EUROPEAN COMMISSION Brussels, 11.9.2019 SWD(2019) 1350 final COMMISSION STAFF WORKING DOCUMENT EVALUATION Accompanying the document Proposal for a

DECISION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL amending Decision No 573/2014/EU on enhanced cooperation between Public

Employment Services (PES)

{COM(2019) 620 final} - {SWD(2019) 319 final}



Table of contents 1.INTRODUCTION 1.1.Purpose 1.2.Scope 2.BACKGROUND TO THE INTERVENTION 2.1.Description of the intervention and its objectives 2.2.Baseline and points of comparison 2.2.1. Pre-2014 cooperation model showed limitations 2.2.2. Enhanced cooperation between PES from 2014 3.IMPLEMENTATION / STATE OF PLAY 3.1.Governance of the Network 3.2.Benchlearning 3.3.Mutual assistance 3.4.Modernising PES in line with the objectives of Europe 2020 3.5.Analytical reports 3.6.Implementation of relevant policy initiatives 3.7.Implementation of the European Social Fund 3.8.Annual work programme 3.9.Promote and share best practices for people not in education, employment or training (NEETs)

3.10.Participation in learning events 4.METHOD 4.1.Short description of methodology and sources 4.2.Limitations and robustness of findings 5.ANALYSIS AND ANSWERS TO THE EVALUATION QUESTIONS 5.1.Relevance 5.1.1. The objectives have been relevant for the Network 5.1.2. The relevance of each objective 5.1.3. The objectives have been resilient to the changing labour market and policy developments 5.1.4. The initiatives of the PES Network have been relevant 5.1.5. Potential for improving relevance 5.2.Effectiveness 5.2.1. Quantitative effects of the enhanced cooperation between PES 5.2.2. Modernisation promoted by benchlearning 5.2.3. Mutual learning between PES has improved 5.2.4. Effectiveness of mutual assistance 5.2.5. PES have taken up the results of Network cooperation 5.2.6. Network contributions to effective policy implementation 5.2.7. Potential for improving effectiveness 5.3.Efficiency

5.3.1. The Network has been efficient in delivering initiatives and reaching objectives 5.3.2. Costs 5.3.3. Benefits 5.3.4. Balance of costs and benefits 5.3.5. Potential to increase efficiency 5.4.Coherence 5.4.1. Consistency of the Decision with EU policies and initiatives 5.4.2. Contributions of the Network to relevant EU policies and initiatives 5.4.3. Large degree of complementarity with the EU policy framework 5.4.4. Potential to increase consistency 5.5.EU added value 5.5.1. Impacts of the Decision 5.5.2. The role of the Network in cooperation between PES 5.5.3. Potential for increased added value 6.CONCLUSIONS AND LESSONS LEARNED Annex 1: Procedural information Annex 2: Stakeholder consultation — Synopsis report 1.Consultation strategy 1.1. Identification of stakeholders 1.2. Consultation activities

2.Consultation activities 2.1. Public consultation 2.2. Targeted consultations 2.2.1. Targeted semi-structured interviews 2.2.2. Written questionnaires 2.2.3. Workshop 2.2.4. Case studies 2.2.5. Summary and reflections on challenges 3.Information on stakeholder groups 3.1. Public consultation 3.1.1. Country of respondents 3.1.2. Sector of activity 3.2. Targeted consultations 3.2.1. Targeted semi-structured interviews 3.2.2. Workshop 3.2.3. Written questionnaires 3.2.4. Case studies 4.Methodology for data processing 4.1. Public consultation 4.2. Targeted consultations

4.3. Case studies 5.Dissemination strategy implemented for the public consultation 6.Overview of the results of the consultations Annex 3: More detailed intervention logic of the evaluation Figures, Boxes and Table Figure 1: Summary of the intervention logic of the Decision Figure 2: Increase in benchmarking scores from the first to the second cycle Figure 3: PES participation across all mutual learning activities, 2011-2014 and 2015-2018 Figure 4: Summary of the evaluation process Figure 5: Composition of the PES population Figure 6: Number of dashboard sessions per country Box 1: The five pillars of benchlearning Box 2: Mutual assistance — Cyprus Box 3: Mutual assistance — Romania Box 4: Examples of reform initiatives resulting from the benchlearning initiative Table 1: Selected examples of contributions of the Network to relevant policies and initiatives Glossary

Term or acronym Meaning or definition AFEPA Adviser for European PES affairs ALMP Active labour market policies CSR Country-specific recommendation EaSI Employment and social innovation programme ECE European Centre of Expertise EMCO Employment Committee ESF European Social Fund ESPN European Social Policy Network EURES European Network of Employment Services FTE Full-time equivalent HoPES Heads of public employment services ILO International Labour Organization LTU Long-term unemployed MISSOC Mutual Information Systems on Social Protection ML Mutual learning NEET Not in employment, education or training NGO Non-governmental organisation OECD Organisation for Economic Cooperation and Development PARES Partnership between Employment Services Initiative PC Public consultation PDCA Plan-do-check-act PES Public employment services Network European Network of Public Employment Services PrES Private employment services TFEU Treaty on the Functioning of the European Union WAPES World Association of Public Employment Services 1.Introduction 1.1.Purpose Decision No 573/2014/EU on enhanced cooperation between Public Employment Services (PES) (the ‘Decision’) establishes the European Network of Public Employment Services (the ‘Network’) from 17 June 2014 until 31 December 2020. Although not required by the Decision, this evaluation has been undertaken in the interest of good administration and in line with the ‘evaluate first’ principle to take into account the lessons of past EU action and to better design future interventions. This evaluation assesses the state of play and degree of implementation of the Decision, and whether the Network has fulfilled its legal obligations and achieved its objectives, further described in Section 2. The evaluation covers all five criteria set out by the Better Regulation requirements: •Relevance: the extent to which the Decision, in particular the Network’s objectives and initiatives as outlined in Articles 3 and 4 respectively, have been relevant and remain so in the current labour market context and the social situation in the EU. •Effectiveness: the extent to which the Network has been effective in delivering the initiatives mentioned in the Decision (Article 4) and reaching its objectives (Article 3). This includes the extent to which the Decision’s objectives have been effectively implemented, assessing indicators of relative success, and their contribution to achieving the Decision’s objectives. •Efficiency: the extent to which the Network has been efficient in delivering the initiatives mentioned in the Decision and reaching its objectives, and whether the costs were proportionate to the benefits achieved. •Coherence: the extent to which the Decision and the initiatives under its aegis are complementary with other EU or national policies and initiatives with similar objectives. •EU value added: the extent to which the Decision has generated added value for the EU as a whole compared to the situation prior to or in the absence of the Decision’s entry into force. Based on the assessments of each criteria, the evaluation draws some overall conclusions, including lessons learned. The results of the evaluation will feed into reflections on the possible future cooperation between European public employment services post-2020. 1.2.Scope The geographical scope of the Network is the EU Member States and the EEA countries Iceland and Norway. Inclusion of the EEA countries is supported by their participation in the Employment and social innovation (EaSI) programme. This has also contributed to extending the range of members in the Network. The timeframe for the evaluation covers the period from the entry into force of the Decision establishing the Network, which is 17 June 2014, until 31 July 2018. The cut-off date was set to maximise the availability of data in the evaluation. Secondary sources used in the evaluation can have different cut-off dates. For example, the 2018 annual report of the European Network of Public Employment Services (PES) covers the full calendar year. The evaluation focuses on the extent to which the Decision has been implemented, and the extent to which this has so far led to the expected outputs. 2.Background to the intervention 2.1.Description of the intervention and its objectives Public employment services are the main agencies carrying out employment policies directly accountable to governments. They are set up to facilitate the labour market integration of jobseekers and the payment of unemployment and/or welfare benefits. Although structured differently in each country, all PES help match supply and demand in the labour market through information, placement and active support services at local, national and European levels. PES are also the main actors implementing activation policies in the Member States, and they play an important role in facilitating successful labour market transitions and integration. The quality of their services has direct consequences for the impact of employment policies on the ground. All of the above make PES key actors in fighting unemployment in Europe and in ensuring the success of the EU 2020 strategy for jobs, and smart, sustainable and inclusive growth. Performance differs considerably across PES, depending on business models, capacity in terms of staff and resources, and management approaches. Additionally, PES operate in different labour market contexts. They face challenges due to changes in the way the labour market functions and experience fluctuating demand from citizens, for instance due to the increased unemployment rates following the economic crisis. Shrinking public budgets and the need for increased cost-effectiveness of PES have prompted several Network members to undertake reforms. The PES need to respond to these challenges, to changes on the labour market and to customers’ expectations of service delivery. European-level collaboration between PES started in 1997, when the Commission set up an informal advisory group of the heads of PES (the European Network of Heads of PES (HoPES)). The aim was to promote cooperation, exchange and mutual learning between the member organisations and to receive feedback on employment policy initiatives. Building on this, in 2013 the Commission proposed to formalise this cooperation to support innovation, benchmarking and mutual learning at European level. This was achieved through the Decision of 15 May 2014 establishing the European Network of Public Employment Services. The Network’s objective is to strengthen the capacity, effectiveness and efficiency of public employment services by providing a platform for comparing their performance at European level, identifying good practices and establishing a mutual learning system. It also aims to give PES more opportunities to help develop innovative, evidence-based policies in line with the Europe 2020 objectives.

Figure 1 below, summarising the intervention logic of this evaluation, gives an overview of needs for stronger and more formalised cooperation. The intervention logic includes the objectives, the problems that the interventions under the Decision intended to solve and how it was expected to work. The Network contributes to the European employment strategy to create more and better jobs throughout the EU, an integral part of the ‘Europe 2020’ strategy for jobs, and smart, sustainable and inclusive growth. The objective of the Decision, as stated in Article 3, is to encourage cooperation between PES through a formalised network, in order to contribute to the ‘Europe 2020’ strategy and to the implementation of relevant EU policies. In this way the Network will support: (a) the most vulnerable social groups with high unemployment rates, especially older workers and young people not in employment, education or training (NEETs); (b) decent and sustainable work; (c) the better functioning of labour markets in the EU; (d) the identification of skills shortages and the provision of information on their extent and location, as well as the better matching of jobseekers’ skills with employers' needs; (e) the better integration of labour markets; (f) increased voluntary geographical and occupational mobility on a fair basis to meet specific labour market needs; (g) the integration of people excluded from the labour market as part of the fight against social exclusion; (h) the evaluation and assessment of active labour market initiatives and their effective and efficient implementation.

Figure 1: Summary of the intervention logic of the Decision NEEDS OBJECTIVE INPUTS

ACTIVITIES OUTPUTS RESULTS IMPACTS Source: Study supporting the evaluation of Decision 573/2014/EU on enhanced cooperation between Public Employment Services, Ecorys for the European Commission, 2019. Article 4 of the Decision sets out the initiatives and activities to be implemented by the Network: ‘(a) the development and implementation of Union-wide, evidence-based benchlearning among PES to compare, with appropriate methodology, the performance of their activities in the following areas: (i) contribution to reducing unemployment for all age groups and for vulnerable groups; (ii) contribution to reducing the duration of unemployment and reducing inactivity, so as to address long-term and structural unemployment, as well as social exclusion; (iii) filling of vacancies (including through voluntary labour mobility); (iv) customer satisfaction with PES services; (b) the provision of mutual assistance, either in the form of peer-to-peer or group activities, through cooperation, exchanges of information, experiences and staff between the members of the Network, including support for the implementation of PES-related country-specific recommendations issued by the Council upon request by the Member State or the PES concerned; (c) contribute to modernising and strengthening PES in key areas, in line with the employment and social objectives of Europe 2020; (d) prepare reports at the request of the European Parliament, the Council or the Commission, or on its own initiative;

(e) contribute to the implementation of relevant policy initiatives; (f) adopt and implement its annual work programme setting out its working methods, deliverables and the details related to the implementation of benchlearning; (g) promote and share best practices on the identification of NEETs and on the development of initiatives to ensure those young people gain the skills necessary to enter and remain in the labour market.’ The aim of the Decision is to: (i) increase cooperation among PES and the implementation of learning and recommendations achieved through this cooperation; (ii) contribute to better performing PES and better functioning labour markets. Contribution to better implementation of EU policy initiatives such as the EU2020 targets is an overarching goal. Due to the broad range of themes and responsibilities covered by the Decision, the outputs take, among others, the form of studies and analysis, learning events, and visits. Dissemination and sharing of knowledge is of key importance. 2.2.Baseline and points of comparison 2.2.1.Pre-2014 cooperation model showed limitations As outlined in the explanatory memorandum for the 2013 proposal for the Decision, experience had shown that PES were not engaging sufficiently in mutual learning and benchmarking activities by themselves (see further details in Figure 3). In summary, four main forms of PES cooperation were in operation. •Voluntary mutual learning opportunities via the ‘PES to PES Dialogue’. This EU-funded programme was established to financially support learning across PES and the implementation of efficient policies. •Strategic dialogue between different employment services (public, private and non-profit) in the context of the Partnership between Employment Services (PARES) initiative. •Informal, voluntary cooperation of the heads of employment services (HoPES) via the HoPES Network. •Bilateral or multilateral cooperation between staff at different PES levels in the context of self-funded exchanges or EU-funded projects. Available evidence from the PES to PES Dialogue programme (annual reports, dissemination of conference papers) indicates that the programme focused on exchanges based on specific guidelines from the European employment strategy. Guidelines addressed for example in 2014 focused on: (i) increasing labour market participation of men and women, reducing structural unemployment and promoting job quality (Guideline 7); and (ii) developing a skilled workforce responding to labour market needs and promoting lifelong learning (Guideline 8). The activities of the PES to PES Dialogue programme included: (i) conferences; (ii) peer reviews intended to identify good practices, explore them and disseminate their core elements; and (iii) follow-up study visits providing ad hoc support provided when needed and requested by PES. High-level advice and expertise was delivered by peer PES experts to increase the operational capacity of PES, to improve service offer and delivery or to implement good practices highlighted by the peer review. In addition, ad hoc analytical papers were published on various themes. Despite progress over the years, this cooperation model showed considerable limitations: •Since participation of national PES in these activities was voluntary, this made it harder to quickly identify low performance by PES and potential structural labour market challenges deriving from this. •There was no reporting mechanism, meaning that policy designers at national and EU level were not systematically informed about the results of the cooperation. •The linkages between benchmarking and mutual learning activities were weak and inconsistent, and the evidence base for the activities of the mutual learning programme were not scientifically robust. •Not all PES participated, and often the least mature PES and those with the greatest labour market challenges participated the least. •PES developed their own tools and approaches (e.g. for digitalisation) at national level, rather than saving time and money by learning from the more advanced PES. As a result of the above, efforts to make PES more comparable by clustering them according to business models had not succeeded by 2013.

2.2.2.Enhanced cooperation between PES from 2014 The aim of the Decision was to fill the gap between the pre-2014 situation and the new requirements for PES to contribute effectively to the Europe 2020 strategy. To contribute to the work of the Employment Committee (EMCO), which supports the coordination of the Member States' employment policies, PES were required to adapt their organisation models, business strategies and processes faster than what had been achieved by the previous, informal network. Formalising the Network was identified as an effective way to provide a platform for comparing PES performance across countries, identify good practices and foster mutual learning. While Member States would remain responsible for organising, staffing and running their PES, a formal PES Network would expand, reinforce and consolidate benchmarking and learning initiatives for the benefit of all PES. The Network was expected to provide a platform for comparing their performance at European level, identifying good practices and fostering mutual learning to foster learning organisations to strengthen service capacity and efficiency. It constituted a concerted action to modernise and empower PES to successfully act in unison in the face of the then predominant economic crisis. A European PES Network established on a solid legal foundation was expected to be able to ramp up coordination between PES and provide the Network with legitimacy to act. A formalised structure was also considered a pre-condition for increasing the Network’s potential to help develop innovative, evidence-based policy implementation measures in line with the Europe 2020 objectives. The Network was also expected to feed into the European Semester surveillance process by identifying low-performing PES at an early stage before their problems became structural. Formalising the Network was also expected to facilitate implementation of labour market projects financed by the European Social Fund (ESF), and the proposed initiative would contribute to improved cost-efficiency. The Network established under the Decision would carry out initiatives in the form of ‘incentive measures’ provided for under Article 149 of the Treaty on the Functioning of the European Union (TFEU) designed to improve cooperation between Member States in the employment field. The Network’s initiatives would be in addition to cooperation among PES within the European Network of Employment Services (EURES) on the basis on Articles 45 and 46 TFEU. 3.Implementation / state of play This section describes the Network’s governance structure and the implementation of each of the initiatives provided for under Article 4 of the Decision. Where thematically appropriate, some initiatives have been linked together. 3.1.Governance of the Network The Network has a governing board made up of the heads of PES from each EU Member State and from Norway and Iceland, plus a representative from the Commission. EMCO has observer status. The Board appoints a chair and a vice chair, while the second vice chair is held by the Board member from the PES of the country holding the EU Presidency. The chair represents the Network. The Board meets twice a year in the Member State that holds the EU Presidency. It sets the strategic direction of the Network, debates labour market developments and monitors the work programme’s implementation. Each Board member appoints an Advisor for European PES Affairs (AFEPA) to help run the Network. AFEPAs meet twice a year in Brussels. The Network’s Board is assisted by a Secretariat provided by the Commission (specifically by the Directorate-General for Employment, Social Affairs and Inclusion (DG EMPL)). The Secretariat, in cooperation with the chair and vice-chairs, prepares the Board meetings and helps implement the annual work programme. It also organises and chairs the meetings of the AFEPAs. 3.2.Benchlearning Benchlearning is defined in the Decision as the ‘process of creating a systematic and integrated link between benchmarking and mutual learning activities that consists of identifying good performances through indicator-based benchmarking systems […] and of using findings for tangible and evidence-informed mutual learning activities, including good or best practice models’. The aim is to support each PES in improving their performance through comparisons and institutional learning from peers. Box 1. The five pillars of benchlearning 1.Quantitative assessment creating transparency on PES performance by collecting, validating and analysing PES data; 2.Qualitative assessment of PES performance against performance enablers, achieved through self-assessment followed by a peer/Commission/external expert assessment on the basis of a site visit; 3.Systematic identification of good practices collated online, such as in the PES Knowledge Centre; 4.A mutual learning programme which builds on the results of and supports better PES performance by focusing on identified strengths and weaknesses; 5.PES follow-up of the benchlearning assessment, by drawing up and implementing action plans to improve performance. Peer-to-PES coaching and learning through learning dialogues. The benchlearning concept is based on five pillars, as explained in Box 1 above. Quantitative and qualitative assessments of PES performance build the benchmarking side. This is the basis for performance transparency. Knowing about PES structures, processes and services, as well as their outcomes, makes it possible to identify good practices and mutual learning activities. This leads to PES action plans to improve performance, benefiting from peer PES support. In the context of the benchmarking (quantitative and qualitative assessments), a structured and systematic analysis has been conducted of PES results and its drivers (‘performance enablers’). In 2015, a thorough analysis of performance data (quantitative benchmarking) was carried out and a process of PES self-review, peer review and expert review (qualitative benchmarking) was implemented. The quantitative and qualitative performance indicators are listed in the Annex to the Decision. On quantitative assessment, each year PES administrative data is collected, validated and analysed to provide the eight mandatory benchmarking indicators listed in the Annex to the Decision: 1.Contribution to reducing unemployment for all age groups and for vulnerable groups: a)Transition from unemployment into employment per age group, gender and qualification level, as a share of the stock of registered unemployed persons; b)Number of people leaving the PES unemployment records, as a share of registered unemployed persons. 2.Contribution to reducing the duration of unemployment and reducing inactivity, so as to address long-term and structural unemployment, as well as social exclusion: a.Transition into employment within, for example, 6 and 12 months of unemployment per age group, gender and qualification level, as a share of all PES register transitions into employment;

b.Entries into a PES register of previously inactive persons, as a share of all entries into that PES register per age group and gender. 3.Filling of vacancies (including through voluntary labour mobility): a.Job vacancies filled; b.Answers to Eurostat’s Labour Force Survey on the contribution of PES to the finding of the respondent’s current job. 4.Customer satisfaction with PES services: a.Overall satisfaction of jobseekers; b.Overall satisfaction of employers. Areas of benchmarking through qualitative internal and external assessments of performance are as listed in the Annex to the Decision: 1.Strategic performance management; 2.Design of operational processes such as effective channelling and profiling of jobseekers and tailored use of active labour market instruments; 3.Sustainable activation and management of transitions; 4.Relations to employers; 5.Evidence-based design and implementation of PES services; 6.Effective management of partnerships with stakeholders; 7.Allocation of PES resources. The qualitative assessment was conducted in two cycles: 2015/2016 and 2017/2018. In the second cycle, the assessment was enriched by a focus on changes implemented since the first cycle and by introducing change management as an additional performance area. The exercise is based on the ‘PDCA cycle’ (plan, do, check, act). This means that the PES are assessed on: (i) their planning and implementation of activities within each performance area; (ii) how they check implementation; and (iii) how they react to needs for revisions or other actions. By the end of 2018, all PES participating in the Network had been visited and assessed twice. At the end of each visit, each PES received a feedback report containing qualitative assessments and performance scores; with 6 as the best score and 1 as the lowest. This provides PES with insight into their strengths and weaknesses in the various performance areas, provides practical suggestions for further improvements, and identifies peer PES as potential partners for exchange (for the first cycle). The feedback report also provides insight into the relevance, coherence and consistency of the reform agenda and an assessment of the PES’s change management (for the second cycle). As follow-up, reports on change are submitted by the PES one year after the visit. These reports give an overview of the recommendations to which the PES has given priority, and of the actions taken to bring about improvement in the specific area. The results of the benchlearning assessments have over the past four years been increasingly used as an evidence base to develop and steer the Network’s mutual learning programme, as reflected in its annual work programmes. Thematic clusters such as digitalisation, services to employers and performance management were central to the 2018 programme. Finally, a collection of potentially transferable PES practices identified during the assessments and the mutual learning activities were shared within the Network for learning purposes. In the PES Practice Repository, more than 170 PES practices had been presented by the end of 2018.

3.3.Mutual assistance Results and assessments from the benchlearning programme have helped PES identify performance areas or tasks where support from other PES could be an appropriate method for initiating or implementing reform. The benchlearning recommendations and the country-specific recommendations coming out of the European Semester process form the basis for selecting performance areas or tasks. Through mutual assistance, the selected PES receive technical support from peer PES to develop specified performance areas or tasks upon request. This includes support for the implementation of European Semester country-specific recommendations. Between 2015 and 2018, mutual assistance projects lasting between 9 and 18 months were carried out in Bulgaria, Cyprus, Greece, Italy, Romania and Spain. PES colleagues from Austria, from VDAB (the PES for Belgium’s Flanders Region), from Denmark, Estonia, France, Germany, Ireland, Latvia, Slovenia, Spain, Sweden and the United Kingdom participated in the projects as peer experts. Box 2. Mutual assistance — Cyprus The mutual assistance project in Cyprus took place in 2017. A follow-up meeting was held in 2018. The project focused on the following thematic areas: The benchmarking site visit identified the following progress as a consequence of the mutual assistance: improvement in digital services, segmentation, better monitoring of ALMPs and implementation of a new service offer to employers. Success factors included: help in identifying key reform priorities, developing a clear action plan and committing to implementing it, and achieving high staff motivation. •Better customer segmentation; •Expanded use of IT for self-service provision; •Evidence-based design and enhanced delivery of active labour market policies (ALMPs) / improved activation services (including outreach) for jobseekers especially from the most disadvantaged groups, e.g. the young, the disabled, the long-term unemployed; •Services for employers; •Performance management/organisational culture/quality management. The projects covered a broad range of tasks. For example, the project in Spain focused on three main topics: a common profiling tool, customer satisfaction and partnership with employers. The aim was to achieve better coordination between the country’s 17 regional employment services, social services and employers so as to better respond to customers’ needs. At this stage, it is too early to assess the impacts on coordination and the role of the central PES. The Cypriot PES requested support to enhance its capacity to serve the long-term unemployed and to improve outreach to non-registered unemployed people. This is further explained in Box 2. Source: PES Secretariat/European Commission 3.4.Modernising PES in line with the objectives of Europe 2020 Modernising a PES involves increasing its level of professionalism, which in turn increases its maturity as an organisation. As explained in Section 3.2 above, PES performance is scored on a scale from 1 to 6 as part of the benchlearning exercise. The following rule is used to approximate maturity in the benchlearning exercise: •A mature organisation with respect to a specific performance area is achieved when all four scores in the PDCA cycle are 5 or higher. •A well-developed organisation with respect to a specific performance area is achieved when at least three of the four scores are 4 or higher. •A developing organisation with respect to a performance area is achieved when at least three of the four scores are 3 or higher. •In all other cases, the maturity of the organisation is considered ‘developable’. Low scores indicate performance areas where a PES can improve, while high scores indicate that PES is strong in these areas. Among the 29 PES that had received their second cycle scores by the end of 2018, 9 PES were at the overall level assessed as ‘mature’ or ‘well developed’. The comparison between the first and second benchlearning cycle shows that on average PES maturity gradually increased (see Figure 2). Out of 28 countries included in Figure 2, 6 improved their scores by at least 20% from the first cycle to the second, and 10 countries improved by at least 10%. The remaining PES show moderate or no progress at all. Figure 2 shows that Cyprus, Bulgaria and Greece, which have received mutual assistance, (see Section 3.2 above), all belong to the group of PES with a high increase in scores from the first to the second cycle. Source: PES Secretariat/European Commission Some PES had a high maturity level already in the first cycle. Their room for further improvements could be more restricted than for PES assessed as less mature in the first cycle. 3.5.Analytical reports Research and studies on PES topics related to priorities of the annual work programme are published each year at the request of the European Parliament, the Council or the Commission, or on the initiative of the Network. Since 2015, about 60 papers and reports have been published in the PES Knowledge Centre. The reports enable members and practitioners to learn about initiatives across the EU and access their impact. The topics of the reports include PES practices and toolkits as well as analytical papers. Examples of reports from 2018 include a study report on PES’ role in outreach to the inactive population and an analytical paper on early activation and employment promotion. 3.6.Implementation of relevant policy initiatives

The Network delivered a common response to the consultation on the European Pillar of Social Rights after assessing its impact on the Network Decision and reflecting on new trends in work patterns and society. In line with the Pillar’s principles, the Network updated its strategy document ‘EU Network of Public Employment Services Strategy to 2020 and beyond’, demonstrating the Network’s ability respond to new EU policies and initiatives. Some examples of its contributions are set out in Table 1 below. The overview in Table 1 demonstrates that the Network has contributed extensively to the implementation of EU relevant policies and initiatives. Its contributions have been more extensive in areas such as implementation of the Youth Guarantee and the integration of the long-term unemployed into the labour market. The Network has also provided contributions to other relevant policy areas such as the European Pillar of Social Rights.

Table 1: Selected examples of the Network’s contributions to relevant policies and initiatives Policies/initiatives Examples of contributions by the Network Council Recommendation of 22 April 2013 on establishing a Youth Guarantee •Formal role in monitoring as enshrined in the Recommendation •PES Network self-assessment report on PES capacities to implement the Youth Guarantee, 2014 •PES Network Catalogue of Measures for implementation of the Youth Guarantee, 2014 •Summary Report: PES Network Conference: Implementation of the Youth Guarantee — Challenges and Success Factors, 2016 •Annual reports on PES Implementation of the Youth Guarantee Council Recommendation of 15 February 2016 on the integration of the long-term unemployed into the labour market •Formal role in monitoring as enshrined in the Recommendation •Peer review on PES approaches for sustainable activation of LTU •PES LTU Working Group, 2016 •PES Network contribution to the consultation on long-term unemployment, 2015 •Proposal to EMCO for quality standards to monitor implementation of the Recommendation, 2016 •Summary report: PES Network Conference: on the integration of the long-term unemployed, 2016 •Thematic review workshop on ‘Sustainable integration into the labour market’, November 2016 •Analytical paper: Sustainable integration into the labour market, 2017 •PES Network study on assessment and early intervention to prevent long-term unemployment, 2017 •Ad hoc module to the 2018 PES capacity questionnaire survey report 2018 European Pillar of Social Rights •Network Board contribution to the Commission’s consultation on the European Pillar of Social Rights, December 2016 Integration of refugees •Thematic paper on ‘Integration of refugees into the labour market’ •Follow-up visit on integration of refugees into the labour market, 2017 •Key considerations on labour market integration of refugees, 2016 and 2019 ESF •In some countries (e.g. Bulgaria, Croatia, France, Luxembourg, Hungary, Lithuania and Romania) the PES actively cooperates with the ESF managing authorities and receives funds for their operational programmes EURES •Cooperation with the European Network of Employment Services (EURES) re-established under Regulation (EU) 2016/589 •Closer cooperation between the PES Network and EURES at European level agreed at the December 2016 meeting of the PES Network Board •A member of the PES Network acts as liaison point with the EURES Coordination Group Source: Study supporting the evaluation of Decision on enhanced cooperation between Public Employment Services, Ecorys for the European Commission, 2019. 3.7.Implementation of the European Social Fund PES are key actors in implementing the European Social Fund (ESF) in some Member States, in particular for measures promoting sustainable and quality employment. Total funding in this area amounts to €40.7 billion (2014-2020, EU and national funding). Out of this amount, the ESF provided direct support of €2.1 billion for reforms to modernise PES and more than €25 billion to develop active labour market policies that are mainly implemented by the PES. One of the ESF’s priorities is the funding of active labour market policies aimed at disadvantaged people in the labour market, such as young people and the long-term unemployed. ESF funding is therefore important for the implementation of policy initiatives for many national PES. In some countries, Bulgaria and Romania for example, ESF funding has played an important role in reforming PES in order to ensure they have the institutional capacity and their staff have the necessary skills to help disadvantaged groups to find jobs. In Romania, the ESF project was implemented by assistance from the Network (see Box 3). Box 3. Mutual assistance — Romania Romania approached the Network with a request for mutual assistance after receiving country-specific recommendations in 2016 and 2017 on the capacity of its PES, specifically related to improving services to employers and jobseekers. PES colleagues from Belgium (VDAB), Germany and Sweden participated in the project, which took place in 2017 and 2018. Understanding labour market needs and close collaboration with employers are crucial for PES. The project included a human resource component, with 400 employees of Romania’s national employment service receiving training aimed at improving the skills needed for an effective relationship with employers. In addition, particular action was taken to: (i) develop a new tool to support jobseekers’ profiles and channel them towards appropriate active measures; (ii) develop the IT system; and (iii) develop the case management approach for the registered unemployed. Funding from the ESF had an important contribution to financing the project. Work is still in progress and measures could lead to a positive impact if they are embedded in a comprehensive implementation strategy. Source: PES Secretariat/European Commission Member States are responsible for appointing a designated managing authority that provides information on the ESF programme, selects projects and monitors implementation. Out of the 31 PES covered by the 2018 PES capacity report, 4 PES acted as managing authority for the ESF and 5 PES acted as managing authority in partnership with other institutions. Other PES are involved because the managing authority has delegated certain responsibilities to them. 3.8.Annual work programme The Network adopts an annual work programme, which is based on exchange of views and a learning needs assessment, agreed upon with all its members. The work programme is designed to assist the PES in: (i) delivering the Network’s mission to promote modernisation; and (ii) helping individual PES to contribute more to the implementation of the EU2020 strategy. Benchlearning is central to the Network’s activities and is key to identifying and addressing mutual learning activities. In recent years, the work programme has been grouped under three headings: benchmarking, mutual learning and network governance. 3.9.Promote and share best practices for people not in education, employment or training (NEETs) Each year, a number of practices to improve the situation of vulnerable groups in the labour market are published and shared on the ‘PES Practices’ page on the European Commission website. Some practices have a broader scope, not targeted at specific groups of vulnerable people. In addition, practices are presented which focus on NEETs as a group with particular needs, for example the ‘Toolkit on Sustainable Activation of NEETs’ and ‘PES Practices for the outreach and activation of NEETs’. 3.10.Participation in learning events Since 2015, the range of learning events has been broadened to better support the learning needs of individual PES. Specifically, the focus has changed towards smaller, more targeted learning formats, with events now organised with specific learning groups in mind. This methodology means that benchlearning site visit reports (conducted in the context of the qualitative assessments) and other documents are used to identify PES with specific learning needs in a particular topic area. PES that are advanced in certain areas are in general invited to host or chair thematic review workshops and follow-up visits, so that they can share their practices with other countries, helping them further develop in that same area. Likewise, advanced PES are invited to participate as external experts in mutual assistance projects. Working groups chaired by, and with participation from, Network members, have a key role in developing knowledge that is relevant to PES, and in preparing (for example) key considerations to be endorsed by the Network Board. The range of learning events has included webinars since 2017 and learning dialogues since 2019. Learning dialogues are small groups of PES, where a PES that is advanced in a certain area shares experience with two to four PES that have learning needs in that area. The coaching approach includes followup support aimed at PES implementing what they have learnt for that specific area.

As illustrated in Figure 3, the number of learning events more than doubled between the 20112014 and 2015-2018 periods. Similarly, the number of participants more than doubled, and the number of learning days increased by 75%. Figure 3: PES participation across all mutual learning activities. 2011-2014 and 2015-2018 Source: PES Secretariat/European Commission 4.Method 4.1.Short description of methodology and sources Following the evaluation roadmap published in October 2017, the evaluation presented in this staff working document was carried out on the basis of information and data collected from different sources in accordance with the Better Regulation guidelines and Toolbox. The information aims to cover all the elements in the intervention logic presented in Figure 1, with a focus on results and impacts of the activities. This evaluation draws to a large extent on information from an external study commissioned from a contractor (the ‘external study’) and the 2017 interim review. The 2017 interim review is the Commission report published (in accordance with Article 10 of the Decision) on 6 June 2017 on the Network’s activities up until that point. Furthermore, desk research was carried out to map changes resulting from the Decision and the Network’s activities; the research covered a range of written sources such as annual reports, data collections and satisfaction surveys. The sources are complementary and mutually reinforcing. The main sources are summarised in Figure 4. Figure 4: Summary of the evaluation process Source: PES Secretariat/European Commission The external study was conducted between June 2018 and March 2019. Its methodological approach combined quantitative and qualitative data, which have been systematically cross-checked to answer the evaluation questions. The study includes a mapping of policy changes implemented by PES following the creation of the Network in June 2014. A range of actors were consulted in the external study (see details in Annex 2). Interviews were conducted with: ·members of each of the 32 PES involved in the Network (when possible both with the Network Board member and the AFEPA); ·selected representatives of relevant EU-level organisations and bodies (e.g. EMCO, EU-level private employment services and temporary work agencies, the European Lifelong Guidance Policy Network); ·selected representatives of relevant international organisations (for example ILO, OECD, World Bank, World Association of Public Employment Services (WAPES)); ·other stakeholders closely involved in the running of the Network (e.g. the PES Secretariat, contractors); ·a sample of individuals who were former members of the Network or involved with PES cooperation pre-2014. In addition, a workshop was conducted with AFEPAs, as well as case studies in five countries (Estonia, France, the Netherlands, Spain and Romania) to further elaborate the evaluation questions. These countries were selected on the basis of several criteria, namely geographic location, population size, economic situation (measured by real GDP growth, employment and unemployment rates), and on the basis of PES performance indicators (maturity levels, increase/decline in PES expenditure, and increase/decline in new vacancies reported). ESF managing authorities and labour and social affairs ministries received a targeted questionnaire, which 16 labour ministries and 7 ESF managing authorities filled out.

An open internet-based public consultation ran from 17 September 2018 to 13 December 2018. An online questionnaire was available in English, French and German, and replies could be made in any of the 24 official EU languages. In particular, the consultation sought to gather views from: (i) individual citizens in their personal or professional capacity; (ii) researchers and academia; (iii) civil society organisations working with jobseekers or on the issue of unemployment/employment services; (iv) social partners; (v) employment and social services providers; and (vi) public authorities at national, regional and local levels. 126 replies from 18 Member States and Pan-EU organisations were received. The evaluation also draws on various other secondary sources, cited throughout in the text. Benchlearning results, the evaluation of mutual learning events and the annual reports are among the main sources, a list of which is set out below: •Annual reports of the Network, which were sent to the European Parliament and to the Council and published online. •The outputs produced by the Network as part of its annual work programme, published on the PES Knowledge Centre and PES Practices site. •Satisfaction surveys and other monitoring activities with regard to the Network’s activities, carried out by the contractors who supported the implementation of the Network’s work programme. In particular, in 2018, the contractor in charge of mutual learning activities carried out an enhanced evaluation to understand the potential impact of mutual learning activities. The enhanced evaluation included: (i) third-stage qualitative interviews with PES who attended a thematic review workshop plus a follow-up visit in 2015 (thus building a continuum with interviews undertaken with 2015 cohorts in 2017); (ii) an online mutual learning participant survey, implemented between May and June 2018; and (iii) follow-up interviews with the respondents to the mutual learning participant survey who agreed to be interviewed in summer 2018. •Yearly survey and report on PES capacity available for 2014-2018, published online. In addition, in 2014-2017 an annual report on PES implementation of the Youth Guarantee was published, followed in 2018 by a report on PES implementation of the Recommendation on the long-term unemployed. •The annual quantitative data collection and analysis gathered via the benchmarking exercise is available for the years 2015-2018. Biennial qualitative assessment site visits to the PES, which involve self-assessment reports, and overall assessment reports are available for the years 2015/2016 and 2017/2018 for the whole Network. For the first cycle, change reports from all 28 Member States, Norway and Iceland had been produced one year after the site visit. •The EMCO thematic reviews of PES/ALMPs and the European Semester. 4.2.Limitations and robustness of findings As per the Decision, the Network should contribute to the improved functioning of the PES themselves, as well as to the policy objectives described in Article 3. Evidence from this evaluation shows that for various reasons many of the envisaged reforms and changes take time to be properly implemented. It is important to point to the contextual factors, such as political decisions, allocation of budgetary resources and changing labour market contexts due to economic and social developments. The period covered by this evaluation may therefore be too short to achieve major changes and not least to measure the impacts, and thus it is not possible at this time to make a full assessment of Network efficiency. This may also mean that there are unrealised gains from plans or initiatives that will only appear in the years to come. Secondly, it is sometimes difficult to assess whether an initiative was implemented only because of the Decision. This in turn makes it hard to make a full assessment of the added value. Estimating the impact of the Network’s activities and of PES initiatives as a result of the former is in some cases difficult and may hide the real impact. This is because of data deficiencies due to different definitions between countries or the different design of measures or activities. Differences in PES responsibility from country to country is furthermore a reason why data comparability could be limited. For example, the supply of active labour market measures is in some countries a responsibility for the PES, while in other countries it is a responsibility of the municipalities. Furthermore, some PES produce most of their services in-house, while other PES have extensive use of contractors. An important information source is the qualitative assessments made by PES representatives. However, since it could be argued that people who are deeply involved in a task could be biased in their assessments, this evaluation draws on a broad range of sources. When possible, reliability is checked by comparing information from different sources such as similar information from the external study and the final evaluation report for the mutual learning event, or by comparing assessments by PES representatives with similar assessments by other stakeholders as the labour ministries. In this way, triangulation or cross-checking of sources mitigates the risk of biased conclusions. 5.Analysis and answers to the evaluation questions 5.1.Relevance 5.1.1.The objectives have been relevant for the Network As described in Section 3, the objectives set in the Decision have been a major guideline for Network initiatives implemented through the annual work programmes (see Chapter 2.1 for overview). There was strong consensus among stakeholders interviewed that the eight objectives from 2014-2018 had a high overall degree of relevance. The relevance of the Decision’s content was also strongly confirmed by the findings from the targeted consultations with labour ministries and ESF managing authorities. Finally, the relevance of the Decision was confirmed by the public consultation. One of the main missions for PES is to help disadvantaged groups such as NEETs or the long-term unemployed to find jobs. This has been and still is a major challenge for PES due to high unemployment in the wake of the 2008 economic crisis and the high number of refugees in many countries. The economic recovery has facilitated labour market access for disadvantaged groups, but the long-term unemployed, young people and older jobseekers are still overrepresented among those seeking assistance from PES. Inclusion of inactive people, who are often further away from the labour market, is becoming a priority task for some PES. The Decision’s objectives encompass key areas of responsibility for PES in the field of employment. By providing a broad framework for Network activities, the objectives enable a flexible approach to addressing priorities for action depending on labour market developments. The PES Network’s 2020 Strategy 2020 and Beyond is updated when necessary to be as relevant as possible to the goals of the Europe 2020 strategy. The range of areas and themes is very broad. At the same time, the external study highlights that there is some variation in the degree of relevance for each of the objectives. 5.1.2.The relevance of each objective Section 2.1 of the external study gives an overview of assessments of the relevance of each of the objectives set in the Decision. The ‘inclusion of the most vulnerable social groups’ (objective a) is regarded by consultees in the external study as one of the most relevant Decision objectives as it is a long-standing issue in different European countries, particularly focusing on the inclusion of the long-term unemployed and NEETs. The consultations and the 2017 interim review of the Decision reveal that objective a) is closely related to objective g) on the ‘integration of people excluded from the labour market as part of the fight against social exclusion’. This demonstrates that some of the target groups for PES meet multiple and even interlinked barriers to integration in the labour market. Social exclusion in itself, for example because of poverty, can reduce a person’s chances in the labour market. Achievements on objective g) can therefore contribute to progress on objective a) and vice versa.

‘Supporting decent and sustainable work’ (objective b) is a relevant objective. Non-standard work (i.e. employment contracts that are neither permanent nor full-time) represent nearly 20% of part-time employment and over 10% of temporary employment in the EU in 2017 (see The Future of Work: Implications and Responses by the PES Network, Section 3.1.3). These types of work can provide opportunities for people who want to work less than full working hours or want to have flexibility. At the same time, this work often comes with a greater unpredictability of working conditions and income flows, especially for workers in low-skilled occupations. Due to the changing labour market characterised by new forms of work, work relationships and forms of work organisation, this objective may become increasingly relevant in the future. A ‘better functioning of the labour markets in the EU’ (objective c) is a prerequisite for optimal use of the labour force and is at the heart of the mission of European PES. The relevance of this objective is confirmed in the external study. A well-functioning labour market depends on a number of factors such as economic growth, the general regulatory framework, access to a skilled labour force and professional PES services. The Network contributes to professionalisation of PES by facilitating cooperation and helping them to learn from each other, as well as cooperation with for example EURES and the private employment services. Facilitating the matching of jobseekers with jobs, facilitating labour market integration and effective active labour market policies are all key to a well-functioning labour market. Objective c) is therefore closely linked to objectives d), e), f) and h). Objective d), ‘identification of skills shortages and the provision of information on their extent and location and the better matching of the skills of jobseekers with the needs of employers’, is assessed as relevant by experts and other stakeholders, for example to fill bottleneck vacancies. ‘Better integration of the EU labour markets’ (objective e) also has close links to objective d), and is clearly a relevant objective for the Network, since it directly concerns cooperation at EU level or bilateral cooperation. Objective f), ‘increased voluntary and geographical and occupation mobility on a fair basis to meet specific labour market needs’ is also considered as relevant. There is no duplication and, according to the external study, there is a good level of coordination between the Network and EURES. Finally, objective h), ‘evaluation and the assessment of active labour market initiatives and their effective and efficient implementation’, has been and remains relevant. The relevance of the objectives could depend on the labour market situation at national level or on a where an economy is in its business cycle. At country level, objective d) is seen as most relevant in countries facing skills shortages. Some consultees in the external study raised the question whether objective f) is as relevant for the Network as some of the other objectives. In Greece, Italy and Spain, unemployment was still above 10% at the end of 2018. In other countries, unemployment had dropped to below 4%, and tackling skill shortages is becoming a more prominent task for PES. This illustrates how important it is that the objectives are designed to be relevant for PES facing different labour market conditions. The feedback from PES in the external study confirms that the objectives as they stand fulfil this purpose. To sum up, there was a strong consensus among stakeholders interviewed that the Decision’s objectives are relevant to the current challenges of different national labour markets. Together, they provide a common strategic framework for PES cooperation to tackle the increasing common global challenges that PES are facing, challenges that require concerted action. 5.1.3.The objectives have been resilient to the changing labour market and policy developments The external study, Section 2.1, has shown that overall the objectives have been relevant to the challenges of the European labour market and to the social situation. The flexibility in their wording has made it possible to address new and unexpected challenges (e.g. adaption of the Network strategy and work programme to the 2015 migrant crisis), thereby showing their continuous relevance. Stakeholders’ assessment of the relevance of the objectives matches well with statistical information. The 2018 assessment report on PES capacity highlights that PES are still dominated by difficult-to-place groups of unemployed people such as young people, the long-term unemployed and older workers. These three groups account for a fairly constant share of the registered jobseekers (see Figure 5). Source: PES Secretariat/European Commission 5.1.4.The initiatives of the PES Network have been relevant Article 4 of the PES Decision sets out seven initiatives (labelled a-g) which the Network should carry out (see Chapter 2.1 for overview). These initiatives have been the major guideline for the activities of the annual work programmes. Overall, the initiatives set out in Article 4 of the Decision are relevant to the needs of PES and are consistent with the Decision’s objectives. This is confirmed by the vast majority of consultees in the external study, Section 2.2, both from PES and from other EU and international institutions. The initiatives provide relevant tools to: (i) enhance cooperation between national PES and a well-constructed mix of approaches to suit the needs of different PES; and (ii) ensure synergies and complementarities between the different initiatives. Consultees all assessed ‘benchlearning’ (initiative a), ‘mutual assistance’ (initiative b), ‘reports’ (initiative d), ‘implementation of relevant policy initiatives’ (initiative e) and ‘promoting and sharing’ (initiative g) as very relevant. Consultees regarded initiative c), ‘contribute to modernising and strengthening PES’ as more overarching although still highly relevant, and implemented by a range of other initiatives such as benchlearning and mutual assistance. In addition, while initiative f), ‘adopt and implement its annual work programme’ was seen as clearly relevant, it could be considered as a requirement of the Network’s function rather than an initiative with the same status as benchlearning and mutual assistance. As evidenced in the Network’s annual work programmes and confirmed by the external study, Section 2.2, the initiatives set in Article 4 of the Decision are well linked to the objectives set in Article 3. The Network has demonstrated flexibility by focusing on activities related to new upcoming topics such as the integration of migrants and refugees and the prevention of unemployment. 5.1.5.Potential for improving relevance The remaining difficulties that these groups (young people, long-term unemployed and older workers) face when trying to (re-)enter the labour market indicate that the objective of supporting vulnerable group with high unemployment rates remains relevant. How PES can contribute to better functioning of the labour market in the changing world of work has been addressed more prominently in the Network’s work programme over the past few years. The challenges posed by the changing world of work are also having an impact on how the objective to ensure decent and sustainable work can be achieved. New working conditions are resulting in employment opportunities in new technology sectors, change or loss of jobs due to the large-scale digitalisation of many professions, more short-term or flexible working and a growing trend towards entrepreneurialism and platform jobs. To achieve the objectives of the Decision, jobseekers need the necessary skills, knowledge and competences. PES can: (i) support jobseekers and employers by identifying future skills needs; (ii) deliver suitable training, career guidance and support to enable jobseekers to manage their own careers; and (iii) forge partnerships to improve their offer in this area. A working group established by the Network in 2018 concluded that the Network needs to evolve as a learning organisation, while continuing with benchlearning and promoting the principles of knowledge-based development. A continuous learning environment is key to PES transforming their services, and to focusing on empowerment of jobseekers.

5.2.Effectiveness 5.2.1.Quantitative effects of the enhanced cooperation between PES The Network is considered generally very effective in enhancing cooperation and improving the functioning of PES, in particular through the benchlearning and mutual assistance initiatives. The vast majority of the stakeholders consulted emphasised benchlearning and mutual assistance as the most effective initiatives because of their concrete results. Stakeholders interviewed in the external study believe that the success of the two benchlearning cycles derives not so much from their value as a tool for ranking performance, but rather as a tool for PES to help them learn and improve. The quantitative performance indicators set in the Annex to the Decision are grouped under four headings: ‘labour market flows’, ‘relations with employers’, ‘customer satisfaction’ and ‘long-term unemployment’. The indicators show that almost all PES can be found at least once among the over-performers, thanks to their achieving better results than what statistically could be expected based on their context, e.g. given the demographics, education level, macroeconomic context, etc. in the country or region that they serve. At the same time: there is still room for improvement. PES access the quantitative benchlearning indicators via the PES Dashboard. The dashboard is the major tool used to promote transparency between all members of the Network. The dashboard does not appear to be consistently used by the Network members (see Figure 6 below). Some of the reasons mentioned by PES stakeholders during the external consultation concerned technical aspects, mentioning that IT system security problems limited access to the dashboard, while others referred to the comparability and presentation of the indicators. The fact that so many different PES have agreed on a set of quantitative and qualitative common indicators for the benchlearning process is in itself an important achievement. Since quantitative information can play a role in the planning of initiatives and reforms, one aim should be to achieve more consistent use of the dashboard among PES.

Figure 6: Number of PES Dashboard sessions per country. March 2018 to February 2019 Source: PES Secretariat/European Commission 5.2.2.Modernisation promoted by benchlearning The quantitative and qualitative benchlearning results indicate that PES performance is related to how PES are organised and how services are offered to customers. Evidence on this relationship has improved the foundation for knowledge-based development, both in national PES and across countries in the Network. Comparing results from the first to the second cycle of benchlearning demonstrates that the exercise has given guidance to PES on their path towards modernisation and change. Modernisation is being pushed further by the strong focus of the second cycle on change processes and management of these changes. Decentralisation and involving PES, jobseekers and employers and other stakeholders in design and implementation of services are now playing a more important role in setting up services that are suited to the needs of jobseekers and employers. Staff commitment to delivering quality customer services in all PES continued to be high, although unfavourable caseloads persist in many PES, and many PES continue to work under difficult conditions. Based on the finding, some trends can be summarised: •Strategic performance management: On the basis of the recommendations made during the first cycle, many PES have critically reviewed their performance management system and especially the structure of their key performance indicators, with the aim of achieving outcome-oriented business development. •Design of operational processes: the second cycle gave evidence that customer needs and expectations are at the heart of effective modernisation of processes. Efficient customer segmentation makes it possible to shift resources to those customers who need in-depth support. •Sustainable activation and management of transitions: Nearly all PES have introduced different channels for communication with customers, and digital tools especially have helped to reduce administrative burden in some PES, freeing up resources for in-depth counselling of jobseekers with special needs (long term unemployed, NEETS, etc). •Relations with employers: The PES are currently developing different concepts on how to satisfy employers’ needs given an increasing share of difficult-to-place customers. The Network European Employers’ Day supports PES visibility in participating countries. •Evidence-based design and implementation of PES services: The use of pilot projects has become more prominent since the first cycle, especially for testing new organisational set-ups, service models and ALMPs for specific groups A systematic evaluation of practices and communication of results has, however, only been implemented by a few PES. •Management of partnerships and stakeholders: The discussion has intensified as to which PES services should be outsourced and which should be kept in-house. One-stop shop solutions providing combined social and labour market services have been introduced in some PES, and others will follow.

Even if it is not possible to reach conclusions on the direct impacts of the various actions, PES maturity has increased overall from the first to the second cycle. The observed trends in findings demonstrate that the systematic benchlearning approach, including the follow-up reports, has resulted in more concerted actions across PES. A clearly defined process starting with the first cycle assessments and recommendations, followed by a change plan, follow-up reports and the renewed assessments in the second cycle, seems to have facilitated commitment to plans and implementation of them. 5.2.3.Mutual learning between PES has improved As an inherent part of benchlearning, mutual learning has been a key pillar of Network activity, together with benchmarking activities. Mutual learning involves the use of evidence to identify and address PES learning needs. The aim of the Network’s activities in this area has been to help PES learn from employment services with particular expertise or insights in a specific area. Mutual learning offers a key mechanism to support the other broader initiatives, activities and objectives specified in the Decision. For instance, it includes the types of enhanced activity intended to modernise and strengthen PES, and supports the implementation of relevant policy initiatives. Quantitative and qualitative benchmarking have led to the creation of an evidence base with information on learning needs as well as expertise and good practices to share. This feeds into the development and implementation of the Network’s mutual learning programme. Mutual learning activities focused on several priority areas in 2014-2018, addressing various Decision objectives. In 2015-2016, the focus was on thematic clusters such as young people, performance management and measurement of customer satisfaction, while in 2017-2018 clusters such as digitalisation and services to employers attracted more attention. Activities related to sustainable integration were relatively important throughout the whole period. 2018 saw the introduction of an enhanced approach to monitoring and evaluation of mutual learning activities. The findings, reported in the 2018 annual report, demonstrate that participants are generally very satisfied and that the available resources are useful in the development of changes, initiatives and reforms in PES. Overall, participants gave higher scores to face-to-face learning events than to reports and papers. This can be a reason to explore new channels for dissemination, for example whether videoconferencing can be a useful tool. However, the higher scoring of face-to-face meetings also highlights the importance of networking and discussions with peers and experts in the specific field. 5.2.4.Effectiveness of mutual assistance Benchlearning is often accompanied by direct mutual assistance (technical assistance through peer PES support to selected PES) offered through the Network. Mutual assistance offers a needs-based learning format responding to individual PES needs. For example, following previous activity linked to the benchlearning exercises, throughout 2015-2018 the Network and its members provided mutual assistance for PES modernisation in Spain, Bulgaria, Romania (Box 3) and in Italy and Cyprus (Box 2), again highlighting the interrelated nature of PES activities supporting the Decision objectives. Mutual assistance is seen as a highly effective initiative of the Network. At the request of individual PES, two to three mutual assistance projects take place each year. In the external study, representatives both of PES receiving assistance and of those providing assistance emphasised the role this activity plays in raising awareness about PES strengths and weaknesses. The respondents cited the importance of comparing systems and of providing concrete opportunities to learn and design change strategies (see Boxes 2 and 3 for descriptions of outcomes). Stakeholders consulted for the French case study (the external study Section 3.1) highlighted that mutual assistance activities, as judged by Pôle emploi (France’s PES), are very interesting not only for the PES receiving the assistance but also for the PES providing it. Similarly, the Estonian case study highlighted the exchange of best practice and mutual assistance opportunities as particularly valuable. This illustrates that mature PES (see Annex 1 in the 2018 Annual Report) can also get inspiration for new reforms by acting as mentors and advisers. The mutual assistance projects should therefore not only be considered as assistance, but also as a learning arena. 5.2.5.PES have taken up the results of Network cooperation Assessments from peer experts during site visits and their recommendations to PES are an important input and provide inspiration for change in PES. This is borne out by the change reports from the PES one year after the first site visit and by the reports on the follow-up of benchlearning recommendations made in response to the annual reports. The assessors can give advice about other PES that could provide assistance, and some countries have approached peer PES for learning and exchange (see the example of Austria in Box 4 below), demonstrating the effectiveness of the benchlearning initiative. Box 4. Examples of reform initiatives resulting from the benchlearning initiative The Austrian PES is setting up a system of causal impact evaluations of ALMP measures in order to identify the efficiency of all measures on a regular basis. The Austrian PES organised an exchange at expert level with the German PES to get inspiration from the German ‘Treatment Effect and Prediction’ system. The Slovenian PES has introduced a ‘Randomized Control Trials’ approach (RCTs) to piloting and evaluation of new services and products. This approach will reduce the need for more extensive empirical methods when evaluating interventions. Through the benchlearning initiative, the Slovenian PES received information about other PES with experience in implementing RCTs. As a result of activities from the second benchlearning cycle, the Spanish PES is identifying and disseminating best practices. An internal benchlearning and mutual learning project for the regional PES in Spain is the main framework for this sharing of practices. Source: PES Network Annual Report 2018 In addition to benchlearning, mutual learning activities are part of the Network cooperation. Evidence from internal evaluations of the impact of mutual learning indicates that 70% of respondents used the knowledge they accumulated during the activities and one third declared that the activities have contributed to changes in their PES. The types of change that have occurred often relate to concrete and specific operational changes. These include: •development of new measures for young people and the long-term unemployed; •development and launch of evaluations of ALMPs; •redesign of approaches to measuring customer satisfaction; and •redevelopment of web-based resources.

Other types of change include: •an increased knowledge base; •changes to a strategic approach; •changes in priorities among senior or middle management; and •raised awareness of a specific topic or approach. This overview demonstrates that changes are often directly targeted at services for young people and the long-term unemployed. Other changes could have more indirect impacts on services by increasing the capacity and professionalism of the PES. Change can be initiated not only by mutual learning, but also through other channels such as political decisions at the national level, for example introducing new measures targeted at young people or the long-term unemployed. To implement such changes, learning from other PES can still be important. Even if there are differences, the main conclusion is that PES use mutual learning results extensively. 5.2.6.Network contributions to effective policy implementation The external study, Section 3.2, demonstrates that Network activities have been most useful in supporting the most vulnerable groups with high unemployment rates and, to a certain extent, the integration of people excluded from the labour market. A large majority of national labour ministries indicated that the Network has been very successful or successful in supporting the most vulnerable groups, while only about one third found that the Network had been very successful or successful in supporting the integration of people excluded from the labour market. The Network prepares reports at the request of the European Parliament, the Council and the Commission. As stated in the annual reports from the Network, a number of reports and analytical papers are published, for example on themes related to the long-term unemployed, young people and support to vulnerable groups. Labour ministries were asked by the external contractor (see Section 3.1 in the external study) how they evaluated the Network’s reports. A total of 5 ministries found this activity very successful, 7 found it successful, and 2 somewhat successful, while 2 ministries did not offer any assessment. Overall, the responses from the labour ministries confirm that reports prepared by the Network have proved useful. The evidence gathered that the Network is providing evidence-based information to the European Semester process, indicating PES’ progress and fostering well-informed dialogue with the Employment Committee. This dialogue is important in developing EU employment policy which is applicable to Member States’ needs, for example to address the increased focus on policies to integrate people excluded from the labour market. 5.2.7.Potential for improving effectiveness Limited internal PES capacity, high staff workloads and infrastructure problems were seen as problematic by some PES consulted in the external study (Section 3.1). Despite technical and financial support at EU level, some PES have limited financial and human capacity to balance national and international commitments, particularly those PES with less resources from national governments. Moreover, many consultees stressed the influence of political decision-making as a potential barrier in cases where the political will is not present to encourage sustainable change. Some PES have experienced difficulties in demonstrating the benefits of the Network activities to their national governments in quantifiable terms. Changing national priorities influence whether change can take place at the national level. This is demonstrated by a combination of findings from the external consultation and the reports from PES on follow-up of benchlearning recommendations. Thus, for a mutual learning activity to be effective (meaning that change is achieved) it is important that it comes at the right time. Effectiveness also depends on the ability to find a balance between launching promising proposals and keeping new ideas within what could realistically be realised. This balance differs across PES, and can change over time. The benchlearning exercise has been important to improve evidence for designing better measures and recommendations at EU level that are also targeted at each PES. The 2017 interim review of the Decision highlights the benchlearning project as a positive innovation in that national institutions can have external references, receive constructive feedback from expert practitioners and engage in tailor-made learning with peers. The evidence gathered by the various consultation activities for this evaluation and reports from PES demonstrate that benchlearning and mutual learning activities have contributed to concrete changes in PES organisational processes and service delivery, thus rendering them more effective. These changes have the potential of contributing further to: (a) improving PES performance; and (b) effectively implementing employment policies at the national and local levels. 5.3.Efficiency 5.3.1.The Network has been efficient in delivering initiatives and reaching objectives The external study, Section 4.1, confirmed that the Network is considered both to be efficient and a good use of national (or regional) and European public resources by the vast majority of actors consulted both at EU level and in national PES (almost 100% of those who expressed an opinion). This view was also held by representatives of international organisations consulted, who felt that the Network provided a key forum where practical issues for PES performance and service delivery can be discussed and ensure efficiency gains by learning from others. However, the highest proportion of respondents in the public consultation was unable to determine the cost-effectiveness. The PES Secretariat has proven to be an important link between the Commission and the Network. The PES Secretariat consists of Commission staff and Seconded National Experts from European PES. This legitimately supports the Commission by being a source of information from and contacts with the Network. This close link is also contributing to the efficiency of the Network. 5.3.2.Costs Examining efficiency requires an assessment of the degree to which the objectives and activities of the Network have been delivered at an appropriate cost. This implies a need to assess the costs and benefits associated with the Network. However, many of the benefits in particular do not lend themselves to monetary valuation. Moreover, a four-year period is in some cases too short to realise the full gains of organisational or cultural changes that have been set in motion. A full cost-benefit analysis can therefore not be carried out.

The external contractor asked the national members of the PES Network to indicate time and other costs used in participating in Network activities (external study Section 4.2). Estimates were often approximate and some consultees did not want to share them or were not able to obtain information on costs. Furthermore, some respondents found it difficult or impossible to separate time used in Network activities from other activities, particularly when related to European or international tasks. The estimates can therefore only give indications of the use of resources. Moreover, estimations of time investments by PES differed quite widely by PES. There are also real differences across countries, depending on the involvement in the Network that should be considered when analysing the average. On average, the PES estimated that around 200 days of staff time per year were committed to Network activities. This approximates to one full-time equivalent per PES per year. Concerning costs such as travel, accommodation, translation and expert inputs, PES considered them hard to estimate as some of them were reimbursed by the Commission, while others could be merged into other budget items. At European level, the average annual cost is around €3.45 million per year, incurred by the European Commission and funded by the Employment and Social Innovation (EaSI) programme. This amount includes: (i) the annual budget to implement the Network’s work programme (€3.12 million); (ii) Commission staff devoted to the PES Secretariat, allowances paid to Seconded National Experts, mission costs for the Secretariat and for Commission representatives to the Board (about €200,000); and (iii) costs for the organisation of meetings (Board and AFEPA meetings) and reimbursing the participants (about €130,000). These resources are in accordance with Article 7 of the Decision. In addition, the Network Secretariat liaises with the AFEPAs and other experts at national level with a view to hosting and participating in learning events, data collections, etc. Compared, for example, to the total direct support from the ESF to reforms to modernise PES (€2.1 billion over the period 2014-2020), the support for this Network is small. Other networks such as the EURES Network, the European Social Policy Network (ESPN) and the European Labour Law Network are also receiving allocations from EaSI. This gives a scale of the financial support involved. In 2018, EaSI allocations to the EURES Network’s tasks related to network meetings, training and support activities was

€4.45 million. EaSI allocations in the field of labour law and employment and labour market policies amounted to €2.2 million to ESPN, while the European Centre of Expertise (ECE) received €2.0 million. However, it remains difficult to properly compare the costs of these networks due to the difference in their tasks and responsibilities. For example, the EURES Network organises training for advisers and line managers. As seen above, a substantial task for the Network covered by this evaluation is mutual learning across national PES. ESPN provides the Commission with an overview of policies on social protection and social inclusion, and acts as the secretariat to the Mutual Information Systems on Social Protection (MISSOC). The ECE is a network of labour law experts and provides an ad hoc capacity in labour market policies analysis through a flexible pool of experts. It assists the Commission in ensuring a correct application of EU law across all Member States and monitoring labour legislation reforms in the EU. 5.3.3.Benefits PES performance and impacts for the users of PES services depend on many different factors. Impacts on PES performance can be indirect, for example through changes in organisational culture, which is normally also affected by a number of other measures and channels. Furthermore, the causality of improvements, or how objectives have been achieved, cannot always be established. For instance, reduced unemployment can be a consequence of better PES performance, but also of changes in demand for labour. However, many PES identified in the external study cases where they felt that their PES and the national labour market had benefited from participation in the Network, for example: •new initiatives or processes introduced following Network activities; •participation in the Network facilitated bilateral relationships between PES; •Network participation stimulated internal learning, reflection and policy debate; •Network participation enhanced PES staff development, motivation and engagement. Many consultees highlighted in the external study that benchlearning and associated visits had a significant impact on helping individual PES understand how they compare with other PES, how they can plan for the future, and how they can evaluate their own activities. It is true that some of the examples mentioned in the consultations, such as new tools and development of staff, could to some extent have been implemented by the use of private sector consultants. This can, however, be costly. Moreover, such support is unlikely to be as relevant as the learning that can be obtained from experienced peers from other PES. Less advanced PES have benefited the most from participating in the Network in terms of progress in performance (see Figure 2). However, comparing maturity levels between the first and second cycles of benchlearning demonstrates that advanced PES have also progressed. 5.3.4.Balance of costs and benefits The benefits of the Network’s activities cannot be directly estimated, neither at the EU level nor the national level. How the Network members themselves view the benefits, however, is borne out by the willingness of PES to participate in benchlearning and mutual learning events and by the PES responses in the external consultation. In the external consultation (Section 4.3), 26 PES responded to the question about the costs and benefits of participation in the Network. Of these, 14 PES stated that benefits significantly exceeded costs. A further 5 stated that benefits slightly exceeded costs, while 3 reported that costs were equal to benefits. Only 1 PES felt that costs exceeded benefits. 3 PES responded that they were not sure. One of the respondents stated: ‘To our knowledge, the EU PES Network absorbs only a very limited amount of EU public resources, but coordinates more than one million public employees who are helping vulnerable and jobless across the Europe. With its activities aimed at better provision of quality services and efforts to improve internal efficiency, we believe that it is good value for money and HR spent.’ These results suggest that the ‘leverage’ of the Network is high, in that it yields benefits which considerably exceed the costs of participation, even if not equally distributed. The impact of scale was mentioned in the consultation as contributing to leverage. Participation in the Network means joining a vast network of PES employees across Europe.

Less mature PES are more likely to report net benefits of the Network participation than the more advanced ones. In some cases, where national budgets would be a barrier to Network participation, the financial investment from the EU was mentioned as a key contributor to the Network’s leverage. This indicates that the contributions from the EU are important both to ensure that the mature PES do not see Network participation as a burden and to allow the less mature PES to participate in activities which are crucial for their further development. 5.3.5.Potential to increase efficiency Some potential means to further improve efficiency were mentioned in the external study, Section 4.1. These relate mainly to better targeting of learning events and potential to further use digital solutions. Some consultees mentioned that study visits could be organised in smaller and more targeted groups, and to increase the focus on training of PES employees so that they can have a greater impact on Network activities on the ground. Finding solutions for the translation of materials, tools and guidelines is mentioned as a means to increase take-up of outputs by staff in local PES who may not speak English. A result of two cycles of benchlearning is the accumulation of an evidence base on the maturity and priorities of PES. This evidence base has made it easier to target learning events. This has paved the way for learning dialogues, a new format of learning event starting in 2019. This peer-coaching approach related to specific PES learning needs is designed to support national PES as they bring about change in an identified area for improvement. Greater efficiency could be obtained by reducing language barriers. Some consultees mentioned that many PES staff cannot read outputs in English. Linked to this, it can be difficult to find PES staff proficient in English who are experts on certain topics and who can attend Network activities. Lack of internal capacities as well as language barriers between experts can restrict the possibility to draw benefits from mutual learning. A higher focus on smaller, targeted events could therefore increase efficiency. Calls for experts to attend mutual assistance projects could, for example, put the emphasis on their knowing the native language in the receiving country. It was also mentioned that there could be a greater focus on dissemination, which might increase impacts, by targeting and distributing outputs more strategically. Efficiency can be increased by making better use of modern e-solutions and online interactive tools for communication, and through the exchange of best practices and dissemination of outputs. Two webinars were organised as part of the 2018 work programme and three webinars will be organised in 2019. However, as pointed out by several consultees during the consultations, personal contact between PES staff is the key to the success of the cooperation. It is therefore important to find the right balance between face-to-face and virtual meetings to achieve the efficiency goals. 5.4.Coherence 5.4.1.Consistency of the Decision with EU policies and initiatives The preamble of the Decision refers to the links between the Network’s responsibilities and other EU policies. Specifically, the Network should: •contribute to the implementation of policy initiatives in the field of employment; •support initiatives aimed at skills matching, decent and sustainable work, voluntary labour mobility and transition from education and training to work; •address the evaluation of active labour market policies; •ensure that it complements and does not replace or duplicate other EU initiatives. Consistency between legal frameworks as well as implementation is thus important. The Europe 2020 strategy is the EU’s agenda for growth and jobs for the current decade. One of the goals is to increase work participation to a rate of 75% for people aged 20-64. In the Council Decision on guidelines for the employment policies of the Member States, guideline 7 declares that the Member States should aim for better and more effective public employment services to reduce and shorten unemployment by providing tailored services to support jobseekers, supporting labour-market demand and implementing performance- measurement systems. The PES Decision clearly states in Article 3 that it aims to “encourage cooperation between Member States [...] in order to contribute to Europe 2020” and, in Article 4, that it should “contribute to modernising and strengthening PES in key areas, in line with the employment and social objectives of Europe 2020”. The new legal basis for EURES laid down in 2016 reinforces the obligations to share information and cooperate across Member States in areas such as apprenticeships. It also lays down a minimum set of support services that PES have to provide. EURES focuses on the pooling of job vacancies and applications and the provision of mobility services in view of facilitating free movement of workers and the further integration of labour markets. The external study explains that whereas the Network focuses primarily on cooperation to improve PES performance and contribute to implementation of employment policies, EURES directly targets employers, workers and jobseekers. To avoid duplication and ensure synergies, the Network Board decided in 2016 to promote closer cooperation between the Network and EURES. The Network has demonstrated the ability to adapt to new legal frameworks for EU policies. The dedicated and relevant chapters of the European Pillar of Social Rights, which was proclaimed by the European Parliament, the Council and the Commission in 2017 (equal opportunities and access to the labour market, fair working conditions and social protection and inclusion), which are aligned with the guidelines from the European employment strategy, have become guiding in setting the Network’s initiatives and actions. An example of this is the increased focus on the role of PES in outreach to the inactive population. The tasks of the Network also include monitoring implementation and carrying out analytical activities such as preparing reports and studies. The Network has established its identity and presence at European level and helps national PES to perform better and deliver employment policies more successfully, taking into account the impact of labour market trends. The Network’s activities have also contributed to better analysis as part of the European Semester and the country-specific recommendations (CSRs) it produces. In 2018, 11 Member States received CSRs related to their PES and active labour market policies (ALMPs). This includes needs for more personalised tailored services for the unemployed, a more comprehensive performance measurement system, better targeting and prioritisation, and better coordination between central and regional administrations. The CSRs focus for some Member States on measures to make ALMPs more effective and accessible to those furthest away from labour market. To address these challenges identified in the European Semester, the PES Network uses its benchlearning approach, striving for continuous performance improvement. One of its unique features is the attention given to organisational factors, drivers and practices that are likely to improve PES performance. Overall, the external study, Section 5.2, revealed that most stakeholders consider that there is a high degree of consistency between the Decision and the EU policy framework. Although improvements could be made in future to improve synergies, no duplication of action was observed. There is some cooperation with other relevant labour market stakeholders (as set out in Article 5 of the Decision) at EU level, but there is scope for improvement in future.

In particular, a large majority of the PES consultees pointed out that the Decision reflects the relevant priorities set out in the Europe 2020 strategy. This complementarity was also emphasised in relation to the shared goals and commitments included in the Decision and other European initiatives such as the Youth Guarantee, the Youth Employment Initiative, the Council Recommendation on the integration of the long-term unemployed into the labour market, and ESF-funded programmes. Synergies were highlighted in relation to the mandate of other EU institutional actors such as the EURES Network and EMCO. The fact that different institutional actors and networks share similar core objectives — e.g. reducing unemployment, improving labour market policies’ effectiveness and the inclusion of vulnerable groups — was acknowledged and seen in very positive terms by the majority of those consulted. The facilitation of information exchange and best practices across institutions and networks at the EU and national level was likewise deemed as particularly important in facilitating complementarity and coherence. 5.4.2.Contributions of the Network to relevant EU policies and initiatives The Network has contributed to implementation of a broad range of EU policy initiatives. Table 1 in Section 3.6 gives an overview of selected examples of the Network’s contributions to policy implementation. The Network’s contributions have been most extensive in implementation of the Youth Guarantee and the integration of the long-term unemployed into the labour market. The Network has also demonstrated flexibility to target new topics such as the integration of migrants and refugees, the prevention of unemployment and addressing skills shortages. The Network has also provided contributions to implementation of the European Pillar of Social Rights. As regards the contributions of the Network to the implementation of the CSRs under the European Semester, PES are continuing to innovate in their development of ALMPs, while general ALMP expenditure is increasing at many PES. Moreover, PES are adapting their staff deployment to the changing environment. This includes specialised teams offering tailored approaches and intensive support to target groups such as the long-term unemployed and customers with disabilities. For example, in 2017 Spain received a CSR to reinforce coordination between regional employment services, social services and employers so as to better respond to jobseekers’ and employers’ needs. In this context, the Spanish PES requested a mutual assistance project from the PES Network to help it work towards addressing the CSR. 5.4.3.Large degree of complementarity with the EU policy framework Other stakeholders confirmed the high degree of complementarity with the EU policy f