COMMISSION STAFF WORKING DOCUMENT EVALUATION Accompanying the document Proposal for a DECISION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL amending Decision No 573/2014/EU on enhanced cooperation between Public Employment Services (PES)
Date: 2019-09-11
Institutions: Directorate-General for Employment, Social Affairs and Inclusion, European Commission
Subjects: EU employment policy, administrative cooperation, employment service, integration into employment

EN 

  EN 

   EUROPEAN COMMISSION  Brussels, 11.9.2019  SWD(2019) 1350 final  COMMISSION STAFF WORKING DOCUMENT EVALUATION Accompanying the document Proposal for a  DECISION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL amending Decision No 573/2014/EU on enhanced cooperation between Public  Employment Services (PES)  {COM(2019) 620 final} - {SWD(2019) 319 final}  
 
 
Table of contents 

1.  INTRODUCTION...................................................................................................................... 5 

1.1.  Purpose ............................................................................................................................ 5 

1.2.  Scope ............................................................................................................................... 5 

2.  BACKGROUND TO THE INTERVENTION .......................................................................... 6 

2.1.  Description of the intervention and its objectives ........................................................... 6 

2.2.  Baseline and points of comparison ................................................................................ 10 

2.2.1. 

Pre-2014 cooperation model showed limitations .................................................. 10 

2.2.2. 

Enhanced cooperation between PES from 2014.................................................... 11 

3.  IMPLEMENTATION / STATE OF PLAY ............................................................................. 12 

3.1.  Governance of the Network .......................................................................................... 12 

3.2.  Benchlearning ................................................................................................................ 12 

3.3.  Mutual assistance .......................................................................................................... 14 

3.4.  Modernising PES in line with the objectives of Europe 2020 ....................................... 15 

3.5.  Analytical reports .......................................................................................................... 17 

3.6. 

Implementation of relevant policy initiatives ................................................................ 17 

3.7. 

Implementation of the European Social Fund ............................................................... 19 

3.8.  Annual work programme............................................................................................... 20 

3.9.  Promote and share best practices for people not in education, employment or training 

(NEETs) ............................................................................................................................ 20 

3.10.  Participation in learning events ..................................................................................... 20 

4.  METHOD ................................................................................................................................. 21 

4.1.  Short description of methodology and sources ............................................................. 21 

4.2.  Limitations and robustness of findings ......................................................................... 24 

5.  ANALYSIS AND ANSWERS TO THE EVALUATION QUESTIONS ............................... 25 

5.1.  Relevance ...................................................................................................................... 25 

5.1.1. 

The objectives have been relevant for the Network .............................................. 25 

5.1.2. 

The relevance of each objective ............................................................................. 25 

5.1.3. 
developments ......................................................................................................................... 27 

The objectives have been resilient to the changing labour market and policy 

5.1.4. 

The initiatives of the PES Network have been relevant ........................................ 28 

5.1.5. 

Potential for improving relevance .......................................................................... 28 

5.2.  Effectiveness ................................................................................................................. 29 

5.2.1. 

Quantitative effects of the enhanced cooperation between PES ........................... 29 

5.2.2.  Modernisation promoted by benchlearning ........................................................... 30 

5.2.3.  Mutual learning between PES has improved ......................................................... 31 

1 

 
5.2.4. 

Effectiveness of mutual assistance ........................................................................ 32 

5.2.5. 

PES have taken up the results of Network cooperation......................................... 32 

5.2.6. 

Network contributions to effective policy implementation ................................... 34 

5.2.7. 

Potential for improving effectiveness .................................................................... 34 

5.3.  Efficiency ...................................................................................................................... 35 

5.3.1. 

The Network has been efficient in delivering initiatives and reaching objectives 35 

5.3.2. 

Costs ...................................................................................................................... 35 

5.3.3. 

Benefits .................................................................................................................. 37 

5.3.4. 

Balance of costs and benefits ................................................................................ 37 

5.3.5. 

Potential to increase efficiency .............................................................................. 38 

5.4.  Coherence ...................................................................................................................... 39 

5.4.1. 

Consistency of the Decision with EU policies and initiatives ............................... 39 

5.4.2. 

Contributions of the Network to relevant EU policies and initiatives .................... 41 

5.4.3. 

Large degree of complementarity with the EU policy framework ......................... 41 

5.4.4. 

Potential to increase consistency ............................................................................ 42 

5.5.  EU added value ............................................................................................................. 42 

5.5.1. 

Impacts of the Decision ......................................................................................... 42 

5.5.2. 

The role of the Network in cooperation between PES .......................................... 43 

5.5.3. 

Potential for increased added value........................................................................ 44 

6.  CONCLUSIONS AND LESSONS LEARNED ...................................................................... 44 

Annex 1: Procedural information .................................................................................................. 48 

Annex 2: Stakeholder consultation — Synopsis report ................................................................. 50 

1.  Consultation strategy ................................................................................................................ 50 

1.1. Identification of stakeholders ............................................................................................ 50 

1.2.  Consultation activities ...................................................................................................... 51 

2.  Consultation activities .............................................................................................................. 51 

2.1. Public consultation ............................................................................................................ 51 

2.2. Targeted consultations ...................................................................................................... 52 

2.2.1. Targeted semi-structured interviews ........................................................................... 52 

2.2.2. Written questionnaires ................................................................................................. 53 

2.2.3. Workshop .................................................................................................................... 53 

2.2.4. Case studies ................................................................................................................. 54 

2.2.5. Summary and reflections on challenges ...................................................................... 54 

3.  Information on stakeholder groups .......................................................................................... 54 

3.1. Public consultation ............................................................................................................ 54 

3.1.1. Country of respondents ............................................................................................... 54 

2 

 
3.1.2. Sector of activity ......................................................................................................... 55 

3.2. Targeted consultations ...................................................................................................... 55 

3.2.1. Targeted semi-structured interviews ........................................................................... 56 

3.2.2. Workshop .................................................................................................................... 56 

3.2.3. Written questionnaires ................................................................................................. 56 

3.2.4. Case studies ................................................................................................................. 56 

4.  Methodology for data processing ............................................................................................. 57 

4.1. Public consultation ............................................................................................................ 57 

4.2. Targeted consultations ...................................................................................................... 57 

4.3. Case studies ....................................................................................................................... 57 

5.  Dissemination strategy implemented for the public consultation ............................................ 57 

6.  Overview of the results of the consultations ............................................................................ 58 

Annex 3: More detailed intervention logic of the evaluation ........................................................ 62 

Figures, Boxes and Table 

Figure 1: Summary of the intervention logic of the Decision 
Figure 2: Increase in benchmarking scores from the first to the second cycle 
Figure  3:  PES  participation  across  all  mutual  learning  activities,  2011-2014  and  2015-
2018 
Figure 4: Summary of the evaluation process 
Figure 5: Composition of the PES population 
Figure 6: Number of dashboard sessions per country 

Box 1: The five pillars of benchlearning 
Box 2: Mutual assistance — Cyprus 
Box 3: Mutual assistance — Romania 
Box 4: Examples of reform initiatives resulting from the benchlearning initiative 

Table  1:  Selected  examples  of  contributions  of  the  Network  to  relevant  policies  and 
initiatives 

3 

 
 
 
 
Glossary 

Term or acronym 

Meaning or definition 

AFEPA 

ALMP 

CSR 

EaSI 

ECE 

EMCO 

ESF 

ESPN 

EURES 

FTE 

HoPES 

ILO 

LTU 

Adviser for European PES affairs 

Active labour market policies 

Country-specific recommendation 

Employment and social innovation programme 

European Centre of Expertise 

Employment Committee 

European Social Fund 

European Social Policy Network 

European Network of Employment Services 

Full-time equivalent 

Heads of public employment services 

International Labour Organization 

Long-term unemployed  

MISSOC 

Mutual Information Systems on Social Protection 

ML 

NEET 

NGO 

OECD 

PARES 

PC 

PDCA 

PES 

Network 

PrES 

TFEU 

WAPES 

Mutual learning 

Not in employment, education or training 

Non-governmental organisation 

Organisation for Economic Cooperation and Development  

Partnership between Employment Services Initiative 

Public consultation 

Plan-do-check-act 

Public employment services 

European Network of Public Employment Services 

Private employment services 

Treaty on the Functioning of the European Union 

World Association of Public Employment Services 

4 

 
1.  INTRODUCTION 

1.1.  Purpose 

Decision  No  573/2014/EU  on  enhanced  cooperation  between  Public  Employment 
Services  (PES1)  (the  ‘Decision’)  establishes 
the  European  Network  of  Public 
Employment Services (the ‘Network’) from 17 June 2014 until 31 December 2020. 

Although  not  required  by  the  Decision,  this  evaluation  has  been  undertaken  in  the 
interest of good administration and in line with the ‘evaluate first’ principle to take into 
account  the  lessons  of  past  EU  action  and  to  better  design  future  interventions.  This 
evaluation assesses the state of play and degree  of implementation of the Decision, and 
whether the Network has fulfilled its legal obligations and achieved its objectives, further 
described  in  Section  2.  The  evaluation  covers  all  five  criteria  set  out  by  the  Better 
Regulation requirements: 

• 

•  Relevance:  the  extent  to  which  the  Decision,  in  particular  the  Network’s 
objectives and initiatives as outlined in Articles 3 and 4 respectively, have been 
relevant  and  remain  so  in  the  current  labour  market  context  and  the  social 
situation in the EU. 
Effectiveness: the extent to which the Network has been effective in delivering 
the initiatives mentioned in the Decision (Article 4) and reaching its objectives 
(Article 3).  This  includes  the  extent  to  which  the  Decision’s  objectives  have 
been effectively implemented, assessing indicators of relative success, and their 
contribution to achieving the Decision’s objectives. 
Efficiency: the extent to which the Network has been efficient in delivering the 
initiatives  mentioned  in  the  Decision  and  reaching  its  objectives,  and  whether 
the costs were proportionate to the benefits achieved. 

• 

•  Coherence: the extent to which the Decision and the initiatives under its aegis 
are  complementary  with  other  EU  or  national  policies  and  initiatives  with 
similar objectives. 
EU  value  added:  the  extent  to  which  the  Decision  has  generated  added  value 
for the EU as a whole compared to the situation prior to or in the absence of the 
Decision’s entry into force. 

• 

Based on the assessments of each criteria, the evaluation draws some overall conclusions, 
including lessons learned. The results of the evaluation will feed into reflections on the 
possible future cooperation between European public employment services post-2020. 

1.2.  Scope 

The geographical scope of the Network is the EU Member States and the EEA countries 
Iceland and Norway. Inclusion of the EEA countries is supported by their participation in 

1 https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv%3AOJ.L_.2014.159.01.0032.01.ENG 

5 

 
                                                           
the  Employment  and  social  innovation  (EaSI)  programme.  This  has  also  contributed  to 
extending the range of members in the Network. 

The  timeframe  for  the  evaluation  covers  the  period  from  the  entry  into  force  of  the 
Decision establishing the Network, which is 17 June 2014, until 31 July 2018. The cut-
off date was set to maximise the availability of data in the evaluation. Secondary sources 
used  in  the  evaluation  can  have  different  cut-off  dates.  For  example,  the  2018  annual 
report of the European  Network of Public Employment Services (PES)2 covers the full 
calendar  year.  The  evaluation  focuses  on  the  extent  to  which  the  Decision  has  been 
implemented, and the extent to which this has so far led to the expected outputs. 

2.  BACKGROUND TO THE INTERVENTION 

2.1.  Description of the intervention and its objectives 

Public  employment  services  are  the  main  agencies  carrying  out  employment  policies 
directly  accountable  to  governments.  They  are  set  up  to  facilitate  the  labour  market 
integration  of  jobseekers  and  the  payment  of  unemployment  and/or  welfare  benefits. 
Although structured differently in each country, all PES help match supply and demand 
in the labour market through information, placement and active support services at local, 
national  and  European  levels.  PES  are  also  the  main  actors  implementing  activation 
policies in the Member States, and they play an important role in facilitating successful 
labour  market  transitions  and  integration.  The  quality  of  their  services  has  direct 
consequences  for  the  impact  of  employment  policies  on  the  ground.  All  of  the  above 
make PES key actors in fighting unemployment in Europe and in ensuring the success of 
the EU 2020 strategy for jobs, and smart, sustainable and inclusive growth. 

Performance differs considerably across PES, depending on business models, capacity in 
terms of staff and resources, and management  approaches. Additionally,  PES operate in 
different  labour  market  contexts.  They  face  challenges  due  to  changes  in  the  way  the 
labour market functions and experience fluctuating demand from citizens, for instance due 
to  the  increased  unemployment  rates  following  the  economic  crisis.  Shrinking  public 
budgets  and  the  need  for  increased  cost-effectiveness  of  PES  have  prompted  several 
Network members to undertake reforms. The PES need to respond to these challenges, to 
changes on the labour market and to customers’ expectations of service delivery. 

European-level collaboration between PES started in 1997, when the Commission set up 
an informal advisory group of the heads of PES (the European Network of Heads of PES 
(HoPES3)). The aim was to promote cooperation, exchange and mutual learning between 
the member organisations and to receive feedback on employment policy initiatives. 

Building  on  this,  in  2013  the  Commission  proposed  to  formalise  this  cooperation  to 
support  innovation,  benchmarking  and  mutual  learning  at  European  level.  This  was 
achieved  through  the  Decision  of  15 May  2014  establishing  the  European  Network  of 

2 https://ec.europa.eu/social/BlobServlet?docId=20613&langId=en 
3 https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv%3AOJ.L_.2014.159.01.0032.01.ENG 

6 

 
                                                           
Public  Employment  Services.  The  Network’s  objective  is  to  strengthen  the  capacity, 
effectiveness  and  efficiency  of  public  employment  services  by  providing  a  platform  for 
comparing  their  performance  at  European  level,  identifying  good  practices  and 
establishing a mutual learning system. It also aims to give PES more opportunities to help 
develop innovative, evidence-based policies in line with the Europe 2020 objectives. 

Figure 1 below, summarising the intervention logic of this evaluation, gives an overview 
of needs for stronger and more formalised cooperation. The intervention logic includes the 
objectives, the problems that  the interventions under the Decision intended to solve and 
how it was expected to work. 

The Network contributes to the European employment strategy to create more and better 
jobs throughout the EU, an integral part of the ‘Europe 2020’ strategy for jobs, and smart, 
sustainable and inclusive growth. 

The objective of the Decision, as stated in Article 3, is to encourage cooperation between 
PES  through  a  formalised  network,  in  order  to  contribute  to  the  ‘Europe 2020’  strategy 
and to the implementation of relevant EU policies. In this way the Network will support: 

(a)  the  most  vulnerable  social  groups  with  high  unemployment  rates,  especially  older 
workers and young people not in employment, education or training (NEETs); 

(b) decent and sustainable work; 

(c) the better functioning of labour markets in the EU; 

(d) the identification of skills shortages and the provision of information on their extent 
and location, as well as the better matching of jobseekers’ skills with employers' needs; 

(e) the better integration of labour markets; 

(f) increased voluntary geographical and occupational mobility on a fair basis to meet 
specific labour market needs; 

(g) the integration of people excluded from the labour market as part of the fight against 
social exclusion; 

(h) the evaluation and assessment of active labour market initiatives and their effective 
and efficient implementation. 

7 

 
 
 
Figure 1: Summary of the intervention logic of the Decision 

NEEDS 

  Strengthen  PES  service  capacity  and  efficiency,  as  efficient  PES  are  an  essential  factor  for  the 

 

successful implementation of employment policies and measures 
Increase the PES potential to contribute to the development of policy implementation measures in line 
with the Europe 2020 objectives. 

OBJECTIVE 

Strengthen  cooperation  between  the  PES  of  the  Member  States  +  Iceland  and  Norway  with  a  view  to 
improving  PES  performance  and  contributing  to  the  implementation  of  policy  initiatives  in  the  field  of 
employment 

Financial resources: ca. €3.5 million/year over 2014-2020 
Staffing of PES Network Secretariat, liaisons (AFEPAs) at national level 

ACTIVITIES 

  Benchmarking self-assessments, site visits, external assessments 
  Provision of mutual assistance among PES via mutual learning events 
  PES Network governance meetings 

OUTPUTS 

  Benchlearning reports and recommendations among PES to compare their activities in given areas (e.g. 

contribution to reducing unemployment) 

  Studies, toolkits, PES practices, PES Knowledge Centre 
  Learning dialogues starting in 2019 
 

Increased cooperation and exchange among PES 

 
  Follow-up and implementation of the benchlearning recommendations 

IMPACTS 

  Strengthened and modernised PES 
  Better performing PES (efficiency and effectiveness) 
  Better support to the unemployed and the inactive not registered with PES 
  Better functioning labour markets 
  Better matching and integration of labour markets 
 
  Lower unemployment, including for vulnerable groups 
  Contribution to meeting the EU2020 target of a 75% employment rate 

Increased voluntary geographical and occupational mobility 

Source:  Study  supporting  the  evaluation  of  Decision  573/2014/EU  on  enhanced  cooperation  between 
Public Employment Services, Ecorys for the European Commission, 2019. 

8 

 EXOGENOUS FACTORS   
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Article 4 of the Decision sets out the initiatives and activities to be implemented by the 
Network: 

‘(a) 
the  development  and 
benchlearning  among  PES 
performance of their activities in the following areas: 

implementation  of  Union-wide,  evidence-based 
the 
to  compare,  with  appropriate  methodology, 

(i)  contribution  to  reducing  unemployment  for  all  age  groups  and  for 
vulnerable groups; 
(ii)  contribution  to  reducing  the  duration  of  unemployment  and  reducing 
inactivity, so as to address long-term and structural unemployment, as well 
as social exclusion; 
(iii) filling of vacancies (including through voluntary labour mobility); 
(iv) customer satisfaction with PES services; 

(b)  the  provision  of  mutual  assistance,  either  in  the  form  of  peer-to-peer  or  group 
activities,  through  cooperation,  exchanges  of  information,  experiences  and  staff 
between the members of the Network, including support for the implementation  of 
PES-related  country-specific  recommendations  issued  by  the  Council  upon  request 
by the Member State or the PES concerned; 

(c)  contribute  to  modernising  and  strengthening  PES  in  key  areas,  in  line  with  the 
employment and social objectives of Europe 2020; 

(d)  prepare  reports  at  the  request  of  the  European  Parliament,  the  Council  or  the 
Commission, or on its own initiative; 

(e) contribute to the implementation of relevant policy initiatives; 

(f) adopt and implement its annual work programme setting out its working methods, 
deliverables and the details related to the implementation of benchlearning; 

(g)  promote  and  share  best  practices  on  the  identification  of  NEETs  and  on  the 
development of initiatives to ensure those young people gain the skills necessary to 
enter and remain in the labour market.’ 

The  aim  of  the  Decision  is  to:  (i)  increase  cooperation  among  PES  and  the 
implementation of learning and recommendations achieved through this cooperation; (ii) 
contribute to better performing PES and better functioning labour markets. Contribution 
to  better  implementation  of  EU  policy  initiatives  such  as  the  EU2020  targets  is  an 
overarching goal. Due to the broad range of themes and responsibilities covered by the 
Decision,  the  outputs  take,  among  others,  the  form  of  studies  and  analysis,  learning 
events, and visits. Dissemination and sharing of knowledge is of key importance. 

9 

 
 
2.2.  Baseline and points of comparison 

2.2.1. Pre-2014 cooperation model showed limitations 

As  outlined  in  the  explanatory  memorandum  for  the  2013  proposal  for  the  Decision4, 
experience  had  shown  that  PES  were  not  engaging  sufficiently  in  mutual  learning  and 
benchmarking5 activities by themselves (see further details in Figure 3). In summary, four 
main forms of PES cooperation were in operation. 

• 

•  Voluntary  mutual  learning  opportunities  via  the  ‘PES  to  PES  Dialogue’.  This 
EU-funded  programme  was  established  to  financially  support  learning  across 
PES and the implementation of efficient policies. 
Strategic  dialogue  between  different  employment  services  (public,  private  and 
non-profit)  in  the  context  of  the  Partnership  between  Employment  Services 
(PARES) initiative. 
Informal, voluntary cooperation of the heads of employment services (HoPES) 
via the HoPES Network. 
Bilateral or multilateral cooperation between staff at different PES levels in the 
context of self-funded exchanges or EU-funded projects. 

• 

• 

Available  evidence  from  the  PES  to  PES  Dialogue  programme  (annual  reports, 
dissemination of conference papers) indicates that the programme focused on exchanges 
based  on  specific  guidelines  from  the  European  employment  strategy.  Guidelines 
addressed  for  example in  2014  focused  on:  (i)  increasing  labour market  participation of 
men and women, reducing structural unemployment and promoting job quality (Guideline 
7);  and  (ii)  developing  a  skilled  workforce  responding  to  labour  market  needs  and 
promoting lifelong learning (Guideline 8). 

The activities of the PES to PES Dialogue programme included: (i) conferences; (ii) peer 
reviews  intended  to  identify  good  practices,  explore  them  and  disseminate  their  core 
elements; and (iii) follow-up study visits providing ad hoc support provided when needed 
and requested by PES. High-level advice and expertise was delivered by peer PES experts 
to  increase the  operational  capacity  of  PES,  to  improve  service  offer  and  delivery  or  to 
implement  good practices  highlighted  by  the  peer  review.  In  addition,  ad hoc  analytical 
papers were published on various themes6. 

Despite progress over the years, this cooperation model showed considerable limitations: 

• 

• 

Since participation of national PES in these activities was voluntary, this made it 
harder  to  quickly  identify  low  performance  by  PES  and  potential  structural 
labour market challenges deriving from this.  
There  was  no  reporting  mechanism,  meaning  that  policy  designers  at  national 
and  EU  level  were  not  systematically  informed  about  the  results  of  the 

4 https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52013PC0430&from=EN 
5 Definition of benchmarking and benchlearning are given under point 3.2 on p.12. 
6  The  PES  Knowledge  Centre,  established  as  an  initiative  under  the  Decision,  includes  all  papers  and 

reports back to 2008 https://ec.europa.eu/social/main.jsp?catId=1163&langId=en 

10 

 
                                                           
• 

cooperation.  
The  linkages  between  benchmarking  and  mutual  learning  activities  were  weak 
and inconsistent, and the evidence base for the activities of the mutual learning 
programme were not scientifically robust.  

•  Not  all  PES  participated,  and  often  the  least  mature  PES  and  those  with  the 

• 

greatest labour market challenges participated the least.  
PES developed their own tools and approaches (e.g. for digitalisation) at national 
level,  rather  than  saving  time  and  money  by  learning  from  the  more  advanced 
PES. 

As  a  result  of  the  above,  efforts  to  make  PES  more  comparable  by  clustering  them 
according to business models had not succeeded by 2013. 

2.2.2. Enhanced cooperation between PES from 2014 

The aim of the Decision was to fill the gap between the pre-2014 situation and the new 
requirements for PES to contribute effectively to the Europe 2020 strategy. To contribute 
to the work of the Employment Committee (EMCO), which supports the coordination of 
the Member States'  employment  policies,  PES  were  required to  adapt  their  organisation 
models,  business  strategies  and  processes  faster  than  what  had  been  achieved  by  the 
previous, informal network. Formalising the Network was identified as an effective way 
to  provide  a  platform  for  comparing  PES  performance  across  countries,  identify  good 
practices and foster mutual learning. 

While Member States would remain responsible for organising, staffing and running their 
PES, a formal PES Network would expand, reinforce and consolidate benchmarking and 
learning  initiatives  for  the  benefit  of  all  PES.  The  Network  was  expected  to  provide  a 
platform  for  comparing  their  performance  at  European  level,  identifying  good  practices 
and  fostering  mutual  learning  to  foster  learning  organisations  to  strengthen  service 
capacity and efficiency. It constituted a concerted action to modernise and empower PES 
to successfully act in unison in the face of the then predominant economic crisis. 

A European PES Network established on a solid legal foundation was expected to be able 
to ramp up coordination between PES and provide the Network with legitimacy to act. A 
formalised  structure  was  also  considered  a  pre-condition  for  increasing  the  Network’s 
potential  to help develop innovative, evidence-based policy implementation measures in 
line  with  the  Europe  2020  objectives.  The  Network  was  also  expected  to  feed  into  the 
European  Semester  surveillance  process  by  identifying  low-performing  PES  at  an  early 
stage before their problems became structural. Formalising the Network was also expected 
to  facilitate  implementation  of  labour  market  projects  financed  by  the  European  Social 
Fund (ESF), and the proposed initiative would contribute to improved cost-efficiency. The 
Network  established  under  the  Decision  would  carry  out  initiatives  in  the  form  of 
‘incentive measures’ provided for under Article 149 of the Treaty on the Functioning of 
the European Union (TFEU) designed to improve cooperation between Member States in 
the  employment  field.  The  Network’s  initiatives  would  be  in  addition  to  cooperation 
among PES within the European Network of Employment Services (EURES) on the basis 
on Articles 45 and 46 TFEU. 

11 

 
3.  IMPLEMENTATION / STATE OF PLAY 

This  section  describes  the  Network’s  governance  structure  and  the  implementation  of 
each of the initiatives provided for under Article 4 of the Decision. Where thematically 
appropriate, some initiatives have been linked together. 

3.1.  Governance of the Network 

The Network has a governing board made up of the heads of PES from each EU Member 
State and from Norway and Iceland, plus a representative from the Commission. EMCO 
has  observer status.  The Board  appoints  a  chair  and  a  vice  chair,  while the  second  vice 
chair  is  held  by  the  Board  member  from  the  PES  of  the  country  holding  the  EU 
Presidency.  The  chair  represents  the  Network.  The  Board  meets  twice  a  year  in  the 
Member State that holds the EU Presidency. It sets the strategic direction of the Network, 
debates 
the  work  programme’s 
implementation.  Each  Board  member  appoints  an  Advisor  for  European  PES  Affairs 
(AFEPA) to help run the Network. AFEPAs meet twice a year in Brussels. 

labour  market  developments  and  monitors 

The  Network’s  Board  is  assisted  by  a  Secretariat  provided  by  the  Commission 
(specifically  by  the  Directorate-General  for  Employment,  Social  Affairs  and  Inclusion 
(DG EMPL)). The Secretariat, in cooperation with the chair and vice-chairs, prepares the 
Board meetings and helps implement the annual work programme. It also organises and 
chairs the meetings of the AFEPAs. 

3.2.  Benchlearning 

Benchlearning  is  defined  in  the  Decision  as  the  ‘process  of  creating  a  systematic  and 
integrated  link  between  benchmarking  and  mutual  learning  activities  that  consists  of 
identifying  good performances through indicator-based benchmarking systems […] and 
of using findings for tangible and evidence-informed mutual learning activities, including 
good  or  best  practice  models’.  The  aim  is  to  support  each  PES  in  improving  their 
performance through comparisons and institutional learning from peers. 

Box 1. The five pillars of benchlearning 

1.  Quantitative  assessment  creating  transparency  on  PES  performance  by 

collecting, validating and analysing PES data; 

2.  Qualitative  assessment  of  PES  performance  against  performance  enablers, 
achieved  through  self-assessment  followed  by  a  peer/Commission/external 
expert assessment on the basis of a site visit; 

3.  Systematic identification of good practices collated online, such as in the PES 

Knowledge Centre; 

4.  A  mutual  learning  programme  which  builds  on  the  results  of  and  supports 
better PES performance by focusing on identified strengths and weaknesses; 

5.  PES  follow-up  of  the  benchlearning  assessment,  by  drawing  up  and 
implementing action plans to improve performance. Peer-to-PES coaching and 
learning through learning dialogues. 

12 

 
The  benchlearning  concept  is  based  on  five  pillars,  as  explained  in  Box  1  above. 
Quantitative  and  qualitative  assessments  of  PES  performance  build  the  benchmarking 
side.  This  is  the  basis  for  performance  transparency.  Knowing  about  PES  structures, 
processes  and  services,  as  well  as  their  outcomes,  makes  it  possible  to  identify  good 
practices  and  mutual  learning  activities.  This  leads  to  PES  action  plans  to  improve 
performance, benefiting from peer PES support. 

In  the  context  of  the  benchmarking  (quantitative  and  qualitative  assessments),  a 
structured  and  systematic  analysis  has  been  conducted  of  PES  results  and  its  drivers 
(‘performance enablers’).  In 2015, a thorough analysis of performance data (quantitative 
benchmarking) was carried out and a process of PES self-review, peer review and expert 
review  (qualitative  benchmarking)  was  implemented.  The  quantitative  and  qualitative 
performance indicators are listed in the Annex to the Decision. 

On quantitative assessment, each year PES administrative data is collected, validated and 
analysed to provide the eight mandatory benchmarking indicators listed in the Annex 
to the Decision: 

1.  Contribution  to  reducing  unemployment  for  all  age  groups  and  for  vulnerable 

groups: 

a)  Transition  from  unemployment  into  employment  per  age  group,  gender 
and  qualification  level,  as  a  share  of  the  stock  of  registered  unemployed 
persons; 

b)  Number of people leaving the PES  unemployment  records,  as a share of 

registered unemployed persons. 

2.  Contribution  to  reducing the duration of unemployment  and  reducing inactivity, 
so  as  to  address  long-term  and  structural  unemployment,  as  well  as  social 
exclusion: 

a.  Transition  into  employment  within,  for  example,  6  and  12  months  of 
unemployment per age group, gender and qualification level, as a share of 
all PES register transitions into employment; 

b.  Entries into a PES register of previously inactive persons, as a share of all 

entries into that PES register per age group and gender. 

3.  Filling of vacancies (including through voluntary labour mobility): 

a.  Job vacancies filled; 
b.  Answers to Eurostat’s Labour Force Survey on the contribution of PES to 

the finding of the respondent’s current job. 

4.  Customer satisfaction with PES services: 
a.  Overall satisfaction of jobseekers; 
b.  Overall satisfaction of employers. 

Areas  of  benchmarking  through  qualitative  internal  and  external  assessments  of 
performance are as listed in the Annex to the Decision: 

1.  Strategic performance management; 
2.  Design  of  operational  processes  such  as  effective  channelling  and  profiling  of 

jobseekers and tailored use of active labour market instruments; 

3.  Sustainable activation and management of transitions; 

13 

 
4.  Relations to employers; 
5.  Evidence-based design and implementation of PES services; 
6.  Effective management of partnerships with stakeholders; 
7.  Allocation of PES resources. 

The qualitative  assessment  was conducted in  two cycles: 2015/2016 and  2017/2018.  In 
the second cycle, the assessment was enriched by a focus on changes implemented since 
the first cycle and by introducing change management as an additional performance area. 

The  exercise  is  based  on  the  ‘PDCA  cycle’  (plan,  do,  check,  act).  This  means  that  the 
PES  are  assessed  on:  (i)  their  planning  and  implementation  of  activities  within  each 
performance area; (ii) how they check implementation; and (iii) how they react to needs 
for revisions or other actions. 

By the end of 2018, all PES participating in the Network had been visited and assessed 
twice7.  At  the  end  of  each  visit,  each  PES  received  a  feedback  report  containing 
qualitative  assessments  and  performance  scores;  with  6  as  the  best  score  and  1  as  the 
lowest. This provides PES with insight into their strengths and weaknesses in the various 
performance  areas,  provides  practical  suggestions  for  further  improvements,  and 
identifies peer PES as potential partners for exchange (for the first cycle). The feedback 
report also provides insight into the relevance, coherence and consistency of the reform 
agenda and an assessment of the PES’s change management (for the second cycle). 

As follow-up, reports on change are submitted by the PES one year after the visit. These 
reports  give an overview of the recommendations to  which the PES  has  given priority, 
and of the actions taken to bring about improvement in the specific area. 

The  results  of  the  benchlearning  assessments  have  over  the  past  four  years  been 
increasingly used as an evidence base to develop and steer the Network’s mutual learning 
programme,  as  reflected  in  its  annual  work  programmes8.  Thematic  clusters  such  as 
digitalisation,  services  to  employers  and  performance  management  were  central  to  the 
2018 programme. Finally, a collection of potentially transferable PES practices identified 
during the assessments and the mutual learning activities were shared within the Network 
for learning purposes. In the PES Practice Repository, more than 170 PES practices had 
been presented by the end of 2018. 

3.3.  Mutual assistance 

Results  and  assessments  from  the  benchlearning  programme  have  helped  PES  identify 
performance areas or tasks where support from other PES could be an appropriate method 
for  initiating  or  implementing  reform.  The  benchlearning  recommendations  and  the 
country-specific recommendations coming out of the European Semester process form the 
basis for selecting performance areas or tasks. 

7 With the exception of the UK, which was only visited in the first cycle and notified the Commission that 

it was not participating in the second cycle. 

8 Annual reports on the Network’s activities are available for 2015-2018: 

http://ec.europa.eu/social/main.jsp?catId=1100&langId=en 

14 

 
                                                           
Through mutual assistance, the selected PES receive technical support from peer PES to 
develop specified performance areas or tasks upon request. This includes support for the 
implementation  of  European  Semester  country-specific  recommendations9.  Between 
2015 and 2018, mutual assistance projects lasting between 9 and 18 months were carried 
out in Bulgaria, Cyprus, Greece, Italy, Romania and Spain. PES colleagues from Austria, 
from VDAB (the PES for Belgium’s Flanders Region), from Denmark, Estonia, France, 
Germany, Ireland, Latvia, Slovenia, Spain, Sweden and the United Kingdom participated 
in the projects as peer experts. 

The projects covered a broad range of tasks. For example, the project in Spain focused on 
three  main  topics:  a  common  profiling  tool,  customer  satisfaction  and  partnership  with 
employers. The aim was to achieve better coordination between the country’s 17 regional 
employment services, social services and employers so as to better respond to customers’ 
needs. At this stage, it is too early to assess the impacts on coordination and the role of 
the central PES. The Cypriot PES requested support to enhance its capacity to serve the 
long-term  unemployed  and  to  improve  outreach  to  non-registered  unemployed  people. 

Box 2. Mutual assistance — Cyprus 

The mutual assistance project in Cyprus took place in 2017. A follow-up meeting was 
held in 2018. The project focused on the following thematic areas: 

•  Better customer segmentation; 
• 
• 

Expanded use of IT for self-service provision; 
Evidence-based  design  and  enhanced  delivery  of  active  labour  market 
policies  (ALMPs)  /  improved  activation  services  (including  outreach)  for 
jobseekers  especially  from  the  most  disadvantaged  groups,  e.g.  the  young, 
the disabled, the long-term unemployed; 
Services for employers;  
Performance management/organisational culture/quality management. 

• 
• 

The benchmarking site visit identified the following progress as a consequence of the 
mutual  assistance:  improvement  in  digital  services,  segmentation,  better  monitoring 
of ALMPs and implementation of a new service offer to  employers. Success factors 
included: help in identifying key reform priorities, developing a clear action plan and 
committing to implementing it, and achieving high staff motivation. 

This is further explained in Box 2. 

Source: PES Secretariat/European Commission 

9  https://ec.europa.eu/info/strategy/european-semester/european-semester-timeline/eu-country-specific-

recommendations_en 

15 

 
 
                                                           
3.4.  Modernising PES in line with the objectives of Europe 2020 

Modernising  a  PES  involves  increasing  its  level  of  professionalism,  which  in  turn 
increases  its  maturity  as  an  organisation.  As  explained  in  Section  3.2  above,  PES 
performance is scored on a scale from 1 to 6 as part of the benchlearning exercise. The 
following rule is used to approximate maturity in the benchlearning exercise: 

•  A  mature organisation with  respect  to  a specific performance area is  achieved 

when all four scores in the PDCA cycle are 5 or higher. 

•  A  well-developed  organisation  with  respect  to  a  specific  performance  area  is 

achieved when at least three of the four scores are 4 or higher. 

•  A developing organisation with respect to a performance area is achieved when 

at least three of the four scores are 3 or higher. 
In all other cases, the maturity of the organisation is considered ‘developable’. 

• 

Low  scores  indicate  performance  areas  where  a  PES  can  improve,  while  high  scores 
indicate  that  PES  is  strong  in  these  areas.  Among  the  29  PES  that  had  received  their 
second  cycle  scores  by  the  end  of  2018,  9  PES  were  at  the  overall  level  assessed  as 
‘mature’ or ‘well developed10’. 

The comparison between the first and second benchlearning cycle shows that on average 
PES maturity gradually increased (see Figure 2). Out of 28 countries included in Figure 
2,  6  improved  their  scores  by  at  least  20%  from  the  first  cycle  to  the  second,  and  10 
countries improved by at least 10%. The remaining PES show moderate or no progress at 
all.  Figure  2  shows  that  Cyprus,  Bulgaria  and  Greece,  which  have  received  mutual 
assistance, (see Section 3.2 above), all belong to the group of PES with a high increase in 
scores from the first to the second cycle. 

Figure 2: Increase in benchmarking scores from the first to the second cycle 

10 https://ec.europa.eu/social/BlobServlet?docId=20613&langId=en 

16 

 
                                                           
Source: PES Secretariat/European Commission 

Some  PES  had  a  high  maturity  level  already  in  the  first  cycle.  Their  room  for  further 
improvements could be more restricted than for PES assessed as less mature in the first 
cycle. 

3.5.  Analytical reports 

Research and studies on PES topics related to priorities of the annual work programme 
are  published  each  year  at  the  request  of  the  European  Parliament,  the  Council  or  the 
Commission, or on the initiative of the Network. Since 2015, about 60 papers and reports 
have  been  published  in  the  PES  Knowledge  Centre11.  The  reports  enable  members  and 
practitioners to learn about initiatives across the EU and access their impact. The topics 
of the reports include PES practices and toolkits as well as analytical papers. Examples of 
reports  from  2018  include  a  study  report  on  PES’  role  in  outreach  to  the  inactive 
population12 and an analytical paper on early activation and employment promotion13. 

3.6.  Implementation of relevant policy initiatives 

The Network delivered a common response to the consultation on the European Pillar of 
Social Rights after assessing its impact on the Network Decision and reflecting on new 
trends  in  work  patterns  and  society.  In  line  with  the  Pillar’s  principles,  the  Network 
updated its strategy document ‘EU Network of Public Employment Services Strategy to 
2020 and beyond’, demonstrating the Network’s ability respond to new EU policies and 
initiatives.  Some examples of its contributions are set out in Table 1 below. 

The overview in Table 1 demonstrates that the Network has contributed extensively to the 
implementation of EU relevant policies and initiatives. Its contributions have been more 

11 https://ec.europa.eu/social/main.jsp?catId=1163&langId=en 
12 https://ec.europa.eu/social/BlobServlet?docId=19219&langId=en 
13 https://ec.europa.eu/social/BlobServlet?docId=19912&langId=en 

17 

 
                                                           
extensive in areas such as implementation of the Youth Guarantee and the integration of 
the  long-term  unemployed  into  the  labour  market.  The  Network  has  also  provided 
contributions to other relevant policy areas such as the European Pillar of Social Rights. 

18 

 
 
Table 1: Selected examples of the Network’s contributions to relevant policies and initiatives 

Policies/initiatives 

Examples of contributions by the Network 

Council 
Recommendation of 
22 April 2013 on 
establishing a Youth 
Guarantee 

Council 
Recommendation of 
15 February 2016 
on the integration of 
the long-term 
unemployed into the 
labour market 

European Pillar of 
Social Rights 

• 
• 

• 

• 

Formal role in monitoring as enshrined in the Recommendation 
PES  Network  self-assessment  report  on  PES  capacities  to  implement  the 
Youth Guarantee, 2014 
PES  Network  Catalogue  of  Measures  for  implementation  of  the  Youth 
Guarantee, 2014 
Summary  Report:  PES  Network  Conference:  Implementation  of  the  Youth 
Guarantee — Challenges and Success Factors, 2016 

•  Annual reports on PES Implementation of the Youth Guarantee 

• 
• 
• 
• 

• 

• 

Formal role in monitoring as enshrined in the Recommendation14 
Peer review on PES approaches for sustainable activation of LTU 
PES LTU Working Group, 201615 
PES  Network  contribution  to  the  consultation  on  long-term  unemployment, 
2015 
Proposal  to  EMCO  for  quality  standards  to  monitor  implementation  of  the  
Recommendation, 2016 
Summary  report:  PES  Network  Conference:  on  the  integration  of  the  long-
term unemployed, 2016 

•  Thematic review workshop on ‘Sustainable integration into the labour market’, 

November 2016 

•  Analytical paper: Sustainable integration into the labour market, 2017 
• 

PES Network study on assessment and early intervention to prevent long-term 
unemployment, 2017 

•  Ad hoc module to the 2018 PES capacity questionnaire survey report 2018 

•  Network  Board  contribution  to  the  Commission’s  consultation  on  the 

European Pillar of Social Rights

, December 2016 

16

Integration of 
refugees 

•  Thematic paper on ‘Integration of refugees into the labour market’ 
• 
Follow-up visit on integration of refugees into the labour market, 2017 
•  Key considerations on labour market integration of refugees, 2016 and 2019 

ESF 

EURES 

• 

In  some  countries  (e.g.  Bulgaria,  Croatia,  France,  Luxembourg,  Hungary, 
Lithuania and Romania) the PES actively cooperates with the ESF  managing 
authorities and receives funds for their operational programmes 

•  Cooperation  with  the  European  Network  of  Employment  Services  (EURES) 

re-established under Regulation (EU) 2016/58917 

•  Closer cooperation between the  PES Network and EURES at European level 

agreed at the December 2016 meeting of the PES Network Board 

•  A  member  of  the  PES  Network  acts  as  liaison  point  with  the  EURES 

Coordination Group 

Source: Study supporting the evaluation of Decision on enhanced cooperation between Public Employment 
Services, Ecorys for the European Commission, 2019. 

14  Official  Journal  of  the  European  Union,  Council  Recommendation  of  15 February  2016  on  the 

integration of the long-term unemployed into the labour market (2016/C 67/01), paragraph (9). 

15 PES Network (2017),  European Public  Employment Services  (PES)  Network  study  on assessment and 

early intervention to prevent long term unemployment. 

16 COM(2017) 287 final, p. 7. 
17 Regulation (EU) 2016/589 of the European Parliament and of the Council of 13 April 2016, OJ L 107, 

22.4.2016, p. 1-28. 

19 

 
 
 
                                                           
3.7.  Implementation of the European Social Fund 

PES are key actors in implementing the European Social Fund (ESF) in some Member 
States,  in  particular  for  measures  promoting  sustainable  and  quality  employment.  Total 
funding  in  this  area  amounts  to  €40.7  billion  (2014-2020,  EU  and  national  funding18). 
Out  of  this  amount,  the  ESF  provided  direct  support  of  €2.1  billion  for  reforms  to 
modernise PES and more than €25 billion to develop active labour market policies that 
are mainly implemented by the PES. 

One  of  the  ESF’s  priorities  is  the  funding  of  active  labour  market  policies  aimed  at 
disadvantaged  people  in  the  labour  market,  such  as  young  people  and  the  long-term 
unemployed.  ESF  funding  is  therefore  important  for  the  implementation  of  policy 
initiatives for many national PES. In some countries, Bulgaria and Romania for example, 
ESF funding has played an important role in reforming PES in order to ensure they have 
the institutional  capacity and their staff have the necessary skills  to  help  disadvantaged 
groups to find jobs. In Romania, the ESF project was implemented by assistance from the 
Network (see Box 3). 

Box 3. Mutual assistance — Romania 

Romania  approached  the  Network  with  a  request  for  mutual  assistance  after  receiving 
country-specific  recommendations  in  2016  and  2017  on  the  capacity  of  its  PES, 
specifically  related  to  improving  services  to  employers  and  jobseekers.  PES  colleagues 
from  Belgium  (VDAB),  Germany  and  Sweden  participated  in  the  project,  which  took 
place in 2017 and 2018. 

Understanding  labour  market  needs  and  close  collaboration  with  employers  are  crucial 
for  PES.  The  project  included  a  human  resource  component,  with  400  employees  of 
Romania’s national employment service receiving training aimed at improving the skills 
needed  for  an  effective  relationship  with  employers.  In  addition,  particular  action  was 
taken to: (i) develop a new tool to support jobseekers’ profiles and channel them towards 
appropriate  active  measures;  (ii)  develop  the  IT  system;  and  (iii)  develop  the  case 
management  approach  for  the  registered  unemployed.  Funding  from  the  ESF  had  an 
important  contribution  to  financing  the  project.  Work  is  still  in  progress  and  measures 
could lead to a positive impact if they are embedded in a comprehensive implementation 
strategy. 

Source: PES Secretariat/European Commission 

information  on 

Member  States  are  responsible  for  appointing  a  designated  managing  authority  that 
the  ESF  programme,  selects  projects  and  monitors 
provides 
implementation.  Out  of  the  31  PES  covered  by  the  2018  PES  capacity  report,  4  PES 
acted  as  managing  authority  for  the  ESF  and  5  PES  acted  as  managing  authority  in 
partnership  with  other  institutions1920.  Other  PES  are  involved  because  the  managing 
authority has delegated certain responsibilities to them. 

18 ESF synthesis report of annual implementation reports for 2017. 
19 https://ec.europa.eu/social/BlobServlet?docId=20575&langId=en 

20 

 
                                                           
3.8.  Annual work programme 

The Network adopts an  annual work programme, which is based on exchange of views 
and a learning needs assessment, agreed upon with all its members. The work programme 
is  designed  to  assist  the  PES  in:  (i)  delivering  the  Network’s  mission  to  promote 
modernisation; and (ii) helping individual PES to contribute more to the implementation 
of the EU2020 strategy. Benchlearning is central to the Network’s activities and is key to 
identifying  and  addressing  mutual  learning  activities.  In  recent  years,  the  work 
programme has been grouped under three headings: benchmarking, mutual learning and 
network governance. 

3.9.  Promote  and  share  best  practices  for  people  not 

in  education, 

employment or training (NEETs) 

Each  year,  a  number  of  practices  to  improve  the  situation  of  vulnerable  groups  in  the 
labour market  are published and shared on the  ‘PES  Practices’ page21 on  the European 
Commission  website.  Some  practices  have  a  broader  scope,  not  targeted  at  specific 
groups of vulnerable people. In addition, practices are presented which focus on NEETs 
as a group with particular needs, for example the ‘Toolkit on Sustainable Activation of 
NEETs’ and ‘PES Practices for the outreach and activation of NEETs’. 

3.10. Participation in learning events 

Since  2015,  the  range  of  learning  events  has  been  broadened  to  better  support  the 
learning  needs  of  individual  PES.  Specifically,  the  focus  has  changed  towards  smaller, 
more targeted learning formats, with events now organised with specific learning groups 
in mind. This methodology means that benchlearning site visit reports (conducted in the 
context  of  the  qualitative  assessments)  and  other  documents  are  used  to  identify  PES 
with specific learning needs in a particular topic area. 

PES  that  are  advanced  in  certain  areas  are  in  general  invited  to  host  or  chair  thematic 
review workshops and follow-up visits, so that they can share their practices with other 
countries, helping them  further develop  in  that same area.  Likewise,  advanced PES  are 
invited  to  participate  as  external  experts  in  mutual  assistance  projects.  Working  groups 
chaired by, and with participation from, Network members, have a key role in developing 
knowledge that is relevant to PES, and in preparing (for example) key considerations to 
be endorsed by the Network Board. The range of learning events has included webinars 
since  2017  and  learning  dialogues  since  2019.  Learning  dialogues  are  small  groups  of 
PES, where a PES that is advanced in a certain area shares experience with two to four 
PES  that  have  learning  needs  in  that  area.  The  coaching  approach  includes  follow-up 
support aimed at PES implementing what they have learnt for that specific area. 

As illustrated in Figure 3, the number of learning events more than doubled between the 
2011-2014  and  2015-2018  periods.  Similarly,  the  number  of  participants  more  than 
doubled, and the number of learning days increased by 75%. 

20 Norway and Iceland do not participate in the ESF. 
21 http://ec.europa.eu/social/main.jsp?catId=1206&langId=en 

21 

 
                                                                                                                                                                            
Figure  3:  PES  participation  across  all  mutual  learning  activities.  2011-2014  and  2015-2018

Source: PES Secretariat/European Commission 

4.  METHOD 

4.1.  Short description of methodology and sources 

Following the evaluation roadmap22 published in October 2017, the evaluation presented 
in  this  staff  working  document  was  carried  out  on  the  basis  of  information  and  data 
collected from different sources in accordance with the Better Regulation guidelines and 
Toolbox.  The  information  aims  to  cover  all  the  elements  in  the  intervention  logic 
presented in Figure 1, with a focus on results and impacts of the activities. 

This  evaluation  draws  to  a  large  extent  on  information  from  an  external  study 
commissioned from a contractor (the ‘external study’) and the 2017 interim review23. The 
2017 interim review is the Commission report published (in accordance with Article 10 
of the Decision) on 6 June 2017 on the Network’s activities up until that point. 

Furthermore, desk research was carried out to map changes resulting from the Decision 
and  the  Network’s  activities;  the  research  covered  a  range  of  written  sources  such  as 
annual reports, data collections and satisfaction surveys. The sources are complementary 
and mutually reinforcing. The main sources are summarised in Figure 4. 

22 https://ec.europa.eu/info/law/better-regulation/initiatives/ares-2017-5258712_en 
23 http://ec.europa.eu/social/BlobServlet?docId=17834&langId=en 

22 

 
 
 
                                                           
Figure 4: Summary of the evaluation process 

Staff working 
document 

External study 

Desk research 

Targeted 
consultations 

Case studies 

Public 
consultation 

Annual reports/surveys 
Benchlearning results 
Annual data collections 
Satisfaction surveys 

Source: PES Secretariat/European Commission 

The  external  study  was  conducted  between  June  2018  and  March  2019.  Its 
methodological  approach  combined  quantitative  and  qualitative  data,  which  have  been 
systematically  cross-checked  to  answer  the  evaluation  questions.  The  study  includes  a 
mapping of policy changes implemented by PES following the creation of the Network 
in June 2014. 

A  range  of  actors  were  consulted  in  the  external  study  (see  details  in  Annex  2). 
Interviews were conducted with:  

  members of each of the  32 PES24 involved in  the  Network  (when possible both 

with the Network Board member and the AFEPA);  

  selected  representatives  of  relevant  EU-level  organisations  and  bodies  (e.g. 
EMCO, EU-level private employment services and temporary work agencies, the 
European Lifelong Guidance Policy Network);  

  selected representatives of relevant international organisations (for example ILO, 
OECD,  World  Bank,  World  Association  of  Public  Employment  Services 
(WAPES));  

  other stakeholders closely involved in  the running of the Network (e.g. the PES 

Secretariat, contractors);  

  a  sample  of  individuals  who  were  former  members  of  the  Network  or  involved 

with PES cooperation pre-2014.  

In  addition,  a  workshop  was  conducted  with  AFEPAs,  as  well  as  case  studies  in  five 
countries (Estonia, France, the Netherlands, Spain and Romania) to further elaborate the 

24 27 Member States PES + 3 Belgian regional PES + Iceland and Norway. 

23 

 
 
                                                           
 
 
 
evaluation  questions.  These  countries  were  selected  on  the  basis  of  several  criteria, 
namely geographic location, population size, economic situation (measured by real GDP 
growth,  employment  and  unemployment  rates),  and  on  the  basis  of  PES  performance 
indicators (maturity levels, increase/decline in PES expenditure, and increase/decline in 
new  vacancies  reported).  ESF  managing  authorities  and  labour  and  social  affairs 
ministries  received  a  targeted  questionnaire,  which  16  labour  ministries  and  7  ESF 
managing authorities filled out. 

An open internet-based public consultation ran from 17 September 2018 to 13 December 
2018. An online questionnaire was available in English, French and German, and replies 
could  be  made  in  any  of  the  24  official  EU  languages.  In  particular,  the  consultation 
sought  to  gather  views  from:  (i)  individual  citizens  in  their  personal  or  professional 
capacity;  (ii)  researchers  and  academia;  (iii)  civil  society  organisations  working  with 
jobseekers  or  on  the  issue  of  unemployment/employment  services;  (iv)  social  partners; 
(v)  employment  and  social  services  providers;  and  (vi)  public  authorities  at  national, 
regional and local levels. 126 replies from 18 Member States and Pan-EU organisations 
were received. 

The  evaluation  also  draws  on  various  other  secondary  sources,  cited  throughout  in  the 
text.  Benchlearning  results,  the  evaluation  of  mutual  learning  events  and  the  annual 
reports are among the main sources, a list of which is set out below: 

•  Annual reports of the Network, which were sent to the European Parliament and 

to the Council and published online25. 

• 

• 

The  outputs  produced  by  the  Network  as  part  of  its  annual  work  programme, 
published on the PES Knowledge Centre and PES Practices site26. 

Satisfaction  surveys  and  other  monitoring  activities  with  regard  to  the 
Network’s  activities,  carried  out  by  the  contractors  who  supported  the 
implementation  of  the  Network’s  work  programme.  In  particular,  in  2018,  the 
contractor  in  charge  of  mutual  learning  activities  carried  out  an  enhanced 
evaluation to understand the potential impact of mutual learning activities27. The 
enhanced  evaluation  included:  (i)  third-stage  qualitative  interviews  with  PES 
who attended a thematic review  workshop plus  a follow-up visit in  2015 (thus 
building  a  continuum  with  interviews  undertaken  with  2015  cohorts  in  2017); 
(ii) an online mutual learning participant survey, implemented between May and 
June  2018;  and  (iii)  follow-up  interviews  with  the  respondents  to  the  mutual 
learning participant survey who agreed to be interviewed in summer 2018. 

•  Yearly  survey  and  report  on  PES  capacity  available  for  2014-2018,  published 
online. In addition, in 2014-2017 an annual report on PES implementation of the 

25 https://ec.europa.eu/social/BlobServlet?docId=20613&langId=en 
26 https://ec.europa.eu/social/main.jsp?catId=1206&langId=en 
27 See Annex 3 to the 2019 annual report, https://ec.europa.eu/social/BlobServlet?docId=20613&langId=en 

24 

 
                                                           
Youth  Guarantee  was  published,  followed  in  2018  by  a  report  on  PES 
implementation of the Recommendation on the long-term unemployed28. 

• 

The  annual  quantitative  data  collection  and  analysis  gathered  via 
the 
benchmarking exercise is available for the years 2015-2018. Biennial qualitative 
assessment  site  visits  to  the  PES,  which  involve  self-assessment  reports,  and 
overall assessment reports are available for the years 2015/2016 and 2017/2018 
for the whole Network29. For the first cycle, change reports from all 28 Member 
States, Norway and Iceland had been produced one year after the site visit. 

• 

The EMCO thematic reviews of PES/ALMPs and the European Semester. 

4.2.  Limitations and robustness of findings 

As per the Decision, the Network should contribute to  the improved functioning of the 
PES themselves, as well as to the policy objectives described in Article 3. Evidence from 
this  evaluation  shows  that  for  various  reasons  many  of  the  envisaged  reforms  and 
changes take time to be properly implemented. It is important to point to the contextual 
factors, such as political decisions, allocation of budgetary resources and changing labour 
market  contexts  due  to  economic  and  social  developments.  The  period  covered  by  this 
evaluation may therefore be too short to achieve major changes and not least to measure 
the impacts, and thus it is not possible at this time to make a full assessment of Network 
efficiency.  This  may  also  mean  that  there  are  unrealised  gains  from  plans  or  initiatives 
that will only appear in the years to come. 

Secondly, it is sometimes difficult to assess whether an initiative was implemented only 
because  of  the  Decision.  This  in  turn  makes  it  hard  to  make  a  full  assessment  of  the 
added value. Estimating the impact of the Network’s activities and of PES initiatives as a 
result  of  the  former  is  in  some  cases  difficult  and  may  hide  the  real  impact.  This  is 
because of data deficiencies due to different definitions between countries or the different 
design  of  measures  or  activities.  Differences  in  PES  responsibility  from  country  to 
country is  furthermore a reason why data comparability  could  be limited. For example, 
the supply of active labour market measures is in some countries a responsibility for the 
PES,  while  in  other  countries  it  is  a  responsibility  of  the  municipalities.  Furthermore, 
some PES produce most of their services in-house, while other PES have extensive use 
of contractors. 

An  important  information  source  is  the  qualitative  assessments  made  by  PES 
representatives. However, since it could be argued that people who are deeply involved 
in a task could be biased in their assessments, this evaluation draws on a broad range of 
sources.  When possible,  reliability is  checked by comparing information  from  different 
sources  such  as  similar  information  from  the  external  study  and  the  final  evaluation 
report for the mutual learning event, or by comparing assessments by PES representatives 

28 https://ec.europa.eu/social/BlobServlet?docId=20186&langId=en 
29 The second cycle does not include the qualitative assessment from the UK. 

25 

 
                                                           
with  similar  assessments  by  other  stakeholders  as  the  labour  ministries.  In  this  way, 
triangulation or cross-checking of sources mitigates the risk of biased conclusions. 

5.  ANALYSIS AND ANSWERS TO THE EVALUATION QUESTIONS 

5.1.  Relevance 

5.1.1.  The objectives have been relevant for the Network 

As described in Section 3, the objectives set in the Decision have been a major guideline 
for  Network  initiatives  implemented through  the  annual  work programmes  (see Chapter 
2.1 for overview). There was strong consensus among stakeholders interviewed that  the 
eight objectives from 2014-2018 had a high overall degree of relevance. The relevance of 
the  Decision’s  content  was  also  strongly  confirmed  by  the  findings  from  the  targeted 
consultations with labour ministries and ESF managing authorities. Finally, the relevance 
of the Decision was confirmed by the public consultation. 

One of the main missions for PES is to help disadvantaged groups such as NEETs or the 
long-term unemployed to find jobs. This has been and still is a major challenge for PES 
due to high unemployment in the wake of the 2008 economic crisis and the high number 
of  refugees  in  many  countries.  The  economic  recovery  has  facilitated  labour  market 
access for disadvantaged groups, but the long-term unemployed, young people and older 
jobseekers are still overrepresented among those seeking assistance from PES. Inclusion 
of  inactive  people,  who  are  often  further  away  from  the  labour  market,  is  becoming  a 
priority task for some PES. 

The Decision’s objectives encompass key areas of responsibility for PES in  the field  of 
employment.  By  providing  a  broad  framework  for  Network  activities,  the  objectives 
enable a flexible approach to addressing priorities for action depending on labour market 
developments.  The  PES  Network’s  2020  Strategy  2020  and  Beyond30  is  updated  when 
necessary to be as relevant as possible to the goals of the Europe 2020 strategy. The range 
of  areas  and  themes  is  very  broad.  At  the  same  time,  the  external  study  highlights  that 
there is some variation in the degree of relevance for each of the objectives. 

5.1.2.  The relevance of each objective 

Section  2.1  of  the  external  study  gives  an  overview  of  assessments  of  the  relevance  of 
each  of  the  objectives  set  in  the  Decision.  The  ‘inclusion  of  the  most  vulnerable  social 
groups’ (objective  a) is  regarded  by  consultees in  the  external  study  as  one  of  the  most 
relevant Decision objectives as it is a long-standing issue in different European countries, 
particularly  focusing  on  the  inclusion  of  the  long-term  unemployed  and  NEETs.  The 
consultations  and  the  2017  interim  review  of  the  Decision  reveal  that  objective  a)  is 
closely  related  to  objective  g)  on  the  ‘integration  of  people  excluded  from  the  labour 
market as part of the fight against social exclusion’. This demonstrates that some of the 
target  groups  for  PES  meet  multiple  and  even  interlinked  barriers  to  integration  in  the 
labour  market.  Social  exclusion  in  itself,  for  example  because  of  poverty,  can  reduce  a 

30 https://ec.europa.eu/social/BlobServlet?docId=20662&langId=en 

26 

 
                                                           
person’s  chances  in  the  labour  market.  Achievements  on  objective  g)  can  therefore 
contribute to progress on objective a) and vice versa. 

‘Supporting  decent  and  sustainable  work’  (objective  b)  is  a  relevant  objective.  Non-
standard  work  (i.e.  employment  contracts  that  are  neither  permanent  nor  full-time) 
represent nearly 20% of part-time employment and over 10% of temporary employment 
in  the  EU  in  2017  (see  The  Future  of  Work:  Implications  and  Responses  by  the  PES 
Network, Section 3.1.3)31. These types of work can provide opportunities for people who 
want to work less than full working hours or want to have flexibility. At the same time, 
this work often comes with a greater unpredictability of working conditions and income 
flows,  especially  for  workers  in  low-skilled  occupations.  Due  to  the  changing  labour 
market  characterised  by  new  forms  of  work,  work  relationships  and  forms  of  work 
organisation, this objective may become increasingly relevant in the future. 

A ‘better functioning of the labour markets in the EU’ (objective c) is a prerequisite for 
optimal use of the labour force and is at the heart of the mission of European PES. The 
relevance of this objective is confirmed in the external study. A well-functioning labour 
market depends on a number of factors such as economic growth, the general regulatory 
framework, access to a skilled labour force and professional PES services. The Network 
contributes to professionalisation of PES by facilitating cooperation and helping them to 
learn  from  each  other,  as  well  as  cooperation  with  for  example  EURES and  the  private 
employment services. 

Facilitating  the  matching  of  jobseekers  with  jobs,  facilitating  labour  market  integration 
and effective active labour market policies are all key to a well-functioning labour market. 
Objective c) is therefore closely linked to objectives d), e), f) and h). 

Objective d), ‘identification of skills shortages and the provision of information on their 
extent and location and the better matching of the skills of jobseekers with the needs of 
employers’, is assessed as relevant by experts and other stakeholders, for example to fill 
bottleneck vacancies. ‘Better integration of the EU labour markets’ (objective e) also has 
close  links  to  objective  d),  and  is  clearly  a  relevant  objective  for  the  Network,  since  it 
directly concerns cooperation at EU level or bilateral cooperation. Objective f), ‘increased 
voluntary and geographical and occupation mobility on a fair basis to meet specific labour 
market needs’ is also considered as relevant. There is no duplication and, according to the 
external  study,  there  is  a  good  level  of  coordination  between the  Network  and  EURES. 
Finally,  objective  h),  ‘evaluation  and  the  assessment  of  active  labour  market  initiatives 
and their effective and efficient implementation’, has been and remains relevant. 

The  relevance  of the objectives  could  depend  on  the labour market  situation  at  national 
level or on a where an economy is in its business cycle. At country level, objective d) is 
seen as most relevant in countries facing skills shortages. Some consultees in the external 
study raised the question whether objective f) is as relevant for the Network as some of 
the other objectives. In Greece, Italy and Spain, unemployment was still above 10% at the 

31 https://ec.europa.eu/social/BlobServlet?docId=20520&langId=en 

27 

 
                                                           
end  of 2018.  In  other  countries, unemployment had  dropped  to  below 4%,  and tackling 
skill shortages is becoming a more prominent task for PES. This illustrates how important 
it is that the objectives are designed to be relevant for PES facing different labour market 
conditions. The feedback from PES in the external study confirms that the objectives as 
they stand fulfil this purpose. 

To  sum  up,  there  was  a  strong  consensus  among  stakeholders  interviewed  that  the 
Decision’s  objectives  are  relevant  to  the  current  challenges  of  different  national  labour 
markets.  Together,  they  provide  a  common  strategic  framework  for  PES  cooperation  to 
tackle  the  increasing  common  global  challenges  that  PES  are  facing,  challenges  that 
require concerted action. 

5.1.3.  The objectives have been resilient to the changing labour market and 

policy developments 

The external study, Section 2.1, has shown that overall the objectives have been relevant 
to the challenges of the European labour market and to the social situation. The flexibility 
in  their  wording  has  made  it  possible  to  address  new  and  unexpected  challenges  (e.g. 
adaption  of  the  Network  strategy  and  work  programme  to  the  201532  migrant  crisis), 
thereby showing their continuous relevance.  

Stakeholders’ assessment of the relevance of the objectives matches well with statistical 
information. The 2018 assessment  report  on PES capacity33 highlights that PES are still 
dominated by difficult-to-place groups of unemployed people such as young people, the 
long-term unemployed and older workers. These three groups account for a fairly constant 
share of the registered jobseekers (see Figure 5). 

Figure 5: Breakdown of registered jobseekers 

32 https://ec.europa.eu/social/BlobServlet?docId=20662&langId=en 
33 https://ec.europa.eu/social/BlobServlet?docId=20575&langId=en  

28 

 
                                                           
Source: PES Secretariat/European Commission 

5.1.4.  The initiatives of the PES Network have been relevant 

Article 4 of the PES Decision sets out seven initiatives (labelled a-g) which the Network 
should  carry  out  (see  Chapter  2.1  for  overview).  These  initiatives  have  been  the  major 
guideline for the activities of the annual work programmes. 

Overall, the initiatives set out in Article 4 of the Decision are relevant to the needs of PES 
and are consistent with the Decision’s objectives. This is confirmed by the vast majority 
of  consultees  in  the  external  study,  Section  2.2, both  from  PES  and  from  other  EU  and 
international institutions. The initiatives provide relevant tools to: (i) enhance cooperation 
between  national  PES  and  a  well-constructed  mix  of  approaches  to  suit  the  needs  of 
different  PES;  and  (ii)  ensure  synergies  and  complementarities  between  the  different 
initiatives.  Consultees  all  assessed  ‘benchlearning’  (initiative  a),  ‘mutual  assistance’ 
(initiative  b),  ‘reports’  (initiative  d),  ‘implementation  of  relevant  policy  initiatives’ 
(initiative e) and ‘promoting and sharing’ (initiative g) as very relevant. 

Consultees  regarded  initiative  c),  ‘contribute  to  modernising  and  strengthening  PES’  as 
more  overarching  although  still  highly  relevant,  and  implemented  by  a  range  of  other 
initiatives  such  as  benchlearning  and  mutual  assistance.  In  addition,  while  initiative  f), 
‘adopt and implement its annual work programme’ was seen as clearly relevant, it could 
be considered as a requirement of the Network’s function rather than an initiative with the 
same status as benchlearning and mutual assistance. 

As evidenced in the Network’s annual work programmes and confirmed by the external 
study,  Section  2.2,  the  initiatives  set  in  Article 4  of  the  Decision  are  well  linked  to  the 
objectives  set  in  Article 3.  The  Network  has  demonstrated  flexibility  by  focusing  on 
activities related to new upcoming topics such as the integration of migrants and refugees 
and the prevention of unemployment.  

29 

 
5.1.5.  Potential for improving relevance 

The  remaining  difficulties  that  these  groups  (young  people,  long-term  unemployed  and 
older workers) face when trying to (re-)enter the labour market indicate that the objective 
of supporting vulnerable group with high unemployment rates remains relevant. 

How PES can contribute to better functioning of the labour market in the changing world 
of work has been addressed more prominently in the Network’s work programme over the 
past few years. The challenges posed by the changing world of work are also having an 
impact on how the objective to ensure decent and sustainable work can be achieved. New 
working conditions are resulting in employment opportunities in new technology sectors, 
change  or  loss  of  jobs  due  to  the  large-scale  digitalisation  of  many  professions,  more 
short-term  or  flexible  working  and  a  growing  trend  towards  entrepreneurialism  and 
platform jobs. 

To achieve the objectives of the Decision, jobseekers need the necessary skills, knowledge 
and  competences.  PES  can:  (i)  support  jobseekers  and  employers  by  identifying  future 
skills needs; (ii) deliver suitable training, career guidance and support to enable jobseekers 
to manage their own careers; and (iii) forge partnerships to improve their offer in this area. 
A working group established by the Network in 2018 concluded that the Network needs to 
evolve as a learning organisation, while continuing with benchlearning and promoting the 
principles of knowledge-based development34. A continuous learning environment is key 
to PES transforming their services, and to focusing on empowerment of jobseekers. 

5.2.  Effectiveness 

5.2.1. 

Quantitative effects of the enhanced cooperation between PES 

The  Network  is  considered  generally  very  effective  in  enhancing  cooperation  and 
improving  the  functioning  of  PES,  in  particular  through  the  benchlearning  and  mutual 
assistance  initiatives.  The  vast  majority  of  the  stakeholders  consulted  emphasised 
benchlearning  and  mutual  assistance  as  the  most  effective  initiatives  because  of  their 
concrete results. Stakeholders interviewed in the external study believe that the success of 
the two benchlearning cycles derives not so much from their value as a tool for ranking 
performance, but rather as a tool for PES to help them learn and improve. 

The  quantitative  performance  indicators  set  in  the  Annex  to  the  Decision  are  grouped 
under  four  headings:  ‘labour  market  flows’,  ‘relations  with  employers’,  ‘customer 
satisfaction’ and ‘long-term unemployment’. The indicators show that almost all PES can 
be found at least once among the over-performers, thanks to their achieving better results 
than  what  statistically  could  be  expected  based  on  their  context,  e.g.  given  the 
demographics, education level, macroeconomic context, etc. in the country or region that 
they serve. At the same time: there is still room for improvement. 

PES  access  the  quantitative  benchlearning  indicators  via  the  PES  Dashboard.  The 
dashboard  is  the  major  tool  used  to  promote  transparency  between  all  members  of  the 

34 https://ec.europa.eu/social/BlobServlet?docId=20520&langId=en  

30 

 
                                                           
Network.  The  dashboard  does  not  appear  to  be  consistently  used  by  the  Network 
members  (see  Figure  6  below).  Some  of  the  reasons  mentioned  by  PES  stakeholders 
during the external consultation concerned technical aspects, mentioning that IT system 
security  problems  limited  access  to  the  dashboard,  while  others  referred  to  the 
comparability and presentation of the indicators. 

The fact that so many different PES have agreed on a set of quantitative and qualitative 
common  indicators  for  the  benchlearning  process  is  in  itself  an  important  achievement. 
Since quantitative information can play a role in the planning of initiatives and reforms, 
one aim should be to achieve more consistent use of the dashboard among PES. 

31 

 
 
Figure 6: Number of PES Dashboard sessions per country. March 2018 to February 2019 

Source: PES Secretariat/European Commission 

5.2.2.  Modernisation promoted by benchlearning 

The  quantitative  and  qualitative  benchlearning  results  indicate  that  PES  performance  is 
related to how PES are organised and how services are offered to customers35. Evidence 
on this relationship has improved the foundation for knowledge-based development, both 
in national PES and across countries in the Network. 

Comparing results from the first to the second cycle of benchlearning demonstrates 
that  the  exercise  has  given  guidance  to  PES  on  their  path  towards  modernisation 
and change36. Modernisation is being pushed further by the strong focus of the second 
cycle  on  change  processes  and  management  of  these  changes.  Decentralisation  and 
involving  PES,  jobseekers  and  employers  and  other  stakeholders  in  design  and 
implementation of services are now playing a more important role in setting up services 
that are suited to the needs of jobseekers and employers. Staff commitment to delivering 
quality  customer  services  in  all  PES  continued  to  be  high,  although  unfavourable 
caseloads  persist  in  many  PES,  and  many  PES  continue  to  work  under  difficult 
conditions. 

Based on the finding, some trends can be summarised: 

• 

Strategic  performance  management:  On  the  basis  of  the  recommendations 
made  during  the  first  cycle,  many  PES  have  critically  reviewed  their 
performance  management  system  and  especially  the  structure  of  their  key 
performance  indicators,  with  the  aim  of  achieving  outcome-oriented  business 
development. 

•  Design of operational processes: the second cycle gave evidence that customer 
needs and expectations are at the heart of effective modernisation of processes. 

35 https://ec.europa.eu/social/BlobServlet?docId=20613&langId=en 
36 https://ec.europa.eu/social/BlobServlet?docId=20613&langId=en 

32 

 
 
 
                                                           
• 

Efficient  customer  segmentation  makes  it  possible  to  shift  resources  to  those 
customers who need in-depth support. 
Sustainable activation and management of transitions: Nearly all PES have 
introduced  different  channels  for  communication  with  customers,  and  digital 
tools  especially  have  helped  to  reduce  administrative  burden  in  some  PES, 
freeing  up  resources  for  in-depth  counselling  of  jobseekers  with  special  needs 
(long term unemployed, NEETS, etc). 

• 

•  Relations with employers: The PES are currently developing different concepts 
on  how  to  satisfy  employers’  needs  given  an  increasing  share  of  difficult-to-
place  customers.  The  Network  European  Employers’  Day  supports  PES 
visibility in participating countries. 
Evidence-based design and implementation of PES services: The use of pilot 
projects has become more prominent since the first cycle, especially for testing 
new  organisational  set-ups,  service  models  and  ALMPs  for  specific  groups  A 
systematic  evaluation  of  practices  and  communication  of  results  has,  however, 
only been implemented by a few PES. 

•  Management  of  partnerships  and  stakeholders:  The  discussion  has 
intensified as to which PES services should be outsourced and which should be 
kept  in-house.  One-stop  shop  solutions  providing  combined  social  and  labour 
market services have been introduced in some PES, and others will follow. 

Even if it is not possible to reach conclusions on the direct impacts of the various actions, 
PES maturity has increased overall from the first to the second cycle. The observed 
trends in findings demonstrate that the systematic benchlearning approach, including the 
follow-up reports, has resulted in more concerted actions across PES. A clearly defined 
process  starting  with  the  first  cycle  assessments  and  recommendations,  followed  by  a 
change plan, follow-up reports and the renewed assessments in the second cycle, seems 
to have facilitated commitment to plans and implementation of them. 

5.2.3.  Mutual learning between PES has improved 

As an inherent part of benchlearning, mutual learning has been a key pillar of Network 
activity,  together  with  benchmarking  activities.  Mutual  learning  involves  the  use  of 
evidence to identify and address PES learning needs. The aim of the Network’s activities 
in  this  area  has  been  to  help  PES  learn  from  employment  services  with  particular 
expertise  or  insights  in  a  specific  area.  Mutual  learning  offers  a  key  mechanism  to 
support the other broader initiatives, activities and objectives specified in the Decision. 
For  instance,  it  includes  the  types  of  enhanced  activity  intended  to  modernise  and 
strengthen PES, and supports the implementation of relevant policy initiatives. 

Quantitative and qualitative benchmarking have led to  the creation of an evidence base 
with information on learning needs as well as expertise and good practices to share. This 
feeds  into  the  development  and  implementation  of  the  Network’s  mutual  learning 
programme.  Mutual  learning  activities  focused  on  several  priority  areas  in  2014-2018, 
addressing various Decision objectives. In 2015-2016, the focus was on thematic clusters 
such  as  young  people,  performance  management  and  measurement  of  customer 

33 

 
satisfaction, while in 2017-2018 clusters such as digitalisation and services to employers 
attracted  more  attention.  Activities  related  to  sustainable  integration  were  relatively 
important throughout the whole period. 

2018  saw  the  introduction  of  an  enhanced  approach  to  monitoring  and  evaluation  of 
mutual  learning  activities.  The  findings,  reported  in  the  2018  annual  report37, 
demonstrate that participants are generally very satisfied and that the available resources 
are  useful  in  the  development  of  changes,  initiatives  and  reforms  in  PES.  Overall, 
participants gave higher scores to face-to-face learning events than to reports and papers. 
This  can  be  a  reason  to  explore  new  channels  for  dissemination,  for  example  whether 
videoconferencing  can  be  a  useful  tool.  However,  the  higher  scoring  of  face-to-face 
meetings  also  highlights  the  importance  of  networking  and  discussions  with  peers  and 
experts in the specific field. 

5.2.4. 

Effectiveness of mutual assistance 

Benchlearning  is  often  accompanied  by  direct  mutual  assistance  (technical  assistance 
through  peer  PES  support  to  selected  PES)  offered  through  the  Network.  Mutual 
assistance offers a needs-based learning format responding to individual PES needs. For 
example,  following  previous  activity  linked  to  the  benchlearning  exercises,  throughout 
2015-2018  the  Network  and  its  members  provided  mutual  assistance  for  PES 
modernisation  in  Spain,  Bulgaria,  Romania  (Box  3)  and  in  Italy  and  Cyprus  (Box  2), 
again  highlighting  the  interrelated  nature  of  PES  activities  supporting  the  Decision 
objectives. 

Mutual  assistance  is  seen  as  a  highly  effective  initiative  of  the  Network.  At  the 
request of individual PES, two to three mutual assistance projects take place each year. In 
the  external  study,  representatives  both  of  PES  receiving  assistance  and  of  those 
providing  assistance  emphasised  the  role  this  activity  plays  in  raising  awareness  about 
PES  strengths  and  weaknesses.  The  respondents  cited  the  importance  of  comparing 
systems  and  of  providing  concrete  opportunities  to  learn  and  design  change  strategies 
(see Boxes 2 and 3 for descriptions of outcomes). 

Stakeholders  consulted  for  the  French  case  study  (the  external  study  Section  3.1) 
highlighted  that  mutual  assistance  activities,  as  judged  by  Pôle  emploi  (France’s  PES), 
are  very  interesting  not  only  for  the  PES  receiving  the  assistance  but  also  for  the  PES 
providing it. Similarly, the Estonian case study highlighted the exchange of best practice 
and mutual assistance opportunities as particularly valuable. This  illustrates that mature 
PES (see Annex 1 in the 2018 Annual Report38) can also get inspiration for new reforms 
by  acting  as  mentors  and  advisers.  The  mutual  assistance  projects  should  therefore  not 
only be considered as assistance, but also as a learning arena. 

37 https://ec.europa.eu/social/BlobServlet?docId=20613&langId=en 
38 https://ec.europa.eu/social/BlobServlet?docId=20613&langId=en 

34 

 
                                                           
5.2.5. 

PES have taken up the results of Network cooperation 

Assessments from peer experts during site visits and their recommendations to PES are 
an important input and provide inspiration for change in PES. This is borne out by 
the change reports from the PES one year after the first site visit and by the reports on the 
follow-up of benchlearning recommendations made in response to the annual reports. The 
assessors  can  give  advice  about  other  PES  that  could  provide  assistance,  and  some 
countries  have  approached  peer  PES  for  learning  and  exchange  (see  the  example  of 
Austria in Box 4 below), demonstrating the effectiveness of the benchlearning initiative. 

Box 4. Examples of reform initiatives resulting from the benchlearning initiative39 

The  Austrian  PES  is  setting  up  a  system  of  causal  impact  evaluations  of  ALMP 
measures  in  order  to  identify  the  efficiency  of  all  measures  on  a  regular  basis.  The 
Austrian  PES  organised  an  exchange  at  expert  level  with  the  German  PES  to  get 
inspiration from the German ‘Treatment Effect and Prediction’ system. 

The Slovenian PES has introduced a ‘Randomized Control Trials’ approach (RCTs) to 
piloting and evaluation of new services and products. This approach will reduce the need 
for  more  extensive  empirical  methods  when  evaluating  interventions.  Through  the 
benchlearning  initiative,  the  Slovenian  PES  received  information  about  other  PES  with 
experience in implementing RCTs. 

As  a  result  of  activities  from  the  second  benchlearning  cycle,  the  Spanish  PES  is 
identifying  and  disseminating  best  practices.  An  internal  benchlearning  and  mutual 
learning project for the regional PES in Spain is the main framework for this sharing of 
practices.  

Source: PES Network Annual Report 2018 

In  addition  to  benchlearning,  mutual  learning  activities  are  part  of  the  Network 
cooperation.  Evidence  from  internal  evaluations  of  the  impact  of  mutual  learning 
indicates  that  70%  of  respondents  used  the  knowledge  they  accumulated  during  the 
activities  and  one  third  declared  that  the  activities  have  contributed  to  changes  in  their 
PES40. 

The types of change that have occurred often relate to concrete and specific operational 
changes. These include: 

• 
• 
• 
• 

development of new measures for young people and the long-term unemployed; 
development and launch of evaluations of ALMPs; 
redesign of approaches to measuring customer satisfaction; and 
redevelopment of web-based resources. 

39 Further examples can be found in the European Network of Public Employment Services, Annual Report 

2018: https://ec.europa.eu/social/BlobServlet?docId=20613&langId=en 

40 ICF — enhanced approach to the monitoring and evaluation of mutual learning activities (see Annex 3 of 

Annual Report 2018). 

35 

 
 
                                                           
Other types of change include: 

• 
• 
• 
• 

an increased knowledge base; 
changes to a strategic approach; 
changes in priorities among senior or middle management; and 
raised awareness of a specific topic or approach. 

This overview demonstrates that changes are often directly targeted at services for young 
people and the long-term unemployed. Other changes could have more indirect impacts 
on  services  by  increasing  the  capacity  and  professionalism  of  the  PES.  Change  can  be 
initiated  not  only  by  mutual  learning,  but  also  through  other  channels  such  as  political 
decisions at the national level, for example introducing new measures targeted at young 
people  or  the  long-term  unemployed.  To  implement  such  changes,  learning  from  other 
PES can still be important.  Even if there are differences, the main conclusion is that PES 
use mutual learning results extensively. 

5.2.6. 

Network contributions to effective policy implementation 

The  external  study,  Section  3.2,  demonstrates  that  Network  activities  have  been  most 
useful in supporting the most vulnerable groups with high unemployment rates and, to a 
certain  extent,  the  integration  of  people  excluded  from  the  labour  market.  A  large 
majority  of  national  labour  ministries  indicated  that  the  Network  has  been  very 
successful or successful in supporting the most vulnerable groups, while only about one 
third  found  that  the  Network  had  been  very  successful  or  successful  in  supporting  the 
integration of people excluded from the labour market. 

The Network prepares reports at the request of the European Parliament, the Council and 
the Commission. As stated in the annual reports from the Network, a number of reports 
and  analytical  papers  are  published,  for  example  on  themes  related  to  the  long-term 
unemployed,  young  people  and  support  to  vulnerable  groups.  Labour  ministries  were 
asked  by  the  external  contractor  (see  Section  3.1  in  the  external  study)  how  they 
evaluated  the  Network’s  reports.  A  total  of  5  ministries  found  this  activity  very 
successful, 7 found it successful, and 2 somewhat successful, while 2 ministries did not 
offer  any  assessment.  Overall,  the  responses  from  the  labour  ministries  confirm  that 
reports prepared by the Network have proved useful. 

The evidence gathered that the Network is providing evidence-based information to the 
European  Semester  process,  indicating  PES’  progress  and  fostering  well-informed 
dialogue with the Employment Committee. This dialogue is important in developing EU 
employment policy which is applicable to Member States’ needs, for example to address 
the increased focus on policies to integrate people excluded from the labour market. 

5.2.7. 

Potential for improving effectiveness 

Limited  internal  PES  capacity,  high  staff  workloads  and  infrastructure  problems  were 
seen as problematic by some PES consulted in the external study (Section 3.1). Despite 
technical  and  financial  support  at  EU  level,  some  PES  have  limited  financial  and 
human  capacity  to  balance  national  and  international  commitments,  particularly 
those  PES  with  less  resources  from  national  governments.  Moreover,  many  consultees 

36 

 
stressed the influence of political  decision-making  as  a potential barrier in cases  where 
the  political  will  is  not  present  to  encourage  sustainable  change.  Some  PES  have 
experienced  difficulties  in  demonstrating  the  benefits  of  the  Network  activities  to  their 
national governments in quantifiable terms. 

Changing  national  priorities  influence  whether  change  can  take  place  at  the  national 
level. This is demonstrated by a combination of findings from the external consultation 
and the reports from PES on follow-up of benchlearning recommendations. Thus, for a 
mutual  learning  activity  to  be  effective  (meaning  that  change  is  achieved)  it  is 
important that it comes at the right time. Effectiveness also depends on the ability to 
find  a  balance  between  launching  promising  proposals  and  keeping  new  ideas  within 
what could realistically be realised. This balance differs across PES, and can change over 
time. The benchlearning exercise has been important to improve evidence for designing 
better measures and recommendations at EU level that are also targeted at each PES. The 
2017  interim  review  of  the  Decision  highlights  the  benchlearning  project  as  a  positive 
innovation in that national institutions can have external references, receive constructive 
feedback from expert practitioners and engage in tailor-made learning with peers. 

The  evidence  gathered  by  the  various  consultation  activities  for  this  evaluation  and 
reports  from  PES41  demonstrate  that  benchlearning  and  mutual  learning  activities  have 
contributed  to  concrete  changes  in  PES  organisational  processes  and  service  delivery, 
thus  rendering  them  more  effective.  These  changes  have  the  potential  of  contributing 
further 
implementing 
improving  PES  performance;  and  (b)  effectively 
employment policies at the national and local levels. 

to:  (a) 

5.3.  Efficiency 

5.3.1.  The Network has been efficient in delivering initiatives and reaching 

objectives 

The  external  study,  Section  4.1,  confirmed  that  the  Network  is  considered  both  to  be 
efficient and a good use of national (or regional) and European public resources by the 
vast majority of actors consulted both at EU level and in national PES (almost 100% of 
those  who  expressed  an  opinion).  This  view  was  also  held  by  representatives  of 
international  organisations  consulted,  who  felt  that  the  Network  provided  a  key  forum 
where  practical  issues  for  PES  performance  and  service  delivery  can  be  discussed  and 
ensure  efficiency  gains  by  learning  from  others.  However,  the  highest  proportion  of 
respondents in the public consultation was unable to determine the cost-effectiveness. 

The PES Secretariat has proven to be an important link between the Commission and the 
Network.  The  PES  Secretariat  consists  of  Commission  staff  and  Seconded  National 
Experts  from  European  PES.  This  legitimately  supports  the  Commission  by  being  a 
source  of  information  from  and  contacts  with  the  Network.  This  close  link  is  also 
contributing to the efficiency of the Network. 

41 https://ec.europa.eu/social/BlobServlet?docId=20613&langId=en 

37 

 
                                                           
5.3.2.  Costs 

Examining  efficiency  requires  an  assessment  of  the  degree  to  which  the  objectives  and 
activities of the Network have been delivered at an appropriate cost. This implies a need 
to  assess  the  costs  and  benefits  associated  with  the  Network.  However,  many  of  the 
benefits in particular do not lend themselves to monetary valuation. Moreover, a four-year 
period  is  in  some  cases  too  short  to  realise  the  full  gains  of  organisational  or  cultural 
changes  that  have  been  set  in  motion.  A  full  cost-benefit  analysis  can  therefore  not  be 
carried out. 

The external contractor asked the national members of the PES Network to indicate time 
and  other  costs  used  in  participating  in  Network  activities  (external  study  Section  4.2). 
Estimates were often approximate and some consultees did not want to share them or were 
not able to obtain information on costs. Furthermore, some respondents found it difficult 
or impossible to separate time used in Network activities from other activities, particularly 
when  related  to  European  or  international  tasks.  The  estimates  can  therefore  only  give 
indications of the use of resources. 

Moreover, estimations of time investments by PES differed quite widely  by PES. There 
are also real differences across  countries, depending on the involvement in the Network 
that should be considered when analysing the average. 

On  average,  the  PES  estimated  that  around  200  days  of  staff  time  per  year  were 
committed to Network activities. This approximates to one full-time equivalent per PES 
per year. Concerning costs such as travel, accommodation, translation and expert inputs, 
PES  considered  them  hard  to  estimate  as  some  of  them  were  reimbursed  by  the 
Commission, while others could be merged into other budget items.  

At European level, the average annual cost is around €3.45 million per year, incurred by 
the European Commission and funded by the Employment and Social Innovation (EaSI) 
programme.  This  amount  includes:  (i)  the  annual  budget  to  implement  the  Network’s 
work  programme  (€3.12  million);  (ii) Commission  staff  devoted  to  the PES  Secretariat, 
allowances paid to Seconded National Experts42, mission costs for the Secretariat and for 
Commission  representatives  to  the  Board  (about  €200,000);  and  (iii)  costs  for  the 
organisation of meetings (Board and AFEPA meetings) and reimbursing the participants 
(about €130,000). 

These  resources  are  in  accordance  with  Article 7  of  the  Decision.  In  addition,  the 
Network Secretariat  liaises with  the AFEPAs  and other experts at national  level  with  a 
view to hosting and participating in learning events, data collections, etc. Compared, for 
example,  to  the  total  direct  support  from  the  ESF  to  reforms  to  modernise  PES 
(€2.1 billion over the period 2014-2020), the support for this Network is small. 

Other  networks  such  as  the  EURES  Network,  the  European  Social  Policy  Network 
(ESPN) and the European Labour Law Network are also receiving allocations from EaSI. 

42 Much of the cost of Seconded National Experts is borne by the national PES and in principle included in 

the national PES cost estimates. 

38 

 
                                                           
This  gives  a  scale  of  the  financial  support  involved.  In  2018,  EaSI  allocations  to  the 
EURES Network’s tasks related to network meetings, training and support activities was  
€4.45  million.  EaSI  allocations  in  the  field  of  labour  law  and  employment  and  labour 
market  policies  amounted  to  €2.2  million  to  ESPN,  while  the  European  Centre  of 
Expertise (ECE) received €2.0 million. However, it remains difficult to properly compare 
the costs of these networks due to  the difference in  their tasks and responsibilities. For 
example, the EURES Network organises training for advisers and line managers. As seen 
above, a substantial  task for the Network covered by this evaluation is  mutual  learning 
across  national  PES.  ESPN  provides  the  Commission  with  an  overview  of  policies  on 
social  protection  and  social  inclusion,  and  acts  as  the  secretariat  to  the  Mutual 
Information  Systems  on  Social  Protection  (MISSOC).  The  ECE  is  a  network  of  labour 
law experts and provides an ad hoc capacity in labour market policies analysis through a 
flexible pool of experts. It assists the Commission in ensuring a correct application of EU 
law across all Member States and monitoring labour legislation reforms in the EU. 

5.3.3.  Benefits 

PES  performance  and  impacts  for  the  users  of  PES  services  depend  on  many  different 
factors.  Impacts  on  PES  performance  can  be  indirect,  for  example  through  changes  in 
organisational  culture,  which  is  normally  also  affected  by  a  number  of  other  measures 
and channels. Furthermore, the causality of improvements, or how objectives have been 
achieved,  cannot  always  be  established.  For  instance,  reduced  unemployment  can  be  a 
consequence of better PES performance, but also of changes in demand for labour.  

However, many PES identified in the external study cases where they felt that their PES 
and  the  national  labour  market  had  benefited  from  participation  in  the  Network,  for 
example: 

• 
new initiatives or processes introduced following Network activities; 
• 
participation in the Network facilitated bilateral relationships between PES; 
•  Network participation stimulated internal learning, reflection and policy debate; 
•  Network  participation  enhanced  PES  staff  development,  motivation  and 

engagement. 

Many  consultees  highlighted  in  the  external  study  that  benchlearning  and  associated 
visits had a significant impact on helping individual PES understand how they compare 
with other PES, how they can plan for the future, and how they can evaluate their own 
activities.  It  is  true  that  some  of  the  examples  mentioned  in  the  consultations,  such  as 
new tools and development of staff, could to some extent have been implemented by the 
use of private sector consultants. This can, however, be costly. Moreover, such support is 
unlikely  to  be  as  relevant  as  the  learning  that  can  be  obtained  from  experienced  peers 
from  other  PES.  Less  advanced  PES  have  benefited  the  most  from  participating  in  the 
Network  in  terms  of  progress  in  performance  (see  Figure  2).  However,  comparing 
maturity  levels  between  the  first  and  second  cycles  of  benchlearning  demonstrates  that 
advanced PES have also progressed. 

39 

 
5.3.4.  Balance of costs and benefits 

The benefits of the Network’s  activities cannot  be directly  estimated, neither at  the EU 
level  nor  the  national  level.  How  the  Network  members  themselves  view  the  benefits, 
however,  is  borne  out  by  the  willingness  of  PES  to  participate  in  benchlearning  and 
mutual learning events and by the PES responses in the external consultation. 

In  the  external  consultation  (Section  4.3),  26  PES  responded  to  the  question  about  the 
costs and benefits of participation in the Network. Of these, 14 PES stated that benefits 
significantly  exceeded  costs.  A  further  5  stated  that  benefits  slightly  exceeded  costs, 
while 3 reported that  costs were  equal  to  benefits.  Only 1 PES  felt that  costs exceeded 
benefits. 3 PES responded that they were not sure. 

One of the respondents stated: 

‘To  our  knowledge,  the  EU  PES  Network  absorbs  only  a  very  limited 
amount  of  EU  public  resources,  but  coordinates  more  than  one  million 
public employees who are helping vulnerable and jobless across the Europe. 
With its activities aimed at better provision of quality services and efforts to 
improve internal efficiency, we believe that it is good value for money and 
HR spent.’ 

These results suggest that the ‘leverage’ of the Network is high, in that it yields benefits 
which considerably exceed the costs of participation, even if not equally distributed. The 
impact  of  scale  was  mentioned  in  the  consultation  as  contributing  to  leverage. 
Participation  in  the  Network  means  joining  a  vast  network  of  PES  employees  across 
Europe. 

Less mature PES are more likely to report net benefits of the Network participation than 
the  more  advanced  ones.  In  some  cases,  where  national  budgets  would  be  a  barrier  to 
Network  participation,  the  financial  investment  from  the  EU  was  mentioned  as  a  key 
contributor to the Network’s leverage. This indicates that the contributions from the EU 
are  important  both  to  ensure  that  the mature  PES  do  not  see  Network participation  as  a 
burden and to allow the less mature PES to participate in activities which are crucial for 
their further development. 

5.3.5.  Potential to increase efficiency 

Some potential means to further improve efficiency were mentioned in the external study, 
Section  4.1.  These  relate  mainly  to  better  targeting  of  learning  events  and  potential  to 
further use digital solutions. 

Some  consultees  mentioned  that  study  visits  could  be  organised  in  smaller  and  more 
targeted groups, and to increase the focus on training of PES employees so that they can 
have  a  greater  impact  on  Network  activities  on  the  ground.  Finding  solutions  for  the 
translation of materials, tools and guidelines is mentioned as a means to increase take-up 
of outputs by staff in local PES who may not speak English.  

A  result  of two  cycles  of  benchlearning is  the  accumulation  of  an  evidence  base  on the 
maturity  and  priorities  of  PES.  This  evidence  base  has  made  it  easier  to  target  learning 

40 

 
events.  This  has  paved  the  way  for  learning  dialogues,  a  new  format  of  learning  event 
starting in  2019.  This  peer-coaching  approach  related  to  specific  PES  learning  needs  is 
designed  to  support  national  PES  as  they  bring  about  change  in  an  identified  area  for 
improvement. 

Greater  efficiency  could  be  obtained  by  reducing  language  barriers.  Some  consultees 
mentioned that many PES staff cannot read outputs in English. Linked to this, it can be 
difficult to find PES staff proficient in English who are experts on certain topics and who 
can  attend  Network  activities.  Lack  of  internal  capacities  as  well  as  language  barriers 
between  experts  can  restrict  the  possibility  to  draw  benefits  from  mutual  learning.  A 
higher  focus  on  smaller,  targeted  events  could  therefore  increase  efficiency.  Calls  for 
experts to attend mutual assistance projects could, for example, put the emphasis on their 
knowing the native language in the receiving country. 

It was also mentioned that there could be a greater focus on dissemination, which might 
increase impacts, by targeting and distributing outputs more strategically. Efficiency can 
be increased by making better use of modern e-solutions and online interactive tools for 
communication, and through the exchange of best practices and dissemination of outputs. 
Two  webinars  were  organised  as  part  of  the  2018  work  programme  and  three  webinars 
will  be  organised  in  2019.  However,  as  pointed  out  by  several  consultees  during  the 
consultations,  personal  contact  between  PES  staff  is  the  key  to  the  success  of  the 
cooperation.  It  is  therefore  important  to  find  the  right  balance  between  face-to-face  and 
virtual meetings to achieve the efficiency goals. 

5.4.  Coherence 

5.4.1.  Consistency of the Decision with EU policies and initiatives 

The  preamble  of the  Decision  refers to  the  links  between the  Network’s responsibilities 
and other EU policies. Specifically, the Network should:  

• 
• 

• 
• 

contribute to the implementation of policy initiatives in the field of employment;  
support  initiatives  aimed  at  skills  matching,  decent  and  sustainable  work, 
voluntary labour mobility and transition from education and training to work;  
address the evaluation of active labour market policies;  
ensure that it complements and does not replace or duplicate other EU initiatives. 
Consistency  between  legal  frameworks  as  well  as  implementation  is  thus 
important. 

The Europe 2020 strategy is the EU’s agenda for growth and jobs for the current decade. 
One of the goals is to increase work participation to a rate of 75% for people aged 20-64. 
In the Council Decision on guidelines for the employment policies of the Member States, 
guideline  7  declares  that  the  Member  States  should  aim  for  better  and  more  effective 
public  employment  services  to  reduce  and  shorten  unemployment  by  providing  tailored 
services  to  support  jobseekers,  supporting  labour-market  demand  and  implementing 
performance- measurement systems. The PES Decision clearly states in Article 3 that it 
aims  to  “encourage  cooperation  between  Member  States  [...]  in  order  to  contribute  to 
Europe  2020”  and,  in  Article  4,  that  it  should  “contribute  to  modernising  and 

41 

 
strengthening  PES  in  key  areas,  in  line  with  the  employment  and  social  objectives  of 
Europe 2020”.  

The new legal  basis  for EURES  laid  down  in 201643  reinforces  the  obligations  to  share 
information and cooperate across Member States in areas such as apprenticeships. It also 
lays down a minimum set of support services that PES have to provide. EURES focuses 
on the pooling of job vacancies and applications and the provision of mobility services in 
view  of  facilitating  free  movement  of  workers  and  the  further  integration  of  labour 
markets.  The  external  study  explains  that  whereas  the  Network  focuses  primarily  on 
cooperation  to  improve  PES  performance  and  contribute  to  implementation  of 
employment  policies,  EURES  directly  targets  employers,  workers  and  jobseekers.  To 
avoid duplication and ensure synergies, the Network Board decided in 2016 to promote 
closer cooperation between the Network and EURES. 

The  Network  has  demonstrated  the  ability  to  adapt  to  new  legal  frameworks  for  EU 
policies.  The  dedicated  and  relevant  chapters  of  the  European  Pillar  of  Social  Rights, 
which was  proclaimed by the European Parliament, the Council  and the Commission in 
2017  (equal  opportunities  and  access  to  the  labour  market,  fair  working  conditions  and 
social protection and inclusion), which are aligned with the guidelines from the European 
employment  strategy,  have  become  guiding  in  setting  the  Network’s  initiatives  and 
actions. An example of this is the increased focus on the role of PES in outreach to the 
inactive population44. 

The  tasks  of  the  Network  also  include  monitoring  implementation  and  carrying  out 
analytical  activities such as  preparing reports and studies. The Network  has  established 
its identity and presence at European level and helps national PES to perform better and 
deliver employment policies more successfully, taking into account the impact of labour 
market trends. The Network’s activities have also contributed to better analysis as part of 
the European Semester and the country-specific recommendations (CSRs) it produces. In 
2018,  11  Member  States  received  CSRs  related  to  their  PES  and  active  labour  market 
policies  (ALMPs).  This  includes  needs  for  more  personalised  tailored  services  for  the 
unemployed,  a  more  comprehensive  performance  measurement  system,  better  targeting 
and prioritisation, and better coordination between central and regional administrations. 
The CSRs focus for some Member States on measures to make ALMPs more effective 
and accessible to those furthest away from labour market. 

To address these challenges identified in the European Semester, the PES Network uses 
its benchlearning approach, striving for continuous performance improvement. One of its 
unique features is the attention given to organisational factors, drivers and practices that 
are likely to improve PES performance. 

43 Regulation (EU) 2016/589 of the European Parliament and of the Council of 13 April 2016, OJ L 107, 

22.4.2016, p. 1-28 

44 https://ec.europa.eu/social/BlobServlet?docId=19219&langId=en 

42 

 
                                                           
Overall,  the  external  study,  Section  5.2,  revealed  that  most  stakeholders  consider  that 
there is a high degree of consistency between the Decision and the EU policy framework. 
Although improvements could be made in future to improve synergies, no duplication of 
action  was  observed.  There  is  some  cooperation  with  other  relevant  labour  market 
stakeholders  (as  set  out  in  Article 5 of  the  Decision)  at  EU  level,  but  there  is  scope  for 
improvement in future. 

In particular, a large majority of the PES consultees pointed out that the Decision reflects 
the relevant priorities set out in the Europe 2020 strategy. This complementarity was also 
emphasised in relation to the shared goals and commitments included in the Decision and 
other European initiatives such as the Youth Guarantee, the Youth Employment Initiative, 
the  Council  Recommendation  on  the  integration  of  the  long-term  unemployed  into  the 
labour market, and ESF-funded programmes. Synergies were highlighted in relation to the 
mandate  of  other  EU  institutional  actors  such  as  the  EURES  Network  and  EMCO.  The 
fact  that  different  institutional  actors  and  networks  share  similar  core  objectives  —  e.g. 
reducing  unemployment,  improving  labour  market  policies’  effectiveness  and  the 
inclusion of vulnerable groups — was acknowledged and seen in very positive terms by 
the  majority  of  those  consulted.  The  facilitation  of  information  exchange  and  best 
practices  across  institutions  and  networks  at  the  EU  and  national  level  was  likewise 
deemed as particularly important in facilitating complementarity and coherence.  

5.4.2.  Contributions of the Network to relevant EU policies and initiatives 

The Network has contributed to implementation of a broad range of EU policy initiatives. 
Table  1  in  Section  3.6  gives  an  overview  of  selected  examples  of  the  Network’s 
contributions  to  policy  implementation.  The  Network’s  contributions  have  been  most 
extensive in implementation of the Youth Guarantee and the integration of the long-term 
unemployed  into  the  labour  market.  The  Network  has  also  demonstrated  flexibility  to 
target  new  topics  such  as  the  integration  of  migrants  and  refugees,  the  prevention  of 
unemployment  and  addressing  skills  shortages.  The  Network  has  also  provided 
contributions to implementation of the European Pillar of Social Rights.  

As regards the contributions of the Network to the implementation of the CSRs under the 
European  Semester,  PES  are  continuing  to  innovate  in  their  development  of  ALMPs, 
while  general  ALMP  expenditure  is  increasing  at  many  PES.  Moreover,  PES  are 
adapting  their  staff  deployment  to  the  changing  environment.  This  includes  specialised 
teams  offering  tailored  approaches  and  intensive  support  to  target  groups  such  as  the 
long-term  unemployed  and  customers  with  disabilities.  For  example,  in  2017  Spain 
received a CSR to reinforce coordination between regional employment services, social 
services and employers  so as to  better respond to jobseekers’ and employers’ needs.  In 
this  context,  the  Spanish  PES  requested  a  mutual  assistance  project  from  the  PES 
Network to help it work towards addressing the CSR. 

5.4.3.  Large degree of complementarity with the EU policy framework 

Other  stakeholders  confirmed  the  high  degree  of  complementarity  with  the  EU  policy 
framework.  There  is  well-functioning  cooperation  between  the  Network  and  other 
bodies within the EU. As an illustration, there are various forms of contact with EMCO 

43 

 
through the Committee’s role as an observer in the Network, through presentations by the 
Network Chair at EMCO meetings and through contacts between the PES Secretariat and 
the EMCO Secretariat. 

However,  some  concerns  were  also  raised  in  the  consultations  about  the  potentially 
overlapping mandates of the Network, the EURES Network, EMCO and — potentially — 
the European Labour Authority. Overall, there was agreement about the need for effective 
cooperation between all relevant bodies at EU and national level to ensure that common 
objectives are achieved and that outcomes are sustainable. However, it was felt that while 
further upfront coordination could avoid some overlaps, most overlaps of topics tend to be 
mutually  reinforcing  and  can  provide  synergies,  thus  addressing  key  topics  (e.g. 
integration  of  migrants)  from  different  perspectives  and  with  different  types  of  actors. 
There  was  no  evidence  found  of  duplication  of  actions  with  other  EU  policies  or 
initiatives. 

PES in all Member States are in one way or another involved in implementing the ESF. 
The  strength  of  the  links  between  national  PES  and  national  ESF  programming  and 
funding  varies  between  countries.  For  instance,  in  several  countries  (e.g.  Bulgaria, 
Croatia,  France,  Luxembourg,  Hungary,  Lithuania  and  Romania)  the  PES  actively 
cooperates  with  the  ESF  managing  authorities  (Ministry  for  Social  Affairs,  Ministry  of 
Labour or other public authorities) and receives funds for their operational programmes. 
In  a  few  other  countries  (e.g.  Austria  and  Greece),  the  influence  of  the  PES  on  ESF 
funding and programming is more limited, due for example to the different parameters for 
cooperation  with  ESF  managing  authorities  or  because  PES  do  not  use  ESF  funds  to 
implement their initiatives, measures or programmes. 

5.4.4.  Potential to increase consistency 

To reinforce the complementarity of activities of the Network with relevant policies and 
initiatives,  the  Network  could  reach  out  more  widely  to  other  relevant  labour  market 
stakeholders  as  envisaged  under  Article 5  of  the  Decision.  These  could  include  other 
providers  of  employment  services,  social  partners  and  NGOs  working  in  the  field  of 
employment. While such cooperation has been observed in certain cases at national level, 
and  established  sometimes  formally  at  EU  level,  it  does  not  yet  appear  to  be  extensive 
across  all  countries  or  at  EU  level.  Building  close  links  between  PES  and  focusing 
discussions on issues of particular relevance to PES has most likely been beneficial in this 
relatively  early  phase  of  inter-PES  cooperation.  However,  it  could  be  beneficial,  as  the 
cooperation  deepens,  to  explore  whether  other  actors  such  as  relevant  EU-level  NGOs, 
social partners and networks should have more structural links with and roles within the 
Network.  While  the  Chair  already  has  a  role  representing  the  Network  to  other 
stakeholders, the external role of the Chair could be further developed. 

5.5.  EU added value 

5.5.1. 

Impacts of the Decision 

The  findings  in  the  external  study,  Section  6.2,  demonstrated  a  number  of  significant 
impacts, outputs and results of the Decision which did not occur before the Network was 

44 

 
set  up in  2014 and would not  have been achieved by voluntary  cooperation of national 
PES, confirming the added value of the PES Network: 

• 

• 

• 

a  structured  framework,  via  the  benchlearning  process,  for  assessing  PES 
performance and capacity and for facilitating comparisons and improvements; 

increased  and  more  structured  cooperation  and  peer  learning  between  PES  to 
tackle labour market needs across the EU; 

greater ownership, collegiality and buy-in by all national PES, in particular via 
their  role  on  the  PES  Network’s  Board  and  in  approving  annual  work 
programme; 

•  more  effective  assessment  and  identification  of  the  needs  and  areas  of 
development for individual PES capacity and performance via the benchlearning 
process; 

• 

• 

• 

• 

• 

the  provision  of  appropriate  individual  support  and  collective  mutual  learning 
for  PES  based  on  precise  needs 
the  benchlearning 
assessments; 

identified 

through 

a greater understanding among EU policymakers of the needs and challenges of 
national (and regional) PES, leading to more effective policy solutions; 

ensuring a collective voice and formal platform for PES in policymaking at EU 
level  in  relevant  fields  through,  for  example,  the  monitoring  of  the  Council 
recommendations establishing the Youth Guarantee and on the integration of the 
long-term  unemployed,  and  through  contributions  to  relevant  policy  initiatives 
such as the European Pillar of Social Rights; 

contributions  to  the  EU2020  objectives  by  providing  targeted  support  to  PES 
and cooperating with EMCO (and the labour ministries) to identify those which 
require additional support; 

a  common  strategic  framework  for  PES  cooperation  to  tackle  the  increasing 
common global challenges that PES are facing, which require concerted action. 

As regards the impact at national level, the size and scale of the added value differs 
across countries, but the vast majority of PES reported that participation in mutual 
learning  events  has  been  useful  for  them.  The  evaluation  of  the  mutual  learning 
events45  indicated  that  for  20%  of  the  respondents,  their  participation  in  the  mutual 
learning  activities  was  critical  to  initiate  and  support  changes.  Over  half  of  the 
respondents  considered  that  the  changes  would  have  happened  anyway,  but  less 
efficiently. 27% of the respondents considered that the changes would have happened in 
the same way without their participation in the mutual learning activities. 

The  benchlearning  approach  has  attracted  interest  outside  of  the  Network,  and 
contributed  to  value  added  also  beyond  the  EU  level.  A  simplified  version  of 
benchlearning is being implemented in Western Balkan countries. In addition, the World 

45  Service  contract  for  measures  to  enhance  cooperation  between  Public  Employment  Services  (PES),  in 
particular services to implement a ‘benchlearning’ concept within the PES Network (VC/2015/0062)  - 
Final Evaluation Report – Year 4 (annex available on request). 

45 

 
                                                           
Association  of  Public  Employment  Services  (WAPES)  is  considering  developing  a 
version to be used on a global basis. 

5.5.2.  The role of the Network in cooperation between PES 

PES representatives in particular confirmed in the external study that information sharing 
about PES practices would not have been as effective and systematic in the absence of the 
Decision  (see  Section  6.3  in  the  external  study).  Furthermore,  without  the  Network’s 
existence,  Network  activities  such  as  the  self-assessments  and  external  assessments 
carried  out  within  the  benchlearning  process,  mutual  assistance  visits  and  other 
opportunities  to  compare  different  practices  and  models  are  unlikely  to  have  been 
implemented. The Network is a mechanism that ensures the continuity of collaboration in 
a  way  that  includes  all  its  members.  While  many  consultees  considered  that  some  PES 
would continue to cooperate in the absence of a formal Network, they also stressed that 
cooperation was unlikely to be maintained with the same frequency, depth, structure and 
commitment from all participating PES. Representatives from PES with less financial and 
human  resources,  and  those  facing  the  greatest  challenges  in  the  development  of  their 
PES, underlined that in the absence of the Network they would not be able to participate 
to the same degree in cooperation with other PES.  Failing to continue the  initiative was 
also seen as a loss of the resources invested to date. 

Due  to  the  Decision,  the  Network  obtained  a  formal  role  in  organising  cooperation  and 
contacts with other stakeholders. This commitment and support of resources provided at 
EU  level  has  contributed  to  ensuring  the  effective  implementation  of  the  Decision,  in 
terms  of  both  financing  (funding  for  Network  initiatives  including  meetings,  events, 
benchlearning expertise) and human resources (the PES Secretariat, contractors) invested 
in  the  running  of  the  Network.  Comparison  with  the  situation  before  2014  as  well  as 
responses from PES in the external study are strong indications that a formal commitment 
to  continuing  to  provide  resources  for  the  Network  is  important  to  maintain  the  scope, 
extent  and  quality  of  PES  cooperation  and  the  Network’s  ability  to  implement  strategic 
EU employment goals. 

5.5.3. 

 Potential for increased added value 

The  2017  interim  review  of  the  Network  activities46  emphasised  that  PES  are  stronger 
when they speak with one voice, and that the Network has been productive in its output. 
Many  of  these  outputs  have  a  wider  benefit  for  researchers  and  decision-makers.    The 
review confirmed that greater visibility of the Network and its outputs would ensure that 
these resources are used effectively. The external study also highlights the potential for 
improved  visibility  of  the  added  value  of  PES  (Section  7.2).  An  agreement  from  the 
Board  to  allow  benchlearning  data  to  be  published,  would  strengthen  the  business  case 
for reform, and further increase the added value of the Network for achieving EU policy 
objectives. 

46 http://ec.europa.eu/social/BlobServlet?docId=17834&langId=en 

46 

 
                                                           
6.  CONCLUSIONS AND LESSONS LEARNED 

Relevance 

The Decision remains highly relevant for PES. The objectives of the Decision, as set out 
in  Article 3,  and  the  initiatives  set  out  in  Article 4,  encompass  the  key  areas  of  PES 
responsibility and provide a robust framework for the policy and concrete activities of the 
Network. 

How  PES  can  contribute  to  the  better  functioning  of  the  labour  market  in  the  changing 
world of work has been addressed more prominently in the Network’s work programme 
over the past years. These challenges are also having an impact on how the objective of 
decent and sustainable work can be achieved, as new working conditions are resulting in 
employment  opportunities  in  new  technology  sectors,  the  large-scale  digitalisation  of 
many  professions,  more  short-term  flexible  working  and  a  growing  trend  towards  self-
employment and platform jobs. By providing a broad framework for Network activities, 
the objectives enable a flexible approach to addressing priorities for action depending on 
labour market developments. 

Effectiveness 

The evaluation reveals that the Network has been effective in delivering its initiatives and 
objectives.  PES  have  taken  up  the  results  of  PES  cooperation,  and  the  PES  have 
increased  their  level  of  maturity  as  organisations.  The  Network’s  effectiveness  is 
illustrated  out  by  the  strong  endorsement  and  implementation  of  the  successful 
benchlearning concept,  as well as by the success of knowledge sharing through mutual 
learning events, reports and PES practices.  

Benchlearning, including the learning dialogues, has proven to be a successful example 
of systematic evidence-based learning, which could be transferable to other policy fields 
at EU level. The fact that so many different PES have agreed on a set of quantitative and 
qualitative  common  indicators  for  the  benchlearning  process  is  in  itself  a  significant 
achievement. 

The  benchlearning  project  has  been  a  key  success  factor  in  the  establishment  of  a 
structured  process  for  learning  and  sharing  of  practices.  The  Network  has  served  as  a 
mechanism ensuring the commitment and participation of all its members. This extensive 
cooperation,  which  includes  the  benchlearning  process,  mutual  assistance  visits,  and 
mutual  learning  activities  to  share  and  explore  different  practices,  are  unlikely  to  have 
been implemented without the Network. 

Efficiency 

The  Network  has  been  efficient  in  delivering  initiatives  and  reaching  objectives.  Some 
Network activities are difficult to quantify, and four years is also a short period to realise 
certain gains and make them visible, such as changes in organisational culture. However, 
the willingness of PES to participate in benchlearning and mutual learning events, as well 
as positive feedback from PES demonstrate that the Network outputs are highly effective 
and  appreciated  by  all  Network  members.  Overall,  less  advanced  PES  have  benefited 

47 

 
more  from  participating  in  the  Network  in  terms  of  the  progress  they  have  made  on 
performance, but advanced PES have also progressed. 

Coherence 

The evaluation reveals that there is a good degree of consistency between the Decision 
and the EU policy  framework (for instance with  concrete policy initiatives, such as the 
recommendations  on  the  Youth  Guarantee  and  the  integration  of  the  long-term 
unemployed  into  the  labour  market).  No  duplication  of  action  is  observed.  As  an 
example,  whereas  the  Network  focuses  primarily  on  cooperation  to  improve  PES 
performance  and  contribute  to implementation  of  employment  policies, EURES directly 
targets employers, workers and jobseekers. 

The Network has contributed extensively to the implementation of EU relevant policies 
and initiatives  in  various areas and by different  means, and has established a collective 
voice and a formal platform in policymaking at EU level, while making contributions to 
the EU 2020 strategy for jobs, and smart, sustainable and inclusive growth. 

The  Network’s  contributions  have  been  most  extensive  in  the  long-standing  issues  of 
implementation of the Youth Guarantee and the integration of the long-term unemployed 
into  the  labour  market.  The  Network  has  also  demonstrated  flexibility  to  target  new 
topics such as the integration of migrants and refugees, the prevention of unemployment 
and  addressing  skills  shortages.  Moreover,  the  Network  supports  national  PES  as  they 
implement country-specific recommendations addressed to the Member States on active 
labour market policies (ALMPs) and PES issues in the context of the European Semester.  

EU value added 

An added value of the PES Network has been a structured framework for assessing PES 
performance  and  capacity  and  for  facilitating  comparisons,  peer 
learning  and 
improvements.  The  evaluation  demonstrates  several  other  results  of  the  Decision  that 
would not have been achieved by voluntary cooperation of national PES. Some important 
achievements  are  greater  ownership,  the  provision  of  individual  PES  learning  and 
collective mutual  learning, ensuring a collective  voice and a formal platform  in  policy-
making at EU level, and contributions to the EU2020 objectives. Some activities, such as 
reports or conferences, are able to reach stakeholders beyond the Network, for example 
researchers, decision-makers, social partners, NGOs and international organisations. The 
Network also plays an important role in representing the unique perspective of PES in the 
design,  implementation  and  monitoring  of  the  success  of  policies  relevant  to  labour 
market  integration  and  retention.  Nevertheless,  there  is  potential  to  develop  stronger 
relationships  with  social  partners,  NGOs  and  ESF  managing  authorities,  and  at  the 
international level in order to increase synergies and mutual benefits. 

To  date,  there  is  no  information  available  on  how  cooperation  across  PES  could  be 
organised if the Decision is not extended. It is, however, likely that information sharing 
would have been less effective and systematic in the absence of the Decision. Although 
some  cooperation  is  likely  to  continue  without  the  Network,  the  formal  framework  for 
cooperation  that  the  Decision  supports,  as  well  as  the  Commission’s  technical  and 
financial support, are considered by stakeholders as crucial. 

48 

 
Lessons learnt 

Overall, the evaluation shows that the Decision has been successfully implemented and 
some lessons learnt could be identified. The persisting difficulties that vulnerable groups 
face when trying to (re-)enter the labour market demonstrate the continued relevance of 
the objective of supporting vulnerable groups with high unemployment rates. 

Less advanced PES have benefitted more from participating in the Network in terms of 
the  progress  they  have  made  on  performance,  but  advanced  PES  have  also  progressed. 
The evaluation clearly underlines the importance of continued focusing on learning and 
sharing experiences. The recently launched learning dialogue is a promising new learning 
format  which  actively  supports  the  national  reform  agenda  via  peer  PES  coaching.  It 
demonstrates  the  evolution  of  individual  peer  PES  learning  and  the  growth  of  the 
Network as a learning organisation itself. Potentials to further improve efficiency of the 
Network include focusing on smaller and more targeted learning events and the potential 
for further use of digital solutions. 

Since quantitative information  can form part of the planning of initiatives and reforms, 
one aim should be to ensure that PES make more consistent use of the PES Dashboard, 
where the quantitative benchlearning indicators can be accessed. This could be achieved 
by allowing benchlearning data to be published, as it would strengthen the business case 
for reform and further increases the added value of the Network for achieving EU policy 
objectives. 

The PES are also key actors implementing the European Social Fund, and play an active 
role in the development of a comprehensive strategy for a policy framework on ALMPs, 
thereby contributing to the effective and efficient spending of EU funds. 

Financial, organisational and expert support from the EU level is essential to ensure the 
ongoing  participation  of  all  PES.  Without  such  support,  some  countries  could  face 
difficulties  participating,  while  commitments  to  long-term  activities  could  become 
difficult, and cooperation could be more fragmented in terms of themes. 

49 

 
 
Annex 1: Procedural information 

Organisation 

DG Employment, Social Affairs and Inclusion (DG EMPL) is the lead for the evaluation 
of  Decision  No  573/2014/EU  on  enhanced  cooperation  between  Public  Employment 
Services (PES). 

The  evaluation  started  with  the  publication  of  the  evaluation  roadmap47  on  27 October 
2017. The evaluation was carried out with the support of the interservice group chaired 
by  DG  EMPL,  to  which  the  following  Directorates-General  were  invited:  ECFIN, 
HOME, RTD and SG. 

Evidence, sources and quality 

Both internal and external expertise were deployed to ensure the quality of the evaluation 
and  the  staff  working  document,  applying  the  triangulation  method  to  cross-check  the 
quality of the findings (see the methodology section below). The main sources are listed 
below (other secondary sources are cited throughout in the text): 

•  External  study  carried  out  by  Ecorys48.  The  quality  of  the  deliverables  was 

endorsed by the inter service group. 

•  Commission report on the PES Network Decision to the European Parliament, the 
Council, the European Economic and Social Committee and the Committee of the 
Regions adopted on 6 June 201749. 

•  Annual reports of the Network for the years 2014-201850. 

•  The  outputs  produced  by  the  Network  (thematic  and  analytical  papers,  PES 
practices,  success  stories,  manuals,  toolkits,  working  papers,  study  visits,  etc.) 
published on the Knowledge Centre51. 

•  Satisfaction surveys and other monitoring activities conducted by the contractors 

supporting the implementation of the Network’s work programme. 

•  Yearly survey and report on PES capacity for the years 2014-201852. 

•  Yearly  reports  on  PES  implementation  of  the  Youth  Guarantee  available  for 

2014-201753. 

47 https://ec.europa.eu/info/law/better-regulation/initiatives/ares-2017-5258712_en 
48 https://publications.europa.eu/en/home 
49 http://ec.europa.eu/social/BlobServlet?docId=17834&langId=en 
50 https://ec.europa.eu/social/main.jsp?catId=1163&langId=en 
51 https://ec.europa.eu/social/main.jsp?catId=1163&langId=en and 

https://ec.europa.eu/social/main.jsp?catId=1206&langId=en 
52 https://ec.europa.eu/social/main.jsp?catId=1163&langId=en 
53 https://ec.europa.eu/social/main.jsp?catId=1163&langId=en 

50 

 
                                                           
•  Annual  data  collection,  validation  and  analysis  gathered  via  the  benchmarking 
exercise for the years 2015-2018. Overall assessment reports from the two cycles 
of benchlearning visits to the PES and their change reports one year after the first 
site visit. 

•  EMCO thematic reviews of PES/ALMPs and the European Semester54. 

•  ESF synthesis report of the annual implementation report for 201755. 

54 https://ec.europa.eu/social/main.jsp?catId=115 
55 http://ec.europa.eu/esf/home.jsp?langId=en 

51 

 
 
                                                           
Annex 2: Stakeholder consultation — Synopsis report 

The  main  goal  of  the  stakeholders’  consultation  was  to  collect  data,  experiences  and 
opinions  on  the  relevance,  effectiveness,  efficiency,  coherence  and  EU  added  value  of 
Decision  573/2014/EU  on  enhanced  cooperation  between  Public  Employment  Services 
(PES)  from  relevant  stakeholders  groups  and  the  general  public.  This  was  done  in  line 
with  the  Better  Regulation  guidelines56,  which  mandate  transparent  and  inclusive 
policymaking by offering interested citizens and stakeholders the opportunity to provide 
feedback on the design of polices and their evaluation methods. 

This  annex  presents  the  consultation  strategy,  activities  undertaken,  overview  of  the 
consulted stakeholders, the analysis process, the dissemination strategy and overview of 
the results of the consultation. Activities and analysis were mainly conducted under the 
external study57. 

1.  CONSULTATION STRATEGY 

The  Commission  published  an  evaluation  roadmap58  to  provide  information  on  the 
content and the strategy of the evaluation of Decision 573/2014/EU. 

The  roadmap  was  available  for  public  feedback  from  27  October  until  24 November 
2017.  It  yielded  two  responses59  from  NGOs.  This  feedback,  which  was  rather  brief, 
supported  the  initiative  and  expressed  a  desire  for  the  initiative  to  continue  post-2020. 
Therefore it is not commented on further. 

The  strategy  in  the  roadmap  made  provision  for  the  consultation  activities  and  the 
stakeholders to be targeted. These are detailed below.  

1.1. Identification of stakeholders 

The main stakeholders  identified were the members of the Network:  the PES  of the 28 
Member  States  and  PES  from  EEA  countries.  Both  Board  members  and  Advisers  for 
European PES Affairs (AFEPAs) were consulted, as well as the Employment Committee 
(EMCO) as an observer to the Network. Other main stakeholders included the Network 
of  Member  State  Labour  Ministries,  EU-level  organisations  representing  private 
employment services and temporary work agencies, the EURES Network and the World 
Association of Public Employment Services (WAPES). 

Stakeholders  not  covered  by  any  targeted  consultation  were  encouraged  to  take  part  in 
the public consultation. Examples of stakeholders falling into this category include social 
partners, local and regional authorities in Member States, civil society organisations, job 
seekers and citizens, and researchers and academia. 

56 https://ec.europa.eu/info/law/law-making-process/planning-and-proposing-law/better-regulation-why-and-how_en 
57 Study supporting the evaluation of Decision on enhanced cooperation between Public Employment Services, Ecorys 

for the European Commission, 2019 

58 https://ec.europa.eu/info/law/better-regulation/initiatives/ares-2017-5258712_en 
59 https://ec.europa.eu/info/law/better-regulation/initiatives/ares-2017-5258712/feedback/add_en 

52 

 
                                                           
The  detailed  description  of  the  different  groups  of  stakeholders  for  each  consultation 
activity is outlined in Section 3. 

1.2.   Consultation activities 

The strategy in the roadmap indicated that the evaluation of the Decision would involve 
targeted  consultations  of  the  main  stakeholders  and  a  12-week  public  consultation  in 
German,  French  and  English,  accessible  via 
the  Commission’s  central  public 
consultations page60. Replies could be made in any of the 24 official EU languages. 

Detailed activities included the following:  

  public consultation;  

 

targeted consultations including:  

o 

o 

targeted  semi-structured  interviews  with  a  broad  range  of  Network 
members  and  other  stakeholders  at  the  national,  EU  and  international 
levels; 
targeted  questionnaires  for  ESF  managing  authorities,  labour  and  social 
affairs ministries; and  

o  an evaluation workshop with the AFEPAs 

  case  studies  (in  five  countries  —  Estonia,  France,  Italy,  Netherlands  and 
Romania) which also included consultations with key stakeholders in the form of 
in-depth interviews and focus groups.  

The  consultation  approach  was  suitable  for  the  scope  of  the  consultation  and 
implemented  as  planned.  The  purpose  of  these  consultation  activities  and  the  range  of 
participants are described in the next section.  

2.  CONSULTATION ACTIVITIES 

This  section  presents  the  consultation  activities  carried  out  for  the  purpose  of  this 
evaluation under the external study. 

2.1. Public consultation 

The public consultation was launched on 20 September 2018 as an online questionnaire 
on the Commission’s public consultation website61. It remained open until 13 December 
2018, by which time the consultation had accumulated a total of 126 respondents. 

60 https://ec.europa.eu/info/consultations/public-consultation-enhanced-cooperation-between-public-employment-

services-1_en 

61 https://ec.europa.eu/info/consultations/public-consultation-enhanced-cooperation-between-public-employment-

services-1_en 

53 

 
                                                           
The  consultation  was  open  to  any  interested  party  and  to  the  general  public.  Care  was 
taken to ensure that a range of key viewpoints were represented, going beyond the close 
circle of those familiar with the 2014 Network set-up, and to encourage participation by 
key  stakeholders  not  expressly  targeted  by  the  targeted  consultations.  For  that  purpose, 
the  public  consultation  was  further  disseminated  and  promoted  by  sending  targeted 
emails to  key  representatives of identified organisations62 and asking them to  share the 
notification  with  the  members  of  their  organisations  or  networks  through  their  email 
distribution lists, websites and social media channels. 

2.2.Targeted consultations 

Targeted consultations were held with key stakeholders, with the aim of producing more 
detailed inputs on issues closest to the stakeholders’ experience and involvement with the 
Network. The targeted consultations consisted of four main activities: (i) interviews with 
key  stakeholders;  (ii)  written  questionnaires;  (iii)  workshops;  and  (iv)  consultation 
activities  in  the  context  of  five  country  case  studies  (focus  groups  and  interviews) 
conducted as part of the external supporting study. 

2.2.1.  Targeted semi-structured interviews 

Interviews were conducted with:  

  members of  each of the  32 PES  involved in  the Network63 (when possible both 

with the AFEPAs and the Board Member);  

  selected  representatives  of  relevant  EU-level  organisations  and  bodies  (e.g. 
EMCO, EU-level private employment services and temporary work agencies, the 
European Lifelong Guidance Policy Network);  

  selected representatives of relevant international organisations (e.g. ILO, OECD, 

World Bank, WAPES);  

  other stakeholders closely involved in the running of the Network (e.g. the PES 

Secretariat, contractors);  

  a  sample  of  individuals  who  were  former  members  of  the  Network  or  involved 

with PES cooperation pre-2014. 

62 These were: social partners at EU and national level, employers, local/regional authorities, organisations or networks representing 

the interests of different groups of jobseekers, members of academic/research networks, citizens. 

63 The PES Network includes PES from each of the EU-28 Member States, as well as  Iceland and Norway (30 countries in total). 
However, since in the case of Belgium there are three participating PES, the total number of PES participating in the Network is 32. 

54 

 
                                                           
The  targeted  semi-structured  interviews  were  conducted  from  September  2018  until 
January  2019  to  best  accommodate  stakeholders’  availability  and  ensure  a  large  and 
representative  pool  of  responses  across  the  Network  as  well  as  from  external 
stakeholders.  One  topic  guide  was  developed  for  interviews  with  PES  Network  Board 
members and AFEPAs, while a separate topic guide was prepared for other stakeholders 
(e.g.  EU-level  and  international  organisations),  which  could  be  adapted  to  each 
interviewee. A third topic guide was subsequently created for Network interviewees who 
asked to do the interview in writing.  

In most cases, the conversations were digitally recorded, with respondents’ prior consent, 
as  an  aide  mémoire  for  the  interviewer  and  to  facilitate  detailed  write-ups.  The 
interviewees  were  informed  that  their  feedback  was  confidential,  and  interview 
conversations  were  recorded  only  with  the  interviewees’  explicit  approval.  In  some 
instances, stakeholders preferred not to be recorded; in these cases, notes were taken.  

The  subject  fields  covered  by  the  topic  guide  aimed  to  encompass  all  five  evaluation 
criteria (relevance, effectiveness, efficiency, coherence and EU added value). Within this 
context,  retrieving  information  for  the  efficiency  criteria  was  challenging  as  very  little 
information about labour (days invested in the Network activities) and non-labour costs 
(other  costs  such  as  travel,  translation,  event  organisation)  was  readily  available  to  the 
PES  stakeholders  interviewed.  Consequently,  following  the  interim  meeting,  a  short 
follow-up with each PES was conducted, asking three cost-efficiency related questions. 
The questions were sent via email in early December; after follow-up, the response rate 
was very high, with 30 PES out of 32 submitting their answers. 

2.2.2.  Written questionnaires 

Two separate questionnaires were designed, with one targeting ESF managing authorities 
and  the  other  labour  ministries.  The  questionnaires  were  sent  on  9 October  2018, 
followed by a reminder sent on 7 November 2018 and a final reminder sent at the end of 
December 2018. 

2.2.3.  Workshop 

A workshop was held at the meeting of the AFEPAs on 11 October 2018. The workshop 
commenced  with  a  presentation  of  the  evaluation  aims,  approach  and  timetable  of  the 
evaluation,  including  the  AFEPAs’  role.  Two  rounds  of  breakout  sessions  (with  four 
subgroups) then took place: part I focused on the  main positive impacts of the Network 
to date, while part II focused on identifying the Network’s future aims and activities, and 
possible  improvements.  A  seven-page  outcome  document  presenting  lessons  from  the 
workshop activities was circulated shortly after the workshop to relevant stakeholders64. 
The  AFEPA  workshop  write-up  also  served  as  a  key  reference  document  providing 
background  knowledge  and  context  for  the  researchers  conducting  the  semi-structured 
interviews. 

64 See Annex 6 to the Study supporting the evaluation of Decision on enhanced cooperation between Public Employment Services, 

Ecorys for the European Commission, 2019 

55 

 
                                                           
2.2.4.  Case studies 

Case  studies  were  carried  out  in  five  countries  (Estonia,  France,  Italy,  the  Netherlands 
and  Romania)  to  gain  deeper  insight  into  the  actions  taken  in  response  to  the  PES 
Decision at national/regional/local level(s) and to assess impact on individual PES. The 
purpose of the case studies was to explore in depth the qualitative evidence and to look 
into  actions  taken  in  response  to  the  PES  Decision  at  national/regional/local  level(s)  to 
assess impact on individual PES and their influence on programming and funding, where 
applicable. 

The four main tasks in the context of the case studies included secondary data analysis 
activities (desk-based research), two primary data collection methods (interviews and, in 
certain  countries,  a  focus  group),  and  analysis  and  reporting.  The  focus  groups  set  out, 
where feasible, to explore the changes triggered by the PES Decision through collective 
discussions with key stakeholders in each selected case study country. Such stakeholders 
included  employment  service  providers  and  representatives  of  PES  ‘users’  (e.g. 
jobseekers, employers, other labour market stakeholders). 

2.2.5.  Summary and reflections on challenges 

The  consultation  methods  were  overall  implemented  as  planned  and  in  line  with  the 
agreed  consultation  strategy.  The  interviews  were  conducted  as  planned,  both  for  the 
targeted consultations and as part of the case studies. The response rate for the in-depth 
interviews  was  very  high  (only  one  individual  included  in  the  consultation  strategy 
neither  replied  nor  delegated  the  interview  to  a  colleague).  The  response  rate  for  the 
targeted  questionnaires  was,  however,  lower:  only  16  labour  ministries  and  7  ESF 
managing authorities filled out the questionnaires sent by the evaluation team. The rate of 
response to the consultations conducted in the five case studies fluctuated. In some case 
study countries, it was necessary to replace the focus groups with additional interviews as 
for different reasons65 it was not possible to organise the interviews within the evaluation 
timeframe. 

3.  INFORMATION ON STAKEHOLDER GROUPS 

3.1. 

Public consultation 

A  total  of  126  consultation  responses  were  received  from  public  consultation 
stakeholders  by  the  cut-off  date  of  13  December.  Their  profile  is  presented  in  the 
following subsections. 

65 For the French study, the PES was unable to find a suitable date in November 2018, December 2018 or January 2019 for the focus 
group;  for  the  Estonian  case  study,  there  were  not  sufficiently  large  numbers  of  individuals  and  organisations  with  sufficient 
knowledge of the PES Network to warrant a focus group; in the Dutch case study, the Dutch PES did not wish for a focus group 
to be organised because there were not sufficiently large numbers of individuals and organisations with sufficient knowledge of 
the PES Network, so instead the Dutch PES facilitated the organisation of interviews with several representatives involved in the 
PES Network in different capacities. 

56 

 
                                                           
3.1.1.  Country of respondents 
Out  of  the  126  responses,  the  highest  numbers  were  from  Italy  (23)  and  France  (22). 
Table 1 below sets out the number of responses received from each country, as well as 
the percentage of the overall number of responses. 

Table 1: Distribution of respondents by country 

Country 

Italy (n= 23) 

Respondents 
18.3% 

France (n= 22) 
Spain (n= 19) 
Belgium (n= 15) 
Germany (n= 13) 
Finland (n= 3) 
Portugal (n= 3) 
Greece (n= 2) 
Netherlands (n= 2) 
Slovenia (n= 2) 
Sweden (n= 2) 
Bulgaria (n= 1) 
Croatia (n= 1) 
Czech Republic (n= 1) 
Hungary (n= 1) 
Latvia (n= 1) 
Liechtenstein (n= 1) 
United Kingdom (n= 1) 
Pan-EU/European umbrella organisation (n = 6) 
Other (n= 7) 
Total (n = 126) 

17.5% 
15.1% 
11.9% 
10.3% 
2.4% 
2.4% 
1.6% 
1.6% 
1.6% 
1.6% 
0.8% 
0.8% 
0.8% 
0.8% 
0.8% 
0.8% 
0.8% 
4.8% 
5.6% 
100% 

3.1.2.  Sector of activity 
The highest number of respondents mentioned ‘public administration’ as their sector of 
activity  (54.8%),  7.1%  mentioned  ‘education’  and  5.6%  ‘other  community,  social  and 
personal services. Table A1.2 in Annex 1 to the external study gives a full overview of 
sectors of activity of the public consultation respondents.   

3.2.  Targeted consultations 

In  line  with  the  consultation  strategy  of  the  evaluation,  targeted  consultations  were 
conducted with a variety of key stakeholders at both national and EU levels, as detailed 
above. The full list of stakeholders participating in the targeted consultations are set out 
in Table A1.3 in Annex 1 to the external study.  

57 

 
 
3.2.1.  Targeted semi-structured interviews 
By  the  closing  date  for  conducting  interviews  in  January  2019,  interviews  with  all  32 
PES included in the consultation strategy had been conducted. This included interviews 
with  the  Network  Board  members  (usually  the  heads  of  PES)  and  interviews  with 
AFEPAs.  Some  21  interviews  with  other  stakeholders  were  also  conducted,  including 
interviews with the Commission PES Secretariat, the Council’s Employment Committee, 
the  World  Employment  Confederation,  EU-level  networks/institutions  (DG  EMPL,  DG 
REGIO, the European Lifelong Guidance Policy Network), WAPES, other international 
organisations (ILO, OECD, World Bank), and other stakeholders with knowledge of pre- 
and post-2014 activities, as well as the contractors that supported the implementation of 
Network activities (ICON Institut and ICF International). 

3.2.2.  Workshop 
PES stakeholders in EU Member States, Norway and Island (AFEPAs and some Board 
members) were further consulted during the AFEPA workshop on 11 October 2018. The 
evaluation workshop was organised during one of the bi-annual AFEPA meetings, which 
are organised under the aegis of the Network activities. A total of 35 AFEPAs (or their 
alternative  representative66)  took  part  in  the  evaluation  workshop  and  substantially 
contributed to increasing the evidence base. 

3.2.3.  Written questionnaires 
The  two  written  questionnaires  derived  from  the  consultation  topic  guide  framework 
primarily targeted two key stakeholder groups: ESF managing authorities and labour and 
social  affairs ministries.  By 14 January 2019, 16  contributions  had been collected from 
labour ministries67 and 7 contributions from ESF managing authorities68. 

3.2.4.  Case studies 
Interviews and focus groups were also conducted as part of the case studies carried out in 
five selected countries (Estonia, France, Italy, the Netherlands and Romania). In Estonia, 
interviews  were  conducted  with  representatives  from  the  Estonian  Unemployment 
Insurance Fund and the Ministry of Social Affairs. In the French case study, interviews 
were  held  with  a  range  of  different  representatives  of  Pôle  emploi,  the  French 
employment  service  and  the  ministry  of  labour.  Interviews  were  conducted  with  the 
Italian Agency for Labour Policies (ANPAL) at the national level, and a focus group was 
organised in which national and regional representatives of the Agency participated. This 
was also the case in Romania, where national and regional representatives of the National 
Agency for Employment participated in the consultation process. Several interviews were 
organised in the Netherlands as well. 

66 In some cases, more than one representative from each PES was present at the meeting (e.g. Ireland). 
67 Austria, Belgium (Wallonia), Croatia, Czech Republic, Denmark, Estonia, Germany, Hungary, Ireland, Latvia, Malta, the 

Netherlands, Romania, Slovakia, Spain and Sweden. 

68 Bulgaria, Cyprus, Czech Republic, Hungary, Latvia, Lithuania, Slovakia and Sweden. 

58 

 
                                                           
4.  METHODOLOGY FOR DATA PROCESSING 

4.1. Public consultation 

The responses to the public consultation were examined through a combination of closed 
and  open-ended  questions.  Closed  questions  provided  respondents  with  different 
categorical  answers  (on  the  challenges  faced  by  PES  and  the  most  important  areas  for 
improvement) or with a rating of the Network’s importance, strength of complementarity, 
success, efficiency and necessity. 

The  analysis  of  results  was  carried  out  using  both  quantitative  methods  (to  analyse  the 
frequencies  of  the  closed  answers)  and  qualitative  methods  (for  the  open  questions,  to 
analyse  complex  concepts  and  to  substantiate  and  interpret  the  quantitative  data  with 
relevant  insights).  In addition,  the responses were categorised using a range of relevant 
typologies (i.e. of respondents, of type of information provided, of country, etc.) 

Quantitative  analysis  includes:  (i)  analysis  of  frequency  distribution  for  each  of  the 
variables related to the closed questions; (ii) cross-tabulations between specific variables 
and characteristics of respondents69; and (iii) analysis of variability, calculating averages 
and  measuring  distances  from  the  average  to  allow  for  comparison.  Qualitative  data 
analysis was used to contextualise the analysis carried out on quantitative data, including 
inclusion of any quotes where relevant, and to identify major trends or changes. 

4.2. Targeted consultations 

The  write-ups  from  the  targeted  consultations  (interviews  and  written  questionnaires) 
were collected and exported into analytical grids, broken down by the different questions 
and by the respective evaluation criteria. The analytical grids were used to carry out an 
in-depth  analysis  of  the  data  collected  to  feed  into  the  relevant  sections  of  the  external 
study  and  the  analysis  carried  out  for  the  staff  working  document.  The  write-up  of  the 
AFEPA workshop was similarly used to feed into the evaluation. 

4.3. Case studies 

Information  gathered  from  the  case  studies  was  written  up  into  extensive  case  study 
templates. The data provided in the different subsections of the templates was used in the 
analysis carried out for the staff working document. 

5.  DISSEMINATION STRATEGY IMPLEMENTED FOR THE PUBLIC CONSULTATION 

The dissemination strategy aimed to ensure that the consultation: (i) reached stakeholders 
not  specifically  targeted  under  the  targeted  consultations,  i.e.  social  partners  at  EU  and 
local/regional  authorities,  organisations  or  networks 
national 
representing 
jobseekers,  members  of 
academic/research networks, citizens; and (ii) went beyond the close circle of those who 

interests  of  different  groups  of 

level,  employers, 

the 

69  The  data  was  disaggregated  by  the  capacity  in  which  the  respondent is answering  (individual  or  professional),  the  respondents’ 
involvement  with  the  PES  network  and  the  type  of  organisation  that  respondents  answering  in  their  professional  capacity  come 
from (government body, PES or NGO and others). 

59 

 
                                                           
are very familiar with the post-2014 Network set-up. Table 2 includes a detailed list of 
stakeholders targeted for the public consultation. 

Table 2: List of stakeholders targeted for the online public consultation 

Stakeholder 
categories 
Social partners  

National, 
local/regional 
stakeholders 

Other actors 

Research and other 
networks 

Stakeholders  Multipliers for survey distribution 

Business 
organisations 

Sectoral 
organisations 

Trade unions 

Chambers of 
commerce  
National and 
local/regional 
PES 

Other national/ 
local actors 

Private 
employment 
services 
Associations 
representing 
local actors 
Think tanks in 
relevant policy 
areas 

Academic 
networks 

NGOs 
representing 
jobseekers or 
specific groups 

International 
organisations  

  European Trade Union Confederation (ETUC) 
  European Trade Union Federations 
  Business Europe 
  European  Association  of  Craft,  Small,  and  Medium-Sized 

Enterprises (UEAPME) 

  European Centre of Enterprises with Public Participation and 

of Enterprises of General Interest (CEEP) 

  The  Association  of  European  Chambers  of  Commerce  and 

Industry (Eurochambers) 

  National PES (AFEPA) and regional/local offices 
  National ESF authorities 
  National labour ministry representatives 
  European Lifelong Guidance Policy Network (ELGPN) 
  EQF national coordination points 
  The  Council  of  European  Municipalities  and  Regions 

(CEMR) 

  World Employment Confederation-Europe 
  Eurocities  

  European Student Union 
  Social Platform 
  European Anti-Poverty Network (EAPN) 
  European Disability Forum (EDF) 
  European Youth Forum (YJF) 
  European Women’s Lobby 
  European Network against Racism (ENAR) 
  European  Confederation  of  cooperatives  active  in  industry 

and services (CECOP) 

  Civil Society Europe 
  Network  of  Experts  on  Social  aspects  of  Education  and 

Training (NESET) 

  Lifelong Learning Platform 
  Euroguidance 
 
ILO 
  Europass 
  EURES Network 

Citizens 

Citizen voice 
channels 

Social media 

  European citizens’ initiative (ECI) 
  European Citizen Action Service (ECAS) 

Twitter:  @EU_Social,  @EU_Commission,  @EURESjob, 
@EuropeanYouthEU, etc. 

60 

 
 
 
 
 
 
 
 
 
 
6.  Overview of the results of the consultations 

Relevance:  There  was  a  strong  consensus  among  consultees  about  the  overall  high 
degree of relevance of the objectives of the Decision, albeit to different degrees. These 
findings were confirmed by the interviews with PES representatives, other stakeholders 
(EU  and  international  level  stakeholders,  as  well  as  participants  with  knowledge  of  the 
cooperation  between  PES  before,  as  well  as  after  2014)  and  those  interviewed  for  case 
studies. All interviewees emphasised that the PES Network itself remains very relevant to 
needs  and  should  continue  beyond  2020.  The  majority  of  consultees,  as  well  as 
participants  in  the  AFEPA  workshop,  underlined  the  capacity  which  the  Network  has 
shown  to  adapt  to  changing  policy  priorities  (e.g.  the  refugee  crisis).  The  relevance  of 
Decision  objectives  was  strongly  confirmed  by  the  findings  from  the  written 
consultations  of  Labour  Ministries  and  ESF  Managing  Authorities.  The  results  of  the 
public  consultation  also  endorsed  the  Decision’s  relevance,  with  53%  of  public 
consultation respondents answering in their personal capacity and 59% answering in their 
professional  or  institutional  capacity  considering  that  PES  cooperation  at  EU  level  is 
‘very  important’  for  developing  national  PES  effectiveness  in  addressing  social  and 
labour market challenges. 

Effectiveness:  All  stakeholders  who  could  express  an  informed  opinion  (based  on 
frequent  participation  in  Network  activities)  reported  that,  overall,  Network  initiatives 
have been effective in supporting the achievement of the objectives. This is equally valid 
for  the  interviewed  PES  Network  members,  as  well  as  EU,  international  and  national 
level  stakeholders that have participated in  or have knowledge about  the  Network. The 
vast  majority  of  stakeholders  –  including  PES  representatives  who  were  interviewed, 
those who participated in the AFEPA workshop and in the case study research, EU and 
international  level  stakeholders,  Labour  Ministries,  and  ESF  Managing  Authorities  - 
emphasised  benchlearning  as  the  most  effective  PES  Network  initiative.  Mutual 
assistance    are  also  perceived  as  very  effective  by  the  PES  representatives  that  have 
participated  in  them  as  learners  or  tutors  alike.  Other  Network  initiatives  (such  as 
contributing to  the implementation  of policy initiatives, the promotion  of best  practices 
and the adoption of the annual work programmes) are also perceived as effective, but to 
different  degrees.  Another 
the  consulted  PES 
representatives  was  the  contribution  of  the  Network  to  the  implementation  of  relevant 
policy initiatives at national level (e.g. the Youth Guarantee). The vast majority of all the 
consulted  PES  representatives  and  other  stakeholders  indicated  that  the  Network  has 
contributed to supporting the most vulnerable groups with high unemployment rates and 
the  integration  of  persons  excluded  from  the  labour  market.  The  majority  of  responses 
received  from  the  national  Labour  Ministries  (12  out  of  16  ministries)  indicated  that 
Network has been successful or very successful in supporting the most vulnerable groups 
with high unemployment rates. However, only a minority (5 out of 16) indicated that the 
Network has been successful or very successful in supporting the integration of persons 
excluded  from  the  labour  market.  Stakeholders  generally  considered  that  the  Network 
activities  have  contributed  less,  although  to  some  degree,  to  supporting  decent  and 

initiative  often  emphasised  by 

61 

 
sustainable  work,  increased  geographical  and  occupational  mobility,  the  evaluation  and 
assessment of ALMPs and their implementation.  

The  public  consultation  results  show  that  one  third  of  the  respondents  who  have  been 
involved  in  Network  consider  that  it  has  been  very  successful  in  supporting  enhanced 
cooperation.  Respondents  who  have  not  been  directly  involved  with  the  Network 
consider  that  it  has  somewhat  successful  in  this  area  (25%),  while  19%  consider  it 
somewhat  unsuccessful,  and  29%  don’t  know..  Most  of  the  respondents  answering  in 
their  professional  or  institutional  capacity  reported  that  the  Network  is  somewhat 
successful in supporting  enhanced cooperation (38%), while 17% of them reported that 
they  do  not  know.  40%  of  respondents  answering  in  their  individual  capacity  reported 
that they do not know to what extent the Network has been successful in this area.  

Efficiency: The interviews and written consultations, as well as evidence from the case 
studies, confirmed that the PES Network is considered both to be efficient and a good use 
of resources by the vast majority of actors consulted both at EU level and in national PES 
(almost  100%  of  those  who  expressed  an  opinion).  This  view  was  also  held  by 
representatives  of  international  organisations  consulted.  Given  the  relatively  modest 
resources  (compared  to  the  scale  of  PES  activity)  which  most  consultees  consider  are 
devoted to the Network and its activities at EU level, the vast majority of consultees felt 
that the benefits of the Network are very important. 

Coherence: Most stakeholders consider that there is a good degree of coherence between 
the Decision and the EU policy framework. Most stakeholders also reported that there is 
some  cooperation  between  the  Network  and  other  relevant  labour  market  stakeholders, 
but  that  there  is  scope  for  improvement,  for  example  by  developing  stronger 
relationships:  at  EU  level  with  the  EU  social  partners,  EU  level  NGOs,  EMCO  and 
EURES;  at  national  level  with  the  social  partners,  NGOs  supporting  jobseekers, 
Managing Authorities of EU Funds (in particular the ESF) and training providers; and at 
international  level  with  the  OECD,  ILO,  WAPES  and  the  World  Employment 
Confederation. 

In  the  public  consultation,  almost  half  (45%)  of  the  respondents  answering  in  their 
professional  or  institutional  capacity  reported  that  a  ‘strong  complementarity’  between 
the objectives of the  Network and other EU and/or national policies aimed at improving 
PES, while 15% reported a very strong complementarity. Almost a third of respondents 
(32%) answering in their personal capacity reported a strong complementarity, while 8% 
reported a very strong complementarity.  25% of respondents answering in their personal 
capacity  reported  that  they  do  not  know  whether  there  is  complementarity  between  the 
Network  objectives  and  other  EU  and/or  national  policies,  compared  to  15%  of  those 
answering in their professional capacity. 

The 
targeted  consultations  (interviews,  case  studies  and  workshop)  with  PES 
representatives  show  that  the  vast  majority  consider  that  there  is  a  high  degree  of 
coherence  of  the  Decision  with  the  European  policy  framework.  In  particular,  a  large 

62 

 
majority of PES consultees emphasised that the Decision reflects the priorities set out in 
the  Europe  2020  strategy.  Complementarity  was  also  emphasised  in  particular  with  the 
Youth  Guarantee,  the  Youth  Employment  Initiative,  the  Council  Recommendation 
2016/C 67/01 on the integration of the long-term unemployed into the labour market, and 
ESF-funded programmes. Synergies were also highlighted in relation to the mandate of 
other EU institutional actors, such as the EURES Network and EMCO.  

Added  value:  PES  representatives,  Labour  Ministries  and  ESF  Managing  Authorities, 
external  stakeholders  at  EU  and  international  level  have  overwhelmingly  reported  that 
from their perspective, the learning process and cooperation would not have occurred to 
the same extent in the absence of the Decision. According to all consultees, the Network 
has  an  added  value  in  driving  improvements  in  PES  organisation  and  performance, 
increasing  effectiveness  and  stimulating  cooperation.  The  consultations  strongly 
emphasised  the  crucial  nature  of  the  commitment  and  support  provided  at  EU  level 
(European  Commission)  to  ensuring  the  effective  implementation  of  the  Decision,  in 
terms of both financial and human resources invested in the running of the Network.  It 
was reported that the absence of a formal commitment to continue to provide resources 
for  the  Network  to  continue  to  function  would  have  a  negative  impact  in  reducing  the 
scope,  extent  and  quality  of  PES  cooperation  and  its  ability  to  implement  strategic  EU 
employment goals. 

The  importance  of  the  EU  support  for  PES  cooperation  is  reinforced  by  the  public 
consultation  results.  More  than  for  any  other  question  in  the  public  consultation,  there 
was  a  strong  consensus  amongst  respondents  about  the  necessity  of  EU  action  for 
strengthening  the  cooperation  between  PES.  A  large  majority  (80%)  of  respondents 
answering  in  their  professional  or  institutional  capacity  reported  that  EU  action  for 
strengthening  the  cooperation  between  PES  is  ‘very  necessary’  (62%)  or  ‘somewhat 
necessary’  (20%);  similarly,  82%  answering  in  an  individual  capacity  felt  that  it  was 
‘very’  (57%)  or  ‘somewhat’  (25%)  necessary.  The  share  of  those  considering  that  EU 
action  is  ‘very  necessary’  was  highest  among  respondents  from  PES  (67%)  and 
NGOs/other stakeholders (65%). 

63 

 
Annex 3: More detailed intervention logic of the evaluation 

64 

Needs Strengthen PES service capacity, effectiveness and efficiency. Increase PES potential to contribute to Europe 2020 objectives, including support for lowering unemployment (including for the most vulnerable social groups); decent and sustainable work; better functioning of EU labour markets; identification of skills shortages and better matching; better integration of labour markets; increased voluntary mobility to meet specific labour market needs; integration of persons excluded from the labour market; evaluation and assessment of active labour market initiatives and their implementation. External (exogenous) factors Labour market functioning and social situation at MS level Structure and funding of PES at MS levelObjectivesStrengthen cooperation between PES in order to: Improve PES performance Contribute to the implementation of policy initiatives in the field of employmentInputs EC contribution of €3.5m/yr (2014-2020) Member State contribution Staffing of PES Network Secretariat Liaison persons at national levelActivities Benchmarking: self-assessments, site visits, external assessments Mutual learning events PES Network governance meetings Mutual assistance Supporting policy implementation Additional activities e.g. monitoring of pan-EU initiatives, ad-hoc studies and research, etc.Outputs Benchlearning reports and recommendations, PES Capacity Reports, PES Implementation of the Youth Guarantee, etc. Studies, toolkits, PES practices, PES Knowledge centre. PES Network meeting minutes, decisions and position papersResults Increased cooperation and exchange amongst PES Follow up and implementation of benchlearning recommendations Changes undertaken by PES (organisational process and service delivery) Changes in ESF programming and/or fundingImpacts Strengthened and modernised PES Better performing PES (efficiency and effectiveness) Better support to the unemployed and inactive not registered with PES Better functioning labour markets Better matching of skills and integration of labour markets Increased mobility (geographical and occupational) Lower unemployment, including for vulnerable groups Contributing to meeting the EU2020 targetsRelevanceCoherenceEffectivenessEfficiencyEU added value