COMMISSION STAFF WORKING DOCUMENT EVALUATION Evaluation of the ESF and YEI Support to Youth Employment
Date: 2020-09-24
Institutions: Directorate-General for Employment, Social Affairs and Inclusion, European Commission
Subjects: EU employment policy, European Social Fund, integration into employment, job access, young person, young worker, youth employment, youth unemployment

EN 

  EN 

   EUROPEAN COMMISSION  Brussels, 24.9.2020  SWD(2020) 216 final  COMMISSION STAFF WORKING DOCUMENT  EVALUATION  Evaluation of the ESF and YEI Support to Youth Employment {SWD(2020) 217 final}  
 
 
Table of contents 

1. 

2. 

INTRODUCTION ................................................................................................................................ 4 

BACKGROUND DESCRIPTION AND PROGRAMME OBJECTIVES ........................................... 6 

2.1.  BASELINE AND POINTS OF COMPARISON .................................................................................. 9 

3. 

IMPLEMENTATION / STATE OF PLAY ........................................................................................ 13 

3.1.  PROGRAMMING AND BUDGET .................................................................................................... 13 

3.2.  TYPE OF OPERATIONS FUNDED .................................................................................................. 15 

3.3.  FINANCIAL AND OPERATIONAL IMPLEMENTATION ............................................................ 17 

4.  METHOD............................................................................................................................................ 18 

4.1.  SHORT DESCRIPTION OF THE METHODOLOGY ...................................................................... 18 

4.2.  LIMITATIONS AND ROBUSTNESS OF FINDINGS ..................................................................... 19 

5.  ANALYSIS AND ANSWERS TO THE EVALUATION QUESTIONS .......................................... 22 

5.1.  EFFECTIVENESS:  HOW  EFFECTIVE  IS  THE  YEI,  AND  OTHER  ESF-FUNDED 

YOUTH EMPLOYMENT OPERATIONS, IN ACHIEVING THEIR OBJECTIVES? .................... 22 

5.2.  EFFICIENCY:  HOW  EFFICIENT  HAS  THE  YEI,  AND  OTHER  YOUTH 
IN  THE 

EMPLOYMENT  OPERATIONS  FUNDED  BY  THE  ESF,  BEEN 
ACHIEVEMENT OF THEIR OBJECTIVES? ................................................................................... 36 

5.3.  RELEVANCE:  HOW  RELEVANT  IS  THE  YEI  AND  THE  OTHER  ESF-FUNDED 

YOUTH EMPLOYMENT OPERATIONS? ....................................................................................... 43 

5.4.  COHERENCE: HOW COHERENT ARE YEI AND THE OTHER ESF-FUNDED YOUTH 
EMPLOYMENT  OPERATIONS  AMONG  THEMSELVES,  AND  WITH  OTHER 
ACTIONS IN THE SAME FIELD? ................................................................................................... 48 

5.5.  EU ADDED VALUE: WHAT IS THE EU ADDED VALUE OF THE YEI AND OTHER 

ESF-FUNDED YOUTH EMPLOYMENT OPERATIONS? ............................................................. 50 

5.6.  SUSTAINABILITY:  HOW  SUSTAINABLE  ARE  THE  YEI  AND  THE  OTHER  ESF-

FUNDED YOUTH EMPLOYMENT OPERATIONS? ..................................................................... 54 

6. 

CONCLUSIONS AND LESSONS LEARNED ................................................................................. 57 

6.1.  OVERALL CONCLUSIONS ............................................................................................................. 57 

6.2.  LESSONS LEARNED ........................................................................................................................ 60 

ANNEX 1: PROCEDURAL INFORMATION ............................................................................................ 65 

ANNEX 2: STAKEHOLDER CONSULTATION ....................................................................................... 67 

ANNEX 3: METHODS AND ANALYTICAL MODELS .......................................................................... 79 

1 

 
 
 
 
 
Figures, Tables and Boxes  

Figure 5 

Figure 6 
Figure 7 

Figure 8 
Figure 9 

Figure 1  Youth employment intervention logic ........................................................................................ 8 
Figure 2  Youth unemployment rate (15-24) (% of active population), 2014 ............................................ 9 
The  rate  of  young  people  (aged  15-24)  not  in  employment,  education  or  training,  by  labour 
Figure 3 
market status (% of population), 2014 ...................................................................................... 10 
Figure 4  Young  people  in  the  EU28  (aged  18-24)  having  left  education  early,  by  country  and  labour 
status, in 2014 and 2018 ............................................................................................................ 10 
Change in unemployment rates for young people aged 15-24 by level of education, 2014-2018 
(in percentage points) ................................................................................................................ 12 
Share of costs eligible for ESF funding by type of operation ................................................... 16 
Implementation  rates  of  YEI/ESF  funding  by  Member  State  (expenditure  declared  to  the 
managing authority vs funding allocated) ................................................................................. 17 
Progress towards the milestones set for 2018 – YEI ................................................................. 23 
Share  of  NEETs  (aged  15-24)  receiving  ESF/YEI  funding  as  a  proportion  of  the  NEET 
population in 2014 by cluster of regions (%) ............................................................................ 27 
Figure 10  How successful were the youth employment operations funded by the European Social Fund 
and the Youth Employment Initiative in providing support to the following target groups? .... 28 
Figure 11  Share of job offers with permanent contracts according to YEI evaluations ............................ 30 
Figure 12  To what extent do you agree/disagree that the following measures have been actually effective 
in  helping  young  people  enter  quality  and  sustainable  employment?  (population:  399 
organisations) ............................................................................................................................ 32 
Figure 13  How would you qualify the following administrative arrangements for the implementation of 
youth employment operations under the ESF and YEI? ........................................................... 41 
Figure 14  Overview of budget allocation to clusters of regions ................................................................ 47 
Figure 15  To  what  extent  are  the  YEI  and  ESF  coherent  with  other  youth  and  youth  employment  EU 
national/regional schemes? (n=399) ......................................................................................... 49 
In  your  opinion,  how  useful  are  the  following  actions  in  helping  young  people  find  a  job, 
improve their professional skills or enter the labour market? (Group A1, A2 and B, n=976) .. 71 
Figure 17  The  support  you  received  helped  you  in…  (Group  A1  and  A2,  n=431,  multiple  answers 
allowed) .................................................................................................................................... 72 
Figure 18  To what extent do  you agree or disagree that the  following activities implemented under the 
Youth Employment Initiative were cost-effective? (Group C, n=206) ..................................... 75 
Figure 19  To  what  extent  are  the  YEI  and  ESF  coherent  with  other  youth  and  youth  employment  EU 
national/regional schemes? (Group C and D, n=399) ............................................................... 76 
Figure 20  Has  the  EU  provided  support  that  goes  beyond  what  is  addressed  by  national  or  regional 
programmes or support schemes aimed at youth employment? (Group C and D, n=282) ........ 76 

Figure 16 

Table 1 
Table 2 
Table 3 

Table 4 
Table 5 
Table 6 
Table 7 

Box 1 
Box 2 
Box 3 
Box 4 
Box 5 
Box 6 

Funding for Youth Employment ESF + YEI, including amendments until 2018 ..................... 13 
Comparative table of unit costs ................................................................................................. 38 
Expenditure per entry: expenditure  minimum-rate threshold of 75.1%  - breakdown by region 
and type of intervention ............................................................................................................ 39 
Use of ESF and YEI funds in implementing national Youth Guarantee schemes, 2016-2019 . 53 
Employment rates by type of operation, upon completion and after 6 months ......................... 55 
Labour market status of individuals who took part in ESF/YEI initiatives ............................... 73 
Number of interviewees by Member State and type of organisation ........................................ 77 

Cooperation and partnerships in Germany and Portugal ........................................................... 35 
Special support for hard-to-reach young people in Belgium ..................................................... 44 
Common standards and tools developed in YEI programmes .................................................. 51 
‘From training to employment’ – the Voluntary Labour Corps project in Poland .................... 51 
Common standards and tools developed in the YEI programme .............................................. 52 
The impact of training on the employment rate in Italy ............................................................ 56 

2 

 
 
 
Term or acronym 

Meaning or definition 

Glossary 

ALMP 
AIR 

Beneficiary 

CIE 
CPR 
CSR  
DG EMPL 
DG REGIO 
EAFRD 
EMFF 
EQ 
ERDF 
ESF 
FEAD 
GDP 
ISCED 
JRC 
MS 
IP 
NEET 
OP 

Operation 

Participant 

PES 
SFC 
VET 
YE 
YEI 
YG 

Active Labour Market Policies 
Annual Implementation Report 
Beneficiary  means  a  public  or  private  body  responsible  for 
initiating or both initiating and implementing operations 
Counterfactual impact evaluations 
Common Provisions Regulation 
Country-specific recommendations 
Directorate-General Employment, Social Affairs & Inclusion 
Directorate-General Regional and Urban Policy 
European Agricultural Fund for Rural Development 
European Maritime and Fisheries Fund 
Evaluation question 
European Regional Development Fund 
European Social Fund 
Fund for European Aid to the Most Deprived 
Gross domestic product 
International Standard Classification of Education 
Joint Research Centre 
Member State 
Investment Priority 
Person not in employment, education or training 
Operational Programme 
Operation means a project, contract, action or group of projects 
selected  by  the  managing  authorities  of  the  programmes 
concerned, or under their responsibility, that contributes to the 
objectives of a priority or priorities 
A  persons  benefiting  directly  from  an  ESF  intervention  who 
can  be  identified  and  asked  for  their  characteristics,  and  for 
whom specific expenditure is earmarked 
Public employment service 
System for Fund Management in the European Union 
Vocational education and training 
Youth employment 
Youth Employment Initiative 
Youth Guarantee 

3 

 
 
 
 
1. 

INTRODUCTION 

This evaluation takes stock of the activities and results achieved by providing EU support 
for  youth  employment  under  the  Youth  Employment  Initiative  and  the  European 
Social Fund. It builds on the evaluations carried out by the Member States of the Youth 
Employment Initiative, delivered by December 20181. It also assesses the extent to which 
the  Youth  Employment  Initiative  contributed  to  meeting  the  objectives  of  the  Youth 
Guarantee. 

The evaluation assessed the effectiveness, efficiency, relevance, EU added value and the 
sustainability  of  youth  employment  activities  funded  by  Youth  Employment  Initiative 
and other actions under the European Social Fund over the period 2014-2018. It covers 
all Member States, particularly those who funded operations under investment priority 
8.ii of the European Social Fund2 and the dedicated investment priority for programming 
the Youth Employment Initiative resources. It also looked at whether the initiatives were 
complementary  and  coherent  with  other  initiatives  in  this  policy  area  over  the  same 
period  (training,  vocational  education,  lifelong  learning,  skills,  mobility  of  young 
workers).  

The conclusions of the evaluation are relevant to the ongoing work in the final stages of 
the Youth Employment Initiative and the 2014-2020 European Social Fund investments 
in  youth  employment.  The  findings  can  also  feed  into  the  process  to  programme 
additional resources under the REACT-EU instrument3 for cohesion policy with the aim 
of  crafting  an  effective  response  to  the  detrimental  impacts  of  the  COVID-19  crisis  on 
employment and on society in general. In particular, support for youth employment and 
protecting  vulnerable  groups  such  as  young  people  not  in  employment,  education  or 
training  have  been  identified  as  some  of  the  key  areas  for  ESF  support  with  additional 
resources allocated for 2020-2022. Member States will have flexibility in organising the 
recovery measures for young people, such as to provide incentives for job hiring.  

1  Article 19(6)  of  Regulation  (EU)  No 1304/2013  of  the  European  Parliament  and  of  the  Council  of 
17 December  2013  on  the  European  Social  Fund  and  repealing  Council  Regulation  (EC)  No 
1081/2006. 

2 ‘Sustainable integration into the labour market of young people, in particular those not in employment, 
education  or  training,  including  young  people  at  risk  of  social  exclusion  and  young  people  from 
marginalised  communities,  including  through  the  implementation  of  the  Youth  Guarantee.’  Article 
3(1) of Regulation (EU) No 1304/2013 of the European Parliament and of the Council of 17 December 
2013 on the European Social Fund and repealing Council Regulation (EC) No 1081/2006. 

3 https://ec.europa.eu/info/sites/info/files/com_2020_451_act_v8.pdf. 

4 

 
 
 
 
                                                 
The results of the evaluation should feed into the next programming period, in the form 
of lessons learned (notably on cost effectiveness, outreach and main target groups). This 
will underpin the negotiation of operational programmes under the European Social Fund 
Plus and the design of operations by the Member States targeting youth employment. The 
findings  can  provide  guidance  for  EU  funding  on  how  to  further  reform  youth 
employment policies and vocational education and training and apprenticeships systems, 
co-funded by the European Social Fund+ and by the Resilience and Recovery Facility as 
part of the reinforced Youth Guarantee and the upcoming policy proposals on vocational 
education  and  training.  The  results  will  also  feed  into  the  Commission’s  ex-post 
evaluation of the European Social Fund. 

5 

 
 
 
 
 
 
2.  BACKGROUND DESCRIPTION AND PROGRAMME OBJECTIVES 

Following the financial and economic crisis of the late 2000s, the employment conditions 
for  young  people  in  the  EU  deteriorated  significantly.  Following  the  Commission’s 
recommendations, the Council issued in April 2013 a Recommendation on establishing a 
Youth Guarantee4. The Youth Guarantee is a political commitment made by the Member 
States  to  give  every  person  under  the  age  of  25  (or  under  30,  if  the  Member  State  so 
decides)5 a good-quality offer of employment, continued education, an apprenticeship or 
a traineeship within four months of becoming unemployed or leaving formal education. 

The Youth Employment Initiative is one of the main EU funding resources to support 
the Youth Guarantee schemes. However, the aim and scope of the programme are more 
focused than Youth Guarantee policy. The Initiative provides direct support exclusively 
to  young people  who are not  in  education, employment or training, including the long-
term  unemployed  or  those  not  registered  as  jobseekers  living  in  regions6  where  youth 
unemployment was over 25% in 2012. It thus aims to ensure that young people in parts 
of Europe with the most acute employment challenges can receive targeted support.   

The Youth Employment Initiative was integrated into the European Social Fund (chapter 
IV of Regulation (EU) No 1304/2013 of the European Parliament and of the Council on 
the  European  Social  Fund  and  repealing  Council  Regulation  (EC)  No 1081/2006,  the 
‘ESF Regulation’), in order to provide support for action under the investment priority in 
Article 3(1)(a)(ii) of the ESF Regulation (the “IP8.ii”):   

‘Sustainable  integration  into  the  labour  market  of  young  people,  in  particular 
those not in employment, education or training, including young people at risk of 
social  exclusion  and  young  people  from  marginalised  communities,  including 
through the implementation of the Youth Guarantee.’  

Action  under  the  Youth  Employment  Initiative  often  takes  the  form  of  pathways  or 
packages of measures with the long-term objective of integrating young people into the 
active labour market. Under the Initiative, each individual participant receives an offer of 
employment and/or continued education and training, a traineeship or an apprenticeship, 

4 https://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:C:2013:120:0001:0006:EN:PDF 

5  Currently,  the  Youth  Guarantee  is  limited  to  those  aged  15-24  only  in  Belgium,  Denmark,  Ireland, 
France,  Luxembourg,  Hungary,  the  Netherlands,  Austria,  Romania  and  Sweden.  In  the  other  17 
Member States, it is open to the 15-29 cohort. 

6  NUTS  level  2  regions.  The  NUTS  classification  (Nomenclature  of  territorial  units  for  statistics)  is  a 
hierarchical system for dividing up the economic territory of the EU for the purpose of, among others, 
socio-economic analyses of the regions where NUTS 2 are basic regions for the application of regional 
policies. 

6 

 
 
 
                                                 
and therefore contributes to the implementation of national Youth Guarantee schemes. A 
single  Youth  Employment  Initiative  operation  can  cover  both  the  offer  itself  and  the 
range  of  support  services  (such  as  job  search  assistance,  counselling,  psychological 
support)  leading  to  the  job/education/training  offer.  However,  the  Initiative  cannot 
directly fund reforms of systems and structures carried out under the Youth Guarantee.  

The  European  Social  Fund  provides  additional  assistance  to  youth  employment  policy, 
and has a broader scope than the YEI. It includes support for system-related activities and 
support for regions not eligible for Youth Employment Initiative funding. 

7 

 
 
 
 
 
Figure 1 

Youth employment intervention logic 

8 

 
 
 
 
2.1.  BASELINE AND POINTS OF COMPARISON 

The  situation  of  youth  employment  at  the  start  of  the  programming  period  for 
EU28 

The economic and financial crisis of the late 2000s had severe effects on young people.  
Young people trying to make the transition from education or training into work faced a 
market with fewer jobs and unparalleled levels of competition, not only from their peers 
but also from adult workers with extensive work experience who had recently been laid 
off.  As  a  result,  they  increasingly  found  themselves  unable  to  get  a  job  without  prior 
experience and yet little chance of getting a job to gain that initial experience.  

At  the  beginning  of  the  programming  period,  the  unemployment  rate  of  young  people 
was about 30% higher than it was for older adults in 20147.  

In 2014, on average across the EU, more than one in five young people was unemployed, 
with wide variations between the Member States. 

Figure 2 

Youth unemployment rate (15-24) (% of active population), 2014 

Source: Eurostat, Labour Force Survey (lfsa_pganws), data extracted on 21 June 2018. 

The unemployment rate was particularly high for low-skilled young people, where one in 
three of the population did not have a job, twice the unemployment rate of high-skilled 
young people8.  

7 The unemployment ratio is the youth unemployment ratio divided by the adult unemployment ratio. Thus, 
all values>1 imply a disadvantage for young people. Note that we are using ratios and not rates (ratio 
is  the  share  of  unemployed  among  all  those  aged  15-24;  unemployment  rates  refer  to  the  share  of 
unemployed among the active population of the same age). 

8 Source: Eurostat, Labour Force Survey (yth_empl_090) 

9 

22.224.523.223.815.812.67.714.923.452.453.224.245.542.736.019.719.222.320.411.712.710.323.934.824.020.229.720.522.917.00102030405060 
 
 
 
                                                 
In  2014,  12.5%  of  young  people  across  the  EU  aged  15-24  were  not  in  employment, 
education or training, with an equal share of inactive and unemployed, up from 10.9% in 
2008. Again, there were huge variations between Member States (Figure 3). 

Figure 3  The rate of young people (aged 15-24) not in employment, education or 

training, by labour market status (% of population), 2014 

Source: Eurostat, Labour Force Survey (yth_empl_150), date of extraction 21 June 2019. 

In  2014,  more  than  one  in  ten  (11.2%)  young  people  aged  18-24  left  school  early, 
meaning  that  they  had  completed  at  most  lower  secondary  education  and  were  not 
currently  enrolled  in  further  education  or  training  (Figure  4).  The  rate  of  early  leavers 
from  education  and  training  leaving  varied  widely  between  countries.  At  EU  level,  six 
out of ten (59.8%) early leavers from education and training were not employed. 

Figure 4  Young  people  in  the  EU28  (aged  18-24)  having  left  education  early,  by 

country and labour status, in 2014 and 2018 

Source: Eurostat, Labour Force Survey (edat_lfse_14), data extracted on 1 July 2019. 

10 

12.512.312.020.28.15.86.411.715.319.117.111.219.322.117.012.09.96.313.610.35.57.712.012.317.09.412.810.27.211.905101520250510152025EU28EUavBEBGCZDKDEEEIEELESFRHRITCYLVLTLUHUMTNLATPLPTROSISKFISEUKNEET rate unemployedNEET rate inactiveTotal NEET rate05101520252014201820142018201420182014201820142018201420182014201820142018201420182014201820142018201420182014201820142018201420182014201820142018201420182014201820142018201420182014201820142018201420182014201820142018201420182014201820142018EU28BEBGCZDKDEEEIEELESFRHRITCYLVLTLUHUMTNLATPLPTROSISKFISEUKNot employedEmployed 
 
 
 
 
 
 
At  EU  level,  there  was  only  a  small  gender  gap  in  the  rate  of  young  people  not  in 
employment,  education  or  training  in  2014  (12.3%  for  men  aged  15-24  v  12.7%  for 
women), though some countries recorded more substantial differences. The gender gap is 
wider in  the labour market  status  of  these  young  people.  At EU level,  out  of all  young 
people not in employment, education or training, 60.2% of young men were unemployed 
and 39.8% were inactive, compared to 43.3% of young women unemployed and 56.7% 
inactive9.  

In 2014, almost a third (31.4%) of young people aged 15-24 lived in households at risk of 
poverty or social exclusion. Poverty was therefore more prevalent amongst this age group 
than in the population as a whole (24.4%). In 2014, young people aged 15-24 were more 
likely (compared to the general population) to be at risk of poverty or social exclusion in 
all EU countries but Estonia10. 

It was especially hard for young people with a migrant background (i.e. people born in a 
country  other than the one in  which they seek work)  to  find  employment,  compared to 
nationals. In 2014, the difference in unemployment rates at EU28 level between the two 
groups was 7.6 percentage points on average. 

Labour market trends for young people since 2014 

Between 2014 and 2018, as the economic situation improved, youth unemployment rates 
fell  in  all  Member  States  and  by  an  average  of  7  percentage  points  at  EU28  level. 
However, not all Member States saw the same degree of improvement.  

There  are  also  differences  in  the trend  of  unemployment rate based on education level. 
Although the overall improvement in the employment rate was higher for the low-skilled 
(primary/lower secondary education) than for medium (upper secondary education) - and 
highly  (tertiary  education)    educated  young  people,  the  unemployment  rate  for  low-
skilled  people  (21.9%)  remains  double  the  rate  of  unemployed  highly  educated  young 
people (11.0%).  

9 Source: Eurostat, Labour Force Survey (yth_empl_150) 

10 Source: Eurostat, EU-SILC (ilc_peps01) 

11 

 
 
 
 
 
 
                                                 
Figure 5  Change in unemployment rates for young people aged 15-24 by level  of 

education, 2014-2018 (in percentage points) 

Source: Eurostat, Labour Force Survey (yth_empl_100), data extracted on 21 June 2018. 

Over that period, the EU28 average rate of young people not in employment, education 
or  training  fell  from  12.5%  in  2014  to  10.5%  in  2018.  More  importantly,  it  fell  in  all 
countries  but  Germany  and  Denmark,  where  it  had  only  marginally  increased.  The 
improvement derives entirely from  the fall  in  the number of unemployed  young people 
(down 3.0 percentage points) as the number of inactive  young people slightly increased 
(up 1.2 percentage points). The composition of the target population is thus shifting, with 
a higher share of inactive young people (58% in 2018)11. 

By  2018,  the  rate  of  early  school  leaving  fell  to  10.6%  while  the  proportion  of  early 
school leavers that were not in employment fell from six in ten to five in ten (52.8%).  

By 2018, the rate of young people at risk of poverty or social exclusion had fallen from 
31.5%  in  2014  to  28.1%,  though  it  remained  higher  than  the  rate  for  the  overall 
population (21.8%).  

For  young  migrants  (15-24  years),  the  unemployment  rate  fell  from  29.2%  in  2014  to 
20.9% in 2018.   

Note that, given the inevitable time lags associated with the quantitative cross-EU data, 
all figures in this report predate the COVID-19 crisis. 

11 Source: Eurostat, Labour Force Survey (yth_empl_150) 

12 

-40-30-20-10010-40-30-20-10010Low (ISCED 0-2)Medium (ISCED 3-4)High (ISCED 5-8)Total (ISCED 0-8) 
 
 
 
 
 
 
                                                 
3. 

IMPLEMENTATION / STATE OF PLAY 

3.1. 

PROGRAMMING AND BUDGET 

Originally, the resources for the Youth Employment Initiative consisted of:  

(1) a dedicated budget line (specific allocation) of €3.2 billion;  
(2) a matching contribution from the European Social Fund of €3.2 billion;  
(3) national co-financing for the European Social Fund.  

The  specific  allocation  for  the  Youth  Employment  Initiative  is  not  complemented  with 
national  co-financing12.  At  the  start  of  the  programming  period,  the  total  EU  budget 
allocated  to  the  Initiative  (under  both  the  Youth  Employment  Initiative  (YEI)    and  the 
European  Social  Fund  (ESF))  was  €6.4  billion  (€7.67  billion  including  national  co-
financing of the European Social Fund).  

Given  the  persistently  high  level  of  youth  unemployment,  in  June  2017,  the  European 
Parliament and the Council agreed to increase funding for the Initiative by another €1.2 
billion,  matched  by  an  equivalent  amount  of  ESF  funding  (€2.4  billion  in  total).  The 
increase in ESF funding was also topped up by eligible funding from Member State own 
resources.  All  in  all  (combining  EU  funding  on  the  dedicated  YEI  budget  line,  the 
matching  funding  under  ESF  and  national  co-financing),  the  total  budget  for  YEI  was 
€10.4 billion, as shown in Table 1 below. The total ESF budget allocated to Investment 
Priority 8.ii (non-YEI) is €8.2 billion, bringing the total investment in youth employment 
operations to around €18.6 billion. 

Table 1  Funding  for  Youth  Employment  ESF  +  YEI,  including  amendments  until 

2018 

Member 
State 

AT 
BE 
BG 
CY 
CZ 
DE 
DK 
EE 
ES 
FI 
FR 

ESF — IP 8ii13 
EU 
amount 
(in 
€000) 

Total 
(in 
€000) 

YEI14 

EU 
amount 
(in €000) 

Total 
(in €000) 

Total 

EU 
amount 
(in €000) 

Total 
(in €000) 

 —    

 —    

 —    

 —    

 —    

 —    

 61,539  
 31,799  
 6,798  

 133,209  
 37,357  
 7,998  

 —    

 —    

 467,029  

 827,733  

 —    
 —    

 —    
 —    

 125,788  
 110,377  
 36,274  
 27,200  

 188,682  
 120,117  
 39,474  
 29,600  

 —    
 —    
 —    

 —    
 —    
 —    

 187,327  
 142,177  
 43,072  
 27,200  
 467,029  

 321,891  
 157,474  
 47,472  
 29,600  
 827,733  

 —    
 —    

 —    
 —    

 420,217  

 589,236  

 2,723,322    2,963,615   3,143,538    3,552,850  

 —    

 —    

 —    

 —    

 —    

 —    

 195,976  

 249,915  

 944,660    1,117,509   1,140,635    1,367,424  

12 Article 22(3) of the ESF Regulation. 
13 Excludes ESF allocations to the Youth Employment Initiative. 
14 Includes ESF allocations to the Youth Employment Initiative. 

13 

 
 
 
                                                 
Member 
State 

ESF — IP 8ii13 
EU 
amount 
(in 
€000) 

Total 
(in 
€000) 

YEI14 

EU 
amount 
(in €000) 

Total 
(in €000) 

Total 

EU 
amount 
(in €000) 

Total 
(in €000) 

GR 
HR 
HU 
IE 

IT 

LT 
LU 
LV 
MT 
NL 

PL 

PT 
RO 
SE 
SI 
SK 
UK 

EU 

 500,842  
 202,590  
 99,531  
 136,291  

 574,249  
 262,277  
 707,113  
 204,436  
 1,821,065    2,288,069    3,114,159    4,555,957  

 500,842  
 28,130  
 602,598  
 136,291  

 574,249  
 220,466  
 108,313  
 204,436  

 —    

 —    

 35,540  
 503,068  

 41,812  
 598,801  

 —    

 —    

1,293,09

2,267,88

5  
 17,453  
 6,819  

8  
 20,533  
 13,638  

 63,565  

 69,174  

 —    

 —    

 —    

 —    

 58,021  

 63,141  

 4,800  

 6,000  

 —    

 —    

 —    
 —    

 —    
 —    

 81,018  
 6,819  
 58,021  
 4,800  

 89,707  
 13,638  
 63,141  
 6,000  

 —    

 —    

1,256,02

1,488,18

8  
 —    

2  
 —    

 421,124  
 191,150  
 73,000  

 496,769  
 382,301  
 91,250  

 —    

 —    

 549,841  

 984,149  

5,535,27

8,236,76

4  

8  

 537,635  

 585,074    1,793,663    2,073,256  

 446,720  
 302,237  
 88,326  
 18,423  
 206,715  
 397,265  

 486,136  
 328,905  
 132,489  
 20,726  
 228,275  
 578,361  

 486,136  
 446,720  
 825,674  
 723,361  
 514,790  
 279,477  
 111,976  
 91,423  
 206,715  
 228,275  
 947,105    1,562,510  

 8,846,846   10,346,81

14,382,12

18,583,57

0  

0  

9  

Source: SFC2014, based on data on the operational programmes reported by the Member States in 
AIR2018 (data extracted on 6 September 2019) 

Youth employment operations under both funds are programmed and implemented under 
the shared management system with the Member States. Member States identify relevant 
operations to support the objectives set in the operational programmes, in agreement with 
the European Commission. Therefore the range of measures supported by the European 
Social  Fund  and  the  Youth  Employment  Initiative  varies  significantly  across  Member 
States (also in view of the funding available to each Member State). 

There are 37 operational programmes running in 20 Member States including YEI (out of 
a total of 187 programmes under the European Social Fund). Only three Member States 
include  YEI  in  more  than  one  programme  (Belgium  and  the  UK  with  two  each,  and 
France  with  several  regional  programmes  implementing  YEI  and  a  dedicated  national 
operational programme for YEI).   

All  Member  States,  including  those  not  eligible  for  the  YEI,  can  programme  ESF 
resources to support youth employment operations. There are 58 operational programmes 
running in 20 Member States that include ESF investments under IP8.ii (access to youth 
employment) and that not covered by the YEI.  

14 

 
 
 
 
 
 
 
 
 
 
Altogether only five countries (Austria, Denmark, Estonia, Finland and the Netherlands) 
did  not  programme  ESF  and  YEI  investment  under  the  youth  employment  investment 
priority. This echoes the fact that these Member States did not receive a country-specific 
recommendation  on  youth  employment  under  the  European  Semester  process  over  the 
years15, which means that these Member States did not present major policy challenges in 
this  policy  area.  However,  even  though  these  Member  States  did  not  allocate  specific 
budget  to  the  youth  employment  investment  priority,  several  operations  are  running  in 
Austria, Denmark and Finland that are mainly focused on young people. 

During  the  negotiations  of  the  2014-2020  cycle  of  operational  programmes,  the 
Commission  called  for  all  action  on  youth  employment  to  be  programmed  under  the 
dedicated investment priority on  youth employment (IP 8.ii), to ensure visibility and  to 
concentrate  action.  Nevertheless,  a  map  of  all  operations  reported  by  the  managing 
authorities under Thematic Objective 8 related to employment-related actions shows that 
ESF support for employment programmed under other investment priorities may also be 
accounted  as  youth  employment.  The  costs  of  these  operations  are  estimated  at  €2.1 
billion, which can be extrapolated to an estimated €3.5 billion (assuming similar levels of 
implementation as the Youth Employment operations under IP8.ii: 66%).  

Based  on  these  additional  calculations,  the  total  estimated  funding  for  Youth 
Employment  schemes  under  the  European  Social  Fund  and  the  Youth  Employment 
Initiative  is  €22  billion,  representing  18%  of  the  total  budget  for  the  European  Social 
Fund (including national co-financing). 

3.2.  TYPE OF OPERATIONS FUNDED 

A  total  of  596  different  ESF  and  YEI  operations16  across  all  28  Member  States  were 
identified under IP 8ii. 

For ESF-funded operations, the budget was allocated as follows: 

  24% (the largest share) to operations that support work-based learning,  
  22% to guidance and support measures,  

15  One  exception  is  Austria  that  received  a  recommendation  on  improving  the  education  outcomes  / 
achievements / basic skills, in particular of young people with a migrant background, for four years in 
a row. 

16   An  ‘operation’  is  defined  in  the  Common  Provisions  Regulation  (Article  2,  Regulation  (EU) 
No 1303/2013) as ‘a project, contract, action or group of projects selected by the managing authorities 
of  the  programmes  concerned,  or  under  their  responsibility,  that  contributes  to  the  objectives  of  a 
priority or priorities; in the context of financial instruments, an operation is constituted by the financial 
contributions  from  a  programme  to  financial  instruments  and  the  subsequent  financial  support 
provided by those financial instruments’. 

15 

 
 
 
                                                 
  13% to measures that support education and training,  
  10% to financial incentives,  
  13% to operations with multiple categories of operations that run under a single 
operation,  with  no  dominant  type.  For  example,  a  single  operation  could 
encompass  a  pathway  of  training  and  education  combined  with  guidance  and 
counselling for individuals.  

Given the specific scope and objectives of the Youth Employment Initiative, the share of 
different types of operations funded is different.  This is confirmed by the  high share of 
the  budget  (20%)  allocated  to  multiple  types  of  operations  in  a  single  approach.  In 
particular,  the  higher  share  of  financial  incentives  (26%  compared  to  10%  for  ESF-
funded  operations),  such  as  wage  subsidies,  reflects  the  slightly  different  set-up  of  the 
Initiative.  However,  the  provision  of  guidance  and  support  accounts  for  a  considerably 
smaller share in YEI operations (11% against 22% under the ESF). The share of funding 
allocated  to  work-based  learning  and  education  and  training  are  relatively  similar  for 
both YEI and ESF-funded operations.  

Figure 6 

Share of costs eligible for ESF funding by type of operation 

Source: Mapping of operations eligible costs declared in AIR 2015 – 2018 

16 

 
 
 
 
3.3. 

FINANCIAL AND OPERATIONAL IMPLEMENTATION 

By  the  end  of  2018,  there  had  been  about  3.8  million  participations17  under  ESF/YEI 
in  employment 
operations.  Some  1.4  million  participations  operations  resulted 
immediately  after  participation,  with  further  results  including  employment-focused 
education and training, and self-employment.   

To  direct  resources  to  youth  employment  measures  at  the  start  of  the  programming 
period,  YEI  investments  were  frontloaded  in  the  first  two  years  of  the  programming 
period,  while  ESF  investments  could  be  allocated  over  the  full  programming  cycle18. 
Some  Member  States  planned  to  continue  YEI  schemes  after  2018  from  the  start,  with 
ESF  funding.  This  initial  frontloading  of  YEI  funding  explains  the  difference  in 
implementation rates between YEI and ESF, with YEI financial execution overall more 
advanced in the Member States that receive YEI funding, in particular in Member States 
that receive a substantial share. The start of the actual implementation was delayed due to 
the time needed to set up control and monitoring systems that satisfy the Regulations and 
legislation.  The  socio-economic  context,  absorption  and  administrative 
national 
capacities of Managing Authorities and beneficiaries appear to be decisive factors for the 
progress in implementation which varies a lot between the different Member States.” 

   Figure 7 
(expenditure declared to the managing authority vs funding allocated) 

rates  of  YEI/ESF 

Implementation 

funding  by  Member  State 

Source: SFC2014, based on AIR 2018, data extracted on 6 September 2019 MD=more developed region, 
LD=less developed region, TR=transition region. No regional data available for YEI funding. 

17 This number denotes the number of participations in all operations, which is not the same as individual 
participants, as an individual can participate in several operations over the course of the programming 
period. 

18 Taking into account the N+3 rule for spending and declaring investments, the last eligible declarations 

for YEI support were initially planned for 2018. 

17 

 
 
 
 
 
 
                                                 
4.  METHOD 

4.1. 

SHORT DESCRIPTION OF THE METHODOLOGY 

The  evaluation  questions,  derived  from  the  Better  Regulation  evaluation  criteria,  were 
broken down into a number of sub-questions. 

The evaluation was based on three main sources of data: 

  a study19 carried out by an external contractor; 

 

the twelve-week public consultation, which was carried out by and analysed with 
the assistance of the external contractor; 

  an  estimate  of  macroeconomic  indirect  effects  of  youth  employment  operations 
the  Dynamic  Computable  General  Equilibrium  model  ‘Rhomolo’ 

using 
developed by the European Commission’s Joint Research Centre.  

There  were  1,376  responses  to  the  public  consultation.  Over  half  (57.6%)  were 
responses by citizens up to 33 years of age, almost a quarter of who had received funding 
under  ESF  or  YEI  operations.  Some  21.8%  of  respondents  were  from  organisations 
involved  in  managing  and  implementing  of  YEI  and/or  ESF-funded  operations.  Where 
data on the gender of respondents was provided, 58.7% of respondents were female. 

The  study  combined  all  information  on  the  other  two  instruments  in  a  single  report, 
which provided the main materials and analysis for this evaluation. The study applied a 
mix of evaluation methods and tools, which are explained in further detail in Annex 3 to 
this Staff Working Document. In summary, the study involved: 

  mapping  and  carrying  out  an  in-depth  analysis  of  youth-related  thematic 
objectives, investment priorities, target populations and type of operations, based 
on 
the  SFC2014  database,  operational  programmes,  AIRs  and  related 
documentation,  including  the  Commission’s  country-specific  recommendations. 
The study used data up to 31 December 2018, as reported by Member States by 
September 2019. For the mapping, the regions were categorised into four clusters 
in terms of their socioeconomic context, and used in the analysis and conclusions 
of this evaluation: 

o  Cluster A – regions with a strong initial socioeconomic context that made 

substantial progress over the period;  

o  Cluster B – regions with a strong initial socioeconomic context that made 

limited progress over the period; 

19 https://ec.europa.eu/social/main.jsp?catId=738&langId=en&pubId=8307&preview=cHJldkVtcGxQb3J0YWwhMjAxMjAyMTVwcmV2aWV3 

18 

 
 
 
                                                 
 
o  Cluster C – regions with a weak initial socioeconomic context that made 

visible progress; 

o  Cluster  D  –  regions  with  a  very  weak  initial  socioeconomic  context  that 

made limited progress.  

 

reviewing  available  literature  (policy  documents,  regulations,  national  and  EU-
wide evaluations, ESF websites and publications);  

  a synthesis of the 2018 compulsory national YEI evaluation reports and screening 
of other ESF evaluations of youth employment programmes using the database of 
evaluations compiled by the Evaluation Helpdesk; 

 

the results of the public consultation; 

  a  comparative  analysis  of  labour  market  trends  and  the  role  of  EU-funded 

operations; 

  a cost-effectiveness analysis;  

  econometric analysis and provision of data to the Joint Research Centre (JRC) for 

the Rhomolo modelling simulations; 

  case  studies  in  10  Member  States20,  covering  20  operational  programmes  and  a 

mix of YEI and ESF youth employment programmes; 

  62 interviews with desk officers, managing authorities and other stakeholders, in 

particular for the case studies; 

 

five focus groups in the Member States and one at EU level; 

  a  set  of  fact  sheets  for  all  countries  with  YEI  and/or  ESF-funded  youth 

employment programmes;  

  a compilation of lessons learned and good practices.  

4.2.  LIMITATIONS AND ROBUSTNESS OF FINDINGS 

The findings and conclusions are reasonably robust, since they are based on a series of 
measurements from the multiple sources and tools used in the evaluation. However, the 
evaluation also noted several limitations. 

Data  quality  and  timeliness.  There  is  a  significant  time  lag  between  reporting  the 
outputs, results and financial implementation,  as a result of work on  data checking and 
reporting. The costs are declared only after checks by relevant authorities. Often, outputs 
are  reported  once  operations  are  completed  and  declared,  and  the  results  reported  later 

20 Belgium, Germany, Greece, Malta, Portugal, Spain. France, Italy, Poland, Slovakia. 

19 

 
 
 
 
                                                 
still. Lastly, since the data is based on participations in operations and not on individual 
participants, the calculation of certain ratios, such as success rate, is challenging. 

Detailed  programme  information.  Detailed  information  on  individual  operations  is 
usually not available, despite the obligation to record and store minimum data. Moreover, 
across the EU, the rules and standards for reporting operations are uneven. This means it 
is  difficult  to  make  a  systematic  comparison.  To  address  this  challenge,  the  evaluators 
focused on the cost of operations and made a number of assumptions such as estimated 
values  by  type  of  operation  where  no  financial  allocation  was  available,  removing 
implausible  values  or  the  assumption  that  operations  not  covered  by  Article  IP  8.ii 
reached similar implementation levels as those covered by Article IP 8.ii.  

Clearly defining youth employment operations. Regarding ESF support, a number of 
Member States decided to run youth employment schemes also across other investment 
priorities. As a result, the EU-level mapping exercise required clear definitions based on 
robust  monitoring  data,  and  estimates  of  the  costs  of  operations  not  covered  by  the 
investment  priority  on  youth  employment  (8.ii).  The  report  makes  it  clear  throughout 
what the underlying data is based on and where reliable comparisons can be drawn. For 
YEI support, the rules require that all YEI funding (including the matching ESF support) 
be programmed under IP 8.ii.  

Public consultation. The public consultation was in the form of an online survey21 run to 
gauge the views of  a wide range of stakeholders, on a voluntary basis. This means that 
the information is not representative. Almost three quarters of the responses came from 
individuals and organisations in four Member States: Italy, Bulgaria, Spain and Slovakia, 
despite extensive efforts to publicise the consultation. This was highlighted in the report. 
Nonetheless, the public consultation provided a substantial amount of useful information 
and insights into the experiences and perceptions of youth employment operations. 

Comparative analysis of labour market trends. At the time of writing the report, data 
in the labour market policy database22 were only available up to 2017; excluding 2018. 

Regional clusters. YEI monitoring data are not broken down by type of region, so data 
on expenditure and participants may cover more than one type of region.  

21  As  indicated  in  the  Commission  Better  Regulation  guidelines,  the  data  gathered  through  public 

consultations does not provide a representative view of the EU population. 

22 Labour Market Policy statistics are one of the data sources for monitoring the Employment Guidelines. 
The scope of LMP statistics is limited to public interventions that explicitly target groups of people with 
difficulties  in  the  labour  market.  Data  on  public  expenditure  and  participants  (stock  and  flows)  are 
collected  annually  from  administrative  sources.  Quantitative  data  on  expenditure  and  participants  are 
complemented by a set of qualitative reports that describe each intervention. 

20 

 
 
 
                                                 
Cost  effectiveness.  Due  to  issues  of  data  timeliness  and  completeness,  the  lack  of  full 
information  on  the  outputs  and  results  made  it  difficult  to  draw  overall  conclusions  on 
cost  effectiveness.  Thus  the  cost  per  result  are  not  yet  meaningful  and  the  cost  per 
participation  (number  of  individual  participants  registered  against  the  total  eligible 
expenditure  declared)  are  but  a  snapshot  at  the  cut-off  date  of  31 December  2018.  The 
approach taken was to combine partial findings from a number of sources (counterfactual 
impact  evaluations  available,  micro-data  from  case  studies  in  the  Member  States, 
econometric analyses, Rhomolo modelling and the public consultations) in order to draw 
valid, if only partial, conclusions. 

Econometric  analysis.   The  explanatory  power  of  the  regressions  –  especially  for  unit 
costs – is low, due to the high degree of variability in the observations. This is the result 
of the complexity of  youth employment  schemes, mixed target  groups and the fact that 
the data is still in the process of being consolidated (as mentioned above). 

Rhomolo  analysis.  General  equilibrium  models  need  to  be  based  on  a  number  of 
simplifying assumptions. Given the overall scale of EU youth employment operations in 
comparison with GDP figures and national programmes, the macro impacts identified by 
the model should be viewed with due perspective. Nonetheless, the work has been useful 
and informative. 

COVID-19  pandemic.  Note  that  the  field  work  for  this  staff  working  document  was 
carried out before COVID-19 pandemic came to Europe. The Commission proposed the 
Recovery Plan for Europe on 27 May 2020, as part of the current and future multiannual 
financial  frameworks.  One  of  the  aims  of  the  Recovery  Plan  is  to  invest  significant 
resources  in  measures  to  support  youth  employment  and  related  reforms  for  the 
remainder of the current implementation period and over the next programming period, 
with  a  view  to  mitigating  the  consequences  of  this  pandemic  on  young  people  and 
turning around the situation for youth employment for the better.  

21 

 
 
 
 
 
 
5.  ANALYSIS AND ANSWERS TO THE EVALUATION QUESTIONS 

5.1.  EFFECTIVENESS:  HOW  EFFECTIVE  IS  THE  YEI,  AND  OTHER  ESF-FUNDED 

YOUTH EMPLOYMENT OPERATIONS, IN ACHIEVING THEIR OBJECTIVES? 

The  section  on  effectiveness  describes  the  extent  to  which  YEI  and  ESF-funded 
operations  have  progressed  towards  the  set  objectives,  broken  down  into  operational 
(outputs), specific (results) and general (longer-term results/impacts) objectives. It covers 
both quantitative (success rates) and qualitative aspects (e.g. the quality of employment 
offers).  This  section  also  analyses 
influenced  performance 
the 
(socioeconomic  context,  implementation  arrangements,  type  of  action  and  outreach 
strategies). 

factors 

that 

Financial implementation and expected outputs are progressing, albeit slower than 
anticipated 

For  the  Youth  Employment  Initiative,  progress  on  implementation  has  been  below  the 
targets set in the operational programmes, both in terms of financial implementation and 
number of participations. By the end of 2018, only France, Hungary, Lithuania, Latvia, 
Poland  and  Slovenia  had  met  the  2018  milestone  targets  set  at  the  start  of  the 
programming period. Despite action taken by the European Commission to facilitate the 
implementation of the Initiative in the early stages23, Member States experienced delays 
in implementing the Initiative in 2014 and even in 2015, as they decided to first ensure 
compliance  with  ex  ante  conditionalities24  including  set  up  of  monitoring  systems  that 
complied with the Regulations25 and national legislation, before initiating the work.  

In addition, a common challenge across Youth Employment  Initiative programmes was 
reaching  young  people  not  in  employment,  education  or  training.  Since  many  of  these 
young people were often not registered as unemployed, identifying the target group often 
required  active  canvassing  and  encouraging  individuals  to  register.  Member  States  and 
implementing  organisations  (beneficiaries  of  YEI  support)  often  reported  gathering  the 
evidence on NEET status as possibly the most challenging legal requirement in relation 
to implementing the Initiative.  

23 There are two main factors to consider. First, Member States could begin implementing the YEI as early 
as  September  2013.  Second,  in  2015,  an  amendment  to  the  ESF  Regulation  substantially  raised  the 
initial  pre-financing  amount  paid  to  operational  programmes  (from  1%  to  30%),  to  help  countries 
facing budget constraints to start implementing the Initiative. 

24 Ex ante conditionality means a specific and precisely pre-defined critical factor, which is a prerequisite 
for and has a direct and genuine link to, and direct impact on, the effective and efficient achievement 
of a specific objective for an investment priority or an EU priority. 

25 Regulation (EU) No 1303/2013 of the European Parliament and of the Council and Regulation (EU) No 

1304/2013 of the European Parliament and of the Council. 

22 

 
 
 
                                                 
Over time, most Member States implementing the Initiative broadened the eligibility of 
investments  to  include  25-29  year  olds.  This  required  refocusing  the  support  to  help 
young people overcome barriers to work and to fund outreach, rather than on developing 
their qualifications and skills.  

Faced  with  these  considerable  challenges  to  meet  the  initial  targets  for  2018,  the  2017 
revision  of  the  multiannual  financial  framework  provided  resources  for  2017-2020, 
which led to an extension of YEI funding until the end of the programming period. This 
extension  was  prompted  by  the  need  to  ensure  that  operations  already  started  and 
considered  crucial  by  the  Member  States  could  continue.  As  a  consequence,  the 
operational  programmes  were  amended  and  the  implementation  targets  were  revised26. 
The EU average financial implementation rate reached 52% by the end of 2018. 

Figure 8 

Progress towards the milestones set for 2018 – YEI 

Source: SFC2014, based on AIR 2018, data extracted on 6 September 2019 (Cluster A – Strong 
start/substantial progress, Cluster B – Strong start/limited progress, Cluster C – Weak start/visible 
progress, Cluster D –Weak start/limited progress) 

According  to  the  econometric  analysis,  the  country  specific  context  affects  how  the 
Initiative is implemented. In addition, the countries in most need of support often face the 
greatest implementation challenges. For example, countries with an unfavourable context 
at the start but where conditions are improving saw a positive correlation with progress in 
the  project  selection  rate.  By  contrast,  in  general  there  was  no  correlation  between  the 
change in the youth unemployment rate and financial progress. 

26  Due  to  the  extension  of  the  implementation  period,  most  Member  States  substantially  lowered  their 
targets for 2018. After the adjusted target following the 2017 reprogramming exercise, the EU average 
milestone  for 2018  was  set at 61% of the revised YEI budgets (or 82% of the  allocated YEI budget 
before the budget increase), instead of 100% as set at the start of the programming period. 

23 

 
 
 
 
 
                                                 
The impact of socioeconomic factors on achieving the output targets, such as difficulty to 
reach target audience, lack of jobs or low education of participants, is more shaded.  

Other factors that hindered implementation were related to the administrative capacity of 
the managing authorities and the absorption capacity of beneficiaries. 

EU-funded  youth  employment  operations  have  made  a  positive  contribution  to  the 
integration of young people in the labour market  

Over 1.5 million completed participations in YEI were recorded in the monitoring system 
for unemployed young people under 25 years of age. This is slightly over half the total 
participations  (2.7  million  total  participations).  However,  it  is  important  to  note  that 
many  participations  continued  after  2018  so  they  were  not  recorded  in  the  study  as 
completed. 

The 1.5 million represents on average 65% of the final target set for this population. Out 
of this total, 0.7 million unemployed people received an offer of training or employment 
immediately after completing the operation (with an average target achievement of 58%). 
Moreover,  0.9  million  people  unemployed  at  the  start  left  the  operation  either  in 
education or training, having gained a qualification or having found employment (60% of 
the  target).  This  is  encouraging  progress,  considering  that  YEI  implementation  is 
currently in its most active phase and will continue at least until the end of 2022, under 
cohesion policy rules. Therefore, it is likely that the overall YEI targets will be achieved. 

There  were  positive  results  for  the  long-term  unemployed  as  result  of  participating  in 
YEI27: 

27 The figures for success rate are the share of each type of result compared with their target population (i.e. 
first three columns are the share of unemployed  that achieved each specified outcome, the next three 
columns give the share of long-term unemployed that achieved each specified outcome). 

24 

 
 
 
 
 
 
 
 
 
 
                                                 
The  results  for  economically  inactive28  participants  have  steadily  improved,  but  the 
target for this group is relatively low (compared with implementation rates, for example, 
but  also  with  the  results for the long-term  unemployed), as  people  are further from  the 
labour market and less responsive to active labour market schemes than people who are 
unemployed.  They  are  also  often  more  difficult  to  reach  out  to  and  may  require  more 
comprehensive support, including motivation, counselling and social inclusion measures. 

Most  participants  who  responded  consider  that  their  participation  in  EU  youth 
employment operations has been beneficial 

The opinions expressed by participants during the public consultation  were consistently 
positive regarding the effects of the EU support they received in helping them integrate 
into the labour market.  

The  most  frequent  types  of  support  the  participants  received  were  support  to  develop 
general  skills  (33.6%  of  respondents),  help  in  looking  for  a  job  (33.4%),  and  gaining 
professional  skills  and  qualifications  (32.3%).  Only  10.9%  of  respondents  said  that  the 
support received did not help. 

28 According to Definitions for ESF common indicators (Annex C1 to Commission Guidance document on 
monitoring and evaluation of ESF) "inactive" are persons currently not part of the labour force (in the sense 
that they are not employed or unemployed according to the definitions provided). 

25 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
                                                 
 
The benefits of youth employment support go beyond access to employment 

The views  expressed by the  organisations  responding to  the public  consultation  echoed 
those expressed by individual participants. According to the organisations, the benefits of 
ESF/YEI funding are mostly linked to the development of skills and qualifications and to 
help  young people find a job, including  young people not in  employment, education or 
training and other disadvantaged individuals.  

Respondents from the organisations involved in running ESF/YEI also expressed broad 
agreement on the following positive results of the Initiative:  

inclusion of marginalised people;  

  promotion of new partnerships and exchanges;  
 
  development of soft skills and group work;  
  youth empowerment;  
 
 
 

increased connection with labour market requirements;  
increased motivation and  
integration into society of young people not in employment, education or training. 

Overall,  respondents  from  organisations  are  also  more  sceptical  than  individual 
respondents  concerning  the  ESF/YEI’s  effectiveness  in  helping  young  people  improve 
their situation on the labour market in a sustainable manner.  

Identifying and reaching out to NEETs was often a challenge 

Despite  the  high  levels  of  young  people  not  in  employment,  education  or  training  in  a 
number  of  countries,  actually  identifying  them  to  provide  support  proved  difficult,  in 
particular in Member States that had not yet set up policy channels and mechanisms to do 
so.  

Several  Member  States  developed  strategies  to  overcome  this  challenge.  For  example, 
they used social networks, media, newsletters or more innovative channels. In Bulgaria, 
concerts  were  organised  and  adverts  were  placed  on  fast  food  trays  to  reach  young 
people. They also used street work to meet young people not in employment, education 
or training in public places where young people gather, e.g. in parks or shopping centres.  

Member States also invested in developing the experience and skills required to reach the 
‘hardest  to  reach’  young  people,  working  with  professional  youth  workers29  who 
represent  local  organisations,  promoting  cooperation  between  the  various  bodies 
involved, especially ‘vertically’ (e.g. from local level to the public employment service), 

29  'Youth worker' means a professional or a volunteer involved in non-formal learning who supports young 

people in their personal socio-educational and professional development 

26 

 
 
 
                                                 
going  beyond  traditional  outreach  channels.  One  example  is  in  Germany,  where  ESF 
funding  successfully  supported  tailored  activities  (support  chains  and  dedicated 
preventative systems of assisted transition from school to work through systematic multi-
party cooperation). 

Resources were channelled to the population most in need 

The  figures  overleaf  show  that  ESF/YEI  funding  reached  a  much  higher  proportion  of 
young  people  not  in  employment,  education  or  training  in  regions  where  the  labour 
market situation of young people was worse at the start30 of the programming period than 
in  regions  with  better  initial  favourable  conditions31,  particularly  when  the  situation 
continued  to  be  unfavourable,  e.g.  when  there  has  been  limited  progress32.  The  higher 
coverage  in  regions  where  the  labour  market  situation  of  young  people  was  worse 
confirms that resources were channelled to the areas (and people) most in need.  

Figure 9 

Share  of  NEETs  (aged  15-24)  receiving  ESF/YEI  funding  as  a 
proportion of the NEET population in 2014 by cluster of regions (%) 

Source: SFC2014, based on AIR 2018, data extracted on 6 September 2019 and Eurostat, Labour Force 
Survey (edat_lfse_22), data extracted on 21 June 2019. 

These findings concur with the Rhomolo modelling analysis, which found that EU youth 
employment  funding  went  mostly  to  the  regions  most  in  need.  The  Rhomolo  analysis 
also found that, in terms of indirect employment, the results are stronger for low-skilled 
participants, which was the main target group. At regional level too, in regions located in 
Southern European Member States where youth unemployment is high, EU funding has 

30 Cluster C and D regions – see section 4.1. 

31 Cluster A and B regions – see section 4.1. 

32  The  data  relate  to  flows  of  funding  to  the  initial  population  of  young  people  not  in  employment, 
education or training. Since people joining this category after the initial stock was recorded could still 
receive  support  in  this  funding  period,  a  perfect  coverage  of  the  whole  NEET  population  would 
produce results above 100%. 

27 

 
 
 
 
                                                 
been  the  most  beneficial  for  participants,  considering  also  the  high  relative  size  of  the 
support provided in those regions. 

Overall participation was well balanced from a gender perspective 

Across the EU, participation is well balanced from a gender perspective, with a broadly 
equal share of men and women (51% against 49%) reached by YEI operations, although 
participation in ESF-funded operations is higher for men (54% against 46%). 

There are considerable differences among the Member States. YEI operations in Croatia 
and  Greece  reached  more  young  women  (66%  for  the  YEI  and  63%  for  youth 
employment  operations  under  the  ESF),  while  youth  unemployment  initiatives  in 
Belgium reached more  young men. The participation rate of young men in ESF-funded 
operations  in  Belgium,  Germany,  France  and  Sweden  was  over  60%.  Cyprus,  Croatia, 
Poland and Slovenia, by contrast, had above-average female participation rates. In some 
countries (Belgium and Spain), the national evaluations concluded that women benefited 
less from the operations under the ESF/YEI than men.   

Reaching  out  to  specific  groups  of  disadvantaged  young  people  proved  particularly 
challenging, requiring multiple methods 

According  to  the  2018  YEI  evaluations,  the  most  vulnerable  young  people  are  often 
under-represented,  and  potentially  under-reported.  This  is  in  line  with  the  replies 
submitted  to  the  public  consultation,  which  noted  difficulties  in  reaching  out  to  and 
supporting vulnerable  young people in  rural  areas, those with  disabilities, ex-offenders, 
those at risk of poverty and the homeless, and young people with multiple disadvantages.  

Figure 10  How  successful  were  the  youth  employment  operations  funded  by  the 
European  Social  Fund  and  the  Youth  Employment  Initiative  in  providing 
support to the following target groups? 

Young people not in employment, education and…
Young people leaving education without…
Young people at risk of social exclusion or…

Young migrants and minorities

Young people in disadvantaged or hard-to-reach…

Youth affected by poverty
Homeless young people

28 
27 
32 
25 
12 

78 

54 

224 

172 

80 

159 

139 
132 
133 

97 
132 

115 
127 

60 

185 

108 

42 

75 

96 
83 

96 
91 

48 

7 
18 
19 
18 
24 
23 
34 

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Very successful

Mostly successful

I don't know / I don't have an opinion Mostly unsuccessful

Not successful at all

Source: Final Report, Public Consultation on the Evaluation of the EU Support to Youth Employment by the 
Youth Employment Initiative and the European Social Fund. Study supporting the 2019 evaluation of 
youth employment (VC/2018/0175) 

28 

 
 
 
 
On  the  basis  of  the  2018  annual  implementation  reports,  and  bearing  in  mind  that  the 
categories below are not mutually exclusive, note that at least at EU(28) level: 

  11.3% of YEI and 13.2% of the ESF participations reported were young migrants, 
participants  with  a  foreign  background  or  minorities  (including  marginalised 
communities such as Roma);  

  4.5% of YEI and 5.2% of ESF participations were young people with disabilities;  

  13.7%  of  YEI  and  14.2%  of  ESF  participations  were  people  with  other 

disadvantages; 

  0.8%  of  YEI  and  0.3%  of  ESF  participations  were  homeless  people  or  people 

affected by housing exclusion; 

  19.3% of YEI and 18.7% of ESF participations were young people in rural areas. 

Under-reporting is quite likely for these indicators, since information on disadvantage is 
not always systematically collected by organisations or volunteered by participants. 

It  is  also  important  to  note  that  not  all  EU  countries  have  the  same  definition  of  the 
disadvantaged, or the same definition of the NEET category. In some regions particularly 
hit  by  the  crisis,  a  substantial  share  of  the  young  population  could  be  considered 
disadvantaged  at  the  beginning  of  the  operation,  while  in  other  parts  of  Europe  the 
definition  covered  only  certain  groups.  Because  of  this,  there  are  notable  differences 
between  the  Member  States.  For  example,  Sweden  recorded  a  significant  share  of 
migrants  (38.7%  of  participations  in  YEI  and  38.6%  in  ESF  operations  against  an 
average  of  11.3%  of  YEI  and  13.2%  of  ESF)  and  a  significant  share  of  people  with 
disabilities (69% of ESF compared to an average of 5.2%).  

The  case  studies  illustrate  the  different  methods  used  to  reach  out  to  disadvantaged 
groups.  

In Portugal, for example, the national operating programme did not define sub-groups of 
young  people  not  in  employment,  education  or  training,  but  referred  only  to  young 
unemployed and young inactive people. The indicators did not differentiate participants 
further. Nevertheless, some operations included an increase in the budget for vulnerable 
groups such as the long-term unemployed, ex-prisoners, people with disabilities, etc. The 
regional operating programme run in the Azores in Portugal reached all target groups of 
disadvantaged young people in all nine islands, except for young people with disabilities. 

Evidence  from  the  case  studies  shows  that  some  of  the  differences  in  target  groups 
reached between the regions could be due to the type of organisations providing support 
rather than to the type of region.  

In Poland, for instance, there were differences in the targeting methods used by the local 
labour  offices  and  the  Voluntary  Labour  Corps.  The  local  labour  offices  operated  on  a 

29 

 
 
 
first come, first served basis with people registered with the public employment service. 
The  Voluntary  Labour  Corps  targeted  NEETs  including  those  not  registered  with  the 
public employment service. Public employment services use additional eligibility criteria, 
including a lack of professional qualifications or qualifications not adapted to the needs 
of  the  labour  market,  disabilities,  gender,  particularly  difficult  social  situations  (e.g. 
people from jobless households, families using social assistance services, single parents 
and people from rural areas).  

Participants were mostly satisfied with the offers received but had contrasting views on 
the quality of the offer 

One  of  the  concerns  of  participants  receiving  YEI/ESF  funding  from  the  start  was  the 
quality and timeliness of the employment or training offers received. 

In  practice,  the  criteria  most  used  to  assess  the  quality  of  job  offers  were  whether  the 
offer was for a permanent or full-time job, and the wage level.  

For most operational programmes for which data on the quality of job offers is available, 
over  a  third  of  the  offers  provided  with  EU  support  were  for  ‘permanent’  employment 
contracts (in the sense that they were for an indefinite period of time). 

Figure 11  Share of job offers with permanent contracts according to YEI evaluations 

70

60

50

40

30

20

10

0

Source: 2018 YEI Evaluation reports  
Data for FR relate to three national measures (‘Accompagnement APEC’, FR-GJ: ‘Garantie Jeunes’, FR-PA: 
‘Parcours Autonomie’). 
Data for CY relate to ‘Acquisition of work experience for young unemployed graduates’. 
Data for ES refer to the situation 12 months after exit, and 6 months for UK-England.  

30 

 
 
 
 
 
 
The results of the public consultation indicate that ESF/YEI participants who took part in 
the  consultation  and  who  started  the  operation  unemployed  were  more  likely  to  have 
transitioned  to  full-time  employment  (39.7%)  than  to  part-time.  However,  among  the 
long-term  unemployed,  only  19.5%  found  full-time  employment  after  support,  7.3% 
found  part-time  work  and  half  remained  unemployed.  When  looking  at  responses  from 
all  types  of  participants,  more  respondents  reported  that  they  found  a  temporary  job 
(20.9% of all ESF/YEI participants) than a permanent one (16.8%). 

Several  YEI  evaluation  reports  highlighted  that  for  most  participants,  the  wage  level 
offered  was  an  issue,  as  it  was  often  at  minimum  wage  level.  This  was  a  particular 
concern in (though not only in) Bulgaria, Croatia, Cyprus and Lithuania.  

In addition to the statistical data, the YEI evaluations carried out by the Member States 
revealed a high level of satisfaction with the support received by participants. In Croatia 
and  Slovenia,  for  instance,  the  satisfaction  rates  are  close  to  70%,  and  in  Italy,  the 
participants surveyed are on average more satisfied with the support received through the 
YEI  than  participants  in  other  employment  support  measures.  There  is,  however,  a 
general  consensus among stakeholders in Italy that the quality of the offers should and 
could be further improved and that it should provide a form of certification of the skills 
acquired. 

The effectiveness of operations was influenced by the type of operations and, to a certain 
extent, socioeconomic factors and organisational arrangements 

The results of the public consultation indicate that paid apprenticeships are considered to 
be the most effective type of intervention by organisations involved in ESF/YEI, together 
with  vocational  education  and  training  and  basic  skills  training,  which  are  considered 
useful  for  most  target  groups,  including  the  most  disadvantaged.  This  is  generally 
supported by the evaluations carried out by Member States and the case studies providing 
specific examples. 

Community  or  voluntary  work,  as  well  as  non-paid  apprenticeships/traineeships,  are 
considered the least effective types of operations in terms of access to employment, but 
not necessarily in terms of employability. 

31 

 
 
 
 
 
 
 
Figure 12  To  what  extent  do  you  agree/disagree  that  the  following  measures  have 
been  actually  effective  in  helping  young  people  enter  quality  and 
sustainable employment? (population: 399 organisations) 

Remunerated…

187 

172 

Vocational oriented education and training

Basic skills training

Guidance and career support

169 

164 

163 

Skills assessment

116 

Subsidies to employers for…

138 

Awareness raising and information actions…

113 

183 

183 

180 

199 

174 

196 

25 

5 
10 

29 

3 
15 

29 

4 
19 

31 

7 
18 

49 

28 

7 

45 

36 

6 

45 

39 

6 

Employment incentives for employers

Second chance education

Start-up support for young entrepreneurs and…

156 

136 

112 

130 

124 

158 

46 

31 

8 

167 

61 

29 

6 

178 

159 

161 

66 

36 

7 

63 

34 

13 

52 

49 

13 

Job mobility measures

101 

161 

71 

54 

12 

Community or voluntary work

49 

148 

89 

85 

28 

Non-remunerated…

44 

139 

62 

115 

39 

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

I strongly agree

I agree

I don't know / I don't have an opinion

I disagree

I strongly disagree

Source: Final Report, Public Consultation on the Evaluation of the EU Support to Youth Employment by the 
Youth Employment Initiative and the European Social Fund. Study supporting the 2019 evaluation of 
youth employment (VC/2018/0175) 

Socioeconomic and geographic factors clearly influenced the effectiveness of operations 

Although  the  econometric  analysis  indicated  that  the  socioeconomic  context  does  not 
seem  to  play  a  major  role  in  achieving  the  targets  for  output  indicators  for  the  YEI, 
regions 
lower 
achievement  rates  for  ESF  operations.  ESF  operations  in  regions  in  a  weak 

less  favourable  economic  situations  recorded  comparatively 

in 

32 

 
 
 
 
 
socioeconomic  situation  at  the  start  and  that  made  limited  progress  (Cluster  D)33  had 
more difficulties in reaching the target number of individuals, whereas ESF investments 
in  regions  where  the  socioeconomic  situation  was  improved  made  steadier  progress 
towards  their  targets.  The  socioeconomic  situation  also  had  a  greater  bearing  on  the 
achievement  of  targets  for  result  indicators  than  for  outputs,  independently  of  delayed 
reporting.  

A  high  share  (47.4%)  of  organisations  responding  to  the  public  consultation  reported 
structural problems (such as the lack of jobs and the low education level of participants) 
as the factors hindering the achievement of ESF and YEI objectives, in addition to a lack 
of  flexibility  to  adapt  operations  to  changing  socioeconomic  contexts.  This  is 
understandable, given the difficult economic environment in which YEI and ESF youth 
employment  operations  were  implemented  in  the  aftermath  of  the  late  2000s  economic 
and financial crisis, in particular in southern Europe.  

The  main  geographical  factor  to  emerge  as  having  an  influence  on  effectiveness  of 
operations  was  the  difference  between  urban  and  rural  areas.  This  is  based  on 
information  from  the  public  consultation  and  the  case  studies,  which  provide  some 
examples.  For  instance,  in  Spain  actions  taking  place  in  large  cities  were  considered 
more effective than those in rural areas, in particular as those run in rural areas typically 
tend to be more expensive due to the larger distances. They also suffer from having less 
infrastructure available and a smaller overall local job market. 

The  administrative  capacity,  experience  and  organisation  of  managing  authorities  and 
beneficiaries also had an impact on effectiveness 

Structural  challenges  relate  not  only  to  the  socioeconomic  context,  but  also  to  the 
capacity  of  managing  authorities  and  beneficiaries  to  deal  with  complex  programming 
requirements,  including  eligibility  criteria  for  young  people  not  in  employment, 
education or training under the YEI, and engaging with stakeholders. This last point was 
particularly  relevant  in  regional  programmes  where  the  successful  implementation  of 
projects depends on cooperation between multiple governance levels and bodies.  

The  2018  national  YEI  evaluations  revealed  that  prior  experience  and  continuity  are 
factors contributing to effective management. In the case studies, experience in managing 
similar measures was reported as helpful in implementing YEI. In Ireland, for instance, 
the long history of some operations (such as BTWEA, which has existed since 1993 or 
‘Community Training Centres’) is considered to contribute to the success of YEI.  

33  This  cluster  comprises  six  less  developed  regions  (in  Bulgaria,  Spain,  France,  Italy,  Hungary  and 
Romania) and five transitional regions (in Belgium, Spain, Greece, France - overseas, and Italy). 

33 

 
 
 
                                                 
Good  governance  was  widely  seen  as  essential  to  the  success  of  the  YEI  evaluations. 
This  is  illustrated  in  Sweden,  where  the  high  quality  implementation  of  the  operations 
(the competence of staff, and low staff turnover) was regarded as a major success factor.  

Partnership and cooperation  were also  mentioned in  several  YEI evaluations  and in the 
public  consultation  as  crucial  aspects  to  support  implementation.  For  instance,  in 
Brussels-Capital  Region,  cooperation  between  Actiris  and  other  partners  improved 
access  to  support for  young people  with  a low level  of qualifications  or those living in 
jobless households. 

The managing  authorities participating in  the EU-level  focus  group also  confirmed that 
cooperation  among  the  several  bodies  involved,  in  particular  among  employers  and 
employment  services,  is  key  to  the  efficient  implementation  of  youth  employment 
operations. This can only work well if the training offer is in line with companies’ need 
for skills or when there is a financial incentive for companies. 

The ability to reach the target population seems to have been the single biggest factor in 
effectiveness 

All  in  all,  there  seems  to  be  a  consensus  among  stakeholders  that  the  factor  with  the 
single  greatest  impact  on  effectiveness  was  the  ability  to  reach  the  target  groups. 
Approximately six out of ten organisations responded to the public consultation that the 
main  hindrance  was  the  difficulty  in  reaching  target  groups,  followed  by  structural 
problems such as the lack of jobs and the low level of education of participants.  

This  was  also  confirmed  in  the  EU-level  focus  group  where  the  managing  authorities 
agreed that reaching those furthest from the labour market was key to the success of the 
programmes. They identified three main essential aspects: local outreach work, local staff 
with the right qualifications and skills, and the use of innovative channels to reach young 
people.  

The composition of the target group also had an impact on the success rate of operations  

The  composition  of  the  target  group  also  has  an  impact  on  effectiveness.  The  more 
ambitious the interventions were in terms of outreach to target groups requiring intensive 
and comprehensive support (for example people with disabilities or other disadvantaged 
people) the more difficult it was to achieve the financial targets and employment targets. 
This  was  also  confirmed  by  the  econometric  analysis.  For  some  disadvantaged 
participants,  access  to  employment  is  not  an  immediate  objective  or  even  an  option; 
instead  the  key  outcomes  were  confidence  building,  gaining  qualifications  and  some 
experience (not just employment).  

34 

 
 
 
 
Although not an explicit objective of YEI, youth employment operations led to structural 
changes in systems 

According  to  over  half  of  the  respondents  from  organisations  (59%)  to  the  public 
consultation,  the  employment  operations  run  under  the  European  Social  Fund  or  the 
Youth  Employment  Initiative  successfully  contributed  to  the  promotion  of  structural 
reforms  in  employment,  education  and  training  systems.  A  number  of  examples 
identified in the case studies confirmed this finding. 

The  main  impact  of  YEI  and  ESF  funding  for  young  people  across  operational 
programmes  was  in  changes  to  the  way  public  employment  services  and  other  bodies 
involved in employment policy work approach youth employment issues. 

Box 1 

Cooperation and partnerships in Germany and Portugal 

In  Saxony-Anhalt  and  North-Rhine-Westphalia,  the  two  initiatives  RÜMSA  and 
KAoA, which bring together all bodies involved in youth employment in the regions, 
are bringing about a system change by coordinating support offered to young people. 
KAoA  has  the  potential  to  spin-off  into  a  broader  municipal  policy-making  and 
coordination tool. 

In Portugal, implementation of the Youth Guarantee  under the Youth Employment 
Initiative  helped  speed  up  and  expand  the  public  employment  service’s  work.  The 
effect was to reach a more diverse target group of young people, extend geographic 
coverage and bring in new outreach channels (e.g. a specific position was created in 
job centres to assist companies and universities). The solid new network of partners 
working  on  the  Youth  Guarantee  enabled  the  service  to  identify  inactive  young 
people (who were not yet registered at the public employment service). 

Macroeconomic effects are small, but positive, especially in the medium to long term 

The  results  from  the  modelling  work  carried  out  by  the  JRC’s  Rhomolo  analysis 
conclude that the ESF/YEI investments in human capital and education for young people 
produce  macroeconomic  returns  on  top  of  the  direct  positive  results  for  participants, 
though  they  need  some  time  to  materialise.  In  the  long  term,  youth  employment 
operations  are expected  to  provide  a boost to  overall EU GDP  and  employment  levels, 
although at a modest rate.  

At EU level, the modelling found that, in addition to the jobs covered in direct relation to 
the funded operations, 11,000 jobs are expected to have been generated by 2023  thanks 

35 

 
 
 
 
to  the  ESF  and  YEI  investments  in  youth  employment,  with  long-lasting  effects 
generated by the structural policies and the change in productivity34. The increase shows 
persistence over the long term (by 2030), indicating that GDP may be 0.06% higher than 
the baseline and over 35,000 jobs are expected to have been created.  

These  positive  but  comparatively  small  effects  should  be  read  in  the  light  of  four  key 
facts: 

 

 

 

the  overall  investment  is  modest  compared  to  GDP  (the  whole  Youth 
Employment  package  accounts  for  just  0.05%  of  the  EU’s  GDP)  and  uneven 
across regions;  
in  addition  to  the  overall  macroeconomic  effects,  the  operations  provide  direct 
benefits to those receiving support;  
these  are  just  partial  estimates  given  the  programmes  are  still  ongoing  and 
reporting inevitably lags actual performance;  

  EU  cohesion  policy  supports  investment  in  physical  capital  along  with  human 
capital, under the European Regional Development Fund. Such investments may 
have synergies with ESF/YEI investments in human capital.  

The key point here is that productivity-enhancing human capital investments ensure that 
jobs are created over the medium to long term. Although the effects may seem modest at 
EU level, they are greater in some regions, with a number of regions located in southern 
European  states  standing  to  reap  most  of  the  benefits,  which  can  be  quite  substantial 
given the scale of EU support. 

5.2.  EFFICIENCY:  HOW  EFFICIENT  HAS  THE  YEI,  AND  OTHER  YOUTH 
IN  THE 

EMPLOYMENT  OPERATIONS  FUNDED  BY  THE  ESF,  BEEN 
ACHIEVEMENT OF THEIR OBJECTIVES? 

Three aspects of efficiency of EU youth employment operations are analysed below: 

  First, the cost effectiveness of operations in delivering outputs and results, and the 

factors involved; 

  Secondly,  the  impact  of  administrative  arrangements  on  the  effectiveness  of 

support; 

  Lastly,  how  visibility  of  the  funding  and  the  operations  contributed  to  outreach 

and hence to successful delivery. 

34 The modelling work was carried out before COVID-19 pandemic. 

36 

 
 
 
 
 
                                                 
Stakeholders viewed vocational education and training, as well as paid apprenticeships, 
traineeships and internships, as the most cost-effective type of operations 

The public consultation elicited broadly similar responses for the ESF and the YEI. Most 
respondents  from  organisations  involved  in  implementing  the  ESF/YEI  agreed  on  the 
cost effectiveness of vocational education and training activities (87.6% ESF, 83% YEI) 
followed  by  apprenticeships,  traineeships  and  internships  (83.3%  ESF,  82.5%  YEI), 
basic  skills  training  (80.9%  ESF,  79.1%  YEI)  and  guidance  and  career  support  (80.4% 
ESF,  76.7%).  In  both  cases,  and  in  line  with  the  public  consultation  responses  on 
effectiveness, voluntary and community work were ranked the least cost-effective. 

Unit  costs  per  output  are  comparable  with  the  YEI  and  other  ESF  youth  employment 
operations, with significant variations between Member States 

Mindful of the above limitations in terms of data availability, this evaluation resulted in 
an  average  cost  per  participation  in  ESF  8.ii  (excluding  YEI)  investments  of  €1,854, 
which  is  higher  than  the  average  unit  cost  for  Thematic  Objective  8  (Promoting 
sustainable  and  quality  employment  and  supporting  local  mobility)  overall  (at  €1,390). 
These figures are also higher than the unit costs for youth operations given in the update 
of the ex-post evaluation of ESF 2007-2013 for Access to Employment (€1,215).  

Investments  in  YEI  have  similar  average  unit  costs  at  €2,035  (EU  average).  The 
econometric  analysis  confirmed  that  there  was  no  statistically  significant  difference 
between the unit costs for YEI and ESF youth employment operations outside YEI. 

A  Commission  study35  also  concluded  that  ‘considering  the  rigorous  data  verification 
process  that  the  data  in  the  sample  has  undergone,  […]  unit  costs  for  the  unemployed 
established at EU level (Simplified Cost Options that Member States can use to  reduce 
the administrative burden of providing evidence for eligibility of expenditure) reflect the 
specific training practices in those Member States’. 

The  calculations  of  both  costs  per  participation  and  simplified  cost  operations  vary 
significantly  between  the  Member  States.  There  are  some  reasons  for  the  substantially 
higher  than  average  values  for  Germany,  Ireland  and  Sweden.  For  Germany,  the  likely 
explanation  is  that  typical  operations  in  Germany  last  longer  (12  months  and  over).  In 
Ireland,  operations  target  specific  groups,  such  as  the  socially  excluded,  long-term 
unemployed, and youth, which increases the cost of the intervention. For Sweden, factors 
are both the type and the duration of operations. 

35  Developing  ‘off-the-shelf’  Simplified  Cost  Options  (SCO)  under  Article  14.1  if  the  European  Social 

Fund (ESF) Regulation, PPMI for DG EMPL, April 2018. 

37 

 
 
 
                                                 
Note that the experience in running YEI and ESF youth employment support operations 
has become a solid basis for the Commission and Member States to work on developing 
simplified  cost  operations  for  the  integration  of  young  people  not  in  employment  or 
education36.  Developing  EU-level  simplified  cost  operations  for  this  target  group  and 
such  measures  could  be  particularly  relevant  for  transnational  cooperation  activities 
involving  several  Member  States,  as  evidenced  in  the  current  period  by  the  ESF 
Transnational cooperation network on youth mobility.  

Table 2 

Comparative table of unit costs 

ESF monitoring data 

YEI 
monitoring 
data 

YEI 
evaluations 
2018 

LMP 
database 
2017 

LMP 
database 
2017 

MS 

Overall 
unit 
cost 8.ii 

Overall 
unit costs 
TO8 

Overall 
unit costs 

Overall unit 
costs 

Overall unit 
costs 
national 

Overall unit 
costs ESF 
co-funded 

(EUR) 

(EUR) 

(EUR) 

(EUR) 

(EUR) 

(EUR) 

SCO study 
based on 
LMP 
database 

Overall 
unit costs 
active 
population 
(EUR) 

SCO study 
based on 
extrapolated 
data from 
LMP 
database 

Overall unit 
costs active 
population 

(EUR) 

BE 
BG 
CY 
CZ 
DE 
ES 
FR 
EL 
HR 
HU 
IE 
IT 
LT 
LU 
LV 
MT 
PL 
PT 
RO 
SE 
SI 
SK 
UK 
EU  

310 
 17,711 
 3,727 
- 
 4,078 
805 
 3,748 
- 
 4,925 
 1,940 
- 
 1,017 
- 
 1,184 
- 
680 
 1,958 
- 
 1,557 
 5,564 
 3,235 
- 
 2,265 
 1,854  

698 
 16,035 
 3,664 
 2,831 
 3,009 
626 
 1,498 
 2,748 
 4,136 
 2,188 
 7,717 
691 
 1,678 
 1,402 
955 
 1,551 
 1,885 
456 
 2,154 
 6,759 
 2,806 
 1,847 
 1,706 
 1,390  

278 
 1,648 
 3,088 
 5,084 
- 
 1,951 
 1,359 
 2,588 
 4,754 
 3,005 
 9,738 
 2,578 
 1,023 
- 
 2,012 
- 
 2,014 
 5,609 
715 
 3,524 
 6,449 
 1,023 
 2,257 
 2,035  

5,700 
2,500 

700 
5,900 

14,500 
4,300 
        4,000 
16,800 
3,900 
3,600 
16,000 
700 
8,700 
3,500 
2,500 
1,400 
15,100 
6,300 

1,278 

3,000 

1602 

224 
3,680 

800 
2,500 

4,800 
6,000 

5,700 
5,400 
5,500 
17,800 

5,800 
45,700 
4,200 
5,600 
3,300 
4,800 

4,200 
2,200 

3,351 
596 
2,696 
521 
6,959 
2,772 
6,274 

4,299 
1,818 
11,119 

1,359 
19,302 
756 

594 
994 
53 
7,303 
854 
424 

Sources: SFC2014, based on AIR 2018, data extracted on 6 September 2019. The figures for BG are 
subject to the same caveats as in the previous table, YEI 2018 evaluations. LMP database  
Data compiled by PPMI based on data extracted from the LMP database and provided by Member States 
for training of all unemployed 

2,064 
689 

3,676 

2,256 

583 

5,863 

36   A key task of the abovementioned Commission study is to develop EU-level off-the-shelf solutions for 
funding  transnational  mobility  of  young  people  not  in  employment,  education  or  training  (NEETs). 
The  development  of  EU-level  SCOs  will  be  based  on  the  current  funding  arrangements  under  TLN 
Mobility  network  and  on  the  collection  and  analysis  of  administrative  and  monitoring  data  from 
countries and regions that have funded such mobility programmes under ESF operational programmes. 

38 

 
 
 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
                                                 
The intensity of support is likely to be a major factor in determining the unit cost and the 
cost effectiveness of operations 

The  focus  on  outreach  to  vulnerable  and  hard-to-reach  groups  means  that  effective 
operations  require  a  significant  investment  in  identifying,  recruiting  and  supporting  – 
often  with  a  high  level  of  intensity  –  those  from  vulnerable  and  hard-to-reach  groups, 
which implies additional costs. This may further increase if inactive NEETs account for  
an increasing share of NEETs supported.  

This  can  also  be  related  to  the  fact  that  training  activities  are  also  considered  the  most 
costly  type  of  operations  (Table  3).  In  this  context,  it  should  be  underscored  that  high 
costs  are  not  in  themselves  indicators  of  low  efficiency.  Cost  effectiveness  is  also 
positive when costly operations are linked to high effectiveness for specific target groups. 

Table 3  Expenditure per entry: expenditure minimum-rate threshold of 75.1% - 

breakdown by region and type of intervention 

Work-based learning or first job 
experience 
Education and training 
Prevention strategies for early 
school leaving 
Guidance and support for 
individuals 
Structural support for 
strengthening institutional 
capacity  
Financial incentives to employers 
and unemployed  

Overall 

YEI 

more 
developed 
regions 

in 
transition 
regions 

less 
developed 
regions 

4,383 

4,006 

5,906 

1,000 

6,340 

7,869 
964 

1,027 

12,295  Data gaps   Data gaps  
964  Data gaps   Data gaps   Data gaps  

469 

457 

904 

537 

Data gaps  

Data 
gaps  

Data gaps   Data gaps   Data gaps  

2,344 

3,088 

2,003 

2,169 

2,585 

Source: Micro-data and own calculations, based on operations completed to 75% or more 

The influence of the socioeconomic context on the cost per participant is not significant 
for outputs, but it does have some impact on results 

According to the econometric analysis carried out, the socioeconomic context tends does 
not  make  a  statistically  significant  difference  to  the  average  cost  per  participant  by 
country. Additional findings from the econometric analysis are that: 

  costs  tend  to  be  higher  in  programmes  run  in  the  cluster  of  countries  with  a  strong 
starting  position  and  that  made  good  progress  (Luxembourg  and  Ireland,  for 
example). Costs are comparatively lower in areas with greater need of support; 
  changes in the unemployment rates are not correlated with cost per participant. 

39 

 
 
 
 
  
 
 
The  costs  of  measures  are  not  directly  related  to  the  characteristics  of  the  target 
population  

According to the econometric analysis, overall, the composition of the target population 
(different  shares  of  inactive,  low-skilled,  disadvantaged  participants,  and  shares  of 
participants  from  minorities  and  with  disabilities)  is  not  strongly  correlated  with  unit 
costs. The exception is for programmes that target a high share of inactive participants, 
which  show  statistically  significantly  lower  unit  costs.  This  finding  would  appear  to 
contradict37 other sources that highlight more intense levels of provision for those young 
people from more disadvantaged backgrounds. For instance, most case studies highlight 
higher  unit  costs  for  hard-to-reach/disadvantaged  groups  (especially  the  long-term 
unemployed) for comparable forms of support. 

Administrative requirements have facilitated implementation, with some exceptions 

Overall,  respondents  had  a  positive  perception  of  the  administrative  requirements  and 
organisational  arrangements  to  implement  youth  operations.  Most  respondents  to  the 
public  consultation  (300  responses  from  organisations)  found  the  administrative 
arrangements for the implementation of projects to be appropriate (see Figure 13).  

The  types  of  administrative  requirements  more  often  rated  as  appropriate  are  those 
related  to  project  implementation  (65.3%),  project  selection  (64%),  communication 
(62.7%),  evaluation  (61%),  reporting  and  monitoring  (55.7%),  setting  up  standard  cost 
options  (51.3%)  and  audits  (50.3%).  A  lower  score  was  given  to  the  arrangements  for 
setting up the management and control systems, with 46.3% rating it as appropriate.  

37  Students  are  by  definition  inactive,  so  when  we  find  high  shares  of  inactive  young  people  in  a 
programme it may indicate a high share of students. More broadly, by definition YEI supports people 
with  some  degree  of  distance  from  the  labour  market.  Being  low  skilled  or  being  inactive  does  not 
necessarily mean these participants are at a greater distance from the labour market. In addition, basic 
skill training and job guidance is arguably cheaper than specialised training for the high skilled. It is 
not  uncommon  to  offer  relatively  low-cost  guidance  to  inactive  participants,  to  get  them  ‘in  the 
system’. 

40 

 
 
 
 
 
                                                 
Figure 13  How would you qualify the following administrative arrangements for the 
implementation of youth employment operations under the ESF and YEI? 

Implementation of projects

Selection of projects

Communication

Evaluation

Reporting and monitoring

Setting up of standard cost options

Audit

Setting up of the management and control system

196 

192 

188 

183 

167 

154 

151 

139 

36 

23 

45 

20 

32 

56 

18 

46 

48 

32 

19 

66 

70 

15 

48 

30 

25 

91 

62 

11 

76 

84 

23 

54 

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Appropriate

Burdensome

Insufficient

I don't know

Source: Draft Final Report, Public Consultation on the Evaluation of the EU Support to Youth Employment 
by the Youth Employment Initiative and the European Social Fund 

However,  the  case  studies  identified  a  number  of  areas  for  improvement,  notably  as 
regards the requirements to provide evidence to support the eligibility of costs regarding 
the support provided to young people not in employment, education or training and the 
problems related to the initial set-up and functioning of the IT systems for the operational 
programmes.  This  was  found  to  be  a  general  weakness  in  all  cohesion  policy 
programmes. 

Respondents  to  the  public  consultation  were  asked  in  an  open  question  to  provide  any 
examples of gold plating  or excessive administrative burden that they had experienced. 
However,  the  examples  given  do  not  seem  to  be  deliberate  on  the  part  of  public 
administrations  or  ESF/YEI  operations,  but  more  due  to  inefficiencies  resulting  from 
either  work  to  implement  EU  regulations,  or  from  incompatibility  in  national  systems. 
Essentially, both the public consultation and the case studies noted improvements since 
the previous programming period and support for the EU’s simplification agenda, which 
will take time to deliver all of the anticipated benefits. 

Reporting and monitoring requirements are complied with, but delays are common  

The  respondents  had  a  generally  positive  perception  on  this  aspect,  after  the  initial 
transition period. Beneficiaries comply with the Commission’s reporting and monitoring 
requirements  and  systematically  use  the  system  for  fund  management  for  reporting 
purposes, in line with the guidance provided and the regulation.  

41 

 
 
 
 
Compared  to  the  previous  period,  the  current  ESF  Regulation  2014-2020  sets  out 
common  output  and  result  indicators  (including  those  setting  a  common  approach  to 
monitoring/reporting on the sustainability of results through systematic measurement of 
longer-term results for participants) and the Commission has issued detailed definitions, 
guidance  and  support  for  setting  up  monitoring  systems.  Member  States  and  the 
Commission  use  this  framework  to  report  progress,  notably  on  the  support  provided  to 
young and disadvantaged people.  

The  main  issues  are  under-reporting,  difficulties  in  measuring  soft  outcomes  and  in 
capturing  the  long-term  benefits  of  YEI/ESF  youth  employment  operations  after  the 
projects end. In addition, there is a significant time lag between reporting outputs, results 
and  financial  implementation,  as  a  result  of  data  checking  and  reporting  requirements. 
Costs are declared only after the authorities carry out checks. Outputs are reported often 
only once operations are completed and declared, with results reported later still.  

The  reporting  of  indicators  focusing  on  employment  and  on  the  formal  qualifications 
gained typically underestimates the total benefits to society resulting from active labour 
market operations, for example the benefits to health and levels of crime. The managing 
authorities  recognise  the  value  of  soft  outcomes,  especially  for  disadvantaged  groups 
with  multiple  problems  and  they  are  said  to  be  highly  valued  by  participants  in  the 
context of youth employment operations. The value is generally in the social skills built 
up  (self-esteem/confidence,  cultural  and  community  interaction  and  interaction)  and  in 
reliability, 
that  employers 
the  non-vocational  skills  sets 
honesty/discipline,  time  keeping  and  personal  organisation).  However,  Member  States 
seldom  monitor  soft  outcomes.  This  may  be  due  to  difficulties  in  devising  a  consistent 
measurement  method  and  in  verifying  recorded  data.  Moreover,  soft  outcomes  are 
specific  to  each  operation,  which  makes  them  unsuited  for  tracking  by  common 
indicators.   

look  for  (including 

Managing  authorities  are  aware  of  the  value  of  capturing  results  after  a  period  of  time 
(six, 12 or more months after participation) and the value of tools such as ad hoc surveys, 
but practice varies. 

From  an  evaluation  perspective,  and  for  the  purpose  of  making  a  pan-EU  comparative 
analysis,  managing  authorities  should  be  able  to  use  national  data  sets  for  econometric 
analysis  or  to  track  individuals  (e.g.  through  tax  and  national  insurances  numbers  or 
surveys, especially for tracking results after a participant has left an operation). However, 
this raises data privacy issues, in particular since the entry into force of the GDPR38. 

38 General Data Protection Regulation. 

42 

 
 
 
 
                                                 
Visibility is an issue 

The visibility of EU funding is not only a legal requirement but also an important aspect 
of youth employment operations, since increasing awareness  contributes to reaching the 
target  population.  Therefore,  good  visibility  is  key  to  boosting  the  efficiency  of 
operations. 

Visibility  covers  both  general  awareness-raising  and  specific  communication  on 
activities. On both counts, there is room for improvement for the ESF and even more for 
the YEI. The public consultation, for instance, revealed that some respondents who had 
received EU funding for youth employment operations were not aware of the ESF or the 
YEI. 

Communication activities are essential in raising awareness. More work seems warranted 
to  find  effective  channels  to  reach  the  target  population.  This  appears  clearly  in  the 
difference between the demand shown in responses to the public consultation for ‘new’ 
forms of communication and ‘old-style’ forms. 

Most respondents viewed social media campaigns to be the best communication channel 
(76.9%),  followed  at  a  distance  by  youth  networks,  clubs  and  online  groups  (42.7% 
overall) and word of mouth (36.5% overall). However,  most participants had found out 
about the support measures they had received via employment services and employment 
info centres (52.9%), and much fewer by word of mouth, family or friends.  Only 26.2% 
had found out about these measures via social media.  

5.3.  RELEVANCE:  HOW  RELEVANT  IS  THE  YEI  AND  THE  OTHER  ESF-FUNDED 

YOUTH EMPLOYMENT OPERATIONS? 

The  following  section  refers  to  the  extent  to  which  the  YEI  and  ESF-funded  youth 
employment  operations  address  the  needs  of  young  people,  in  particular  young  people 
not  in  employment,  education  or  training.  Given  the  changing  socioeconomic  context, 
especially the fall in the youth unemployment rate, this section also provides an analysis 
of how the implementation solutions have adapted to the changing needs.   

Youth  employment  operations  funded  by  the  ESF  and  the  YEI  addressed  the  most 
relevant needs and groups 

In line with the purposes of the Youth Guarantee, the YEI was created to provide support 
for  young  people  not  in  employment,  education  and  training  to  help  them  access  the 
labour market. These needs were not only formulated in terms of providing jobs, but also 
in terms of developing their professional and personal skills to help young people make 
the  transition  to  employment.  The  country-specific  recommendations  issued  by  the 

43 

 
 
 
 
Commission  to  the  Member  States  between  2014  and  2018,  though  not  exhaustive, 
provide a good overview of the main needs at the beginning of the programming period: 

Improving basic training and skills; 

 
  Reaching  out  to  inactive  young  people  and  helping  them  find  the  way  back  to 

employment; 
Providing counselling and mentoring;  
Facilitating the transition from school to work; 

 
 
  Offering  quality 
employment; 
Providing incentives to employers for hiring young people. 

 

traineeships  and  apprenticeships  as  a  first  step 

towards 

The  bulk  of  expenditure  for  ESF/YEI-funded  operations  is  channelled  to  work-based 
learning,  education  and  training  in  professional  skills,  guidance  and  support  for 
individuals  and  combined  operations  as  pathways  to  employment.  For  the  YEI,  a 
substantial  share  (26%)  has  been  allocated  to  financial  incentives  to  employers.  The 
range  of  operations  funded  confirm  that  programmes  were  aligned  with  the  identified 
needs of young people. 

Box 2 

Special support for hard-to-reach young people in Belgium 

The Wallonia-Brussels AIR for 2018 in Belgium identified the difficulties experienced 
by  young  people  not  in  employment,  education  or  training  and  participating  in  YEI 
operations.  They  include  their  low  skills  and  competence  levels,  lack  of  motivation, 
instability, problems linked to multiple disadvantages (e.g. access to housing, health, 
mental health problems and family problems). These issues require special support, 
such  as  individual  and  constant  guidance,  adaptation  and  close  collaboration  with 
specialist  institutions  (e.g.  medical  centres,  mental  health  centres,  diagnoses  of 
learning  difficulties,  emergency  accommodation)  to  prevent  the  young  people  from 
dropping out of operations. There is often a problem with attendance (absences and 
late arrivals).  

The  Wallonian  ‘Sac  a  dos’  project  finances  several  activities  carried  out  by  the 
beneficiary.  It  includes  individual  guidance  and  counselling,  workshops  to  help 
participants  improve  their  self-confidence  and  self-esteem  and  developing  personal 
skills (such as autonomy, living in a group, expressing oneself) and short projects (of 
about four days) where young people can give back to the community and discover 
or  improve  specific  techniques  in  a  range  of  sectors  (such  as  in  construction,  or  in 
the  ‘green  sector’).  It  also  funds  short  training  sessions  (minimum  two  weeks)  in 
several  businesses  (such  as  in  a  flower  shop  or  a  fitness  centre)  to  give  the  young 
people  work  experience.  In  2018,  the  project  provided  assistance  to  50  young 
people,  with  the  number  rising  each  year  (especially  among  homeless  young 
people).  There  are  no  official  statistics  tracking  the  situation  of  young  people  once 
they have left the project, but it would appear that most go back to school or enrol 
in a vocational training course. 

44 

 
 
 
 
 
 
Stakeholders  and  participants  rated  the  youth  employment  operations  as  relevant  and 
useful 

During the public consultation, participants reported that training to build general skills, 
support  in  overcoming  barriers  to  work/training,  and  training  to  obtain  qualifications 
were the most useful types of operations. 

Most  organisations  (over  90%)  that  responded  to  the  public  consultation  consider  the 
following  actions  as  relevant  or  very  relevant  to  help  young  people  find  quality  and 
sustainable jobs: vocational-focused education and training, guidance and career support, 
paid apprenticeships/traineeships/internships and basic skills training. 

By  contrast,  they  ranked  support  to  set  up  a  business  (71%),  find  a  job  or  opportunity 
abroad (70.3%) and support to get back to school (64.8%) as relatively less useful. The 
contributing  organisations  considered  community  or  voluntary  work  and  non-
remunerated apprenticeships to be the least relevant (47.5%). 

Gender  issues  are  taken  into  consideration  initially,  but  there  is  a  lack  of  active 
monitoring and targeting  

In general, programme documents contain detailed strategies on how attention to gender 
equality  is  mainstreamed.  The  monitoring  systems  meet  the  obligation  to  disaggregate 
data collected by  gender but  often  did  not  define gender-specific indicators that extend 
beyond levels of participation and provide results from a gender perspective. The same 
applies to the targets. Although all Member States report outputs and results by gender, 
only 41% of the output indicators focusing on youth employment with targets are gender-
specific  targets  and  measure  whether  implementation  meets  with  the  programme’s 
ambitions for gender equality. In terms of results, this is slightly lower as only 28% of all 
result  indicators  are  gender-specific  targets,  corresponding  to  fewer  than  half  of  the 
operating programmes. 

Only  a  small  number  of  operating  programmes  (in  Austria,  Spain  and  Italy)  included 
dedicated  youth  employment  investments  in  gender  equality  (IP  8.iv),  accounting  for  a 
total amount of €18 million. Other ESF/YEI operations (IP 8.ii) generally tend to focus 
on gender equality as a cross-cutting issue.  

The operations did not always reach the most vulnerable groups 

The strict eligibility criteria for  young people not  in  employment, education or training 
mean  that  some  vulnerable  (young)  people  groups  were  not  directly  targeted  by  YEI 
operations,  and  sometimes  not  reached  by  ESF  operations  either.  In  focus  group 
discussions,  participants  noted  that  YEI  currently  does  not  allow  youth  employment 
operations  to  focus  on  prevention  work  targeting  groups  at  risk,  for  example  potential 
early  school  leavers.  Some  countries  were  in  favour  of  extending  the  definition  of  the 

45 

 
 
 
target  population  (beyond  those  still  at  school)  such  as  Italy  where,  due  to  severe 
unemployment in less developed regions, there is a need to extend the support to people 
up to the age of 34  years. In addition, some countries underlined the complexity of the 
NEET concept, as it requires verifying negative conditions. 

Programmes sought to  address the most relevant and vulnerable groups  by taking steps 
such as: 

 
 
 

defining specific output targets (e.g. Spain, Luxembourg, Romania); 
creating mechanisms to identify the most disadvantaged groups (e.g. Malta); 
putting a specific focus on disadvantaged young people (e.g. Germany, Spain and 
Greece). 

There are various ways in which operational programmes under the ESF address the most 
relevant  target  groups  and  tackle  disadvantage.  There  are  no  strict  eligibility 
requirements  under  the  ESF  for  participation,  so  funding  can  be  allocated  to  any  such 
target  groups,  either  under  the  dedicated  investment  priority  for  youth  employment,  or 
under broader types of objectives and investment priorities.  

Support for youth employment focused on regions most in need of support 

The budget allocation to different clusters of regions, as presented in Figure 14 overleaf, 
clearly shows how YEI focuses predominantly on regions where youth unemployment is 
most problematic (72% of the budget was allocated to regions in a very weak situation at 
the start of the process and that made limited economic progress).  

The  Rhomolo  modelling  study  showed  that  some  regions  (particularly  in  southern 
Europe) that depend  greatly on youth employment reap much of the benefits of YEI. It 
also highlighted that the allocated YEI amounts are proportionately higher in regions that 
have much higher youth unemployment rates, NEET rates or social exclusion and at-risk-
of-poverty rates.  

ESF  funds  are  spread  across  different  clusters  of  regions,  but  the  fact  that  almost  no 
youth  employment  operations  are  allocated  to  regions  that  had  favourable  youth 
employment figures in 2014 and that had improved further over recent years shows how 
ESF funding has been targeted to regions where youth employment is most problematic.  

When  combining  ESF  and  YEI  funding,  the  regions  with  the  highest  youth 
unemployment figures also received over half of all the youth employment budget (56%). 
Regions that also had high youth unemployment at the start of the programming period 
but that improved over time were the second largest beneficiary of funding (28%).  

46 

 
 
 
 
 
Figure 14 Overview of budget allocation to clusters of regions 

Source: SFC2014, based on AIR 2018, data extracted on 6 September 2019 (Cluster A – Strong 
start/substantial progress, Cluster B – Strong start/limited progress, Cluster C – Weak start/visible 
progress, Cluster D – Very weak start/limited progress). 

Operational programmes were flexible and able to adapt to changes  

Operational programmes were able to adapt to changes stemming from the business cycle 
such as the fall in youth unemployment and the rising share of inactive young people in 
the target group, but also to unexpected developments, such as the immigration crisis.  

Most Member States set specific objectives and target groups in a relatively broad way. 
As  a  result,  minor  annual  adjustments  to  the  programming  did  not  have  to  go  through 
formal amendment procedures.  

The additional budgets made available to YEI did, however, require formal amendments 
to the programmes. They are also evidence of how the procedures were flexible enough 
to  provide  for  additional  budget  to  be  programmed  where  necessary  and  facilitated  the 
adjustment of formal targets set in programmes.  

The budgetary changes made were mainly to the volume of operations supported, without 
leading to significant changes in the programme strategy.  

Most adaptations to the programme were to accentuate the focus of regional targeting, to 
broaden target groups and the type of actions. Only in a limited number of programmes 
was  the  underlying  programme  strategy  revised  (e.g.  North  Rhine-Westphalia,  Saxony-
Anhalt or the operational programme for knowledge education development to adapt to 
the changing socioeconomic context). 

Flexibility  was  thus  a  key  factor  in  the  adaptation  of  programmes  and  cooperation 
between programme stakeholders was an essential success factor in this regard. 

47 

 
 
 
 
5.4.  COHERENCE:  HOW  COHERENT  ARE  YEI  AND  THE  OTHER  ESF-FUNDED 
YOUTH  EMPLOYMENT  OPERATIONS  AMONG  THEMSELVES,  AND  WITH 
OTHER ACTIONS IN THE SAME FIELD? 

The  ESF  Regulation  (Article 4)  requires  that  the  strategy  and  actions  set  out  in  the 
operational programmes are consistent with, and respond to:  

the challenges identified in the national reform programmes,   

 
  other  national  strategies  (where  relevant)  that  aim  to  combat  unemployment, 

 

poverty and social exclusion, and  
the  relevant  Council  recommendations,  in  order  to  contribute  to  achieving  the 
headline  targets  of  the  Europe  2020  strategy  on  employment,  education  and 
poverty reduction.  

This section looks at coherence between YEI/ESF-funded youth employment operations 
among themselves and with the national policies.  

The  design  and  programming  of  the  ESF  and  the  YEI  contributed  to  their  mutual 
complementarity 

training,  apprenticeship,  hiring 

On  the  one  hand,  YEI  was  designed  to  support  similar  operations  to  the  ESF  (such  as 
self-employment 
employment  and 
programmes),  though  it  focuses  exclusively  on  young  people  not  in  employment, 
education or training in  eligible regions. YEI is therefore technically complementary to 
the  ESF,  available  to  the  regions  most  affected  by  youth  unemployment  and  providing 
support for particularly disadvantaged young people.  

incentives, 

On the other hand, ESF also offered system support or was often budgeted as a follow-up 
measure to operations run under the YEI. 

YEI and ESF youth employment operations are complementary and coherent with other 
EU schemes 

The issue of complementarity and coherence with other EU funds in the same areas was 
barely covered in the national YEI evaluations. The public consultation showed that most 
organisations involved in running youth employment operations considered that the YEI 
and ESF were coherent with each other and with other EU schemes in similar or related 
areas. 

48 

 
 
 
 
 
Figure 15  To what extent are the YEI and ESF coherent with other youth and youth 

employment EU national/regional schemes? (n=399) 

Existing national/regional programmes

Erasmus+

172 

162 

40 

45 

142 

46 

21 

170 

EURES

124 

33 

13 

ERDF

110 

46 

18 

229 

225 

European Solidarity Corps

81 

35 

13 

270 

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

They are complementary

They are not aligned

They overlap

I don't know

Source: Final Report, Public Consultation on the Evaluation of the EU Support to Youth Employment by the 
Youth Employment Initiative and the European Social Fund 

This  point  was confirmed by the case studies  and the analysis of the policy framework 
for the ESF and other EU funding schemes.  

The  YEI  and  youth  employment  measures  under  the  ESF  are  specifically  focused  on 
access  to  the  labour  market  for  young  people  and  helping  them  become  active  on  the 
labour  market.  None  of  the  other  EU  schemes  replicate  or  unnecessarily  duplicate  YEI 
and  youth  employment  measures under  the  ESF. Some funds (ERDF, EAFRD, EMFF) 
are  geared  towards  creating  sustainable  jobs  open  to  young  people.  The  EAFRD  and 
EMFF  target  young  people  by  providing  support  for  farming  and  fisheries  ventures, 
including helping them to develop skills for these sectors. 

The  EURES  and  Erasmus+  programmes  have  a  strong  complementarity  with  YEI  and 
ESF  youth  employment  measures.  EURES  facilitates  information  sharing  on  job 
opportunities while Erasmus+ facilitates mobility and learning for young people. 

Institutional coordination is essential element to ensure coherence 

Coordination  is  key  to  ensuring  complementarity  with  other  EU  funds  or  programmes. 
Conversely, poor coordination can be an impediment to success. Although it did not find 
evidence of extensive overlaps, a study commissioned by DG REGIO found evidence of 
poor coordination only in a limited number of cases39. 

39 European Commission (2018) study on the coordination and harmonisation of ESI  funds and other EU 

instruments. 

49 

 
 
 
 
                                                 
There is overall complementarity between EU-funded youth employment operations and 
national schemes, but some cases of overlapping and replacement were reported 

According to  the results  of the public consultation,  most respondents  (43%) considered 
that  YEI/ESF  youth  employment  programmes  are  complementary  to  national/regional 
programmes. This was confirmed by the case studies run in Italy, France and Germany. 

However, the analysis of YEI evaluations showed that in a few Member States, the YEI 
programmes  sometimes  replaced  measures  that  used  to  be  financed  by  the  national 
budget, raising a question as to what extent EU resources have been used to complement 
national action and budgets. 

In  general,  there  are  clear  lines  of  demarcation.  In  some  countries  and  regions,  this  is 
facilitated  by  coordinating  partnerships  that  take  a  holistic  approach  to  youth 
employment  operations  across  EU  and  national  programmes.  Several  EU  programmes 
have a focus on young people, but none have the clear employment focus or the breadth 
of the YEI or ESF youth employment operations. 

5.5.  EU  ADDED  VALUE:  WHAT  IS  THE  EU  ADDED  VALUE  OF  THE  YEI  AND 

OTHER ESF-FUNDED YOUTH EMPLOYMENT OPERATIONS? 

To  analyse  EU  added  value,  the  evaluation  used  a  wider  framework  extending  beyond 
what  could  be  achieved  by  Member  States  at  national  and/or  regional  levels.  This 
framework  helps  assess  the  extent  to  which  the  YEI/ESF-funded  youth  employment 
operations  was  additional  to  national  funding  and  programmes,  and  the  ways  in  which 
the support helped reform and improve national systems.  

Without  YEI  and  ESF  funding,  some  operations  would  not  have  been  implemented  or 
would have been more limited 

At a time of economic  crisis,  EU  youth  employment operations  enabled  the running of 
operations that would have been difficult to achieve otherwise. According to the replies 
from the public consultation, the main effects of YEI were to strengthen approaches by 
providing extra funds, and by providing individualised support to participants. 

According  to  the  information  available  to  the  Commission  on  national  labour  market 
policies,  in  several  Member  States,  some  of  the  labour  market  measures  specifically 
designed to help disadvantaged young people (i.e. unemployed, employed but at risk and 
inactive  young  people)  were  fully  funded  by  the  ESF/YEI,  i.e.  there  are  no  equivalent 
measures  funded  only  through  national/regional  resources.  This  reflects  the  significant 
volume effect of YEI and ESF-funded operations. For instance, all apprenticeships were 
co-funded by ESF/YEI in Ireland, Spain, Portugal and Finland. This was also the case for 

50 

 
 
 
 
traineeships  in  Bulgaria,  Greece,  Latvia,  Malta,  Portugal  and  Slovenia,  for  institutional 
training  in  Estonia,  Ireland,  Latvia,  Malta,  Portugal  and  Slovenia,  and  for  employment 
incentives in Estonia, Cyprus, Lithuania, Spain and Slovakia.  

The case studies provided further examples where YEI and other ESF-funded operations 
produced effects at national and regional level that would not have been achieved without 
EU support.  

Box 3 

Common standards and tools developed in YEI programmes  

A particularly striking example is the national YEI evaluation report for Bulgaria. If it were 
not for YEI funding, the level of youth unemployment and inactivity in the country would 
have  been  significantly  higher:  nearly  4,000  young  people  would  be  out  of  the  labour 
market, approximately 9,000 people would have not received further vocational training, 
19,000 would have not increased their qualification through internships, and some 26,500 
would have been out of employment. 

YEI  and  ESF-funded  operations  covered  target  groups  that  were  not  covered  by  other 
national or regional interventions 

National  evaluations,  case  studies  and  the  public  consultation  provide  evidence  of  the 
fact that the YEI enabled an expansion and refining of the scope of public employment 
operations. The YEI and the ESF brought to the forefront the needs of the target group 
(the NEETS in high youth unemployment regions) which would not have been a specific 
focus under other funding schemes. 

Box 4 

‘From  training  to  employment’  –  the  Voluntary  Labour  Corps  project  in 
Poland 

The Voluntary Labour Corps runs daily activities for the benefit of young people. As part 
of the project, it was possible to test the entire support path from educational activities to 
employment,  enriched  with  new  activities  to  increase  the  effectiveness  of  support  and 
ensure that the employment goal is met. Comprehensive support provided as part of the 
project (broader than the support provided in standard actions), improved the work with 
young  people.  The  experience  acquired  fed  into  the  activities  carried  out  and,  as  far  as 
possible, will lead to an expansion in the scope of standard forms of support.  

It was important to reach young people in the most difficult situations, who cannot access 
standard  activities,  and  to  work  with  them  every  day.  To  achieve  this,  each  local 
coordinator looked after approximately 10 people, which required 24/7 engagement. They 
were in constant telephone contact with them, often taking them to classes, making sure 
that they did not give up the project. This is one of the most difficult aspects of this job, 
because  it  is  associated  with  upbringing  and  teaching  social  behaviour,  with  the  aim  of 
helping  young  people  to  function  independently.  Equally  important  was  to  strengthen 
organisational structures at local and national level. The Voluntary Labour Corps has since 
gained  recognition  among  young  people  as  an  institution  providing  information  on  the 
support available to them. 

51 

 
 
 
 
 
 
The project led to the development of intensive, and sometimes tailor-made, approaches 
to target groups (vulnerable young people in different contexts), who might not otherwise 
have  received  such  specific  services.  It  also  raised  the  level  of  ambition  of  support 
objectives. 

Though rather limited information is available on the possible effects of the operations, 
in  some  countries  it  is  recognised  that  the  YEI/ESF  helped  to  raise  awareness  about 
young  people  not  in  employment,  education  or  training  to  pay  specific  attention  to 
helping this target group. 

The positive trends in NEET rates up until early 2020 are likely to be reversed as a result 
of  the  ongoing  COVID-19  pandemic.  Support  to  young  people  not  in  employment, 
education or training will continue to be at the core of the anti-crisis measures currently 
running  in  the  Member  States  with  funding  from  the  current  programmes,  notably  the 
ESF, as well as the future ESF+. As those instruments invest in both people and systems, 
they provide comprehensive support to respond to the challenges. 

Outreach  to  and  specific  measures  for  this  target  group  have  been  identified  as  a  key 
policy areas where Member States will step up their action. They will continue to learn 
from  each  other  in  the  context  of  the  Council  Recommendation  on  a  Bridge  to  Jobs  – 
reinforcing the Youth Guarantee, which the Commission proposed on 1 July 202040.  

Box 5 

Common standards and tools developed in the YEI programme  

In  Italy,  the  national  YEI  operational  programme  was  highly  innovative,  in  terms  of  the 
new  method  of  collaboration  brought  in  by  the  central  and  regional  authorities,  and  the 
new  common  standards  and  tools  that  have  now  become  a  legacy  of  the  programme. 
Most  of  the  tools  developed  were  mainstreamed  in  the  2015  reform  of  employment 
services  and active  labour  market policies.  The  national YEI operational  programme  also 
spearheaded  the  involvement  of  private  employment  services  and  supported  the 
development  of  the  active  labour  market  policy  system.  It  contributed  to  boosting  the 
capacity  of  the  public  employment  service,  in  particular  in  terms  of  the  number  of 
registered participants. 

 EU-funded youth employment operations have supported the implementation of national 
Youth Guarantee schemes 

The YEI was  created with  the political  will  and aim  ‘to  complement other ESF-funded 
operations and national actions targeting NEET, including through the implementation of 
the Youth Guarantee’41. 

40 https://ec.europa.eu/social/main.jsp?langId=en&catId=89&newsId=9719&furtherNews=yes 

41 Recital (11) of Regulation 1304/2013. 

52 

 
 
 
 
                                                 
YEI and ESF-funded operations run in the context of the Youth Guarantee appear to have 
had a positive impact on employment results. According to information collected by the 
European  network  of  Public  Employment  Services,  from  2016-2019,  ESF  funding  was 
used in  at  least  23 Member States to  support the implementation  of the  national  Youth 
Guarantee  schemes,  and  YEI  funding  was  used  in  at  least  19  Member  States.  In  11 
Member States, ESF was the main source of funding of the Youth Guarantee. The YEI 
was identified as the main source of funding of the Youth Guarantee in two countries.  

Table 4  Use of ESF and YEI funds in implementing national Youth Guarantee 

schemes, 2016-2019  

MS 

BE 
BG 
CZ 
DK 
DE 
EE 
IE 
EL 
ES 
FR 
HR 
IT 
CY 
LV 
LT 
LU 
HU 
MT 
NL 
AT 
PL 
PT 
RO 
SI 
SK 
FI 
SE 
UK 
Total 
Main source 

2016-
2017 
 
 
 
N/A 
N/A 
 
N/A 
 
N/A 
 
 
 
 
 
 
N/A 
 
 

 
 
 
 
 
 

N/A 
19 
11 

ESF 
2017-
2019 
 
 
 
N/A 
N/A 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
N/A 
23 
15 

2016-
2019 
 
 
 
N/A 
N/A 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
N/A 
23 
11 

2016-
2017 
 
 
 
N/A 
N/A 

YEI 
2017-
2019 
 
 
 
N/A 
N/A 

2016-
2019 
 
 
 
N/A 
N/A 

N/A 
 
N/A 
 
 
 
 
 
 
N/A 
 

 
 
 
 
 

N/A 
16 
4 

 
 
 
 
 
 
 
 

N/A 

 
 
 
 
 

 
N/A 
17 
3 

 
 
 
 
 
 
 
 
 
N/A 
 

 
 
 
 
 

 
N/A 
19 
2 

 ESF/YEI funds are used to fund the national Youth Guarantee scheme.  
 ESF/YEI are identified as the main source of funding of the national Youth Guarantee scheme. 
N/A: information not available 
Notes: 
BE: between 2017-2019, BE-Actiris identified ESF as the main source of funding for the YG. 
DK: No centralised national data on the specific amount of funds allocated to the Youth Guarantee.  
FR: ESF funds were available from 2014 to 2020 and funded 50% of the scheme. As of the end of 2016, 
more regions were covered by the ESF. The YEI funds the scheme at a 92% funding level, but only in a 
few regions and certain provinces in regions already covered by the ESF. 
Source: European Commission, Report on PES Implementation of the Youth Guarantee. The report of 
September 2017 covers the period from spring 2016 to spring 2017, while the report of September 2019 
covers the period from spring 2017 to spring 2019.  

53 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Aggregate  monitoring  data  on  the  Youth  Guarantee  and  YEI  for  the  period  2014-2018 
indicate  that  one  in  six  (16%)  participating  young  people  aged  15-29  who  received  a 
Youth  Guarantee  offer  (of  employment,  education,  an  apprenticeship  or  a  traineeship) 
had received support under the YEI. There is only a small difference between age groups: 
15.5%  of  offers  were  taken  up  by  young  people  aged  15-24  and  17.5%  of  offers  were 
taken  up  by  those  aged  25-29  (includes  only  the  countries  that  expanded  the  Youth 
Guarantee to also cover this age group). In Spain, Cyprus, Latvia, Lithuania, YEI outputs 
account  for more than one in four Youth Guarantee offers, and in  Italy  more than half. 
By  contrast,  YEI  outputs  account  for  fewer  than  one  in  ten  Youth  Guarantee  offers  in 
Czechia,  Romania  and  Slovenia.  In  total,  the  cumulated  YEI  outputs  reported  between 
2014  and  2018  account  for  over  half  (55.1%)  of  all  subsidised  Youth  Guarantee  offers 
(i.e. those provided using public funds targeted either directly at  young people or more 
broadly to the unemployed).  

The  YEI  seems  to  have  been  particularly  heavily  used  to  (co)fund  Youth  Guarantee 
offers.  It  was  used  to  cover  at  least  80%  of  all  subsidised  Youth  Guarantee  offers  in 
Belgium,  Greece,  France,  Italy,  Latvia  and  Lithuania,  though  only  marginally  used 
(<10%  of  subsidised  Youth  Guarantee  offers)  in  Czechia,  Romania  and  Slovenia.  This 
does not mean that the Youth Guarantee was underfunded in these countries, rather that 
the subsidised offers were mainly funded from other sources (ESF/national).  

5.6. 

SUSTAINABILITY: HOW SUSTAINABLE ARE THE YEI AND THE OTHER ESF-
FUNDED YOUTH EMPLOYMENT OPERATIONS? 

According to the information available, the rate of participants making the transition to 
employment  in  YEI/ESF  operations  generally  improved  over  time.  The  impact  on 
employment is sustainable for low-skilled participants, who were the main focus of the 
operations, and for medium and high-skilled participants.  

Continuity of EU-supported operations after the funding stops is largely dependent on the 
availability of alternative funding. Nevertheless, the systemic changes achieved through 
the  ESF/YEI  operations  are  likely  to  remain  independently  of  the  funding.  From  a 
macroeconomic  perspective,  the  impact  of  the  operations  on  GDP  and  particularly  on 
employment are expected not only to last but even to increase over the medium to long-
term, peaking in 2026-2030. 

The effects of the operations for participants seem to last and increase over time 

Participation  in  YEI  and  ESF  youth  employment  operations  would  appear  to  have  a 
positive impact on the employability of participants, becoming more visible over time.  

The national YEI evaluations indicated that in most cases, the employment rate increased 
over this period.  

54 

 
 
 
Table 5 

Employment rates by type of operation, upon completion and after 6 
months  

Type of measure 

Education/training 

Recruitment incentives 

Support for 
entrepreneurship 
Guidance services 

Combined activities 

Employment rate  
immediately after exit 

·  BE-Brussels (VDAB 
training) 
·  IE (Youthreach) 
·  PT (Internships) 
·  IE (JobsPlusYouth) 
·  PT (Hiring support) 
·  IE (BTWEA) 

·  BE-Brussels (YEI NL 
guidance) 
·  LT (Find Yourself) 
·  LT (New Start) 

Employment rate  
6 months after 
exit 

37.5% 

65.5% 

10% 
48% 
68% 
79% 
78% 

9.5% 

13-20%* 
17-24%* 

14% 
63% 
54% 
80% 
77% 

40.7% 

23-43% 
24-37% 

Source: 2018 YEI evaluation reports 
Note: The shares given for Lithuania vary according to the target groups (young people who are 
economically inactive, close to the labour market or far from the labour market) 

The  counterfactual  impact  evaluations  carried  out  by  the  Member  States  back  up  this 
finding. In Italy for example, the counterfactual impact evaluation concludes that 54.7% 
of young people who participated in YEI are in employment 18 months after completing 
the  programme  (compared  to  42.4%  of  their  peers  who  did  not  participate).  The 
Counterfactual Impact Evaluation for France highlight the value of the Youth Guarantee, 
estimating  that  at  18  months  the  employment  rate  for  those  on  the  Youth  Guarantee 
would  be  33.4%,  against  24.3%  for  those  who  were  not.  In  Spain,  the  proportion  of 
participants in ‘Training and apprenticeship  contracts’ who were in  full-time temporary 
contracts after 18 months, was larger among those supported by the YEI than for those 
not  supported  (i.e.  the  control  group  -  68%  against  52%.  The  Counterfactual  Impact 
Evaluations  for  YEI 
in  Portugal  conducted  by  JRC  (Competence  Centre  on 
Microeconomic  Evaluation)42  shows  that  individuals  participating  to  YEI  interventions 
have a higher probability of being employed 36 months after the intervention starts. The 
effect  increases  with  the  duration  of  the  intervention  and  is  stronger  for  hiring  support 
schemes  relative  to  internships.  More  specifically,  the  average  effect  on  employment 
ranges between 7.7 and 31.7 percentage points. On average, individuals participating in 
YEI  activities  earn  between  145  and  313  euros  more,  respectively  for  the  shortest 
internship and for internship plus hiring support, 3 years after the program start. 

42  See  Duarte,  N.,  Geraci,  A.,  Granato,  S.,  Mazzarella,  G,  Mortágua,  M.  “The  evaluation  of  the  Youth 
Employment Initiative in Portugal using Counterfactual Impact Evaluation methods”, JRC Technical 
Report nr. 120942. 

55 

 
 
 
 
 
 
 
 
                                                 
The  case  studies  indicate  that  the  young  participants  gained  a  better  knowledge  of  the 
labour market and improved their skills and employability. 

Box 6 

The impact of training on the employment rate in Italy 

The  national  operational  programme  ‘Youth  Employment  2014-2020’  ran  in  all 
regions  (except  the  Province  of  Bolzano).  It  targeted  young  people  not  in 
employment,  education  or  training  aged  15-29  years  and  since  end-2017,  it  also 
covers  young  unemployed  people  in  less  developed  and  transition  regions.  The 
programme  involved  providing  extra-curricular  traineeships,  training,  community 
service and accompanying paths to support self-employment.  

Some 56% of participants had  an upper  secondary  education, 24%  had a low level 
of secondary education and 19% had completed higher education. In the programme 
as  a  whole,  the  average  effect  of  the  treatment  increases  over  time  since  the 
participants started receiving support. Participation in the programme means that in 
the  first  semester  lock-in  effects  are  dominant  and  the  employment  rate  of 
participants is lower than the control group. 18 months after the start of the support 
(approximately  one  year  after  the  end  of  the  support),  the  average  impact  on  the 
employment  rate  of  young  people  under  30  years  is  positive,  ranging  from  an 
increase  of  4  percentage  points  (in  northern  regions)  to  17.1  percentage  points  (in 
central regions).  

Changes to employment policies seem likely to last if the funding continues 

The  change  of  scope  and  the  more  targeted  approach  to  the  needs  of  young  people, 
especially those in a vulnerable and disadvantaged situation, seems to have had positive 
effects  and  contributed  substantial  added  value.  However,  programme  continuity  can 
only be guaranteed if sufficient funding is made available in the future. 

The  analysis  revealed  that  many  activities  funded  under  the  ESF  would  not  continue 
without  the  funding,  particularly  where  national  funding  is  scarce  (as  it  is  in  southern 
Europe).  However,  Spain  has  examples  of  programmes  continued  even  without  EU 
funding  where  stakeholders  have  integrated  operations  into  their  core  activities,  or  in 
Italy  where  the  new  partnership  approach  was  implemented  at  national  level/  regional 
level, involving private and public employment  services, and most importantly creating 
an  integrated  approach  between  employers  and  employment  services  and  local 
organisations. In addition, where ESF/YEI funds strengthened and extended the scope of 
existing measures, this tended to ensure sustainability (in Germany, France and Malta). 

56 

 
 
 
 
 
  
 
 
6.  CONCLUSIONS AND LESSONS LEARNED  

The analysis carried out for the evaluation leads to the conclusion that YEI and other ESF 
youth  employment  operations  have  been  relevant  in  terms  of  meeting  the  needs  of  the 
target  population  and  effective  in  providing  solutions  that  addressed  those  needs,  in 
particular  for  young  people  not  in  employment,  education  or  training.  It  also  indicates 
that,  overall,  the  operations  boosted  the  employability  of  participants,  which  may  help 
them enter the labour market at a later stage.  

This  is  in  itself  a  positive  outcome.  It  links  with  the  objective,  set  in  the  Youth 
Guarantee,  of  helping  young  people  integrate  into  and  remain  in  the  labour  market, 
especially those further from the labour market.  

A  key  challenge  for  the  evaluation,  being  carried  out  at  an  intermediate  stage  of 
implementation,  is  that  the  programmes  evaluated  are  still  ongoing  and  a  series  of 
operations are incomplete (therefore the data on outputs and results are also incomplete).  

6.1. 

OVERALL CONCLUSIONS 

As  a  general  conclusion,  the  study  finds  that  YEI/ESF  youth  employment  operations 
have helped improve the employability of young people across Europe. Some aspects 
of  the  operations  were  particularly  relevant,  such  as  innovative  methods  to  reach  the 
target population, coordination between partners, and holistic approaches to employment. 

Based on the evidence available, it seems justified to conclude that, without EU support, 
fewer  young  people  not  in  employment,  education  or  training  would  have  received 
support, and the overall NEET population would have been higher. The rate of YEI/ESF 
participants making the transition to employment generally improved over time. 

In  addition  to  achieving  good  results  in  terms  of  access  to  employment,  employment-
focused education and training, and self-employment, the operations boosted the overall 
employability of participants, notably  by increasing their self-esteem, building positive 
attitudes to employment, and building their competencies, ensuring that the young people 
receiving support do not return to the NEET group. YEI also helped meet the information 
needs  of  participants,  especially  where  operations  worked  closely  with  the  public 
employment  service,  helping  to  ensure  that  young  people  have  better  access  to  the 
opportunities available.  

In the vast majority of cases, the operations reached the target groups they aimed to 
reach,  which  expanded  over  the  period  to  cover  the  25-30  age  group.  Despite  the 
challenge  of  reaching  the  target  groups  furthest  away  from  the  labour  market, 
several outreach approaches were successful, many of them based on using social media.  

57 

 
 
 
 
Due to its initial frontloading, YEI performed better than the ESF-programmed funding 
for  youth  employment  (e.g.  52%  financial  implementation  against  27%  for  ESF  youth 
employment  operations  at  the  end  of  2018).  Taking  into  account  the  learning  curve  in 
implementing  targeted  support  to  NEETs,  the  eligibility  of  YEI  was  extended  until  the 
end of the multiannual budget period and additional funds were budgeted for 2017-2020 
to allow support to be maintained in the regions  with high youth unemployment and to 
reach the targets set.  

The organisations involved in ESF/YEI considered paid apprenticeships to be the most 
effective type of measure, together with vocational education and training and basic 
skills  training.  The  latter  are  considered  useful  for  most  target  groups,  including  the 
most disadvantaged, such as young people affected by poverty, young people who are at 
risk  of  social  exclusion,  young  migrants,  young  homeless  people  and  young  people 
leaving education without a qualification. Several Member States provided support in the 
form  of  support  pathways.  This  proved  to  be  effective  approach,  in  particular  for 
participants  further  away from  the labour market,  who need preparatory support  before 
they can make the transition to work. 

The  average  unit  cost  per  participation  is  around  €2,000  with  significant  variations  in 
costs  and  unit  costs  between  types  of  measures,  and  between  Member  States.  The  cost 
per  participant  does  not  vary  significantly  between  YEI  and  ESF  youth  operations. 
However, cost effectiveness is not determined by costs alone.  Vocation education and 
training  can  involve  high  costs,  but  these  measures  have  shown  to  be  efficient  in 
different contexts, when linked to work experience, thereby justifying the higher cost. By 
contrast,  guidance  is  provided  at  a  relatively  low  cost,  but  if  it  is  not  tailored  to  the 
individual, it is generally less effective in terms of generating employment results. 

In some cases, the support provided by YEI and the ESF contributed (indirectly, for the 
YEI,  as  it  targeted  individuals  and  not  systems),  to  structural  changes  in  national 
systems of education and training and public employment systems. The influence on 
youth  policies  is  perhaps  more  evident  as  YEI  and  ESF  helped  raise  awareness  of  the 
challenges that young people face in accessing training and the labour market, especially 
those further away from the labour market, and to create interventions to support them. 
The ESF/YEI has brought about some significant changes in the strategic approach of 
public  employment  services  and  other  bodies  to  youth  unemployment,  including 
cooperation  between  the  multiple  bodies  involved  and  adjustments  to  the  services  they 
provide. 

The administrative arrangements facilitated implementation, with  few examples of gold 
plating.  Bringing  in  the  simplified  costs  options  system  helped  reduce  administrative 
burden, after initial delays and after building capacity in setting up the system. 

58 

 
 
 
Managing authorities needed time to take on new monitoring and reporting requirements 
and faced delays in setting up online monitoring systems and databases. Monitoring and 
reporting  arrangements  are  now  operational.  There  are,  however,  information  gaps 
and  delays  in  reporting,  notably  as  regards  outputs  and  results.  This  is  due  in  part  to 
checks carried out before declaring and reporting costs and performance indicators.  

YEI  and  the  other  ESF-funded  youth  operations  are  relevant  to  meet  the  needs  of 
young  people.  The  action  taken  by  the  Member  States  to  meet  these  needs,  combined 
with a general economic uplift following the crisis, have yielded positive results. YEI and 
ESF-supported  actions  contributed  to  this  positive  development,  some  of  which  were 
found to have helped young people improve their skills and enter the labour market.  

In  addition  to  specific  YEI/ESF  actions,  the  way  in  which  they  were  designed  and 
implemented  (the  YEI/ESF  approach  or  strategy)  is  also  relevant,  for  instance,  they 
started with a needs analysis and focused on targeting or step-by-step implementation. 

As  a  consequence  of  the  design  of  YEI  operations,  support  for  youth  employment 
focused on the regions with the highest youth unemployment. The relevance of the youth 
employment  operations  is  also  evident  in  the  capacity  of  programmes  to  adapt  to 
changing  conditions,  such  as  the  drop  in  the  number  of  young  people  not  in 
employment, education or training, by changing budget allocations or shifting operations 
to  focus  on  certain  regions  or  target  groups  that  were  most  in  need.  In  addition,  the 
design  of  programmes  proved  to  be  flexible  enough,  in  most  cases,  to  accommodate 
any required changes without major reprogramming.  

There is, however, scope for improvement. Some of the actions were less useful, due to 
limited work to identify the needs or insufficient targeting. Some needs were difficult to 
address  due  to  limited  capacity  (e.g.  lack  of  specialised  social  workers),  eligibility 
conditions or the situation of extreme vulnerability of target groups. There is also scope 
to  improve  the  employment  offers  provided,  in  some  cases  (in  terms  of  duration  and 
wage levels). 

Evidence  shows  that  YEI  and  other  ESF-funded  youth  operations  were  coherent 
amongst themselves, notably due to the way they were designed and programmed.  The 
key  factors  that  contributed  to  this  complementarity  include  the  specific  types  of 
operations  supported,  capitalising  on  past  experience  with  similar  operations  and 
institutional  cooperation  amongst  stakeholders.  This  is  also  a  key  factor  in  achieving 
complementarity of YEI and ESF-funded youth operations with other related actions, i.e. 
actions  supported  by  other  programmes  and  funds  at  EU,  national  or  regional  level. 
Coordinating committees  play an important  role in  ensuring complementarity and there 
are interesting examples of such set-ups at regional level. 

59 

 
 
 
The  YEI  and  ESF  have  demonstrated  considerable  European  added  value.  This 
includes  volume  effects  by  supporting  interventions  that  were  not  funded  by  other 
national or regional programmes and enabling additional actions to take place. It also had 
important scope effects by widening the range of existing action and expanding target 
groups or including groups not covered by other programmes. 

Although  role  effects  were  less  evident  so  far,  YEI  and  ESF-funded  youth  operations 
were  instrumental  in  raising  awareness  of  the  situation  of  NEETs  in  the  Member 
States.  In  fewer  cases,  the  YEI  and  ESF  had  role  effects  in  terms  of  bringing  in 
innovative  actions  that  were  later  mainstreamed  into  national  youth  employment 
policies. 

YEI  has  made  a  positive  impact  on  sustainable  employment.    The  impact  on 
employment  is  sustainable  for  the  low-skilled,  the  main  focus  group  of  the  operations, 
and for medium and high-skilled young people.  

Continuity of the EU-supported operations after the funding stops is largely dependent on 
availability  of  alternative  funding.  Nevertheless,  the  systemic  changes  achieved  by 
running ESF/YEI operations are likely to be lasting, independently of the funding.  

The YEI Youth Guarantee-funded operations appear to have had a positive impact on 
employment.  According  to  information  collected  by  the  European  network  of  Public 
Employment  Services,  between  2016  and  2019,  ESF  funds  were  used  in  at  least  23 
Member  States  to  help  rollout  the  national  Youth  Guarantee  schemes,  and  YEI  funds 
were used in at least 19 Member States. In 11 Member States, ESF was the main source 
of  funding  of  the  Youth  Guarantee.  The  YEI  was  the  main  source  of  funding  of  the 
Youth Guarantee in  two countries.  Aggregate  monitoring data for the  Youth  Guarantee 
and  YEI  for  the  period  2014-2018  indicate  that  one  in  six  (16%)  participating  young 
people aged 15-29 that received a Youth Guarantee offer (of employment, education, an 
apprenticeship or a traineeship) was supported by the YEI. 

6.2. 

LESSONS LEARNED 

Effective outreach is one of the main challenges in youth employment policy   

Identifying and recruiting young people not in employment, education or training to join 
youth employment operations requires a range of innovative approaches, especially when 
the share of economically inactive young people in the target group rises.  

Giving  a  very  precise  definition  of  that  target  population  has  resulted  both  in 
opportunities and challenges.  Focusing the support of the NEET population has had a 
number  of  positive  outcomes  in  terms  of  the  public  employment  system  and  the 

60 

 
 
 
 
assistance  provided.  It  has  put  the  spotlight  on  a  specific  part  of  the  population 
particularly  affected  by  the  crisis,  but  had  fallen  somehow  off  the  radar  of  mainstream 
employment policies. It also helped create the mechanisms and processes that may help 
reach out to these young people and provide the solutions they need. 

Effective  outreach  to  young  people  was  identified  as  one  of  the  main  challenges  in 
youth  employment policy. Despite the high levels of  young people not  in  employment, 
education  or  training  in  a  number  of  countries,  it  was  difficult  to  identify  and  reach 
potential  candidates  for  youth  employment  operations.  Moreover,  the  strict  eligibility 
criteria  for  young  people  not  in  employment,  education  or  training  mean  that  some 
vulnerable  (young)  people  groups  were  not  directly  targeted  by  YEI  operations,  and 
sometimes not reached by ESF operations either. 

In  addition,  the  most  vulnerable  young  people  are  often  under-represented,  and 
potentially  under-reported.  This  is  in  line  with  the  replies  submitted  to  the  public 
consultation, which noted difficulties in reaching out to and supporting vulnerable young 
people in rural areas, those with disabilities, ex-offenders, those at risk of poverty and the 
homeless, and young people with multiple disadvantages. 

Concerning  the  gender  issue,  the  programme  documents  contain  detailed  strategies  on 
how  attention  to  gender  equality  is  mainstreamed.  The  monitoring  systems  meet  the 
obligation  to  disaggregate  data  collected  by  gender  but  often  did  not  define  gender-
specific indicators that extend beyond levels of participation and provide results from a 
gender perspective. The same applies to the targets. 

The EU-level focus group highlighted three principles for outreach, generally agreed by 
the participating Member State representatives, namely:  

 

local  round  work  to  assess  the  nature  and  extent  of  the  challenge  and  potential 
solutions geared to the local context;  

  using qualified outreach staff; and  
 

identifying innovative communication, or ‘interception’ channels to identify and 
reach potential participants. 

Although the current overall positive outlook for youth unemployment in the EU might 
be significantly affected by the ongoing COVID-19 pandemic, the lesson remains valid 
as even in times of crisis, the most in-need young people in the target group must not be 
forgotten. 

This  is  reflected in  the  Commission’s  proposal for a regulation on the European Social 
Fund  Plus  by  maintaining  the  provision  that  Member  States  support  access  to 
employment for all jobseekers, in particular young people.  

61 

 
 
 
Good cooperation and partnerships is key to success  Stakeholders involved in  youth 
employment  policies  emphasise  the  importance  of  cooperation  at  local  and  regional 
level, to both identify young people not in employment, education or training and to run 
programmes efficiently. Efficiency could be further improved by building up the capacity 
of programme authorities and social partners, as provided for in the proposed regulation 
on European Social Fund Plus. 

Both  the  case studies and the replies  to the public consultation  highlighted the positive 
effect of aligning EU and national youth employment policies in programming. In the 
same vein, the public consultation highlighted the value of cooperation between different 
types  of  organisations,  including  trade  unions,  schools,  research  centres  and  youth 
organisations.  

Regarding  outreach,  cooperation  with  youth  organisations  helped  to  identify  NEETs 
who are not registered with the public employment service or in education. Cooperation 
during  the  operation  ensures  that  it  provided  support  for  the  range  of  services  this 
population  of  young  people  needed.  It  was  also  very  helpful  to  involve  the  local 
authorities  in  driving  the  process.  Involving  local  companies  helped  raise  awareness  of 
the Youth Guarantee and generate employment offers. It also proved useful to ensure and 
formalise the participation of young people in planning activities in order to ensure that 
the operation provides the right services.  

Evidence from the case studies illustrates the value gained from working in partnership 
and conversely, the missed opportunities, when cooperation and partnership is not strong 
– especially in outreach work, as shown above. At a policy level, aligning the ESF/YEI 
with  national  and  other  youth  employment  policies  and  programmes  is  critical  in 
maximising  the  added  value  of  the  EU  youth  employment  interventions.  The  replies  to 
the  public  consultation  also  highlighted 
importance  of  having  functioning 
partnerships as one of the key factors for effective programmes.  

the 

Efficiency  could  be  further  improved  by  providing  better  information  and  further 
boosting the capacity of programme authorities and social partners, as provided for in the 
proposal for a regulation on European Social Fund Plus. 

Tailored offers are most beneficial in the long term  

Individuals  furthest  away  from  the  labour  market  benefit  greatly  from  tailored  and 
intensive guidance and support. 

A significant proportion of young people targeted by youth employment  actions are not 
job ready, and require preparatory and ongoing support to help them make the transition 
to  successful  employment,  qualification  or  labour  market  inclusion.  It  has  proved 

62 

 
 
 
essential to involve trained youth workers to help disadvantaged young people make 
the transition from inactivity to participating in training or employment schemes. 

Very often this group has low educational attainment levels and requires a blend of basic 
skills training and vocational education, in combination with employment interventions. 
During  the  EU-level  focus  group  work,  a  number  of  representatives  emphasised  the 
importance  of  working  with  schools  as  most  of  the  problems  start  before  young 
people  reach  the  age  to  qualify  as  NEET,  i.e.  at  15,  though  there  are  limits  to  using 
YEI  and  ESF  in  working  with  young  people  at  an  earlier  age.  Belgium,  Latvia  and 
Slovakia emphasised the need to cooperate with schools to reach these young people as 
early as possible. In this respect, the definition of NEET can be restrictive since it only 
includes  young people already  not  in  employment,  education or training, not  those  that 
may soon join the category. 

Many  ESF/YEI  operations  focus  on  work  experience  to  give  an  opportunity  to  young 
people who would otherwise find it very hard to find a placement or traineeship. Often 
work  experience  is  more  attractive  to  young  people  who  have  experienced  failure  at 
school.  It  gives  them  a  chance  to  discover  what  they  are  interested  in  and  where  their 
strengths lie. It also helps them develop social skills that are essential to the workplace. 
To conclude, work experience has proved to be a vital route into employment and helps 
young people develop their social skills. 

Many Member States have adopted a ‘pathway approach’ (or combined approach) to 
tackling  youth  employment.  This  is  particularly  the  case  with  YEI  support.  Many 
young people not in employment, education or training need to participate in a series of 
interventions  in  order  to  make  the  transition  into  work  or  education  and  therefore  they 
benefit  from  a  comprehensive  approach.  In  Germany  and  Poland,  the  individualised 
approach to assisting and supporting  young people (in particular those most in need) is 
working  well.  Organisations  responding  to  the  public  consultation  highlighted  the 
importance  of  tailored  approaches  for  programmes  to  be  effective,  ensuring  that  they 
focused on individual needs. 

Measuring the effectiveness of youth employment programmes can be improved, as 
can the visibility of EU funding 

Measuring  the  effectiveness  of  youth  employment  operations  improved  significantly 
over  the  current  programming  period  but  there  is  still  scope  for  improvement.  The 
problem of under-reporting would be aided by more regular reporting by  the managing 
authorities  instead  of  the  current  yearly  practice.      Achieving  a  greater  degree  of 
standardisation  of  soft  outcomes  and  longer-term  results  in  addition  to  the  compulsory 
indicators, at least at national level, would also aid future evaluations.  

63 

 
 
 
To  reduce  the  administrative  burden,  the  next  regulation  should  provide  the  legal  basis 
for using administrative data in the future.  

Further  efforts  should  be  made  to  make  microdata  more  available  and  evaluation 
practices  would  benefit  from  a  further  increase  of  the  number  of  counterfactual  impact 
evaluations and more stable information on unit costs, as the operations come to an end. 

There  seems  to  be  an  issue  of  the  visibility  of  EU  support  provided  for  youth 
employment.  Broader  communication  strategies  can  be  used  to  address  this  issue,  by 
using  social  media  and  forging  partnerships  to  work  with  schools,  community 
organisations and frontline services. Increased visibility would also aid better outreach. 

64 

 
 
 
 
 
ANNEX 1: PROCEDURAL INFORMATION 

1.  LEAD DG, DECIDE PLANNING/CWP REFERENCES 

This evaluation was carried out by DG EMPL as an initiative  published in Decide 
with  the  reference  number  PLAN/2018/2966.  It  was  published  in  July  2018 
(https://ec.europa.eu/info/law/better-regulation/initiatives/ares-2018-3876690_en). 

2.  ORGANISATION AND TIMING 

An interservice steering group (ISSG) was set up in July 2018, which included the 
following DGs: EMPL, AGRI, BUDG, EAC, GROW, HOME, JRC, REGIO, RTD, 
SG and the Legal Service. 

The timeframe for the evaluation was as follows: 

20 July – 17 August 2018 
26 July 2018 
20 September 2018 

13 December 2018 

18 December 2018 
19 February 2019 
14 June 2019 
7 November 2019 
31 January 2020 
18 May 2020 
25 May-3 June 2020 

Publication of the roadmap and feedback period 
1st meeting of the ISSG 
Request for services for the external study to underpin the 
evaluation launched 
Signature of the contract for the study with the consortium 
Fondazione Giacomo Brodolini – Applica – Metis  
2nd meeting of the ISSG: kick-off meeting for the study 
3rd meeting of the ISSG: draft inception report of the study 
4th meeting of the ISSG: draft interim report of the study 
5th meeting of the ISSG: draft final report of the study 
Final delivery of the Rhomolo modelling 
Approval of the external study 
Discussion of the draft staff working document  

3.  EXCEPTIONS TO THE BETTER REGULATION GUIDELINES 

N/A 

4.  CONSULTATION OF THE REGULATORY SCRUTINY BOARD (IF APPLICABLE) 

N/A 

5.  EVIDENCE, SOURCES AND QUALITY 

The evaluation was based on three main sources of information: 

- 

the twelve-week public consultation, carried out and analysed with the assistance 
of the external team of experts; 

65 

 
 
 
 
- 

- 

an estimate of the macroeconomic effects of youth employment operations using 
the  dynamic  computable  general  equilibrium  model  (Rhomolo)  run  by  the  Joint 
Research Centre under an administrative agreement between DG EMPL and JRC; 

a study  carried out  by  an external  team experts,  under contract (VC/2018/0715) 
through DG EMPL Multiple Framework Contract VT/2016-027 for the provision 
of services related to the implementation of the Better Regulation Guidelines. The 
external study combined the results of the two other sources with additional work. 

The quality of the final report of the external contractors was assessed as good by the 
interservice steering group. 

The conclusions and findings of the evaluation are considered  to be robust, mindful 
of the limitations and mitigating measures described in section 4.2 of the evaluation. 

66 

 
 
 
ANNEX 2: STAKEHOLDER CONSULTATION 

1.  The consultation strategy 

1.1. Objectives 

This synopsis report outlines the consultation organised to evaluate ESF support for 
Youth Employment from 2014 till 2018 and presents the main findings. 

To  ensure  transparency  and  involve  the  stakeholders,  the  process  followed  the 
standards  and  methods  set  out  in  the  Better  Regulation  guidelines.  The  various 
consultations have followed the roadmap and consultation strategy. The roadmap43 of 
the  evaluation  itself  was  published  on  the  Better  Regulation  website  and  open  for 
public feedback between 20 July 2018 and 17 August 201844. 

1.2. Consultation stakeholders  

The stakeholders targeted by the consultation were organisations or individuals that: 

  had an interest in youth employment operations funded under the ESF/YEI; 

  had or might have participated in the operations; 

  had expertise in the subject; 

  and had or might have run or been involved in running the operations. 

Therefore, the following groups were formed for the consultation: 

1.  Participants:  people  who  have  received  support  under  the  Youth  Employment 
Initiative or other youth employment operations, with the aim of gaining an insight 
into the extent to which the objectives of the funding have been achieved in terms 
of  their  integration  in  the  labour  market,  and  which  factors  played  a  role  in  this 
that weren’t directly identified by the monitoring mechanisms; 

2.  Young people not in education, employment and training not reached by the 
operations  (NEETs):  the  consultation  tools  collected  the  views  of  members  of 
this  group  (the  main  target  for  the  Youth  Employment  Initiative)  who  had  not 
participated in the operations, with the aim of understanding why they had not; 

43  https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/1863-Evaluation-of-the-support-

to-youth-employment-by-the-Youth-Employment-Initiative-and-the-European-Social-Fund. 

44 Feedback on the roadmap: https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/1863-

Evaluation-of-the-support-to-youth-employment-by-the-Youth-Employment-Initiative-and-the-
European-Social-Fund/feedback?p_id=273121. 

67 

 
 
 
                                                 
3.  Bodies  involved  in  running  the  operational  programmes,  such  as  managing 
authorities and other Member State representatives, social and economic partners 
represented in the monitoring committees; 

4.  Organisations 

involved 

in  delivering  youth  employment  operations  as 
beneficiaries  or  project  partners  and  their  EU-level  representatives:  public 
administrations at the national, regional or local levels, public and private training 
and  education  providers,  workers’  and  employers’  organisations,  youth 
organisations, NGOs, charities and companies. Their feedback was relevant to all 
evaluation questions, particularly to identify any issues of efficiency and delivery 
in running the youth employment operations; 

5.  Organisations and individuals not directly involved in running the operations, 
but who have a stake in youth employment issues, notably with regards to young 
people and, specifically, young people at risk of social exclusion, who contributed 
their views on how the operations met the specific needs of young people; 

6.  Academic  and  research  bodies  with  expertise  and  knowledge  of  youth 
employment  policies  and  issues,  who  provided  insights  into  the  relevance  and 
coherence of measures; 

7.  Organisations  representing  employers,  who  should  contribute  feedback  on  the 
factors that played a  role in  the successful  integration of participants,  and  young 
people in general, in the labour market;  

8.  Young people in general, who contributed their opinions on the relevance of the 

action taken; 

9.  The  general  public,  i.e.  any  individual  or  organisation  outside  the  previous 
groups who wanted to provide their views on EU support for youth employment. 

1.3. Consultation methods and tools 

Type of stakeholder 
consultation 

Type of stakeholders  

Timeframe 

Evaluation 
partnership meetings 

Managing 
bodies 

authorities/intermediate 

February 
February 2020 

2019 

– 

Open 
consultation 

public 

Open to all stakeholders and the general 
public 

24  May  –  16 August 
2019 

in  10 
Field  work 
Member  States  (case 
studies)  in  the  form 
of 
semi-structured 
interviews  and  focus 
groups 

Managing 
socioeconomic 
researchers 
participants. 

authorities, 

partners; 

beneficiaries, 
evaluators/ 
representatives  of 

and 

July 2019 - September 
2019 

EU-level 
groups 

focus 

Members  of  ESF 
group 

technical  working 

2 October 2019 

68 

 
 
2.  Synopsis Report 

2.1. Stakeholders involved in managing the ESF 

Stakeholders  involved  in  managing  the  funds  were  consulted  regularly  during  the 
ESF  evaluation  partnership,  which  comprises  Member  State  representatives  of  ESF 
monitoring and evaluation capacity. They  carried out  numerous  tasks, ranging from 
preparing the evaluation questions to presenting the findings. 

Date 
ESF 
of 
evaluation partnership 
meeting  

Topic discussed 

18 February 2019 

Update on the state of play of the evaluation study 

12 November 2019 

The draft findings of the thematic evaluation and conclusions 
from the public consultation  

11 February 2020 

Findings and conclusions of the study 

Draft  reports  were  circulated  to  the  partnership  members,  who  were  given  the 
opportunity to comment on the outputs of the evaluation. The comments made by the 
managing authorities were mostly to flag inconsistencies in data related to individual 
Member States, which were then corrected. 

2.2. The public consultation 

2.2.1.  Description of the activity 

The online public consultation was launched on 24 May 2019 and ran until 16 August 
2019 using the EU Survey tool. The questionnaire comprised 26 questions (excluding 
profiling and closing sections), structured around the main evaluation questions and 
including both open-ended and closed questions.  

2.2.2.  Stakeholders  

The public consultation was structured around five groups of respondents, defined as 
follows: 

  Group  A.1:  Individual  citizens  33  years  old  or  under  who  had  taken  part  in  the 

ESF/YEI – i.e. ‘ESF/YEI participants’; 

  Group A.2: Individual citizens 33 years old or under who either did had not taken 
part in the ESF/YEI, had taken part in other support schemes not funded by the 
ESF/YEI,  or  had  taken  part  and  didn’t  know  if  it  was  ESF/YEI-funded  –  i.e. 
‘other young people’; 

  Group B: Individual citizens over 33 years old – i.e. the ‘general public’; 

  Group  C:  Organisations  involved  in  managing  the  ESF/YEI  (such  as  managing 
authorities,  intermediate  bodies,  beneficiaries,  and  social  partners  involved  in 

69 

 
 
 
running  or  monitoring  the  ESF/YEI)  likely  to  have  a  direct  and  detailed 
knowledge of YEI/ESF – i.e. ‘organisations involved’;  

  Group  D:  Organisations  not  involved  in  managing,  monitoring  and  running  the 
ESF/YEI, i.e. entities or organisations that presumably do not have a direct stake 
in the ESF/YEI – i.e. ‘other organisations’. 

The number of questions for the respondents varied by profile of respondent.  

2.2.3.  Results 

The public consultation received 1,376 responses.  

It  achieved  a  satisfactory  level  of  overall  representation  of  different  respondents’ 
profiles,  in  particular  for  young  people,  who  were  a  key  target  of  the  consultation. 
Most  respondents  were  under  33  years  of  age  (57.6%)  who  either  took  part  in 
ESF/YEI  (24.7%)  or  did  not  take  part  in  ESF/YEI  (32.9%).  Just  over  one  fifth  of 
respondents (21.8%) belonged to organisations involved in managing the ESF/YEI. 
The  remaining  respondents  were  either  from  the  general  public  or  belonged  to 
organisations not involved in the schemes. 

Most  of  the  respondents  who  participated  in  some  form  of  youth  employment 
support  (339  ESF/YEI  participants  and  92  other  participants)  did  so  through 
information on job opportunities, guidance and tutoring, support to find work 
experience  and  training  for  general  skills.  The  least  used  type  was  support  for 
going back to school or setting up a business. There were no significant differences 
in the type of support received by ESF/YEI participants and other participants.  

Almost one third of respondents replied on behalf of an organisation. Overall, the 
most responses from organisations were from public authorities (38.8%), followed 
by  companies  and  business  organisations  (22.3%)  and  NGOs  (18.5%). 
Beneficiary organisations (applying for ESF/YEI funds and running the projects) are 
the most represented among organisations involved (Group C).  

Over half of respondents are familiar with or have an idea of the goal and scope 
and know at least one activity funded by the ESF or YEI, or by both. 

There is an important caveat regarding representativeness. The public consultation 
run  as  a  voluntary,  online  survey  may  not  have  reached  all  target  groups  in  the 
desired  way.  The  responses  to  the  public  consultation  show  a  very  unbalanced 
geographical  distribution,  with  four  countries  covering  almost  three-quarters  of 
responses: Italy (20.4% of responses, considering only those referring to one single 
country), Bulgaria (18.8%), Spain (16.4%) and Slovakia (16.4%). All other countries 
elicited much fewer responses and two countries no responses at  all (Denmark and 
Luxembourg). Almost 60% of individual respondents were women; 40% were men. 

Relevance 

The questions on relevance explore the extent to which the different types of actions 
and support are in  line  with  the needs of  young people,  in  particular  young  people 
not  in  education,  employment  or  training,  in  terms  of  helping  them  find  a  job, 
improve their professional skills or get closer to the labour market.  

When  asked  to  rank  the  usefulness  of  youth  employment  support  actions, 
individual respondents (as opposed to those answering on behalf of an organisation) 

70 

 
ranked  information  on  job  opportunities,  guidance  and  tutoring  followed  by 
support  to  find  work  experience;  training  for  general  skills;  support  in 
overcoming barriers to work and training; and training for qualifications as the 
most  useful  forms  of  support  (between  84.8%  and  82.3%  say  they  were  mostly  or 
very useful).  

Fewer  respondents  ranked  support  in  setting  up  a  business,  in  finding  a  job  or 
opportunity  abroad  and  support  to  get  back  to  school  to  be  most  useful  (between 
71% and 64.8%). For more details see Figure 16. Respondents suggested additional 
support  that  could  be  useful,  such  as  training  in  social  skills  and  psychological 
support.  

Figure 16  In your opinion, how useful are the following actions in helping young 
people find a job, improve their professional skills or enter the labour 
market? (Group A1, A2 and B, n=976) 

Information on job opportunities, guidance, tutoring

Support to find a work experience

Training for general skills

Support in overcoming barriers to work/training

Training for qualifications

Support in setting up a business

Support finding a job/experience abroad

469 

522 

469 

463 

453 

356 

36 

70 

43 

301 

43 

61 

47 

342 

345 

65 

62 

36 

67 

49 

50 

349 

75 

63 

34 

374 

351 

318 

135 

83 

64 

334 

126 

104 

59 

Support to go back to school

307 

324 

170 

116 

57 

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Very useful Mostly useful

I don't know / I don't have an opinion Mostly useless Not useful at all

The  ESF/YEI  action  considered  most  relevant  by  responding  organisations  was 
vocational education and training, followed by guidance and career support; paid 
apprenticeships,  traineeships  and  internships;  and  basic  skills  training.  Community 
or  voluntary  work  and  non-paid  apprenticeships  were  ranked  the  least  relevant 
(47.5% find it relevant or very relevant vs over 90% for other types of support).  

In  the  open  questions,  a  few  respondents  suggested  that  social  skills  training  and 
emotional support (such as counselling or psychological support) are also relevant 
types of support that are less often provided by ESF/YEI operations. 

Overall,  most  responding  organisations  reported  that  EU  support  was  flexible 
enough  to  adapt  to  emerging  needs.  Respondents  highlighted  the  capacity  of  the 
ESF/YEI to adapt to the needs of young people and to the socioeconomic context in 
the  different  regions  and  Member  States.  Conversely,  some  others  sent  negative 
replies citing a lack of capacity of the programmes to adapt to the specific needs of 
young people and to tailor the actions to specific national and regional needs. 

71 

 
 
 
 
Effectiveness 

Responses  to  the  questions  on  effectiveness  describe  the  extent  to  which  ESF/YEI 
measures achieved the expected results. 

A key  factor  in the  effectiveness  of  ESF, and especially YEI, is  the programmes’ 
capacity to reach out to and involve those farthest from the labour market. Overall, 
respondents to the public consultation identified social media campaigns as the best 
channels  to  inform  young  people  of  support  initiatives.  In  distant  second  place 
were youth networks, clubs and online groups, and word of mouth. However, when 
young  participants  were  asked  how  they  actually  learned  about  the  support 
measures  they  took  part  in,  most  said  through  employment  services  and 
employment  info  centres,  followed  at  a  distance  by  word  of  mouth,  family  or 
friends and only a smaller share learned about them via social media. 

The  main  results  that  ESF/YEI  participants  achieved  through  the  programmes 
were  in  improvements  to  their  skills  and  qualifications,  whether  to  develop 
general skills (38.3% of respondents), professional skills and qualifications (35.7%), 
or  to  start  or  resume  education  and  training  courses  (29.2%).  A  smaller  but  still 
significant  share  acknowledged  that  ESF/YEI  support  helped  them  join  the  job 
market,  either  by  finding  a  temporary  or  a  permanent  job  (20.9%  and  16.8% 
respectively)  or  by  improving  their  employment  situation  (18.3%).  One  in  ten 
respondents reported that the support received did not help. 

Figure 17  The support you received helped you in… (Group A1 and A2, n=431, 

multiple answers allowed) 

Developing general skills

130 

15 

Looking for a job

100 

Gaining professional skills and qualifications

121 

44 

18 

Entering/going back to education and training

99 

13 

Finding a temporary job

Improving quality of position on the market

71 

68 

17 

11 

Finding a permanent job

57 

16 

It did not help

32 

15 

A.1

A.2

Finding a part-time job

17 

2 

Others

5 

6 

Moving abroad to study/work

9 

1 

0

20

40

60

80

100

120

140

160

72 

 
 
 
 
Compared  to  other  participants,  the  ESF/YEI  participants  are  more  likely  to 
receive  support  to  improve  their  education  and  qualifications  and  to  develop 
their skills in general, although the employment outcomes are similar (in terms of 
finding  either  a  permanent  or  temporary  job).  The  other  participants  reported  in 
higher numbers that the support had helped them find a job.  

Respondents were also asked about their current and past labour market situation 
(before  they  received  support).  Of  the  ESF/YEI  participants,  46.7%  are  currently 
unemployed and 29.5% employed full time. Before receiving support, 88.1% were 
unemployed and 1.6% were employed full time, showing a consistent increase in the 
employment  rate.  Of  the  other  participants,  51.1%  are  currently  unemployed  and 
28.4%  are  employed  full  time.  Before  receiving  support,  77.2%  were  unemployed 
and 0.6% were employed full time. 

Table 6 

Labour market status of individuals who took part in ESF/YEI initiatives 

Labour market status 

Employed full-time 
Employed part-time 
In a traineeship, internship or apprenticeship 
In formal education 
In vocational training 
In voluntary service 
Self-employed 
Unemployed for 12 months or more 
Unemployed for less than 12 months 
Total 
Total unemployed 

N. of respondents in group A1, absolute 
value (% in brackets) 

Before support 

After or during 
support 

5 (1.6) 
8 (2.6) 
5 (1.6) 
10 (3.2) 
6 (1.9) 
2 (0.6) 
1 (0.3) 
125 (40.3) 
148 (47.3) 
310 (100) 
273 (88.1) 

94 (29.5) 
16 (5.0) 
33 (10.3) 
3 (0.9) 
7 (2.2) 
6 (1.9) 
11 (3.4) 
80 (25.1) 
69 (21.6) 
319 (100) 
149 (46.7) 

According to the organisations that responded, the main results of the ESF/YEI 
programmes  were to develop  the skills  and qualifications  of young people  and 
to  help  them  find  a  job,  including  for  young  NEETs  and  other  disadvantaged 
individuals,  such  as  the  disabled.  This  echoes  the  results  reported  by  individual 
recipients.  Overall,  organisations  were  more  sceptical  concerning  the  effectiveness 
of  ESF/YEI  in  helping  young  people  improve  the  quality  of  their  employment. 
Respondents  also  reported  that  ESF/YEI  generated  ‘soft  outcomes’  in  terms  of 
increased  partnerships  and  collaboration  among  stakeholders,  of  integrating 
marginalised  people,  developing  soft  skills  and  group  work,  empowering  young 
people, increasing their fit with job market requirements, increasing their motivation 
and integrating young people not in employment, education and training into society. 

In  terms  of  target  groups,  the  organisations  that  responded  reported  that  the 
ESF/YEI is  relatively  more successful  in helping young NEETs and, to  a lesser 
extent,  young  people  leaving  education  without a  qualification  and  those  at  risk  of 
social exclusion or marginalisation than in helping young people in rural or hard-to-
reach areas, or those at risk of poverty.  

Individual  respondents  gave  a  positive  assessment  on  all  facilitating  factors 
mentioned,  in  particular  concerning  the  provision  of  financial  benefits  to 

73 

 
 
 
participants  and  measures  that  are  aligned  with  labour  market  needs.  Respondents 
from  organisations,  especially  from  organisations  involved  in  ESF/YEI,  focus  on 
the importance of having operations that are well integrated with general youth 
policies  and  on  providing  accompanying  service  or  financial  and  non-financial 
incentives to support young people during their participation. They also mentioned 
flexibility  in  implementation  relatively  more  frequently  as  an  important  factor. 
Additional factors are cooperation among stakeholders (e.g. enhancing the capacity 
of measures to adapt to local specificities and improve dialogue between public and 
private institutions), focus on individual needs and follow-up activities.  

Young respondents who did not participate in ESF/YEI mostly mentioned the lack 
of information as the reason for not participating. 

Organisations indicated the  difficulty  in reaching out  to  target  groups  as the most 
frequently  selected  factor  hindering  effectiveness  (60.9%).  Other  factors 
mentioned, although less frequently, are structural problems such as the lack of jobs 
and  the  low  level  of  education  of  participants,  the  administrative  burden  for 
beneficiaries, and the lack of involvement of stakeholders. Of the countries with the 
highest  response  rate,  Portugal  cited  difficulties  in  reaching  out  to  target  groups 
relatively more frequently (81.3%), followed by Spain (67.6%). 

Efficiency 

The questions on efficiency focus on the ‘value for money’ of measures and whether 
resources invested by the ESF/YEI are proportionate to the results achieved.  

For the ESF, respondents from organisations involved agreed mostly on the cost 
effectiveness  of  vocational  education  and  training  activities,  followed  by 
apprenticeships,  traineeships  and  internships,  basic  skills  training  (80.9%)  and 
guidance and career support (between 83.3% and 80.4%). Agreement seems to be 
quite low regarding community and voluntary work (41.6%). The assessment of the 
YEI is similar. 

74 

 
 
 
Figure 18  To what extent do you agree or disagree that the following activities 

implemented under the Youth Employment Initiative were cost-effective? 
(Group C, n=206) 

Vocational education and training (occupational skills)

Apprenticeships, traineeships and internships

Basic skills training (e.g. IT, language, and job…

Guidance and career support

Employment incentives for employers

Skills assessment (recognition and validation of prior…

Awareness raising and information actions

Second chance education

Start-up support

Structural support for strengthening institutional…

Job mobility measures

76 

88 

71 

85 

70 

52 

47 

56 

56 

50 

44 

44 

95 

82 

92 

73 

18 

18 

7 

4 

6 

8 

2 

8 

24 

23 

8 

3 

8 

12 

4 

9 

75 

25 

18 

7 

11 

81 

82 

71 

71 

33 

17 

4 

19 

33 

23 

4 

17 

34 

20 

3 

22 

32 

16 

7 

24 

70 

31 

25 

7 

23 

73 

68 

41 

14 

5 

29 

35 

29 

5 

25 

Community or voluntary work

22 

54 

47 

38 

10 

35 

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

I strongly agree

I agree

I don't know / I don't have an opinion

I disagree

I strongly disagree

Not applicable

Overall,  most  respondents  from  organisations  involved  in  the  programmes 
judge  the  administrative  arrangements  for  project  implementation,  project 
selection,  communication  and  evaluation  to  be  appropriate  (between  65%  and 
61%). Fewer than half of respondents judged the management and control system to 
be appropriate. 

Coherence 

The  questions  on  coherence  assess  the  extent  to  which  ESF/YEI  activities  are 
aligned  and  complementary  with  other  youth  and  youth  employment  national  or 
regional schemes. 

Most organisations could not rate the coherence of ESF or YEI with other youth 
and  youth  employment  national/regional  schemes  for  all  of  the  suggested 
schemes  (Erasmus+,  EURES,  ERDF  and  European  Solidarity  Corps).  The  only 
exception  regards  coherence  with  existing  national  or  regional  schemes,  with 
43.1% of respondents saying that overall they are coherent, with a higher rating from 
organisations  involved  than  from  organisations  not  involved  (45.3%  vs  36.4%). 
However, overlapping or non-alignment of schemes does not seem to be an issue. 

75 

 
 
 
 
 
Figure 19  To what extent are the YEI and ESF coherent with other youth and youth 

employment EU national/regional schemes? (Group C and D, n=399) 

Existing national/regional programmes

172 

40 

45 

142 

Erasmus+

162 

46 

21 

170 

EURES

124 

33 

13 

ERDF

110 

46 

18 

229 

225 

European Solidarity Corps

81 

35 

13 

270 

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

They are complementary

They are not aligned

They overlap

I don't know

EU added value 

The EU added value questions assess the additional value resulting from ESF or YEI 
support, compared to what could be achieved by Member States at national, regional 
and local levels with own funds. This question was only put to organisations. 

Organisations  gave  an  overall  positive  perception  of  an  EU  added  value  resulting 
from ESF or YEI support, mostly because it expanded coverage of the assistance 
provided  (63.5%)  and  increased  the  assistance  provided  (47.2%).  Very  few 
respondents (3.9%) said that it did not make a real difference. Overall, respondents 
from the organisations involved gave a more positive response while more group D 
respondents reported that it did not make a real difference (11.6%, v 1.4%). 

Figure 20  Has the EU provided support that goes beyond what is addressed by 
national or regional programmes or support schemes aimed at youth 
employment? (Group C and D, n=282) 

Other

Yes, it has widened the assistance provided

Yes, it has increased the assistance already…

Yes, it increased coverage of specific target groups

Yes, the European Social Fund and the Youth…

41 

I don’t know/ I don’t wish to answer 

No, did not make a real difference

13 

11 

Yes, it alowed to test and implement innovative…

0 

226 

179 

133 

116 

0

50

100

150

200

250

76 

 
 
 
 
 
Closing questions 

A high share of young respondents would be interested in taking part in EU youth 
employment activities in the future (66.8%). The type of activities  that appear to 
interest them most are training courses to improve job skills, work experience such 
as traineeships and internships, basic skills training and support to find a job. 

2.3. Member State-level interviews and focus groups 

2.3.1.  Stakeholders involved 

Interviews  and  focus  groups  with  stakeholders  at  Member  State-level  were  carried 
out during the evaluation in 10 Member States and for 20 operational programmes. 
The  types  of  stakeholders  consulted  were  managing  authorities,  beneficiaries, 
socioeconomic partners, evaluators/researchers and representatives of participants – 
e.g. youth organisations. 

Table 7 

Number of interviewees by Member State and type of organisation 

MS 

Belgium 
France 
Germany 
Greece 
Italy  
Malta 
Poland 
Portugal 
Slovakia 
Spain 

Managing authority and 
other governmental 
bodies/agencies/institutes 

Socioeconomic 
partners and 
other 
organisations 

Evaluators/researchers 

3 
8 
8 
7 
12 
2 
4 
4 
20 
4 

2 
1 

5 
2 

4 

4 

1 
1 

2 

In addition to interviews, focus groups were carried out in in Spain, France, Poland, 
Greece and Portugal in order to obtain detailed information from key stakeholders on 
the  evaluation  questions.  Stakeholders  involved  included  national  and  regional 
managing  authorities,  PES  and  other  beneficiaries  and  other  private  or  public 
organisations  such  as  research  institutes.  Where  focus  groups  could  not  be  held, 
additional interviews were organised as well as wrap-up meetings with interviewees. 

2.3.2.  Results 

The  case  studies  were  used  as  the  main  source  of  information  to  answer  the 
evaluation questions and feed into case study reports.  

The  interviews  for  the  case  studies  highlighted  an  increase  in  the  time  required  to 
locate  and  recruit  from  harder-to-reach  groups.  Member  States  also  use  different 
targeting  policies,  reflecting  national  priorities  and  national  assessments  of  needs, 
with  some  countries  using  the  youth  programmes  to  target  young  people  closer  to 
employment,  including  graduates,  and  with  the  results  generally  showing  higher 
levels  of  effectiveness.  ESF/YEI  operations  showed  a  high  level  of  flexibility  and 
innovation  in  tackling  the  challenge  of  accessing  hard-to-reach  and  vulnerable 
groups. 

77 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
2.4. EU-level focus group 

2.4.1.  Stakeholders involved 

An EU-level focus group was held on 2 October 2019 in Brussels. It was organised 
back-to-back with the Technical Working Group and the aim was to discuss some of 
the evaluation questions with a smaller group of participants, focusing on key issues 
and gaps. The focus group comprised representatives from managing authorities and 
European Commission officials. There were 15 participants from 11 Member States.  

2.4.2.  Results 

Effectiveness/outreach 

The discussion confirmed the preliminary findings of the study and highlighted that 
managing  authorities  face  difficulties  with  outreach,  especially  to  reach  the  most 
disadvantaged  young  people  in  the  broader  target  group  of  young  people  not  in 
employment, education or training (NEETs). Member States discussed  the country-
specific obstacles they face and the remedial actions taken to overcome them. 

Efficiency/partnerships 

The  discussion  confirmed,  in  line  with  the  preliminary  findings  of  the  evaluation, 
that cooperation among the multiple bodies involved, in particular among employers 
and  employment  services,  is  key  to  the  efficient  implementation  of  youth 
employment measures. It also confirmed that this is best facilitated when employers 
have a clear interest to work with the employment services, which can only happen if 
the training offer is in line with the companies’ needs for skills or when they have a 
financial incentive. 

Sustainability 

The focus group discussion revealed that managing authorities are aware of the need 
to measure and monitor the sustainability of results of youth employment operations, 
particularly  with  regard  to  making  employment  outcomes  sustainable  over  time.  It 
also revealed that Member States have difficulties in doing so, even though the result 
indicators  (short  and  longer  term),  administrative  data  sets  and  ad  hoc  surveys  are 
useful  support  tools.  Sustainability  can  also  be  interpreted  in  terms  of  systemic 
changes such as an improved governance of youth employment policies or increased 
institutional capacity. 

78 

 
 
 
ANNEX 3: METHODS AND ANALYTICAL MODELS 

1.  Overall approach to the evaluation work 

In line with the Better Regulation guidelines, DG EMPL decided to base this evaluation 
on the work carried out by external evaluators and took the following approach: 

  collect and analyse the relevant evidence; 
  provide answers to all evaluation questions; 
  present evidence-based conclusions. 

2.  Rationale of the evaluation 

The  main  objective  of  this  evaluation  is  to  assess  the  youth  employment-focused 
operations and their relevance, effectiveness, efficiency, coherence and EU value added. 
The  evaluation  includes  both  an  individual  assessment  of  each  country  and  a  cross-
cutting and comparative assessment. An external contractor collected and examined data 
covering 2014-2018 and prepared a study, including a forward-looking analysis of future 
implementation. 

3.  Evaluation questions and structure of the report 

The evaluation was based on the following evaluation questions: 

Question 

Question 1 - Effectiveness: How effective is the YEI, and other 
ESF-funded  youth  employment  operations,  in  achieving  their 
objectives? 

1.1.  To  what  extent  have  the  financial  implementation 
and  the  achievement  of  the  expected  outputs  progressed 
according  to  the  targets  set  in  the  programmes?  What 
in 
were 
implementation, ESF absorption …)? 

the  main 

involved 

(delays 

factors 

1.2.  How  and  to  what  extent  does  YEI  contribute  to  the 
achievement  of  the  general  objective  of  sustainable 
integration (also after the end of the operation) of young 
people  into  the  labour  market  and  to  the  specific 
objectives  under  ESF?  How  did 
to 
addressing the problems faced by NEETs? 

it  contribute 

to 

Link 
the 
section  of 
the SWD   

and 

Data 
info 
sources 

5.1. 

5.1. 

(1),  (2),  (3), 
(4) 

(1),  (2),  (3), 
(4) 

5.1. 

(1),  (2),  (3), 
(4) 

1.3. To what extent were the target groups reached by the 

5.1. 

(1),  (2),  (3), 

79 

 
 
 
operations,  including  disadvantaged  persons,  those  from 
marginalised  communities  and  those  leaving  education 
without  qualifications?  To  what  extent  was  gender 
balance achieved? 

(4) 

1.4.  What  was  the  quality  and  timeliness  of  the  offers 
received by the participants? 

5.1. 

(1), (2), (3) 

1.5. Which types of interventions were the most effective 
and  most  sustainable,  for  which  groups  and  in  which 
contexts  (e.g.  more  developed,  less  developed  and 
transition regions; urban and rural areas etcetera)?  

1.6.  What  main  factors  (geographical,  socioeconomic, 
the 
organisational …)  had  a  bigger 
effectiveness  of  ESF  and  YEI  operations  in  the  field  of 
youth employment, by type of operation? 

impact 

in 

1.7.  To  what  extent  YEI  and  ESF  contributed  to 
structural  changes 
in  national  education  systems, 
vocational  training  systems,  public  employment  systems 
or youth policies?  

5.1. 

(1), (2), (3),  

5.1. 

(1), (2), (3),  

5.1. 

(1), (2), (3),  

Question  2  –  Efficiency:  How  efficient  has  the  YEI,  and  other 
youth-employment  operations  funded  by  the  ESF,  been  in  the 
achievement of their objectives? 

5.2. 

(1),  (2),  (3), 
(4) 

2.1. To what extent were operations cost-effective? What 
types of operations were more and less cost-effective? In 
what contexts? What were the determining factors? 

2.2.  Are  there  significant  cost  differences  between 
Member  States/Regions  in  the  implementation  of  the 
operations? What are these differences related to? 

extent  were 

the  organisational 
2.3.  To  what 
arrangements, 
including  management  and  control 
systems  at  all  levels  conducive  to  the  effectiveness  of 
in 
operations?  Was 
particular gold plating involved? 

there  administrative  burden, 

5.2. 

(1), (2), (3) 

5.2. 

(1), (2), (3) 

5.2. 

(1), (2), (3) 

2.4.  In  particular,  how  timely  and  efficient  were  the 
procedures for reporting and monitoring?  

5.2. 

(1), (2), (3) 

2.5.  How  visible  were  YEI  and  other  ESF-funded  youth 
employment operations? 

5.2. 

(1), (3) 

Question 3 - Relevance: How relevant is the YEI, and the other 
ESF-funded youth employment operations? 

5.3. 

(1), (2), (3) 

3.1.  To  what  extent  were  the  objectives  and  the 
operations  funded  by  the  YEI  relevant  to  the  needs  of 
young  people  in  Europe?  To  what  extent  were  the 
objectives  and  operations  of  other  ESF-funded  youth 
employment operations relevant for them?  

5.3. 

(1), (2), (3) 

80 

 
3.2. To what  extent were OPs  flexible and able to  adapt 
to  changes  in  the  implementation  context,  notably  the 
evolution in the situation of youth employment? 

3.3.  Were  the  most  relevant  groups,  in  the  different 
socioeconomic  contexts  (e.g.  more  developed, 
less 
developed  and  transition  regions;  urban  and  rural  areas 
etcetera), targeted from the design stage? Were the most 
important  needs  of  these  groups  addressed?  How  were 
gender issues addressed? 

5.3. 

(1), (2), (3) 

5.3. 

(1), (2), (3) 

Question  4  –  Coherence:  How  coherent  are  YEI  and  the  other 
ESF-funded  youth  employment  operations  among  themselves, 
and with other actions in the same field? 

5.4. 

(1), (2), (3) 

4.1.  In  which  manner  were  the  YEI  and  other  ESF-
funded  youth  employment  operations  complementary 
with  each  other?  What  were  the  main  factors  in  this 
regard? 

4.2.  To  what  extent  were  they  complementary  and 
coherent with other policy objectives funded by the ESF 
and  other  programmes  and  policy  initiatives  oriented  to 
young people and youth employment at the EU level (e.g. 
ERDF, EAFRD, EMFF, Erasmus+, EURES …)? 

4.3.  To  what  extent  were  they  complementary  and 
coherent  with  other  activities  oriented  to  young  people 
and youth employment at national/regional level? 

5.4. 

(1), (2), (3) 

5.4. 

(1), (2), (3) 

5.4. 

(1), (2), (3) 

Question 5 – EU added value: What is the EU added value of the 
YEI and other ESF-funded youth employment operations? 

5.5. 

(1), (2), (3) 

5.1.  To  what  extent  did  the  YEI  and  other  ESF-funded 
operations  produce  effects  at  the  national  and  regional 
level  that  would  not  have  taken  place  without  the  EU 
intervention?  This  question  shall  be  addressed  from  the 
following perspectives: 

5.5. 

(1), (2), (3) 

  volume  effect:  have  the  operations  added  to 
existing 
actions  or  directly  produced 
beneficial  effects  that  can  be  measured  in 
terms of volume? 

  scope  effect:  have  the  operations  broadened 
existing  actions  by  addressing  groups  or 
policy  areas  that  would  otherwise  not  have 
been addressed? 

 

role  effect:  have  the  operations  supported 
innovation and the transfer of ideas that have 
been  subsequently  rolled  out  in  different 
contexts? 

  process 

effect:  Have  Member  State 

81 

 
administrations 
participating 
organisations  derived  benefits  from  being 
involved in the operations? 

and 

5.2.  To  what  extent  do  YEI  operations  and  other  ESF-
funded  youth  employment  operations  contribute  to  the 
objectives of the YG? 

5.5. 

(1), (2), (3) 

Question  6  –  Sustainability:  How  sustainable  are  YEI  and  the 
other ESF-funded youth employment operations? 

5.6. 

(1), (2), (3) 

6.1.  To  what  extent  the  effects  of  YEI  and  ESF  support 
are likely to continue after the end of the funding, both at 
individual and youth employment policy level? 

5.6. 

(1), (3) 

Data and info sources to reply evaluation questions: (1) – external study; (2) – SFC2014; 
(3) – national evaluations; (4) – EUROSTAT data 

The  evaluation  work  and  the  report  were  structured  around  the  conclusions  reached  on 
each of the questions. 

4.  Methodology and data sources 

The evaluation was based on a complex methodology aimed at collecting solid evidence 
and providing well-informed answers to the evaluation questions. 

It consisted of: 

  extensive desk research; 

  an open public consultation; 

 

interviews; 

  case studies; 

  synthesis of 2nd YEI evaluations; 

  a cost-effectiveness analysis; and 

 

focus groups. 

To better understand and evaluate ESF and YEI-funded operational programmes, it was 
important to look not only at the national situation, but also the situation in the specific 
regions  in  which  the  operational  programmes  are  run,  since  there  can  be  significant 
differences  between  different  parts  of  the  country.  This  was  recognised  in  the 
categorisation  of  regions  for  implementing  the  Structural  Funds  (into  more  developed, 
less developed and transition regions). The aim here was not, however, to re-describe the 
context  with additional detail  by type of region, but  to identify clusters  of regions  with 
similar characteristics that can be used together with the ESF/YEI monitoring data to see 
if the underlying socioeconomic  situation and how it has developed have an impact  on 
the effectiveness and efficiency of implementation.  

82 

 
 
To  this  end,  data  on  selected  indicators  (based  on  the  availability  of  data  at  regional 
level) for regions (NUTS 2 level) were aggregated to produce data by type of region in 
each  country  (i.e.  maximum  three  types  of  region  per  country,  total  of  50  regions)  and 
then  used  to  develop  two  composite  indicators:  one  describing  the  situation  at  the 
beginning of the programming period in 2014 and one describing how it evolved between 
2014  and  2018.  By  combining  the  data  for  the  two  composite  indicators,  four  broad 
clusters of regions were identified: 

  Cluster A: Regions with a strong starting point that made significant progress;  

  Cluster B: Regions with a strong starting point that made little progress;  

  Cluster C: Regions with a low starting point that made significant progress;  

  Cluster D: Regions with a low starting point that made little progress. 

The following steps were taken to create composite indicators and to cluster regions by 
typology: 

1.  Indicators were selected based on the availability of data at regional level (i.e. only 
indicators with more or less complete data by NUTS 2 regions could be used). The 
indicators had to be youth-related and cover the general economic context since the 
underlying  economic  situation  generally  has  a  significant  impact  on  potential 
labour market and programme outputs and outcomes; 

2.  The  indicators  were  calculated  by  type  of  region  for  each  country.  For  indicators 
for which low values suggest a positive situation (i.e. the rate of young people not 
in employment, education or training and the early school leaving rate) reciprocal 
values were used so that when the composite indicator is calculated, higher values 
indicate a more favourable situation or greater progress.  

3.  The  change  between  2014-18  was  calculated  (or  other  dates  depending  on  data 
availability) for each indicator in absolute numbers (percentage points used for % 
indicators, PPS for GDP per capita); 

4.  The indicator values for 2014 and for 2014-18 change were standardised using z-
scores (or standard scores) to indicate how many standard deviations an element is 
from the mean45; 

5.  The value of the two  composite indicators  was calculated  (describing the  starting 
point  and  change)  based  on  the  average  standardised  values  of  each  contributing 
indicator; 

6.  Countries and regions were clustered by typology using the k-Means method46. .  

45 A z-score can be calculated from the following formula: z = (X - μ) / σ, where z is the z-score, X is the value of the 
element, μ is the population mean, and σ is the standard deviation. A z-score less than 0 represents an element less 
than  the  mean;  A  z-score  greater  than  0  represents  an  element  greater  than  the  mean;  A  z-score  equal  to  0 
represents  an  element  equal  to  the  mean;  A  z-score  equal  to  1  represents  an  element  that  is  1  standard  deviation 
greater than the  mean; a z-score equal to 2, 2 standard deviations greater than the  mean; etc. A z-score equal to  -1 
represents an element that is 1 standard deviation less than the mean; a z-score equal to -2, 2 standard deviations less 
than the mean; etc. 
46 K-means provides a simple  way to organise  the  values of a given dataset into a predefined number of 
clusters  (k  clusters).  It  is  based  on  defining  a  centroid  for  each  cluster  and  regularly  adjusting  their 
positions.  Starting  with  k  centroids  spaced  as  far  away  from  each  other  as  possible,  each  point  in  the 

83 

 
 
                                                 
4.1. Synthesis of 2018 YEI evaluation reports 

The  synthesis  of  2018  YEI  evaluation  reports  was  also  formally  considered  part  of 
Task 1. It was carried out based on 21 evaluation reports submitted by the Member States 
up to mid-September 2019 (see annex 7 for the full list). 

By that time, evaluation reports were available for all Operational Programmes except for 
the 
Operational  Programme  Wallonie-Bruxelles  (the  final  evaluation  report  of 
Operational Programme will include a specific section on the evaluation of YEI)[1]. 

The approach used to prepare the synthesis of the evaluation reports was as follows: 

  Review of the executive summary in English if available 
  Translation of the evaluation report for languages not covered by the research 

team (including the executive summary if not available in English) 

  Detailed analysis of the evaluation report 
  Summary  of  evaluation  findings  (in  Word)  using  the  template  elaborated  by 

the Evaluation Helpdesk (including the quality assessment) 

  Editing  of  each  summary  by  Terry  Ward,  the  quality  manager  (but  also 

directing work on the Evaluation Helpdesk) 

  Categorisation of the reported findings in an Excel database 
  Preparation of the synthesis  
  Final editing of the synthesis by the quality manager. 

In  addition  to  the  YEI  evaluation  reports,  other  evaluations  of  youth  employment 
programmes were screened using the database of evaluations compiled by the Evaluation 
Helpdesk. The relevant findings for these are not included in the Synthesis of the 2018 
YEI evaluation reports (since these only deal with YEI evaluations) but are reflected in 
the replies to the evaluation questions. 

4.2. Comparative analysis of the evolution of the labour market and the role of 

the EU funded operations (see also section 2.1. of the main report) 

The aim of the comparative analysis is to analyse the context and evolution of the youth 
labour situation across the EU in relation to the YEI and ESF activities and to identify, as 
far  as  possible,  factors  that  have  contributed  to  the  effectiveness  and  efficiency  of  YEI 
and  ESF  youth-related  operations.  The  outcomes  are  used  to  put  the  results  of  the 
ESF/YEI  operations  into  context  when  answering  the  evaluation  questions  and  prepare 
the background information included in the case study fact sheets. The analysis focuses 
on the following aspects: 

dataset  is  associated  to  the  nearest  centroid.  The  centroid  is  then  moved  to  the  average  of  the  points 
assigned to it and the process is repeated until no further changes in the position of the centroids is possible 

[1]  The  final  version  of  the  reports  for  Hungary  and  England  (expected  in  the  course  of  2019)  were  not 

available yet by mid-September 2019. 

84 

 
 
 
                                                                                                                                                 
 
 
 
 

Socioeconomic context at national and regional level; 
Administrative and organisational procedures; 
Actions to support young people; and 
Efficiency and effectiveness of ESF/YEI operations. 

The work carried out for each of these areas is briefly presented below. 

To  complete  the  comparative  analysis  for  the  aspects  mentioned  above  the  following 
activities were carried out:  

  Selection  and  analysis  of  indicators  at  national  level  relevant  to  youth 
employment  derived  from  readily  available  statistics  published  by  Eurostat. 
The  analysis  looks  at  changes  over  time  (2014  –  2018  or  latest  year  with 
available data) and includes comparisons between countries and different age 
groups. 

  Clustering  of  regions  based  on  the  general  socioeconomic  context  in  four 

broad clusters: 

  Regions with low starting point and little progress, 
  Regions with low starting point and significant progress, 
  Regions with strong starting point and little progress, 
  Regions with strong starting point and significant progress,   
  Comparative analysis of administrative and organisational procedures in terms 
of the delivery of youth employment activities in the EU Member States. The 
aim  is  to  briefly  describe  the  main  actors  providing  youth  employment 
activities  –  YEI/ESF  Managing  Authorities  and  Youth  Guarantee 
implementing authorities  – and to  provide some insight  on the role of Public 
Employment  Services  (PES)  in  implementing  the  national  Youth  Guarantee 
schemes. To  collect  information on major actors, three different  data sources 
have  been  used:  ESF/YEI  Operational  Programmes  to  identify  the  ESF/YEI 
Managing Authorities; the national Youth Guarantee implementation plans to 
identify  the  Youth  Guarantee  implementation  authorities;  and  the  Labour 
Market  Policies  (LMP)  database  to  identify  the  Public  Employment  Services 
(PES).  Additionally,  to  conduct  the  analysis  on  the  role  of  national  PES  in 
implementing  the  national  Youth  Guarantee  schemes  and  the  progress  made 
through  time,  the  annual  PES  Network  reports  on  the  implementation  of  the 
Youth  Guarantee  were  utilised.  The  findings  of  the  reports  are  based  on 
responses  provided  by  national  PES  to  an  email  questionnaire  distributed  by 
the European Commission. 

  Comparative  analysis  of  interventions  specifically  targeting  young  people 
implemented  at  the  national  level  and  main  target  groups.  The  analysis  is 
based  on  information  obtained  from  the  EU  Labour  Market  Policies  (LMP) 
database  (managed  by  DG  EMPL)  and  focuses  on  three  different  aspects: 
types of interventions; target population (i.e. specific groups of young people); 
and level of funding. The aim of the analysis is to assess the degree to which 
ESF  and  YEI  funding  has  contributed  to  existing  interventions  or  whether  it 
has been used to introduce new interventions providing more/different options 
to young people or to target groups that are not targeted by existing nationally 
funded interventions.  

85 

 
 
 
 
 
  Assessment  of  the  effectiveness  and  efficiency  of  the  ESF/YEI  Operational 
Programmes  by  combining  ESF/YEI  monitoring  data  with  context  indicators 
at national and regional level. 

4.3. Cost-effectiveness analysis  

The analysis involves two main sub-tasks. The first is to provide an indicative estimation 
of efficiency across the Member States starting from the mapping of the costs per outputs 
and  results  for  Investment  Priority  8.ii.  Our  analysis  highlights  the  range,  nature  and 
overall extent of costs per person or operation and the costs per type of operation based 
on  monitoring  data  and  econometric  analysis.  The  second  sub-task  involves  a  detailed 
cost-effectiveness analysis for a sample of operations. We also relate costs to evidenced 
causal  effects,  i.e.  effects  of  the  operation  beyond  its  outputs  and  results  (net  results) 
drawing mainly on the  analysis of YEI and counterfactual impact evaluations  in  which 
we identify evidenced examples of effective operations and their costs. We also explore 
whether  lump  sums,  standard  scales  of  unit  costs  or  flat  rates  used  as  a  means  for 
simplification  for  beneficiaries  and  lowering  administrative  burden  for  partners,  led  to 
cost-effectiveness. 

5.  Overall assessment of strengths and weaknesses of the methodology and data 

The  various  sources  were  triangulated,  and  where  the  evidence  was  insufficient  or 
inconclusive,  the  approaches  were  combined:  data-based,  documentary  and  perception-
based sources as well as quantitative and qualitative techniques, depending on the nature 
of the evaluation question and the strengths of the relevant data and approaches. 

The  evaluation  drew  on  a  number  of  sources  providing  opinions  and  perceptions, 
including  surveys,  interviews  and  the  public  consultation.  Data  on  opinions  and 
perceptions  are  important  evidence  where  evaluation  questions  elicit  the  views  of 
stakeholders.  In  some  cases,  respondents  or  interviewees  may  be  the  only  source  of 
knowledge or witness accounts of events when no other sources are available. 

Under the Better Regulation guidelines, the open public consultation is an important tool 
for collecting stakeholder input and views on EU policy initiatives. It cannot be expected 
to provide a fully  representative view of EU public opinion, but it does offer a  channel 
for  the  people  who  care  about  a  given  issue  to  voice  their  opinion.  As  expected, 
respondents’  knowledge  and  involvement  in  running  youth  employment  operations 
differed, but the design of the public consultation made it possible to distinguish between 
the respondents who were well-informed and others. This distinction fed into the analysis 
of the responses, and triangulated with other sources of evidence. 

In  conclusion,  while  acknowledging  that  there  are  some  data  gaps  and  methodical 
limitations  (see  above),  the  evaluation  presents  well-informed,  evidence-based  and 
reliable answers to the questions, to the extent possible.  

86