COMMISSION STAFF WORKING DOCUMENT EVALUATION Ex Post Evaluation of the 2014-2020 European Social Fund (ESF) and Youth Employment Initiative (YEI)
Date: 2026-02-10
Institutions: Directorate-General for Employment, Social Affairs and Inclusion, European Commission
Subjects: EU employment policy, EU initiative, European Social Fund, employment aid, fight against unemployment, job creation, labour market, social integration, youth employment, youth unemployment

EN 

  EN 

   EUROPEAN COMMISSION  Brussels, 10.2.2026  SWD(2026) 60 final  COMMISSION STAFF WORKING DOCUMENT  EVALUATION  Ex Post Evaluation of the 2014-2020 European Social Fund (ESF) and Youth Employment Initiative (YEI)            {SWD(2026) 61 final}  
 
 
Table of contents 

1. 

INTRODUCTION ................................................................................................................. 4 

1.1. 

PURPOSE AND SCOPE OF THE EVALUATION ..................................................................................... 4 

1.2. 

EVALUATION METHODOLOGY AND LIMITATIONS ........................................................................... 6 

2.  WHAT WAS THE EXPECTED OUTCOME OF THE INTERVENTION .................... 9 

2.1. 

DESCRIPTION OF THE INTERVENTION AND ITS OBJECTIVES ............................................................. 9 

2.2. 

POINT(S) OF COMPARISON ............................................................................................................ 17 

3.  HOW HAS THE SITUATION EVOLVED OVER THE EVALUATION PERIOD? .. 18 

3.1. 

ECONOMIC AND SOCIAL EVOLUTION ............................................................................................ 18 

3.2. 

FINANCIAL AND OPERATIONAL IMPLEMENTATION OF ESF AND YEI ............................................ 21 

4. 

EVALUATION FINDINGS ............................................................................................... 25 

4.1. 

TO WHAT EXTENT WAS THE INTERVENTION SUCCESSFUL AND WHY? ........................................... 25 

4.2. 

DID ESF AND YEI MAKE A DIFFERENCE AND TO WHOM? ............................................................. 58 

4.3. 

ARE THE ESF AND YEI STILL RELEVANT? ................................................................................... 64 

5.  WHAT ARE THE CONCLUSIONS AND LESSONS LEARNED? .............................. 70 

6.  ANNEXES............................................................................................................................ 84 

ANNEX I:   PROCEDURAL INFORMATION ............................................................................. 85 

ANNEX II:  METHODOLOGY AND ANALYTICAL MODELS USED .................................... 86 

ANNEX II.A:  METHODOLOGY FOLLOWED BY EXTERNAL CONTRACTOR ......................... 87 

ANNEX II.B:  INTERVENTION LOGIC ....................................................................................... 122 

ANNEX II.C:  EVALUATION MATRIX ........................................................................................ 142 

ANNEX II.D:  ADMINISTRATIVE COSTS ................................................................................... 156 

ANNEX III:  OVERVIEW OF BENEFITS AND COSTS .......................................................... 157 

ANNEX IV:  STAKEHOLDERS CONSULTATION - SYNOPSIS REPORT ........................... 157 

ANNEX V: KEY DATA AND ADDITIONAL INFORMATION .............................................. 168 

 
 
 
 
 
 
 
 
 
Term or acronym 

Meaning or definition 

Glossary 

AIR  

AMIF 

AROPE 

CARE 

CBA  

CF 

CIE 

CLLD 

CPR 

CRII/CRII+  

CSOs 

CSRs  

DG EMPL  

EAFRD 

EaSI 

ESI 

EESC 

EGF 

ERDF  

EMFF 

EPSR 

ESIF  

ESF  

ESF+ 

ESL  

Annual Implementation Report 

Asylum, Migration and Integration Fund 

At-risk of poverty or social exclusion 

Cohesion’s Action for Refugees in Europe 

Cost-Benefit Analysis 

Cohesion Fund 

Counterfactual Impact Evaluation 

Community-Led Local Development  

Common Provisions Regulation 

Coronavirus Response Investment Initiative/ Coronavirus 
Response Investment Initiative Plus 

Civil Society Organisations 

Country specific recommendations 

Directorate-General for Employment, Social Affairs & Inclusion 

European Agricultural Fund for Rural Development 

Employment and Social Innovation 

European Structural and Investment  

European Economic & Social Committee 

European Globalisation Adjustment Fund 

European Regional Development Fund 

European Maritime and Fisheries Fund 

European Pillar of Social Rights 

European Structural and Investment Funds  

European Social Fund 

European Social Fund Plus 

Early school leaving 

 
 
 
 
 
 
ET 2020  

EU  

EUR  

EURES 

FEAD 

GDP 

ILs 

IP  

JRC 

JTF 

MA  

MS 

NEET  

NGO 

NUTS  

OP  

PA  

PES 

Strategic framework for European cooperation in education and 
training 2020 

European Union  

Euros 

EURopean Employment Services 

Fund for European Aid to the Most Deprived 

Gross Domestic Product 

Intervention Logics 

Investment Priority 

Joint Research Centre 

Just Transition Fund 

Managing Authority 

Member State 

Not in Education, Employment and Training 

Non-Governmental Organisation 

Nomenclature of Territorial Units for Statistics  

Operational Programme 

Partnership Agreement 

Public Employment Service 

REACT-EU 

Recovery Assistance for Cohesion and the Territories of Europe 

RRF 

SCOs 

SFC  

SILC 

SURE 

SWD 

TA 

TO  

Recovery and Resilience Facility 

Simplified Cost Options 

System for Fund Management in the European Union 

Statistics on income and living conditions 

Support to mitigate Unemployment Risk in an Emergency 

Staff Working Document 

Technical Assistance 

Thematic Objective  

 
 
 
 
 
UK 

YEI  

United Kingdom 

Youth Employment Initiative 

 
 
 
 
 
 
1. 

INTRODUCTION 

1.1. 

PURPOSE AND SCOPE OF THE EVALUATION 

The European Social Fund (ESF) is the main funding instrument through which the European 
Union translates its labour market,  education, and social inclusion policies into actions.  The 
2014-2020  ESF  programme  had  the  objective  of  promoting  employment,  helping  people  to 
access better and fairer job opportunities, supporting upskilling and reskilling, fostering social 
inclusion, and reinforcing the institutional capacities of public administrations, public services 
and  stakeholders.  Following  Article  174  of  the  Treaty  on  the  Functioning  of  the  European 
Union1 (TFEU), the ESF contributes  to the  economic, social,  and territorial cohesion  of the 
European Union (EU).  

During 2014-2020, the ESF responded to four out of the eleven Thematic Objectives (TO) that 
were identified across the European Structural and Investment (ESI) Funds: TO 8 which aimed 
at  promoting  sustainable  and  quality  employment  and  supporting  labour  mobility;  TO  9, 
promoting  social  inclusion,  combating  poverty  and  any  discrimination;  TO  10,  investing  in 
education and training for skills and lifelong learning; and TO 11, enhancing the capability of 
public authorities and efficient public administration. Recovery Assistance for Cohesion and 
the Territories of Europe (REACT-EU2) was established as a new thematic objective (TO13). 
At the same time, the ESF's support for these TO’s was guided by specific Investment Priorities 
(IPs),  which  outlined  the  key  areas  for  investments.  Additionally,  the  ESF  supported  the 
implementation of the Youth Employment Initiative (YEI)3 through IP 8.ii, targeting regions in 
which youth unemployment was over 25%. 

The purpose of this evaluation is to assess the achievements of European Social Fund and Youth 
Employment  Initiative  during  the  programming  period  2014-2020.  It  also  provides  lessons 
learned, which can be relevant for the implementation of the European Social Fund Plus (ESF+) 
and contribute to the design of future policies and programmes beyond 2027. 

This staff working document (SWD) presents the main findings of the evaluation in accordance 
with Article 57 of the Regulation (EU) No 1303/20134. It complies with the provisions of the 
European Commission’s Better Regulation Guidelines5, which envisage the assessment of five 
evaluation criteria: (i) to which extent the objectives have been achieved — i.e. effectiveness; 

1 Available at: https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:12016ME/TXT  
2 Recovery Assistance for Cohesion and the Territories of Europe (REACT-EU) was a crisis response programme 
launched in December 2020, see more information in footnote 13. 
3 The Youth Employment Initiative was launched by the European Union in 2013 as part of broader efforts to 
combat youth unemployment across member states. It specifically targeted regions with youth unemployment rates 
above 25%, aiming to address the challenges faced by young people, particularly those who are not in employment, 
education,  or  training  (NEETs).  The  YEI  complemented  the  ESF  activities  by  providing  additional funding  to 
support these regions. This funding was used for a variety of interventions, including apprenticeships, traineeships, 
job placements, and continued education and training opportunities. 
4  During  the  2014-2020  period,  the  ESF  was  governed  by  the  European  Parliament  and  Council  Regulation 
1303/2013 (Common Provisions Regulation (CPR hereafter)) and by the Regulation 1304/2013 on the European 
Social Fund (ESF). Additional information on legal requirements available in Annex V. 
5 Better Regulation Guidelines; https://ec.europa.eu/info/law/law-making-process/planning-and-proposing-
law/better-regulation-why-and-how/better-regulation-guidelines-and-toolbox/better-regulation-toolbox-0_en  

4 

 
 
 
 
(ii)  whether  operations  were  cost-effective  —  i.e.  efficiency;  (iii)  to  what  extent  were  the 
objectives and operations funded relevant to the needs of target groups — i.e. relevance; (iv) 
whether the EU support was internally coherent, as well as coherent with other EU funds and 
policies — i.e. coherence; and (v) whether it has produced results beyond what would have 
been achieved by Member States acting alone — i.e. EU added value.  All  these criteria are 
assessed  in  depth  through  specific  evaluation  questions6.  The  evaluation  also  analyses 
sustainability, visibility and simplification7 assessing the fund’s performance in relation to the 
objectives set out in the ESF Regulation8.  

This evaluation covers the implementation years 2014 to 2022. It considers the European Union 
27 Member States as well as the United Kingdom (UK). It covers all ESF TOs and ESF IPs, 
including  YEI,  the  Coronavirus  Response  Investment  Initiative9  and  Coronavirus  Response 
Investment Initiative Plus10 (CRII/CRII+), Cohesion’s Action for Refugees in Europe (CARE11) 
and REACT-EU12 support13. 

The findings and lessons learned are mainly based on the supporting study “Study supporting 
the  ex-post  evaluation  of  the  2014-2020  European  Social  Fund  and  Youth  Employment 
Initiative” (hereafter also called the “supporting study” or the “study”) that was commissioned 
to  an  external  contractor14,  which  triangulates  a  broad  range  of  sources  of  evidence. 
Furthermore, important conclusions on the impact of the ESF and YEI were drawn from the 
Meta-analysis of the European Social Fund counterfactual impact evaluations performed by the 
Joint Research Centre (JRC) of the European Commission15, as well as the working paper “The 

6 See Annex II.C for a detailed presentation of the evaluation questions followed. 
7  Visibility  is  analysed  in  section  4.3  as  a  separate  sub-section.  The  analysis  of  sustainability  is  part  of  the 
effectiveness criterion and simplification as part of the efficiency criterion. 
8 Regulation (EU) No 1304/2013 
9 CRII-Regulation (EU) 2020/460 - https://eur-lex.europa.eu/eli/reg/2020/460/oj 
10 CRII+ Regulation (EU) 2020/558 - https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32020R0558   
11 CARE- Regulation (EU) 2022/562 - https://eur-lex.europa.eu/legal-
content/EN/TXT/?uri=uriserv:OJ.L_.2022.109.01.0001.01.ENG. The Council adopted the Cohesion's Action for 
Refugees in Europe (CARE) in April 2022, amending funding frameworks to address both the pandemic's 
impact as well as refugee inflows, providing flexibility for resource transfers between programmes financed by 
the ESF and ERDF 
12 REACT-EU-Regulation (EU) 2021/177 - https://eur-lex.europa.eu/legal-
content/EN/TXT/?uri=celex%3A32021R0177. Report from the Commission to the European Parliament and the 
Council on the evaluation of REACT-EU. Published 10 October 2025. Available at: 
COM_2025_634_F1_REPORT_FROM_COMMISSION_EN_V4_P1_4084848.pdf 
13 Coronavirus Response Investment Initiative/ Coronavirus Response Investment Initiative Plus (CRII/CRII+), 
Cohesion’s Action for Refugees in Europe (CARE) and Recovery Assistance for Cohesion and the Territories of 
Europe (REACT EU) were crisis response programmes.  CRII/CRII+ was launched in March and April 2020 to 
address the COVID-19 pandemic, it was later supplemented by REACT EU, launched in December 2020 intended 
to support longer term recovery and in later allocations was intended to support other crises. CARE, launched in 
April 2022, supported Member States and regions in providing emergency assistance to displaced persons fleeing 
from Russia’s war of aggression against Ukraine. 
14  Available  at:  Study  supporting  the  ex-post  evaluation  of  the  2014-2020  European  social  fund  and  youth 
employment initiative - Publications Office of the EU and Annexes. Carried out by Ecorys Europe, Ismeri Europa, 
3s Unternehmensberatung GmbH, and Wmp consult, coordinated by the Commission’s Directorate-General for 
Employment, Social Affairs and Inclusion, with the support of an Inter Services Steering Group (ISSG) c.f. Annex 
I. 
15  Haepp,  T.  and  Serrano  Alarcon,  M.,  Meta-analysis  of  the  European  Social  Fund  counterfactual  impact 
evaluations: Brief update with alternative measures, European Commission, Ispra, 2024, JRC137571, Available 
at: https://publications.jrc.ec.europa.eu/repository/handle/JRC137571 

5 

 
 
 
ex-post  macroeconomic  evaluation  of  the  2014-2020  European  Social  Fund,  Youth 
Employment  Initiative  and  REACT-EU  labour  market  interventions”16  issued  by  the  JRC 
(hereafter  also  called  ‘JRC’s  working  paper’  or  ‘JRC’s  report’).  The  latter  provides  a 
quantitative analysis  of  macroeconomic impacts  of the  ESF,  the  YEI and  the labour market 
interventions  of  the  REACT-EU  programme,  using  the  spatial  dynamic  general  equilibrium 
model  RHOMOLO  to  analyse  how  GDP,  employment,  wages  and  various  measures  of 
inequality respond to ESF investment. 

1.2.  EVALUATION METHODOLOGY AND LIMITATIONS 

This evaluation uses as reference the Intervention Logics (ILs) of ESF and YEI as a whole, and 
those specific to each of the TOs (See Annex II.B). The ILs present the theory of change and 
guide  the  causal  pathways  between  the  programme’s  rationale,  objectives,  inputs,  measures 
taken,  output,  results  and  impacts  of  the  funds.  They  are  combined  with  the  evaluation 
framework (See Annex II.C), which addresses the research questions set out for each evaluation 
criterion described in Section 1.1. It presents the judgement criteria, indicators and sources that 
were  used  and  provides  the  elements  of  the  methodology  which  were  used  to  collect  the 
required evidence. The monitoring system established for ESF encompasses key components 
of  the  Intervention  Logics,  and  its  effectiveness  is  evaluated  in  the  report.  The  range  of 
qualitative and quantitative evidence gathered was assessed and triangulated to evaluate how 
the process and outcome causal chains depicted in the intervention logics were supported. 

The  evidence  and  methodology  used  are  designed  to  evaluate  these  cause-and-effect 
relationships. The range of evidence included 27 in-depth case studies17, which included 380 
interviews with country level representatives involved in the implementation and monitoring 
of the funds; 28 individual country factsheets18; 150 interviews conducted with EU and national 
stakeholders19 between February and June 2023; a survey to Managing Authorities conducted 
between March and June 202320; a public consultation21 published on Europa “Have your say” 
website, open for 12 weeks from 16 October 2023 to 23 January 2024; desk research to collect 
and analyse Operational Programmes (OPs) evaluations; monitoring data from the System for 
Fund Management in the EU (SFC) 2014; Annual Implementation Reports; a literature review; 
an assessment of the socio-economic context based on the evolution of relevant socioeconomic 
indicators during the evaluation period; JRC’s macro-economic analysis using the RHOMOLO 

16  Casas,  Pablo and Christou,  Tryfonas and García  Rodríguez,  Abián and Lazarou,  Nicholas  Joseph and Salotti, 
Simone (2025): The  ex-post  macroeconomic  evaluation  of  the  2014-2020  European  Social  Fund,  Youth 
Employment Initiative and REACT-EU labour market interventions. Link to the publication: https://mpra.ub.uni-
muenchen.de/123410/ 
17  See  Supporting 
https://url.de.m.mimecastprotect.com/s/qy8iCvQn4XFE2jAqHQfZsQRm4E?domain=webgate.ec.europa.eu  
18  See  Supporting 
https://url.de.m.mimecastprotect.com/s/tNu3Cw0o4YFvAzykHqhOsJUJJM?domain=webgate.ec.europa.eu  
19 See Annex IV summarising stakeholders’ consultations 
20 114 responses received achieving a 61% response rate, covering all EU27 Member States and the UK 
21 704 respondents took part in the public consultation  

study  Annexes.  All  country 

factsheets  are  available 

study  Annexes.  All 

to  download 

download 

available 

studies 

from: 

from: 

case 

are 

to 

6 

 
 
 
model22;  JRC’s  meta-analysis23  of  ESF  counterfactual  impact  evaluations;  as  well  as  a  cost-
benefit analysis24. Finally, 6 focus groups25 with key stakeholders took place on between 4 June 
2024 and 13 June 2024 to discuss preliminary evaluation findings and lessons learned. Further 
information on the methodology followed by the contractor of the study and the JRC support is 
provided in the Annex II. 

The evaluation research was subject to several limitations, associated with the availability of 
data,  the  differences  in  cut-off  dates  used,  the  data  quality,  representativeness  in  the 
consultations, or limited evidence on more recent programmes. To start with, socio-economic 
indicators  were  not  always  available  or  up  to  date.  Additionally,  incomplete  information  or 
inconsistencies in the ESF monitoring data submission in SFC201426 hindered the analysis to 
some extent. Due to missing data when producing the supporting study, several cut-off dates 
were agreed to perform the analysis27. Another difficulty present in the analysis of monitoring 
data was the lack of clear linkages in the reporting between output and results indicators and 
financial indicators in the monitoring system. This made the identification of linkages between 
the measures  and the intended target groups difficult and rendered the identification of  how 
much  the  achievement  of  a  specific  output  or  result  cost  difficult.  Furthermore,  the  limited 
evidence on recently introduced programmes such as CARE or REACT-EU at the moment of 
writing  the  report  hindered  detailed  information  on  their  impact  in  this  evaluation.  Lastly, 
overrepresentation  of  certain  stakeholders  in  the  public  consultation  (e.g.  47%  of  the  total 
responses coming from  Public Authorities, 43% of the total responses coming from Poland) 
and unavailability of some participants in the targeted consultations affected the interpretation 
of results. 

The table below presents an overview of the main limitations of the methodology and how the 
limitations  were  considered  during  the  supporting  study’s  design  and  implementation,  with 
measures introduced accordingly.  

22  Casas,  Pablo and Christou,  Tryfonas and García  Rodríguez,  Abián and Lazarou,  Nicholas  Joseph and Salotti, 
Simone (2025): The  ex-post  macroeconomic  evaluation  of  the  2014-2020  European  Social  Fund,  Youth 
Employment Initiative and REACT-EU labour market interventions. Link to the publication: https://mpra.ub.uni-
muenchen.de/123410/ 
23  Haepp,  T.  and  Serrano  Alarcon,  M.,  Meta-analysis  of  the  European  Social  Fund  counterfactual  impact 
evaluations: Brief update with alternative measures, European Commission, Ispra, 2024, JRC137571, Available 
at: https://publications.jrc.ec.europa.eu/repository/handle/JRC137571 
24 See Annex III 
25 Conducted in June 2024 with at total of 62 attendants.  
26 System for Fund Management in the European Union (SFC2014) 
27Notably, some submissions by Members States were incomplete or inconsistent, failing to adhere to established 
naming or classification rules. To address these issues, specific datasets and cutoff dates were agreed upon with 
the contractor of the supporting study. In September 2023, an extraction of financial and monitoring data from 
SFC2014, reflecting the status as of the end of 2022, was provided. This extraction served as the definitive data 
collection threshold for the study. However, the data from France’s national employment OP, unavailable in the 
the extract of September 2023 was incorporated into the final dataset in November 2024. Subsequent data cleaning 
included  translating  indicators  and  priority  axes  from  native  languages,  reformatting  numbers  for  large  data 
operability, reclassifying indicators to resolve code ambiguities, and updating NUTS2 region classifications to 
2021  standards.  These  measures  ensure  data  accuracy  and  consistency,  facilitating  reliable  analysis  of  ESF 
financial allocations, programme progress, and participant outcomes. 

7 

 
 
 
 
Table 1: Summary of study limitations and mitigating measures in place 

Limitations 

How it has been addressed 

Socio-economic indicators from official statistics 

•  UK data was unavailable from 2019 onwards, thus socio-
economic analysis for the UK was limited to data up until 
2019. 

•  Some time series data have not been updated or do not 
cover the entire evaluation period (for instance poverty 
and statistics on income and living conditions (SILC) are 
not articulated by region before 2020).  

•  Alternatives were sought for missing 
socio-economic indicators. If an 
indicator was not available for the year 
of interest, the latest available data was 
used and divergences were noted. 

Monitoring data and targets 

•  For the sake of evaluation, the expenditure data is only 

available up to 2022, because data on the subsequent years 
will only be released in 2026. 

•  ESF monitoring data in SFC2014 is largely complete but 

presents some inconsistencies and errors. 

•  Targets for the programme indicators are affected by 

several limitations: their coverage is not uniform, there is 
no available information about how they were defined, 
some changed during the period, their achievement 
highlights a high variability which reflects definition 
issues in addition to performance issues.  

•  Specific datasets and cutoff dates were 

used. 

•  Data cleaning measures, including 

translating indicators, reformatting data, 
and updating classifications, were 
applied to ensure data accuracy and 
consistency. 

Mapping of ESF-related investment priorities, target populations and types of operations 

•  The limited information included in the national lists of 
operations hampered a detailed categorisation of the 
interventions. 

•  REACT-EU operations were characterised by incomplete 
updates (both Kohesio and National Lists) and missing 
mandatory information on TOs.  

•  The identification of the CARE operations was constrained 

•  Secondary desk research, such as 

thematic reports, EU-level sources, and 
national evaluations, was used to fill 
information gaps. 

•  National evaluations were referenced 

where available to support the analysis. 

as not all OPs with a specific CARE axis allocated 
resources to projects by the end of 2022, making the 
analysis potentially incomplete. 

Public Consultation 

•  A high number of responses came from Poland (43%; 

n=305), which can introduce biases in the interpretation of 
results.  

•  Triangulation of data was undertaken, 

and appropriate clarifications were added 
when interpreting and reporting on the 
results of the Public Consultation. 

Case Studies 

8 

 
 
 
 
 
 
 
 
 
•  Case study reports were produced before the datasets were 
fully finalised, especially regarding longer-term result 
indicators, affecting the completeness of the monitoring 
data included in some reports.  

•  Due to the diversity of programme-specific indicators 

across MSs, programmes, and IPs, it was not feasible to 
analyse them. 

•  REACT-EU and CARE case studies: limited availability 
of desk study materials and sparse/absent evaluations due 
to the short and late-introduced projects within the ESF 
period, complicating access to official sources for most 
selected MSs; thus, the assessment heavily relies on 
interviews, with only few projects evaluated to date. 

•  When selected interviewees were 

unavailable, secondary desk research 
was conducted to supplement missing 
data. 

•  National evaluations, thematic reports, 
and other secondary sources were used 
to address gaps in the data. 

•  Findings on the sustainability, visibility, and 

•  The meta-analysis of impact evaluations 

Analysing in-depth the ESF impact evaluations 

was complemented with findings from 
interviews and other qualitative data 
where counterfactual data was 
unavailable. 

•  TO 11 was excluded from the scope due 

to lack of available evaluations. 

communications criteria are limited in national 
evaluations. 

•  Some MSs (IT, PL, DE) are more represented than others 

due to a higher number of reliable evaluations. 

•  TO 11 is not considered in the analysis as few evaluations 

addressed this thematic objective.  

•  Findings of the Meta-analysis of the ESF counterfactual 
impact evaluations were generally based on interventions 
implemented before 2020 and rarely included 
interventions implemented during or after the COVID-19. 
Findings did not include TO 9 and TO 10, because no, or 
too few, counterfactual evaluations were carried out in 
these thematic objectives. 

Source: Supporting study 

2.  WHAT WAS THE EXPECTED OUTCOME OF THE INTERVENTION 

2.1.  DESCRIPTION OF THE INTERVENTION AND ITS OBJECTIVES 

The European Social Fund is a key instrument of the European Union, rooted in Article 174 of 
the Treaty on the Functioning of the European Union, which underpins the EU's commitment 
to economic, social, and territorial cohesion. As a fundamental component of the EU's cohesion 
policy,  the  ESF  operated  through  a  shared  management  system,  where  the  European 
Commission and member states collaborated to  design  and implement  national and regional 
programmes. This collaborative approach allowed for tailored responses to the unique socio-
economic  challenges  faced  by  different  regions,  leveraging  the  ESF's  resources  to  promote 
employment, enhance education, training and skills development, and foster social inclusion 
across Member States. 

Within the framework of Cohesion policy and the Europe 2020 strategy, the ESF's primary role 
was  focused  on  addressing  key  challenges  related  to  employment,  social  inclusion,  and 
education, focusing on four Thematic Objectives (TO 8, TO 9, TO 10, TO 11)28. Following the 
COVID-19 pandemic, TO 13 was introduced with the financing of REACT-EU to address crisis 
response and recovery. It provided additional resources to the ESF and the European Regional 

28 For further explanation on each TO as well as a separate description for the YEI, see Annex II.B 

9 

 
 
 
 
Development  Fund  (ERDF)  to  support  employment,  social  inclusion,  healthcare,  and  the 
transition to a green and digital economy29. 

The CPR and the ESF Regulation introduced several mechanisms to enhance the effectiveness 
and  impact  of  the  ESF.  These  included:  (i)  a  minimum  share  of  cohesion  policy  funding 
allocated  to  the  ESF  (as  required  in  Art  92  of  the  CPR);  (ii)  thematic  concentration  of  the 
Operational  Programmes,  by  which  all  of  them  needed  to  comply  with  the  requirement  of 
focusing a minimum share of ESF allocation to the five largest IPs, which are those that bring 
the greatest added value to the Europe 2020 strategy (60% for OPs in less developed regions, 
70% for transition regions, and 80% for more developed regions) (Art 4 of the CPR); and (iii) 
an allocation of 20% of the total ESF resources to social inclusion (TO 9). 

The overall Intervention Logic of the ESF and YEI 

The  ESF  and  YEI  invested  in  human  capital  with  a  view  to  achieving  the  employment  and 
social policy goals supported by Article 174 of the Treaty on the Functioning of the European 
Union  and  explained  under  Section  1.1.  The  Intervention  Logic  represents  the  underlying 
framework  that  guides  the  implementation  of  the  European  Social  Fund  and  the  Youth 
Employment  Initiative  during  the  2014-2020  programming  period.  It  outlines  the  causal 
relationships  between  the  investments  made,  the  activities  implemented,  and  the  expected 
outcomes and impacts of the programmes. In essence, the intervention logic explains how the 
ESF and YEI interventions are designed to address specific social and economic challenges, 
and  how  they  are  expected  to  contribute  to  the  achievement  of  the  programmes'  overall 
objectives. By mapping out the sequence of events, from the initial needs assessment to the 
ultimate  objectives,  the  intervention  logic  provides  a  clear  and  coherent  narrative  of  the 
programmes' theory of change, allowing stakeholders to  understand the rationale behind  the 
investments and the expected results. This section presents the Intervention Logic of the ESF 
and YEI, highlighting the key components, inputs, and expected outcomes of the programmes, 
and  providing  a  foundation  for  understanding  the  results  and  impacts  presented  in  the 
subsequent sections of this report. 

The 2014-2022 implementation period was characterised by evolving needs and the emergence 
of new crisis. Overall, the EU has faced a persisting need to address youth unemployment30, as 
well  as  different  needs  that  appeared  and  evolved  during  the  programming  period.  The 
implementation  period  was  characterised,  at  the  beginning,  by  the  enduring  effects  of  the 
financial  crisis  of  2008-2012,  which  then  slowly  faded  away,  and  in  the  last  years,  by  the 
emergence of new crises. The COVID-19 pandemic severely affected the EU economy in 2020, 
straining public health and social protection systems, and the Russia’s war of aggression against 
Ukraine  has  further  exacerbated  the  challenges  facing  the  UE,  prompting  the  need  for 
adaptability of the fund31. Furthermore, the labour market faced increasing skills mismatches 
over  time,  hindering  the  adaptation  of  the  workforce  to  the  demands  of  a  rapidly  evolving 
economy and  limiting the potential  of businesses, notably small  and medium enterprises,  to 
create jobs and drive economic growth. These interconnected challenges underscored the need 

(EU)  No 

29REACT-EU  Regulation, 
content/EN/TXT/?uri=CELEX%3A32020R2221) 
30 To provide support focusing on the integration of young people in the labour market, YEI was implemented into 
ESF. Action under the IP 8.ii (See Article 3(1)(a)(ii) of the ESF Regulation). 
31 See Section 3.1 for an understanding on the economic and social evolution. 

https://eur-lex.europa.eu/legal 

2020/2221 

(Available 

at 

: 

10 

 
 
 
for  targeted  interventions  that  could  address  the  complex  and  multifaceted  nature  of 
unemployment, skills, education and social exclusion. 

In view of these described needs, Article 2 of the ESF Regulation defined that the ESF aims, 
among others, to promote high levels of employment and job quality, improve access to the 
labour market, support workers’ adaptation to change, encourage a high level of education and 
training for all, support the transition from education to employment for young people, combat 
poverty, enhance social inclusion, and promote gender equality, non-discrimination and equal 
opportunities.  The  ESF  also  aimed  to  support  Member  States  in  pursuing  the  priorities  and 
headline  targets  of  the  Europe  2020  Strategy  taking  into  account  the  country-specific 
recommendations (CSRs) issued in the context of the European Semester. In addition, the YEI’s 
main objective was to support the fight against youth unemployment in the worst-affected EU 
regions. It  targeted all people aged 15 to  24 and paid  a particular attention to  young people 
neither in employment, nor in education or training (NEETs).  The initiative is closely linked 
to the Council recommendation establishing a Youth Guarantee, which was formally adopted 
by the Council of the European Union in April 2013. The Youth Guarantee aims to ensure that 
every young person under 25 receives a good quality offer of employment, continued education, 
an  apprenticeship,  or  a  traineeship  within  four  months  of  becoming  unemployed  or  leaving 
formal education, thus reinforcing the efforts of the YEI by providing a strategic framework for 
Member States to enhance youth employment and training opportunities. In order to contribute 
to the general objectives, specific objectives of ESF during the programming period 2014-2020 
were defined under the thematic objectives set out in points (8), (9), (10) and (11) of the first 
paragraph of Article 9 of  the CPR, as  well as specified in the  points  (a), (b),  (c) and (d) of 
Article 3 of the ESF Regulation. Namely, the ESF aimed at: promoting sustainable and quality 
employment  and  supporting  labour  mobility  (TO  8);  promoting  social  inclusion,  combating 
poverty and any discrimination (TO 9); investing in education, training and vocational training 
for skills and life-long learning (TO 10); enhancing institutional capacity of public authorities 
and stakeholders and efficient  public administration (TO 11);  and  supporting  and protecting 
employment, reinforcing healthcare systems as a response to COVID-19 (TO 13). Through the 
CARE initiative, the ESF also aimed at addressing the crisis of refugees following Russia’s war 
of  aggression  against  Ukraine.  Further  provisions  are  laid  out  in  Articles  6-10  of  the  ESF 
Regulation relating to the horizontal principles, social innovation and transnational cooperation.  

In addition, ESF operational objectives emerge from the specific objectives. They are defined 
as the investment priorities defined in Article 3 of the ESF Regulation. Among others32, ESF 
aims  to  support  the  access  to  employment  for  job-seekers  and  inactive  people;  sustain  the 
integration of young people in the labour market; achieving equality between men and women 
in all areas, contribute to active inclusion; enhancing access to affordable, sustainable and high-
quality  services;  promoting  social  entrepreneurship;  reducing  and  preventing  early  school-
leaving; improving the quality and access to tertiary and equivalent education; enhance equal 
access to lifelong learning for all age groups; promote capacity building for stakeholders.  

The  implementation  of  the  ESF  and  YEI  programmes  during  the  2014-2020  programming 
period relied on a range of resources and investments, which are referred to as 'inputs' in the 

32 For a detailed list on the investment priorities under each Thematic Objective, see Annex II.B and Figure 1: 
Intervention Logic of the ESF (including REACT-EU and excluding YEI) 

11 

 
 
 
 
intervention logic. These inputs represent the financial, human, and material resources that were 
mobilized to support the implementation of the programmes, including the allocation of EU and 
national  funding,  the  engagement  of  stakeholders  and  partners,  and  the  deployment  of 
institutional and administrative capacities. Together with the national contributions, the total 
ESF and YEI 2014-2020 budget was EUR 150.9 billion (current prices), which included the 
allocation to ESF of EUR 120.3 billion, EUR 20.1 billion by REACT-EU, as well as slightly 
over EUR 10.4 billion that was reserved for implementation of the YEI.  The EU contribution 
of this budget was EUR 113.4 billion, consisting of of the original ESF allocation (EUR 84.5 
billion),  the  allocations  to  ESF  from  REACT-EU  (EUR  19.9  billion)  and  a  share  (EUR  8.9 
billion)  that  was  allocated  to  YEI  funding.  This  amounts  to  an  average  co-financing  rate  of 
75%,  meaning  that  75%  of  the  total  budget  was  financed  by  the  EU,  and  25%  by  national 
contributions.  In  percentage  terms,  the  funding  was  distributed  as  follows  across  various 
initiatives: 80% of the total EU and national contribution was allocated to the ESF (of which 
49.5% was dedicated to less developed regions, 13.6% to transition regions, and 36.9% to more 
developed  regions),  7%  to  the  YEI,  and  13%  to  REACT-EU.  Allocations  per  Thematic 
Objective are available in the Annex II.B and section 3.2. 

The inputs are intended to support activities that contribute to achieving the objectives. In the 
context of the  ESF and  YEI, these activities are  delivered through  Operational  Programmes 
(OPs),  which  are  the  operational  plans  signed  in  partnership  with  national  and  regional 
authorities in Member States (MS) outlining particular areas where the fund is to be invested. 
During the 2014-2022 evaluation period, the ESF was implemented in 27 Member States and 
the United Kingdom through an overall number of 188 OPs, including mono and multi-fund 
programmes.  Three  of  these  OPs  (in  Greece,  Spain  and  France)  were  fully  dedicated  to 
Technical  Assistance  (TA)33.  The  activities  implemented  through  these  OPs  are  diverse, 
including support to human capital and skills, matching labour market demand and supply, and 
self-empowerment and social inclusion measures. The YEI was implemented in 19 Member 
States and the United Kingdom with 34 Operational Programmes supported. 

Each step of the intervention logic corresponds to specific parts of the monitoring framework 
of  ESF.  Under  shared  management,  the  investments  are  monitored  by  financial  indicators 
broken  down  by  thematic  policy  fields  (the  thematic  objectives).  The  policies’  outreach  is 
depicted by (common and programme specific) output indicators. Result indicators describe the 
change in the participant’s situation. 

The activities implemented through the OPs are designed to produce direct and tangible outputs. 
Outputs represent the immediate products or deliverables of the activities. They are the direct 
consequence  of  the  activities  and  are  expected  to  contribute  to  the  achievement  of  the 
operational objectives of the intervention logic. These outputs were recorded through common 
output indicators, which provided a standardised way of measuring the results of the funds. The 
outputs achieved through the ESF and YEI included, for example, the number of participations 
in  training  and  employment  programmes,  broken  down  by  categories  such  as  employment 
status,  age, educational  attainment,  and disadvantaged groups34.  In terms  of entities, outputs 
cover  the  number  of  projects  implemented  by  social  partners  or  NGOs,  number  of  projects 
dedicated to sustainable participation and progress of women in employment, projects targeting 
public administration or public services and supported SMEs. By tracking these outputs, it was 

33 Other countries also have TA, but not in the form of a dedicated OP, but rather as part of other OPs. 
34 ESF Regulation Annex I specifies the list of common ESF and YEI output indicators 

12 

 
 
 
possible  to  assess  the  immediate  impact  of  the  activities  and  identify  areas  where  the 
intervention was having a direct and tangible effect. 

By the end of 2022 a total of 64.6 million participations35 were recorded across ESF and YEI 
interventions,  including  REACT-EU  operations.  Moreover,  the  ESF  has  supported  over  2 
million small and medium-sized enterprises from 2014 to 2022. More details on the outputs 
achieved are reported in Sections 3.2 and 4.1. 

The outputs achieved through the ESF and YEI interventions were expected to lead to a set of 
intended  results,  addressing  the  general  and  specific  objectives  of  the  programmes.  These 
results represent the changes or effects that occur as a consequence of the outputs, and are a key 
component of the intervention logic (i.e. among others, to promote high levels of employment 
and job quality, improve access to the labour market, support workers’ adaptation to change, 
encourage  a  high  level  of  education  and  training  for  all,  support  the  transition  between 
education and employment for young people, combat poverty, enhance social inclusion, and 
promote gender equality). The results can be categorised into two types: immediate and longer-
term. Immediate results occur when a participant leaves the operation, and are reported through 
common immediate result indicators, measuring for instance inactive participants engaged in 
job searching, participants gaining qualifications, or participants in employment. They refer to 
individuals and not to entities. For YEI operations, immediate results include, among others, 
unemployed  participants  completing  a  YEI  intervention,  participants  receiving  an  offer  of 
employment, continued education, apprenticeship or traineeship upon leaving, etc. 

Longer-term results, on the other hand, are defined to capture effects that occur six months after 
a  participant  has  left  the  operation,  and  measure  for  instance  the  number  of  participants  in 
employment,  with  an  improved  labour  market  situation.  For  YEI  interventions,  longer-term 
results  include  participants  in  continued  education,  training  programmes  leading  to  a 
qualification, an apprenticeship or a traineeship six months after leaving;  in employment, in 
self-employment,  etc.  In  addition,  OPs  may  use  programme-specific  output  and/or  result 
indicators, which capture very concrete aspects of certain programmes36.  

The outputs described above led to positive ESF and YEI immediate results (after leaving the 
operation) for 22.3 million participants corresponding to a success rate of 35%. More details on 
the results achieved are depicted in Section 3.2 and 4.1 of the report. 

The  immediate  results  achieved  through  the  ESF  and  YEI  interventions  are  expected  to 
contribute  to  a  set  of  broader,  long-term  impacts  that  reflect  the  ultimate  objectives  of  the 
programmes. Impacts represent the lasting and profound effects of the interventions. They are 
the culmination of the activities, outputs, and results, and are expected to have a significant and 
lasting  influence  on  the  social  and  economic  context  in  which  they  operate.  In  contrast  to 
results, which are more immediate and tangible, impacts are more strategic and far-reaching, 

35 Output and result indicators count “participations” rather than “participants”, as one person can take part in ESF 
programmes  multiple  times,  however,  for  readability  purposes,  the  report  uses  the  term  “participations”  and 
“participants” interchangeably. Data based on the supporting study. Data were extracted in September 2023.  
36 ESF Regulation Annex I and II specify the list of common ESF and YEI immediate result indicators and longer-
term  result  indicators.  For  more  information,  see  also  the  Guidance  document  “Monitoring  and  Evaluation  of 
European Cohesion Policy: European Social Fund” (2018) published by the European Commission. 

13 

 
 
 
such as the improvement of labour and social conditions in the EU, the support to disadvantage 
groups, the modernisation and support of innovative social support strategies and tools. 

Under shared management, the net contribution of the policies’ positive impact is estimated at 
the level of the individual, using counterfactual impact evaluations (CIEs) performed at national 
level,  with  support  from  the  European  Commission.37  As  far  as  labour  market  impacts  are 
concerned,  the  results  of  CIEs  are  summarised  via  a  meta-analysis  (first  edition  in  202238, 
regularly updated39 since) that allows to compute average impacts across the EU. To estimate 
the ESF and YEI’s macroeconomic effect, the Commission relies on JRC’s Rhomolo model. 
Other expected impacts are measured by reported stakeholders’ perceptions. 

The intervention logic describes how the ESF and YEI are expected to work and what are the 
expected outcomes, results and impacts. However, the ESF and YEI funded policies might also 
have unintended effects which are not part of the intervention logic. Some unintended effects 
emerged  during  the  evaluation  fieldwork,  mainly  concerning  the  occurrence  of  “creaming" 
effect in some instances (e.g. TO8), involving a tendency to focus support on groups with less 
complex  needs.  According  to  the  analysis,  such  effect  sometimes  occurred  but  did  not 
significantly affect  the functioning  of operations and the intervention logic presented. Other 
unintended effects  that emerged during the implementation of the funds  are analysed  where 
found to be relevant, such as in the analysis on efficiency (e.g. unintended inequalities in the 
application process and requirements).

37DG  EMPL,  together  with  the  JRC  and  DG  REGIO,  support  work  with  the  national  authorities  to  improve 
evaluation capacity and culture, including capacity to conduct counterfactual impact evaluations. However, access 
to data poses a significant challenge. There is a lot of data available in administrative registers, such as employment 
registers or tax registers, which can be useful for evaluating policies financed by the ESF. Due to uncertainties 
about the legal basis often this data is often not available for evaluations, leading to a lack of evidence on policy 
effectiveness. 
38 Meta-analysis of the ESF counterfactual impact evaluations - Publications Office of the EU 
39 JRC Publications Repository - Meta-analysis of the European Social Fund counterfactual impact evaluations: 
Brief update with alternative measures 

14 

 
 
 
Figure 1: Intervention Logic of the ESF (including REACT-EU and excluding YEI) 

Source: Supporting Study 

15 

 
 
 
Figure 2: Intervention Logic of YEI 

Source: Supporting Study

16 

 
 
 
 
2.2. 

POINT(S) OF COMPARISON  

The evaluation covers the implementation period from 2014 to 2022. It uses key comparison 
points  or  evidence  to  perform  the  analysis  across  the  evaluation  criteria  (when  evidence  is 
available, and the comparison is relevant and informative). The following items are considered 
for comparison:  

•  Data from the ESF during the previous programming period between 2007-2013, which 
allowed a direct comparison with 2014-2020 performance. This includes the following: 

o  The  share  of  participations  during  2007-2013  resulting  in  positive  achievements 
including  job  searching,  gaining  a  qualification  or  in  education/training  or  in 
employment upon leaving; 

o  The EU budget of the ESF in the previous programming period.;  
o  number of entities successfully supported in 2007-2013 
o  ESF’s key target groups reached during 2007-2013  

•  ESF  and  YEI  before  and  after  the  COVID-19  pandemic  to  assess  and  account  for  the 
changed delivery context as a result of this significant crisis affecting all Member States;  
•  Findings from the interim thematic evaluation reports40 completed, allowing an assessment 

of progress since the earlier part of the programming period; 

•  Socio-economic  context  indicators  from  2013  (as  a  baseline  before  the  2014-2020 

programme) allowing an assessment of change; 

•  Findings  from  other  related  research  or  evaluations  at  EU  or  national  level,  including 
impact  evaluations  with  counterfactuals  to  add  context  and  additional  information  to 
assessments made. 

This comparative analysis enables comparison points for the assessment of the support provided 
by  ESF  and  YEI  throughout  the  evaluation  report  (assessing  them  jointly  or  separately  as 
possible and specified in the text). The comparison can only be indicative, however, due to the 
influence of other contextual factors over time.  

40 For example: Fondazione G. Brodolini, Metis GmbH, Applica, Ockham IPS (2020) Study for the Evaluation 
of ESF Support to employment and Labour Mobility, for the European Commission, Directorate General for 
Employment, Social Affairs and Inclusion.  
European Commission, Directorate-General for Employment, Social Affairs and Inclusion, Donlevy, V., Sennett, 
J., Georgallis, M. et al., Study for the evaluation of ESF support to education and training (Thematic Objective 10) 
– Final report, Publications Office, 2020, available at: https://data.europa.eu/doi/10.2767/301732 
Metis GmbH, Fondazione G. Brodolini, Applica, Ockham IPS (2020) Study for the Evaluation of ESF Support to 
Youth  Employment,  for  the  European  Commission,  Directorate-General  for  Employment,  Social  Affairs  and 
Inclusion. p.141. Available at: https://op.europa.eu/en/publication-detail/-/publication/d063ca61-bcd8-11ea-811c-
01aa75ed71a1 
ICF, Cambridge Econometrics and Eurocentre (2020) Study supporting the 2020 evaluation of promoting social 
inclusion,  combatting  poverty  and  any  discrimination  by  the  European  Social  Fund  (Thematic  Objective  09). 
p.120.  Available 
https://op.europa.eu/en/publication-detail/-/publication/8788ec85-2308-11eb-b57e-
at: 
01aa75ed71a1/language-en 
Atkinson, I. et al (2023) Study supporting the preliminary evaluation of the support provided by ESF and FEAD 
at: 
under 
https://ec.europa.eu/social/main.jsp?catId=738&langId=en&pubId=8542&furtherPubs=yes   

the  Coronavirus  Response 

and  CRII+).  Available 

Investment 

Initiatives 

(CRII 

17 

 
 
 
 
3.  HOW HAS THE SITUATION EVOLVED OVER THE EVALUATION PERIOD? 

3.1. 

 ECONOMIC AND SOCIAL EVOLUTION 

3.1.1 ECONOMIC EVOLUTION  

Following  the  double-dip  recession  caused  by  the  financial  crisis  in  2007-2008,  the  EU 
economy grew steadily around 2% per year in the first part of the evaluation period. At the 
same time, employment growth resumed and the unemployment rate dropped gradually to 6.8% 
of the population in the labour force in 2019 (see Figure 3: Evolution of the economic situation in 
the  EU27  (2007-2022)).  The  outbreak  of  the  COVID-19  pandemic  led  to  an  abrupt  halt  of 
economic activity in 2020, leading to sharp drops in real GDP. The impact on the labour market 
remained,  however,  limited  as  a  result  of  the  large-scale  roll-out  of  job  retention  schemes. 
However, the EU economy and the labour market recovered rapidly, supported by public policy 
efforts  at  the  national  and  EU  level  and  also  proved  resilient  in  light  of  Russia’s  war  of 
aggression against Ukraine that resulted in high energy prices and high inflation in 2022. All 
throughout  the  evaluation  period,  structural  changes,  such  as  population  aging,  rapid 
digitalisation and the green transition, impacted the demand for occupations and required skills.  

Figure 3: Evolution of the economic situation in the EU27 (2007-2022)41 

Source: Eurostat  

3.1.2 SOCIAL EVOLUTION  

Against  this  background,  the  ESF  intervention  tailored  employment,  welfare,  education  and 
training  policies  to  include  those  excluded  from  the  labour  market,  workers  in  precarious 
situations and those lacking access to essential social, housing, and health care services. It also 
sought  to  address  disparities  across  Member  States  in  terms  of  unemployment  rates,  labour 
market tightness and with respect to social inequalities and remaining concerns with respect to 

41 The unemployment rate is only available since 2009 based on a comparable methodology.  

18 

24681012-6-4-2024682007200820092010201120122013201420152016201720182019202020212022Real GDP growth rateEmployment growth rateUnemployment rate (rhs)y-o-y% change% ofpopulation in the labour force 
 
 
 
 
equal  access  to  high  quality  education  and  jobs,  especially  for  the  young  as  well  as  skills 
mismatches and high school drop-out-rates.  

The ESF intervention was strongly linked to the Europe 2020 Strategy42 launched in 2010 and 
its headline targets related to the ESF set for the end of 2020: 

➢  At least 75 % of the population aged 20-64 should be employed. 

➢  The share of early school leavers should be under 10% and at least 40% of the younger 

generation should have a tertiary degree.  

➢  20 million less people should be at risk of poverty. 

At the ESF programme level, the focus on social innovation and a specific earmark of 20% of 
the programme's resources for social inclusion and combating poverty also reflects the link to 
the Europe 2020 Strategy. 

By the end of the evaluation period,  the employment rate of the population aged 20-64 rose 
from 66.8% in 2013 to 74.6% in 2022, the share of early school leavers dropped by more than 
2 pps. to 9.6% and the tertiary educational attainment of the population aged 30-34 increased 
to almost 43%. Also, some progress was made towards reducing the number of people at risk 
of poverty or social exclusion, but not to the same extent as for the other targets. Table A.13 in 
Annex VII presents the headline indicators associated with these targets43 for the EU27 average, 
as well as the new headline targets set for 2030 by the European Pillar of Social Rights44(EPSR) 
Action Plan45 in 2021.   

Despite  this  progress,  the  aforementioned  challenges  persisted  all  throughout  the  evaluation 
period.  

For labour market inequalities, this is for instance evidenced by persisting gaps in employment 
rates by gender, disability status and country of origin. The gender employment rate gap barely 
decreased  during  the  evaluation  period,  and  it  remained  high  in  2022  at  almost  11  pps. 
Furthermore, there was hardly a change in the disability employment rate gap (22.7 pps. in 2014 
vs. 21.4 pps. in 2022) and the gap between persons with foreign origin and natives has even 
increased between 2013 and 2022 (from 3.5 pps. to 4.1 pps.). Other persisting inequalities relate 
to  access  to  high  quality  jobs,  wages46  and  working  conditions.  In  addition,  the  COVID-19 
pandemic  might  have  undone  some  of  the  progress,  resulting  in  disproportional  declines  in 

at: 

42 European Commission (2010). Europe 2020 Strategy – A strategy for smart, sustainable and inclusive growth. 
Available 
https://ec.europa.eu/eu2020/pdf/COMPLET%20EN%20BARROSO%20%20%20007%20-
%20Europe%202020%20-%20EN%20version.pdf 
43 The labour market and education and training indicators in the Table are based on the European Union Labour 
Force Survey (EU-LFS), which revised some labour status definitions in 2021, causing a break in the time series.  
Eurostat provides break-free time-series for the most important labour market indicators, including employment 
rates by age and sex, while it does not provide break-free time series for the education and training indicators 
(https://ec.europa.eu/eurostat/statistics-explained/index.php?title=EU_labour_force_survey_-
correction_for_breaks_in_time_series#Comparability_over_time_-_correction_of_breaks_in_time_series).  The 
AROPE indicator is based on the EU-SILC survey which also underwent a revision in 2021, therefore there is a 
break in the time series since 2020.  
44 https://employment-social-affairs.ec.europa.eu/european-pillar-social-rights-20-principles_en 
45https://employment-social-affairs.ec.europa.eu/policies-and-activities/european-pillar-social-rights-building-
fairer-and-more-inclusive-european-union/european-pillar-social-rights-action-plan_en 
46 In the EU, the unadjusted gender pay gap on gross hourly wages, although narrowing, remained high: in 2014 
women’s earnings were 15.7% lower than those of men, dropping to 12.7% lower than men’s in 2021 and 2022 
(Eurostat  Gender 
https://ec.europa.eu/eurostat/statistics-
explained/index.php?title=Gender_pay_gap_statistics#Gender_pay_gap_levels_vary_significantly_across_EU 

Statistics,  March 

Pay  Gap 

2024 

19 

 
 
 
employment and total hours worked (or increases in temporary layoffs) for young, women, less 
educated and foreign-born workers during the first lockdown period in early 2020.47  

In  the  area  of  education,  challenges  persisted  during  the  period,  even  though  the  headline 
Europe  2020  targets  have  been  achieved.  For  instance,  the  prevalence  of  low  educational 
attainment  among  the  EU  population  remained  high  (with  20.6%  of  persons  aged  25-64  at 
ISCED levels 0-2 in 2022) and adults’ participation in learning activities over the previous four 
weeks remained low (at 11.9% of the EU population aged 25-64 in 2022). Furthermore, while 
the percentage of male early leavers from education and training as well as the tertiary education 
attainment rate of men have moved into the right direction during the evaluation period, they 
still  missed  the  overall  EU-level  targets.  Similarly,  the  percentage  of  young  people  not  in 
employment, education or training has declined to 11.7% in 2022 on average in the EU27 but 
remained high especially in some Central and Eastern as well as Southern European countries. 
On a more general level, skill shortages and gaps persisted48 in large parts of Europe.  

With respect to poverty and social inclusion, some progress was made. However, the share of 
people at risk of poverty or social exclusion in the EU27 still amounted to more than one-fifth 
(21.6%)  of  the  population  in  2022.  In  addition,  the  pandemic  affected  negatively  the  most 
vulnerable with a greater need for direct social support and lower access to Information and 
Communications Technology, for instance via the reduction in the provision of services, the 
increased use of teleworking and distance learning.  

Across these three broad policy areas, positive convergence trends were observed during the 
evaluation period. However, significant differences persisted across and within countries and 
regions.  A  strong  convergence  process  was  registered  in  national  and  regional  employment 
rates, but gaps between the best and worst performers remained high, especially on the regional 
level.49  With  regard  to  social  inclusion  and  poverty  reduction,  most  Central  and  Eastern 
European countries converged to the EU average during the evaluation period. Still, the at-risk 
of poverty or social exclusion (AROPE) rate remained high in Southern European countries. 50 
The  landscape  of  educational  attainment  has  become  somewhat  more  uniform  during  the 
evaluation period, but large gaps between Member States persist.51 In addition, the latest trends 
in educational achievements of 15-year-olds measured by the PISA tests raised concerns. The 
2022 results showed unprecedented drops compared to 2018 in reading and mathematics in all 
EU countries, pointing to the adverse effect of the COVID-19 pandemic, and revealed large 

47 European Institute for Gender Equality (2021), “Gender equality and the socio-economic impact of the COVID-
19 pandemic”; European Institute for Gender Equality, EIGE, Vilnius; Luxembourg: Publications Office of the 
European Union, 2021 
48 European Commission (2024), Employment and Social Developments in Europe (ESDE), report 2023. 
49  European  Commission  (2024),  Ninth  Report  on  Economic,  Social  and  Territorial  Cohesion.  Available  at: 
https://ec.europa.eu/regional_policy/information-sources/cohesion-report_en 
50  European  Commission  (2024),  Ninth  Report  on  Economic,  Social  and  Territorial  Cohesion.  Available  at: 
https://ec.europa.eu/regional_policy/information-sources/cohesion-report_en 
51 In 2022, the proportion of tertiary-educated people among those aged 30-34 was above 45% in 15 Member 
States (compared to only 7 in 2014). Still, it remained below 30% in Romania and Italy and the share of people 
25-64 years old with low educational attainment remained significantly higher than the EU average in Italy, Malta, 
Portugal, and Spain. 

20 

 
 
 
differences  across  countries.52  Furthermore,  the  ongoing  megatrends  often  impact  the  EU’s 
poorer regions more strongly. 53  

3.2. 

FINANCIAL AND OPERATIONAL IMPLEMENTATION OF ESF AND YEI 

3.2.1 FINANCIAL IMPLEMENTATION 

Allocation and crisis response 

The  implementation  of  ESF  took  place  in  27  Member  States  and  the  UK  and  the  YEI  was 
implemented  in  19  Member  States  and  the  UK,  with  an  overall  number  of  188  Operational 
programmes,  comprising  of  both  national  and  regional  programmes,  covering  ESF  and  YEI 
interventions of EUR 131 billion, of which EUR 93.5 billion coming from the EU 

The distribution by thematic objective shows that total funding (ESF, YEI and REACT-EU) 
was  primarily  allocated  to  promoting  sustainable  and  quality  employment  (TO  8),  which 
received 32% of the resources. This was followed by operations to promote social inclusion, 
combat poverty, and eliminate discrimination (TO9) with 22%, and investments in education, 
training, and vocational training for skills and lifelong learning (TO10) with 22.5%. Initiatives 
to  enhance  the  institutional  capacity  of  public  authorities  and  stakeholders,  and  to  ensure 
efficient  public  administration  (TO11)  received  2.9%  of  total  EU  resources.  Technical 
Assistance  accounted  for  around  3.3%.  The  most  recent  interventions  for  “Fostering  crisis 
repair”  (TO13),  introduced  after  the  COVID-19  pandemic,  received  17.3%  of  the  EU  total 
resources.  Allocations  are  also  in  line  with  the  thematic  concentrations  imposed  by  EU 
regulations54.  On  a  Member  State  level,  prioritisation  is  evident  and  aligned  with  country-
specific recommendations55. Between 2012 and 2024, more than 50% of the CSRs related to 
the  ESF  were  directed  at  TO  8,  affecting  all  27  Member  States  plus  the  UK.  These  CSRs 
specifically emphasised the need to strengthen labour market policies, increase participation in 
the labour market for disadvantaged groups (such as women, youth, and adults) and improve 
the  efficiency  of  labour  market  services.  For  instance,  Spain,  Luxembourg  and  Slovakia 
allocated more than 50% of ESF funding to TO 8. Netherlands (71.2%) and Cyprus (56.2%) 
distinguished themselves for the large amount of funding dedicated to social inclusion (TO 9), 
while Portugal leads the ranking regarding the support for education which was made available 
under TO 10, with 50.3% of funds allocated to this TO.56 The share of resources allocated to 
TO 11 was particularly high in Hungary, Bulgaria and Romania. 

52 European Commission: Directorate-General for Education, Youth, Sport and Culture (2024), The twin challenge 
of  equity  and  excellence  in  basic  skills  in  the  EU  –  An  EU  comparative  analysis  of  the  PISA  2022  results, 
Publications Office of the European Union, 2024. Available at: https://data.europa.eu/doi/10.2766/881521 
53  European  Commission  (2024),  Ninth  Report  on  Economic,  Social  and  Territorial  Cohesion.  Available  at: 
https://ec.europa.eu/regional_policy/information-sources/cohesion-report_en 
54 At least 20% of the total ESF resources in each Member State was allocated to the TO 9 "promoting social 
inclusion, combating poverty and any discrimination" (Article 9 of Regulation (EU) No 1303/2013).  In addition, 
MSs had to pursue thematic concentration according to the following modalities: (a) more developed regions were 
required to concentrate at least 80 % of the ESF allocation to each  operational programme on up to five of the 
investment  priorities;  (b)  transition  regions  were  required  to  concentrate  at  least  70  %  on  up  to  five  of  the 
investment priorities and (c) Less developed regions, were required to concentrate at least 60 % on up  to five of 
the investment priorities. 
55 See Annex IV, Appendix 7 of supporting study 
56 See Annex V for Distribution of EU and national planned funds by EU fund, country and category of  region 
(national co-financing included) Top up funding of REACT-EU and YEI by country in 2022 (million EUR, %) 
21 

 
 
 
 
At  the  end  of  2022,  49.5%  of  the  total  planned  ESF  funding  was  channelled  to  the  less 
developed regions, about 36.9% to the more developed regions and the remaining 13.6% to the 
transition regions. In countries receiving Cohesion Fund support57, the intensity of the support 
per inhabitant is more than double the intensity than in countries not receiving Cohesion Fund, 
and this difference increases when purchasing power parity is considered58. However, among 
the former, the differences in both the intensity of the support (cost per inhabitant) and the level 
of assistance (expenditure per participant) are larger than in the latter59.   

Resources  and  flexibility  increased  during  the  programming  period  to  respond  to  emerging 
challenges, including through the Coronavirus Response Investment Initiatives, REACT-EU, 
CARE and SURE60. In particular, with the reallocations of the Cohesion Policy funds and the 
addition  of  REACT-EU,  total  EU  resources  available  for  ESF  support  reached  EUR  113.4 
billion in 2022, an increase of almost  26.5% compared to the initial amount in 2015. When 
including the national co-financing, the resources exceed EUR 150 billion over the period. 

Figure 4: Distribution of different EU financial sources in ESF programmes in 2022 (EU amounts) 

Source: Ecorys, Study supporting the ex-post evaluation of the ESF during the programming period 2014-2020, 
2025. Figure 10. 

The total declared expenditure reached EUR 111.7 billion in 2022. The overall absorption rate, 
i.e. the share of total eligible declared expenditure to Managing Authorities (MAs) against total 
eligible  cost  of  selected  operations,  reached  98.4%  for  the  ESF  and  96.6%  for  the  YEI  on 
1 August 2025.61 In the previous programming period 2007-2013 and by the end of the period, 
a total of €116.76 billion was reported as certified, out of €114.64 billion allocated to the entire 
ESF, which corresponds to 101.8%. Most MS had reached full implementation by the end of 

57 EUR 469 per inhabitant compared to EUR 208 per inhabitant for countries not receiving Cohesion Fund.  
58 EUR 666 and EUR 112 per inhabitant. Values in EUR deflated by the EU27 Price Level Index in 2020 (Eurostat). 
59 See supporting study, analysis on the basis of SFC2014 data up to end of 2022, extracted in September 2023 
60  The  SURE  instrument  acted  as  a  second  line  of  defence,  supporting  short-time  work  schemes  and  similar 
measures,  to  help  Member  States  protect  jobs  and  thus  employees  and  self-employed  against  the  risk  of 
unemployment and loss of income. 
61 Data retrieved on 1 August 2025 from the  Open Data Portal for the European Structural Investment Funds  - 
European Commission | Cohesion Open Data  

22 

 
 
 
 
 
 
the 2015.  Implementation  rates  were  often higher  than 100%  as  some  MS submitted higher 
payment claims than the allocated budgets. 

Looking at the performance of declared expenditure in individual countries, some countries met 
and even exceeded their expenditure targets: Malta achieved the highest spending rate (137%), 
followed by Cyprus and Luxembourg with rates around 100%. 

For  the  same  period,  the  total  value  of  contracted  projects  amounted  at  EUR  165.8  billion 
(combined ESF, YEI and REACT-EU funds), about 9.5% higher than the available funding.  
YEI recorded  even higher selection rates  compared to  ESF (120.5% versus  108.7%)62  Such 
progress is due to the fact that in the last years of the period several Member States preferred 
to  overbook  the  available  resources  to  maximise  the  possibility  of  reaching  the  expenditure 
targets. 

As regards the total number of operations implemented, it exceeded 1.3 million, out of which 
more than 1 million were implemented under ESF and 266 864 under YEI. The average eligible 
cost per operation was EUR 123 740 at EU level. Average costs of YEI operations were about 
three  times  lower  than  those  registered  for  ESF  (EUR  47 184  for  YEI  compared  to  EUR 
142 781 in the case of ESF). 

3.2.2 OPERATIONAL IMPLEMENTATION 

Participations and their characteristics 

With a total of 64.6 million participations, including 2.6 million participations in REACT-EU 
operations63, ESF and YEI funded operations reached a significant part of the general population 
of the EU27 and the UK. In addition, over 2 million micro-enterprises and SMEs (including 
cooperative  enterprises,  enterprises  of  the  social  economy)  have  been  supported  by  ESF. 
Approximately  5.1  million  products  were  delivered  in  projects,  and  476  439  structures 
(networks,  partnerships,  and  other  type  of  entities)  have  been  supported  by  ESF  and  YEI. 
118 723  projects  by  social  partners  or  non-governmental  organisations  were  reported. 
Moreover, Member States reported by end 2022 to have supported 58 282 projects dedicated to 
the sustainable participation and progress of women in employment. Another 81 206 projects 
targeted public administration / public services64. 

By labour market status, the highest share of participations was registered by inactive persons 
(39%  in  ESF  operations  and  18%  in  YEI),  followed  by  unemployed  persons  (34%  in  ESF 
operations and 82% in YEI) and employed persons (for 27% in ESF funded operations65). These 
differences are coherent with the differing missions and scopes of ESF and YEI operations. 

In  line  with  the  objectives  set  out  in  the  Regulation  (see  section  2.1),  the  distribution  of 
participations  also  showed  differences  by  region,  disadvantages  and  age.  From  a  regional 
perspective, 47% of all participations came from less developed regions, with a majority being 
inactive and/or over 54 years old.  More developed regions covered 37% of participants and 
resources in these regions were targeted towards disadvantaged groups. Furthermore, 15% of 
the total participants had a migrant, foreign, or minority background, and 19% of participants 

63 Reported by 22 Member States at the end of 2022.  
64 Information based on the ESF Synthesis Report of AIR 2022, data extracted on July 6, 2023. 
65 Employed persons were not eligible for YEI operations. 
23 

 
 
 
 
 
belonged to ‘other disadvantaged groups’66. The share of women was slightly higher than that 
of men (53% over 47% of total participations). 

Intensity of support  

On  average,  the  total  of  ESF  and  YEI  funding  corresponded  to  EUR  294  per  inhabitant  (in 
current prices). This intensity of support was stronger in countries receiving Cohesion Fund 
support67  than countries  not  receiving Cohesion  Fund  (by a factor of 2, respectively  6 when 
adjusting for purchasing power parity), reflecting the ESF’s mission to contribute to territorial 
cohesion.68  

Based on the total declared expenditure, the average cost per participation69 was EUR 1 732 in 
the EU. Across countries, declared expenditure and the number of participations is correlated, 
however  significant  differences  exist  between  Member  States,  with  the  average  cost  per 
participation ranging from a maximum of EUR 4 546 per participant in Czechia to EUR 780 
per  participant  in  Bulgaria.  These  differences  become  even  larger  when  taking  purchasing 
power  differences  into  account  and  are  also  prevalent  between  countries  receiving  and  not 
receiving Cohesion Fund support. This may hint at different levels of efficiencies but is also 
related to  different models  of intervention, types  of measures  used  and targeting of specific 
groups.70  

Results  

ESF  and  YEI  interventions  reached  a  total  of  64.6  million  participations.  Out  of  these, 
22.3 million reported a positive result immediately after leaving the ESF or YEI intervention:  

•  Over 10 million participants had obtained a qualification; 

•  More  than  7  million  were  in  employment  or  self-employment  and  over  1.5  million 

previously inactive participants started searching for jobs; 

•  Over 3 million were enrolled in further education training; 

•  Out  of  3.9  million  participants  in  YEI-funded  projects,  1.1  million  unemployed  -  of 
which  around  0.3  million  long-term  unemployed  and  almost  0.2  million  NEET  - 
received an offer of employment, further education, apprenticeship or traineeship upon 
leaving the project. 

Overall, this means that 35 % of recorded participations resulted in at least one of these positive 
short-term results. While this figure may seem low at first glance, it has to be highlighted that 

66 Figures  representing  participants  with  characteristics  in  disadvantaged groups  may be  underrepresented  in  a 
significative way because: characteristics ‘rural background’ and ‘homelessness’ were only reported as minimum 
requirement in 2017 and as belonging to a minority is often based on self-identification and underreported when 
the minority is facing discrimination (especially in the case of Roma minority).  
67 Analysis refers to countries receiving Cohesion Fund support during the programming period and those that did 
not.  Countries  receiving  Cohesion  Fund  support  in  the 2014-2020 period  were:  Bulgaria,  Croatia,  Cyprus,  the 
Czech Republic, Estonia, Greece, Hungary, Latvia, Lithuania, Malta, Poland, Portugal, Romania, Slovakia and 
Slovenia.  Countries  not  receiving  Cohesion  Fund  in  the  2014-2020  period  were:  Austria,  Belgium,  Denmark, 
Finland, France, Germany, Luxembourg, Ireland, Italy, Netherlands, Spain, Sweden, United Kingdom. 
68  See  supporting  study,  Ecorys,  Study  supporting  the  ex-post  evaluation  of  the  ESF  during  the  programming 
period 2014-2020, 2025. 
69 Calculated as the total declared expenditure, including indirectly targeted expenditure on technical assistance 
and on reinforcing services and institutions, divided by the number of total participants at the end of 2022.  
70  See  supporting  study,  Ecorys,  Study  supporting  the  ex-post  evaluation  of  the  ESF  during  the  programming 
period 2014-2020, 2025. 

24 

 
 
 
the monitoring system is based on “hard” indicators that capture final desired outcomes. It does 
not capture however intermediate steps or more “soft” outcomes that are targeted by operations 
e.g. in the area of social inclusion. In addition, while ESF typically target individuals with low 
employability, there is empirical evidence demonstrating the positive impact on, for example, 
employment probability of participants, as described in Chapter 4.  

Longer-term result indicators show that 6.9 million participants that had received ESF or YEI 
support were employed six months after leaving, out of which 2.6 million were participants 
registered as part of a disadvantaged group and 0.4 million were above 54 years old. In addition, 
1.4 million participants recorded an improved labour market situation71 six months after leaving. 
These results and the corresponding success rate of around 10% positive longer-term results 
per participation are likely to be underestimated as not all countries have reported yet on the 
relevant indicators72. 

The composition of achieved results varied by thematic objective, depending on the dominant 
activities and target groups. In TO 8, activities yielded the largest positive results in bringing 
persons  into  employment,  with  around  5  million  participants  being  in  employment  or  self-
employment upon leaving and 4.1 million participants being employed six months after. By 
contrast,  the  dominant  result  in  TO  10  was  obtaining  a  qualification,  which  applied  to  5.8 
million participations.  

Across Member States, there is a significant variation in the relationship between outputs and 
results73, which can reflect different target groups, the use of different types of activities as well 
as reporting practices. 

4.  EVALUATION FINDINGS  

The evaluation has assessed the effectiveness, efficiency, coherence, relevance, EU added value 
and  visibility  of  the  European  Social  Fund  and  Youth  Employment  Initiative  during  their 
implementation period from 2014 to 2022. It thus aims (where possible) to draw conclusions 
about the causal effects of the EU intervention on the desired outcomes.  

4.1.  TO WHAT EXTENT WAS THE INTERVENTION SUCCESSFUL AND WHY? 

4.1.1 EFFECTIVENESS 

71 Persons who are employed when entering ESF support and who, following the support, transited from precarious 
to stable employment, and/or from underemployment to full employment, and/or have moved to a job requiring 
higher competences/skills/qualifications, entailing more responsibilities, and/or received a promotion 6 months 
after leaving the ESF operation. 
72 Some countries and some programmes reported longer-term result indicators annually, while others will only 
do so in the Final Implementation Report (FIR) in 2026. Consequently, these results may increase when all the 
FIRs are available, especially in the countries with very low or zero results reported at the end of 2022. 
73  See  supporting  study,  Ecorys,  Study  supporting  the  ex-post  evaluation  of  the  ESF  during  the  programming 
period 2014-2020, 2025. 

25 

 
 
 
 
 
 
 
4.1.1.1 ACHIEVEMENT OF OPERATIONAL OBJECTIVES74 

Achievement of targets 

On the EU level, aggregate targets were set for the ESF in the context of the annual EU budget 
performance and reporting exercise75, notably on the shares of participants in employment/self-
employment, gaining a qualification as well as on inactive participants looking for a job. The 
ESF reached two out of these three targets by the end of 2022, with 32% of unemployed and 
inactive  participants  being  in  employment  or  self-employment  upon  leaving  compared  to  a 
target of 24%, and 25% of all participations gaining a qualification compared to a target of 23%. 
It fell however short the target of 16% of inactive participants being engaged in job searching 
upon leaving, which was reached in 2018, but the ratio decreased afterwards to 13% at the end 
of 2022, also related to a shift of the programme focus during the pandemic towards maintaining 
jobs and providing services on the other target groups (including the inactive), rather than on 
pure activation measures.76 YEI interventions reached all EU level targets.  

Target group reach 

The  regulation  lays  out  that  the  ESF  and  YEI  shall  benefit  people,  including  disadvantaged 
people such as the long-term unemployed, people with disabilities, migrants, ethnic minorities, 
marginalised communities and people of all ages facing poverty and social exclusion and shall 
contribute to territorial cohesion. 

Among all participations77, almost half of the persons reached had a low educational attainment 
level and around 20% of participations were from persons at risk of poverty or social exclusion 
(see Table 2). Also, 15% of participations were made up by migrants and persons with foreign 
background  or  from  marginalised  communities  and  7%  came  from  people  with  disabilities. 
When  compared  to  data  from  a  similar  stage  of  the  2007-2013  programming  period,  the 
proportion  of  migrants  and  those  from  minority  or  marginalised  communities  and  that  of 
persons with  disabilities among  total  participations  has increased (compared to  9% and 5%, 
respectively for 2007-2013)78. This is first evidence that targeting the more disadvantaged parts 
of the society has effectively been achieved.  

Comparing  these  shares  to  the  proportion  of  the  respective  target  groups  in  the  overall  EU 
population reveals that migrants, Roma and those with the lowest education levels were over-
represented in the ESF data relative to their population in the EU, further indicating that there 
is a good reach of these target groups. At the same time, participations of those facing social 
exclusion  closely  mirrored  prevalence  within  the  EU  population  speaking  for  an  adequate 
targeting. However, those with disabilities appeared under-represented in terms of the degree 

74 See answers to evaluation questions 1, 2 and 4 of the supporting study. 
75See  EU  core  performance  indicators  available  at:  https://commission.europa.eu/strategy-and-policy/eu-
budget/performance-and-reporting/programme-performance-statements/european-social-fund-
performance_en#mff-2014-2020--european-social-fund  
76See archive of the 2024 exercise here: 
https://wayback.archive-it.org/12090/20250228102931/https://commission.europa.eu/strategy-and-policy/eu-
budget/performance-and-reporting/programme-performance-statements/european-social-fund-
performance_en#mff-2014-2020--european-social-fund 
77 The monitoring system is set up on the basis of participation records. This means that one individual can take 
part in multiple ESF funded operations, which will then be counted as several participations. An identification of 
the exact number of individuals benefiting from ESF+ support is therefore not possible on an aggregate level.  
78  European  Commission,  Directorate-General  for  Employment,  Social  Affairs  and  Inclusion, ESF  ex-post 
evaluation 
report  –  Final  version,  Publications  Office, 
synthesis  2007-2013  –  EU 
2016, https://data.europa.eu/doi/10.2767/1734 

synthesis 

26 

 
 
 
to which they are reached through ESF support.79 Evidence from the YEI case study also hinted 
at a lower effectiveness in reaching the most marginalised young persons among the broader 
NEET population80. 

Table 2:  Participation of ESF target groups as a proportion of overall ESF participations 

Target group 

Persons at risk of poverty or social exclusion 
Migrants, people with foreign backgrounds or 
from marginalised communities such as Roma 
Persons with ISCED level 2 education or lower 

Persons with disabilities 

Proportion of overall participations –  
2014-2020 programming period 
19% 

Proportion of overall participations – 
2007-2013 programming period 
n.a. 

15% 

47% 

7% 

9% 

45% 

5% 

Source: Supporting study, Section 4.1.1.4, ESF ex-post evaluation synthesis 2007-2013 

In addition, the placement of the ESF and YEI intervention in the context of the Europe 2020 
goals  suggested  a  strong  targeting  of  unemployed  and  inactive  persons  compared  to  the 
preceding  programming  period.  Overall,  economically  inactive,  including  NEETs,  made  up 
39% of total participations; the unemployed, including long-term unemployed, 34%; and those 
in employment 27%. While this distribution was still broadly balanced, it suggests a stronger 
focus  on  the  non-employed.  This  is  also  confirmed  by  the  comparison  to  the  preceding 
programming period, where the shares of inactive and unemployed people were lower (with 
36% and 30%) and the employed made up a larger fraction of 33% of total participations.  

These findings are further nuanced by other data collected in the supporting study. Overall, the 
MA survey, interviews and case studies indicate that stakeholders perceived that ESF and YEI 
actions have reached the intended target groups. However, specific operations within certain 
IPs had challenges in reaching the most disadvantaged groups, for example the most vulnerable 
or  marginalised  within  the  larger  NEET  group,  and  the  MA  Survey  confirmed  greater 
challenges  in  reaching  older  persons,  persons  with  disabilities,  Roma  and  people  that  are 
homeless or affected by housing exclusion. Still, interviewees across stakeholder groups and 
respondents of the MA survey felt that the minimum allocation of 20% to the thematic objective 
on  social  inclusion  had  supported  targeting  of  harder  to  reach  populations.  Civil  Society 
Organisation (CSO) representatives at EU and national levels were more sceptical on the extent 
that this would translate into effective engagement and results. 

Contribution to CPR thematic objectives 

The Common Provisions lay out thematic objectives that are to be supported by the respective 
funds81 (including European Regional Development Fund - ERDF82), which can be summarised 
for  the  ESF  and  YEI  to  four  broad  policy  areas:  Employment,  Social  inclusion,  Education, 
training and vocational training and Capacity building. 

In  the  broad  policy  area  “Employment”,  the  ESF  made  a  strong  contribution  to  supporting 
people into employment and increasing social inclusion. Specifically, in TO 8 alone, around 5 

79 In 2022, persons with disabilities were estimated to represent 27% of the EU population. See Supporting study, 
Section 4.1.1.4 
80 See Supporting study, YEI – 8.ii case study.  
81 Thematic objectives set out in points (8), (9), (10) and (11) of the first paragraph of Article 9 of the CPR. 
82 The ERDF 2014-2020 has also contributed to those policy areas, through support to infrastructure, equipment 
and cooperation across borders investments under thematic objectives 8, 9, and 10. Effects thereof are not analysed 
in the text. 

27 

 
 
 
 
million participants were supported into employment and around 1.9 million YEI participants 
were either in employment, education or gained a qualification upon leaving by end of 2022. 
Support was also provided to improve employability of disadvantaged persons in TO 9 and 10, 
with  millions  moving  closer  to  the  labour  market  by  seeking  employment  or  taking  part  in 
further education or training upon leaving. Thus, not only was the scale of operations large, 
with ESF support reaching 13.3% of the average EU27 + UK unemployed and 3.5% of inactive 
persons annually, but it also resulted in an improved labour market situation for a large number 
of participants. Furthermore, alongside other instruments like SURE, the ESF also contributed 
to maintain jobs and support adaptation during the COVID-19 pandemic through CRII/CRII+.83 
The positive contribution was further confirmed by the primary data gathered for the supporting 
study84, as well as being broadly reflected in applicable national evaluations85.  

Interventions in the field of “social inclusion” reached almost 15 million persons under TO 9, 
with a significant reach among disadvantaged groups and marginalised communities. This is 
also  reflected  to  the  achieved  results,  with  around  2.8  million  disadvantaged  participants 
seeking or entering employment, education or training upon leaving support86 and around 1.1 
million participants who gained a qualification upon leaving support87. In addition, around 115 
000  SMEs  and  social  enterprises  as  well  as  23  000  NGOs  and  social  partner  projects  were 
supported in this TO. A positive contribution was also broadly echoed in the TO9 case studies 
and  primary  data  collected  via  surveys  and  interviews  as  well  as  in  national  evaluations. 
However, the results were slightly more nuanced than in the employment domain, with slightly 
less Managing Authorities declaring that operations had met their objectives to a large extent 
(59%, 67 out of 114, vs 65%, 74 out of 114 for TO8) and some interviewees from CSOs feeling 
that operations still struggle in some contexts to reach the most marginalised groups, such as 
Roma or persons with disabilities.  

Operations in the area of “education, training and vocational training” reached more than 20 
million participants in TO10, with a broad reach to disadvantaged or marginalised groups. Of 
these, 5.8 million gained a qualification and around 1.4 million disadvantaged participants were 
seeking or entering employment, education, or training upon leaving. The overall results in this 
policy  area  were  however  higher,  as  1)  other  TOs  also  contribute  to  education  and  training 
outcomes measured by the monitoring system and 2) not all operations under TO10 have the 
objective of gaining a qualification, for instance activities that use informal learning pathways 
or that aim at reducing school dropout. Further evidence collected in the supporting study, also 
speaks  for  a  strong  contribution  by  the  ESF/YEI  in  this  policy  area.  In  particular,  a  high 
proportion  of  surveyed  Managing  Authorities  felt  that  operations  in  TO10  had  met  their 
objectives to a large extent (63%, 72 out of 114).  

83  See supporting study, Annex VIII. Available at: Annexes.   
84 Including data from the surveys, stakeholder interviews and case studies undertaken for the study. 
85 Relevant evaluation evidence is considered in more detail in assessing the changes brought by ESF support at 
Section 4.1.1.5. 
86 Data from SFC relating to CR05 results under TO 9 – i.e. disadvantaged participants engaged in job searching, 
education/ training, gaining a qualification, or in employment, including self-employment, upon leaving. Further 
detail is provided in Section 3.3.8. 
87 Data from SFC relating to CR05 results under TO 9 – i.e. participants gaining a qualification upon leaving. 

28 

 
 
 
Interventions relating to institutional capacity building under TO 11 were allocated a relatively 
small proportion of overall ESF funding (4%) and limited to a subset of countries88. By the end 
of  2022,  under  TO11  more  than  3  500  projects  targeting  public  administrations  and  labour 
market  institutions  had  been  supported  by  the  ESF.  However,  the  case  studies  results  were 
inconclusive  when  it  comes  to  achievements  of  intended  objectives  of  selected  operations. 
Overall, there is less evidence including from primary data collection in the supporting study89, 
so that an overall judgement on the effectiveness is not warranted. 

Overall,  the  evidence  presented  in  this  subsection  suggests  that  ESF  and  YEI  interventions 
effectively reached a large part of the EU population and achieved many positive results among 
its  participants.  Focussing  on  participants  entering  employment  and  gaining  a  qualification, 
which can be directly compared to the preceding programming period, the ESF has been more 
effective, reaching higher shares of these common results.  It also contributed to achieve the 
thematic  objectives  laid  out  in  the Common Provisions.  In  addition,  reach-out  was  good  on 
most of the targeted groups, although for some subgroups (e.g. most disadvantaged among the 
NEET population) effectiveness in targeting was lower (e.g. due to their disengagement from 
public services and society more generally or limitations in terms of resources, capacity and/or 
expertise  of  delivery  organisation).  Regarding  thematic  objectives,  the  ESF  made  a  strong 
contribution in the employment and education and training policy areas and also contributed 
positively to the social inclusion and poverty alleviation targets of the Europe 2020 strategy as 
well as to the third chapter of the European Pillar of Social Rights.  

4.1.1.2 CHANGES RESULTING FROM ESF AND YEI SUPPORT90 

On a micro level 

This  section  analyses  the  changes  brought  to  target  groups  and  which  types  of  operations 
appeared  more  or  less  effective,  based  on  evaluation  evidence91,  case  study  findings  and 
triangulation from other sources.  

In  the  broad  policy  area  “Employment”,  ESF  and  YEI  funding  increased  opportunities  for 
participants to find - or maintain - a job, or to improve their working conditions. This is shown 
by  the  JRC  update  to  the  meta-analysis  of  ESF  counterfactual  impact  evaluations9293  which 
estimates  a  balanced  average  impact  on  employment  of  5.9  pps.  for  the  2014-2020 
programming period, meaning that the probability of being in employment of ESF participants 

88 Member States eligible for cohesion funding or with at least one less developed region. 
89 For instance, almost two-thirds of respondents to the  Managing Authority survey reporting that they did not 
know if operations had met their objectives. 
90 See answers to evaluation questions 3, 5, 7 and 9 of the supporting study. Available at: Study supporting the ex-
post evaluation of the 2014-2020 European social fund and youth employment initiative - Publications Office of 
the EU. 
91  79  national  impact  evaluations  were  analysed  as  well  as 12  implementation-focused evaluations,  for further 
details see supporting study, Annex II and Annex IV. Available at: Annexes. 
92  These  methods  compare  a  ‘treatment’  group  of  individuals  who  participate  in  the  intervention  with  a 
‘comparison’ group of individuals not affected and use the difference in outcomes as an estimate for the treatment 
effect. 
93  Haepp,  T.  and  Serrano  Alarcon,  M.,  Meta-analysis  of  the  European  Social  Fund  counterfactual  impact 
evaluations: Brief update with alternative measures, European Commission, Ispra, 2024, JRC137571, available at: 
https://publications.jrc.ec.europa.eu/repository/handle/JRC137571,  On  the  basis  of  ESF  counterfactual  impact 
evaluations covering the 2014–2020 programming period published until November 2023. 

29 

 
 
 
 
is  substantially  higher  than  that  of  comparable  non-participants.  Importantly,  these  numbers 
should be read having in mind that they mostly refer to individuals who are difficult to employ. 
It is estimated that on average the employment rates for participants increased from 40.2% to 
46.1%  corresponding  to  a  15%  increase94.  The  highest  impacts  were  found  for  employment 
subsidy and financial aid programmes with 15.1 pps, followed by internships/traineeships (6.7 
pps.), vocational training (5.2 pps.), other training (4.9 pps.) and 2.2 pps. for mentoring/social 
support. The lowest (and, on average, negative) effect was estimated for public employment 
programmes (-10.2 pps). These results are consistent with the findings of most studies in the 
academic literature95. The most likely reasons for the low probability of success of public work 
programmes are the lock-in effect (participants suspending their job search efforts and devoting 
time  to  programme  activities)  and  stigmatisation  (employers  associating  participation  in  the 
programme  with  low  productivity).  Public  employment  programmes  also  tend  to  pursue 
broader social inclusion objectives, which are not captured by evaluations focusing solely on 
employment outcomes. At the same time, while the majority of public work programmes show 
low or negative impacts on employment probabilities, some programmes have demonstrated 
positive  effects,  suggesting  that  favourable  employment  outcomes  are  possible  when 
programmes are well designed and well targeted.  

The overall positive effect of active labour market policies on employment is expected, in turn, 
to have beneficial economic effects at the level of individuals and for government revenues at 
the Member State level. Case study evidence also highlighted the importance of softer outcomes 
for participants, particularly for those further from the labour market, for instance in terms of 
enhanced  confidence,  skills  such  as  personal  interaction,  and  interview  preparation  and 
techniques. Particularly  effective in  increasing  the probability of finding a job  or improving 
participants’ job positions were operations on education and vocational training, work-based 
learning  and  financial 
traineeships,  childcare  services  and 
entrepreneurship courses. Likewise, while limited in number, national level impact evaluations 
of ESF education and training support typically identify a positive net effect96 at the level of 
individuals.  

incentives  as  well  as 

For  interventions  in  the  policy  area  “social  inclusion”,  national  evaluations  showed  that 
operations  were  effective  in  reaching  target  groups,  and  in  increasing  employment  rates, 

94  According to the coding of the 11 CIE reports published since February 2022, the reference employment rate 
in those studies was 40.2 percent. While this number does not correspond to all the CIEs of the 2014-2020 period, 
it provides an estimate  of the possible reference employment rate  (employment rate without ESF support) and 
facilitates the general interpretation that the ESF interventions have increased the employment rate of programme 
participants from 40.2 to 46.1 percent.  
Source  on  reference  employment  rate  (footnote  9):    Haepp,  T.  and  Serrano  Alarcon, M.,  Meta-analysis  of  the 
European  Social  Fund  counterfactual  impact  evaluations:  Brief  update  with  alternative  measures,  European 
Commission, Ispra, 2024, JRC137571, available at:  
https://publications.jrc.ec.europa.eu/repository/handle/JRC137571  
95 See for example, David Card, Jochen Kluve, Andrea Weber, What Works? A Meta Analysis of Recent Active 
Labor Market Program Evaluations, Journal of the European Economic Association, Volume 16, Issue 3, June 
2018, Pages 894–931, https://doi.org/10.1093/jeea/jvx028 
96 See supporting study, Annex IV, Appendix 5 for more detailed findings of the meta-analysis of national impact 
evaluations. Available at: Annexes. 

30 

 
 
 
including for the most disadvantaged97. ESF support also helped in terms of access to services, 
in increasing self-confidence and participants’ quality of life in general98. Case study evidence 
highlighted in particular the positive effects on moving disadvantaged and often inactive groups 
closer to and into employment as well as the positive soft outcomes achieved for target groups, 
particularly improvements to mental health, confidence, motivation, self-esteem, and improved 
integration in local communities. Effectiveness varied across operations, with the development 
of  “one-stop-shops”  for  access  to  multiple  services  and  operations  providing  individually 
tailored support and those adopting a holistic approach standing out positively.  

Based  on  national  evaluations,  case  study  evidence  and  a  JRC  education  meta-analysis99, 
operations in  the area of  “education, training and vocational  training” improved educational 
environments  for  both  students  and  teachers  and  resulted  in  improved  competencies  and 
qualifications. The JRC Meta-analysis100 used 26 studies (of which 20 relate to the 2014-2020 
programming  period)  and  showed  that  average  impacts  on  various  education  outcomes  are 
positive at around 3.2 pps. (for studies measuring the outcomes in percentage points) and 7.1% 
(for studies measuring the outcomes in percentage changes). Evaluation reports find positive 
and  significant  impacts  in  44%  of  the  cases,  no  significant  results  in  43%  of  cases  and 
significant,  negative  impacts  in  13%.  The  share  of  positive  and  significant  outcomes  is  the 
highest for interventions targeting students (68%) and for outcomes such as grades (70%) and 
progression to the next school year (68%). Case studies point to key changes in terms of the 
achievement of qualifications, increased participation in tertiary education and post-secondary 
education as well as reductions in early school leaving101. Furthermore, stakeholders perceived 
the  focus  on  people  from  underrepresented  socio-economic  backgrounds  positively  and  the 
collected evidence indicates effective targeting and positive results for these groups. However, 
case study evidence on significant changes at the level of institutions or education and training 
systems was more limited, with the IP 10.ii case study  (improving the quality and access to 
tertiary and equivalent education and training) identifying positive effects at the level of, for 
example, supporting the introduction of new or enhanced higher education courses and some 
examples of a contribution of more structural changes for example in Latvia102, Germany and 
Spain103.  

97 For example, under IP8.i (Access to employment) the Netherlands supported job coaches for workers with a 
disability starting in a new job, achieving a 26% successful exit rate. In Cyprus, operations under IP9.iii supported 
continued  development  of  a  Disability  and  Functionality  Assessment  System  including  Disability  Assessment 
Centres, which allowed an extension of support measures and services to people with autism.  
98 For example, in Latvia operators in healthcare and social services were targeted to improve the social inclusion 
of people with disabilities. 
99 HAEPP, T., Meta-analysis of the European Social Fund counterfactual impact evaluations: First quantitative 
results  on  education  outcomes,  European  Commission, 
JRC143146.  Available  at: 
https://publications.jrc.ec.europa.eu/repository/handle/JRC143146 
100 Not all of the education studies used for the meta-analysis refer to the 2014-2020 programming period. Out of 
26 studies corresponding to 271 estimates, 20 refer to the 2014-2020 programming period. 
101 See supporting study, Annex VIII IP10.i case study. Available at: Annexes. 
102 See supporting study, Annex VIII IP 10.ii case study. Available at: Annexes. 
103 See supporting study, Annex VIII IP 10.iv case study. Available at: Annexes. 

Ispra,  2025, 

31 

 
 
 
 
 
On a macro level 

Given the limited size of the ESF and YEI funding (including REACT-EU, it corresponds to 
around 1.3% of 2014 nominal  GDP in the EU to be spent  across all implementation years), 
large macroeconomic impacts cannot be expected. In addition, the presence of external factors 
(such as the impact of national policies and budgets and the general economic environment) 
makes the isolation of the economic impact of ESF and YEI operations difficult. Econometric 
and/ or modelling approaches can, nonetheless, be used to derive an estimate on the basis of 
available data and assumptions. Such estimates might still be subject to influence from external 
factors if the used model’s underlying assumptions are not correct.  

The RHOMOLO model operated by the EC’s Joint Research Centre was used to simulate the 
macroeconomic impact  of the 2014-2020 ESF  (including REACT-EU) and YEI programme 
compared to a “no-policy baseline”.104 In terms of employment, a positive impact is estimated 
on  the  number  of  individuals  being  in  work  relative  to  the  baseline  in  which  no  policy  is 
implemented. Generally positive impacts were also estimated with respect to reducing skills 
mismatches. In addition, the model estimates that in 2022, the EU wide GDP impact of ESF 
investments (including YEI and REACT-EU) stands at 0.086%, meaning that 9 years after the 
start of the programme, EU GDP is estimated to be 12.2 billion euro higher than the baseline 
GDP. The return on investment in terms of GDP shows that 1 euro spent is estimated to yield a 
total of 1.256 euro in 2030105 (GDP multiplier based on cumulative change) for ESF, YEI and 
REACT-EU combined106.  Compared to the CBA results described in section 4.1.2.1, the macro-
economic  modelling  applied  led  to  lower  impact  results  due  to  different  reasons.  While 
macroeconomic models as Rhomolo simulate system responses and offer a comprehensive view 
of long-term, interactions and indirect effects, they can lead to more conservative estimates. 
The CBA focused on discounting net returns of projects and can sometimes overstate benefits 
due to methodological limitations and underlying assumptions. 

Among  the  thematic  objectives,  TO  11  (institutional  capacity  building)  stands  out,  with  the 
highest return on investment in terms of GDP (a total of 4.4 euros in 2030 for every 1 euro spent 
in  this  TO).  These  larger  economic  returns  are  due  to  public  infrastructure  investments 

104 Casas, Pablo and Christou, Tryfonas and García Rodríguez, Abián and Lazarou, Nicholas Joseph and Salotti, 
Simone (2025): The  ex-post  macroeconomic  evaluation  of  the  2014-2020  European  Social  Fund,  Youth 
Employment Initiative and REACT-EU labour market interventions. 
105  Certain benefits and costs could not be quantified in monetary terms in the CBA, such as employer benefits 
(with the exception of employee training), improvement in the quality of life, reduction of dependence on certain 
services, integration into communities, macroeconomic effects, benefits related to good governance and capacity 
building as well as indirect costs. As a result, these aspects were only considered qualitatively, and their exclusion 
affected the monetary return calculations. Moreover, the reported cumulative costs were used as observed, hence 
at current prices and, therefore, were not corrected for differences in purchasing power. Macroeconomic knock-
on effects are further benefits that were ignored in the CBA but are covered in the RHOMOLO modelling produced 
by the EC’s Joint Research Centre. 
106 Looking only at YEI, the multiplier is 0.94 euro, and for REACT-EU only, it is 1.7 euro in 2030. For further 
details on results by TO or  by country, please see  Casas, Pablo and Christou, Tryfonas and García Rodríguez, 
Abián and Lazarou, Nicholas Joseph and Salotti, Simone (2025): The ex-post macroeconomic evaluation of the 
2014-2020 European Social Fund, Youth Employment Initiative and REACT-EU labour market interventions. 

32 

 
 
 
 
increasing the public capital stock, subsequently raising productivity for all firms which then 
leads to a stimulation of labour demand.  

A further assessment of the economic benefits of the ESF is presented when discussing the cost-
benefit analysis in the section on efficiency.  

On a structural level 

Given  the  relatively  small  size  of  the  ESF/YEI  budget  compared  to  national  ones,  the  ESF 
contributions to changes on a more structural level were comparatively more limited. Still, there 
was  some  evidence  of  positive  influence  of  the  ESF  on  policies  and  some  more  specific 
examples of changes at a more systemic level. This holds in particular for the implementation 
of the Youth Guarantee, with the YEI having a significant contribution to the delivery of the 
Youth  Guarantee107  and  the  ESF  and  YEI  being  a  source  of  its  funding  in  the  majority  of 
Member States. In line with previous findings108, stakeholders showed broad recognition of the 
YEI  as  crucial  intervention,  leading  to  a  more  coherent  policy  and  funding  approach  to  the 
integration of NEETs into the labour market as well as generating considerable EU value added 
by supporting interventions that were not funded by national policies 109. In addition, support to 
structural changes especially around digitalised services was identified under TO11 operations. 
However,  a  direct  association  to  ESF  support  is  not  possible,  inter  alia  due  to  the  context-
specific nature of structural reforms and their longer implementation horizon110 111 .  

A positive influence of the ESF is also confirmed by a study on achieving structural changes in 
ESF112.  Using  a  broad  definition  of  structural  reforms,  the  ESF  contributed  with  funding  to 
almost half of the analysed reforms between 2014 and 2022, and stakeholders acknowledged 
the  importance  of  the  ESF’s  contribution  to  designing  and  implementing  these  reforms. 
Additionally,  the  study  shows  a  significant  contribution  of  ESF  towards  capacity  building 
efforts,  highlighting  the  support  of  operations  under  TO11  as  one  of  the  key  facilitators  of 
structural change.  

107 European Commission, Directorate-General for Employment, Social Affairs and Inclusion, Jeffrey, P., Naylon, 
I.,  Parissaki,  M.  et  al., Study  for  the  evaluation  of  ESF  support  to  youth  employment,  Publications  Office, 
2020, https://data.europa.eu/doi/10.2767/681170 
108 European Commission, Directorate-General for Employment, Social Affairs and Inclusion, Jeffrey, P., Naylon, 
I.,  Parissaki,  M.  et  al., Study  for  the  evaluation  of  ESF  support  to  youth  employment,  Publications  Office, 
2020, https://data.europa.eu/doi/10.2767/681170 
109 See supporting study, Annex VIII. Available at: Annexes. 
110 See also European Commission, Directorate-General for Employment, Social Affairs and Inclusion, Thijs, N., 
Nakrošis,  V., Progress  Assessment  of  ESF  Support  to  Public  Administration  (PAPA)  –  Summary  report, 
Publications Office, 2020, available at: https://data.europa.eu/doi/10.2767/03954 
111 See also supporting study, Annex VIII. For instance, in IP level case studies, consulted stakeholders commonly 
noted that other cohesion/structural funds, alongside national policy reforms and strategies, were equally or more 
significant drivers of change in most cases  
112 European Commission, Directorate-General for Employment, Social Affairs and Inclusion, Ž. Martinaitis, A. 
Morozovaitė, M. Saletti, E. Colaiacomo, C. Pagnini, and G Dente, Study supporting the Impact assessment of the 
future European Social Fund proposal - Achieving structural change: conditions for success in the areas of ESF 
intervention.  Final  report,  Publications  Office,  2025,  Study  supporting  the  impact  assessment  of  the  future 
European Social Fund proposal - Publications Office of the EU. 

33 

 
 
 
 
Country-specific  recommendations  (CSR)113  issued  to  Member  States  on  how  to  address 
structural challenges were also  taken into account in the 2014 to 2020 programming period. 
Between  2012  and  2014,  CSRs  on  labour  market  policies,  quality  employment,  and  labour 
mobility were the most frequent with 204 recommendations issued, followed by 116 in the area 
of education and training.114 The supporting study found a broad correlation between Member 
States’ socio-economic contexts (e.g. the prevalence of relatively low employment rates), CSRs 
issued to them, and ESF allocations at TO level. For instance, the CSRs addressed to Portugal 
placed a focus on improving the quality and relevance of the education system to the labour 
market and more than half of total resources were allocated to TO 10. 88% of relevant CSRs 
issued to Member States were linked to ESF or YEI funds and all stakeholder groups expressed 
positive views on the contribution of the ESF and YEI to implementing CSRs. However, a clear 
conclusion on the effectiveness of the ESF and YEI contribution cannot be made due to two 
factors. First, while case studies data provide multiple examples of how operations responded 
to CSRs, stakeholders were unable to attribute this to the active and systematic consideration 
of the CSRs as opposed to a natural overlap between the focus of CSRs and the ESF. Second, 
stakeholders assessed the potential of ESF funding to  fully  address  the recommendations as 
limited compared to other national reforms or interventions due to its scope and scale.   

The collected evidence demonstrates contributions to increased employment opportunities, the 
promotion  of  social  inclusion,  upskilling  of  workers  and  enhanced  education  provision. 
Effectiveness shows some variation at the level of operations. In addition, some evidence points 
to contributions at the macro and structural level, even though these are rather limited inter alia 
due to the small size of the ESF and YEI budget compared to national ones. 

4.1.1.3 THE ROLE OF ESF AND YEI IN CRISIS RESPONSE115 

Both  the  COVID-19  pandemic  and  Russia’s  war  of  aggression  against  Ukraine  represented 
serious  crises  that  the  ESF  sought  to  address  during  the  2014-2022  evaluation  period.  The 
CRII(+), REACT-EU (adopted in December 2020)  and CARE (adopted in 2022)  initiatives 
aimed at mitigating the economic and social impacts of the crises, while, in the case of REACT-
EU,  also  supporting  the  EU’s  green  and  digital  transition  objectives.  For  that  purpose,  they 
offered,  on  the  one  hand,  flexibilities  to  swiftly  respond  to  the  emerging  crisis  needs  via 
reorientated  funds,  and  on  the  other,  additional  funds  via  REACT-EU.  The  REACT-EU 
initiative continued and expanded the existing crisis response. Moreover, it supported longer-
term recovery efforts and prepared structural actions for twin transition. 

113In the context of the European Semester, the Commission provides guidance to Member States on an annual 
basis  on  how  to  respond  to  challenges  and  to  deliver  on  policy  objectives  in  the  so-called  Country  Specific 
Recommendations; https://commission.europa.eu/business-economy-euro/european-semester_en  
114 See supporting study, Annex IV appendix 7. Available at: Annexes. 
115  See  answer  to  evaluation  question  Q10  of  the  supporting study.  Available  at:  Study  supporting  the  ex-post 
evaluation of the 2014-2020 European social fund and youth employment initiative - Publications Office of the 
EU 

34 

 
 
 
 
The  preliminary  CRII  evaluation116  highlighted  that  the  rapid  adjustments  facilitated  within 
ESF programming were pivotal to support the broader pandemic response’s effectiveness. 23 
countries reallocated ESF funding between TOs, with CRII flexibilities enabling almost EUR 
2 billion of funding to be re-allocated to social inclusion responses (TO 9) and EUR 215 million 
to  employment-related  operations117.  Evidence  from  the  CRII  case  study  confirmed  that  the 
rapid response, procedural simplification, and additional flexibilities effectively facilitated the 
adjustment of resource allocation to national and regional crisis response needs. Additionally, 
both the preliminary CRII evaluation and case study evidence indicates that the crisis response 
measures enabled by the CRII/CRII+ were more easily and rapidly adapted compared to the 
tools deployed in response to the financial crisis of 2007-2008.  

Resource and time constraints were challenges despite the flexibilities offered. Nevertheless, 
they did not affect the overall stakeholder judgement of the effective and valuable response. 
Also, no strong evidence emerged relating to concerns that reduced allocations to TO 10 could 
result in possible long-term effects and a diversion from longer-term cohesion policy objectives 
118 119.   

During  the  preparation  of  the  supporting  study120,  information  on  the  implementation  of 
REACT-EU  and  CARE  including  from  result  indicators  and  dedicated  evaluations  was  still 
limited. Furthermore, interventions were still being implemented during evidence gathering for 
the  ex-post  evaluation,  implying  also  the  lack  of  final  implementation  reports.  However, 
preliminary evidence from the REACT-EU and CARE case studies121 demonstrates a broadly 
positive  picture.  For  REACT-EU,  this  holds  in  particular  for  the  prevention  of  job  losses 
through funding short-time work schemes (alongside the Support to mitigate Unemployment 
Risks in an Emergency (SURE) instrument) and for CARE, stakeholders noted that operations 
are proving broadly effective in labour market integration of persons fleeing the war in Ukraine 
and learning the receiving country’s language. 

Beyond the supporting study, the Commission’s dedicated REACT-EU evaluation published in 
October 2025122 finds that REACT-EU was rapid and - compared to its overall size - effective 
in addressing the health and economic impacts of the COVID-19, crisis as well as in bridging 
the two programming periods123. It brought forward the implementation of the next framework, 
notably through the recovery measures laying the foundation for the 2021-2027 programmes to 
enhance resilience of Member States.  

To  respond  swiftly  to  migratory  challenges  following  Russia’s  war  of  aggression  against 
Ukraine, CARE/FASTCARE measures provided Member States with flexibility, liquidity and 
simplification  to  facilitate  the  use  of  cohesion  policy  funds124.  According  to  the  supporting 

116  European  Commission:  Directorate-General  for  Employment,  Social  Affairs  and  Inclusion,  Atkinson,  I., 
Andriescu, M., Spyridopoulos, K., Broughton, A. et al., Study supporting the preliminary evaluation of the support 
provided by ESF and FEAD under the Coronavirus Response Investment Initiatives (CRII and CRII+)  – Final 
report, Publications Office of the European Union, 2023, https://data.europa.eu/doi/10.2767/160531  
117 Ibid 
118 Ibid.  
119 See supporting study, Annex VIII. Available at: Annexes. 
120 For which a large part of primary research was completed by September 2023. 
121 See supporting study, Annex VIII. Available at: Annexes. 
122 Report from the Commission to the European Parliament and the Council on the evaluation of REACT-EU. 
Published 10 October 2025. Available at: 
COM_2025_634_F1_REPORT_FROM_COMMISSION_EN_V4_P1_4084848.pdf 
123 Ibid. 
124 European Court of Auditors Special report 05/2025: Cohesion’s Action for Refugees in Europe 

35 

 
 
 
study,  the  support  effectively  reached  the  refugees  and  adapted  services  to  their  needs.  For 
example, Hungary successfully financed public education for 5 700 young persons who fled 
Ukraine. In Italy, Germany and Sweden, stakeholders noted that support proved effective in 
addressing labour market integration and providing language support for refugees.  

4.1.1.4  KEY FACTORS INFLUENCING EFFECTIVENESS125 

Evidence from the interviews, public consultation, desk research and case study research were 
triangulated  in  the  supporting  study  to  identify  the  key  factors  that  facilitated  or  hindered 
effectiveness.  The  following  subsection  groups  these  factors  by  broad  topic  and  indicates  a 
facilitating factor with a (+) and a hindering factor with a (-).   

•  Stakeholder involvement and synergies  

(+) Presence of well-developed networks and strong cooperation among stakeholders 
and partners. This allowed to draw on the expertise of partners close to target groups in 
engaging  these  and  responding  to  their  (changing)  needs,  enhancing  effectiveness  in 
achieving intended outputs and results. 

(+) Effective synergies  with related operations, other policy instruments and funding 
sources.  This  facilitated  referral  between  operations  and  results 
in  positive 
reinforcement. 

•  Analysis of target groups and labour markets 

(+)  Effectively  defining  and  understanding  target  groups,  drawing  on  insights  from 
various sources, including the expertise of organisations working with particularly hard 
to reach groups ‘on the ground’, alongside effective dissemination and communication 
to support outreach and engagement. 

(+)  Effective  mechanisms  to  collate  and  disseminate  learning  from  and  to  delivery 
organisations, for example via working groups on specific topics, as well as follow-up 
actions based on these findings. 

(+)  Sufficient  ex-ante  and  ongoing  analysis  of  labour  market,  education  and  social 
inclusion priorities at all levels, thereby facilitating the needs’ assessment across target 
groups and effective use of data and intelligence to match labour markets’ and target 
groups’ needs and capacities. 

•  Governance and institutional support 

(+) Effective governance and implementation approaches; specifically, adequate  staff 
resources  and  skills  as  well  as  well-designed  governance  structures  with  appropriate 
stakeholder  representation  and  the  ability  to  assess  and  adapt  implementation  as  it 
progresses. 

(+) Reduced administrative burdens and reporting requirements through simplification. 
Compared  to  the  previous  programming  period,  this  refers  specifically  to  the 

125 See answer to evaluation questions 1, 2, 4 and 6 of the supporting study. Available at: Study supporting the ex-
post evaluation of the 2014-2020 European social fund and youth employment initiative - Publications Office of 
the EU 

36 

 
 
 
introduction  of  simplified  cost  options,  alongside  the  digitalisation  of  funding 
application processes and procedures. 

(+) The close link to the Europe 2020 Strategy and the EPSR as well as the requirement 
to allocate a minimum 20% of the ESF allocation to social inclusion has helped to better 
target programmes, to support advocacy for the inclusion of disadvantaged people and 
to enhance the focus on broader social objectives alongside employment support. 

(-)  Administrative  burdens  or  costs,  particularly  for  smaller  organisations  and  the 
complexity  of  ESF  management.  The  issue  was  a  particular  challenge  in  relation  to 
understanding guidance, monitoring data, and ESF audit requirements. 

(-)  Limitations  in  terms  of  resources,  capacity  and/or  expertise,  particularly  amongst 
smaller  or  more  specialist  delivery  organisations  and  in  relation  to  gender 
mainstreaming and non-discrimination and how to address discrimination in contexts 
where prejudices are still strong. 

(-) Lack of a supporting national policy framework and levels of awareness of the social 
economy and its potential contributions. 

•  Reaching target groups 

(+) Adopting practical, individualised and tailored approaches to support individuals. 

(-) Difficulties in motivating harder to reach target groups due to the disengagement of 
such individuals from public services and society. This limits participation numbers and 
hampers effective implementation.  

(-) ‘The creaming effect’, in the sense of delivery organisations being incentivised via 
targets or payment structures to focus support at those groups more easily reached or 
closer to the labour market.  

(-) Limited flexibility to shift attention and resources between target groups.     

•  Technical issues 

(-) Difficulties in monitoring of participants’ development or results, especially those 
after six months, including due to data-sharing or access limitations. 

(-)  Delays  concerning  slow  public  procurement,  technical  delays  in  implementing 
digital systems, and later than anticipated implementation.  

4.1.2. EFFICIENCY 

The efficiency of operations is primarily evaluated through a cost-benefit analysis. This analysis 
informs  judgments  on  efficiency  across  various  operations  and  contexts,  highlighting 
significant cost differences across Member States and Regions.  

Beyond  this  core  analysis,  the  examination  also  encompasses  broader  aspects  of  efficiency, 
including: 

-  The impact of organisational structures and implementation approaches on efficiency. 

37 

 
 
 
-  The  identification  of  unnecessary  administrative  burdens  and  instances  of  'gold 

plating126' 

-  The extent to which simplification solutions have been utilized. 

-  The role of reporting and monitoring in supporting efficient and cost-effective delivery. 

A  comprehensive  review  of  these  dimensions  has  revealed  both  programme-wide  common 
themes and issues specific to individual TOs, which are presented here-below. 

4.1.2.1 COST EFFECTIVENESS OF DIFFERENT ACTIVITIES127 

Looking at the data presented so far in the analysis, the declared expenditure at the end of 2022 
of  EUR  111.7  billion  across  64.6  million  participations results  in  a  cost  of  EUR  1,729  per 
participation.  

A  CBA  of  ESF  and  YEI  interventions  was  conducted  in  the  supporting  study128,  leveraging 
monitoring  data  on  output  and  result  indicators  as  well  as  financial  data  to  assess  the  cost-
effectiveness  of  operations  across  different  Thematic  Objectives  and  Member  States.  This 
analysis  was  supplemented  with  external  data,  such  as  wage  levels  and  tax  rates,  and 
assumptions, like those related to career breaks129 and discount rates. Methodological details are 
available in Annex V of the supporting study. 

The CBA compared costs and benefits of the ESF and YEI. Data on costs were taken from the 
SFC  monitoring  data,  namely  the  money  that  the  Managing  Authorities  have  paid  to 
beneficiaries – the so-called declared expenditures. These costs include both the share borne by 
Member States and the share reimbursed by the European Commission and were assumed to 
include all direct costs for the purpose of the present analysis. Data on indirect costs may be 
available from a limited number of case studies but were ignored in the present CBA where the 
focus was on a workable methodology for all Member States and IPs.  

Costs  were  in  most  cases  calculated  per  participant  since  the  calculation  of  unit  costs  and 
benefits  per  participant  improves  the  comparability  of  costs  and  benefits  between  Member 
States and investment priorities .  

Costs per participant were calculated as the ratio of declared expenditures to the total number 
of participants based on cumulative data up to and including 2022 (making the assumption that 
each  participation  recorded  in  the  system  corresponds  to  one  participant).  The  reported 
cumulative costs were used as observed, hence at current prices instead of price levels of a fixed 
year. Costs per participant per IP are calculated excluding expenditures on REACT-EU. The 
EU average cost per participant was calculated by dividing the part of total declared expenditure 

126 Gold-plating is an expression which refers to Member States going beyond what is strictly required by EU 
legislation when they implement it at national level. This may enhance the benefits but can also add unnecessary 
costs for businesses and public authorities which are mistakenly associated with EU legislation. 
127 See answers to evaluation questions in section 4.1.2.1. Costs and benefits of operations of the supporting study 
and Annex III of the present SWD. Available at: Annexes. 
128 See Annex V of the supporting study. Available at: Study supporting the ex-post evaluation of the 2014-2020 
European social fund and youth employment initiative - Publications Office of the EU and Annexes. 
129 In accordance with the Better Regulation Guidelines where applicable to calculate the so-called discount rate 
38 

 
 
 
 
(that in the present CBA was estimated to be spent directly on participants across all Member 
States)  by  the  total  number  of  participants.  Thus,  costs  are  not  corrected  for  differences  in 
purchasing power. 

For certain IPs where certain Member States invest predominantly in projects (e.g. by social 
partners or NGOs, projects for women to employment, public administration projects, SMEs 
supported),  the  split  of  costs  between  participants  (individuals)  and  entities  or  projects  was 
estimated. The approach was to use data from other Member States that only support individuals 
or only entities to estimate the split of costs in Member States where a mix of individuals and 
entities or projects were supported. In doing so, the CBA also took account of differences in 
purchasing power parities. The cost per entity is then estimated as the remaining expenditures 
divided by the number of entities. 

Results were expressed per unit for the same indicators as the costs. In most cases, these were 
results per participant. For most Member States and investment priorities, data on results were 
available  from  common  indicators.  Benefits  of  increased  productivity  are  in  most  cases 
excluded from the analysis, except for employee training. Macroeconomic knock-on effects are 
further benefits that were ignored in the CBA but are covered in the RHOMOLO modelling 
produced by the EC’s Joint Research Centre. 

The  analysis  estimated  costs  and  benefits  at  the  level  of  investment  priorities  and  Member 
States. The results of interventions generated various public and private benefits, including: 

•  Public  benefits  to  governments  in  terms  of  higher  income  tax  revenue,  higher  VAT 
revenue,  and  reduced  benefit  expenditures  as  a  result  of  improved  labour  market 
outcomes (e.g.  better employment and wages, less unemployment, a more productive 
workforce)130 

•  Private benefits to participants such as the increase in household income after entering 
employment, made possible through e.g. support for individuals, projects and small and 
medium-sized enterprises131 

It is to be noted that certain benefits and costs could not be quantified in monetary terms, such 
as employer benefits (with the exception of employee training), improvement in the quality of 
life, reduction of dependence on certain services, integration into communities, indirect costs, 
macroeconomic effects, and benefits related to good governance and capacity building. As a 

130 Public benefits from increasing employment consist not only in income tax revenues, employee social security 
contributions and employer social security contributions, but also in the increase of VAT on increased consumer 
expenditures. Entering employment increases the household income of the participants, a higher household income 
means that more income is spent on consumer needs, which in turn generates additional VAT revenues. Because 
income gains are greater for higher educated workers than for lower educated workers, VAT revenues will also 
differ by educational level. Entering employment reduces social assistance transfers, which is a substantial part of 
public benefits. Training, education and qualifications open doors for employment or higher positions. 
131 The ESF interventions generate benefits to workers, which are equal to the returns because workers are assumed 
not to bear any cost. This may be too optimistic for some interventions such as training of unemployed and inactive 
participants who may face a lock-in effect of not being employed while in training.  The private return to ESF 
investments to the jobseeker in IP9.i, 9.ii, 9.iii and 9.vi are naturally positive because the intervention increases 
the likelihood of employment. There are other private benefits like health or wellbeing improvement, but this could 
not be monetised. 

39 

 
 
 
result,  these  aspects  were  only  considered  qualitatively,  and  their  exclusion  affected  the 
monetary return calculations. 

To estimate which part of the result may be attributable to ESF and YEI, (i.e. net effectiveness 
rate),  results  of  supported  individuals  needed  to  be  compared  with  results  for  people  with 
comparable  characteristics  who  were  not  supported,  the  so-called  counterfactual.  A  meta-
analysis study on counterfactual impact evaluations has been done by JRC, who calculated the 
net  effectiveness  defined  as  the  result  that  is  attributable  to  ESF  for  different  types  of 
interventions132.  From  this  analysis,  the  present  CBA  used  data  on  net  effectiveness  for  the 
2014-2020 programming period (more specifically, only the data based on the balanced method 
which  uses  only  the  main  result  of  each  study  to  ensure  that  they  all  have  an  equal  weight 
irrespective of the number of estimates provided).  

This results in average impacts on employment at the EU level for different intervention types 
(see Table 3 below). 

Table 3: Balanced average133 impact in employment of ESF by type of intervention (percentage points 
increase in employment probability for participants compared to non-participants) 

Type of intervention 

Vocational training 

Other training 

Mentoring and social support 

Employment subsidies and financial aid 

Internships and traineeships 

Public employment 
Average134 

ESF 2014-2020; Balanced estimate 

5.2 

4.9 

2.2 

15.1 

6.7 

-10.2 

5.9 

Source: Haepp and Serrano-Alarcón (2024)135. See also Section 4.1.1.2 above. 

It is to be noted that the JRC meta-analysis did not estimate the net effectiveness for individual 
investment priorities or Member States. For the CBA, therefore, assumptions were made about 
the assumed predominant types of interventions within each investment priority and those were 
mapped with the corresponding net effectiveness rate from the meta-analysis or rates from other 
studies (see Table 4). If many types of interventions were used in one IP, the overall balanced 

132  Haepp,  T.  and  Serrano  Alarcon,  M.,  Meta-analysis  of  the  European  Social  Fund  counterfactual  impact 
evaluations: Brief update with alternative measures, European Commission, Ispra, 2024, JRC137571, Available 
at: https://publications.jrc.ec.europa.eu/repository/handle/JRC137571 
133 The table shows the balanced average effect, explained in Haepp and Serrano-Alarcón (2024). This selects only the single 
most representative coefficient from each intervention based on a pre-defined selection procedure, in contrast to the average 
effect, which assigns the same weight to each effect that has been coded, meaning that effects from an intervention for which 
many effects from different age and gender groups could be retrieved from a study report enter multiple times in the calculation 
of the overall average effect. 
134 The average net effectiveness of 5.9 percentage points indicates that participants in ESF interventions have 5.9 
percentage points higher probability to be employed than those who are not participating (counterfactual). 
135  Haepp,  T.  and  Serrano  Alarcon,  M.,  Meta-analysis  of  the  European  Social  Fund  counterfactual  impact 
evaluations: Brief update with alternative measures, European Commission, Ispra, 2024, JRC137571, Available 
at: https://publications.jrc.ec.europa.eu/repository/handle/JRC137571 

40 

 
 
 
 
average net effectiveness rate of 5.9 percentage points was applied. Also, a uniform ESF net 
effectiveness rate is applied across all Member States, irrespective of the country-specific rates. 

Table 4: Assumed average ESF net effectiveness assumption per investment priority136 

IP 

8i 

8ii 

8iii 

8iv 

8v 

Short description 

Assumed main 

intervention 

Rationale 

Net % 

Access to employment 

Overall balanced average  Many types of interventions 

Youth employment 

Vocational training 

YEI evaluation report 

Self-employment 

Financial aid 

Start capital 

Mentoring, social support 

Promotion, counselling 

Gender equality 

Work adaption 

Overall balanced average  Many types of interventions 

Employee training 

Mostly employed participants 

Outplacement 

Mix of interventions 

5.9 
2.9 a) 

15.1 

2.2 

5.9 
---b) 
5.9 

4.9 

8vi 

Active 

and 

healthy 

Other training 

Training improves adaptability 

ageing 

8vii 

Labour mobility 

Other training 

Job counselling 

Training improves adaptability 

4.9 

Study on LMP services 

0.6-2.3 

9i 

9ii 

9iii 

Active inclusion 

Overall balanced average  Many types of interventions 

Demarginalisation 

Vocational training 

Low-med educated participants 

Combating 

discrimination 

Varies 

Each country different measure 

9iv 

Access to basic services 

Training of medics 

Employee training 

Health promotion or checks 

Integration of services 

9v 

9vi 

10i 

Social employment 

Public employment 

Non-regular employment  

Local development 

Programme specific 

Mostly projects 

Early school leavers 

School counselling 

Main result is education 

10ii 

Access 

to 

tertiary 

Education grants 

Main result is education 

education 

10iii 

Lifelong learning 

Other training 

Non-employed participants 

Employee training 

Employed participants 

10iv 

Labour market relevance  Overall balanced average  Many types of interventions 

5.9 

5.2 

-- 

-- b) 
---c) 

-10.2 

--- 
12 d) 
15 d) 

4.9 
---b) 

5.9 

Source: supporting study, Annex V 
a)  For Youth employment, both ESF and YEI, the net effectiveness rate has been taken from  the Study for 

the Evaluation of ESF Support to Youth Employment137 

b)  For employee training the net effectiveness has been estimated directly in an OECD paper138 

136 The JRC net effectiveness rates are applied to individual investment priorities based on assumptions about the 
prevailing type of intervention. If many types of interventions were used in one investment priority, the average 
net effectiveness rate of 5.9% is used. The net effectiveness rate is the part that can be reliably attributed to the 
intervention. For instance, in the case of IP8i, the net effectiveness is 5.9 pps, and 20.9% of the participants entered 
employment,  meaning  that  only  15  percent  of  the  participants  (20.9%  minus  5.9  pps)  would  have  entered 
employment without the intervention.  
137 See https://ec.europa.eu/social/BlobServlet?docId=22747&langId=en  
138 Fialho, P., G. Quintini and M. Vandeweyer (2019), Returns to different forms of job related training: Factoring 
in  informal  learning,  OECD  Social,  Employment  and  Migration  Working  Papers  No.  231,  Available  at: 

41 

 
 
 
 
c)  For integration of services the return has been estimated in a recent study139 
d)  For IP10i and IP10ii other sources are used to estimate the additionality of the intervention  

It should be noted that compared to the macro-economic modelling, the CBA applied led to 
higher  impact  results  due  to  different  reasons  (e.g.  different  methodologies  i.e.  CBA 
discounting  net  returns  while  macro-modelling  simulating  system  responses,  or  timeframes 
considered). Macroeconomic models as Rhomolo offer a comprehensive view of long-term, 
interactions  and  indirect  effects,  they  can  lead  to  more  conservative  estimates,  while  CBA 
focuses  on  specific  projects  and  can  sometimes  overstate  benefits  due  to  methodological 
limitations.  

Returns at EU level 

Benefits and costs were compared as a difference (called a return) or as a ratio (called a benefit-
cost ratio) between the two140. The evidence points to a largely positive cost effectiveness of 
ESF  in  terms  of  net  returns  on  investment  and  benefit-to-cost  ratios.  This  implies  that  the 
benefits  are  achieved  at  relatively  low  costs,  overall  and  at  the  level  of  investment  priority, 
notably for the following IPs: 

+ 

+ 

+ 

+ 

IP8.i:    

IP8.iii:   

IP9.i:    

IP10.i:   

+ 

IP10.ii:  

Access to employment (with a return of EUR 26,000 and a ratio of 16) 

Self-employment (with a return of EUR 40,000 and a ratio of 20.5) 

Active inclusion (with a return of EUR 31,000 and a ratio of 16.2) 

Early school leaving prevention and learning pathways (with a return of 
EUR 27,000 and a ratio of 43) 

Access to tertiary education (with a return of EUR 29,000 and a ratio of 
14.8) 

However,  some  benefits  could  not  be  quantified  due  to  a  lack  of  data  and  the  difficulty  in 
estimating  them,  such  as  most  benefits  to  employers  as  well  as  intangible  benefits  (e.g. 
improvements to the quality of life), which means a significant advantage was not accounted 
for in the overall return on investment calculation. 

Table 5 below shows the overall present value of private and public returns (benefits – costs) 
on investment per participant at EU level (see column D). It is noted that a strongly negative 
benefit-cost ratio may indicate either significantly negative benefits or low costs, rendering the 
exact  value  of  negative  ratios  less  meaningful.  In  such  cases,  negative  ratios  are  only 

https://www.oecd-ilibrary.org/social-issues-migration-health/returns-to-different-forms-of-job-related-
training_b21807e9-en 
139 Ecorys and Budapest Institute (2020), Study about the methodology to measure the returns on investment from 
integrated 
https://op.europa.eu/en/publication-detail/-
at: 
/publication/e2a4c40c-3b60-11eb-b27b-01aa75ed71a1/language-en 
140 Ratios of private + public benefits to costs, 2014-2022: To compare the benefits in proportion to the cost, 
quantifiable private and public benefits were added and then divided by the cost (benefit-cost ratio) for each IP. 
42 

schemes,  Available 

assistance 

social 

 
 
 
 
 
 
 
 
 
qualitatively indicated. A more detailed breakdown of the cost-benefit analysis and results for 
each TO is presented after this section.  

Table 5: EU Private and public lifetime return per participant (EUR), by investment priority, 2014-2022 

Cost per 

participant 

Private 
return141 

Public 
return142 

Sum of 
returns143 

Benefit-cost 
ratio a 

IP 

8i Access to employment 

8ii Youth employment 

8iii Self-employment 

8iv Gender equality 

8v Work adaptation 

8vi Active and healthy ageing 

8vii Labour mobility 

9i Active inclusion 

9ii Demarginalisation 

9iii Combating discrimination 

9iv Access to basic services 

9v Social employment 

9vi Local development 

10i Early school leavers 

10ii Access to tertiary education 

10iii Lifelong learning 

10iv Labour market relevance 

(A) 

(B) 

(D=B+C) 

(E = 1+ D/A) 

(C) 

15 316 

9 876 

26 292 

10 901 

2 062 

NA 

NA 

19 588 

4 430 

NA 

1 825 

11 152 

7 341 

13 862 

11 843 

2 189 

NA 

NA 

11 211 

4 916 

NA 

NA 

26 468 

17 217 

40 154 

22 744 

4 251 

NA 

NA 

30 799 

9 346 

0 

NA 

68 327 

-/- 103 866 

-/- 35 539 

9 539 

17 824 

23 530 

4 110 

1 799 

7 749 

9 180 

5 076 

7 266 

-/- 450 

17 288 

27,004 

28,606 

11,376 

1,349 

1 767 

2 151 

2 060 

1 649 

1 203 

-- 

-- 

2 028 

1 693 

1 200 

1 577 

2 176 

1 636 

642 

2 073 

1 260 

2 182 

16.0 

9.0 

20.5 

14.8 

4.6 

NA 

NA 

16.2 

6.5 

NA 

NA 

NEG 

11.6 

43 

14.8 

10 

1.6 

Source: Supporting study, cost-effectiveness analysisNA means: unit costs could be calculated meaningfully for too few 
Member States to calculate an EU average 
 a) Because return = benefit – cost, it follows that benefit = cost + return, and benefit/cost = (cost + return)/cost = 1 + return/cost 
= 1+D/A 
-/- means negative value 

The benefit-cost ratio for each IP of TO 8, 9 and 10 were also presented by country and at EU 
level144.  The benefit-cost  ratio varied significantly across different  countries as they targeted 

141 The participant return (B) is equivalent to the private benefit, as ESF participants do not incur costs to participate 
in the intervention, except for IP10.i and IP10.ii (participants forgo income and pay college fees). 
142 The public return (C) is calculated as the difference between the public benefit and the cost per participant. 
143 The sum of private and public returns is the overall monetisable return of ESF, which can be expressed as a 
difference between benefits and costs, called returns (D), which in this study is calculated at the level of investment 
priorities, or as a ratio of benefits to costs or benefit-cost ratio (E). Benefits are greater than the cost if the return 
is greater than zero (column D), or if the benefit-cost ratio is greater than one (column E). 
144 See more detailed overview tables presenting i) unit costs, ii) successful exit rates per individual participant, 
iii) unit costs per successful participant, iv) public return (to the State) per participant of 2014-2022 for each IP in 
section 4.1.2.1. Costs and benefits of operations of the supporting study. 

43 

 
 
 
 
 
 
 
 
 
different groups or implemented different types of measures or because there are differences in 
efficiency145. Moreover, these results also differ from the macro-economic modelling applied 
(Rhomolo) due to different reasons as explained in section 4.1.1.2 above. 

 TO 8 

Within TO 8 the combined private and public returns on investment are highest for two areas: 
access  to  employment  (IP8.i)  and  self-employment  (IP8.iii).  The  public  returns  on  self-
employment  were  particularly  substantial,  primarily  due  to  the  effectiveness  of  financial 
support for inactive individuals starting their own businesses.  

Additionally,  investments  in  youth  employment  (IP8.ii)  and  gender  equality  (IP8.iv)  also 
yielded  positive  returns,  with  estimated  values  of  around  EUR  17,000  and  EUR  23,000  per 
participant  respectively.  These  investments  focused  on  more  disadvantaged  groups,  yet  still 
generated  positive  outcomes.  Specifically,  gender  equality  support,  which  largely  involved 
childcare assistance and access to employment for women re-entrants and migrant spouses, has 
shown to produce positive returns. 

At the EU level, the sum of discounted future benefits is between 5 and 20 times higher than 
the cost for all five IPs where the EU average could be meaningfully calculated.  

TO 9 

Investments under the TO 9 framework yielded varying returns on investment. Notably, active 
inclusion  initiatives  (IP9.i)  and  vocational  training  to  combat  marginalization  (IP9.ii) 
generated  the  most  positive  combined  private  and  public  returns,  with  estimated  values  of 
approximately EUR 31,000 and EUR 9,000 per participant, respectively, at the EU level. 

In almost all Member States, active inclusion (IP9.i) produced positive public returns, while 
vocational training (IP9.ii) yielded positive public returns in nearly all Member States, except 
for two with relatively high ESF unit costs. Furthermore, investments in access to basic services 
(IP9.iv) generated small but positive public returns in most Member States, primarily due to the 
labour  market  effects.  Although  the  primary  objectives  of  these  investments  were  not 
employment or qualification gains, the outcomes were still moderately positive. 

In  contrast,  investments  in  social  employment  (IP9.v)  produced  substantial  negative  returns, 
estimated at approximately EUR 36,000 per participant at the EU level. This was primarily due 
to two factors: the high costs of providing a full-time minimum wage until retirement age, and 
the reduced likelihood of regular employment by about 10 pps for those employed in public 

145 For TO 11 (capacity building) and 13 (REACT-EU), benefits cannot be monetized because the type of benefit 
is generally unknown. For TO 11, participants are typically officials working at for example Public Employment 
Services, to improve their skills to support or to monitor job seekers. However, what benefits bring this support is 
unknown.  For  REACT-EU,  even  though  funds  are  administratively  allocated  to  one  of  the  ESF  investment 
priorities,  the  measures  can  be  different  from  the  standard investment  priorities,  and  common  output  or  result 
indicators for REACT-EU cannot be attributed to specific investment priorities as is the case for TO 8 through TO 
10. Hence, for both TO 11 and 13 only result ratios and unit costs are presented in the annex related to the CBA 
to reflect cost-effectiveness. As a result, benefit-cost ratios were only provided for IPs belonging to TO 8, 9 and 
10 and, therefore, it was not possible to compute an average benefit-cost ratio for the whole ESF. 

44 

 
 
 
 
work schemes. However, investments in social enterprises can still be justified as a means of 
providing meaningful support to individuals with severe disabilities. Periodic re-examinations 
of disability, such as those implemented in Belgium, could potentially mitigate the negative 
effects.  Also,  it  needs  to  be  noted  that  these  schemes  often  have  mainly  a  social  inclusion 
dimension. They rather target softer, intangible results such as improved well-being or a sense 
of belonging instead of moving participants into regular employment. 

The benefits  of  combating  discrimination (IP9.iii) and  local development  (IP9.vi) initiatives 
were  more  challenging  to  quantify.  Many  measures,  such  as  investments  in  awareness  of 
discrimination,  cannot  be  monetised  due  to  a  lack  of  data  on  their  impact  on  workers. 
Additionally, local development investments often have varying focuses on employment and 
differing ESF unit costs, which may be influenced by the reallocation of costs to participants. 

For the three IPs where benefit-to-cost ratios at the EU level could be meaningfully estimated 
and were positive, the ratios vary between 6 and 16.  

TO 10 

Investments in education yielded significant returns, both privately and publicly, particularly 
when focused on young individuals. At the EU level, the returns on investment are substantial:  

- Early school leaving prevention (IP10.i) generated approximately EUR 27,000 per participant. 

- Tertiary education access (IP10.ii) yielded around EUR 29,000 per participant. 

Lifelong  learning  investments  (IP10.iii)  also  showed  promising  returns,  especially  for 
unemployed  or  inactive  workers,  as  it  increased  their  employment  prospects.  However,  for 
employed workers, investments are only estimated as fruitful if average costs remain low (under 
EUR 1,000 per participant). 

Investments in labour market relevance of education (IP10.iv) also yielded a positive overall 
return on investment of around EUR 1,350 per participant, although accompanied by a negative 
public return of EUR 450 per participant. It is to be noted, however, that many of the benefits 
of such investments cannot be monetized, such as making vocational education more attractive, 
reducing early school leaving, and keeping curricula up-to-date.  

The sum of private and public benefits was positive for all IPs and Member States under TO 10. 
Ratios  of  about  100  or  higher  are  explained  by  low  unit  costs,  as  is  the  case  for  IP10.iii  in 
Bulgaria (unit cost EUR 23), IP10.i in France (unit cost EUR 258) and IP10.iii in Spain (unit 
cost EUR 124). At the EU level, the ratio varied from 1.6146 for IP10.iv to 43 for IP10.i.  

TO 11 

IP11.i (good governance) 

The ESF invested in IP11.i (good governance) in a mix of participants and entities. The main 
result for participants was the achievement of a qualification or improved skills. In contrast, 
entity-level  results  were  more  varied,  encompassing  employee  training,  service  integration, 

146 meaning that benefits are 160% of the cost (e.g. EUR 160 of benefits compared to EUR 100 of costs results in 
a benefit-to-cost ratio of 1.6) 

45 

 
 
 
quality  management  enhancement,  and  ICT  tool  implementation.  The  success  rates  for 
supported participants were generally high at 80% to 100%.  

When calculating unit costs, it was noted that Malta was the only Member State that reported 
only participants as output. Therefore, the other countries reported a mix of outputs (participants 
and  entities)  for  which  the  assumption  was  made  that  half  of  the  expenditures  were  on 
participants.  Four  countries  (EE,  PO,  SI,  SK)  invested  in  addition  in  a  mix  of  projects  and 
entities, in which cases further additional assumptions were made. 

For entities the aims of interventions vary, from training of employees, integration of services, 
improved  quality  management  and  the  implementation  of  ICT  tools.  The  aim  of  project 
interventions of IP11.i is rarely specific but rather described in terms such as “implementation 
of processes” or “reduced administrative time”. The success rates per entity or per project vary 
between a few per cent to about 100%. 

 IP11.ii (capacity building) 

Under  IP11.ii  (capacity  building),  only  seven  Member  States  (France,  Croatia,  Italy,  Malta, 
Poland, Portugal and Slovenia) invested, mostly in the capacity of civil society organizations. 
Costs per participant varied from about EUR 400 in Italy to about EUR 6 000 in France. The 
support to projects and entities was difficult to separate and are not presented. Investments in 
participants resulted in achievements of close to 100% qualification gains in France, Italy and 
Slovenia, and about 50% in Malta. Portugal only invested in entities at a cost of close to EUR 
200 000 per entity, to promote capacity building and regional development, with undocumented 
results. 

TO 13 – REACT-EU 

The  primary  focus  of  investments  under  the  REACT-EU  initiative  was  on  short-term  work 
measures, with additional investments made in Germany and Spain for access to healthcare, 
social  services,  and  childcare,  and  in  Belgium,  Bulgaria,  and  Finland  for  the  activation  of 
individuals.  Activation and job preservation were the most common aims of the intervention, 
although  Germany  and  to  a  lesser  extent  Spain  in  addition  invested  in  people  to  gain  a 
qualification under TO13. In the Netherlands and Belgium, ESF resources were primarily used 
to  support  the  administrative  management  of  COVID-19  measures,  rather  than  investing  in 
short-term work measures. 

During  the  COVID-19  pandemic,  the  conventional  concern  about  lost  productivity  due  to 
employee  training  was  less  relevant,  as  the  period  of  inactivity  presented  an  ideal  time  for 
investing in skills development. The estimated costs per participant supported under REACT-
EU  varied  widely  across  countries,  ranging  from  approximately  EUR  5  in  Austria  to  EUR 
12,600 in Slovakia.  

OECD  estimated  mixed  impacts  for  short-time  work  measures  during  the  financial  crisis147 
However,  a  similar  assessment  cannot  be  made  for  short-time  work  measures  during  the 
COVID-19  pandemic  due  to  the  lack  of  comparable  treatment  and  control  groups  among 

147 Hijzen, A. and D. Venn (2011), The Role of Short-Time Work Schemes during the 2008-09 Recession, OECD 
Social, Employment and Migration Working Papers No. 115, Available at: https://doi.org/10.1787/1815199X 

46 

 
 
 
supported  and  unsupported  enterprises.  Nevertheless,  the  justification  for  short-time  work 
measures during the pandemic was clear as the alternative would have been widespread layoffs 
that would have overwhelmed social insurance institutions and public employment services, 
resulting  in  significant  unemployment  benefit  expenditures  and  the  collapse  of  viable 
businesses. 

While  short-time  work  measures  may  have  reduced  job  losses,  they  may  also  have  limited 
"creative  destruction"  like  the  one  triggered  through  former  employees  starting  their  own 
business or through resources (financial, human) reallocated from declining firms and sectors 
to thriving ones. However, the pandemic was not an ideal time for entrepreneurship, which may 
have  mitigated  the  costs  associated  with  reduced  "creative  destruction".  On  the  other  hand, 
short-time  work  measures  provided  an  opportunity  for  employee  training,  which  was 
particularly beneficial for workers who would have otherwise been unemployed. This not only 
helped to maintain workforce skills but also reduced the administrative burden and expenditures 
associated with processing unemployment benefits. 

The  Commission’s  dedicated  REACT-EU  evaluation  published  in  October  2025  finds  that 
REACT-EU was rapid and - compared to its overall size - effective in addressing the health and 
economic  impacts  of  the  COVID-19,  crisis  as  well  as  in  bridging  the  two  programming 
periods148 but could also not include a cost benefit analysis and an efficiency assessment due to 
the lack of data on implementation. 

In conclusion, the cost-benefit analysis revealed generally positive outcomes in terms of net 
public  returns  on  investment,  though  variations  in  cost-effectiveness  were  observed  across 
different Investment Priorities. Unit costs differed significantly across countries, primarily due 
to distinct target groups and the implementation of diverse measures. 

4.1.2.2 FACILITATING FACTORS TO EFFICIENCY149   

Administrative procedures within the ESF have undergone significant improvements during the 
programming period, primarily driven by the implementation of simplified cost options (SCOs) 
proposed by the Commission, digitalisation and streamlined reporting. These innovations have 
collectively contributed to a notable reduction in administrative burdens. 

Implementation of simplified cost options  

One  key  factor  in  enhancing  the  efficiency  of  ESF  administrative  processes  has  been  the 
widespread  adoption  of  SCOs,  such  as  flat  rates  and  standard  unit  costs.  These  measures 
facilitated simplified financial reporting and monitoring, thereby alleviating the administrative 
burden  on  beneficiaries.  A  substantial  majority  (89%)  of  respondents  in  the  Managing 
Authority  survey  reported  that  SCOs  improved  operational  efficiency.  Case  studies  from 

148 Report from the Commission to the European Parliament and the Council on the evaluation of REACT-EU. 
Published 10 October 2025. Available at: 
COM_2025_634_F1_REPORT_FROM_COMMISSION_EN_V4_P1_4084848.pdf 
149 See answers to evaluation questions in section 4.1.2.6 of the supporting study. Available at: Study supporting 
the  ex-post  evaluation  of  the 2014-2020  European  social  fund  and  youth  employment  initiative  -  Publications 
Office of the EU  

47 

 
 
 
 
multiple  Member  States  have  demonstrated  the  positive  impact  of  SCOs  on  reducing 
administrative burden. As such, SCOs were the most important simplification proposed by the 
Commission that had to a significant extent improved the efficiency of operations compared to 
the previous programming period. 

However, some Member States, such as Spain, initially faced administrative challenges with 
the introduction of SCOs, which required the development  of new certification methods for 
each operation. Moreover, some case studies highlighted instances where the new rules were 
not clearly explained to beneficiaries, leading to concerns.  

Related to SCOs, there was limited reference to Joint Action Plans as an additional mechanism 
to  use SCOs in  a more strategic and secured  framework. As such, evidence is  lacking  as to 
whether Joint Action Plans improved the efficiency of operations.  

Digitalisation 

The digitalisation of application and reporting procedures has also significantly supported the 
efficient implementation of ESF and YEI operations across the EU and therefore was identified 
a key good practice to reduce unnecessary administrative burden. Many managing authorities 
reported that they have successfully leveraged online tools and platforms to enhance operational 
efficiency.  Although  some  Member  States,  such  as  Romania  and  Slovakia,  encountered 
challenges when introducing new IT platforms, the potential for positive returns in the future 
remains, pending the resolution of difficulties, e.g. in introducing new IT platforms, delays or 
a lack of linkage between different systems or in using the systems which contributed to the 
administrative burden on its users. Nonetheless, in general digitalisation has been a welcome 
development.  

Streamlined reporting 

Other good practice elements of simplification were identified in the case study work on IP9vi 
(Local development). In light of, and responding to, the resource intensive nature of the small 
allocations  associated  with  Community-Led  Local  Development  (CLLD)  activity,  some 
Member States introduced much lighter monitoring and reporting structures to help ease the 
burden on recipient organisations as well as Managing Authorities. Simplified reporting at the 
end of the funding period was also identified including ‘soft’ evaluation of projects introduced 
in some Member States so that project owners completed a short self-evaluation form, rather 
than  undertaking  an  external  evaluation.  Lighter  touch  reporting  was  also  introduced  in  the 
context of crisis response programmes as shown in other sections of the present report. 

4.1.2.3 ADMINISTRATIVE BURDEN150   

150   See answers to answers to evaluation questions in section 4.1.2.2 of the supporting study. Available at: Study 
supporting the ex-post evaluation of the 2014-2020 European social fund and youth employment initiative - 
Publications Office of the EU 

48 

 
 
 
According to estimates based on a 2018 study and spending data collected by end 2023151, the 
total costs of administering the ESF and YEI programmes for public authorities amounted to 
EUR 3.6 billion, or 2.7% of total spending. This level is considered modest, as administrative 
costs for public authorities below 4% are generally not regarded as excessive152153. In relative 
terms the average administrative costs for ESF programme authorities are estimated to be EUR 
27,600 per million euros spent. The majority of these costs fell on managing authorities (80%), 
followed by audit authorities (13%) and the costs for other authorities (certifying authorities, 
national coordination and authorities involved in programme preparation) were relatively small. 
The  most  cost-intensive  tasks  for  programme  authorities  were  selection  of  operations  and 
information  to  beneficiaries  (EUR  2,616  per  million  spent),  followed  by  verifications  for 
reimbursement of beneficiaries (EUR 2,391 per million) and information and communication 
(EUR  1,668  per  million).  Average  costs  are  influenced  by  programme  size,  with  smaller 
programmes tending to be more costly per million spent. Based on estimates for all ESIF funds, 
Thematic Objective (TO) 11 (Institutional capacity and efficient public administration) had the 
highest administrative costs, while TO 10 (Education, training and lifelong learning) recorded 
the lowest.  

The administrative costs are also incurred by beneficiaries and consisted of costs of preparing 
a funding application (overall EUR 3.3–6.6 billion; 2.5%–4.9% of total spending)) and project 
management  costs  ranging  from  overall  EUR  14.6  to  19.2  billion  (10.8%–14.3%  of  total 
spending).  

It  should  be  noted  that  administrative  cost  estimates,  especially  for  beneficiaries,  should  be 
treated with caution. The cost estimates are based on an extrapolation from a 2018 study and 
not on actual costs incurred during the period. This also implies that the regulatory changes 
after 2018 were not considered. The sample size for beneficiaries was rather limited – only 66 
ESF  beneficiaries.  Moreover,  the  cost  estimates  for  beneficiaries  do  not  only  cover  the 
administrative costs stemming from cohesion policy regulatory requirements, and it was not 
possible to distinguish them from those arising under the national regulatory framework. 

In the 2014-2020 programming  period several  simplification measures154 were introduced to 
streamline  the  processes  and  increase  flexibility.  These  measures  have  likely  produced 
efficiency gains for programme authorities. The 2018 study suggests that the overall costs for 
programme authorities have decreased by 10% or from EUR 30,520 to EUR 27,600 per million 
spent in 2014-2020 in comparison to the 2007-2013 programming period. An especially high 
decrease was for the managing authorities which fell from EUR 27,800 per million spent to 
EUR 22,200 per million spent. However, the decreases for managing authorities were partly 

151 European Commission: Directorate-General for Regional and Urban Policy, Spatial Foresight and t33,  New 
assessment of ESIF administrative costs and burden, 2018, https://data.europa.eu/doi/10.2776/36663. Calculations 
are based on raw data of this study and spending data by end 2023. For more information on methodology, see 
Annex II. 
152 European Court of Auditors Special Report 07/2020, point 68. Implementing Cohesion policy: comparatively 
low costs, but insufficient information to assess simplification savings 
153 New assessment of ESIF administrative costs and burden, 2018, https://data.europa.eu/doi/10.2776/36663, page 
8-9 
154 See section 4.1.2.2. of this SWD. For an overview of the simplification measures see: European Commission, 
Sweco, t33 & Spatial Foresight, ‘Use of New Provisions on Simplification during the Early Implementation Phase 
of ESIF’, 2017. https://ec.europa.eu/regional_policy/sources/studies/new_prov_simplification_esif_en.pdf. 

49 

 
 
 
offset  by  increases  for  audit  authorities  (from  EUR  1,180  to  EUR  3,500  per  million  spent), 
certifying  authorities  (from  EUR  400  to  EUR  700)  and  authorities  involved  in  programme 
preparation (EUR 1,100 to 1,200).  

A comparison of costs by selected tasks shows a significant decrease in costs for information 
and communication activities (from EUR 6,100 to EUR 1,400 per million spent) in 2014-2020 
as compared to 2007-2013 programming period. The costs of administrative verifications of 
payment claims also declined, from EUR 3,400 to EUR 2,400, which can be explained by the 
simplifications introduced in the 2014–2020 programming period, particularly the extended use 
of  simplified  cost  options.  Increased  attention  to  monitoring  and  evaluation  did  not  lead  to 
higher administrative costs. On the contrary, monitoring and evaluation costs for programme 
authorities fell from EUR 2,200 to EUR 1,600, suggesting that common indicators and various 
supporting  tools  (SFC,  guidance  on  monitoring  and  evaluation)  supported  the  work  of 
managing  authorities.  Cost  increases  were  observed  for  tasks  related  to  the  selection  of 
operations and information provided to beneficiaries, which rose from EUR 1,800 to EUR 2,400 
per  million  spent.  In  addition,  the  costs  of  audit  operations  tripled  (from  EUR  460  to 
EUR 1,400), suggesting that part of the control workload associated with the verification of 
payment claims shifted from managing authorities to audit authorities. 

Despite  the  substantial  strides  made  in  reducing  administrative  burdens  compared  to  the 
previous  programming  period  and  the  estimated  reduction  in  administrative  costs  for 
programme  authorities,  ESF  programmes  kept  facing  negative  perceptions  of  unnecessary 
administrative  burdens.  Stakeholders  from  national  and  regional  MAs  and  beneficiary 
organisations across Member States reported that complicated and burdensome administrative 
processes  including  "gold  plating"155  were  present.  These  issues  were  acknowledged  as 
hindering the efficiency of operations, often leading to project implementation delays.  

While these issues varied across Member States, some respondents to different consultations 
felt  that  such  burden  existed  but  had  a  limited  impact  on  efficiency,  and  the  benefits  of 
simplification  offset  them.  Managing  Authorities  survey  results  showed  that  61%  of 
respondents  (69  out  of  114)  believed  ESF-related  administrative  burden  significantly  or 
moderately  hindered  efficiency  for  project  beneficiaries,  and  51%  (58  out  of  114)  reported 
similar burden for MAs. The high volume of data and documentation required from applicants 
to  request  the  subsidies  was  considered  high  and  unintendedly  conducive  to  inequalities 
between small and larger entities in terms of capacity and ease of access to the measure. 

Challenges  remain  in  specific  IPs  or  Member  States,  particularly  in  areas  such  as  financial 
reporting, claim preparation and submission, documentation, audit requirements, and control 
systems. Examples of issues causing administrative burden across different TOs identified in 
the case study work were the following: 

-  Audit  and  reporting  requirements:  unclear  requirements  for  collecting  participants' 
data,  timesheets,  and  supporting  documentation  contributed  to  high  administrative 
burdens (e.g. Italy, Czechia, Denmark). They resulted in delays, additional resources 

155 Gold-plating is an expression which refers to Member States going beyond what is strictly required by EU 
legislation when they implement it at national level. This may enhance the benefits but can also add unnecessary 
costs for businesses and public authorities which are mistakenly associated with EU legislation. 

50 

 
 
 
needed,  and  challenges  in  securing  partners  due  to  uncertainties  associated  with 
reimbursement  rates.  Different  levels  of  audits  (at  regional,  national  and  EU  level) 
were viewed as causing delays and complications, which ultimately have an impact on 
efficiency. 

-  Project administration and management: difficulties in setting up and running project 
administration, such as registering hours and determining eligible hours (e.g. Greece, 
The  Netherlands,  France);  high  administrative  burdens  due  to  largely  paper-based 
processes and justification documents (e.g. Hungary, Croatia). 

-  Unconnected management systems and different reporting requirements and deadlines 
in certain countries156: these issues added to the administrative burden; Bureaucratic 
and directive oversight: Perception of national institutions being overly directive and 
bureaucratic (e.g. Estonia). 

-  Public procurement procedures: lengthy public procurement procedures led to delays 
in implementation and cancelled calls (e.g. Lithuania); Staff turnover and changes in 
legislation: high turnover in administrative staff due to changes in country legislations, 
resulting  in  the  loss  of  competences  and  skills  acquired  in  previous  programming 
periods; Delays in implementation as a result of disagreements among stakeholders, 
staff shortages, and frequent changes in the legislative framework 157. 

-  Eligibility and registration requirements: The need for participants to be registered as 
unemployed  by 
the  Public  Employment  Service  (PES)  created  additional 
administrative procedures for beneficiary organisations (e.g. Croatia, Latvia, Lithuania 
concerning YEI). 

A minority view emerged from some MAs and public authority stakeholders, who considered 
the administrative burden to be acceptable and necessary. They argued that proper justification 
and transparency were essential for efficient management of funds, and that the administrative 
burden was justified by the project outcomes and resources used. 

4.1.2.4 ADMINISTRATIVE BURDEN IN RESPONDING TO CRISES158  

The  evaluation  of  crisis  response  programs,  CRII  and  CRII+,  revealed  mixed  views  on  the 
administrative  burden  associated  with  these  programs.  While  simplified  and  accelerated 
processes  helped  Member  States  respond  quickly  to  crises,  MAs  reported  difficulties  in 
familiarising  themselves  with  the  rules,  such  as  the  administrative  requirements159,  and 

156 For more information refer to the case study on TO 10 
157 C.f. Evaluation Helpdesk (2023) Evaluations of Cohesion policy programmes published in 2014-2020 period: 
Thematic report: TO 11, Enhancing institutional capacity   
158 See answers to answers to evaluation questions in section 4.1.2.4. of the supporting study. Available at: Study 
supporting  the  ex-post  evaluation  of  the  2014-2020  European  social  fund  and  youth  employment  initiative  - 
Publications Office of the EU  
159 The case study research revealed that in several Member States, including Italy, Portugal, and Estonia, MAs 
found the volume of data and documentation required from applicants to be overly high, particularly for small 

51 

 
 
 
 
 
flexibilities. Key challenges included high administrative workload for managing and planning 
authorities, increased data and documentation requirements, which were deemed burdensome 
and may have created inequalities between small and large entities and stringent national audit 
requirements in some Member States. 

However,  some  positive  examples  were  noted  such  as  an  open-ended  funding  application 
process for CARE funding in Sweden or the setup of guidance webinars and a dedicated task 
force  in  Italy,  which  helped  to  simplify  the  implementation  process  and  provide  effective 
support to beneficiaries. 

4.1.2.5 THE MANAGEMENT AND CONTROL SYSTEM REQUIREMENTS160  

Evidence indicates that the monitoring and control system was largely effective, demonstrating 
flexibility, partnership, and a results-driven approach.  The monitoring requirements, as defined 
in the EU regulation, were broadly  functioning well, largely providing a complete overview 
and  quality  data  on  the  participants  and  their  basic  characteristics  on  a  regular  basis.  This 
allowed  for  the  performance  analysis  during  implementation  and  was  also  achieved  at  a 
reasonable cost.  In  addition,  MA stakeholders recognised  significant  steps made to  simplify 
ESF/YEI  processes  by  many  national  ESF  monitoring  committees.  Digitalisation  and  the 
effective use of online tools and platforms were widely discussed as elements aiding efficiency 
of monitoring and evaluation and reducing costs. 

Still, opportunities for efficiency improvement exist, particularly in areas such as digitalisation, 
financial reporting, and streamlining national audit requirements and control systems. In this 
context,  a  number  of  stakeholders  indicated  excessive  monitoring  and  financial  reporting 
requirements  as  a  primary  source  of  administrative  burden,  in  particular  related  to  control 
systems and auditing processes at regional, national, and EU levels, which hindered efficiency. 
For instance, among MA respondents, 45% (51 out of 114) considered that the complexity of 
monitoring  arrangements  hindered  the  efficient  implementation  of  operations.  Beneficiary 
stakeholders, through the consultations and case study work, likewise outlined challenges with 
particular aspects of the data required for monitoring either in terms of the nature of the data or 
in terms of the format it was collected. Somewhat in contrast, the case study research identified 
some  positive  examples  of  high-quality  monitoring  activity  being  facilitated  despite  the 
perceived challenging requirements (e.g. The Netherlands). 

Similarly, the evaluation system showed significant improvements concerning the number and 
quality  of  evaluations.  In  the  2014-2022  period  1 352  evaluations  involving  ESF  and  YEI 
interventions were carried out, which is significantly more than in the previous programming 
period. The attention on impact and results evaluations was also important, with around 30% 
of  the  evaluations  focused  on  final  effects.  The  methodological  approaches  also  showed  an 
increasing maturity in the evaluation system with, for instance, 35% of the impact and results 

entities.  As  a  result,  this  may  have  inadvertently  created  inequalities  in  access  to  the  support.  Additionally,  a 
previous evaluation of CRII identified that in other Member States, such as Greece and Poland, the administrative 
burden was also attributed to strict national audit requirements. 
160 See answers to evaluation questions in section 4.1.2.3. of the supporting study. Available at: Study supporting 
the  ex-post  evaluation  of  the 2014-2020  European  social  fund  and  youth  employment  initiative  -  Publications 
Office of the EU 

52 

 
 
 
evaluations  using  counterfactual  methods,  which  were  almost  absent  in  the  previous 
programming  period.  In  general,  evaluation  has  increased  its  importance  and  has  become  a 
more  systemic  and  well-integrated  component  of  the  monitoring  and  control  system. 
Information available from the national evaluations are incomparably richer than in the past 
programming period.  

Yet,  some  specific  limitations  remained  which  undermined  the  ability  to  evaluate 
implementation more deeply, notably: 

-  Capacity  constraints:  some  Member  States  struggled  with  evaluation  quality, 
particularly  countries  with  a smaller administration  or those without prior evaluation 
experience. 

-  Perceived  obligations:  evaluation  was  sometimes  seen  as  a  formal  requirement,  not 

influencing decision-making mechanisms. 

-  Limited scope: evaluation was often restricted to supporting implementation, rather than 

informing strategic choices or programming. 

These areas can be targeted in future programmes for further support from the Commission to 
address capacity constraints and promote more effective evaluation practices. 

Both, monitoring and evaluation systems could be improved by introducing certain novelties, 
which are described in ‘Lessons learned’ (c.f. section 5). 

The 2021-2027 framework has already incorporated several simplification measures such as 
reduced list of policy objectives, clearer intervention logic through indicators, lighter reporting 
and single audit  arrangements, which  demonstrates  a positive step towards simplifying ESF 
implementation processes. 

4.1.2.6 OTHER HINDERING FACTORS TO EFFICIENCY161   

Additionally, other hindering factors to efficiency identified by stakeholders across different 
groups include: 

-  Limited  administrative  capacity  of  project  beneficiaries,  exacerbated  by  a  heavy 
administrative burden, lengthy project processes, and complex guidance manuals. 

-  Delays in funding, high staff turnover, and lack of specialised technical skills. 

-  Data protection requirements, which create misunderstandings and additional resource 
needs in terms of time and costs, hindering the efficient implementation of projects. 
Specifically in Germany, strict data protection rules were cited as causing a significant 
administrative burden.  

161 See answers to evaluation questions in section 4.1.2.5 of the supporting study. Available at: Study supporting 
the  ex-post  evaluation  of  the 2014-2020  European  social  fund  and  youth  employment  initiative  -  Publications 
Office of the EU  

53 

 
 
 
 
4.1.3. COHERENCE 

4.1.3.1  COMPLEMENTARITY  AND  SYNERGIES  OF  ESF  AND  YEI  WITH  NATIONAL  AND 
REGIONAL ACTIONS AND POLICIES162 

The complementarity and synergies of ESF and YEI with national and regional strategies or 
activities were strong. From the case studies and stakeholder interviews, coherence with other 
national  or regional activities was assessed by stakeholders of all types as very good. Some 
84%  of  respondents  to  the  MA  survey  (N=114)  reported  a  view  that  ESF  operations  were 
coherent  with  national/regional  policies  to  a  large  extent.  This  was  supported  by  numerous 
examples identified from the case studies, mapping of operations and examples provided by 
national and regional stakeholders of national strategic documents being in alignment with ESF 
objectives.  The  high  degree  of  coherence  between  national  and  EU  policies  and  funds  was 
found  to  have  been  greatly  aided  by  years  of  experience  in  delivering  the  ESF.  The 
embeddedness of ESF in public administrations, particularly in labour market institutions, was 
widely seen as facilitating the transfer of programming and managing mechanisms into national 
measures, with substantial similarities and cross-fertilisation among national and EU-funded 
programmes. Case study evidence on social innovation however points at a need for greater 
convergence between European, national and regional resources dedicated to social innovation 
to facilitate implementation and scaling up of social innovation projects at local level163.  

The ESF and YEI demonstrated good level of internal coherence with the different parts of the 
programmes,  working  well together to  achieve their objectives. The various projects  funded 
under different areas of intervention (Thematic Objectives) were distinct yet complementary. 
Deeper analysis revealed only limited overlaps and in general Thematic Objective 9 operations, 
focusing on particular target groups like Roma and people with disabilities, were distinct from 
Thematic Objective 8 operation for the same target groups. Adjustments made in response to 
COVID-19 also maintained coherence by targeting different groups and using varied delivery 
mechanisms. Despite rapidity, REACT-EU remained consistent with the broader objectives of 
cohesion policy as well as acted in synergy with other instruments, in particular the SURE.  

4.1.3.2 COMPLEMENTARITY WITH OTHER EU INSTRUMENTS164 

There was robust evidence of dedicated EU support being coherent with other EU funds. EU 
level interviewees consistently noted what they saw as a strong coherence of ESF with other 
funds, such as European Regional Development Fund (ERDF) (including Interreg), the Fund 
for  European  Aid  to  the  Most  Deprived  (FEAD),  or  Erasmus+  in  terms  of  complementary 
objectives,  scope  and  target  groups.  This  emphasis  on  ERDF  is  reflected  in  the  public 
consultation  results  (701  respondents),  where  61%  of  respondents  reported  the  ESF  was 
complementing or reinforcing ERDF. This was in turn also reflected in the EESC evaluation 
exercise with  67  civil  society organisations, where 80% of respondents  believed ERDF was 
fully or at least moderately integrated with the ESF. With respect to TO 8, it was reported by 

162 See answer to evaluation question 18 of the supporting study. 
163 See case study “Social Innovation” available in the supporting study.  
164 See answer to evaluation question 16 of the supporting study. 

54 

 
 
 
 
the supporting study of the ex-post evaluation of ESF and YEI that a few ERDF Investment 
Priorities directly contributed to TO 8. At the same time, the ERDF ex-post evaluation found 
scope to leverage synergies with ESF more consistently and effectively, such as in  fostering 
equal  access  to  services,  the  ICT  sector  and  territorial  instruments.  ERDF  allocations  were 
higher  under  TO9  and  TO10.  Stakeholders  in  the  ERDF  evaluation  also  pointed  to 
administrative challenges that prevented optimal synergies with the ESF across interventions, 
including in the area of e-government and e-business.  

Regarding  other  instruments,  the  ESF  also  presented  strong  coherence  with  the  FEAD. 
According to the ex-post evaluation of FEAD 2014-2020, the legal texts governing the ESF 
regulation and FEAD regulation, demonstrate a good degree of complementarity, with specific 
provisions designed to ensure complementarity between these funds or to prevent overlaps or 
duplication. 

Examples from the case studies in the ex-post evaluation of FEAD highlight good practices in 
several  Member  States  in  terms  of  complementarities  between  FEAD  and  the  ESF  such  as: 
designing  ESF  and  FEAD  operational  programmes  to  complement  each  other;  actively 
informing and supporting food aid recipients to access ESF-funded training and employment 
measures  including  via  helpdesks  and  teams  of  professionals  delivering  accompanying 
measures; or using ESF-funded measures to support specific target groups for training or other 
labour market-related operations.  

Operations  funded  by  the  EARDF  focused  on  contributing  to  promoting  employment  and 
creating new jobs in rural areas. The Employment and Social Innovation (EaSI) strand focused 
more on operations that are upstream with respect to the ESF, but nonetheless the EURopean 
Employment Services (EURES) axis in particular was considered by the supporting study of 
the ex-post evaluation of ESF & YEI as offering specific synergies to TO 8, specifically that of 
improving labour mobility and particularly its support of language training. 

Activities supported by the Asylum, Migration and Integration Fund (AMIF), and in particular 
those related to the legal migration and integration specific objective under the AMIF, were 
considered to be complementary by the stakeholders’ views recorded in the study supporting 
the 2014-2020 ex-post evaluation of ESF & YEI to operations carried out under IP8.i. given 
they both promote access to employment and labour mobility.  

The use of ESF funds under TO 9 in conjunction with other EU Funds was identified mainly in 
relation to ERDF165 . A common pattern in the examples identified was where ERDF provided 
funds for physical capital measures such as infrastructure and equipment while ESF provided 
funds  for  human  capital  measures  including  staff  and  activities.  Examples  identified  in  the 
earlier  research  included  in  Spain,  where  an  ESF-ERDF  plan  was  developed  to  support  the 
eradication of shanty towns. ERDF funds supported the provision of adequate, and quality non-
segregated  social  housing  for  families  who  live  in  slums  and  in  deprived  homes  while  ESF 
support  for  TO  9  financed  social  integration  activities  targeting  families  that  had  recently 
moved from slums to homes. Coherence of TO 9 operations with other funding than ERDF was 

165 Study supporting the 2020 evaluation of promoting social inclusion, combatting poverty and any 
discrimination by the European Social Fund (Thematic Objective 09) - Publications Office of the EU 

55 

 
 
 
evident in the previous research but to a lesser degree in terms of fewer examples identified. 
Several country examples highlighted the complementarity in the coordinated implementation 
of FEAD in coordination with ESF, including in Italy, a project that aimed to support people 
experiencing homelessness drew on both ESF support to social inclusion and FEAD funding in 
a complementary manner.  

Evidence of synergies and coherence at an implementation level was more varied in respect of 
TO10. In Czechia, under IP10ii the ERDF was used to purchase new equipment, which was 
complemented by the ESF fund to train teachers on the use of the new equipment. In Croatia, 
the IP10.iii case study highlighted that the combination of resources facilitated the “e-schools” 
operations,  which  was  crucial  for  the  transition  to  distance  learning  during  the  COVID-19 
pandemic. 

Evidence to assess synergies between TO 11 operations and other EU Funds was much more 
limited.  A small number of examples were cited by national MA stakeholders through the case 
study research. For example, IP11.ii operations in the ESF OP in French Guyana were aimed 
to support the creation of the ‘Collectivité Territoriale Unique de Guyane,’ in complementarity 
with  ERDF-ESF  OP  –  Guyane  Conseil  Régional  2014-2020,  jointly  tackling  the  need  to 
reinforce the effective professionalisation of public actors. However, MA or public authority 
stakeholders variously reported in the consultations, MA survey or public consultation practical 
difficulties  in  creating  complementarity  in  practice,  notably  due  to  different  administrative 
requirements across funds; in particular, the fact that co-financing opportunities between EU 
funds (e.g. ESF and ERDF), although possible in theory, were in reality not reflected in the 
launch  of  fund-coordinated  announcements  or  calls.  The  sometimes-complex  institutional 
structure within which funds were managed was also cited. 

The Recovery and Resilience Facility, introduced in 2021 as a central instrument of the EU’s 
response to the COVID-19 crisis, played a significant role in the transition between the 2014–
2020 and 2021–2027 programming periods. As concerns the period of this evaluation, evidence 
from  the  case  study  published  alongside  the  RRF  Mid-Term  Evaluation  indicates  that  no 
substantial substitution effects occurred between the RRF and the 2014–2020 Cohesion Policy 
Operational  Programmes.  Only  limited  cases  of  reallocation  or  reprogramming  were 
identified—such as in the Murcia region of Spain—though it remains unclear whether these 
involved ESF resources specifically. By contrast, more  pronounced substitution effects have 
been observed between RRF and ESF+ projects over the period 2021-27, as highlighted in the 
Mid-Term Evaluation of the ESF+166. These effects were confirmed by managing authorities 
and national stakeholders through surveys and interviews, pointing to instances of overlap and 
potential  competition  for  funding  (some  of  the  projects  originally  planned  in  the  2021-27 
Cohesion Policy programmes were shifted under the national Recovery and Resilience Plans).  

166 Staff Working Document available at: EUR-Lex - 52025SC0391 - EN - EUR-Lex 
Executive summary available at: https://eur-lex.europa.eu/legal-
content/AUTO/?uri=CELEX:52025SC0392&qid=1764846889685&rid=2 

56 

 
 
 
In terms of complementarities between ESF and RRF objectives, evidence from the case study 
on the functioning of the RRF and other EU funds167, conducted in the context of the Mid-Term 
Evaluation of the RRF, shows that Recovery Plans enhanced policy areas already covered by 
Cohesion  Policy,  providing  supplementary  financial  resources  and  equally  addressing  the 
challenges and priorities as defined in the Country Specific Recommendations. Both the ESF 
and RRF strengthened social inclusion by financing initiatives that aim to break the cycles of 
poverty  and  social  exclusion:  both  the  ESF  and  RRF  funded  projects  and  investments  to 
improve  access  to  employment,  education  and  training  for  vulnerable  groups.  These  were 
complemented  by  structural  reforms  contained  in  national  Recovery  and  Resilience  Plans. 
Moreover, both funds address the problem of inequalities and focus on boosting employment 
and enhancing skills: the ESF aimed at supporting disadvantaged groups by designing inclusive 
labour markets and lifelong learning opportunities, ensuring the workforce to adapt to evolving 
demands; likewise, the RRF incorporates measures to improve socio-economic development 
across  regions  to  address  territorial  imbalances,  with  the  aim  of  creating  jobs  and 
upskilling/reskilling the labour force, particularly in areas like digital skills and green jobs. By 
focusing on social welfare and health resilience, the ESF and the RRF contribute to building a 
social safety net: both the ESF and the RRF supported the access to efficient social services. 
Furthermore, many national Recovery and Resilience Plans included investments and reforms 
to improve health infrastructure and the social care systems168. Overall, both funds contribute 
to execute the principles of the European Pillar of Social Rights 169. 

4.1.3.3 COMPLEMENTARITY WITH WIDER EU POLICIES170 

Overall, dedicated EU support through the ESF and YEI programme was strongly coherent with 
other EU policies. Throughout consultations with all types of EU and national stakeholders and 
the case study work, stakeholders consistently reported that Member States used key EU-level 
policies  as  the  main  point  of  reference  when  designing  interventions.  Most  commonly 
mentioned EU policies were the Europe 2020 Strategy, the European Pillar of Social Rights 
and the European Skills Agenda.  Other EU level policies or Council Recommendations seldom 
mentioned include the European Disability Strategy, the European Green Paper on Ageing and 
the Council Recommendations on establishing a Youth Guarantee, on validation of non-formal 
and informal learning, and on long-term unemployed in the labour market.   

Member States used the EU-level policies as the main point of reference when designing ESF 
interventions  and  this  was  evident  in  TO  8  case  study  countries  too  (e.g.  Greece,  Slovakia, 
Belgium,  Ireland).  Across  all  TO  9  case  studies,  MA  and  national  ministry  stakeholders 
repeatedly  asserted  that  TO  9  operations  across  the  different  IPs  were  coherent  with  the 
European Pillar of Social Rights principles and Action Plan promoting the need to tackle the 
gender employment gap and increase the provision of formal early childhood and education 
and care services. The link with the Europe 2020 strategy was also visible and highlighted by 

167 https://commission.europa.eu/document/download/c203ce47-c5d4-4fa1-abfc-
50343d9ddcb6_en?filename=case-study-on-the-functioning-of-the-rrf-and-other-eu-funds.pdf 
168 Recovery and Resilience Plans - Reform Support - European Commission 
169 The European Pillar of Social Rights in 20 principles - Employment, Social Affairs and Inclusion 
170   See answer to evaluation question 17 of the supporting study. 

57 

 
 
 
stakeholders in IP9.v operations across all Member States engaged in the case studies (including 
Greece,  Bulgaria,  Poland,  Croatia,  Romania  and  France),  as  their  main  goals  included 
increasing the employability of vulnerable groups. This was supported by the previous mid-
term thematic research on social inclusion where these were the main EU policies mentioned 
in planning documents for TO 9 operations171. 

The key EU policies of the European Pillar of Social Rights and Europe 2020 strategy were 
equally  cited  by  MA  stakeholders  in  the  case  study  research  as  being  aligned  with  TO  10 
operations, but no specific examples were provided through this strand. TO 10 operations were 
funding initiatives in the field of early childhood education and care which aligned with the 
Council  Recommendation  on  High-Quality  Early  Childhood  Education  and  Care  Systems. 
Strong alignment between the New Skills Agenda for Europe and TO 10 was also evident, with 
a  direct  reference  to  ESF  made  in  the  Communication’s  introduction  and  its  actions  clearly 
aligned  with  ESF  TO  10,  including  in  particular,  IP10.iii  (lifelong  learning)  aligning  with 
Action  1  (Upskilling  pathways:  new  opportunities  for  adults)  and  IP10.iv  (labour  market 
relevance) aligning with Action 6 (Vocational Education and Training - VET). Other policies 
the  Council 
cited 
Recommendation  on  Common  values,  inclusive  education  and  the  European  dimension  of 
teaching and the EU Digital Education Plan.  

the  previous  research  with  evidence  of  alignment 

included 

in 

For TO 11, MAs and public authority stakeholders regularly pointed to good coherence among 
TO 11 activities and other EU policies, with operations creating the conditions for guaranteeing 
the fulfilment of the principles set in wider strategies, such as those pursued by the European 
Pillar for Social Rights, and other sectoral policies. 

4.2.  DID ESF AND YEI MAKE A DIFFERENCE AND TO WHOM? 

4.2.1. EU ADDED VALUE 

Like  the  mid-term  evaluation  performed  of  the  2014-2018  ESF  support  to  employment  and 
labour mobility, social inclusion and education and training172, the present evaluation found that 
ESF and YEI continued to bring unique value and produced several beneficial effects that would 
not have taken place in its absence173. 

The added value of ESF and YEI can be reflected in terms of four different effects174: 

171 Study supporting the 2020 evaluation of promoting social inclusion, combatting poverty and any 
discrimination by the European Social Fund (Thematic Objective 09) - Publications Office of the EU 
172 See Brussels 27.1.2021 SWD (2021) 11 final based on three thematic supporting studies for TO8, 9, and 10. 
173 All type of stakeholders, and particularly national stakeholders involved in managing or delivering ESF support 
made  strong  assertions  that  the  effects  identified  would  otherwise  not  have  taken  place  without  ESF  support. 
However, this could not be robustly assessed for each Member States due to the  data  collection approach and 
sampling. 
174 See answer to evaluation question 25 of the supporting study. 

58 

 
 
 
 
 
➢  Volume effects175:  

ESF and YEI made a significant difference in terms of volume effects, allowing Member States 
to reach more individuals and entities that would have otherwise been possible. To start with, 
the total EU budget for ESF and YEI increased compared to the previous programming period 
(from EUR 76.8 billion176 to EUR 93.5 billion), with an additional increase of EUR 19.9 billion 
of REACT-EU funding to the ESF. 

The  ESF  and  YEI  have  enabled  an  increase  in  support  to  a  higher  number  of  participants. 
Beneficiary  organisations  confirmed  the  increased  volume  of  participants  thanks  to  ESF 
support. For instance, in Cyprus, ESF support was crucial in financing interventions for youth 
employment amidst fiscal austerity measures, while in France, ESF funding amplified support 
to job seekers and inactive people. In Greece, the Harmonisation of Family and Professional 
Life operation saw a substantial rise compared to the previous programming period, and in Italy, 
more persons over 54 were supported. Other examples include Portugal, where ESF funding 
led to a 0.9% increase in participants in training or educational activities compared to 2014177. 
These  volume  effects  were  supported  by  the  majority  of  the  Managing  authorities  survey 
respondents,  as  well  as  from  EU  and  national  stakeholders178.  In  addition,  86%  (568)  of 
respondents to the Public consultation stated that more can be done with ESF than with national 
or local resources only.  

In terms of additional instruments, REACT-EU enabled to increase the number of individuals 
supported, for instance in Sweden, where more than twice the initial target of participants was 
reached  for  training  programmes  and  employment  opportunities.  Moreover,  CRII/CRII+ 
operations and CARE’s funding have contributed to a rapid increase in resources,  allocating 
resources that governments could not provide from national sources alone, or not sufficiently 
fast. In Poland, 4,000 people received tangible assistance and material, health or social support. 

As mentioned in Section 3.2, the intensity of the support has been bigger in countries receiving 
Cohesion Fund support countries compared to countries that did not. Overall, the ESF and YEI 
have played a crucial role in scaling up support to those in need, particularly during times of 
crisis, and have demonstrated significant EU added value through their ability to increase the 
volume of support provided. 

➢  Scope effects179:  

The  ESF  and  YEI  operations  broadened  the  actions  to  diverse  target  groups  and  new  or 
underdeveloped policy areas. 

While  ESF  and  YEI  generally  continued  to  serve  the  same  main  target  groups,  the  funds’ 
support  increased  for  hard-to-reach  groups  (see  section  4.1).  Consulted  Managing 
Authorities180 and beneficiary stakeholders confirmed the significant and positive impact of 
the support for groups that were not adequately covered in previous programming periods, or 

175 Volume effects refer to the provision of additional resources and increased in participation. 
176  European  Commission  (2016)  ESF  ex-post  evaluation  synthesis  2007-2013  –  EU  synthesis  report  –  Final 
version, p.11. Available at: https://data.europa.eu/doi/10.2767/1734.   
177 Evidence gathered in the supporting study from consulted Managing Authorities stakeholders and case study 
research. 
178 94% (107 out of 114) Managing Authorities survey respondents reported that the ESF produced volume effects 
to a large or moderate extent. 
179 Scope effects refer to the broadening of actions and target groups. 
180 87% of the  MA survey  respondents (99 out of 114) reported scope effects to a large or moderate  extent in 
relation to target group reach and policy areas. 

59 

 
 
 
were not as adequately covered by national policies and which would not have been possible 
without the ESF and YEI. Additional examples for TO8, TO9 and TO10 include broadening 
the range of lower socio-economic status students accessing scholarships in Croatia, Greece, 
Lithuania or Malta; identifying NEETs in Bulgaria and Latvia as a specific target group, which 
would  not  have  been  offered  support  otherwise;  extending  childcare  services  available  to 
children  with  mild  disabilities  in  Greece,  and  the  provision  of  services  to  old  people  in 
mountainous  or  island  regions,  which  was  not  previously  present.  Support  from  CARE 
enabled  broader  and  more  individually  tailored  measures  to  Ukrainians  fleeing  Russian 
military  aggression.  In  addition,  expansions  of  action  and  beneficiaries  to  more  developed 
regions were evident as a result of the REACT-EU funding available during the periods of 
crisis. 

Additionally, the ESF and YEI showed scope effects through the broadening of policies and 
types of services available. For instance,  in  Lithuania, consulted stakeholders reported that 
IP8.ii-YEI projects introduced new types of support for young people in the country that had 
not  been  available  before  (e.g.,  psychological  or  legal  support,  traineeships,  and 
apprenticeships). The funds also supported areas of intervention where there was insufficient 
national funding for employment and social inclusion (Slovakia) as well as in adult learning 
(Germany)181 ; focused on areas that are not on the national policy agenda, and addressed more 
long-lasting  objectives  than  national  programmes,  as  well  as  relying  on  broad  stakeholder 
engagement (Sweden).  

Lastly, ESF and YEI enabled activities at larger scale and in several new areas, including for 
Ukrainians fleeing Russia’s military aggression (e.g. Finland).  

➢  Role effects182:  

One area where ESF has added value to national and regional policies is the sustained focus 
on  the  implementation  of  systematic  and  innovative  actions.  More  specifically,  the 
implementation of ESF and YEI generated role effects linked to supporting innovation and the 
transfer of ideas and enabled changes in national systems/ structures in some instances.  

As  per  the  ESF  Regulation,  social  innovation  is  relevant  to  all  the  TOs.  The  Regulation 
highlights that “the ESF shall promote social innovation within all areas falling under its scope, 
in particular with the aim of testing, evaluating and scaling up innovative solutions, including 
at the local or regional level, in order to address social needs.” The Regulation then specifies 
that “ESF should encourage and support innovative social enterprises and entrepreneurs as 
well  as  innovative  projects  taken  on  by  non-governmental  organisations  and  other  actors 
within the social economy. In particular, testing and evaluating innovative solutions before 

181 A recent study analysed the effectiveness, coherence and efficiency of the ESF supported voucher and grant 
schemes in different Operational Programmes in Germany for the period 2014-2020, which aimed at supporting 
non-formal job-related adult learning and contributing to upskilling and reskilling.  
The  study  highlighted  that  the  instruments  introduced  help  to  fill  gaps  left  by  the  main  instruments  of  public 
funding used within the German adult learning system. Concretely, ESF-co-funded instruments for adult learning 
became particularly important in areas where it was more difficult to adopt formal laws due to the complex split 
of responsibilities for funding adult learning between the German Länder and the federal government. 
Further information is available in the study: “Study on the effectiveness, coherence and efficiency of the ESF 
supported  voucher  schemes  in  different  operational  programmes  (funding  period  2014-2020)  -  European 
Commission” 
182 Role effects refer to innovative actions or mainstream of activities. 

60 

 
 
 
scaling  them  up  is  instrumental  in  improving  the  efficiency  of  policies  and  thus  justifies 
specific support from the ESF.” 

In terms of programming of social innovation, Article 11 of the ESF Regulation introduces 
the  opportunity  to  identify  a  specific  priority  axis  dedicated  to  the  development  of  social 
innovations in the Operational Programmes. It also allows the maximum co-financing rate for 
a  priority  axis  wholly  dedicated  to  social  innovation  or  to  transnational  cooperation,  or  a 
combination of both, to be increased by ten percentage points (but not exceeding 100%).  

Successful  examples  were present  in  Italy,  where ESF funding offered opportunities to  try 
various social innovation approaches and subsequently replicate them at the national, regional, 
or local level. Impact and implementation evaluations also showed that TO10 interventions 
introduced new approaches and enabled the adoption of new methods and practices, especially 
in  relation  to  teaching.  For  instance,  in  Portugal,  the  ESF  enabled  more  flexible  training 
models  in  line  with  the  needs  of  the  labour  market  and  regional  characteristics,  while  in 
Belgium-Flanders,  ESF  supported  operations  enabling  the  testing  of  an  innovative 
community-building approach and a network of partners. 

The evidence is, however, not even across TOs and IPs. TO8 operations presented fewer role 
effects compared to other TOs. Some successful cases in this TO included companies that have 
gained greater innovative capability and competitiveness through the implementation of new 
technologies  (e.g.,  Germany)183;  mainstreaming  horizontal  principles  such  as  equal 
opportunities  and  sustainable  development  (IP8.ii-YEI  in  Spain),  or  allowing  cross-
fertilisation  of  ideas.  From  stakeholder  consultations  and  case  study  research  for  the 
supporting study, role effects were reported by 58% (69 out of 114) of the respondents to the 
MA survey, and 41% of respondents in the public consultation agreed that the ESF enables 
experimenting with new ways of delivering services (n=269). 

The presence of role effects was also evidence by impacts on the national and regional systems 
and  structures.  This  has  been  achieved  through  decentralization  and  community-led 
development, where ESF/YEI have led to changes in the responsibilities of municipalities and 
NGOs, engaging local communities in addressing social exclusion, as seen in (Bulgaria) where 
municipalities  were  able  to  provide  targeted  social  services  to  address  local  needs. 
Additionally,  ESF  support  has  strengthened  local  institutions  and  social  partners,  enabling 
them to diversify their activities, such as in Croatia where financing focused on thematic areas 
like social development  and strengthening democratic political  culture, which had received 
less  funding  from  national  and  local  sources.  Managing  authorities  and  beneficiaries 
interviewed supported that the ESF has also promoted policy shifts and innovative measures, 
such as in Latvia where ESF interventions contributed to the de-institutionalization of long-
term care, and in Greece where the ESF supported the development of a National Strategy for 
Accessibility to Higher Education. Lastly, it was noted that the YEI led to a re-orientation of 
national policies in France in line with priorities established at the European level.  

➢  Process effects184:  

The  ESF  and  YEI  have  also  had  a  range  of  positive  process  effects  on  the  stakeholders, 
organisations  and  authorities  involved.  61%  of  MA  survey  respondents  (69  out  of  114) 
reported  process  effects  that  included  enabling  collaboration  among  stakeholders  towards 
long-term objectives, increasing the capacity of government administrations and participating 

183 Reported by consulted MA and public authority stakeholders 
184 Process effects refer to the improvement of systems and structures in place. 

61 

 
 
 
beneficiaries, and enhancing the efficiency of their processes. National stakeholders of all type 
interviewed  for  the  supporting  study  also  expressed  the  view  that  ESF  management  and 
implementation has been a cumulative learning process, which led to new capabilities, benefits 
and increased professionalisation. 

National evaluations also show that ESF allowed MA and implementing bodies to gain new 
competences and experience for successive programming periods. Under the TO8, operations 
allowed to create new networks for the YEI implementation (e.g. Sweden), as well as increase 
capacity building for beneficiary organisations and the development of new partnerships (e.g. 
Belgium, IP8.ii). New tools developed for ESF operations have been subsequently used by 
public authorities for services funded by other sources (e.g. Lithuania, IP8.i). Under TO9, ESF 
and YEI allowed managing and implementing bodies to adopt new approaches and practices 
that could be successfully replicated in the next programming periods. The active involvement 
of public institutions and the creation of effective networks with relevant stakeholders (e.g., 
municipalities,  trainings  centres,  NGOs,  social  centres,  etc.)  has  also  been  highlighted  by 
different  evaluations  (e.g.,  Germany,  Poland,  Sweden).  Under  TO10,  process  effects  were 
visible through increased operational capacity of beneficiary organisations to navigate ESF 
requirements  (Hungary,  IP10.ii)  or  improved  capacity  of  the  municipalities  to  support 
schoolwork  while  integrating  transparency  and  accountability  into  regulations  (Portugal, 
IP10.i). Additionally, under TO11, the ESF and YEI contributed to the development of a new 
regulatory framework for the functioning and further training of the civil service (Slovakia, 
IP11.i)  and  knowledge  sharing  and  institutional  cooperations,  increasing  social  dialogue 
(Poland, IP11.ii).  

The ESF and YEI led to sustainable changes by increasing the capacity and cooperation of 
government  administrations and stakeholders. These programmes have enabled beneficiary 
organisations, especially NGOs, to continue or expand their activities and enhance their ability 
to meet ESF requirements. In addition, various ESF-co-funded vouchers and grants under the 
2014- 2020 funding period showed good sustainability selected for continuation in the ESF+ 
2021-2027  period185.  However,  stakeholders  commonly  reported  the  concern  about  over-
reliance  on  EU  funding,  which  could  create  future  funding  gaps  if  national  funding  is  not 
maintained. A number of factors are likely to play a role and mediate sustainable outcomes, 
such as national and local policy environments and available funding, further investments, the 
ownership over current practices and outcomes, as well as specific factors linked to each area 
of intervention. 

4.2.2. VISIBILITY 

This section assesses to what extent operations funded by the ESF and YEI were visible to 
relevant  stakeholders,  beneficiaries  and  the  wider  public186.  On  the  basis  of  the  evidence 
gathered,  overall  good  levels  of  visibility  were  observed,  with  differences  between  TOs, 
Member States and operations, and less visibility towards the general public. 

Throughout the implementation of the ESF/YEI programmes, diverse activities were carried 
out to promote the visibility and awareness of the EU’s role in the support provided towards 

185 See “Study on the effectiveness, coherence and efficiency of the ESF supported voucher schemes in different 
operational programmes (funding period 2014-2020) - European Commission” 
186 See evaluation question 30 of the supporting study. 

62 

 
 
 
 
 
beneficiaries and the general public. The most common visibility actions implemented include 
the set-up of a dedicated website about the project and available support, the clear display of 
the EU logo, and specific communication campaigns and events.  

Examples  included  Poland,  where  support  to  newly  arrived  Ukrainians  fleeing  Russia’s 
military  aggression  was  in  both  Polish  and  Ukrainian;  or  Spain,  where,  to  promote  YEI 
visibility, they used indicators to measure the visibility of the supported projects. Up to 2020, 
in total 1 389 events were organised with overall more than 200 000 participants. 

Overall,  as  per  the  supporting  study,  the  activities  carried  out  were  effective  in  achieving 
reasonable  levels  of  visibility  of  the  EU’s  involvement  in  the  projects  supported  by  the 
ESF/YEI,  in  particular  for  groups  which  were  more  closely  involved  in  the  management, 
implementation  or  coordination  of  the  projects.  In  the  survey  of  ESF/YEI  managing 
authorities, respondents reported that visibility has increased in comparison to the previous 
programming  period,  and  some  suggested  that  the  emphasis  on  visibility  activities  was 
strengthened.  

However, evidence as regards the visibility of the ESF/YEI towards the general public is less 
robust, and where it is available, it suggests a more mixed picture, indicating differing levels 
of visibility across Member States. Moreover, differences in  visibility also  emerged across 
TOs. For instance, while evidence from stakeholders and case studies suggests that there were 
overall good levels of visibility of ESF/YEI projects supported under TO 8, TO 9 and TO 10, 
this  was  less  the  case  for  operations  under  TO  11  and  TO  13  (REACT-EU).  For  TO  11, 
visibility  depended  on  the  nature  of  the  supported  operations,  with  notable  differences  in 
visibility  for  projects  aimed  at  public  officials,  which  were  often  promoted  via  internal 
channels,  and those targeting public services for citizens which  were more visible. Results 
from  the  public  consultation  also  suggest  a  low  visibility  of  TO  13,  with  few  respondents 
providing any replies, indicating little knowledge about REACT-EU both among the general 
public  and  organisations  involved  in  ESF  programmes.  Evidence  from  case  studies  also 
suggests that there were limited activities and media coverage specifically covering support 
from  REACT-EU.  In  general,  challenges  related  to  the  COVID-19  pandemic  in  terms  of 
ensuring  visibility  were  also  identified  across  the  case  studies,  as  many  communication 
activities could not be implemented as planned. 

The factors which were considered most successful by stakeholders of all types in promoting 
visibility  of  the  projects  supported  by  ESF/YEI  are  continuous  communication,  including 
across programming  periods, having  dedicated websites in  place that are of high  quality,  a 
clear  branding  and  use  of  the  EU  logo,  using  social  media,  cooperation  between  different 
actors  and  community  outreach.  In  this  context,  a  combination  of  different  activities  and 
channels emerged as a key element facilitating visibility. 

Results  from  the  public  consultation  suggest  that  the  multiplier  effect  was  also  relevant  in 
raising awareness of the available support: ESF beneficiaries that replied to the consultation 
indicated that they learned about the support through several additional channels, including 
their  employer,  education  and  training  institutions  or  through  employment  services  or 
information centres187.  

Particularly good results in terms of enhancing visibility and awareness of the ESF, including 
towards  the  general  public,  were  reached  with  the  bottom-up,  community-led  local 

187 However, the sample of respondents to the public  consultation who received ESF support is small, with 47 

replies received. 

63 

 
 
 
development approach of IP9vi, as shown in the case study conducted on TO 9 (see Supporting 
Study  Annexes).  This  approach  facilitated  access  to  ESF  support  and  empowered  local 
communities to decide how ESF support was allocated, thereby strengthening visibility too. 

Nonetheless, communication and visibility were often mentioned by stakeholders as a key area 
where further improvements could be made. Evidence from a review of national evaluations 
of ESF and YEI programmes in several Member States and related to different TOs suggest 
that  there  was  a  need  to  strengthen  the  dissemination  of  different  approaches  and  good 
practices of visibility activities as well as the dissemination of achieved results. 

4.3.  ARE THE ESF AND YEI STILL RELEVANT? 

To be able to assess whether the ESF and YEI were relevant to the needs of their target groups, 
this section examines the relationship between the objectives of the two funds and the needs 
and challenges which they intended to alleviate, including how well they were addressed and 
how they potentially evolved over the implementation period. It also looks into the continued 
relevance of the ESF/YEI in view of emerging needs and future trends. 

4.3.1 RELEVANCE OF ESF AND YEI188 

This evaluation found that ESF and YEI exhibited an overall good relevance to the needs of 
the supported targeted groups, namely unemployed people, inactive people, (young) NEET 
and disadvantaged groups, albeit with more mixed views and nuanced findings when it comes 
to specific targets groups, while there is less evidence as regards the relevance to the target 
groups’ needs of the scope and intensity of support provided. 

Evidence from the evaluation (MA survey, public consultation, case studies and participation 
data)  suggests  that  the  objectives  of  and  operations  supported  by  ESF/YEI  were  largely 
relevant in addressing the needs of the target groups concerned and that target groups were 
appropriately  defined.  In  particular,  data  from  the  participations  in  ESF/YEI-supported 
operations indicate  that the provided support was well aligned with  the needs of the target 
groups  as  presented  in  the  programmes,  and  that  the  ESF/YEI  were  able  to  reach  a 
considerable  share  of  people  from  disadvantaged  backgrounds  (see  section  4.1).  When 
considering the annual coverage rates, unemployed people were the most covered by ESF/YEI 
operations, with on average 13.3% of all unemployed people in the EU (incl. the UK) having 
received support from the two funds and reaching as many as 40% of unemployed people in 
Hungary or 33.6% in Belgium.  

In addition, the evaluation (analysis performed in the context of the supporting study) also 
indicated  that  the  link  established  in  the  programming  period  with  the  CSRs  addressed  to 
Member States in the context of the European Semester has helped enhancing the focus of the 
measures  on  addressing  the  main  labour  market  and  social  issues,  thereby  contributing  to 
increasing the relevance of ESF/YEI support (see also section 4.1).  

However, more mixed views as regards the relevance of the ESF/YEI to the needs of specific 
target groups were expressed by national and regional stakeholders as well as representatives 
from civil society organisations. Such stakeholders were of the view that relevance was stronger 
for broader target groups (e.g., unemployed), whereas accounting for the needs of more specific 
target  groups  such  as  the  Roma,  refugees  and  migrants,  persons  with  disabilities  or  older 

188 See answers to evaluation questions 19 and 20 of the supporting study. 

64 

 
 
 
 
workers as well as persons with intersecting vulnerabilities was considered more difficult. This 
was  also  recognised  by  respondents  to  the  MA  survey,  where  only  27%  (of  in  total  114 
respondents)  considered  that the Roma were adequately  prioritised, whereas the majority of 
respondents  did  state  that  this  was  the  case  for  the  unemployed  (87%)  and  the  long-term 
unemployed  (82%).  In  addition,  evidence  from  the  case  studies  also  pointed  to  a  risk  of 
‘creaming’ effects, i.e., the risk of prioritising individuals with a higher likelihood of achieving 
employment to the detriment of those who are furthest from the labour market. 

Factors enabling relevance 

Evidence  from  the  MA  survey  and  case  studies  suggests  that  the  relevance  of  supported 
interventions overall increased both as compared to the previous programming period as well 
as  within  the  2014-2020  period.  This  can  be  attributed  to  several  factors;  for  instance, 
stakeholders consulted in the case studies stated that a better knowledge of the needs of target 
groups had been developed based on the experience of the previous programming period which 
has helped increase relevance. Other factors that emerged from the case studies as contributing 
to improving relevance of ESF/YEI-supported interventions relate to regular and continuous 
monitoring of implementation, and the application of the partnership principle and multi-level 
governance. Evidence from various case studies further indicates that involving stakeholders 
from the design phase of operational programmes as well as during their implementation was a 
crucial aspect for the relevance of ESF/YEI operations. In particular, engaging CSOs and NGOs 
that  have  direct  knowledge  of  or  experience  with  specific  target  groups  in  the  design  and 
implementation  of  interventions  contributed  to  strengthening  the  relevance  of  the  ESF/YEI. 
Nonetheless, while overall the application of the partnership principle has been strengthened as 
compared to the previous programming period (see also section 4.1), the case studies also point 
towards a lower involvement of stakeholders in some Member States, in particular those with 
more centralised programme management or with less of a tradition of working with partners.   

Relevance across TOs 

Respondents  to  the  MA  survey  also  indicated  differences  as  regards  the  relevance  of  ESF 
interventions to the needs of supported target groups across TOs. Whereas around two thirds of 
respondentsconsidered  that  interventions  implemented  under  TO  8  (65%  of  in  total  114 
respondents), TO 9 (62%) and TO 10 (63%) were to a large extent relevant to the needs of the 
defined target groups, this was less the case for TO 11 (only 19% of respondents to the MA 
survey considered supported interventions as relevant to a large extent) and YEI (only 31%). 
For the latter two, the majority of respondents indicated that they ‘do not know’, which might 
point  towards  an  overall  lower  knowledge  of  operations  under  TO  11  and  YEI189.  The  case 
studies conducted on the different TOs provide a number of examples and insights as regards 
commonalities and differences in ensuring relevance to the target groups’ needs. 

Evidence from the case study on TO 8 indicates that a number of Member States were able to 
rely on experience from previous programming periods to ensure increased relevance of ESF 
operations.  For  instance,  in  Denmark,  specific  elements  from  projects  implemented  in  the 
previous period that were considered as success factors were replicated in similar projects for 

189 TO 13 is not covered as respondents to the MA survey were not specifically asked about their views as regards 
the relevance of TO 13 in the relevant survey question. 

65 

 
 
 
the 2014-2020 period. The case study on the YEI underlines the importance of involving and 
cooperating with CSOs to ensure the relevance of supported operations. For instance, in Ireland, 
cooperation  with  relevant  CSOs  with  a  youth  focus  has  helped  adopt  a  holistic  approach  to 
respond to the specific needs of the target group (early school leavers), considering also mental 
health  support.  Nonetheless,  interviews  conducted  with  stakeholders  from  Cyprus  and 
Lithuania  also  suggest  some  difficulties  in  reaching  the  most  vulnerable  NEET  and  in 
accounting  for  intersectional  vulnerabilities  of  young  NEET  (e.g.,  young  people  with 
disabilities, rural youth). Furthermore, there is also evidence from the case study indicating that 
the  focus  and  definition  of  YEI  operations  did  not  allow  to  address  the  needs  of  young, 
employed people in terms of e.g., training or upskilling activities. 

Stakeholders  interviewed  in  the  context  of  the  case  studies  covering  operations  under  TO  9 
overall considered that targeted outreach and specifically tailoring interventions to marginalised 
communities  and  potential  beneficiaries  at  risk  of  social  exclusion  has  helped  ensure  the 
relevance of ESF operations to responding to the needs and difficulties these groups encounter 
when it comes to access to employment, essential and social services. For instance, in some 
Member States, an approach was adopted where other relevant actors in close contact with the 
intended  target  groups  were  reached  to  raise  awareness  and  ultimately  facilitate  their 
participation  (e.g.,  in  Italy,  care-givers  were  engaged;  in  Lithuania,  social  enterprises 
employing  people  with  disabilities  were  targeted).  Nevertheless,  as  also  mentioned  above, 
respondents to the MA survey largely perceived that the needs of the Roma were not adequately 
addressed.  Participants  to  the  focus  group  on  TO  9  referred  to  the  issue  of  persisting  social 
stigma which can further hamper reaching the most vulnerable target groups.  

Examples from the case study of TO 10 highlight the importance of systematically assessing 
and considering the needs and specific circumstances of the target groups. For instance, under 
IP10.i, Spanish authorities systematically analysed the reasons for early school leaving (e.g., 
income needs, lack of family support), on the basis of which they were able to design drop-out 
prevention measures accounting for these underlying reasons that were potentially preventing 
target groups from participating in ESF measures.  

As mentioned above, most respondents to the MA survey indicated that they were not able to 
assess the relevance of interventions implemented under TO 11, likely linked to a more limited 
knowledge of this TO and considering its narrower scope. Nonetheless, evidence from the case 
study  points  towards  the  importance  of  involving  relevant  stakeholders  in  the  design  and 
implementation  of  interventions  to  ensure  that  the  nature  and  content  of  the  supported 
operations (e.g., trainings, digitalisation of processes) is relevant and timely.  
In the future, the positive results of ESF and YEI support across all TOs190, both on micro and 
macro levels (see section 4.1), are likely to continue after the end of EU funding. In particular, 
88% of MA survey respondents (100 out of 114) suggested that results for participants are likely 
to be sustained. Notably, activities and operations that continue under the ESF+ or are integrated 
into national or regional initiatives contribute to the long-lasting effect of results and moreover 
indicate  the  continued  relevance  of  the  support  provided  to  the  target  groups.  For  instance, 
services  have  continued  after  2022  in  Italy  (childcare  services  vouchers),  Malta  (Work 
Programme Initiative), Finland (one-stop-shops to help young people integrate into the labour 

190  As  reported  by  national  stakeholders  of  all  types,  including  Managing  Authorities,  public  authorities  and 
beneficiary stakeholders; and variously reported in national evaluations. 

66 

 
 
 
market now embedded in the national system) or Lithuania, where operations financed by IP10ii 
have now been taken over by state financing191. 

Continued relevance in changing circumstances 

As suggested in previous sections, due to the flexibility enabled in the implementation of ESF 
programmes  during  the  COVID-19  pandemic  and  the  crisis  of  Ukrainians  fleeing  Russia’s  
military aggression with the introduction of the CRII/CRII+, REACT-EU and CARE initiatives, 
the ESF has been able to ensure continued relevance of its objectives and support in line with 
evolving  needs  and  circumstances.  In  this  regard,  the  case  study  of  TO  13  (REACT-EU) 
suggests that supported operations were highly relevant in addressing newly emerged needs, 
such as regarding remote education or to protect jobs (e.g., support to short-time work schemes 
or  to  the  adaptation  of  workers  and  enterprises  to  the  changed  circumstances).  This  is  also 
confirmed by the REACT-EU evaluation192. 

In addition to the flexibility provided with the above-mentioned initiatives, Member States were 
also able to introduce formal amendments to their programmes following regular procedures 
foreseen in the set-up of the funds, to reflect changed circumstances and priorities and thereby 
contribute  to  ensuring  continued  relevance.  For  instance,  in  Bavaria  (Germany),  operations 
were amended to reduce the planned focus on supporting the long-term unemployed given that 
their needs were not as high as expected. 

Relevance achieved through contributions to horizontal principles193, the EU 2020 strategy and 
the European Pillar of Social Rights 

All  Member  States  addressed  the  horizontal  principles  of  gender  equality  and  non-
discrimination in their OPs, thereby also contributing to enhancing the relevance of the support 
operations  in  view  of  their  target  groups.  However,  only  a  subset  provided  for  specific 
allocations to these themes including Czechia, Germany, Poland and the Netherlands194. 93% 
of MA respondents  reported that the principle of gender equality, equal opportunities and non-
discrimination195 was applied to a large or moderate extent, representing the strongest response 
on  horizontal  principles.  Stakeholders  of  all  types  underlined  the  positive  improvement, 
compared  to  the  past,  in  the  commitment  and  capacity  to  concretely  address  the  non-
discrimination  principle.  Mainstreaming  gender  equality,  equal  opportunities  and  non-
discrimination principles took place through systemic measures such as knowledge-sharing and 
awareness-raising, tackling social stereotypes, supporting stakeholders’ capacity-building and 

191 For more information, see the supporting study Evaluation Question 28. 
192 Report from the Commission to the European Parliament and the Council on the evaluation of REACT-EU. 
Published 10 October 2025. Available at: 
COM_2025_634_F1_REPORT_FROM_COMMISSION_EN_V4_P1_4084848.pdf  
193  The  horizontal  principles  pertain  to  the  promotion  of  gender  equality  and  non-discrimination,  sustainable 
development as well as the involvement of partners and were to be reflected in operational programmes and by 
appropriate amounts of ESF funding. See Chapter II, articles 6, 7, 8 of the ESF Regulation. 
194 Specifically, according to the ESF categorisation data, only 17 Member States reported allocating around EUR 
4.46 billion on 25 898 operations related to the gender equality secondary theme, see Case study report on Gender 
Equality. 
195 In accordance with Article 7 of the ESIF Common Provisions Regulation, (EU) No 1303/2013 (Available at: 
https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32013R1303 and in Articles 7 and 8 of the ESF 
regulation, 
https://eur-lex.europa.eu/legal-
(EU) 
content/EN/ALL/?uri=celex:32013R1304 

1304/2013, 

Available 

No 

at: 

67 

 
 
 
involving NGOs and associations with experience and knowledge of working with vulnerable 
groups, the latter identified as a key factor in strengthening the relevance of provided support 
(see  above).  Still,  challenges  persist  such  as  more  limited  knowledge,  experience  and 
institutional  capacity;  difficulties  in  actively  involving  NGOs  and  representatives  of 
discriminated groups in programme design and implementation; as well as limited effectiveness 
of implemented actions. Also, there is a minority view that there were gaps between formal 
statements and actual implementation.    
The implementation of the partnership and multi-level governance principle196 has improved 
compared to the 2007-2013 programming period and contributed to strengthening relevance197. 
Member States and Managing Authorities adopted a variety of arrangements and operational 
mechanisms to ensure effective involvement of partners in different stages of the programming 
cycle. As noted above and underlined in the relevant case study, this has been a key factor in 
enhancing the relevance of supported operations to the target groups’ needs, by engaging the 
knowledge  and  networks  of  local  actors  and  authorities.  However,  effective  and  efficient 
implementation of the partnership principle depended strongly on the capacity of both partners 
and Managing Authorities. At the same time, the application of the principle was also found to 
contribute to supporting the capacity building of both public administrations and other involved 
stakeholders.  The  COVID-19  pandemic  had  negative  implications  on  this  principle  due  to 
meeting restrictions and fast reallocation needs, however, in some Member States the creation 
of  new  working  groups  and  task  forces  strengthened  cooperation  and  partnership  processes 
became  more  digital198,  favouring  the  involvement  of  a  larger  number  of  organisations  and 
stakeholders, although sometimes at a more superficial and less effective level. 

Interviewed  national  and  regional  level  stakeholders  of  all  types  stated  that  the  principle  of 
sustainable  development  was  taken  on  board  well  in  ESF  operations.  However,  case  study 
evidence was more limited and less consistent than for the other two principles and according 
to  the  survey  among  Managing  Authorities,  this  principle  was  less  applied  in  ESF 
implementation than for the other horizontal principles. Still, research conducted in the context 
of  the  supporting  study  indicated  the  importance  for  sustainable  development  of  involving 
relevant stakeholders as well as target groups early on, as of the design phase of the operations. 

Some stakeholders also expressed dissatisfaction with the degree to which horizontal principles 
had been used to inform the design and implementation of ESF  and YEI  operations and the 
variety of implementing and monitoring practices across countries. Furthermore, stakeholders 
generally felt that the need to rapidly respond to crises took precedence over more considered 
application of the horizontal principles.  

196 The legal basis for the 2014-2020 programming period lays in article 5 of the Reg(EU)1303/2013 , which refers 
both  to  the  Partnership  Agreement  and  Programme  levels.  Available  at:  https://eur-lex.europa.eu/legal-
content/EN/ALL/?uri=celex:32013R1304 
197 For more information, see the supporting study Evaluation Question 21. 
198 European Committee of the Regions, Holstein, F., Böhme, K., Münch, A. et al., Application of the principles 
of partnership and multi-level governance in Cohesion Policy programming 2021-2027, European Committee of 
the Regions, 2021, available at: https://data.europa.eu/doi/10.2863/602544 

68 

 
 
 
The evaluation also found that some ESF and YEI operations were relevant in promoting social 
innovation. The relevance of social innovation operations under the ESF varied widely between 
countries and projects (as per the evidence gathered in the case study on social innovation), 
depending on the existing social innovation ecosystem and the capacity to involve the key social 
innovation actors. Social innovation was reflected in a number of Member States, with a total 
of 78 operational programmes in 22 Member States allocating finances in a specific priority 
axis  or  focussing  on  social  innovation  as  a  secondary  theme.  However,  challenges  existed 
relating to finding a common way to approach the concept and to address its implementation 
through  operations.  With  respect  to  the  ESF’s  regulatory  and  administrative  framework, 
stakeholders  felt  that  it  needed  further  tailoring  and  simplification  to  better  support  the 
implementation of social innovation interventions. 

Finally, the collected evidence also points to a contribution to the Europe 2020 employment 
rate target as well as to the first chapter of the European Pillar of Social Rights. This is also 
mirrored  in  the  MA  survey,  where  79%  (90  out  of  114)  indicated  a  large  or  moderate 
contribution to the employment rate target and 87% (99 out of 114) feeling that the ESF and 
YEI had contributed to a large or moderate extent to the first chapter of the European Pillar of 
Social Rights. Stakeholder views corroborated these aspects but pointed to the need to support 
more disadvantaged and marginalised groups. Concerning social inclusion, 52% (59 out of 114) 
of the MA survey respondents felt that the ESF/YEI contributed to a large or moderate extent 
to the Europe 2020 poverty alleviation target and 59% (67 out of 114) felt that the ESF/YEI 
had  contributed  to  a  large  or  moderate  extent  to  the  third  chapter  of  the  EPSR.  Relating 
education and training, stakeholders felt that the ESF/YEI contributed to both the Europe 2020 
targets in this domain199. Therefore, the ESF has made a substantial contribution to expanding 
and improving education, training and vocational training. 

4.3.2 FUTURE TRENDS200 

Beyond the relevance of ESF/YEI interventions  during the evaluation  period,  ‘mega’-trends 
such as the digital transformation, demographic ageing, environmental and climate challenges 
and migration have profound consequences for the future relevance of ESF/YEI-type funding, 
triggering new and deepened labour market and social challenges across Member States, but 
also presenting new opportunities to be exploited.  

Notably,  the  EU  is  faced  with  urgent  and  increasing  labour  and  skills  shortages  in  various 
sectors and occupations, for both low-skilled and high-skilled workers201, heightening the need 
to  upskill  and  reskill  the  workforce  to  meet  the  evolving  demands  of  the  labour  market. 
Moreover, the digital transition and increasing use of artificial intelligence tools are markedly 
impacting  on  employment  and  working  conditions  across  all  sectors,  requiring  that  workers 
develop new and updated skills and competences, while they can exacerbate already existing 

199 Reducing early school leavers to 10% and increasing the share of the population aged 30-34 having completed 
tertiary education to 40%.  
200 See answer to evaluation question 24 of the supporting study. 
201 See also Draghi’s Report on Competitiveness, available here: https://commission.europa.eu/topics/eu-
competitiveness/draghi-report_en 

69 

 
 
 
 
socio-economic inequalities and disparities in terms of access to the labour market, to training 
and to essential services, e.g., of rural and less developed areas. Similarly, climate change and 
the green transition call for equipping the workforce with the right skills and competences to 
foster the development and deployment of clean technologies. At the same time, an inclusive 
approach in  this regard is needed, to  avoid  exacerbating existing  inequalities across groups, 
sectors and regions. On the other hand, demographic change in terms of a declining working 
age population will further increase current labour shortages, social and territorial disparities 
and pressure on the EU social and welfare model, whereas migration and attracting skills from 
abroad is often considered as a way of countering labour shortages. In this context, the social 
inclusion of third-country nationals also remains a key policy concern. Finally, some population 
groups  are  at  an  increasing  risk  of  falling  into  poverty  and  social  exclusion,  such  as  low-
educated persons, NEETs, or people living in rural and remote areas, and progress in reducing 
poverty rates across the  EU and in  particular those of  children at  risk of  poverty and social 
exclusion has stalled. 

Therefore, in view of these new and persisting policy challenges, the relevance of funding in 
policy areas that were covered by ESF/YEI can be expected to remain high and even increase 
in the future considering the different mega-trends described above that already have significant 
and transformative impacts on EU labour markets, skills needs and socio-economic inequalities 
as well as on the overall competitiveness and social model of the Union. 

5.  WHAT ARE THE CONCLUSIONS AND LESSONS LEARNED? 

In  this  section,  we  set  out  the  conclusions  and  lessons  learned  deriving  from  the  evidence 
gathered and reflected in previous sections. Lessons learned are presented in boxes after each 
set of conclusions. 

5.1 EFFECTIVENESS 

Based  on  the  available  evidence,  ESF  and  YEI  operated  effectively  during  the  2014-2022 
evaluation  period,  achieving  their  objectives  and  contributing  to  wider  EU  policies,  while 
successfully  reaching  the  intended  target  groups.  To  a  lesser  extent,  they  assisted  Member 
States in tackling the challenges raised in Country Specific Recommendations and in achieving 
changes at a macro or more structural level. With the flexibilities introduced via CRII/CRII+, 
CARE  and  REACT-EU,  they  effectively  allowed  Member  States  to  quickly  adapt  to  the 
emerging  needs  due  to  the  COVID-19  pandemic  and  Russia’s  war  of  aggression  against 
Ukraine.  

Achievement of operational objectives 

Operations reached a total of 64.6 million participations. Immediate common result indicators 
reveal  that  35%  of  ESF  participants  at  the  end  of  2022  had  achieved  a  positive  result  upon 
leaving ESF and YEI funded operations, including supporting 7.4 million participants to find a 
job and 10.3 million gaining a qualification. In addition, 1.5 million were engaged in job search 
and 3.0 million were in education/training after the intervention. 6.9 million participants with 
positive results are disadvantaged participants. Focussing on participants entering employment 
and  gaining  a  qualification,  which  can  be  directly  compared  to  the  preceding  programming 
period, the ESF has been more effective, reaching higher shares of these common results. 

70 

 
 
 
The evidence points at a good effectiveness of ESF and YEI interventions in reaching the target 
groups. Compared to the previous programming period, the shares of inactive and unemployed 
participants among all participants have been increased, as has the ESF’s reach among migrants 
and persons from minority or marginalised communities. In addition, the shares of those facing 
social exclusion closely mirror prevalence within the EU population and the shares of Roma 
people as well as of those with the lowest education levels are over-represented in the ESF data 
relative to their population. However, there is room for improvement. In particular, challenges 
persist in reaching the most disadvantaged groups (such as the most vulnerable or marginalised 
within the larger NEET group and people that are homeless or affected by housing exclusion) 
e.g. due to their disengagement from public services and society more generally or limitations 
in  terms  of  resources,  capacity  and/or  expertise  of  delivery  organisation.  Effectiveness  in 
reaching persons with disabilities is also still limited.  

Regarding thematic objectives, the ESF made a strong contribution in the employment policy 
area. Stakeholders reported that it contributed to reaching the Europe 2020 employment rate 
target as well as implementing the first chapter of the European Pillar of Social  Rights. The 
policy  area  of  education  and  training  also  benefitted  from  a  strong  ESF  contribution,  and 
stakeholders reported that ESF/YEI contributed to both the Europe 2020 targets in this domain. 
The ESF also contributed positively to the social inclusion and poverty alleviation targets of 
the Europe 2020 strategy as well as to the third chapter of the European Pillar of Social Rights. 
Despite the complexities of addressing social inclusion and poverty challenges, notable efforts 
have been made to reach disadvantaged groups, laying the groundwork for continued progress 
and improvement in this critical area. 

Changes resulting from ESF and YEI support 

ESF and YEI programmes were overall effective in achieving changes at the individual level 
such  as  contributions  to  increased  employment  opportunities,  decreased  unemployment,  the 
promotion  of  social  inclusion  as  well  as  successful  upskilling  of  workers  and  enhanced 
education  provision  for  young  people.  On  average,  evidence  available  from  110  national 
counterfactual impact evaluations shows that support from ESF and YEI made a difference by 
increasing participants' chances to find employment by nearly 6 percentage points compared to 
non-participants.  It  is  estimated  that,  on  average,  the  employment  rates  for  participants 
increased from 40.2% to 46.1% corresponding to a 15% increase. Importantly, these numbers 
should  be  read  having  in  mind  that  ESF  mostly  targets  individuals  who  are  relatively  more 
difficult  to  employ.  Particularly  effective  in  increasing  the  probability  of  finding  a  job  or 
improving  participants’  job  positions  were  operations  on  education  and  vocational  training, 
work-based  learning  and  financial  incentives  as  well  as  traineeships,  childcare  services  and 
entrepreneurship  courses.  Similarly,  the  development  of  “one-stop-shops”  for  access  to 
multiple services and operations providing individually tailored support and those adopting a 
holistic approach stand out positively.  

Contributions to changes on a more macroeconomic or structural level have to be looked at 
taking the scope and scale of ESF and YEI funding as well as other  – powerful – factors of 
influence into account. Therefore, the availability of robust statistical evidence is more limited 
than for changes on the individual level. Based on the Rhomolo model, ESF and YEI funding 
had a positive effect on employment rates of participants and a positive impact on GDP and 
employment levels. One euro spent on the ESF, YEI and REACT-EU is estimated to yield a 
total  of  1.26  euro  in  2030.  In  addition,  some  more  specific  examples  of  changes  at  a  more 

71 

 
 
 
systemic  level  were  observed,  especially  in  the  context  of  the  implementation  of  the  Youth 
Guarantee.  In  addition,  a  large  majority  of  relevant  country-specific  recommendations  was 
taken  up  in  programming  and  all  stakeholder  groups  expressed  positive  views  on  the 
contribution  of  the  ESF  and  YEI  to  implementing  country-specific  recommendations.  The 
supporting study found a broad correlation between Member States’ socio-economic contexts 
(e.g.  the  prevalence  of  relatively  low  employment  rates),  CSRs  issued  to  them,  and  ESF 
allocations at TO level. However, it is not possible to assess the precise contribution due to, 
among others, the large overlap between focus of the European semester and the objectives of 
the  ESF  as  well  as  fact  that  a  number  of  CSRs  are  broad,  calling  for  significant  structural 
reforms, on which the ESF has limited influence.  

Role of the ESF in crisis response 

Both  the  COVID-19  pandemic  and  Russia’s  war  of  aggression  against  Ukraine  represented 
serious crises that the ESF addressed while negotiations and the setup of programmes for the 
2021-2027  programming  period  were  still  ongoing.  Additional  funds  and  reprogramming 
options were introduced via CRII, CRII+, REACT-EU and CARE, in coordination with support 
under ERDF. Evidence shows that CRII/CRII+ made a valuable and substantial contribution to 
the pandemic response, enabling ESF operations to be adapted, and at times newly developed, 
to address emerging needs from COVID-19. These crisis response measures were also more 
easily and rapidly adapted compared to the tools deployed in response to the financial crisis of 
2007-2008. Moreover, REACT-EU extended the crisis response both in volume and scope. The 
additional financing helped alleviate financial constraints in public services, as well as prepared 
the ground for longer-term investments with a view to twin transition. CARE effectively helped 
people fleeing from Russia’s war of aggression against Ukraine to learn the receiving country’s 
language and integrate in the labour market.  

Facilitating and hindering factors 

The  analysis  on  key  factors  influencing  effectiveness,  based  on  triangulation  of  different 
sources,  showed  that  facilitating  factors  include  effective  governance  and  implementation 
approaches,  the  presence  of  well-developed  networks,  strong  cooperation  among  relevant 
stakeholders and partners as well as effective synergies with related operations, other policy 
instruments and funding sources. Further facilitating factors were the analysis of labour market, 
education  and  social  inclusion  priorities,  the  effective  definition  and  understanding  target 
groups as  well as effective mechanisms to  learn  from  previous experience. Finally, reduced 
administrative  burden  and  reporting  requirements  through  simplification  were  key  factors 
facilitating  effective  implementation.  In  the  context  of  CRII/CRII+  the  rapid  response  and 
additional flexibilities were key.  The main hindering factors  are the reverse of the elements 
mentioned above.  

Lessons learned on ‘Effectiveness’ 

Despite  being  overall effective  across the various assessed domains, a number of areas for 
further improvements could be considered. Importantly, most of these lessons learned have 
already been incorporated under ESF+ in the 2021-27 programming period:  

Effectiveness in leading to change 

72 

 
 
 
  
•  Specific  and  targeted  approaches  to  supporting  certain  groups  show  higher 
effectiveness  in  reducing  the  employability  gap  between  those  without  and  with 
disabilities and between the overall working-age population and older workers. 

o  In the 2021-2027 programming period, a new impetus to social innovation was 
given  for  instance  with  the  launch  of  the  European  Competence  Centre  for 
Social Innovation202. This is espected to help develop more targeted approaches 
to support certain groups.  

•  To ensure that horizontal principles  (such as gender equality) are more consistently 
and concretely embedded in ESF operations, specific monitoring requirements could 
be set that track the expenditure and the outcomes related to each horizontal principle. 
Additional  weight  could  also  be  placed  in  national  evaluations  on  assessing 
contributions to horizontal principles.  

o  In the 2021-2027 programming period, secondary dimensions to track, among 
others, gender equality contributions were added to the financial planning and 
reporting.  This  should  provide  a  clearer  picture  of  how  much  ESF+  funds 
contribute to horizontal principles.  

Increasing the ability to assess effectiveness 

•  More guidance could be provided on target setting and efforts on validating the targets 
set by Member States to ensure greater consistency in target setting and thus increase 
the value of their analysis in evaluations.  

o  In the 2021-2027 programming period, the Commission increased its offer to 
check on draft methodological documents on indicators and targets, provided 
targeted  guidance  documents  and  dedicated  sessions  of  its  data  network 
meetings with Managing Authorities to the topic of target setting in an effort 
to improve and share knowledge.   

•  Annual reporting of all indicators, including longer-term result indicators, could also 
be considered so that performance in  terms  of results can be more consistently  and 
reliably assessed throughout programme delivery. 

o  In the 2021-2027 programming period, annual reporting on most indicators was 

introduced to allow for a better performance assessment.  

5.2 EFFICIENCY 

Stakeholder consultations and case studies consistently highlighted that both EU institutions 
and  national/regional  authorities  made  notable  efforts  to  reduce  unnecessary  administrative 
burden e.g. substantial strides were made driven by simplified cost options, digitalisation efforts 
and streamlined reporting. The use of SCOs was widely recognised as a key factor in reducing 
administrative  burdens,  facilitating  efficient  payment  administration  to  beneficiaries  and 
improving financial reporting and monitoring. Digitalisation, and online tools, also resulted in 

202 See: https://european-social-fund-plus.ec.europa.eu/en/esf-social-innovation-initiative 

73 

 
 
 
 
 
 
decreased  administrative  costs,  particularly  for  Managing  Authorities  and  some  light 
monitoring and reporting structures were introduced in certain situations. 

However,  challenges  remain  in  specific  IPs  or  Member  States,  particularly  in  areas  such  as 
financial reporting, claim preparation and submission, documentation, audit requirements, and 
control systems. Also, some beneficiary organisations faced challenges with the introduction 
of  SCOs  and  digitalisation  due  to  limited  knowledge,  in  particular  of  EU  funding  and  cost 
justification  requirements.  Nonetheless,  the  long-term  potential  of  digitalisation  and  SCOs 
remains promising, pending the resolution of existing challenges. 

Overall cost-effectiveness of ESF and YEI 

The cost benefit analysis carried out as part of this evaluation found largely positive evidence 
of  cost  effectiveness  in  terms  of  net  public  returns  on  investment,  although  with  some 
differences across Investment Priorities and countries. The sum of private and public returns on 
investment per participant were highest for direct investments in employment measures such as 
under IP8.i access to employment (EUR 26 000), IP8.iii self-employment (EUR 40 000), IP9.i 
active inclusion (EUR 31 000), and for investments in formal education such as IP10.i early 
school leaving and learning pathways (EUR 27 000) and IP10.ii access to tertiary education 
(EUR 29 000). It should be noted that certain benefits could not be monetised due to a lack of 
data  and  the  impossibility  to  robustly  estimate  them  in  quantitative  terms  (e.g.,  benefits  to 
employers).  

In all TOs, unit costs varied among countries, also reflecting the targeting of different groups 
and the implementation of different types of measures.  

Facilitating factors to efficiency   

The European Social  Fund has made significant  improvements in  administrative procedures 
during the programming period 2014-2020, driven by:  

•  Simplified Cost Options: Flat rates and standard unit costs have simplified financial 

reporting and monitoring, reducing administrative burdens.  

•  Digitalisation:  Online  tools  and  platforms  have  enhanced  operational  efficiency, 
although some Member States faced challenges with introducing new IT systems.  

•  Streamlined  reporting:  Some  Member  States  introduced  lighter  monitoring  and 
reporting  structures,  including  simplified  reporting  and  self-evaluation  forms,  to 
ease the burden on recipient organizations.  

These  innovations  have  collectively  contributed  to  a  notable  reduction  in  administrative 
burdens. However, some challenges remained, including limited knowledge of EU funding and 
cost  justification  requirements  among  beneficiary  organisations,  initial  administrative 
challenges  with  introducing  SCOs  in  some  Member  States,  delays  or  difficulties  with 
introducing new IT systems. 

Overall,  significant  progress  was  made  in  simplifying  ESF  processes,  and  the  long-term 
potential  of  digitalisation  and  SCOs  remains  promising,  pending  the  resolution  of  existing 
challenges. In contrast, there was low take up of Joint Action Plans, likely due to low awareness. 

General administrative burden 

74 

 
 
Total costs of administering the ESF and YEI programmes for public authorities are estimated 
at EUR 3.6 billion, or 2.7% of total spending. This level is considered modest, as administrative 
costs for public authorities below 4% are generally not regarded as excessive. The most cost-
intensive  tasks  for  programme  authorities  were  selection  of  operations  and  information  to 
beneficiaries, while it was preparing a funding application in the case of beneficiaries. 

Significant strides were made in reducing administrative burdens in ESF programmes compared 
to previous periods, driven by simplified cost options, digitalisation, and streamlined reporting, 
which reflected a clear commitment to improving efficiency and reducing complexity. These 
simplification  measures  are  estimated  to  have  produced  efficiency  gains  for  programme 
authorities of around 10% decrease of costs. 

Despite these advancements, some stakeholders (national/regional Managing Authorities and 
beneficiary organizations) still perceived lingering administrative challenges in specific IPs or 
Member  States,  such  as  "gold  plating"  and  complex  processes  in  project  management.  In 
particular,  areas  such  as  public  procurement,  reporting,  claim  preparation  and  submission, 
documentation, audit requirements, and control systems were acknowledged as needing further 
improvement. Unconnected management systems, staff turnover and changes in legislation or 
eligibility requirements were also issues occasionally contributing to project implementation 
delays. The recognition of these challenges by stakeholders highlights opportunities for targeted 
improvements, ensuring future programming periods can build on existing successes.  

Administrative burden in responding to crisis 

Streamlined and expedited processes for amending Operational Programmes and reallocating 
resources enabled Member States to respond swiftly to emerging crises  203  However, in the 
evaluation of crisis response programmes, notably CRII and CRII+, Managing Authorities of 
certain  countries  reported  difficulties  in  navigating  the  rules  or  fulfilling  stringent 
administrative and national audit requirements with high data/documentation demands, which 
were perceived as burdensome and potentially unfair to smaller entities.  

To improve the effectiveness of these types of programmes, stakeholders pointed to the need to 
strike the right balance between simplifying and making flexible administrative processes and 
ensuring clear guidance, accountability and transparency. By adopting flexible and supportive 
administrative practices, such as those implemented in Sweden and Italy, Member States can 
reduce the administrative burden and facilitate access to funding for beneficiaries. 

Management and control systems 

The  monitoring  and  evaluation  systems  generally  functioned  well,  providing  regular 
comprehensive, high-quality data on participants and enabling effective performance analysis 
at reasonable costs. The system was praised for its flexibility, partnership approach, and results-
driven  focus.  However,  challenges  remained  in  aligning  expenditure  with  performance 
indicators and clarifying the relationship between common and specific indicators. Similarly, 
there  were  opportunities  for  improvement,  particularly  in  further  digitalisation,  streamlined 
financial reporting, audit requirements and control systems. 

203 Evidenced in case study research for the study on CRII, REACT-EU and CARE. See Atkinson, I. et al (2023) 
Study supporting the preliminary evaluation of the support provided by ESF and FEAD under the Coronavirus 
Response 
at: 
(CRII 
https://ec.europa.eu/social/main.jsp?catId=738&langId=en&pubId=8542&furtherPubs=yes   

Investment 

Initiatives 

Available 

CRII+). 

and 

75 

 
 
 
The number and quality of evaluations at national and regional level improved significantly. 
However, limitations persisted such due to capacity constraints in smaller administrations or 
countries lacking evaluation expertise, perceived formality of evaluations, which often failed 
to  influence  decision-making  or  the  limited  scope,  focusing  on  implementation  rather  than 
strategic programming or the design of new programmes. 

Future  programs  could  address  these  by  bolstering  capacity  and  aligning  evaluations  with 
strategic  goals.  Overall,  while  current  systems  are  functional,  targeted  improvements  in 
efficiency, digital tools,  and evaluation  practices  could  provide for more  critical  and deeper 
insights as well as operational effectiveness.  

Other hindering factors to efficiency   

Stakeholders have identified following factors leading to additional costs, higher administrative 
burdens and lower efficiencies:  

•  Limited  administrative  capacity  of  project  beneficiaries,  worsened  by  heavy 
administrative burdens, lengthy project processes, and complex guidance manuals. 

•  Operational  challenges  such  as  funding  delays,  high  staff  turnover,  and  a  lack  of 

specialised technical skills. 

•  Data  protection  requirements  causing  misunderstandings  and  demanding  extra 
resources (time/costs), with Germany highlighting strict rules as a major administrative 
burden. 

Lessons learned on ‘Efficiency’ 

There were positive improvements on efficiency as a result of the simplifications proposed by 
the Commission and the other good practices implemented by the Member States. Processes 
could still be simplified further to reduce the administrative burden, streamline procedures, 
and enhance the overall efficiency of the ESF. A number of areas for further simplification or 
reform  emerged  from  the  evaluation  which  could  be  considered  as  future  developments. 
Importantly,  the  2021-2027  framework  has  already  incorporated  several  simplification 
measures  in  these  domains,  including  fewer  policy  objectives,  clearer  indicators,  lighter 
reporting  requirements  and  single  audit  arrangements.  This  shows  a  positive  step  towards 
simplifying ESF implementation processes that could be further enhanced in the future. 

General simplification measures could include:   

•  Making cost options flexible to external shocks by allowing due consideration to the 

rise of living costs and inflation.  

•  Streamlining  national  application  and  selection  processes  where  redundant  or 

excessive  information not directly related to project activities is requested. 

•  Simplifying  monitoring  and  reporting  requirements  by  reducing  data  collection 

requirements and by shortening annual implementation reports.  

76 

 
 
 
  
•  Streamlining  and  expediting  the  amendment  of  Operational  Programmes  and 
reallocating resources enabled Member States to respond rapidly to emerging crises as 
done in the crisis response programmes CRII and CARE. 

•  Simplifying information requirements for small institutions like NGOs and regional 
bodies  and  offering  them  better  capacity  building  support  to  avoid  that  too  large  a 
proportion of the ESF and YEI grant funds goes to administration, audit and control.  

•  Streamlining  auditing  requirements  by  reducing  control  or  verifications  and 
documentation  requirements,  by  requiring  a  proportionate  audit  trail  system  and  by 
eliminating excessive auditing processes, different audit rules and interpretations, and 
repeated requests for the same information. 

Streamlined and effective monitoring systems could be improved with: 

•  Clearer  linkages  between  expenditure  and  indicators  allow  detailed  analysis  of 

expenditure and intensity of the support on participants specifically.  

•  The obligation to publish the list of projects, already in force, could additionally be 
associated with a simple classification of the intervention and other information such 
as  target  groups  or  delivery  mechanism,  being  mindful  of  not  adding  excessive 
additional  burdens,  to  facilitate  improved  understanding  of  linkages  with  the 
Intervention Logic as well as performance, efficiency and effectiveness.  

•  Enhancing the usability and significance of the programme specific indicators. At the 
moment, those indicators are not clearly and functionally connected to the performance 
framework and the common indicators. Creating clear categories of specific indicators 
(e.g. for persons, systems and equipment), providing an explicit link between specific 
and common indicators (e.g. what part of a common indicator is explained by a specific 
indicator) and explaining what internal objective the specific indicator relates to would 
assist in more clearly identifying the performance story. 

•  Reviewing  indicators  so  they  more  robustly  capture  the  quality  of  operations. 
Indicators  were,  in  many  cases,  ‘overly  quantitative’  and  overlooked  the  quality  of 
operations (targets are monitored the main achievement of the operation was not really 
measured).  

Streamlined and effective evaluation systems could be enhanced through: 

•  Building evaluation capacity in Member States (particularly countries with a smaller 
administration or those without prior evaluation experience, to improve the quality and 
usefulness of evaluations) and broadening the scope of evaluation. This would inform 
strategic choices and programming, rather than just supporting implementation. 

•  Further  support  from  the  Evaluation  Helpdesk  and  increasing  the  promotion  of 
effective evaluation practices. This would help reducing the perception of evaluation’s 
being obligatory instead of a tool for decision-making.  

77 

 
 
 
 
5.3 COHERENCE 

Complementarity and synergies of ESF and YEI support with national and regional policies 

The evaluation showed that ESF and YEI operational programmes had strong complementarity 
with national and regional strategies. Most stakeholders, including those surveyed, agreed that 
ESF operations aligned well with national and regional policies. 

As regards internal coherence, ESF and YEI had good level of it with the different parts of the 
programmes  working  well  together  to  achieve  their  objectives.  The  various  projects  funded 
under different areas of intervention (Thematic Objectives) were distinct yet complementary. 
Adjustments made in response to COVID-19 also maintained coherence by targeting different 
groups and using varied delivery mechanisms. Member States generally drew on lessons from 
previous programming periods, involving multiple stakeholders in the design phase and using 
EU-level policies as references, ensuring complementarities and reducing overlap.   

Complementarity with other EU instruments  

The evaluation concluded that there was good external coherence with the European Regional 
Development  Fund,  as  it  was  also  pointed  out  in  the  2014-2020  ERDF  ex-post  evaluation, 
through the use of a targeted policy mix. Numerous examples of complementarities are reported 
in  the  evaluation.  At  the  same  time,  the  ERDF  ex-post  evaluation  found  scope  to  leverage 
synergies with ESF more consistently and effectively, such as in the ICT sector and territorial 
instruments. Stakeholders in the ERDF evaluation also pointed to administrative challenges that 
prevented optimal synergies with the ESF in e-government and e-business interventions.  

ESF stakeholders highlighted that there was also coherence with other funds such as FEAD, 
and  Erasmus+.  According  to  the  ex-post  evaluation  of  FEAD  2014-2020,  the  legal  texts 
governing  the  ESF  regulation  and  FEAD  regulation,  demonstrate  a  good  degree  of 
complementarity, with specific provisions designed to ensure complementarity between these 
funds  or  to  prevent  overlaps  or  duplication.  Examples  from  the  case  studies  in  the  ex-post 
evaluation  of  FEAD  highlight  good  practices  in  several  Member  States  in  terms  of 
complementarities between FEAD and the ESF.  

In relation to other funds, the RRF, introduced in 2021, played a significant role in the transition 
between the 2014–2020 and 2021–2027 programming periods. As concerns the period of this 
evaluation, the RRF complements Cohesion Policy by enhancing policy areas and addressing 
challenges aligned with Country Specific Recommendations. Both the ESF and RRF strengthen 
social  inclusion  and  address  inequalities  while  contributing  to  the  European  Pillar  of  Social 
Rights. The case study on the functioning of the RRF and other EU funds, within the context 
of the Mid-Term Evaluation of the RRF, reported no significant substitution effect with respect 
to the 2014-2020 Cohesion Policy Operational Programmes, except from minor reallocations 
identified in Spain (e.g. Murcia), which nonetheless cannot be confirmed as related to an ESF 
programme.  

Overall, challenges remained due to complex institutional structures which did not allow for 
further coherence with other EU funding instruments. 

Complementarity with wider EU policies 

In  relation  to  the  coherence  with  EU  policies,  it  was  shown  that  there  was  good 
complementarity, with national authorities and stakeholders commonly using EU-level policies 
as  references  when  designing  interventions.  The  Country  Specific  Recommendations  of  the 

78 

 
 
European semester, the  Europe 2020 Strategy, the European Pillar of Social Rights, and the 
European  Skills  Agenda  were  frequently  mentioned,  showing  alignment  with  employment 
objectives  and  coherence  with  social  inclusion  goals.  Overall,  the  ESF  objectives  related  to 
employment  were  the  most  coherent  with  EU  policy  objectives,  while  social  inclusion  and 
training policy objectives were prioritised to a lesser extent.  

Lessons learned on ‘Coherence’ 

•  Most  Member  States  have  centralised  mechanisms  that  ensure  that  funding  and 
operational objectives are allocated appropriately across ESF operations. This internal 
coherence could be further built-upon reducing the risk of overlaps between activities 
and  to  reinforce  shared  objectives  and  complementarities  between  operations  by 
engaging and sharing knowledge with all relevant stakeholders.   

•  Many  Member  States  used  EU  level  policies  as  the  main  point  of  reference  when 
designing interventions, providing a framework and focus for key mainstream issues 
such as unemployment. However, it was noted that in general few references were made 
in planning documents to EU policies for specific target groups (e.g. Roma and persons 
with a disability). Member States could consider the full range of EU policies in their 
planning process regarding all target groups and types of operations. 

5.4 RELEVANCE 

Relevance to the needs of the target groups 

The evaluation provides robust evidence that ESF/YEI programmes were relevant to and well 
aligned with the needs of their target groups, in particular unemployed and inactive people as 
well as (young) NEET. Respondents to the MA survey reported that a better knowledge of the 
target groups’ needs, based on the experience of previous programming periods, has contributed 
to increasing the relevance of operations in the 2014-2020 period.   

ESF/YEI  operations  were  able  to  reach  people  from  disadvantaged  backgrounds,  including 
people living in rural areas, migrants, people from marginalised communities, and the homeless. 
However,  stakeholders  shared  that  there  could  be  scope  for  better  reflecting  in  ESF/YEI 
interventions the specific needs of target groups such as Roma, refugees and migrants, persons 
with disabilities or older workers as well as persons with intersecting vulnerabilities. 

The application of the partnership principle and the involvement of stakeholders (in particular 
NGOs  and  CSOs  with  direct  knowledge  of  the  specific  target  groups’  needs)  in  the  design, 
implementation and monitoring of interventions consistently emerged as a key factor for the 
relevance of the ESF/YEI. In addition, the importance of regular and continuous monitoring of 
implementation and of systematic needs assessments was also raised by stakeholders. 

Some differences can be observed in the relevance of ESF/YEI operations across TOs, with the 
majority of respondents to the MA survey considering that interventions under TO 8, TO 9 and 
TO 10 were relevant to a large extent, whereas most respondents indicated an overall lower 

79 

 
 
  
 
 
knowledge of TO 11 and YEI operations and their relevance. The flexibility provided under TO 
13 (REACT-EU) and the introduction of the CRII/CRII+ and CARE initiatives is considered to 
have helped ensure the continued relevance of the ESF in view of changing needs. 

The horizontal principles of gender equality and non-discrimination, sustainable development 
and  involvement  of  partners  have  been  widely  considered  in  the  OPs.  However,  the  overall 
picture  is  more  mixed  in  terms  of  achievements.  Concerning  gender  equality  and  non-
discrimination,  challenges  persisted  such  as  limited  knowledge  and  institutional  capacity  or 
regarding  the  active  involvement  of  relevant  stakeholders  in  programme  design  and 
implementation.  With  respect  to  the  partnership  principle,  its  effective  implementation 
depended strongly on the capacity of both partners and Managing Authorities. Finally, evidence 
on the effectiveness of the implementation of the sustainable development principle was more 
limited than for the other two horizontal principles.  

Future trends  

Looking  ahead,  ‘mega-trends’  such  as  the  digital  transition,  demographic  ageing,  climate 
change and migration, are expected to have significant impacts on the structure and functioning 
of EU labour markets and social systems. Notably, these trends will further increase the need 
to  urgently  address  current  and  emerging  skills  and  labour  shortages,  a  policy  priority  that 
already features prominently in the political agenda204.  

 Lessons learned on ‘Relevance’ 

•  Member  States  could  rely  more  on  the  use  of  ex-ante  impact  assessments  and 
systematic  needs  assessments  to  inform  the  design  of  interventions  tailored  to  the 
specific  needs  and  circumstances  of  target  groups,  in  particular  considering  their 
intersectional vulnerabilities. In this regard, exchanges and the dissemination of good 
practices  could  be fostered as well  as capacity-building  to  strengthen the skills  and 
competences  of  managing  authorities  and  implementing  bodies  to  conduct  ex-ante 
impact and needs assessments. 

•  A  multi-faceted and holistic approach specific to the target group can be beneficial to 
address the concrete needs of those target groups that are particularly vulnerable and/or 
outside  of  the  labour  market,  incl.  the  Roma,  people  with  disabilities  or 
migrants/refugees,  considering  the  various  challenges  and  obstacles  they  encounter 
when accessing the labour market, essential services and social system.  Alternative 
approaches such as targeting adjacent groups (e.g., care-givers, health care and social 
service providers, social enterprises) can be helpful in reaching the most vulnerable 
groups and addressing their needs. 

• 

Important enablers could be: continued support to the partnership principle and further 
promotion  of  the  involvement  of  stakeholders  in  the  design,  implementation  and 
monitoring of operations (in particular NGOs and CSOs with direct knowledge of the 
specific target groups or representing the target groups), incl. by providing dedicated 
capacity-building opportunities to public administrations, implementing bodies as well 

204  See  for  instance  the  Commission  Communication  on  the  Union  of  Skills  (COM(2025)  90  final)  and  the 
Commission Communication on A Competitiveness Compass for the EU (COM(2025) 30 final). 

80 

 
 
 
 
stakeholders  directly  or  encouraging  the  set-up  of  dedicated  partnership  fora  at  the 
appropriate (regional/local) level. 

•  Furthermore, there is room for further promoting capacity-building and the exchange 
of  experiences  and  good  practices  for  the  effective  and  efficient  monitoring  of  the 
implementation  of  programmes,  fostering  careful  monitoring  of  target  groups  in 
particular where these are defined more broadly (incl. to address the risk of ‘creaming-
effects’). 

• 

In  view  of  the  multiple  crises  affecting  the  implementation  of  the  ESF/YEI 
programmes,  allowing for  sufficient  flexibility  in  the design  and implementation  of 
future funding instruments could account for changing circumstances, while ensuring 
stability and predictability for beneficiary organisations. 

•  Finally, there is scope to increase awareness on future trends with a significant impact 
on labour markets and social systems, to contribute to ensuring that emerging needs 
and challenges can be tackled in a proactive manner. 

5.5 EU-ADDED VALUE 

Added Value of ESF and YEI: volume, scope, role and process effects 

The  evaluation  has  provided  strong  evidence  of  the  ESF  and  YEI  having  produced  several 
beneficial  effects  beyond  what  would  have  been  achieved  by  Member  States  acting  alone, 
adding significant value in terms of volume, scope, role and process effects. 

•  Volume  effects:  ESF  and  YEI  have,  to  a  significant  extent,  generated  a  difference  in 
terms of volume effects, allowing Member States to reach more individuals and entities 
than would have otherwise been possible solely through national funding, as well as a 
larger number and scale of activities. In addition, during the pandemic, ESF and YEI 
proved  significant  EU  added  value  through  their  ability  to  increase  the  volume  of 
support provided and expand the type of services. 

•  Scope  effects:  The  ESF  and  YEI  operations  broadened  the  actions  to  diverse  target 
groups  and  new  or  underdeveloped  policy  areas,  increasing  the  support  for  hard-to-
reach groups compared to the previous programming period, as well as the institutions 
and enterprises supported.  It  also  allowed the support to  new services  available, and 
areas of intervention where there was insufficient national funding. 

•  Role  effects:  The  implementation  of  ESF  and  YEI  generated  role  effects  linked  to 
supporting innovation and the transfer of ideas and enabled changes in national systems 
or  structures  in  some  instances.  While  evidence  is  not  even  across  TOs  and  IPs, 
successful cases implementing social innovation approaches and its replication in the 
national regional, or local level are present. The evaluation showed that ESF and YEI 
strengthened social partners and local institutions and promoted policy shifts. However, 
less clear evidence is available on the contribution to structural changes. 

•  Process effects: The ESF and YEI have produced to a moderate extent process effects 
on  the  stakeholders,  organisations  and  authorities  involved.  ESF  and  YEI  operations 
enabled  Member  States  administrations  and  participating  organisations  to  gain  new 
competences. The funds also enabled collaboration among stakeholders towards long-

81 

 
 
 
term objectives, increasing the capacity of government administrations and participating 
beneficiaries,  and  enhancing  the  efficiency  of  their  processes.  It  allowed  drawing 
lessons  learned  from  previous  experiences,  which  resulted  in  the  provision  of  new 
implementing tools. 

The  available  monitoring  information,  complemented  by  stakeholder  interviews,  the  MA 
survey, the public consultation and case study research provided a positive picture, identifying 
data,  numerous  examples  and  assertions  from  national  stakeholders  that  ESF  and  YEI  have 
generated effects  that would not  have been possible in  their absence. The available national 
impact and implementation evaluations, as well as the results of the reports summarising the 
mid-term  evaluations  on  the  TOs,  confirm  these  findings  of  EU  added  value  across  all 
dimensions.  Stakeholders  of  all  types,  but  particularly  national  stakeholders  involved  in 
managing or delivering ESF support, made strong assertions that the identified effects would 
otherwise  not  have  taken  place  or  would  have  to  a  lesser  extent  without  ESF  support, 
showcasing the beneficial effects beyond what would have been achieved by Member States 
alone. 

Lessons learned on ‘EU-added value’ 

•  Building  on the ESF achievements, continued investment  can unlock synergies and 
amplify  the  positive  impacts  achieved  so  far.  To  maximise  the  fund's  value,  it  is 
essential to conduct an early assessment, in close cooperation with national authorities, 
to identify which objectives can be best pursued through existing operations, which 
innovative  approaches  can  be  scaled  up  and  replicated  across  Member  States,  and 
where EU support can add the most value in ensuring the long-term sustainability of 
these operations 

•  There is a need to avoid the risk that EU-funding could replace national  funding in 
areas that are not policy priorities. Therefore, ensuring further alignment in terms of 
priorities between EU- and national-level investments avoiding funding gaps could be 
considered for the future. Enhancing sustainability of operations by integrating into 
national and regional initiatives could help. However, a number of factors are likely to 
play  a  role  and  mediate  sustainable  outcomes,  such  as  national  and  local  policy 
environments and available funding, further investments, the ownership over current 
practices and outcomes, as well as specific factors linked to each area of intervention. 

•  The use of flexibilities provided by crisis instruments to face the COVID-19 pandemic 
deepened and  expanded  the support for participants most in need, which would not 
have  been  possible  without  EU  support.  While  being  mindful  of  the  original 
programme goals, an enhanced flexibility in programming could allow ESF to better 
contribute to emerging crisis and objectives. 

5.6 VISIBILITY 

ESF/YEI programmes have overall achieved reasonable levels of visibility, with good levels of 
visibility in particular for those organisations and stakeholders which were directly involved in 
the implementation of the supported projects, although evidence suggests notable differences 

82 

 
 
 
 
across TOs and Member States. Successful communication and awareness-raising activities that 
were highlighted by stakeholders in the context of the supporting study include high quality and 
dedicated websites, the display of the EU logo, specific communication campaigns and events, 
the use of social media and continued communication, including across programming periods. 
The combination of different communication and awareness-raising activities also emerged as 
a key factor that can facilitate visibility.  

Nonetheless, less visibility is perceived among the general public. In addition, visibility was 
most often raised by stakeholders as one of the main areas for improvement, and evidence from 
national evaluations highlights the need to disseminate good practices as well as of results of 
ESF/YEI support. 

Lessons learned on ‘Visibility’ 

•  A  multi-faceted  approach  towards  achieving  higher  levels  of  visibility  could  be 
promoted,  which  combines  different  communication  channels,  formats  and  tools 
(including  communication  and  local/social  media  campaigns,  events,  dedicated 
community outreach, etc.), considering the specificities of the target groups at hand. 

•  More emphasis could be put on setting up high-quality websites with clear and concise 
information on the support provided to ensure that potential beneficiaries as well as 
beneficiary  organisations  can  benefit  from  a  single  entry-point  when  looking  for 
relevant information and to foster a coherent brand identity. 

•  Exchanges  between  Member  States  of  good  practices  and  approaches  towards 
achieving visibility could be enhanced in the future. In addition, the dissemination of 
results achieved in EU-funded projects could be strengthened, including via different 
(EU-level) channels and fora. 

83 

 
 
  
6.  Annexes 

Annex I:    

Annex II:   

Procedural information 

Methodology and analytical models used 

Annex II.a:    Methodology followed by external contractor 

Annex II.b:   

Intervention logic  

Annex II.c:    Evaluation matrix 

Annex II.d:    Administrative costs 

Annex III:   

Annex IV:   

Annex V:  

Annex VI:  

Overview of benefits and costs 

Stakeholders consultation - synopsis report 

Public consultation report 

Factual summary report 

Annex VII:  

Key data and additional information 

Annex VIII:  

Focus group summary report   

84 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
ANNEX I:   PROCEDURAL INFORMATION 

Lead DG, Decide Planning/ CWP references 

The  evaluation  was  led  by  the  Directorate-General  Employment,  Social  Affairs  and  Inclusion 
Directorate G – Funds, Programming and Implementation Unit G.5. – Better Regulation. The Decide 
planning reference is PLAN/2022/953 

Organisation and timing 

An  interservice  Steering  Group  (ISG)  was  set  up,  coordinated  by  the  Directorate-General  for 
Employment, Inclusion and Social Affairs (represented by staff from the Better Regulation unit and 
other policy and geographical Units) including members from the Secretariat-General (SG); the Joint 
Research  Centre  (JRC-Seville  and  JRC-Ispra);  the  Directorate-Generals  for  Migration  and  Home 
Affairs (HOME); Justice and Consumers (JUST); Regional and Urban Policy (REGIO); Agriculture 
and Rural Development (AGRI); Digital Services (DIGIT); Maritime Affairs and Fisheries (MARE); 
Education, Youth, Sport and Culture (EAC). 

The  ISG also  acted as steering group  for the external  study supporting  the evaluation,  which  was 
carried  out  by  a  consortium  of  Ecorys,  Ismeri  Europa,  and  3s,  under  the  Framework  Contract 
VC/2021/0336.  The 
to  Ecorys  on  September  14th,  2022 
awarded 
(REF.ARES/2022/6355724). The contract for the study was signed on September 29th, 2022. 

tender  was 

On 30 May 2024 the contractor sent the draft final report (ARES 2024/3871739) and on the 29 July 
2024 the contractor sent the revised final report. From July 2024 to August 2025 successive rounds 
of revisions took place until reaching the published version205. 

The call for Evidence published from 17 October 2023 to 23 January 2024 explained the context, 
purpose and scope of the evaluation and informed stakeholders that an external evaluation study was 
being carried out, supported by a public consultation. The public consultation was open for 12 weeks 
from 16 October 2023 until 23 January 2024. The factual summary was published on 22 March 2024. 
Targeted consultations were carried out mainly from April to June 2023. See Annex IV for further 
details on the stakeholder consultations carried out and the Supporting Study.  

Exceptions to the better regulation guidelines  

All Better Regulation requirements were fulfilled.  

Consultation of the RSB (if applicable)  

The evaluation was not selected for the scrutiny of the Regulatory Scrutiny Board.  

205 Available at: Study supporting the ex-post evaluation of the 2014-2020 European social fund and youth employment 

initiative - Publications Office of the EU and Annexes. 

85 

 
 
 
 
  
 
  
 
Evidence, sources and quality  

The  evaluation  was  based  on  the  evidence  collection  and  analysis  carried  out  by  external  experts 
(contract with Ecorys/Ismeri/3S to carry out the supporting study206, see above) as well as additional 
sources:  the  ESF  Synthesis  Report  of  2022  AIR207,  the  JRC  report  “The  ex-post  macroeconomic 
evaluation of the 2014-2020 European Social Fund, Youth Employment Initiative and REACT-EU 
labour market interventions”208; the meta-analysis of Counterfactual impact evaluations209; additional 
support provided by the Evaluation Helpdesk for the European Commission summarising national 
evaluations. Cut-off data between the different sources differ in some cases, which are mentioned in 
the report. 

Annex II provides additional information on the methodological approach taken. To ensure quality, 
the ISG provided feedback throughout all steps of the evaluation process and ensured accuracy of the 
factual information and data in each study deliverable.  

ANNEX II:  METHODOLOGY AND ANALYTICAL MODELS USED 

The  main  objective  of  this  ex-post  evaluation  is  to  provide  an  assessment  of  the  effectiveness, 
efficiency, coherence, EU added value, relevance and visibility of ESF and YEI for the period 2014-
2022.   

DG EMPL took a mixed approach to this evaluation, by relying on the work of external evaluators to 
(i) collect and analyse the relevant evidence (including consultation work); (ii) provide initial answers 
to  all  evaluation  questions;  and  (iii)  present  evidence-based  conclusions  and  lessons  learnt.  The 
supporting  study  followed  a  mixed  methods  data  collection  approach,  combining  qualitative  and 
quantitative research tools and sources to collect wide evidence on ESF and YEI performance. The 
evaluation did not assess the performance of the fund in each individual Member State but focused 
on evaluating the performance of ESF and YEI at a broader EU-level. 

The  external  contractor  collected  and  examined  evidence  covering  2014-2022  for  the  supporting 
study and performed. This included a collection of case studies and country fiches, available in the 

206 Available at: Study supporting the ex-post evaluation of the 2014-2020 European social fund and youth employment 
initiative - Publications Office of the EU and Annexes. 
207 European Commission (2023) ESF Synthesis Report AIR 2022. Available at: ESF Synthesis Report of 2022 AIRs - 
European Commission 
208 Casas, Pablo and Christou, Tryfonas and García Rodríguez, Abián and Lazarou, Nicholas Joseph and Salotti, Simone 
(2025): The ex-post macroeconomic evaluation of the 2014-2020 European Social Fund, Youth Employment Initiative 
and REACT-EU labour market interventions. Link to the publication: https://mpra.ub.uni-muenchen.de/123410/ 
209 It should be noted that at the moment of preparing this SWD, the available updated meta-analysis was used: Haepp, 
T. and Serrano Alarcon, M., Meta-analysis of the European Social Fund counterfactual impact evaluations: Brief update 
with 
at: 
https://publications.jrc.ec.europa.eu/repository/handle/JRC137571.  In  2025,  a  new  update  was  published  using  112 
Counterfactual Impact Evaluations. In the 2025 update, ESF made a difference by increasing participants' chances to find 
employment, on average, by 6.6 percentage points (or by 16%) compared to non-participants. Available at: Haepp, T., 
Jessen,  J.,  Pompili,  M.  and  Seebauer,  J.,  Meta-Analysis  of  ESF/YEI  Counterfactual  Impact  Evaluations  -  A 
comprehensive  update  including  new  methodology,  Publications  Office  of  the  European  Union,  Luxembourg,  2025, 
https://data.europa.eu/doi/10.2760/1873376, JRC141819  

JRC137571,  Available 

alternative  measures, 

Commission, 

European 

Ispra, 

2024, 

86 

 
 
 
 
 
published  supporting  study  annexes210.  Evidence  was  complemented  with  additional  sources. 
Moreover, JRC carried out a quantitative analysis to assess the macroeconomic impacts of the ESF 
and YEI, which served for complementing the analysis. The Joint Research Centre simulations using 
the  RHOMOLO211  model  to  evaluate  the  macroeconomic  impact  of  the  2014-2020  ESF  and  YEI 
programme compared to the “no-policy baseline” 2013. 

General equilibrium models like RHOMOLO must rely on simplifying assumptions. The modelling 
results should therefore  be interpreted bearing inter alia in  mind that modelling  assumptions have 
been  used  to  incorporate  information  on  the  amount  of  money  spent  in  the  programmes  into  the 
scenario simulations and on how to translate the inputs into outputs and that the modelling results are 
affected by the way in which the interventions are introduced into the model. 

The intervention logics  – one for ESF and YEI  as a whole, and one for each Thematic Objective 
(Annex II.B) set out a high-level understanding of the rationale, inputs, activities, outputs, results and 
impacts of ESF and YEI and provided a framework for a standardised approach to the evaluation. 
The  intervention  logics  formed  the  foundation  of  the  evaluation’s  analytical  framework 
encompassing  the  evaluation  inquiries.  To  conduct  the  evaluation,  the  supporting  study  team 
completed eight tasks (see Figure A. 1).  

The evaluation questions to which the supporting study gave answer are included in the Annex II.C 
“Evaluation Matrix” covering the five evaluation criteria. 

ANNEX II.A:  METHODOLOGY FOLLOWED BY EXTERNAL CONTRACTOR 

The external contractor followed a set of tasks to perform the analysis of the evidence in the supporting study 
to this evaluation. This annex presents the methodology followed for each of the tasks. 

210 Available at: Annexes. 
211  Casas,  Pablo and Christou,  Tryfonas and García  Rodríguez,  Abián and Lazarou,  Nicholas  Joseph and Salotti, 
Simone (2025): The  ex-post  macroeconomic  evaluation  of  the  2014-2020  European  Social  Fund,  Youth  Employment 
Initiative and REACT-EU labour market interventions. Link to the publication: https://mpra.ub.uni-muenchen.de/123410/ 
87 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Figure A. 1: Overview of the methodology 

Source: Ecorys/ Ismeri 

Mapping of ESF investments and operations 

SFC2014 data collection and analysis (Task 1.1) 

Task 1.1 focused on identifying and mapping data on European Social Fund Operational Programmes' financial 
and physical (output and result) indicators to be utilized in Task 1. This involved sourcing monitoring data 
from  openly  available  databases  (https://cohesiondata.ec.europa.eu/)  and  the  SFC2014  platform.  The 
monitoring data provides comprehensive information on the financial and physical implementation of ESF 
OPs, essential for analysing the distribution of funding across Investment Priorities , the demographics reached 
by ESF funds, and the outcomes achieved by the end of 2022 compared to planned targets. Monitoring data 
was extracted from the EC SFC2014 platform in September 2023. 

As per Article 74(4) of Regulation (EU) No 1303/2013, SFC2014 is the designated electronic data exchange 
system for managing shared funds between Member States and the European Commission for the 2014-2020 
programming period. Each Member State appoints a responsible authority to manage the system, including 
user verification, data protection, and notifying the Commission of any changes affecting responsibilities212. 

SFC2014 is a vital tool for monitoring activities, offering features that streamline procedures. For instance, 
forms are either interactive or pre-filled based on previously recorded data, reducing the need for repetitive 
data  entry;  calculations  are  performed  automatically  to  minimise  user  input  and  reduce  encoding  efforts; 

212 Commission implementing regulation (EU) 184/2014 of 25 February 2014, Art. 3.3 https://eur-lex.europa.eu/legal-

content/EN/TXT/HTML/?uri=CELEX:32014R0184 

88 

 
 
 
 
 
 
 
 
 
routine automatic controls ensure internal consistency of data and compliance with applicable rules, sending 
out alerts to inform users about permissible or prohibited actions. Finally, SFC2014 maintains and provides 
access  to  historical  data  for  all  information  entered  related  to  any  operational  programme213.The  platform 
promotes efficiency and uniformity in data handling and introduces a consistent nomenclature system, enabling 
flexible analyses across different countries, programmes, priority axes, fields of intervention, and regions. 

SFC2014 data is a cornerstone for EU evaluations and was extensively used in this study. It enabled detailed 
mapping of ESF financial resource allocation to various TOs and IPs, assessment of  implementation progress, 
iachieved  outcomes,  and  target  populations.  This  data  was  also  critical  for  estimating  the  costs  of  ESF 
participants, feeding into the Cost-Effectiveness Analysis in Task 4. 

Despite improvements since the 2007-2013 programming period, ESF monitoring data in SFC2014 used for 
the study  still  presented inconsistencies  and  errors, affecting  data  reliability  and  complicating aggregation. 
Notably,  some  submissions  were  incomplete  or  inconsistent,  failing  to  adhere  to  established  nomenclature 
rules.  Additionally,  the  codification  of  data  was  not  uniform;  programme-specific  indicators,  which  are 
individually  negotiated  between  Member  States  and  the  European  Commission,  often  remained  non-
comparable and were frequently defined in the native languages of the Member States. Furthermore, reporting 
procedures varied, with some programmes only reporting fully implemented operations, while others included 
partially implemented ones, leading to potential distortions in unit cost analysis. These differences were more 
pronounced  during  the  programming  period,  but  even  in  2022  not  all  interventions  were  completed,  in 
particular  for  REACT-EU.  The  above-mentioned  issues  could  not  be  addressed  because  it  would  need  a 
revision of the entire monitoring dataset and an interaction with all the MAs; however, we carried out a cross-
check on the monitoring data summarised by the ESF Synthesis Report of 2022 AIRs214 to be aligned with the 
better verified and screened information.  

Specific datasets and cutoff dates were agreed upon with DG EMPL. In September 2023, an extraction of 
financial and physical monitoring data from SFC2014, reflecting the status as of the end of 2022, was provided. 
This extraction served as the definitive data collection threshold for the study. However, some additional data 
from one Member State up to the end of 2022 was accepted in November 2024 at the request of the Commission 
and incorporated into the final dataset. Subsequent data cleaning included translating indicators and priority 
axes from native languages, reformatting numbers for large data operability, reclassifying indicators to resolve 
code ambiguities, and updating NUTS2 region classifications to 2021 standards. These measures ensured data 
accuracy and consistency, facilitating reliable analysis of ESF financial allocations, programme progress, and 
participant outcomes, thereby supporting comprehensive and effective evaluation. 

Methodology for the mapping of ESF operations (Task 1.2) 

The mapping of ESF operations under sub-task 1.2 ‘Collection list of operations and/or databases and analysis 
of ESF operations’ aimed at mapping the ESF types of operations financed under each TO and IP to analyse 
how  ESF  operations  have been implemented  over  the  evaluation  period  while identifying related financial 
resources. It encompassed the analysis and classification of the implemented operations reported in the lists 
published by MAs according to Regulation (EU) 1303/2013, Annex XII from the Supporting study.   

The main methodological steps for the collection and classification of the lists were the following:   

•  Selecting the sources used to retrieve each country’s lists of operations and evaluating the implications 
of selecting a unified database (Kohesio215) or publicly disclosed lists from national MAs published 
on programmes’ websites.   

•  Retrieving each country’s most recent lists of operations with the support of national experts when 

needed (e.g., in the case of GR and HU).  

213 Ibid. Art.4 
214European Commission (2024) ESF Synthesis Report of 2022 AIRs.  Available at: https://european-social-fund-
plus.ec.europa.eu/en/publications/esf-synthesis-report-2022-airs 
215 The Kohesio database is freely accessible at: https://kohesio.ec.europa.eu/data/ 

89 

 
 
 
•  Assembling  country  lists,  integrating  operations  financed  by  multiple  OPs  in  a  single  layout  to 

facilitate data aggregation and analysis in later stages.  

•  Preparing the lists by:  

o 

translating  information  into  English  through  the  European  Commission  E-translation 
system216; 

o  converting, where necessary, national currency eligible costs in EUR by means of average 

ECB exchange rates217; 

o  performing additional adjustments to variables and data cleaning, as well as the identification 

of COVID-19, REACT-EU, and CARE related operations;  

o  Marking operations to support classification by selecting those with high relative financial 

weight and those financed through CRII/CRII+, REACT-EU, and CARE.  

•  Assigning categories to all selected operations in accordance with the type, purpose, scope, and target 

of the measure.  

•  Producing a structured database of the ESF operations (including information on priority axis and call 

for proposal/tender, where available).  

In the following paragraphs, each stage is addressed following the chronological and operational sequence 
listed above, from the selection of sources to the complete classification of each list. We also highlight, where 
relevant,  country-specific  peculiarities  in  the  management  of  EU  funds  that  have  emerged  during  the 
implementation of this complex and multifaceted sub-task. These observations have been integrated in this 
annex so as to characterise the different approaches adopted by national MAs, as well as underline some of the 
challenges that needed to be overcome during sub-task 1.2.  

The selection of sources  

An assessment was made regarding the potential utilisation of the Kohesio database during the Inception phase. 
Under  the  management  of  DG  REGIO,  Kohesio  serves  as  a  single  repository  that  assembles  all  lists  of 
operations published by national MAs, with the objective of facilitating integration.  

A  comprehensive  evaluation  was  conducted  to  determine  the  viability  of  drawing  data  from the  Kohesio 
database.  The  assessment  revealed  a  satisfactory  coverage  of  operations  in  relation  to  those  declared  by 
Member States at the end of 2021 and reported in the SFC. However, several limitations emerged, namely:   

•  Underreporting of operations financed under the Youth Employment Initiative;  
•  Kohesio lists from the version available in early 2023 were not updated (the lists were updated between 

November 2020 – November 2021) and consequently information was not homogeneous;  

•  UK data was not included in Kohesio;   
•  Most importantly, the Kohesio database publishes the variables requested by EU Regulations within 
the lists of operations, whereas, in many cases, MAs include other important variables in the lists of 
operations  published  on  their  websites,  including  and  not  limited  to  priority  axes,  public  calls  for 
projects, links to specific measures or funds (CRII/CRII+, REACT-EU, or CARE), status of project, 
and implementing organisations. Moreover, REACT-EU operations are organised based on the same 
category of expenditures for ESF (for code 102 to 120, excluding the Technical Assistance  – TA – 
operations);  thus  it  is  not  possible  to  identify  REACT-EU  operations  relying  exclusively  on  the 
category of intervention.  

Therefore, our approach mainly relied on the direct collection of the lists of operations published by the MAs 
on their websites to identify ESF types of operations. Despite the wide degree of heterogeneity in layout and 
published information, national lists frequently offered additional information on OPs, such as priority axes, 

information  on 

216  Further 
https://commission.europa.eu/resources-partners/etranslation_en 
217 See https://www.ecb.europa.eu/stats/policy_and_exchange_rates/euro_reference_exchange_rates/html/index.en.html 
90 

the  European  Commission’s  Machine  Translation  system 

is  available  at: 

 
 
 
types  of  measures,  calls,  location  identifiers,  implementation  details  and  official  websites,  among  other 
variables not covered by Kohesio.  However, when data was incomplete or the lists required adjustments or 
integrations, missing information was extracted from Kohesio; this information was used whenever national 
lists  were  unretrievable,  outdated,  or  partial.  Integrations  of  information  from  national  lists  and  official 
documentation (AIRs and OPs) were performed to overcome data fragmentation (e.g., for BE, SI, MT, FI).  

The retrieval of country lists  

This task was carried out at national level and required the compilation, for each country, of a single aggregated 
list of all operations under the relevant OPs (excluding technical assistance OPs). Highly regionalised Member 
States, such as France, Italy, Spain and Germany, have a relatively large number of OPs made up of both 
regional and national programmes, leading to fragmentation and heterogeneity in the way MAs organise and 
present the lists of operations on their websites.  

Figure A. 2: Number of Operational Programmes by country as included in ESF mapping task 

Source: our elaboration of SFC2014 data, up to end of 2022, extracted in September 2023  

The assembly of lists  

Once all lists of operations of a Member State were gathered, a process of assembly and adjustment followed; 
lists of operations were first converted in a provisional format optimised for cross-referencing of information, 
triangulation of sources, and ease of handling in the process of classification, and later simplified to a database 
format to rationalise the process of data storage, transmission, and analysis.   

The relevant variables reported for each list of operations are reported in the table that follows.  

Table A1. Relevant variables reported for each list of operations  

Provisional Format  

Database Format  

op_id  

op_nameen  

op_nameorig  

ms  

ms2  

op_startdate  

op_enddate  

cofrate  

eligibleexp  

eubudget  

Operation identification code  

op_id  

Operation identification code  

English translation of operation name  

op_nameen  

English translation of operation name  

Original operation name  

op_nameorig  

Original operation name  

Full name of country  

Acronym of country  

ms2  

eligibleexp  

Acronym of country  

Total eligible expenditure of operation in €  

Start date of operation  

BeneficiaryLabel_en  

English translation of beneficiary  

End date of operation  

categoryint2  

Adjusted intervention category  

EU co-financing rate  

to2  

Adjusted TO  

Total eligible expenditure of operation in 
€  

fund2  

Adjusted ESIF fund; 1=ESF, 2=YEI  

Eligible expenditure financed by EU  

opcci  

Code of operational programme  

91 

 
 
 
categoryint  

categoryint2  

to  

to2  

fund  

fund2  

opcci  

opname  

Region  

todrop  

todrop2  

benef_name  

Beneficiary 
Kohesio  

identification 

code 

on 

opname  

BeneficiaryLabel  

Original beneficiary name  

region  

BeneficiaryLabel_en  

English translation of beneficiary  

region_category  

Intervention category as reported in list of 
operations  
Intervention  category  as  adjusted  by 
compiler  

op_summaryen  

adjustmentsfields  

TO as reported in list of operations  

flag_react_eu  

Name of operational programme  

Name of region  

Region  category, 
developed, 
2=In 
developed  

if  available;  1=More 
3=Less 

transition, 

English translation of project description  

Descriptor of any  adjustment performed on 
the operation  
Flag  for  operations  related  to  COVID, 
CRII/CRII+, REACT-EU, or CARE  

TO as adjusted by compiler  

typeop_main  

Main type of assigned category  

ESIF fund as reported in list of operations  typeop2  

Secondary type of assigned category  

ESIF fund as adjusted by compiler  

typeop3  

Code of operational programme  

notetpyeop  

Tertiary  type  of  assigned  category  (mainly 
for TO 10 operations)  
Additional 
operation type  

description 

qualitative 

of 

Name of operational programme  

website  

URL of website of operation  

nuts3_code  

NUTS3 code of region  

Name of region  

target1  

target2  

Dummy variable for omission from list of 
operations (1)  

target3  

Primary operation target.  

Secondary operation target.  

Tertiary operation target.  

Reason for omission  

target4  

Additional operation target.  

op_summaryen  

English translation of project description   target5  

Additional operation target.  

op_summaryorig  

Original project description  

target6  

Additional operation target.  

adjustments  

Dummy  variable  for  modification  of 
operation variables (1)  

target7  

Additional operation target.  

adjustments2  

Applied modification  

target8  

Additional operation target.  

doublesindescription  

90percent  

flag_react_eu  

variable 

Dummy variable for recurring or repeated 
descriptions  
Dummy 
constituting  90%  of 
expenditure  
Flag  for  operations  related  to  COVID, 
CRII/CRII+, REACT-EU, or CARE  

operations 
eligible 

total 

for 

notetarget  

notetarget  

axis_ifavailable  

website  

URL of website of operation  

call_ifavailable  

Additional qualitative descriptor of target.  

Additional qualitative descriptor of target.  

OP  Priority  Axis  number/code  pertinent  to 
the operation, if available.  
Public  call  number/code  pertinent  to  the 
operation, if available.  

typeop_main  

Main type of assigned category  

year_start  

Starting year of operation  

typeop2  

Secondary type of assigned category  

year_start_COVID  

Additional  qualitative  description  of 
operation type  

COVID-CARE_flag  

Pre-  or  post-COVID  operation;  0=before 
2020, 1=during or after 2020.  
1 if operation pertains to COVID initiatives; 
2 if operation pertains to CARE initiatives.  

Primary operation target.  

REACT_flag  

1 if operation pertains to REACT initiatives.  

Secondary operation target.  

Column  to  add  Priority  Axes,  Calls, 
operation  postcode,  and  other  relevant 
information  

notetpyeop  

target1  

notetarget  

additional fields  

Source: Supporting study  

Notably, the provisional format used for the classification of operations concerned individual countries and 
thus a total of 28 of such lists was produced; each of these lists were remodelled to the database format and 
consolidated  in  a  single  document.  To  prevent  the  inhibition  of  data  analyses  due  to  the  sheer  size  of  the 
consolidated list, many variables were removed or simplified in the conversion from provisional to database 
format. Only redundant or inconsequential variables in the context of this study were omitted from the database 

92 

 
 
  
  
  
  
format: national-language name of beneficiaries and summaries, unamended categories of interventions and 
TOs, variables to assist classifiers such as double descriptions or pertinence to the 90% of national expenditure 
represent some of the variables not included in the final format. Other variables were amended to streamline 
data elaboration; operations’ start and end dates, for example, can be key elements for the identification and 
classification of an operation. However, since the only necessity was to confirm whether the operation started 
before  or  after  COVID-19  pandemic’s  onset,  a  dummy  variable  supported  the  optimisation  of  elaboration 
procedures.  

The adjustment of lists  

Once MAs’ lists of operations for a given country were put together and unified in the provisional format, a 
series  of  actions  were  performed  to  guarantee  readability  in  the  classification  process.  When  presented  in 
Member States’ native language, operation names, beneficiary names, and project descriptions were translated 
into English through the European Commission e-Translation service76 and added to the pertinent columns. 
Due  to  occasional  incompatibilities  in  the  formatting  or  encoding  of  original  lists  of  operation,  additional 
translation services were employed.  

The consolidation of currencies was essential for the elaboration of financial data: MAs from countries within 
the Eurozone reported all financial data of eligible expenditure in Euros; this was also the case for Croatia, 
which joined the Eurozone effectively on January 1st, 2023, and where the MAs have retroactively updated 
their lists of operations by converting the value of operations to the new currency. Conversely, other Member 
States reported expenditure values in national currency (BG, CZ, DK, HU, PL, RO, SE, UK). In this case, a 
conversion was applied using European Central Bank’s data218 as a reference point; the applied conversion rate 
equalled the average exchange rate reported by the ECB website between the dates of the earliest and latest 
operations reported in country lists.  

Applying a single average exchange rate for operations distributed over a timespan of nearly 10 years has 
evident implications. First, currency volatility is smoothed out in the process of calculating average exchange 
rates, as fluctuations influenced by economic factors and geopolitical events are levelled and longitudinally 
distributed. By the same token, the varying impact of currency fluctuations across countries and projects can 
further skew representations. Second, the value of projects occurring in high-variation years does not fully 
reflect the true value at the time, possibly leading to over or underestimations of the real purchasing power of 
spent funds. Third, extraordinary exchange rate events such as the array of measures developed to contain the 
COVID-19 pandemic likely implied further distortions. Awareness of these aspects was crucial in ensuring the 
accurate interpretation of data.  

Operations that lay outside of the scope of this study were removed from the list, such as Technical Assistance 
operations, operations without information of eligible expenditure, or operations with expenditure equal to 
zero.  

To compensate for data omissions, outdatedness, or articulated expenditure allocations, adjustments for each 
list of operations were needed on a case-by-case basis. Adjustments were performed exclusively on essential 
variables to the purposes of the mapping, such as category of intervention, TO, fund, name, beneficiary and 
summary to ensure usage of the correct criteria in the classification. Whenever needed, we applied adjustments 
to eligible expenditure in order to account for shifts in the allocation of expenditure, the integration of missing 
operations for which funding was declared in the monitoring documentation, the balancing of the distribution 
of  expenditure  for  individual  operations  that  were  later  split  to  achieve  contingent  purposes,  and  other 
necessary recalibrations. In the database, the column adjustments outlined all such instances. An exemplary 
case  is  that  of  Hungary,  where  several  pre-existing  operations  from  different  OPs  and  TOs  were  formally 
repurposed  to  address  the  pandemic  crisis;  as  only  a  fraction  of  the  allocated  expenditure  was  redirected 
towards specific emergency efforts, new operations have been manually added to the list by drawing on the 
reported amount of eligible expenditure destined to the crisis effort from the original operations. The specific 

218 See https://www.ecb.europa.eu/stats/policy_and_exchange_rates/euro_reference_exchange_rates/html/index.en.html 

93 

 
 
 
processes  used  for  COVID-19,  REACT-EU,  and  CARE  operations  are  detailed  at  greater  length  in  the 
dedicated section below.  

The final step for the preparation of lists consisted of the selection of operations to classify. All the largest 
operations representing at least 90% of the total OP eligible expenditure were marked for classification. This 
was necessary because resources were not sufficient for classifying all the operations and, for some operations, 
information was not complete even after the integrations.  Operations were sorted by value of expenditure, and 
the sequence of operations up to the first to exceed 90% of the total expenditure reported in SFC was marked 
for classification. We then controlled for categories of intervention, ensuring the selection of a minimum of 
90% of total eligible expenditure of operations from every category of intervention presented in each country 
list. Finally, we attempted to identify all operations related to COVID-19, REACT-EU, and CARE through 
keywords in operations’ titles and summaries, priority axes, project calls and cross referencing with OPs, AIRs, 
and the recent “Study supporting the preliminary evaluation of the support provided by ESF and FEAD under 
the  Coronavirus  Response  Investment  Initiatives  (CRII  and  CRII+)219.  All  COVID-19,  REACT-EU,  and 
CARE  operations  were  marked  for  classification  irrespectively  of  category  of  intervention  or  eligible 
expenditure.   

The categories of operations  

For TO 8-TO 9-TO 10, the categories of ESF operations followed those already used in the previous three 
thematic studies220, with small adjustments. In practice, this involved categorising each operation mainly based 
on the type of instruments employed to carry out the measure (TO 8), the overarching social objective of the 
operation (TO 9), and the targeted educational level (TO 10). As far as TO 11 is concerned, the categories 
were established on the basis of the analysis carried out in the PAPA study221 and were chiefly distinguished 
according to the domain of administrative capacity they acted upon.   

With regard to the ESF operations financed in response to the COVID-19 pandemic, those categories used in 
the recent “Study supporting the preliminary evaluation of the support provided by ESF and FEAD under the 
Coronavirus Response Investment Initiatives (CRII and CRII+)” were reproposed for the ESF mapping.  

Finally, in relation to REACT-EU we used the same categories of operations applied to the previous TOs (TO 
8, 9, 10 and 11), since REACT-EU (TO 13) is linked to the same ESF categories of expenditure (from 102 to 
120).  

The categories used for the classification of the ESF operations are presented in the Table below.   

Table A2. Categories for the analysis of the types of operations  

TOs  

Categories of types of operations  

TO 8/YEI  
(Categories 
expenditure   
102-108)  

of 

•  Work-based  learning:  this  includes  mainly  operations  concerning  on-the-job  training  and  other  forms  of  work 
experience,  such  as  traineeships,  apprenticeships,  internships,  community  work,  etc.  Principally  targeted  at  several 
types of unemployed, but also to inactive. Sometimes, these operations can be combined with training (for example in 
the form of internships at the end of training courses).  

219 European Commission: Directorate-General for Employment, Social Affairs and Inclusion, Atkinson, I., Andriescu, 
M., Spyridopoulos, K., Broughton, A. et al., Study supporting the preliminary evaluation of the support provided by ESF 
and FEAD under the Coronavirus Response Investment Initiatives (CRII and CRII+) – Final report, Publications Office 
of the European Union, 2023, https://data.europa.eu/doi/10.2767/160531 
220 FGB, Metis, Applica, Ockham, (2020), Study for the Evaluation of ESF Support to Employment and Labour Mobility, 
Brussels: European Commission. FGB, Metis, Applica, Ockham, (2020), Study for the Evaluation of ESF Support to 
Youth  Employment,  Brussels:  European  Commission.  ICF,  Cambridge  Econometrics,  Eurocentre  (2020),  Study 
supporting the 2020 evaluation of promoting social inclusion, combatting poverty and any discrimination by the European 
Social Fund (TO 9), Brussels: European Commission. Ecorys, Ismeri Europa (2020), Study for the Evaluation of ESF 
Support to Education and Training (TO 10), Brussels: European Commission. 
221 European Commission, Directorate-General for Employment,  Social  Affairs and Inclusion, Thijs, N., Nakrošis, V. 
(2020). Progress Assessment of ESF Support to Public Administration (PAPA): summary report,  Publications Office. 
Available at: https://data.europa.eu/doi/10.2767/03954 

94 

 
 
 
•  Employment  related  Training:  training  classes,  individualised  training,  vocation  training,  vouchers  for  training. 
These  operations  are  not  usually  related  to  general  education,  they  are  more  related  to  improve  professional  and 
occupation (or basic) skills. These operations can be targeted to unemployed, inactive and employed people. Often, 
they are combined with some form of guidance and support.  

•  Guidance and support for employment: career support, counselling, profiling, mentoring, skills assessment, personal 

development/support etc. Mainly covering the unemployed people. Often in combination with training.  

• 

• 

• 

• 

Financial incentives for employment: incentives scheme and subsidies for jobs, wage incentives, etc. The main targets 
are the unemployed and employers.  

Support  for  entrepreneurs:  support  for  self-employment/start-up,  entrepreneurship  training,  counselling  for  self-
employment,  promotion  of  entrepreneurial  culture.  This  type  of  operations  can  reach  different  target  groups.  It  is 
strongly related to the IP 8.iii (which is almost entirely dedicated to this type of operations).    

Support for the adaptation of workers and enterprises and for workers’ well-being: training, guidance, services, 
campaigns, health measures for older workers, etc. Mainly targeting employed and enterprises. Strongly related to IP 
8.v and IP 8.vi.  

Services  supporting  women’s  employment:  measures  for  work-life  balance,  childcare  services  and  other  actions 
supporting women’s employment. Mainly for women, but not only. These operations are strongly linked to the IP 8.iv. 
When  interventions  targeting  women  are  in  the  form  of  training,  incentives,  work-based  learning,  etc.  they  were 
classified in the previous types of operations.    

•  Capacity of labour market institutions: tools and instruments for PESs or other labour market institutions, human 
resources  training,  partnerships,  studies  and  services.  The  main  target  are  the  labour  market  institutions.  Strongly 
related to the IP 8.vii (which is almost entirely associated to this type of operation).  

• 

Support  for  entering  employment:  operations  aimed  at  reducing  barriers  to  employment  (guidance,  training, 
internships, incentives). Similar to several types of operations of TO 8, but more focused on vulnerable target groups 
(Long-term unemployed, disabled, people at risk of social exclusion, migrants, NEETS, etc.). Employment is the focus 
of these operations, which however could be also aimed at favouring activation and inclusion of people. In principle, 
IP 9.i and 9.ii are the main contributors to these types of operations.  

•  Enhancement of basic skills of vulnerable people: operations supporting vulnerable adults very far from the labour 
market. The main aim of these operations is not mainly related to the employment but to equipping people with basic 
and soft skills such as language, mathematical literacy, self-perception, self-efficacy, etc. In principle, IP 9.i, 9iii, and 
9iv are the main contributors to these types of operations.  

• 

Support for inclusive education in schools: operations supporting students with educational needs and at risk of early 
school leaving, favouring an inclusive education, eliminating segregation or disparities in schools, engaging families 
in education, etc. IPs 9.i and 9.iv. tend to be the main contributors to these types of operations.  

•  Access to inclusive services and the empowerment of people:  operations favouring social inclusion of vulnerable 
people (or preventing social exclusion since the young age) by enhancing the quality and the inclusivity of services. 
These  operations  can  support  services  of  general  interest  (childcare,  long-term  care,  elderly  care,  health…), 
health/services centres, integrated services, housing services. Furthermore, this category includes operations related to 
personal  targeted  services  aimed  at  facilitating  the  activation  and  empowerment  of  people.  These  operations  are 
targeted at individuals and entities. All IPs can contribute to this category, especially IPs 9.i and 9. iv.  

• 

Support  for  social  entrepreneurship:  operations  aimed  at  encouraging  the  integration  into  the  labour  market  of 
vulnerable people, through the support of social enterprises, and at favouring the growth of the social economy. The 
operations  support  both  people  and  enterprises  and  are  strongly  connected  to  the  IP  9.v  (which  almost  entirely  is 
dedicated to this type of operations).   

•  Operations influencing attitudes and systems: information campaigns for raising awareness about social inclusion 
or existing services, development of tools supporting coordination of social services (guidelines, protocols, other tools, 
training  to  professionals,  studies…),  operations  supporting  communities  (CLLD).  These  operations  target  mainly 
systems and entities, population in general, as well as territories. They can be found in all IPs.  

TO 9  
(Categories 
expenditure   
109-114)  

of 

TO 10  
(Categories 
expenditure   
115-118)  

Formal education and training  

of 

• 

Pre-primary  education  and  training:  operations  supporting  pre-primary  education,  targeting  different  groups 
(families, children, teachers, and other staff).  

95 

 
 
• 

Primary  education  and  training:   operations  supporting  primary  education,  targeting  different  groups  (families, 
children, teachers, and other staff).   

•  General  secondary  and  post-secondary  education  and  training:  operations  supporting  schools  and  students’ 
competences in secondary and post-secondary education (also in this case target groups are different: families, students, 
teachers, and other staff).  

•  Vocational education and training - secondary and post-secondary: operations supporting vocational education and 
training, work-based learning. They involve mainly learners, teachers, trainers, and other staff, but also families.  

•  Tertiary education and training: interventions and services supporting the higher education institutions; can include 

support for researchers and internationalisation of higher education institutions. Strongly related to IP 10.ii.  

Non formal education and training  

• 

Job  related  (non-formal)  education  and  training:  training  courses,  services,  individual  support  related  to  job, 
targeting unemployed, employed, and inactive. This type can include system actions related to the employment (such 
as system for forecasting professions, assessing labour needs…).  

•  Non job related (non-formal) education and training: basic education, interventions for life-long learning, etc. These 

operations target several types of groups, unemployed, employed and inactive.    

• 

Services to citizens and accessibility: operations enhancing the provision of public administration services to citizens 
and accessibility of official information and documentation; operations related to transparency and accountability are 
included in this category.   

TO 11  
(Categories 
expenditure   
119-120)  

of 

•  Civil servants and human resource management: operations supporting the development of PA staff’s technical 
skills, adaptation to innovative and/or reformed administrative procedures, and overall greater professionalization of 
civil servants. Dedicated training, education, competence appraisal frameworks all contribute to this category.  

•  Organization of government and policymaking: operations aimed at enhancing horizontal and vertical coordination 
between levels of government, reorganizing competences, and improving global efficacy of policymaking also through 
support of evidence-based measures.  

• 

Judicial  System:  operations  targeting  the  organization  of  courts,  procedures,  structure,  and  staff  which  aim  is  to 
streamline judicial system’s operation, improve functioning and efficacy, and avoid backlogging.  

Employment area 

•  Actions to protect jobs  

•  Actions to support workers  

•  Actions to support employers and the self-employed  

•  Actions to support NEET young people through the YEI  

•  Other employment actions  

Social inclusion 

ESF 
COVID-19 
related operations  
(Horizontal  to  the 
categories 
of 
expenditures), 
under TO 8-TO 11  

•  Actions to promote the social inclusion of vulnerable groups through providing direct targeted support  

•  Actions to promote social inclusion through ensuring access to services  

•  Other social inclusion actions  

Education and training  

1.  Actions to ensure the continuity of education and training  

2.  Equipment/other capital investment to ensure the continuity of education and training   

3.  Actions to increase the medical and social care workforce through fast-tracking curricula or qualifications  

4.  Other education training actions  

Healthcare 

•  Actions to support healthcare workers and patients  

•  Actions to support healthcare systems  

96 

 
 
  
  
  
  
  
•  Other healthcare actions  

The same categories used as for TO 8-TO 11 (see above).  

TO  13  –  REACT-
EU  
Source: Supporting study. Definition based on previous studies on TOs and on the “Study supporting the preliminary 
evaluation of the support provided by ESF and FEAD under the Coronavirus Response Investment Initiatives (CRII and 
CRII+)”.  

The classification of ESF operations  

Overall, the typology described earlier was applied to map in detail the individual operations contained in the 
provisional format lists as created from the lists published by the MAs. From an operational point of view, the 
assignment of each operation to a group was based on the analysis of three main fields available in the lists of 
operations:   

• 
titles of the operations,   
•  names of beneficiaries222, and   
• 
summaries of the operations.   

The method for classifying operations followed a semi-automatic procedure based on two main steps:   

• 

• 

first, a selection of keywords (see the table below) was used for an initial attribution of an operation 
to the appropriate type(s).  
second, the initial attribution was cross-checked by reading the summary of the operations and, 
where necessary, by consulting the relevant websites. Particular attention was devoted to the most 
relevant operations in financial terms and for all REACT-EU/CRII/CARE operations identified. 
Moreover, when the list of operations included information on the respective call for proposals 
(this is the case of Italy, Portugal, Croatia, Bulgaria, and Czech Republic), we consulted the call 
itself to better assign each operation to the relevant type.  

Table A3. Keywords used to assign operations to a certain type  

TOs  

Keywords  

TO 8-YEI  

TO 9  

TO 10  

TO 11  

work experience, work-based learning, internships/traineeships/apprenticeships, public employment, training/vocational training 
course/classes,  guidance  support/assistance,  mentoring,  counselling,  profiling,  skills  assessment,  skills  enhancement,  career 
support, incentives schemes, wage subsidies, financial aids/support, self-employment/business creation support, entrepreneurship 
support/training,  enterprises  creation,  continuing  training,  lifelong  learning,  health  ageing,  safety  jobs,  work  life  balance, 
childcare, gender pay gap, gender mainstreaming, institutional capacity, integrated services/support  
guidance support/assistance, counselling, basic skill, skills assessment/development, incentives/subsidies for job, training courses, 
internships/traineeships/apprenticeships/work experience, basic education, inclusive education, school completion/segregation, 
discrimination, parents/families engagement, health/social services, childcare, long-term care, elderly care, integrated services, 
social housing, social services, poverty, low-work intensity, social exclusion, material deprivation, deinstitutionalisation, social 
entrepreneurship/enterprises, social economy, corporate social responsibility, community-led local development, communities, 
disability/vulnerability, homeless support, Roma support, severe deprivation, addictions, labour market participation  
children/child care, early childhood education and care, kindergarten, infant, pre-primary, primary, elementary, secondary, high-
school, basic skills, early school leaving, special education needs, digital skills, teachers, trainers, vocational education, vocational 
training, dual learning, apprenticeships, internships, traineeships, tertiary, pre-tertiary, university, bachelor/baccalaureate, Master, 
PhD,  doctoral,  STEM,  graduation/degree,  continuing  training,  employees/unemployed  support,   enterprises/business  support, 
companies,  lifelong  learning,  non-formal  learning,  competences  recognition,  skill  development,  skills  intelligence,  skills 
forecasting  
services  to  citizens,  accessibility,  public  administration  services,  administrative  capacity,  institutional  capacity,  information 
systems, civil servants, administrative procedures, competences, judicial system  

Source: our elaboration  

222 This was important mainly in the field of TO 10, to distinguish the types of schools which were beneficiaries of funds, 

when reported clearly. 

97 

 
 
 
 
As mentioned previously, each operation was assigned to a specific type according to the kind of instruments 
used  by  the  measure  (TO  8),  the  ultimate  social  purpose  of  the  operation  (TO  9),  the  educational  level  it 
addressed (TO 10), and the domain to which it applied (TO 11). Given the essentially interpretative component 
of  this  exercise,  classifiers  were  instructed  to  categorise  according  to  the  predominant  character  of  the 
operation. However, since operations often featured projects with multiple logics of intervention, a secondary 
field for the assignment of categories was added; Column typeop2 was filled with the same criteria used for 
the primary type, and served as qualitative, ancillary labels for complex operations.  

TO  10  operations  occasionally  included  a  third  type,  also  inscribed  in  column  typeop2,  due  to  their 
classification hinging on the educational level they applied to; for instance, operations for the upskilling of 
kindergarten  teachers  that  could  also  involve  elementary  and  middle  school  teachers  would  receive  three 
categories,  the  main  type  being  0.preprimary  with  1.primary  and  2.generaleducation  both  reported  as 
secondary  type.  For  the  purpose  of this  study,  all  the  quantitative analyses  and elaborations of  operations’ 
categories  included  in  this  report  account  exclusively  for  the  main  typology  assigned,  whereas  secondary 
categories were useful descriptors to assist the qualitative interpretation of operations in other study processes.  

Finally, it is worthy of mention that since no degree of estimation is involved in these  calculations, a small 
error might only stem from the possible misclassification of individual operations. Nevertheless, the margin 
of error of this procedure is significantly smaller than those arising from estimation approaches that rely on 
samples of operations or on planned operations and consequently work on a smaller informative basis than this 
and run the same risks of misclassification.   

Identification of ESF operations related to CRII/CRII+, REACT-EU and CARE: issues, difficulties and 
strategies to overcome them  

The  identification  of  ESF  operations  related  to  CRII/CRII+,  REACT-EU  and  to  CARE  within  the  lists  of 
operations was problematic for several reasons:  

o 

•  As far as REACT-EU is concerned, the lists of operations according to the EU regulations do not have to 
contain information on the priority axis or TO, which are elements useful to identify the operations funded 
by REACT-EU. To overcome this potential barrier, we applied different strategies for each specific case:  
In  most  cases,  the  consultation  of  national  lists  of  operations  was  preferred  to  the  Kohesio 
database. MAs-published lists of operations usually contain more variables than the mandatory 
minimum requirements, which, in multiple instances, allowed the identification of REACT-EU 
operations223. Some notable examples are: the Italian lists of operations “Opencoesione” which 
clearly identified REACT-EU operations by including the “13.i” flag; in Portugal, the centralised 
web  page  hosting  all  list  of  operations  presented  a  specific  list  for  REACT-EU  projects;  the 
Slovenian,  Croatian,  Danish,  Estonian,  Latvian,  Romanian,  Swedish  and  Slovak  lists  reported 
information  on  the  Priority  Axis  or  TOs  related  to  each  operation;  the  Irish  list  contained  a 
“REACT-EU flag”; in Belgium the list of Wallonie OP had a flag for REACT-EU operations, 
while  in  the  case  of  the  Flemish  OP,  there  was  a  flag  for  REACT-EU  on  the  projects’  web 
repository; in Austria, the REACT-EU flag was contained in the code assigned to each operation 
and this was true also for some Spanish OPs (for example the Valenciana – ESF OP);  
In the other cases, we searched for the keyword “REACT-EU” in the name or in the summary of 
the operations and then cross-checked this information with the Annual Implementation Reports 
(updated at the end of 2021), as well as the information available on the OP web pages.  

o 

Finally, considering that not all the available lists of operations are necessarily updated, we carried out a final 
check based on a triangulation with SFC financial data updated at the end of 2022224. This was done for all 28 
countries, including those with lists of operations which enabled us to identify REACT-EU through different 

223 Given that the Kohesio project has just the aim to make transparent the ESF projects, these variables are not taken into 
account in the preparation of the Kohesio database, which in fact does not allow to  distinguish between ESF and ESF 
REACT-EU operations. 
224 “ESIF 2014-2020 categorisation ERDF-ESF-CF planned vs implemented” updated on 6 March, reporting financial 
data provided by the MAs in January 2023 and related to the end of 2022. Available at: https://cohesiondata.ec.europa.eu/ 
98 

 
 
 
flags (see above). When we identified some OPs not fully in line with the SFC financial data we checked other 
sources, such as AIRs and websites. For example: the Italian Opencoesione database, though reporting accurate 
data  on  REACT-EU  for  almost  all  programmes,  was  not  fully  updated in  the  case  of  the  Governance  and 
Institutional Capacity OP, in which there are EUR 509 million allocated to selected projects under REACT-
EU in SFC, but not in Opencoesione. In this case, we identified the REACT-EU projects for this OP on the 
internet225 and then added the information gathered to the Opencoesione list. France presented an analogous 
case  through  the  allocation  of  EUR  314  million  for  the  social  and  occupational  integration  of  vulnerable 
jobseekers via REACT-EU; while not reported in the OP list, the matching difference in eligible expenditure 
between OP and SFC data and explicit references to the measure in the relevant AIR were sufficient evidence 
to justify the operation’s manual addition to the country list. Such interventions were similarly performed for 
some Spanish OPs226, as well as for Cyprus, Germany, Hungary, Malta, Latvia, and Poland.  

These approaches enabled the mapping of 97% of the EUR 14 6 billion of ESF REACT-EU (TO 13) allocated 
to selected projects at the end of 2022.  

As regards the ESF operations financed in response to the COVID-19 pandemic, we relied on the findings of 
the completed CRII+ study, which identified 354 ESF “operations”227 financed in response to the COVID-19 
under TO 8-TO 11. Starting from the CRII+ study’s database, we searched for all the operations, identified in 
the context of this Study, in the ESF lists of operations. Different keywords were used to identify relevant 
operations (e.g., we searched for keywords such as “COVID”, “CRII”, “pandemic”, “corona”, “coronavirus” 
in the title and summary of the operation). We found almost 100% of all the operations already identified in 
the CRII+ study for nearly all the countries. In Italy, some operations were not identified, but this is probably 
due to the fact the CRII+ study mapped mostly the planned operations and not those actually implemented. In 
fact, it is worth noting that Opencoesione also contains a flag of ESF operations which were re-programmed 
in order to respond to the COVID-19 crisis on the basis of specific agreements between the MAs (national and 
regional) and the Ministry for the South and the Territorial Cohesion. The ESF amount related to COVID 
operations which we identified in Italy was fully consistent with the latest “Report on progress of the European 
Cohesion policy funds in Italy” (i.e., more than EUR 3 billion dedicated to CRII operations)228 meaning that 
our mapping of actually implemented operations (up to October 2022) was complete. Also, for Portugal, some 
operations mentioned in the CRII+ study, especially for the Lisboa, Norte and the Algarve OPs, could not be 
identified.  Nonetheless,  our  analysis  was  able  to  identify  operations related  to COVID-19  which  were  not 
mentioned in the CRII+ study. This applied also to Croatia and the UK under TO 888.   
Finally, with regards to the operations related to CARE, we  knew from the financial data provided by the 
MAs89 that only some of the OPs which adopted a specific axis (under REACT or not) for CARE operations 
had allocated resources to selected projects90 at the end of 2022, while others just planned the interventions91. 
However,  Member  States  could  allocate  resources  to  address  the  migratory  challenges  resulting  from  the 
Russia’s  war  of  aggression  against  Ukraine  just  by  reallocating  resources  between  the  existing  priorities. 
Therefore,  to  identify  CARE  operations  we  used  several  keywords  (e.g.,  CARE,  Ukrainian,  War).  This 
enabled, for instance, identification of  relevant operations in Sweden or Italy. Overall, it is worth noting that 
this  last  analysis  was  not  exhaustive,  since  the  lists  of  operations  did  not  yet  contain  information  on  the 
operations implemented in relation to CARE92; moreover, we could not rely on final implementation reports 
published within the evaluation timeframe to integrate or triangulate the analysis.  

of 

the 

List 

under 

projects 

REACT 

financed 

225 
http://www.pongovernance1420.gov.it/it/progetto_cat/react/ 
226 Especially the Valenciana OP, Pais Vasco OP, Extremadura OP and the Madrid OP. 
227 It is worth noting that the term operation in the CRII+ study means not strictly a single operation, as defined in the EU 
Regulations, but can mean “measures”, “calls”, “specific  objectives”, etc. It is also worth noting that the mapping of 
CRII+ study was made mostly on the base of operations programmes, therefore examining the planned action and not the 
real implemented operations at that moment. 
228 Il Ministro per gli Affari Europei, il Sud, le Politiche di Coesione e il PNRR, 2023, Relazione sullo stato di attuazione 
della 
23-27. 
nazionali 
https://www.agenziacoesione.gov.it/download/relazione-sullo-stato-di-attuazione-della-politica-di-coesione-europea-e-
nazionale-2014-2020/ 

Programmazione 

2014-2020, 

available 

coesione 

europea 

politica 

pages 

OP, 

at: 

di 

e 

99 

 
 
 
Concluding Remarks  

This annex sought to detail the essential steps required for the mapping of ESF-YEI operations in the 2014-
2022 evaluation period. The intent was to describe the approach adopted for each stage of the process, from 
selection of sources to the assembly of lists and, finally, the classification of operations itself.  

Some noteworthy country-specific specificities have been highlighted to capture the multitude of dissimilar 
management approaches carried out by MAs; while by no means exhaustive, the selection presented ought to 
suggest the operational flexibility needed to uniform and process a vastly heterogeneous dataset.  

The scale of this task was unprecedented; the process of manually analysing each operation and assigning it to 
a  certain  group  had  to  be  simultaneously  performed  across  all  TOs  in  all  Member  States  and  the  UK  and 
required  the  massive  effort  of  a  dedicated  team  of  classifiers  that  completed  the  task  over  the  span  of  six 
months.  

Overall, 1 025 234 operations were classified, covering 96.3% of total ESF-YEI expenditure and 83.7% of all 
operations for the 2014-2022 evaluation period. The broad coverage of operations and the depth of analysis of 
each project constituted an invaluable resource that provided significant added value to the evaluation in terms 
of detailed information on the distribution and specific features of actual ESF investments in the 2014-2022 
evaluation period.   

Analysis of the evolution of the socio-economic context (Task 2) 
The purpose of the analysis carried out was twofold: to give an account of the relevant socio-economic context 
at the beginning of the evaluation period and to describe its evolution over the years, highlighting the effects 
of the COVID-19 pandemic and the Russia’s war of aggression against Ukraine; and, to analyse the public 
expenditures in the labour, social and education area, at EU and national level, in order to identify general 
trends and to assess the added value of ESF funded interventions.  

The results of this analysis made it possible to identify the main trends, challenges and needs related to the 
Thematic Objectives, in order to assess the relevance of ESF support. The results also served accountability 
purposes, as they fed into the country fiches (Task 8), showing the role played by the ESF in each Member 
State.  Furthermore,  by  analysing  the  evolution  of  the  socio-economic  context  and  by  borrowing  from  the 
results of the mapping activity (Task 1) and insights from the case studies (Task 6), the flexibility of the ESF 
and its capacity to adapt was also investigated.  

The analysis also considered the role of external factors affecting the effectiveness and efficiency of the ESF. 
At the beginning of the 2014-2020 programming period the EU was still dealing with the long-term effects of 
the great recession of 2007/2008. After some years in which figures concerning employment and poverty had 
improved, in 2020 the COVID019 pandemic crisis was a watershed moment causing structural changes in 
social conditions of European people and deeply affecting the implementation of the ESF. The same is true for 
the Russia’s war of aggression against Ukraine and its consequences on the global economy. EU countries and 
regions  were  differently  affected  by  the  above-mentioned  external  factors229,  due  to  specific  contextual 
elements, which in some territories can positively influence and magnify the effectiveness of the ESF, while 
in  others  they  can  constitute  an  obstacle  to  ESF  implementation  despite  the  efforts  of  policy  makers.  The 
financial crisis, the pandemic, the Russia’s war of aggression against Ukraine and the associated rise in energy 
prices are the most relevant factors that have challenged the implementation of the ESF.  

For  this  reason,  we  considered  as  the  socio-economic  context  developed  over  the  programming  period  in 
particular pre- and post- the onset of the pandemic. 

The socio-economic context was analysed according to the following points:  

229  As  underlined  for  example  in  the  Eighth  Report  on  Economic,  Social  and  Territorial  Cohesion.  See: 

https://ec.europa.eu/regional_policy/sources/docoffic/official/reports/cohesion8/8cr.pdf  

100 

 
 
 
 
•  2013 was defined as a starting date for the analysis in agreement with DG EMPL230 to identify 
overarching change dynamics and the relative progress or regression of Member States (in the 
dimensions of GDP and productivity, employment, youth employment, education and training, 
social inclusion, governance indicators); 

•  The  analysis  was  contextualised  considering  external  factors  affecting  the  effectiveness  and 
efficiency of EU support, e.g., the COVID-19 crisis and, to the extent possible, the Russia’s war 
of aggression against Ukraine and the response through the Cohesion’s Action for Refugees in 
Europe. As regards the consequences of COVID-19, the analysis of the relevant indicators was 
done through comparison points, before and after the onset of the COVID-19 pandemic, in order 
to identify the most relevant changes. Furthermore, some specific indicators capturing the effects 
of the pandemic, as well as the Russia’s war of aggression against Ukraine, have been considered. 
•  A cross-country analysis was conducted per TO, to be able to assess, afterwards, the impact of 

context on effectiveness and efficiency of the ESF;  

•  The differences across Member States were taken into account and the main features of national 

socio-economic contexts and, where possible, regional contexts were identified; 

•  To be able to assess the role of the ESF in national policy frameworks, an analysis of expenditure 
levels  of  public  policy  related  to  employment,  health,  social  inclusion  and  education  was 
undertaken; 

•  The Country-specific recommendations from 2012 to 2020 (due to the COVID-19 crisis and the 
adjustments that occurred in the European Semester, 2021 CSRs focused mainly on fiscal policies) 
were mapped. The analysis mapped the relevant CSRs for ESF and their potential relevance for 
IPs covered in the evaluation, showing the differences among countries in terms of ESF policy 
areas most covered by the CSRs. The role of ESF in relation to the CSRs was then examined in 
other tasks such as Task 3 (consultation) and, especially, as part of the case studies (task 6). 

Socio-economic context data identification and collection 

The table below sets out a list of relevant indicators that were used, drawing from the socio-economic analyses 
carried out in the context of the previous preparatory studies commissioned by the Commission to support the 
ex-post evaluation231.  Many of  these  indicators  refer to  the  EU2020 strategy  (e.g.,  Employment rate  20-64 
years-old) as well as to other policy frameworks – the European Pillar of Social Rights in particular –, such as 
the indicator referred to people At Risk of Poverty and/or Exclusion (AROPE232).  

and 

Available 

230  When  relevant  indicators  to  be  used  in  the  analysis  are  not  available  for  2013,  we  will  consider  the  nearest  year 
available (for example, 2011/2012 or 2014/2015). 
231 For example: Fondazione G. Brodolini, Metis GmbH, Applica, Ockham IPS (2020) Study for the Evaluation of ESF 
Support to employment and Labour Mobility, for the European Commission, Directorate General for Employment, Social 
Affairs 
at: 
Inclusion. 
https://ec.europa.eu/social/main.jsp?catId=738&langId=en&pubId=8330&furtherPubs=yes 
European  Commission,  Directorate-General  for  Employment,  Social  Affairs  and  Inclusion,  Donlevy,  V.,  Sennett,  J., 
Georgallis, M. et al., Study for the evaluation of ESF support to education and training (Thematic Objective 10) – Final 
report, Publications Office, 2020, available at: https://data.europa.eu/doi/10.2767/301732 
Metis GmbH, Fondazione G. Brodolini, Applica, Ockham IPS (2020) Study for the Evaluation of ESF Support to Youth 
Employment, for the European Commission, Directorate-General for Employment, Social Affairs and Inclusion. p.141 
Available at: https://op.europa.eu/en/publication-detail/-/publication/d063ca61-bcd8-11ea-811c-01aa75ed71a1 
ICF, Cambridge Econometrics and Eurocentre (2020) Study supporting the 2020 evaluation of promoting social inclusion, 
combatting poverty and any discrimination by the European Social Fund (Thematic Objective 09). p.120. Available at: 
https://op.europa.eu/en/publication-detail/-/publication/8788ec85-2308-11eb-b57e-01aa75ed71a1/language-en 
Atkinson, I. et al (2023) Study supporting the preliminary evaluation of the support provided by ESF and FEAD under 
the 
at: 
Initiatives 
Investment 
https://ec.europa.eu/social/main.jsp?catId=738&langId=en&pubId=8542&furtherPubs=yes   
232 The AROPE indicator is measured as the sum of persons who are either at risk of poverty, or severely materially and 
socially  deprived  or  living  in  a  household  with  a  very  low  work  intensity.  The  AROPE  rate  is  the  share  of  the  total 

Coronavirus 

Available 

Response 

CRII+). 

(CRII 

and 

101 

 
 
 
 
Most indicators are relevant for more than a single TO: the unemployment rate, for instance, when broken 
down to account for specific target groups (young people, women, etc.) was also relevant for inclusion and 
equal  opportunities, and  non-discrimination (horizontal  principle).  Following the  suggestions made  by  DG 
Employment,  we  also  considered  persons  with  disabilities  and  migrants  when  looking  at  TO  8  and  TO  9 
indicators. 

Table A4. List of indicators on socio-economic context 

Indicator 

Source 

Level of analysis 

TO involved 

1.  Employment 

(20-64),  by 
rate 
educational attainment (lfsa_ergan) 

sex  and 

Eurostat 

2.  Activity rate by sex, age, education attainment 

(fsa_argan) 

Eurostat 

National/Regional 

TO 8, TO 9 

National/Regional  

TO 8 

3.  Unemployment  rate,  by  age,  sex,  education 

(lfsa_urgan) 

Eurostat 

National/Regional 

TO 8, YEI, TO 9 

4.  Long-term unemployment, 25-64 (une_ltu_a) 

Eurostat 

National/Regional 

TO 8, TO 9 

5.  Employment rate gap (in pp) between migrants 

and nationals (lfsa_ergacob) 

Eurostat 

National 

TO 8, TO 9 

6.  Self-employed 

total 
employment, 25-64 (lfsa_esgan and lfsa_egan) 

(%)  of 

share 

as 

Eurostat 

National  

TO 8 

7.  NEET  rate,  by  sex,  age,  labour  market  status 

(edat_lfse_20) 

Eurostat 

National/Regional  

TO 8, YEI, TO 9 

8.  Gender equality index 

European Institute for Gender 
Equality 

National  

TO 8 

9.  Gender employment gap (Women employment 
rates, % of total population, and difference with 
men, 25-64) (lfsa_ergaed) 

10.  Part-time employment as percentage of the total 

employment by sex (lfsa_eppgan) 

11.  Gender  pay  gap 
(earn_gr_gpgr2) 

in  unadjusted 

form 

Eurostat 

Eurostat 

Eurostat 

12.  Active ageing index by country 

DG EMPL and the Population 
Unit  of  the  United  Nations 
for 
Economic  Commission 
Europe (UNECE) 

National  

TO 8, TO 9 

National  

National 

National  

TO 8 

TO 8 

TO 8 

13.  Proportion of people at risk of poverty or social 
(ilc_peps01  and 

(AROPE) 

rate 

exclusion 
ilc_peps01n) 

Eurostat 

National/Regional*  

TO 9, YEI 

14.  Material 

and 

social 

deprivation 

rate 

(ilc_mdsd07) 

15.  Severe housing deprivation rate (ilc_mdho06a) 

Eurostat 

Eurostat 

National/Regional 

National  

TO 9 

TO 9 

population which is at risk of poverty or social exclusion. In 2021, the AROPE indicator was modified according to the 
new EU 2030 target (“the number of people at risk of poverty or social exclusion should be reduced by at least 15 million 
by 2030). The modifications concern the severe material deprivation component of the indicator and the definition of the 
(quasi)-jobless household sub-indicator. The new version of the indicator measures the social exclusion situation of those 
in the working age instead of 18 to 59. The new indicator is published by Eurostat with the code ilc_peps01n and the time 
series starts from 2015. 

102 

 
 
 
19.  Children  receiving  formal  childcare  services 

(silc_ats01) 

Eurostat 

National 

Indicator 

16.  Self-perceived health (hlth_silc_01) 

Source 
Eurostat 

17.  In-work at risk (of poverty) rate (ilc_iw01) 

Eurostat 

18.  Self-reported  unmet  for  medical  examinations 

(hlth_silc_08) 

Eurostat 

OECD 

OECD 

Eurostat 

Eurostat 

Eurostat 

20.  Population  by  educational  attainment  level 

(lfsa_pgaed) 

21.  Early school leavers by sex (% 18-24 years old 

population) – (edat_lfse_14) 

22.  Pupils aged between 4 years old and the starting 
age  of compulsory  education, by  sex  (as  % of 
the population of the corresponding age group) 

23.  Underachieving  15-year-old  students  (%)  by 

field: mathematics, reading, science 

24.  PISA  average  scores  for  15  years  old  in 

mathematics, reading, science 

25.  Population  aged  30  to  34  having  completed 
tertiary  or  equivalent  education,  by  sex 
(edat_lfse_03) 

26.  Students  in  tertiary  education  as  %  of  20-24 

years old population (educ_uoe_enrt08) 

27.  Participation rate (%) in education and training 
of  25-64  years  old  people  by  sex  and  level  of 
education attainment (trng_lfs_02) 

28.  Participation rate in education and training, by 
employment  status  (%  of  25-64  years  old 
population)  (trng_lfs_01)  and  by  degree  of 
urbanisation (trng_lfs_14). 

29.  Employment rate of recent graduates by sex and 
level  of  education  (levels  3-4,  general  and 
vocational; levels 5-8) – (edat_lfse_24) 

30.  Distribution  of  pupils  and  students  in  general 
and vocational programmes by education level 
(duc_uoe_enra13)  and  Exposure  of  VET 
graduates to work-based learning (only for 2021 
– edat_lfs_9919) 

Level of analysis 

TO involved 

National  

National 

National  

National 

National 

TO 9 

TO 9 

TO 9 

TO 9 

TO 10 

TO 10 

TO 10 

Eurostat 

National 

TO 10, TO 8 

Eurostat 

National/Regional 

TO 10, TO 9, YEI 

Eurostat 

National  

National/Regional 

TO 10 

National 

TO 10 

National 

TO 10, TO 8 

Eurostat 

National/Regional 

TO 10, TO 8 

Eurostat 

National/Regional 

TO 10 

Eurostat 

National 

TO 10 

31.  Disability employment gap by level of activity 

limitation and sex (hlth_dlm200) 

Eurostat 

32.  Excess mortality by month (demo_mexrt)** 

Eurostat  

National 

National 

TO 9 

All 

33.  Health  personnel 
(hlth_rs_prshp1)**.  

employed 

in  hospital 

Eurostat  

National  (not  for  all  countries 
available) 

TO 8, TO 9 

34.  Children  cared  only  by  their  parents  by  age 

group (ilc_caparents)** 

Eurostat 

National 

TO 8, TO 9, TO 10 

103 

 
 
Indicator 

35.  Index of total actual hours worked in the main 
job  by  sex  and  age  group  (2021  =  100)  – 
quarterly data (lfsi_ahw_q)** 

36.  Decisions  granting  temporary  protection  by 
citizenship,  age  and  sex  –  monthly  data 
(migr_asytpfm) *** 

37.  Decisions  granting  temporary  protection  to 
unaccompanied minors by citizenship, age and 
sex – monthly data (migr_asyumtpfm)*** 

Source 

Eurostat 

National 

Level of analysis 

TO involved 

Eurostat 

National 

TO 8 

TO 9  

Eurostat 

National 

TO 9, TO 10 

Note:  *Some  Regions  (NUTS  2)  missing.  **=Indicator  specifically  related  to  COVID-19.  ***=Indicator  specifically  related  to  the  Russia’s  war  of 
aggression against Ukraine. 

As far as institutional capacity (TO 11) is concerned, the available indicators from World Bank (Regulatory 
quality  and  Government  effectiveness)  and  from  University  of  Gothenburg  (the  European  Quality  of 
Government  Index)  were  used.  Data  from  the  European  Quality  of  Government  Index  by  University  of 
Gothenburg also enabled a specific focus on a regional level (NUTS2).      

Table A5. Indicators on administrative capacity (TO 11)  

Index/indicator 
Regulatory quality  
Government effectiveness  
European Quality of Government Index 

Source 
World Bank 
World Bank 
University of Gothenburg  

Level of analysis 
National  
National  
National/Regional 

Data on policy expenditures completed the set of information needed for the context analysis. This information 
enabled  an  analysis  of  the  role  played  by  the  ESF  in  each  country,  through  a  comparison  between  ESF 
expenditure  and  total  public  expenditure  in  employment,  education,  health  and  social  inclusion  at  country 
level.  

The table below summarises the data produced by Eurostat regarding expenditure, based on European policy 
taxonomies: 

•  Labour market policy database: for TO 8 and YEI we relied on the European Commission database 
on  expenditure  and  participants  of  labour  market  policies.  The  policies  are  classified  in  nine 
categories regarding employment service (cat. 1), active policies (measures, from cat. 2 to cat. 7) 
and  passive  policies  (support,  cat.  8  and  9)233.  Data  were  available  concerning  the  evaluation 
period  2014-2022.  For  the  comparison  with  YEI,  the  policies  addressing  young  people  were 
identified by looking at the age of participants234, as these are divided between under and over 25 
years old. 

•  Database  on  General  government  expenditure  by  functions:  statistics  on  General  government 
expenditure  by  functions  (COFOG)235  feature  data  at  EU  level.  Education  (GF08)  and  social 
protection  (GF10)  are  two  of  the  ten  functions  making  up  general  government.  By  applying 
national account indicators, it was possible to isolate the types of expenditures that were similar 
to ESF expenditure, for instance by taking out the compensations of the employees from the total. 

Table A6. Indicators on policy context  

Type of data 
Labour  market  active  and  passive  policies 
expenditure and participants by type of action 
General government expenditure by functions 

Source 

DG EMPL 

Eurostat 

Level of analysis 

National  

National 

TO involved  

TO 8, YEI 

TO 9, TO 10 

233 See https://webgate.ec.europa.eu/empl/redisstat/databrowser/view/lmp_expsumm/default/table. 
234 See https://webgate.ec.europa.eu/empl/redisstat/databrowser/view/lmp_partsumm/default/table.    
235 See https://ec.europa.eu/eurostat/databrowser/view/gov_10a_exp/default/table?lang=en  

104 

 
 
 
 
Consultation activities (Task 3) 
The aim of Task 3 was to undertake a set of interlinked targeted consultation activities to help gather the views 
and opinions of a wide range of relevant stakeholders of ESF and YEI programmes. The task had three strands: 
stakeholder consultations, a survey among Management Authorities and a public consultation exercise. Below 
is a summary of the activities completed. 

Stakeholder consultations 

In  total,  stakeholder  consultations  were  completed  with  150  stakeholders  at  EU  and  national  levels. 
Seventeen EU-level stakeholder interviews were completed over February and March 2023. The interviews 
included: 

•  Seven interviews with European Commission representatives involved in the management of 

OPs.  

•  Three interviews with EU social and economic partners.  
•  Six  interviews  with  different  civil  society  organisations  and  networks  representing  groups 

relevant to ESF and YEI support.  

•  One  interview  with  a  representative  of  a  research/academic  organisation  focusing  on 

employment, training, education and labour market policy.  

In  addition,  124  interview  sessions,  involving  133  individuals  were  completed  with  national  level 
stakeholders between April and June 2023, covering specifically: 

•  30  national  and  regional  authorities  involved  in  the  management  of  the  ESF  operational 
programmes,  including  ESF  Managing  Authorities  and  ESF  Committee  Members  at  national 
level (governmental representatives).  

•  70  national and regional ministries  and  other  public  institutions  responsible for  employment, 
training, education and labour market policy as well as ESF monitoring and implementation, 
including Ministries of employment and social affairs, finance and education, Public Employment 
Services, regional and/or local government bodies involved in the implementation of ESF OPs, 
Youth  Guarantee  coordinators,  ESF  national monitoring  committees, and  national  ESF  partner 
organisations. 

•  10  stakeholders  involved  in  labour  market  policies,  including  trade  unions,  employer 

organisations and chambers of commerce.  

•  14 national, regional and local level civil society organisations and networks representing groups 

relevant to ESF and YEI support.  

Additional national stakeholders were also consulted as part of the case study data gathering; this is outlined 
in the case study section below. 

MA Survey 

An  email  invitation  to  complete  the  survey  was  sent  directly  to  MAs  using  contact  lists  provided  by  the 
European Commission. After a fieldwork period of 10 weeks, the MA survey closed on the 2nd June 2023 
with 114 responses received in total. The survey achieved a 61% response rate, covering all EU27 Member 
States and the UK as shown in the Figure below. 

105 

 
 
 
 
 
 
 
 
Figure A. 3: MA Survey response rate by country  

Source: MA survey data 

Public consultation 

A total of 704 respondents took part in the public consultation which was open between 16 October 2023 and 
23 January 2024. The vast majority of responses (99%; n=697) came from within the EU (see Figure below), 
with all 27 Member States and the United Kingdom being represented. The highest number of responses 
came from Poland (43%; n=305), Spain (8%; n=59) and Germany (5%; n=36). Responses were also received 
from four non-EU countries,236 accounting for 1% of all responses combined (n=7). There was no evidence of 
a  coordinated  campaign  in  the  responses  received  from  Poland.  However,  the  preponderance  of  Polish 

236 The non-EU countries represented are the UK, Zimbabwe, Russia and Serbia. 

106 

 
 
 
 
 
 
 
 
respondents within the achieved sample distribution required careful triangulation with other sources when 
developing study findings, along with appropriate caveats being added when drawing on the results of the 
public consultation. 

Figure A. 4: Distribution of responses by country of origin 

Country of origin (N=704)

43%

50%

45%

40%

35%

30%

25%

20%

15%

10%

5%

0%

8%

5%

4% 4% 3% 3% 3% 2% 2% 2% 2% 2% 2% 2% 1% 1% 1% 1% 1% 1% 1% 1% 1% 1% 1% 1%

PL

ES DE BG BE

IT HU AT

FR EL HR CZ

IE

FI

LV LY

SL

SE

SK NL CY

LU MT RO EE DK PT

Responses  were  received  both  from  organisations  (86%;  n=607)  and  individuals  (14%;  n=97).  Among 
different  stakeholders,  the  highest  number  of  responses  (47%;  n=330)  came  from  public  authorities.  The 
second  largest  group  was  EU  citizens  (14%;  n=96).  Academic/research  institutions  and  non-governmental 
organisations  (NGOs)  respectively  accounted  for  10%  (n=72)  and  9%  (n=65)  of  respondents. 
Companies/businesses, business associations and trade unions together represented 6.5% of total respondents 
(n=46). The remaining 95 respondents included other categories of organisations and one non-EU citizen. 

Figure A. 5: Distribution of responses by type of respondent 

47%

0,5
0,4
0,3
0,2
0,1
0

I am giving my contribution as... (N=704)

14%

13%

10%

9%

3%

2%

2%

Public
authority

EU citizen

Other

Academic/
research
institution

NGO

Company/
business

Business
association

Trade union

In line with the Better Regulation requirements for public consultation, a full report and a factual summary 
detailing the  process and  findings  from  the  public  consultation  were  produced. Initial  drafts  of  these  were 
provided to the Commission on 5th March 2024. Following comments from the client team and Steering Group, 
a  final  version  of  the  Factual  Summary  was  produced  and  published  on  19th  March  2024,  to  meet  the 
requirement to publish this within eight weeks of the closure of the public consultation.  

107 

 
 
 
 
 
 
 
 
Cost effectiveness analysis (Task4) 
Cost-benefit analysis of ESF interventions was based on monitoring data on common output, result 
indicators and financial data. Where Programme Specific Indicator (PSI) output values were much 
higher than common indicator output values, PSIs were used. A limitation of the monitoring data 
was that finances, outputs and results are not reported at the level of individual interventions. Hence, 
where  relevant,  assumptions  were  made  about  the  types  of  interventions  within  an  Investment 
Priority based on characteristics of participants, because different interventions may achieve different 
results with different benefits. Based on those assumptions, the different operations are schematised 
into an IP-level intervention logic with aggregated benefits and costs.  

The  analysis  also  used  external  data  on,  for  example,  wage  levels  and  tax  rates,  as  well  as 
assumptions,  for  example about career  breaks, following  the  Better  Regulation  Guidelines  where 
relevant to calculate the so-called discount rate. Costs and benefits were estimated at the level of 
Investment  Priorities  and  Member  States,  and  only  at  the  country  level.  The  various  results  of 
interventions generated public benefits to the government and private benefits to the participant, and 
in  some  cases  additional  private  benefits  to  businesses.  Certain  benefits  and  costs  could  not  be 
monetised  (e.g.,  benefits  to  employers  with  the  exception  of  employee  training,  indirect  costs, 
macroeconomic knock-on effects, benefits for TO 11 on good governance and capacity building and 
for TO13 on REACT-EU) and therefore were only considered qualitatively. As a result, the monetary 
return calculations were affected by this limitation and the CBA calculations did not include them.  

The cost effectiveness of ESF is analysed per country and per Investment Priority in four steps, as 
technically summarised below.  

Step 1: estimating annually recurring benefits per successful participant 

What makes the exit from an ESF intervention successful depends on the type of intervention. Many 
interventions  aim  at  increasing  employment  and/or  earnings,  either  directly  or  indirectly  through 
education  or  training.  The  associated  annual  private  benefit  per  participant  is  the  increase  in 
employment likelihood multiplied by the increase in annual earnings: 
Annual private benefit = Δ employment likelihood x Δ net household income 
The change in net household income for unemployed and inactive participants is calculated as the 
difference between the net wages after the ESF intervention and the average social assistance level, 
assuming  due  to  lack  of  data  that  the  partner  is  also  unemployed  or  inactive,  because  otherwise 
inactivity may be a choice. For employed participants employee training, the change in net household 
income is estimated as the change in net wage assuming a partner who is also employed, because 
this  is  the  most  frequent  household  situation.  In  the  case  of  education,  the  benefits  are  further 
multiplied by the likelihood of successfully completing education.  
The main annual public benefits from employed workers considered in the analysis are:  
-  Higer income tax revenue 
-  Higher VAT revenue 
-  Reduced benefit expenditures  

In most EU countries, the income tax rate increases with the income level and assumptions will be 
needed about the average income of successful participants, and from there the applicable average 
income tax rate. VAT revenue increases with income from work because people with higher income 
have higher consumer expenditures. The share of household income spent on consumer goods and 
services can be calculated with (Eurostat or national) Household Budget Survey data. Although VAT 
rates vary between different types of goods, as an approximation the “standard” VAT rate could be 
assumed,  which  ranges  from  17%  in  Luxembourg  to  27%  in  Hungary.  For  participants  who  are 

108 

 
 
 
unemployed,  inactive  or  threatened  with  job  loss,  the  reduced  benefit  expenditures  can  be 
approximated with the average social assistance benefit. In formula: 
Annual public benefit = (income tax rate + VAT rate * consumption share) * annual private benefit 
+ Δ employment likelihood * average social assistance benefit level 
For employed participants not threatened with job loss, Δ employment likelihood is simply zero.  

Step 2: multiplying annually recurring benefits with a present-value factor 

The present-value factor is just one number to multiply annual benefits with to obtain a weighted 
sum of all future benefits, where benefits further in the future are weighted less as follows: 
Discounted benefit t years in the future = annual benefit / (1+r)t  
The larger the discount rate “r” or the further in the future “t” is, the smaller the discounted benefit 
is. Shortly before the retirement age, the number of future benefits is limited but the benefits are not 
discounted  much.  At  a  young  age  of  participants,  there  is  a  large  number  of  future  benefits  but 
benefits far in the future are discounted more heavily. In the yearly Education at a Glance reports, 
OECD uses a discount rate of 2% per year for formal education, perhaps reflecting that future benefits 
are considered to be quite certain. If future benefits are more uncertain, one may want to discount 
those more and assume for example a discount rate of 4% per year.  
In ESF monitoring data, the number of participants is broken down into three age groups: below age 
25, between age 25 and 55, and above age 55. For a participant exiting to employment, one career 
break is assumed as follows:  
-  For participants < age 25 (on average 22 years old): a career break between age 35 and 45 
-  For participants aged 25-55 (on average 40 years old): a career break between age 50 and 55 
-  For participants aged 55+ (on average 58 years old): no career break  

The present value of future benefits per successful participant can then be calculated as: 
Present value = ([% <25] * [PV <25] + [% 25-55] * [PV 25-55] + [% 55+] * [PV 55+]) * annual 
benefit 

Step 3: multiplying with net effectiveness rate 

A change in employment or income can generally be attributed to many factors. In a meta-study, 
JRC estimated the contribution of ESF to the employment likelihood, at the EU level and for different 
types  of  interventions  (across  all  IPs),  in  the  form  of  so-called  net  effectiveness  rates.  The 
introduction to Section 4.1.2.1 describes these in more detail. The private and public benefits are 
multiplied with these net effectiveness rates to estimate the ESF contribution to these benefits.  

Step 4 Comparing benefits and costs  

Some costs of the ESF interventions are borne by ESF, but especially for education and training some 
costs  are  borne  by  participants,  employers  and/or  the  State,  such  as  foregone  income  while  in 
education, lost productivity during employee training and the funding of formal education by the 
State. The non-ESF costs are subtracted from the benefits.  
The result (benefits minus non-ESF costs) is then compared to ESF costs in two ways: 
-  By also subtracting the ESF costs, giving the so-called return of ESF 
-  By dividing the benefits minus non-ESF costs by ESF costs, giving the benefit-cost ratio 

The  downside  of  the  return  is  that  it  does  not  show  at  what  amount  of  costs  the  benefits  were 
achieved. The downside of the ratio is that it will always be low-cost interventions that jump out with 
high ratios, and that the magnitude of negative ratios (if benefits are negative) is difficult to interpret.  

109 

 
 
 
 
Impact analysis (Task 5) 
Task 5 collected and analysed evidence on the impacts of ESF interventions at both a macro and 
micro level.  

With regard to the macro level, the task firstly provided the appropriate support to the JRC team 
working on the macroeconomic general equilibrium model RHOMOLO to produce estimates of the 
macroeconomic impacts of ESF interventions financed by the ESF and Youth Employment Initiative 
TOs 8, 9, 10 and 11. Concretely this involved providing annual and regionalised data – NUTS2 level 
– separately for each TO. Secondly, under this task the findings from RHOMOLO were analysed 
and used to answer to the evaluation questions related to the impacts of ESF. 

With regard to the micro level, the task first synthesised the findings of the study “Meta-analysis of 
the ESF counterfactual impact evaluations” and the later updated results237 on the effectiveness of 
the ESF and YEI operations on employment. Furthermore, in order to complement the quantitative 
findings of the meta-analysis, the task analysed in-depth the counterfactual impact evaluations (CIEs) 
on the ESF interventions implemented in the 2014-2020 programming period, which were available 
up to mid-2024 in the Evaluation Library produced by the Helpdesk project. The analysis focused on 
the  impact  for  individuals  participating  in  ESF  interventions  and  linked,  as  far  as  possible,  the 
findings of the CIEs to the type of operations identified in Task 1.   

Task 5 is articulated into two sub tasks: sub-task 5.1, focussing on the macro impacts (RHOMOLO 
model) and the sub-task 5.2, relating to micro impacts emerging from the ESF CIEs. 

Methodological note on the regionalisation of expenditure and common output 
indicators (Task 5.1) 

The regionalisation of expenditure and output indicators under Task 5.1 set out to circumscribe the 
resources allocated to and the population involved in ESF/YEI interventions by investment priority 
(IP) in each NUTS2 region of the 27 Member States and the United Kingdom.  

Regionalisation of ESF expenditure  

In the regionalisation of the ESF expenditure, we analysed the latest SFC2014 extractions which 
contained  data  on  expenditure  by  the  IPs  (Interventions  fields  from  102  to  120),  the  year  of 
implementation, the category of region and the related NUTS2 data for all the ESF OPs. With regard 
to location dimension (namely the pertinent NUTS2 region), the level of available data was often 
limited to NUTS1, or even at national level. As such, the regionalisation process required estimating 
representatively resources and participants to locations lacking NUTS2 data.  

We checked and adjusted data where possible to return NUTS0 and NUTS1 figures to the correct 
NUTS2. Several dissimilar occurrences were encountered: 

•  Data was not regionalised, but the Member State was a single NUTS2 Member State; 

this was the case for CY, EE, MT, LU, LT and LV. 

•  A regional OP covered only one NUTS2 location, but the MA assigned the amount to a 
NUTS code of a higher level (1 or 0). The data was easily corrected by applying the 
pertinent NUTS2 classification. 

237  The  initial  analysis  used  original  findings  from  the  study:  Ismeri  (2022),  ‘Meta  analysis  of  the  ESF 
counterfactual impact evaluations’ (VT/2020/052) custom-made by the European Commission – DG EMPL. 
Available  at  https://ec.europa.eu/social/main.jsp?catId=738&langId=en&pubId=8512&furtherPubs=yes.    It 
should be noted that updated analysis was then published using an alternative calculation method for average 
impacts: Haepp, T. and Serrano Alarcon, M., Meta-analysis of the European Social Fund counterfactual impact 
evaluations:  Brief  update  with  alternative  measures,  European  Commission,  Ispra,  2024,  JRC137571, 
Available at: https://publications.jrc.ec.europa.eu/repository/handle/JRC137571  

110 

 
 
 
• 

In several cases, only one NUTS2 location was eligible for funding as “less developed”, 
“transition”  or  “more  developed”  regions,  even  if  the  information  on  NUTS2  was 
missing in  the  SFC  dataset.  When this  was the  case, the  incomplete  information  was 
simply replaced by the correct NUTS2 code. 

Such adjustments allowed us to reliably assign non-regionalised expenditure at NUTS2 level. This 
was necessary to minimise the application of estimation techniques to monitoring indicators, as these 
techniques  always  imply  a  margin  of  error.  However,  whenever  the  disambiguation  of  non-
regionalised  indicators  at  NUTS2  level  was  not  possible  through  analytical  procedures  and 
assignments, we proceeded to an estimation procedure based on a double approach: 

•  For  the  programme  (and  IPs)  for  which  more  than  90%  of  funds  were  already 
regionalised, the non-regionalised share was broken down at NUTS2 level, using the 
NUTS2  distribution  of  regionalised  funds.  This  was  done  controlling  for  the  type  of 
region as well as the Investment Priority 

•  For the other OPs (and IPs), for which there was less than 90% of the total available 
NUTS2  distribution,  regionalisation  was  done  using  several  indicators  showing  the 
distribution  of  the  population  at  regional  level.  The  estimation  was  conducted  by 
applying the weights taken from the indicators related to the population for each OP, 
year and controlling also for the type of region. Several checks were then carried out to 
assess the internal validity of the estimation (for example expenditures by country and 
year, type of regions, OPs, comparing actual and estimated values).  
Indicators were regionalised according to the reference population for each IP, matched 
by logic and thematic proximity, as presented in the table below. 

• 

111 

 
 
 
 
Table  A7.  Reference  population  and  associated  IP/FOI  for  NUTS2  regionalisation  of  common  and 
financial indicators 

PI 

Field of intervention 

Variable used to regionalise 

8.1 
8.2 

8.3 
8.4 
8.5 
8.6 
8.7 
9.1 
9.2 
9.3 
9.4 
9.5 
9.6 
10.1 
10.2 
10.3 
10.4 
11.1 
11.2 
TA 
TA 
TA 

102 
103 

104 
105 
106 
107 
108 
109 
110 
111 
112 
113 
114 
115 
116 
117 
118 
119 
120 
121 
122 
123 

total unemployed and inactive 15-64  
total unemployed 15-24 , (or total unemployed and inactive people 15-64 when 15-24 figures are 
not available) 
total unemployed and inactive 15-64  
women unemployed and inactive 15-64 
total employed population  
total unemployed and inactive 15-64  
total unemployed and inactive 15-64  
people at risk of poverty or social exclusion 
Foreign citizens (or Population 15-64 when foreign citizens are not available) 
Population 15-64  
People at risk of poverty or social exclusion 
Population 15-64  
Population 15-64  
Population 0-19  
Population 20-29  
Population 15-64 
Population 15-64  
Total public Employment 
Total public Employment 
Population 15-64  
Population 15-64  
Population 15-64 

Note: given that the same ESF categories (from 102 to 120) were used for the financial resources under TO 13, we applied 
the same indicators seen above for the IPs from 8.i to 11.ii, since the IPs correspond to the categories of expenditures.  
Source: Supporting Study 

Regionalisation of common output indicators 

The approach used for the distribution of non-regionalised output indicators on participants hinged 
on the same procedures as above: 

• 

• 

assigned a single NUTS2 region to the Member States with a single NUT2 region (CY, 
LU, MT, EE, LV, LT) 
assigned a single NUTS2 region to the ESF regional OPs covering only one NUTS2 
region 
assigned a single NUTS2 when, in a country or within a NUTS1 region, only one NUTS2 
region which is a less developed, more developed or a transition region exists. 
The  output  indicators  that  could  not  be  regionalised  through  the  previous  methods  were 
distributed according to the quota of the regional expenditure, calculated as explained above. 
The following additional steps were performed: 

• 

•  The inconsistencies between expenditure and output indicators datasets were adjusted 
with  ad  hoc  verification,  including  mismatched  investment  priorities,  categories  of 
region, NUTS2 region, and REACT-EU interventions (these inconsistencies interested 
51 OPs in 16 Member States). 

•  Verification  of  the  overall  coherence  of  figures  and  results  with  source  materials, 
identifying  and  addressing  the  remaining  incongruities  (2  OPs  in  France  and  Spain 
respectively). 

Overall,  anytime  OPs  presented  a  degree  of  over  or  underestimation  of  indicators  following  the 
regionalisation process, Operational Programmes and Annual Implementation Reports (AIRs) were 
consulted. The presented configurations of allocated funds on priority axes and selected IPs allowed, 
in line with the assumptions above, the estimation of the regional concentration of indicators based 
on the relative financial weights of specific IPs in pertinent NUTS2 regions of the same category. 

112 

 
 
REACT-EU  monitoring indicators  faced two  key  issues  for  regionalisation.  Firstly,  all indicators 
were assigned to investment priority 13i, hindering their attribution to specific intervention fields 
and  local  dimensions.  Secondly,  the  region  categories  flagged  for  REACT-EU  measures  posed 
geographic  challenges  for  output  indicators.  Addressing  these  issues  was  essential  to  inform  the 
regional  concentration  of  outputs.  The  consultation  of  Operational  Programmes  and  Annual 
Implementation  Reports  (AIRs)  allowed  these  gaps  to  be  filled.  The  primary  asset  in  this  regard 
concerned  the  distinction  of  allocations  by  funds  and  priority  axes,  marking  a  first  distinction 
between ordinary and REACT-EU indicators; indications about investment priorities, categories of 
region, and location dimensions, in most cases, were reported following the ordinary procedures, 
which  limited  the  degree  of  uncertainty  to  the  specific  weight  of  distribution  within  clusters  of 
regions  in  the  same  category.  Drawing  from  the  financial  configuration  of  expenditure  in 
approximate  NUTS2  according  to  investment  priority  or  categories  of  region,  distribution  was 
calculated  and  assigned  to  output  indicators  lacking  location.  Notably,  incongruities  between 
financial  and  output  indicators  datasets  also  implied  common  indicators  under  IPs  which  had  no 
certified expenditure allocation. In these instances, provided that no additional information could be 
surmised from OPs or AIRs, regionalisation criteria observed the concentration of the population 
with selected characteristics in the affected NUTS2 regions. 

113 

 
 
Impact analysis (Task 5.2) 

Task 5.2 analysed the impact of ESF interventions on participants, based on reliable impact assessments 
carried out by Managing Authorities during the evaluation period, using either a counterfactual, theory-
based or qualitative/quantitative approach. The analysis also considered a sample of implementation 
evaluations  with  the  aim  of  covering  those  interventions  or  countries  not  tackled  by  reliable  impact 
evaluations. 

Overall approach 

Task 5.2 was undertaken through a qualitative and narrative approach based on a literature review of 
ESF  and  YEI  impact  and  implementation  evaluations.  The  review  of  relevant  evaluations  -  either 
counterfactual,  theory-based  and/or  qualitative/quantitative  –  addressed  the  following  evaluation 
questions in a narrative way: 

•  How and to what extent does the ESF contribute to the achievement of the objectives of 
YEI, TO 8, TO 9 and TO 10, in terms of a positive evolution of the employment situation, 
the quality of jobs created, labour mobility, self-employment, the competences of people, 
early school leaving, social inclusion, etc.? 

•  Which types of operations were the most effective and most sustainable? 
•  For which groups and in which contexts (e.g., more developed, less developed and transition 

• 

regions) are the ESF interventions more effective? 
(partly) What main factors (geographical, socioeconomic, organizational, etc.) contribute to 
explaining the impacts of the ESF operations? 

The analysis was based on secondary information sources, namely evaluation reports drafted during the 
2014-2022 period and collected in the Evaluation Library produced by the Evaluation Helpdesk project. 

The analysis was not limited to counterfactual evaluations, but also included findings from other types 
of impact and implementation evaluations, thereby including evidence from mixed-method approaches. 
This enabled coverage of all evaluation criteria and task 5.2-related questions foreseen for the study. 

Selection of evaluations through the Evaluation Helpdesk database 

The main information source is the Evaluation Helpdesk database238, which collects and categorises all 
evaluation reports published by MAs across EU Member States and the United Kingdom. Since 2015, 
the database has collated published evaluation reports relating to Operational Programmes implemented 
in Member States.  

Each month, new reports are analysed by the Helpdesk team, summarised, and assessed based on five 
main criteria: 

•  Clarity and suitability of the evaluation design. 
•  Appropriateness and correctness of the techniques applied. 
•  Data quality. 
•  Validity of findings. 
•  Validity of policy implications. 

238 This library contains the evaluations carried out by Member States on interventions supported by cohesion 
policy in the 2021-2027 and 2014-2020 periods and the evaluations assessing the impact of investments from the 
2007-2013  period,  published  since  2015  See:  https://ec.europa.eu/regional_policy/policy/evaluations/member-
states_en 

114 

 
 
 
These five criteria are given a score, ranging from 0 (when a specific element is missing) to 4, when the 
report is well detailed and relied on thorough analysis. The average score was then used to categorise 
evaluations as “reliable” (when the average score is above 3) or “not reliable” (average score below 
2.75). This assessment by the Evaluation Helpdesk’s experts is not available to the wider public and it 
was  only  used  by  the  European  Commission  for  analytical  purposes.  To  ensure  the  robustness  and 
reliability of the evidence used for this report, only evaluations assessed as reliable by the Evaluation 
Helpdesk were considered. 

As  of  September  2023, the  Helpdesk  database included  362  impact  evaluations covering  2014-2020 
ESF/YEI programming, as well as multi-funded programmes (ESF and ERDF and/or CF). Among them, 
66 evaluations were not reliable (scoring from 1.25 to 2), while 205 were categorised as “partly reliable” 
(between 2.25 and 2.75), as they feature some deficiency in the methodology adopted, or in relation to 
the quality and reliability of data used. The remaining 91 evaluations were those considered as reliable 
by  the  expert reviewers  (average  score  3-4).  Nevertheless,  only 79  out  of the  91  impact evaluations 
identified could effectively contribute to this analysis, as some multi-funded impact evaluations do not 
report any relevant findings on ESF/YEI interventions.  

Given the above, our team identified 12 additional reliable implementation evaluations (ongoing, mid-
term, in-depth analysis on specific topics, etc.) to gather further evidence on the results of ESF/YEI-
funded  interventions  analysed  through  different  evaluation  methods.  This  approach  enabled  a  wider 
range of evaluations to be analysed, including some Member States not covered by impact evaluations 
(Denmark, Finland, Greece, Hungary and the Netherlands). Overall, 91 evaluations (both impact and 
implementation evaluations) were reviewed. 

The  selected  evaluations  covered  18  countries  (17  EU  Member  States  and  the  UK),  guaranteeing  a 
balanced representation at geographical level (Northern, Southern, Eastern and Western Europe). They 
also included all types of regions (more developed, less developed and in transition) and a range of OPs 
(at national and regional levels).  

The available evaluations mainly focused on TO 8 (54% of selected evaluations), followed by TO 9 
(22%) and TO 10 (21%). Since the sample included only a few evaluations addressing TO 11, which 
were not statistically relevant and whose findings were limited, not allowing for comparisons, TO 11 
was not part of the exercise. Further details on the evaluations’ featured are given in Annex IV, Appendix 
5. 

Additional methodological considerations 

It is important to note that, unlike the meta-analysis of CIEs undertaken previously by Ismeri239, the 
present report was based on findings from reliable impact and implementation evaluations carried out 
through  different  methodologies,  thus  privileging  qualitative  information,  while  quantitative  results 
were analysed primarily through updated monitoring data (result indicators).  

As highlighted in previous sections, selected evaluations involved 18 countries (17 EU Member States 
and the UK) covering all categories of regions, alongside OPs implemented at both national and regional 
levels. Nonetheless, some countries (Italy, Poland, Germany) were more represented than others because 
of their higher number of reliable evaluations. 

Generally, selected evaluations covered TOs 8, 9 and 10 and all the evaluation criteria. Nevertheless, in 
most cases findings were focused on specific topics following evaluations’ specific objectives. While 
effectiveness is tackled in all reports, other themes such as sustainability, visibility and communication 
were  addressed  much  less  frequently.  Therefore,  findings  on  these  criteria  were  limited,  context-
specific, and could not be generalised to the programme level. In addition, EU added value was inferred 
to the extent possible by considering information related to other evaluation criteria, while not generally 

239 Ismeri (2022), ‘Meta analysis of the ESF counterfactual impact evaluations’ (VT/2020/052) custom-made by 
the 
at 
https://ec.europa.eu/social/main.jsp?catId=738&langId=en&pubId=8512&furtherPubs=yes 

Commission 

Available 

European 

EMPL. 

DG 

– 

115 

 
 
being directly tackled in the analysed evaluations. Moreover, since it was decided to omit evaluations 
also  tackling  the  2007-2013  period,  this  report  focused  on  the  2014-2020  period,  not  providing 
comparisons between the two programming cycles. 

Although  68%  of  the  selected  evaluations  were  published  after  2020,  most  findings  relate  to 
interventions implemented before that year (thus not considering the effects of the COVID-19 pandemic, 
nor  the  Russia’s  war  of  aggression  against  Ukraine).  For  the  same  reason,  the  evaluations  typically 
covered operations that were still being implemented and/or refer to closed operations whose durable 
effects could not be comprehensively assessed. Therefore, evidence was triangulated with findings from 
other relevant tasks to ensure a more comprehensive response to the evaluation questions. 

Overview of selected evaluations 

Main characteristics 

The following figure shows the number of selected impact and implementation evaluations per country 
included  in  the  Helpdesk  database.  Among  these  countries,  Italy  (17  evaluations),  Poland  (17 
evaluations), Germany (15), and Portugal (6 evaluations) are the mostly represented.  

Figure A. 6: Number of impact and implementation evaluations selected by MS 

15

17

17

3

2

1

1

1

2

1

2

4

2

2

4

3

1

2

1

1

2

1

6

18
16
14
12
10
8
6
4
2
0

Source: Evaluation Helpdesk database, September 2023 

Around  52,7%  evaluations  (48  out  of  91)  covered  measures  financed  by  regional  OPs,  while  the 
remaining 47,25% of analysed evaluations related to measures financed by national OPs. 

As for the distribution of regional OPs impact and implementation evaluations, 47,9% of them were 
referred to more developed regions (23 evaluations), followed by evaluations referring to in transition 
regions (11 evaluations), and evaluations referring to less developed regions (14 evaluations). 

The sample mostly included ESF evaluations (68%) followed by evaluations considering ERDF/ESF 
evaluations (11%), and ESF/YEI evaluations (10%), as shown in the figure below.  

Impact and implementation evaluations were referred to measures that can cover one or more TOs. Most 
evaluations covered TO 8 (47%, namely 37 evaluations), followed by TO 9 (30,7%, with 24 evaluations) 
TO 10 (19,23%, with 15 evaluations), and, at last, TO 11 (covering 2,5% of the evaluations). 

The selected evaluations were divided into two groups based on the type of measure they focused on 
(i.e., from which funds the measures analysed are covered):  

• 
• 

evaluations covering measures financed by ESF, CF and ERDF, 
evaluations covering measures financed by ESF, CF and ERDF + YEI. 

It is worth noting that the Helpdesk evaluation database categorises evaluations according to the type of 
evaluation, namely: impact evaluations (I), process evaluation (P), and monitoring evaluation (M). 

116 

 
The following analysis on impact and implementation evaluations by type of measure/Fund addresses 
three main aspects: type of evaluation, adopted evaluation method(s) and used data. 

ESF/YEI evaluations 

There were 72 ESF Impact and implementation evaluations (not including YEI which is analysed in the 
following paragraph). The analysis presented herein focused on mixed types of evaluations. The joint 
use  of  impact  and  monitoring  evaluations  was  the  most  frequent  case  (I+M),  representing  29%  of 
evaluations.  This  case  was  followed  the  combination  of  impact,  monitoring  and  process  evaluations 
(I+P+M), covering 22% of evaluations (16 evaluations), and impact evaluations (I), corresponding to 
the  21%  of  the  total  sample  of  evaluations  (15  evaluations).  At  last,  the  less  frequent  cases  were 
represented by the joint use of impact (I) and process (P) evaluations (13%, namely 9 evaluations), the 
joint use of process and monitoring evaluation (P+M), corresponding to 10%, monitoring evaluations 
(M, namely 4%), and process evaluation (P, 1% evaluations). 

The  methods  used  for  the  selected  impact  evaluations  were  counterfactual  approach  (counterfactual 
impact  evaluation  -  CIE),  qualitative  methods  (QUAL),  quantitative  methods  (QUAN),  cost-benefit 
analysis  (CBA),  theory-based  evaluation  (TBE)  and  macro  modelling  (MOD).  The  most  frequent 
methods used, either alone or combined, were the following: 

• 

• 

• 

• 

• 

and 

counterfactual  approach 

joint  use  of  qualitative  quantitative  methods 
(QUAL+QUAN+CIE), adopted in 17 evaluations. 
joint  use  of  qualitative  and  quantitative  methods  (QUAL+QUAN),  adopted  in  13 
evaluations. 
and 
(TBE+QUAL+QUAN), adopted in 12 evaluations. 
joint use of theory-based evaluation, counterfactual approach, qualitative and quantitative 
methods (TBE+QUAL+QUAN+CIE), for a total of 8 evaluations. 
counterfactual approach (CIE), 6 evaluations. 

theory-based  evaluation,  qualitative  and  quantitative  methods 

joint  use  of 

There were 19 Impact and implementation evaluations covering measures financed by YEI, alongside 
other funds. The most frequent case was represented by impact evaluations (I), covering the 32% of the 
evaluations (namely 6 evaluations), followed by impact and monitoring evaluations (I+M), covering the 
26% of the evaluations (5 evaluations),  and by impact, monitoring and process evaluations (I+P+M), 
in total 4 evaluations (21%), and lastly, impact and process evaluations (16%), namely 3 evaluations. 

As for the methods adopted, the most frequent case in the sample under analysis was the combination 
of  counterfactual impact evaluation and quantitative methods  (CIE+QUAN),  with  6  evaluations.  All 
other combinations of evaluation methods were generally less frequent (1 or 2 impact evaluations). 

As previously discussed, implementation evaluations were taken into account for some of those Member 
States for which reliable impact evaluations were not identified, in order to still provide reliable hints 
on the main results of the ESF and YEI measures. Implementation evaluations covered 5 Member States: 
Denmark, Finland, Greece, Hungary and the Netherlands. 

Most  implementation  evaluations  (8  out  of  12)  covered  measures  financed  by  national  OPs,  and  4 
implementation evaluations cover measures financed by regional OPs.  

117 

 
 
 
 
 
 
 
Integration of the meta-analysis of ESF CIEs findings 

The  analysis  also  took  into  consideration  the  original  findings  of  the  meta-analysis  of  the  ESF 
counterfactual impact evaluations240 and its update implemented in 2024241. It is worth recalling that, 
unlike the present analysis based on impact and implementation evaluations carried out through a variety 
of methods during the 2014-2020 programming period, the meta-analysis focused only on CIEs realised 
in the 27 EU Member States and the UK covering both the 2007-2013 and 2014-2020 programming 
periods.  

The meta-analysis aimed at finding systematic evidence of the effectiveness and sustainability of ESF 
and YEI operations, covering TO 8 (Employment), TO 9 (Social inclusion), and TO 10 (Education and 
training) and analysing effects in terms of employment and education outcomes, while it did not tackle 
social inclusion outcomes. Moreover, most of the evaluations analysed in the meta-analysis report were 
related to interventions implemented before 2020, thus assessing interventions not yet impacted by the 
COVID-19 pandemic nor by the Russia’s war of aggression against Ukraine.  

For this reason, the European Commission – DG EMPL entrusted the JRC to provide an update on the 
descriptive  statistics  presented  in  the  2022  meta-analysis  by adding  16  CIE  reports  published  in the 
Helpdesk database between January 2022 and November 2023 to the previously collected 94 CIE reports 
on  employment  outcomes.  This  informed  an  update  of  the  previously  calculated  average  effect  on 
employment rates based on all available studies. The updated analysis provided figures based on both 
the  previous  methodology  (which  considered  all  coefficients  provided  in  the  reports  that  estimated 
effects, allowing for a large variation in the number of coefficients provided by each report leading to a 
higher role of certain reports in the calculation of the general overall statistics for the ESF), as well as 
on a balanced methodology that gave an equal weight to each intervention irrespective of the number of 
coefficients provided in the report that fit the three pre-specified categories of the meta database (whole 
sample, gender subsamples, age group subsamples).  

The analysis considered data on the average effects on employment probability by intervention type (not 
broken down by gender and age group) provided by the updating of the initial study made by JRC in 
January 2024, while outcomes on target groups and on the dynamics related to programme duration and 
the time horizon were derived from the 2022 meta-analysis. 

Finally, to ensure continuity in  the evaluation of ESF/YEI outcomes and coherence with the overall 
methodology  adopted,  the  analysis  considered  updated  figures  based  on  the  previously  adopted 
methodology only for the 2014-2020 programming period. The following table summarises the main 
findings from the meta-analysis adopted for this report, which includes the updating by JRC in 2024 and 
original findings of the original meta-analysis carried out in the 2022. 

Table A8. Main findings from the meta-analysis on CIEs  

Findings of CIEs on employment outcomes 

Overall effect on employment 
probability 
(2024) 

Average programme effect is a 5.9 percentage points increase in the employment probability of 
treated individuals 

Most effective types of 
operations (average effect) 

11.9 percentage points for employment subsidies/financial aid 
9.5 percentage points for traineeships 

240 Ibid. 
241  Haepp,  T.  and  Serrano  Alarcon,  M.,  Meta-analysis  of  the  European  Social  Fund  counterfactual  impact 
evaluations: Brief update with alternative measures, European Commission, Ispra, 2024, JRC137571, available at: 
https://publications.jrc.ec.europa.eu/repository/handle/JRC137571 

118 

 
 
 
(2024) 

6.9 percentage points for vocational training 

Most effective types of 
operations (balanced average 
effect) 
(2024) 

15.1 percentage points for employment subsidies/financial aid 
6.7 percentage points for traineeships 
5.2 percentage points for vocational training 

Least effective types of 
operations (average effect) 
(2024) 

5.7 percentage points for other training 
3.8 percentage points for mentoring/social support 
0.1 percentage points for public employment 

Least effective types of 
operations (balanced average 
effect) 
(2024) 

Highly affected target groups 
(2022) 

4.9 percentage points for other training 
2.2 percentage points for mentoring/social support 
-10.2 percentage points for public employment 

Unemployed: 55% of the reported estimates are positive significant, and 21% are negative 
significant 
Long-term unemployed: 51.9% of the estimates are positive significant and 21.1% negative 
significant 
Atypical workers: 44.9% of estimates are positive significant, and 20.2% are negative significant 

Less affected target groups 
(2022) 

Youths (up to 25 years old), with only 42% of the estimates being positive significant and 29% 
negative significant (but effect sizes increase over time following the programme, making the 
investment worthwhile for youths due to the extended payoff period). 

Effects by gender 
(2022) 

Women, 46% of the reported estimates are positive significant, while 21.4% are negative significant 
Men, 50% of the reported estimates are positive significant, while 22.2% are negative significant 

Dynamics related to programme 
duration and the time horizon 
(2022) 

The longer the intervention’s duration (25-48 months), the smaller the effects. 
Shorter interventions (up to 24 months) seem more effective 

Findings of CIEs on education outcomes 

Main target groups 
(2022) 

Unemployed  
NEETs 

Impacts of the interventions 
assessed 
(2022) 

Mixed effects (negative and positive): no type of support, more positive than others, emerged from 
the analysis. For instance, several evaluations found positive effects for interventions targeting the 
unemployed and the NEETs (i.e., vocational training). Still, this same typology had negative impacts 
in some other CIEs analysed. 
High shares of not statistically significant estimates on the short or medium period suggesting that 
interventions are likely to produce any real effect only in the longer run. 

Case studies (Task 6) 
The aim of the case studies was to add considerable depth to the overall study’s understanding of the 
delivery and results of the ESF by Thematic Objective and Investment Priority. They also sought to 
offer a specific, detailed, assessment of the implementation and results of the CRII, REACT-EU, and 
CARE, alongside exploring the effects of social innovation and the horizontal principles in the ESF 
context.  

27 case study reports were produced as follows:  

•  20 IP level case studies 
•  3 case studies evaluating the key crisis response instruments (CRII/ CRII+, REACT-EU and 

CARE) 

119 

 
 
 
•  1 case study focused on social innovation, and  
•  3  case  studies  focused  on  horizontal  principles  (partnership  and  multilevel  governance, 
equality between men and women and non-discrimination, and sustainable development). 

A sampling exercise identified 5-6 countries that would be the focus of country level research that fed 
into each case study. In total, 380 country level stakeholder interviews were completed across the case 
studies. Alongside interviews, country experts conducted desk-based research, including reviewing the 
relevant  data  from  this  current  study  and  identifying  and  reviewing  Member  State  level  secondary 
evidence.  The  latter  included  Member  State-level  ESF  and  YEI  evaluations,  any  relevant  academic 
research conducted at the Member State level, and any additional and relevant background documentary 
sources at national level that could inform the context presented within the case studies. This country 
level evidence was collated by country experts into key findings notes. In total 224 key findings notes 
were completed, compiling this country level information for each IP/initiative (e.g. CRII)/cross-cutting 
theme (e.g. social innovation) and quality assured by the core research team. 

The key findings notes were one of the principal sources of evidence used by case study leads to produce 
each case study report. In addition, case study leads had access to IP level/crisis response data collected 
and analysed in other strands of the study, such as financial implementation and types of operations 
data. It should be noted that the case study reports were produced between November – January 2024 
and as such drew on wider data and findings available at this point of the study. The case study reports 
were intended to be a data source in themselves, rather than a specific reporting output; as such the data 
presented in the case study reports has fed into analysis and triangulation rather than being updated with 
new findings from other evaluation activities. 

Focus groups (Task 7)  
After the submission of the draft final report, six thematic focus groups were carried out between June 
4th and June 13th 2024, four covering TO 8-11, one for YEI, and one addressing the reaction to the crises 
(REACT-EU/CRII/CARE). All of them involved relevant stakeholders to fill remaining knowledge gaps 
and their findings were integrated into the final report. In total, 62 attendants participated across the six 
focus groups. Focus groups were run online as a cost-effective and accessible approach to engaging 
participants.  

Identifying and securing participants  

Each  focus  group  brought  together  between  6  to  10  representatives  from  stakeholders  active  in  and 
knowledgeable about ESF and YEI, and of the changes brought by the REACT/CRII/ CARE to their 
implementation including:  

•  National Managing Authorities of the programmes 

•  Representatives  of  national  ministries  or  national  ministries  of  education,  employment  and 

social affairs  

•  Regional  and  local  government  bodies  in  charge  of  employment  and  social  policies  and/  or 

involved in ESF design or implementation 

•  Representatives  of  organisations  involved  in  the  delivery  of  operations  as  beneficiaries  or 

project partners  

•  National public employment services 

•  EU-level civil society organisations and networks representing groups relevant to ESF 

•  Academics or independent experts 

•  Representatives of Commission services 

120 

 
Potential participants were proactively identified during the delivery of the research and a longlist of 
participants developed in close consultation with DG EMPL. This approach aimed to ensure a balanced 
representation  across  the  EU  and  a  diversity  of  views;  while  making  sure  the  overall  number  and 
selection of participants encourages effective discussions where everyone is given the opportunity to 
actively participate.  

Developing the focus group, concept note, agenda and discussion note 

A concept note, including the overall agenda of the focus groups, was discussed and agreed with the 
Commission. The concept note outlined the overarching approach and agenda as well as any specificities 
for the different focus groups, including an assessment of priority topics and findings for consideration 
by focus groups based on the research results and input from the Commission.  

Focus groups were broken down into two main sessions: the first part helped test the study’s emerging 
findings and conclusions, including identifying information to fill gaps or areas for development; and 
the second aimed to discuss learning points and policy implications for the future. The draft agenda was 
updated based on the findings from the research and adapted to reflect the specificities of each of the 
focus groups.  

Before the focus groups, the core study team sent participants a short discussion note outlining the key 
objectives and expected results of this event, an overview of the key findings, the draft conclusions and 
recommendations of the study, of specific relevance to the topic of each workshop as well as transversal 
themes.  The  discussion  paper  also  signposted  key  questions  and  issues  for  discussion  with  the 
participants. A GDPR compliant privacy statement was also provided to participants.  

Organisation and delivery of the focus groups 

Focus groups were held in English, without interpretation. Workshops were supported by a moderating 
team, including technical support to ensure active participant engagement. Microsoft Teams was used 
to run the focus groups as it provides certain interactive features that allowed for smooth running and 
interactive facilitation of sessions. The contributions of each participant were noted for reporting. With 
the explicit permission of the participants, the focus groups were audio-recorded to allow checking of 
notetaking accuracy and/or to facilitate the production of anonymised quotes.  

Focus group reports  

The findings from the focus groups were used to sense-check, fine-tune and fill gaps in the findings and 
conclusions  as  well  as  inform  the  key  learning  points  included  in  the  Final  Report.  Findings  and 
participation  details  from  all  the  focus  groups  were  compiled  into  the  consultation  synopsis  report 
included in the Annex IV. 

Key figures and evidence (Task 8) 
The aim of Task 8 was to prepare individual country factsheets for each of the 27 EU Member States 
and the UK, providing a clear overview of the results of ESF and YEI support achieved in the evaluation 
period  2014-2022.  The  individual  factsheets  contain  both  quantitative  and  qualitative  information 
presenting key figures and evidence in an easily readable format.  

Developing the structure, format and design of country factsheets 

The country factsheets were produced in two separate phases and formats: a first phase using Microsoft 
Word and a second phase producing a full designed version resulting in PDF versions of the factsheet. 
Most of the work was carried out during the first phase using Microsoft Word as it was easier for country 
experts, DG EMPL staff members, members of the ISG and translators to make comments and edits on 
the content of the draft factsheets in Word. Once the content of the factsheets had been agreed on the 
Word versions, final designed/PDF versions of the factsheets were produced.  

121 

 
ANNEX II.B:  INTERVENTION LOGIC 

This evaluation builds on the intervention logics of the ESF and YEI as a whole, as well as for 
each Thematic Objectives (TO 8, 9, 10, 11, 13). The ILs include a complex set of operational, 
specific  and  general  objectives.  These  objectives  are  mirrored, 
respectively,  by 
activities/outputs, results, and impacts, which are the main reference for the present assessment. 
This section clarifies the Intervention Logics for each Thematic Objective. 

Thematic Objective 8 has the objective of “promoting sustainable and quality employment 
and supporting labour mobility” and is operationalised through seven Investment Priorities 
(IPs)  that  can  be  clustered  into  three  broad  categories:  i)  investment  priorities  directly 
supporting access to employment (8i - access for job seekers and the economically inactive; 8ii 
- Sustainable integration into the labour market of young people; 8iv - equality between men 
and  women;  and  8vi  -  active  ageing);  ii)  IPs  supporting  businesses  (8iii  -  self-employment, 
entrepreneurship  and  business  creation;  and  8v  -  adaptation  of  workers,  enterprises  and 
entrepreneurs to change.); iii) and an n IP supporting institutions (8vii - modernisation of labour 
market institutions).  

Thematic  Objective  9  aims  to  “promote  social  inclusion,  combating  poverty  and  any 
discrimination”  through  six  IPs  that  tend  to  focus  either  on  a  specific  target  group  (9ii  - 
marginalised  communities)  or  an  issue  (9iii  -  discrimination  and  equal  opportunities;  9iv  - 
access  to  services;  and  9v  -  and  the  social  economy).  IP  9vi  supports  a  specific  instrument 
chosen  to  deliver  policy  (community-led  local  development  strategies)  while  IP  9i  (active 
inclusion) is very broadly defined. 

Thematic  Objective  10  has  the  goal  of  “investing  in  education,  training  and  vocational 
training for skills and lifelong learning”, through the implementation of four IPs covering all 
phases of education and targeting multiple groups: 10i - reducing and preventing early school 
leaving and promoting equal access to education, including formal, non-formal and informal 
learning pathways for reintegrating into education and training; 10ii - improving the quality and 
efficiency of, and access to, tertiary and equivalent education and training; 10iii - enhancing 
equal access to lifelong learning for all age groups in formal, non-formal and informal settings; 
and 10iv - improving the labour market relevance of education and training systems. 

Thematic Objective 11 focuses on “enhancing institutional capacity of public authorities 
and stakeholders and efficient public administration”; ESF support to TO 11 is delivered 
through two investment priorities: 11i - investment in institutional capacity and in the efficiency 
of public administrations and public services at the national, regional and local levels with a 
view to  reforms,  better regulation  and good governance;  and 11ii  -  capacity  building  for all 
stakeholders  delivering  education,  lifelong  learning,  training  and  employment  and  social 
policies, including through sectoral and territorial pacts to mobilise for reform at the national, 
regional and local levels. 

Thematic Objective 13 and its unique IP 13.1 has the objective of “fostering crisis repair in 
the context of the COVID-19 pandemic and its social consequences and preparing a green, 

122 

 
 
 
digital and resilient recovery of the economy”. To this aim investments under TO 13 could 
also pursue priorities relating to the other ESF Thematic Objectives. 

TO 8: promoting sustainable and quality employment and supporting labour mobility 

Needs:  The lingering effects  of the 2008 and 2012 crises on employment,  which resulted in 
substantial job losses and slow recovery, needed to be addressed. In 2013, employment rate 
stood at 66.8% of the EU population. High and uneven unemployment levels across Member 
States  and  regions  had  to  be  tackled  to  foster  economic  growth  and  social  cohesion. 
Additionally,  there  was  a  need  to  bridge  the  skills  gap  and  provide  adequate  skills  to  the 
workforce, enabling them to adapt to changing labour market needs. Vulnerable groups, such 
as  young  people,  low-skilled  workers,  and  migrants,  required  support  to  enhance  their 
employability.  The  impact  of  an  ageing  workforce  on  the  labour  market  also  needed  to  be 
mitigated, including promoting longer working lives, higher productivity, and sustainability of 
social protection systems. Furthermore, entities, such as businesses and public organizations, 
needed support to adapt to changing circumstances and embrace innovation and technological 
progress. Finally, labour mobility, which was limited, needed to be promoted to facilitate the 
free  movement  of  workers  across  the  EU,  addressing  skills  shortages  and  surpluses,  and 
enhancing economic growth and competitiveness. 

Objectives: In view of the needs, the Thematic Objective 8 promoted sustainable and quality 
employment and supports labour mobility. It included seven operational objectives (described 
under  the  Investment  Priorities)  that  articulated  its  strategy,  targeting  particular  groups 
(jobseekers,  young  people,  entrepreneurs,  women,  workers,  older  workers),  covering 
individuals in various employment conditions (unemployed, long-term unemployed, employed, 
and inactive), and supporting employment and self-employment. TO 8 also included support 
for national services and institutions dedicated to the development of the labour market. 

Inputs: By the end of 2022, the planned resources for TO8 included 8 included EUR 27.3 billion 
from the ESF and EUR 11.8 billion from national co-financing. At the same date, the declared 
expenditure of EU resources in TO 8 was around EUR 33 billion of ESF and national resources.  
The major expenditures were devoted to promoting access to employment (43% of the total for 
TO8), supporting young people entering the labour market (18%), and adapting workers and 
enterprises to technological changes (21%). Significant expenditure was also directed towards 
self-employment and entrepreneurship, which accounted for around 9% of the total. 

Activities:  The  inputs  are  intended  to  support  activities  that  contribute  to  achieving  the 
objectives.  Member  States  implement  the  funds  through  Operational  Programmes.  The 
classification and analysis of the operations highlighted several prevailing categories and social 
mechanisms pursued. Interventions to support unemployed people in finding a job were based 
on  three  main  types  of  activities:  a)  improving  their  skills  to  make  them  more  employable 
(training); b) reducing labour costs for enterprises (employment subsidies); and c) facilitating 
the matching between workers and firms (internships, apprenticeships, guidance). These types 
of interventions were often combined and represent key mechanisms through which the ESF 
sought to use available resources to generate intended outputs, results and impacts. To promote 

123 

 
self-employment and business creation, services to support new entrepreneurs were typically 
provided, sometimes integrated with small grants to provide initial capital. In-house training 
was the main activity/mechanism designed to strengthen the adaptation of employees and adults 
to  new  technologies  and  organisational  changes.  Other  operations,  generally  defined  as 
capacity-building activities, focused on labour market institutions. Finally, a significant amount 
of resources (more than 1% of the total for TO 8) was reprogrammed to support the income of 
employed and unemployed people during the COVID-19 pandemic. 

Outputs: Under ESF only, operations that promote sustainable and quality employment (TO8) 
supported 18.5 million participations from 2014 to 2022.  More than half of participants were 
unemployed, over one third were employed and around one in ten were economically inactive. 
Unemployed people formed 54% of participants, while operations under Investment Priority 8.i 
(Access to employment for jobseekers and inactive people) supported 45% of participants.  

Results: By the end of 2022, 3.9 million participants were in employment after participating in 
an ESF-supported intervention, and 2.9 million participants were in employment 6 months after 
leaving. Other more qualitative results achieved concerned the empowerment of participants in 
terms of their ability to participate in the labour market, the strengthening of the accessibility 
and capacity of employment services, and enhanced adaptability of enterprises to technological 
change 

Impacts:  Impacts  of  TO  8  concerned  increased  participation  and  employment  in  the  labour 
market, improvements in the quality and productivity of work, and support to economic growth. 
TO 8 contributed to employment resilience during the COVID-19 crisis as well as to an increase 
in  employment  rates  compared  to  the  initial  years  of  the  period.  In  quantitative  terms,  a 
contribution to long-term European growth is confirmed by the RHOMOLO model analysis242. 
The return on investment in terms of GDP of 1 euro spent on the TO 8 (considering in this case 
ESF and YEI) in 2030 is 1.256 euro. A positive net impact of the activities carried out under 
the TO 8 is evidenced by a meta-analysis of more than 100 counterfactual evaluations243. 

Unintended  effects:  Unintended  effects  mainly  concerned  the  "creaming"  effect  in  some 
instances, involving a tendency to focus support on groups with less complex needs and which, 
according  to  the  following  analysis,  sometimes  occurred  but  did  not  significantly  affect  the 
functioning of TO 8 operations and the intervention logic presented. 

242 Casas, Pablo and Christou, Tryfonas and García Rodríguez, Abián and Lazarou, Nicholas Joseph and Salotti, 
Simone (2025): The  ex-post  macroeconomic  evaluation  of  the  2014-2020  European  Social  Fund,  Youth 
Employment Initiative and REACT-EU labour market interventions.  
243  Haepp,  T.  and  Serrano  Alarcon,  M.,  Meta-analysis  of  the  European  Social  Fund  counterfactual  impact 
evaluations: Brief update with alternative measures, European Commission, Ispra, 2024, JRC137571, available at: 
https://publications.jrc.ec.europa.eu/repository/handle/JRC137571 

124 

 
 
 
 
 
 
 
 
Figure A. 7: Intervention Logic of Thematic Objective 8 (only ESF) 

Source: Supporting study

125 

 
 
 
Thematic Objective 9 “promote social inclusion, combating poverty and any discrimination” 

Needs: TO 9 aimed to tackle pressing social issues, including a large population at risk of social 
exclusion,  challenges  faced  by  disadvantaged  groups  in  integrating  into  society  and  the 
workforce, regional disparities in poverty levels, increasing in-work poverty and high number 
of people excluded from adequate healthcare, social and educational services. The economic 
crises of 2009 and 2012 exacerbated these problems, which persisted as significant concerns 
beyond 2014. Additionally, TO 9 concentrates on the needs of vulnerable populations, such as 
marginalised  communities,  people  with  disabilities,  and  migrants,  who  were  not  adequately 
supported by existing national policies.  

Objectives: TO 9 aimed to promote social inclusion, fight poverty, and combat discrimination. 
It pursues five sub-objectives focused on specific positive actions:  i) promoting active inclusion 
in the labour market; ii) supporting integration of marginalised communities such as the Roma; 
iii)  combating  all  forms of  discrimination  and  promoting  equal  opportunities;  iv)  enhancing 
access to sustainable and high-quality social services; v) promoting social entrepreneurship and 
vocational integration in social enterprises and vi)  supporting innovation in social inclusion 
approaches  via  community-led  local  development  strategies.  The  sub-objectives  targeted 
different components of social inclusion (labour market participation, anti-discrimination etc.) 
with  a  focus  on  marginalised  groups  and  the  strengthening  of  policy  instruments  (social 
services, community mobilisation, and social enterprises).  

These strategic objectives were integrated with operational objectives to ensure outreach to the 
most disadvantaged groups and the activation and participation  of organisations  such as  co-
operatives and social economy enterprises and stakeholders close to the target groups in policy 
implementation. 

Inputs: The total budget allocated to TO 9 was EUR 35.9 billion, comprising EUR 24.9 billion 
from the European Social Fund and EUR 11 billion in national co-financing. Moreover, EC and 
national authorities contributed providing human resources. Promoting the active inclusion of 
disadvantaged people into the labour market absorbed 64% of the total declared expenditure of 
TO 9 by the end of 2022 and enhancing access to sustainable and high-quality social services 
around  26%,  which  confirms  the  general  effort  to  strengthen  social  policy  instruments.  In 
contrast,  the  remaining  four  priorities  received  relatively  limited  funding,  targeting  more 
specific territorial or social issues and playing a lesser role in the overall allocation of resources. 

Activities:  TO  9  aimed  to  address  the  needs  and  meet  the  objectives  outlined  above  via 
implementing  a  wide  range  of  operations.  Operation  types  most  frequently  used  in  TO  9  to 
support  labour  inclusion  (training,  guidance,  work  mediation)  were  similar  to  the  TO  8 
interventions, but were typically adapted to the needs of disadvantaged groups (including, for 
instance, support for confidence building and activities to reduce social isolation alongside, or 
prior  to,  more  work-focused  activity).  They  likewise  often  involved  a  higher  intensity  or 
duration in terms of time and resources dedicated to participants. The reinforcement of social 
services  included  the  increase  in  the  quality  and  quantity  of  personnel  (for  example,  via 
training),  which  was  often  viewed  as  a  precondition  for  increasing  both  the  number  and 

126 

 
 
 
 
diversity of participants able to more easily and effectively access services. These interventions 
were,  in  some  cases,  linked  to  reform  of  the  national  social  assistance  and  instruments  and 
consequently  assumed  a  more  systemic  and  general  scope.  Other  activities  were  aimed  at 
supporting education for disadvantaged groups and promoting local development actions for 
social inclusion. Support for new entrepreneurship was also a significant tool activated by TO 
9, promoting both self-employment and social entrepreneurship (such as cooperative and social 
enterprises).  

Outputs: TO 9 involved more than 14.8 million people, of which 55% were women. Moreover, 
the  ESF  faced  significant  challenges  in  involving  disadvantaged  people  in  active  social 
inclusion  measures,  as  well  as  reducing  the  risk  of  leaving  these  people  without  tangible 
assistance.  In  this  respect,  TO  9  involved  a  large  number  of  participants  from  marginalised 
groups  and  promoted  a  more  systemic  approach  to  their  social  inclusion  (people  with 
disabilities were the 15% of total participants, migrants 26%, and people with primary or lower 
secondary education 49%). ESF activity, which was largely based on outreach projects, rather 
than simply services expected to be accessed by individuals in need, as in national policies, 
enabled a more flexible and targeted approach to these groups. For instance, the large number 
of migrants  involved in ESF operations highlights the capacity of TO 9 to address  new and 
emerging problems in many countries and regions.  

Results:  TO  9  activities  produced  significant  results,  such  as  2.8  million  disadvantaged 
participants  engaged  in  job  searching,  in  education  /  training,  gaining  a  qualification  or  in 
employment, including self-employment, upon leaving ESF activity, 1.7 million participants in 
employment upon leaving and 1.8 million participants in employment 6 months after leaving. . 
Other  qualitative  outcomes  included  the  increased  employability  and  self-empowerment  of 
participants, with these aspects being key factors in facilitating a progressive and sustainable 
integration of the most disadvantaged participants into the labour market. In addition, in many 
countries there was a general strengthening of social services, often accompanied by significant 
social policy reforms. 

Impacts: The expected impact of TO 9 was to improve the conditions and activation of the most 
vulnerable groups in the labour market and to increase the capacity of national institutions and 
services  to  address  social  problems.  The  results  identified,  alongside  complementary 
interventions such as the Fund for European Aid to the Most Deprived, should help to reduce 
the number of people at risk of poverty and suffering from severe deprivation, as well as having 
contributed to an improvement in the efficiency and effectiveness of social services. Outcome 
indicators show a significant success rate in getting people into the labour market, but official 
statistics show a deterioration in overall social conditions at EU level, indicating a significant 
but  still  insufficient  impact  of  ESF  and  national  policies,  which  account  for  the  bulk  of 
expenditure in this area. More detailed impacts on individual empowerment or estimations of 
net impacts were not systematically measured under this thematic objective, but case studies 
showed important impacts on the modernisation and strengthening of social services in many 
Member States, especially where these services were weaker. 

127 

 
 
 
 
Context: Contextual conditions generally worsened or became very serious during the period 
(COVID-19  pandemic,  increase  in  income  inequalities,  new  and  reinforced  migration  flows 
from  many  different  countries),  requiring  the  ESF  to  respond  rapidly  and  significantly.  The 
increasingly older populations in Member States also increased risks of marginalisation and the 
demand for social services. In addition, structural trends and polarisation in the labour market 
in many Member States created new forms of poverty and made social mobility more difficult. 
These  two  conditions  may  have  weakened  the  effects  of  the  ESF  or  limited  them  to  more 
“defensive” support. 

128 

 
 
Figure A. 8: Intervention Logic of Thematic Objective 9 

Source: Supporting study

129 

 
 
 
TO  10:  Investing  in  education,  training  and  vocational  training  for  skills  and  life-long 
learning 

Needs: In 2013, the early school leaving rate stood at almost 12% of the population aged 18 to 
24  in  the  EU  27  and  the  tertiary  educational  attainment  rate  was  just  below  36%  of  the 
population aged 30-34, remaining short of meeting the Europe 2020 targets (less than 10% and 
at  least  40%,  respectively).  In  addition,  shill  shortages  and  gaps  persisted  in  large  parts  of 
Europe,  also  liked  to  the  heightened  need  for  digital  literacy.  At  the  same  time,  adult 
participation in formal, non-formal and informal education and training just reached 10% in 
2014244. Among the overall needs to be addressed in the 2014-2020 programming period, TO 
10 focused on the needs to  facilitate access  to  and reduce dropout  rates  from  education and 
training and to improve the quality of education systems as well as to align them with social 
and productive needs where needed. These needs were also reflected in the significant number 
of CSRs related to the focus of TO 10 investments, accounting for around 25% of the total ESF-
related CSRs. These related to the adaptation of education to labour market needs, measures to 
reduce school dropout rates, the improvement of the quality of education, and fostering of skills 
development. During this period, additional needs arose due to increasing problems in access 
to education by migrants and disadvantaged segments of the population, as well as increasing 
inequality in education provoked by the COVID-19 pandemic. 

Objectives: In view of these needs, Thematic Objective 10 promotes a high level of education 
and  training  for  all,  supports  the  transition  between  education  and  employment  for  young 
people, enhances social inclusion, and equal opportunities. It includes 4 operational objectives 
(described under the Investment Priorities) that articulate its strategy, targeting all levels of the 
education  and  training  systems  (including  tertiary  education  and  life-long  learning),  various 
characteristics  thereof  (access,  efficiency,  quality,  and  labour  market  relevance)  as  well  as 
specific groups (young, disadvantaged, workers). 

Inputs: By the end of 2022, around EUR 36.3 billion of the ESF and YEI funds had been spent 
on  TO  10  (24.1%  of  the  budget  for  ESF  and  YEI),  including  REACT-EU.  The  major 
expenditures were devoted to preventing early school leaving, enhancing life-long learning and 
improving  the  education  system.  Significant  expenditure  was  also  directed  to  support  for 
tertiary education, which accounted for around 15% of the total.  

Activities:  The  classification  and  analysis  of  the  operations  highlighted  two  main  types  of 
activities.  Interventions  to  reduce  school  drop-out  consisted  mainly  of  support  for  less 
academically  able  pupils,  for  example  by  promoting  alternative  dual  channels  with  a  strong 
vocational focus or by reinforcing education and training activities or a combination thereof. 
Given the needs of this target group, operations also provided highly targeted support, using 
pathways  providing  training,  qualification,  counselling  and  job  placement.  To  support 
vocational education and training operations included traditional training activities as well as 
the  promotion  of  various  types  of  training  (formal,  informal,  and  non-formal)  and  life-long 

244 This indicator is based on the EU-Labour Force Survey and considers participation in training activities in the 
last 4 weeks before the interview.  

130 

 
 
learning. In addition to this, interventions to support tertiary education included the provision 
of scholarships and post-graduate high-quality skills programmes and operations to increase the 
labour  market  relevance  of  education  and  training  systems  were  implemented,  for  example 
developing mechanisms for skills anticipation. In reaction to the outbreak of the COVID-19 
pandemic, interventions to accelerate the adoption of e-learning and distance interactions aimed 
at reducing the resulting increasing inequality in education. 

Outputs:  Under  ESF  and  YEI,  operations  under  TO  10  supported  more  almost  24  million 
participations from 2014 to 2022. Given that this TO mainly involved young persons still in the 
process of completing their education, 67% of participants were inactive and 62% had primary 
or secondary education levels.  Activities also reached a significant number of persons without 
any  educational  attainment  record  (more  than  1  million),  NEETs  (almost  600  000),  persons 
with disabilities (more than 500 000) as well as migrants, minorities and persons with a foreign 
background (more than 2 million).  

Results: Operations under this TO led to 5.8 million participants gaining a qualification after 
participating in an ESF-supported intervention, 1.4 million continuing in education or training 
and  over  391  thousand  participants  engaging  in  job  searching  upon  leaving.  Among  the 
disadvantaged participants, 1.4 million engaged in job searching, were in further education and 
training or employment, or gained a qualification upon leaving the intervention. In addition, 
ESF  support  helped  to  improve  teacher’s  skills  and  the  quality  of  education  as  well  as  its 
alignment with labour market needs.  

Impacts: Impacts of TO 10 concerned reduced early school leaving, youth unemployment and 
the  NEET  rate.  It  also  facilitated  the  transition  from  education  to  the  labour  market  and 
increased  the  labour  market  relevance  of  education  and  training  programs.  Further  impacts 
include the personal development of participants, social inclusion through education and active 
citizenship effects. 

131 

 
 
 
 
 
 
 
 
 
Figure A. 9: Intervention Logic of Thematic Objective 10 

Source: Supporting study

132 

 
 
TO 11 – Enhancing the capability of public authorities and stakeholders and efficient public 
administration  

Needs: Public administrations play a key role in delivering on common policy objectives and 
on  the  EU2020  targets,  yet  administrative  and  institutional  capacities  differ  significantly 
between Member States and are often weak, which can have an adverse impact on the delivery 
of public services and public and private investments as well as on the implementation of EU 
funds. In view of technological developments, public administrations are confronted with the 
need to fundamentally transform their procedures, services and governance models, triggering 
a need to support reform and restructuring processes and to provide learning and development 
opportunities for staff. Moreover, stakeholders are important actors in the implementation of 
ESF  interventions,  while  they  often  lack  the  capacity  to  participate  in  their  design  and 
implementation. 

Objectives: Consequently, TO 11 is dedicated to enhancing the institutional capacity of public 
authorities  and  of  stakeholders  and  fostering  efficient  public  administrations.  To  reach  this 
general objective, activities supported under this TO were expected to contribute to building 
institutional  capacity  and  efficiency  of  public  administrations  and  public  services  at  the 
national, regional and local levels to promote reforms for better regulation and good governance 
(IP11i), and to capacity-building of stakeholders in education, lifelong learning, training and 
employment  and  social  policies,  empowering  them  to  contribute  to  employment  and  social 
policymaking (IP11i), by reaching public administrations and relevant entities directly and by 
providing relevant (digital) solutions. Both the ESF and the European Regional Development 
Fund (ERDF) provided support to activities under TO 11. 

Inputs: The total budget allocated to TO 11 amounts to EUR 4.1 billion (EUR 3.2 billion of 
Union contribution and EUR 0.9 billion represent the national share), representing the lowest 
budget allocation of all thematic objectives, with 4% of the ESF budget dedicated to supporting 
capacity building  of public authorities and of stakeholders. The large majority of  the  Union 
funding is allocated to IP 11i (EUR 3 135 million, which represents 97% of the budget allocated 
to  TO  11),  whereas  EUR  103  million  (3%  of  the  Union  budget  allocated  to  TO  11)  of  the 
programmed funding supports investments under IP 11ii. More than half of the Member States 
programmed activities under this TO (in total 18 Member States245), and thereby in most cases 
(for 13 Member States246) also addressed relevant CSRs that were directed at them247. 

Activities: The operations supported under TO 11 reflect an overall focus on skills, the quality 
of  public  services  and  institutional  and  governance  change  and  largely  consist  of  training 

245 These were  Bulgaria, Cyprus, Czechia, Estonia, France, Greece, Croatia, Hungary, Italy, Lithuania, Latvia, 
Malta, Poland, Portugal, Romania, Slovakia, Slovenia and the United Kingdom. 
246 Of the Member States having programmed activities under TO 11, Bulgaria, Croatia, Czechia, Estonia, Greece, 
Hungary, Italy, Latvia, Malta, Portugal, Romania, Slovakia, and Slovenia received CSRs in the 2012-2014 period 
aimed at improving the capacity of the public sector. 
247 To note that funding under TO 11 was only available in Member States eligible for the Cohesion Fund or that 
have at least one less developed region. Consequently, Member States not implementing TO 11 are AT, BE, DE, 
DK, ES, FI, IE, LU, NL, SE. 

133 

 
 
measures,  targeting  different  levels  of  government  as  well  as  social  partners  and  other 
stakeholders, incl. NGOs. Accordingly, examples of activities implemented include upskilling 
and  training  activities,  organisational  advice  (e.g.,  to  support  reorganisation  processes), 
technical  assistance,  the  hiring  of  specialised  personnel,  and  improving  the  quality  and 
accessibility of public services incl. via digital solutions and e-government services. Notably, 
activities under TO 11 also contributed to supporting public administrations in responding to 
the COVID-19 pandemic, and activities were also often targeted at the judicial system.  

Outputs:  Under  this  TO,  outputs  typically  correspond  to  the  number  of  public  officials 
participating in training and upskilling measures and the number of supported projects to e.g., 
revise processes in public administrations, in line with the supported activities which primarily 
focused  on  the  (staff  of)  institutions  and  on  procedures.  Overall,  almost  13  000  public 
administrations  and  organisations  were  reached,  and  in  total  952  117  participations  were 
recorded. Support was provided to 3 724 projects targeting public administrations at national, 
regional or local level, and more than 1 200 projects involved NGOs and social partners.   

Results:  The  main  results  under  TO  11  were  expected  to  reflect  the  enhanced  skills  and 
capacities of public officials as well as of social partners and other stakeholders, to enable public 
administrations to strengthen good governance, and to ensure efficient procedures and public 
service delivery. Nonetheless, the most telling results under TO 11 were often procedural, such 
as reductions in the time required for certain administrative procedures, e.g., via revised and/or 
simplified  procedures,  the  adoption  of  digital  solutions,  or  the  implementation  of  new  IT 
systems. 

Impacts: Interventions under TO 11 were expected to contribute to the overall strengthening of 
public administrations, the judiciary, and civil society organisations, to improve public services, 
and to support (administrative) reform processes. At the same time, impacts in terms of reduced 
administrative costs due to a more efficient delivery of public services and investments were 
also expected. 

134 

 
 
 
 
 
 
 
 
Figure A. 10: Intervention Logic of Thematic Objective 11 

Source: Supporting study 

135 

 
 
 
 
 
TO  13:  fostering  crisis  repair  in  the  context  of  the  COVID-19  pandemic  and  its  social 
consequences and preparing a green, digital and resilient recovery of the economy 

Needs:  TO 13 includes all the interventions funded by the Recovery Assistance for Cohesion 
and the Territories of Europe (REACT-EU)248. The needs to be addressed by TO 13 were the 
health emergency caused by the COVID-19 pandemic and the subsequent need for economic 
recovery that, to a different extent, concerned all Member States. 

Inputs: The Recovery assistance for cohesion and the territories of Europe extended the crisis-
response  and  crisis-repair  measures  delivered  through  the  coronavirus  response  investment 
initiative and the coronavirus response investment initiative plus and constitutes a bridge to the 
long-term recovery plan. The programme supported investment projects that foster crisis-repair 
capacities and contribute to a green, digital and resilient recovery of the economy, including 
support for maintaining jobs (by short-time work schemes and support for the self-employed). 
As part of Next Generation EU, REACT-EU was unique among all anti-crisis initiatives in that 
it provided additional funding to cohesion policy. Consequently, the funding was programmed 
in a new thematic objective with a view to structural investments serving longer-term recovery.  

In 2022, REACT-EU’s scope was expanded to address the needs arising due to people fleeing 
Russia's aggression against Ukraine. The CARE+ regulation, adopted in April 2022, built upon 
the earlier CARE framework by providing additional pre-financing to Member States through 
REACT-EU. This funding aimed to ease the financial strain on Member States receiving the 
high number of Ukrainians fleeing Russia’s war of aggression against Ukraine. The regulation 
also  introduced  flexible  measures,  including  unit  costs  for  basic  needs  and  support  for 
individuals granted temporary protection. Out of the 187 ESF programmes, 89 used REACT-
EU, with a total allocation of 20 billion euro (with almost 100% EU co-financing, national co-
financing amounted to EUR 0.2 billion). The 40% ESF share among all REACT-EU funding is 
significantly higher than the proportions in the whole of cohesion policy (26%). 

Activities: In ESF programmes, REACT-EU focussed primarily on 

•  adaptation  of  workers,  enterprises  and  entrepreneurs  to  change  (in  particular  job 

• 

maintenance through short-time work schemes) and 
reducing  and  preventing  early  school-leaving  and  promoting  equal  access  to  good 
quality  early-childhood,  primary  and  secondary  education  (in  particular  provision  of 
equipment and facilities to prevent dropouts and reduce inequality due to the pandemic). 

The remaining half of ESF REACT-EU included notably support for health system personnel 
and  adaptation  of  civil  servants  to  new  challenges.  The  common  thread  of  logic  in  the 
interventions  was  the  primary  aim  of  longer-term  recovery  through  crisis  response.  For 
example,  educational  equipment  and  training  for  personnel  and  pupils  addresses  both 
immediate  learning  needs  in  the  changed  circumstances  due  to  the  pandemic,  as  well  as 
educational inequalities and may serve the digital transition.  

248 REACT-EU was adopted in December 2020 as part of the COVID-19 crisis response. REACT-EU is financed 
by  the  EU  Recovery  Plan,  Next  Generation  EU,  and  serves  as  a  supplement  to  the  European  Regional 
Development Fund (ERDF) and the European Social Fund (ESF) rather than as a new funding instrument 

136 

 
 
Outputs: In 2022, TO 13 recorded around 2.6 million participations, with 43% of participants 
being  employed.  The  comparatively  high  share  of  employed  persons  reflects  the  fact  that 
actions to avoid job-losses as a result of the COVID-19 pandemic, for example through short 
time working schemes, were important in the period following REACT-EU’s introduction. Of 
the  remaining  participants,  28%  were  inactive,  and  29%  unemployed.  Participants  were 
primarily aged between 25-54 (55%) and had education levels below tertiary education (73%).  

Results: The common results reflect those specific objectives that aimed at maintaining jobs 
and adapting personnel in public services: 250 thousand found employment and another 250 
thousand gained a qualification. However, the impact on the healthcare system and the twin 
transitions (green and digital) is less evident from quantitative data.  

137 

 
 
 
 
 
Figure A. 11: Intervention Logic of Thematic Objective 13 

Source: Supporting study

138 

 
 
 
The Intervention Logic of YEI  

Needs - The high level of youth unemployment in many countries and regions created a need 
for specific support, especially after the economic crisis that hit the EU before 2014. The YEI 
was  a  specific  initiative  included  in  the  action  of  the  ESF  and  provided support to young 
people living in Member States and regions where youth unemployment was higher than 25%. 
In particular, the YEI was aimed at supporting NEET young people who face greater difficulties 
in entering the labour market and are at risk of social exclusion249. 

Objectives - The YEI’s main objective was to support the fight against youth unemployment in 
the worst-affected EU regions. It targeted all people between the ages of 15 and 24 years old 
and paid a particular attention to people not in employment, education or training, with Member 
States given the freedom to decide to extend support to include young people under the age of 
30 through the YEI’s implementation. YEI only had one specific objective, namely ensuring 
that all young people under the age of 25 years receive a good-quality offer of employment, 
continued education, an apprenticeship or a traineeship. 

Inputs– The YEI mobilised around EUR 9 billion of EU funding  and its interventions were 
programmed under ESF IP 8ii – YEI (Sustainable integration of young people into the labour 
market).  

Activities  -  The  most  frequently  funded  activities  were  vocational  training,  traineeships, 
apprenticeships,  guidance,  and  support  for  self-employment.  The  important  focus  on  young 
NEET people required development of new strategies and tools to engage this target group. 

Outputs – The YEI engaged 3.9 million young people, of whom 49% were young women and 
34% were young people with primary or lower secondary education. 

Results - 6 months after leaving, some 35% of YEI participants were in employment or self-
employment  and  11%  were  in  continued  education  or  training  programmes  leading  to  a 
qualification.  In  general,  the  expected  results  on  young  participants  included  increased 
employability through improved skills and active participation in the labour market. In addition, 
the YEI demonstrated a high capacity to focus EU and national policies on the needs of young 
people and highlighted the risks of labour market marginality among young people, creating a 
new awareness of structural bottlenecks to labour market access.  

Impacts –YEI resulted in positive impacts on young people's participation in the labour market 
and on young people's self-empowerment.  In the  meta-analysis  of the counterfactual impact 

249 During the programme preparation years, 20 Member States had received a CSR on the need to increase the 

participation of young people in the labour market. 

139 

 
 
 
 
 
 
 
 
 
evaluations250, the net impacts for young people were not particularly high in the medium term, 
but their importance increased in the long term, confirming the benefits on participants' attitudes 
and behaviour. Finally, it is important to highlight the impact on the design of labour policies, 
where the youth dimension has gained increasing attention in all Member States, in part via the 
introduction of a Youth Guarantee supported in many countries implementing the YEI. 

250  Haepp,  T.  and  Serrano  Alarcon,  M.,  Meta-analysis  of  the  European  Social  Fund  counterfactual  impact 
evaluations: Brief update with alternative measures, European Commission, Ispra, 2024, JRC137571, Available 
at: https://publications.jrc.ec.europa.eu/repository/handle/JRC137571. 

140 

 
 
 
Figure A. 12: Intervention Logic of YEI 

Source: Supporting Study

141 

 
 
 
ANNEX II.C:  EVALUATION MATRIX 

Criteria 

Evaluation questions 

Judgement criteria 

Indicators 

Tasks and sources 

To what extent was the EU support successful – or not – and why? 

1. 

To what extent have the programme 
milestones and targets been 
achieved? What were the main factors 
involved? How do the achievements 
relate to the values reported for the 
common indicators? (To what extent 
are the achievements indicated by 
common and programme-specific 
indicators coherent?) 

1.  Extent to which milestones and targets 
have been achieved per TO/IP/fund 
(including REACT-EU and CARE which 
will be analysed separately) 

2.  Relevance and appropriateness of set 

targets to values reported for common 
and programme specific indicators   
3.  Comparison point:  ESF and YEI before 
and after the COVID-19 pandemic/ 
Targets set in original Operational 
Programmes 

4.  Level of achievement of set targets 
(output and result including soft 
outcomes) per TO/IP/fund (including 
REACT-EU and CARE which will be 
analysed separately) 

5.  Level of achievement of the performance 

framework targets 

6.  Number and type of targets not achieved 

per MS, OP, TO (including TO 13) and IP 

7.  Achieved/not achieved targets of 

T1 Mapping 
•  Monitoring data (SFC data, 
Open data platform, 
Launchpad)  

T6 Case studies 
Interviews 
• 
Desk analysis of national 
• 
evaluations/documents 
(including AIRs as relevant) 

common and specific indicators and 
difference in the performance between 
TOs, IPs, MS and OPs 

T3 Consultation activities 
• 
•  MAs survey 

Stakeholder interviews 

Effectiveness 

2.  How and to what extent did EU 
support contribute to the 
achievement of the thematic 
objectives set in the CPR and the 
wider EU policies including the 
Europe 2020 Strategy and the 
European Pillar of Social Rights as 
well as more specific EU policies, such 
as on education and training? 

8.  Extent to which labour market, education 
and social inclusion improved during 
2014-2020 programming period, also 
considering the pandemic impact   
9.  Extent to which the ESF contributed to 

labour policy, social policy and 
education policy in terms of: 
10.  Financial resources 
11.  Addressed needs 
12.  Participation (e.g., in LM, education 

etc.) 

13.  Actions delivered 

14.  Comparison point:  ESF and YEI before 
and after the COVID-19 pandemic/ 
Socio-economic context indicators from 
2013 as a baseline before the 2014-2020 
programme 

142 

15.  Average values of socio-economic 

indicators per country/region, including 
target indicators of the European 2020 
Strategy and of the European Pillar of 
Social Rights (taking into account the 
changes before and after the pandemic 
and, to the extent possible, the war). 
16.  Relative weight of the ESF expenditure 

with respect to national policies 
expenditure 

17.  Number of participants (people and 

• 

entities) in ESF interventions in different 
policy fields (as absolute values and with 
respect to the relevant context e.g., LM 
features, tertiary education etc.) 

18.  Type of operations implemented under 

each TO/IP  

T1 Mapping 
•  Monitoring data (SFC data, 
Open data platform, 
Launchpad) Data on 
operations (lists of 
operations and Kohesio) 

T2 Context 
• 

Socio-economic data (e.g., 
Eurostat, OECD, World 
Bank) 
Policy expenditure data 
(e.g., Eurostat, European 
Commission) 

 
 
 
 
 
 
 
 
Criteria 

Evaluation questions 

Judgement criteria 

Indicators 

Tasks and sources 

3. 

To what extent was the EU support 
used to implement the Country 
Specific Recommendations? 

19.  Extent to which the operations 

supported by ESF met the objectives set 
out in relevant Country Specific 
Recommendations 

4. 

To what extent were the target 
groups reached (also geographically) 
by the operations including 
disadvantaged persons, especially 
those from marginalized communities 
including Roma and those leaving 
education without qualifications, 
long-term unemployed, older 
workers? Factors influencing the 
reach of the target groups should be 
explained. How effective was the 
requirement to allocate a minimum 
percentage of the budget to social 
inclusion in reaching the social 
inclusion target groups? 

23.  Extent to which disadvantaged persons 

have been reached by ESF 
24.  Degree of differentiation of the 

interventions according to different 
needs of the target groups 

25.  Observed relations between the reach of 

the target groups and the ways of 
delivering the ESF support, especially for 
those disadvantaged.  

26.  Innovation, experimented by 
MS/Regions, in dealing with 
disadvantaged groups through the ESF 
27.  Comparison point:  ESF and YEI before 
and after the COVID-19 pandemic/ 
Targets set in original Operational 
Programmes/ ESF and YEI during the 
previous programming period between 
2007-2013 

20.  Evaluative judgement on the contribution 
to CSRs according to national Authorities 
(in the context of case studies, the extent 
to which ESF contributed to implement 
CSRs will be assessed). 

21.  Resources spent for operations being 

relevant for the CSRs 

22.  Participants (people and entities) in 
operations relevant for the CSRs 

28.  Number and characteristics of people 
who improved their situation upon 
leaving the ESF intervention in terms of 
participation in the labour market, self-
employment, skills, participation in 
education and training, qualification 
attainment, access to services for more 
disadvantaged, etc. by TO/IP 

29.  Coverage rate of target groups such as 

unemployed women, NEETs, 
unemployed, low-educated persons, 
Roma, long terms unemployed and older 
workers, etc. by TO/IP 

30.  Characteristics of the interventions 

delivered for different target groups, also 
considering the type of actors involved 

31.  Factors emerged from the field work 

influencing the reach of the target groups 
(taking into account, to the extent 
possible, the changes before and after 
the pandemic and the war). 

32.  Whether and to what extent, according to 
key stakeholders, the minimum budget 
allocated to social inclusion had a critical 
mass effect compared to the past (2007-
13) 

143 

T2 Context 
•  Mapping of CSRs (also 

based on previous thematic 
studies) to reconstruct and 
understand the needs for 
structural reforms in the 
MSs as part of the context     

T6 Case studies on IPs of TO 8, 9, 
10 and YEI 
T1 Mapping 
•  Monitoring data (SFC data, 
Open data platform, 
Launchpad)  

T2 Context 
• 

Socio-economic data (e.g., 
Eurostat, OECD, World 
Bank) 
Policy expenditure data 
(e.g., Eurostat, European 
Commission) 

• 

T3 Consultation 
• 
•  MAs survey 

Stakeholder interviews 

T6 Case studies on: 
• 
• 

IPs of TO 8, 9, 10 and YEI 
CRII, REACT-EU and CARE 

T7 Focus groups 
•  Managing Authorities and 

stakeholders 

 
 
 
 
 
 
 
 
Criteria 

Evaluation questions 

Judgement criteria 

Indicators 

5.  Which changes (intended and 

33.  Identified effects (intended and 

37.  Level of achievement of the targets, 

unintended) did the EU support bring 
to the target groups (both individuals 
and entities)? How were these 
changes, including soft outcomes, 
assessed and used? Which types of 
operation are or were the most/least 
effective, for which groups and 
gender and in which contexts? 

unintended) are aligned with the 
objectives of the ESF 

34.  Changes concerning the condition of 
participants, including soft outcomes 
(e.g., personal and interpersonal, 
organisational, analytical skills) are 
explained/not explained by the relevant 
theory of change  

broken down by gender  

38.  Evaluative judgment based on evidence 
from specific indicators/evaluations/case 
studies/participant survey on participant 
changes achieved, including soft 
outcomes (e.g., using scoring systems 
and scales aimed at measuring outcomes 
systematically and numerically). 

35.  Changes occurred have/have not 

39.  Evaluative judgement based on level of 

influenced the implementation of the 
OPs 

36.  Results achieved by ESF depend on the 

type of operation financed or/and the 
target group or/and the context   

6.  Which factors facilitate or hinder the 
effectiveness of ESF operations? Rank 
these factors by order of importance 
and justify your ranking. In particular, 
how does the implementation 
framework in Member States/regions 
encourage/discourage the 
participation of beneficiaries? 

42.  Evaluative judgement on the basis of 
evidence collected as to potential 
barriers/success factors underlying 
outputs and results, distinguished by 
target group, especially those further 
away from labour market and at risk of 
exclusion or discrimination 

effectiveness at IP level, as resulting from 
indicators, case studies and participant 
survey, in different national and, where 
necessary, regional contexts (taking into 
account, to the extent possible, the 
changes before and after the pandemic 
and the war). 

40.  Changes in output/outcome/result 

indicators by IP, intervention and across 
target groups identified and explained by 
available evaluations 

41.  Number and value of interventions 

defined or modified drawing on findings 
of the evaluations  

43.  Factors more frequently reported by MAs, 
stakeholders, evaluations, studies on TOs 
to explain success/unsuccess of 
operations and supported by clear 
justifications and robust evidence 
44.  Ranking of facilitating/hindering factors 

by various dimensions (e.g., geographical, 
TO, IP, type of operation, features of 
implementation framework etc.) 

Tasks and sources 

T1 Mapping 
•  Monitoring data (SFC data, 
Open data platform, 
Launchpad)  

T3 Consultation 
• 

Public consultation 

T5 Impact analysis 
• 

CIEs 

T6 Case studies on: 
• 
• 

IPs of TO 8, 9, 10 and YEI 
Desk analysis of EU and 
national 
evaluations/documents 
(including AIRs as relevant) 

T7 Focus groups 
•  Managing Authorities and 

stakeholders 

T3 Consultation: 
•  MAs survey 

T6 Case studies on: 
• 
• 

IPs of TO 8, 9, 10 and YEI 
Desk analysis of EU and 
national 
evaluations/documents 
(including AIRs as relevant) 

45.  Evaluative judgement on IPs or/and OPs 

T7 Focus groups 

which have mostly encountered 
difficulties and challenges in the reach of 
target groups   

•  Managing Authorities and 

stakeholders 

144 

 
 
 
 
 
 
 
 
Criteria 

7. 

Evaluation questions 
To what extent did the EU support 
contribute to structural changes in 
employment, social inclusion and 
education policies and systems in the 
EU and UK as a whole and in Member 
States and regions in particular? 

Judgement criteria 
46.  Evaluative judgement on the basis of all 
evidence collated as to the contribution 
to structural change in EU, in MS and at 
regional level. 

47.  Comparison point:  ESF and YEI before 
and after the COVID-19 pandemic/ 
Targets set in original Operational 
Programmes/ ESF and YEI during the 
previous programming period between 
2007-2013  

8.  What was the concrete contribution 
of EU support to the implementation 
of the horizontal principles and social 
innovation? 

53.  Extent to which horizontal principles 

influenced the design of intervention, 
the selection of the operations and the 
evaluation process 

54.  Extent to which ESF contributed 
concretely to implementation of 
horizontal principles and social 
innovation  

55.  Comparison point:  ESF and YEI during 
the previous programming period 
between 2007-2013  

Indicators 

48.  Findings of the contribution analysis 

about the role of the ESF in: 

49.  reducing disparities/promoting structural 

changes in employment, 
education/training (e.g., in terms of 
improving education outcomes, 
improving quality of education, reducing 
early school leaving, increasing tertiary 
education attainment, improving digital 
skills, enhancing participation in ECEC), 
social inclusion at regional level, 

50.  fostering positive changes – or limiting 
negative ones – in relation to objectives 
of the wider European policies, 

51.  countering effects of the pandemic crisis. 
52.  Findings from the assessment of the 

contribution to the implementation of 
the CSRs 

Tasks and sources 

T2 Context 
• 

Analysis of CSRs 

T3 Consultation 
• 

Stakeholder interviews 

T6 Case studies on: 
• 
• 
• 

IPs of TO 8, 9, 10 and YEI 
CRII, REACT-EU and CARE 
Horizontal principles and 
Social Innovation 

T7 Focus groups 
•  Managing Authorities and 

stakeholders 

56.  Number and types of actors representing 
gender, environment and civil society 
involved in delivery ESF support 

T3 Consultation 
• 
•  MAs survey 

Stakeholder interview 

57.  Specific needs of women and 

discriminated groups addressed by ESF 
58.  Number and value of operations/projects 
which experimented social innovation  

T6 Case studies on: 
• 

Horizontal principles and 
Social Innovation 

9.  What other socio-economic impacts 

59.  Extent to which it is estimated that ESF 

60.  Evidence on micro and macro impacts 

can be identified from EU support, 
both at micro and macro levels? 

10.  To what extent did the operations 

introduced by CRII and CRII+, REACT 
EU and CARE manage to address the 
negative social effects of the crises? 

impacted on GDP and other 
macroeconomic variables (e.g., 
employment), and brought about further 
effects on individuals   

61.  Extent to which the ESF has met the 
needs brought about the pandemic 
outbreak and promptness of the support 

145 

based on CIEs and the RHOMOLO model. 

T5 Impact analysis 
• 

CIEs including the meta-
analysis 

•  Macro impacts (RHOMOLO) 

64.  Value of resources spent scope and type 

of financed operations 

T1 Mapping 
• 
• 

SFC data  
Data on lists of operations  

 
 
 
 
 
 
 
Criteria 

Evaluation questions 

Judgement criteria 

Indicators 

62.  Extent to which the ESF has met the 

65.  Number of additional responses using 

needs brought about by the 
consequences of the Russia’s war of 
aggression against Ukraine 

63.  Comparison point:  ESF and YEI before 
and after the COVID-19 pandemic 

11.  To what extent were operations cost-

effective? What types of operation 
were the most/least cost-effective? 
Why? 

Efficiency 

75.  Extent to which the invested resources 
were appropriate (benefits greater than 
costs) with respect to achievements 
76.  Differences between IPs regarding the 
cost/benefits ratio were relevant 

146 

flexibilities without formal OP 
amendments 

66.  Number of MSs and no. of amendments 
using simplified OP amendment process  

67.  Number of participants and entities 

supported in combating or counteracting 
the effects of COVID-19 

68.  Number of participants who benefited 
from support in short-time work 
arrangements 

69.  Number of participants maintaining their 

jobs 6 months after the end of support; 
and gaining a qualification upon leaving, 
supported in actions combating the 
effects of COVID-19 

70.  Number of health care personnel who 

benefitted from ESF support 

71.  Number of people vaccinated with EU 

support 

72.  Value of COVID-19 relevant items 

purchased (e.g., medical equipment, 
medicines, IT equipment)  
73.  Value of all vaccination costs 
74.  Evaluative judgement: stakeholders’ 

opinion about alternative scenarios with 
no ESF investments faced the socio-
economic effects of the pandemic and 
the war 

77.  Unit costs per output and results (for 
each TO and for each IP at EU level)  
78.  Direct and indirect costs of interventions 

and IPs 

79.  Direct and indirect benefits of 

interventions and IPs  

Tasks and sources 

T6 Case studies on: 
• 

CRII, REACT-EU and CARE 

T1 Mapping  
•  Monitoring data (SFC data, 
Open data platform, 
Launchpad) Data on lists of 
operations 
T4 Cost-Effectiveness 

 
Criteria 

Evaluation questions 
12.  Are there significant cost differences 
between Member States/Regions in 
the implementation of the 
operations? Why? 

disproportionately differ across Member 
States/Regions 

81.  Influence of contextual factors and 
administrative frameworks on 
costs/benefits ratio  

Judgement criteria 

80.  Extent to which the costs 

Indicators 
82.  Comparison of costs/benefits, in terms of 

Tasks and sources 

T4 Cost-Effectiveness 

13.  Was there unnecessary administrative 
burden, in particular gold plating? To 
what extent did simplifications 
proposed by the Commission, such as 
the simplified cost options and joint 
action plans, improve the efficiency of 
the operations? Which other good 
practices to reduce unnecessary 
administrative burden can be 
identified? 

84.  Differences in delivery mechanisms and 
relative variation in the administrative 
burden among MS/OPs. 

85.  Whether the procedures adopted by 

MAs exceeded what the ESIF’s regulation 
requested or required more input (e.g., 
human resources) to comply with the 
additional national obligations.  
86.  Extent to which MS/Regions have 

adopted simplification solutions, for 
what IP and, if/where feasible, 
operations  

T6 Case studies 

T4 Cost-Effectiveness 

T6 Case studies 

type and scope, per OP and comparison 
of results for a sample of OPs   
83.  Comparison of the components 

articulating the costs (direct and indirect) 
between MS/Regions in order to identify 
and explain any difference resulted from 
the analysis of cost-effectiveness  
87.  Administrative burden stemming from 

legal obligations 

88.  Relevant differences in timing 

implementation of interventions reported 
by actors   

89.  Tentative identification of OPs and, if 
feasible, of operations which applied 
simplification options and/or 
experimented further solutions 
90.  Key features of good practices 

implemented to reduce unnecessary 
administrative burden   

14.  To what extent did the monitoring 
and reporting systems of Member 
States provide quality data on a 
regular basis? Has this occurred at a 
reasonable cost? Under which 
conditions could the costs have been 
lower? 

91.  Extent to which monitoring data 

95.  Degree of digitalisation of the 

reflected/was up to date to what was 
happening on the ground 

92.  Extent to which the indicators have been 

regularly quantified 

93.  Extent to which the benefits are greater 

monitoring, reported by actors (e.g., 
measured by the use of a comprehensive 
IT system for data collection and 
reporting, real-time data transfer into the 
ESF national system etc.) 

than the costs 

96.  Monitoring data provided by MS during 

T1 Mapping  
•  Monitoring data (SFC data, 
Open data platform, 
Launchpad)  
T4 Cost-Effectiveness 

T6 Case studies 

Coherence 

15.  To what extent was the dedicated EU 

support internally coherent? 

94.  Administrative burden linked to the 

monitoring/reporting  

99.  Whether the interventions were aligned 
with the IP under which they were 
financed 

100. 

Whether interventions were 

or were not duplicated 

147 

the period and missing values 
97.  Good practices of data collection, 

validation and transfer  

98.  Difference between benefits and costs of 

monitoring/reporting  

102. 

Number and value of 
interventions implemented under 
different OPs and/or different IPs being 
similar as to eligible area, target group, 
type and intensity of the support  

T1 Mapping  
•  Monitoring data (SFC data, 
Open data platform, 
Launchpad) 
T3 Consultation 
• 

Stakeholder interview 

 
 
 
 
Criteria 

Evaluation questions 

Judgement criteria 

Indicators 

Tasks and sources 

101. 

Comparison point:  ESF and 

103. 

Evaluative judgement on 

•  MAs survey 

YEI before and after the COVID-19 
pandemic/ Findings of interim thematic 
evaluation reports completed/ESF and 
YEI during the previous programming 
period between 2007-2013/Findings 
from other related research or 
evaluations at EU or national level 

linkages, synergies and/or 
complementarities within the OPs and 
between NOPs and ROPs 

T6 Case studies: 
• 

Interviews with Managing 
Authorities 

16.  To what extent was the dedicated EU 
support coherent with other EU 
funds? 

104. 

Extent to which ESF 

106. 

Number and value of ESF 

complements or integrates other EU 
funds 

105. 

Comparison point:  ESF and 

YEI before and after the COVID-19 
pandemic/ Findings of interim thematic 
evaluation reports completed/ESF and 
YEI during the previous programming 
period between 2007-2013/Findings 
from other related research or 
evaluations at EU or national level 

interventions linked with other 
interventions co-financed by FEAD or EGF 
or AMIF or - where appropriate - other 
EU funds such as ERDF. 

107. 

Number of local strategies, 
partnerships, etc. using ESF and other EU 
funds in a complementary manner  

17.  To what extent was the dedicated EU 
support coherent with other EU 
policies? 

108. 

Whether the ESF operations 

110. 

Number and value of ESF 

were aligned or contributed to other EU 
policies per TO 

interventions clearly referred to other EU 
policy 

109. 

Comparison point:  ESF and 

111. 

Value of resources allocated 

T6 Case studies: 
• 

Interviews with national 
policy makers (e.g., minister 
representatives and MAs)  
Analysis of National 
strategic documents 

• 

T1 Mapping 
• 

Data on lists of operations 

T6 Case studies: 
• 

Analysis of National 
Strategic documents 

18.  To what extent was the dedicated 
support coherent with other 
national/regional level activities? 

for type of actions aligned to or 
contributing to other EU policies 

YEI before and after the COVID-19 
pandemic/ Findings of interim thematic 
evaluation reports completed/ESF and 
YEI during the previous programming 
period between 2007-2013/Findings 
from other related research or 
evaluations at EU or national level  

112. 

Extent to which ESF 

114. 

Evaluative judgement based 

interventions were planned to make a 
reference to other national/regional 
strategies and plans  

113. 

Comparison point:  ESF and 

YEI before and after the COVID-19 
pandemic/ Findings of interim thematic 

148 

on documentary evidence 

115. 

Number of national/regional 

plans for employment, inclusion, 
education, administrative capacity which 
envisaged interventions financed under 
ESF 

• 
• 

T6 Case studies: 
• 

Strategic documents (es. 
National Reform 
Programmes) 
Programme’s documents 
Desk analysis of national 
evaluations/documents 
(including AIRs as relevant) 

 
 
 
Criteria 

Evaluation questions 

Judgement criteria 
evaluation reports completed/ESF and 
YEI during the previous programming 
period between 2007-2013/Findings 
from other related research or 
evaluations at EU or national level 

Indicators 

Tasks and sources 

• 

Interviews with Managing 
Authorities 

116. 

Is the intervention still relevant? 

19.  To what extent were the objectives 

and the operations funded by the EU 
support relevant to the needs of the 
target groups?  

117. 

The degree to which the 
ESF funded objectives and operations 
focused on the needs of the target 
groups identified in the programming 
documents  

118. 

Comparison point:  ESF and 

119. 

Number of participants 

(people and entities) supported by ESF 
belonging to the target groups (taking 
into account, to the extent possible, the 
changes before and after the pandemic 
and the war). 

YEI before and after the COVID-19 
pandemic/ Findings of interim thematic 
evaluation reports/ESF and YEI during 
the previous programming period 
between 2007-2013/Findings from other 
related research or evaluations at EU or 
national level 

120. 

Evaluative judgement based 
on contents and scope of the operations 
financed to address the target groups   

• 

Relevance 

20.  Were the most relevant groups in the 
different socio-economic contexts 
adequately prioritised and targeted, 
starting from the design stage? Which 
target groups were not reached and 
why?  

121. 

The extent to which the 

128. 

Characteristics of the ESF 

target groups were clearly identified, and 
reached, in the different stages: 
needs assessment,  
122. 
programming,  
123. 
design,  
124. 
implementation,  
125. 
monitoring.  
126. 
Comparison point:  ESF and 

127. 

YEI before and after the COVID-19 
pandemic 

participants (people and entities) 

129. 

Number and types of 

groups mainly supported by the ESF 
under each IP in various Member States 
(taking into account, to the extent 
possible, the changes before and after 
the pandemic). 

130. 

Evaluation judgement, 

successfully addressed challenges 
concerning target groups 

131. 

Barriers which hampered the 

reach of the target groups  

149 

T1 Mapping  
•  Monitoring data (SFC data, 
Open data platform, 
Launchpad)  

T2 Context 
• 

Socio-economic data (e.g., 
Eurostat, OECD, World 
Bank) 
Policy expenditure data 
(e.g., Eurostat, European 
Commission) 

T6 Case studies: 
• 

Data and other secondary 
resources at MS level 
Desk analysis of national 
evaluations/documents 
(including AIRs as relevant) 

•  MAs survey 
T1 Mapping  
•  Monitoring data (SFC data, 
Open data platform, 
Launchpad)  

T2 Context 
• 

Socio-economic data (e.g., 
Eurostat, OECD, World 
Bank) 
Policy expenditure data 
(e.g., Eurostat, European 
Commission) 

• 

• 

T6 Case studies: 
• 

Data and other secondary 
resources at MS level 

 
 
 
Criteria 

Evaluation questions 

Judgement criteria 

Indicators 

21.  How and to what extent was 

132. 

Extent to which the 

136. 

Number and value of 

partnership and multi-level 
governance implemented? To what 
extent have the stakeholders closest 
to the needs of the target groups 
(such as non-governmental 
organisations, social partners, training 
providers) been involved? Were these 
stakeholders encouraged to be 
involved, and by whom (e.g., 
EU/national level, regional level etc) 
and how? 

22.  To what extent were OPs flexible and 
able to adapt to changes in the 
implementation context or political 
priorities at EU and country level, in 
particular the crisis linked to the 
pandemic or the Ukrainian crisis? 

• 

• 

Tasks and sources 
Desk analysis of national 
evaluations/documents 
(including AIRs as relevant) 
Interviews with Managing 
Authorities and other 
stakeholders 

T3 Consultation: 
•  National stakeholder 
targeted survey 

T6 Case studies: 
• 

• 

Case study on partnerships 
and social innovation 
Desk analysis of national 
evaluations/documents 
(including AIRs as relevant) 

stakeholders have been involved in 
different stages 

133. 

Whether and how the 

Authorities put in place adequate actions 
to engage stakeholders  

actions aimed at involving the 
stakeholders during the whole 
programming (taking into account, to the 
extent possible, the changes before and 
after the pandemic). 

134. 

Extent to which the 

137. 

Number and type of 

involvement of the stakeholders allowed 
an enhancement of the ESF 
programming 

stakeholders involved in designing, 
selecting operations and monitoring 

138. 

Evaluative judgement on 

135. 

Comparison point:  ESF and 

YEI before and after the COVID-19 
pandemic/ Findings of interim thematic 
evaluation reports/ESF and YEI during 
the previous programming period 
between 2007-2013/Findings from other 
related research or evaluations at EU or 
national level 

139. 

Extent to which the OPs 
have proved a continuity relevance to 
needs during the period. 

140. 

Whether the OPs have 

supported actions to face the immediate 
needs and/or the socio-economic effects 
of the pandemic and of the war against 
Ukraine. 

150 

contributions given by the stakeholders 
to the ESF programming and 
implementation 

T7 Focus groups 
•  Managing Authorities and 

stakeholders 

141. 

Changes in (evolution of) 

needs addressed by OPs (taking into 
account, to the extent possible, the 
changes before and after the pandemic). 

142. 

Number of new target 

groups addressed and/or target groups 
having benefited from stronger support 
by ESF  

143. 

Amount of financial 

resources allocated for new type of 
operations  

T1 Mapping  
•  Monitoring data (SFC data, 
Open data platform, 
Launchpad)  
T3 Consultation: 
•  National stakeholder 
targeted survey 
Stakeholder interviews 

• 

T6 Case studies: 
• 

Data and other secondary 
resources at MS level 
Case study on partnerships 
and social innovation 

• 

 
 
 
 
Criteria 

Evaluation questions 

Judgement criteria 

Indicators 

23.  Were the changes and measures 

introduced by CRII and CRII+, REACT-
EU and CARE relevant to MS needs in 
order to react to crises? Which 
changes and measures were the most 
appropriate? Which other changes 
and measures would have been 
useful (if any)? 

144. 

Extent to which the OPs 
were re-programmed to deal with the 
crisis. 

150. 

Evaluative judgement based 

on evidence from programming 
documents 

145. 

Degree to which the ESF 

151. 

Number and types of 

pursued objectives and financed 
operations focused on new needs due to 
the crisis  

146. 

Extent to which the 

introduced measures have contributed 
to contain the pandemic and alleviate its 
socio-economic effects 

147. 

Extent to which the 

introduced measures have contributed 
to address the consequences of the war 
against Ukraine 

groups mostly hit by the crisis which have 
not been supported (es. unemployed 
previously in employment, medical staff, 
independent workers, students, etc.) 
Number of new groups 

152. 

benefiting from introduced measures and 
type of support provided to them  

153. 

Number and types of needs 
due to the pandemic effects according to 
the stakeholder’ opinion not addressed 
by the ESF 

148. 

Evaluative judgement on 

154. 

Any needs due to the effects 

the basis of evidence collated on the 
degree to which the needs of target 
groups were met or not in practice and 
why 

149. 

Comparison point:  ESF and 

YEI before and after the COVID-19 
pandemic 

of the war according to the stakeholder’ 
opinion not addressed by the ESF 

24.  Are there any future trends including 

155. 

Whether future trends 

megatrends (such as demographic 
change, migration etc.) that could 
affect the future relevance of the ESF 
programmes and how? 

could bring about structural changes in 
the context, making, for example, more 
relevant to the ESF objectives targets 
such as older migrants, older workers 
and so on 

156. 

Statistics and evaluative 
judgement based on socio-economic 
demographic trends, based on available 
studies 

157. 

Statistics and evaluative 

judgement based on evidence on global 
trends triggered by the pandemic and 
Ukrainian crisis 

151 

• 

Tasks and sources 
Case studies on CRII, 
REACT-EU and CARE 
Interviews with Managing 
Authorities 
T7 Focus groups 

• 

•  Managing Authorities and 

stakeholders 

T1 Mapping  
•  Monitoring data (SFC data, 
Open data platform, 
Launchpad)  
T3 Consultation: 
•  National stakeholder 
targeted survey 

T6 Case studies: 
• 

Case study on partnerships 
and social innovation 
Case studies on CRII, 
REACT-EU and CARE 
Interviews with Managing 
Authorities and other 
stakeholders 
Desk analysis of national 
evaluations/documents 
(including AIRs as relevant) 

T7 Focus groups 

•  Managing Authorities and 

stakeholders 

T2 Context 
• 

Socio-economic data (e.g., 
Eurostat, OECD, World 
Bank) 
Policy expenditure data 
(e.g., Eurostat, European 
Commission) 

T3 Consultation: 

• 

• 

• 

• 

 
 
 
 
Criteria 

Evaluation questions 

Judgement criteria 

Indicators 

158. 

How did the EU intervention make a difference and to whom? 

EU added 

value 

25.  To what extent did EU support 
produce effects at national and 
regional levels that would otherwise 
not have taken place without it? 

o 

o 

o 

o 

Volume effects: Have the 
operations added to existing 
actions or directly produced 
beneficial effects that can be 
measured in terms of volume? 

Scope effects: Have the 
operations broadened existing 
actions by addressing groups, 
systems or policy areas that 
would otherwise not have been 
addressed? 

Role effects: Have the 
operations supported innovation 
and the transfer of ideas that 
have been subsequently rolled 
out in different contexts? To 
what extent has the EU support 
contributed to structural changes 
in national systems promoting 
employment, social inclusion and 
education and training? 

Process effects: Have Member 
State administrations and 
participating organisations 
derived benefits from being 
involved in the operations? To 
what extent are the effects of EU 
support likely to continue after 
the end of the funding, both at 

Tasks and sources 
•  National stakeholder 
targeted survey 

T7 Focus groups 
•  Managing Authorities and 

stakeholders 

T1 Mapping:  

•  Monitoring data (SFC data, 
Open data platform, 
Launchpad)  

T3 Consultation: 
•  National stakeholder 
Targeted survey 
• 
T5 Impact analysis 
• 
• 
T6 Case studies: 
• 
• 

CIEs 
RHOMOLO estimations 

EU strategic documents  
Desk analysis of national 
evaluations/documents 
(including AIRs, National 
Partnerships agreements, 
National Social Strategies, 
CSR and relevant literature 
as relevant) 

159. 

Volume effects: The ESF has 

increased the number of participants 
(people and entities) to national/regional 
policies and/or the geographical 
coverage of the policies. 

160. 

Scope effects: compared to 
existing actions, the degree to which ESF 
has reached new groups or harder-to-
reach groups, provided more 
opportunities of support, supported 
systems with specific actions, 
developed/extended policy areas.  

161. 

Role effects: the extent to 

which the ESF operations have 
supported the transfer of new ideas and 
innovation or/and contributed to 
structural changes. 

VOLUME 

164. 

‘Gross’ volume effects in 

terms of scale of output and results data 
(taking into account, to the extent 
possible, the changes before and after 
the pandemic and the war). 

165. 

CIE evidence from MS 
evaluations on ‘net’ volume effects 
(analysis to be carried out at IP level to 
the extent possible) 

SCOPE 

166. 

Proportion of specific 

groups reached by the ESF (or groups 
that would not have been supported 
otherwise) 

162. 

Process effects: the extent 

167. 

Extension in access to 

to which the ESF has brought about 
sustainable changes in the cooperation 
between administrations and other 
entities, integration between policy 
systems (e.g., social assistance and 
employment services, education and 
training, etc.) and empowerment of the 
actors involved in the operations.  

services (or services that would not have 
been offered otherwise)  

T7 Focus group 
•  Managing Authorities and 

168. 

New types of activities and 

stakeholders 

services provided by the ESF, in particular 
for hard-to-reach groups  

ROLE 

169. 

Number and value of ESF 

163. 

Comparison point:  ESF and 

YEI before and after the COVID-19 
pandemic/ Findings of interim thematic 
evaluation reports/ESF and YEI during 
the previous programming period 

actions mainstreamed into 
national/regional policies, activities or 
interventions 

170. 

Number and value of new 

policy instruments and services  

152 

 
 
 
 
Criteria 

Evaluation questions 

individual, institutional and 
policy levels? 

Judgement criteria 

Indicators 

Tasks and sources 

between 2007-2013/Findings from other 
related research or evaluations at EU or 
national level 

171. 

Evaluative judgement based 

on evidence of structural changes in 
national systems promoting TO 8, TO 9, 
TO 10 or TO 11 

PROCESS 

172. 

Evaluative judgement on 

enhanced capacity of the administrations 

173. 

Number and value of 

actions developed integrating services 
and activities  

174. 

Number of new strategies 

and partnerships established 

26.  What are the key results and impacts 

175. 

Extent to which the ESF 

177. 

Variation in the values of 

of the ESF programme that made a 
difference in comparison with the 
previous programming period? 

made a difference in 2014-2020 
assuming as comparison point key 
results of the previous programming 
period. 

results and output indicators (including 
soft outcomes, where available, measured 
using scoring systems and scales) in 
2007-2013 and 2014-2020. 

176. 

Comparison point:  ESF and 

178. 

Variation in micro and 

YEI during the previous programming 
period between 2007-2013 

macro impacts between 2007-2013 and 
2014-2020 

179. 

Qualitative evidence based 

on the perception of the stakeholders 

27.  To which group(s) of stakeholders did 

180. 

More involvement of 

182. 

Number and type of 

the ESF programme make a tangible 
difference in comparison with the 
previous programming period? 

stakeholders belonging to civil society, 
ONGs, etc. or public administration 
which were little or not involved in the 
previous programming period. 

181. 

Comparison point:  ESF and 

YEI during the previous programming 
period between 2007-2013 

stakeholder involved 

183. 

Evaluative judgement based 

on the role of the different stakeholders 
in design, implementing and 
monitoring/assessing the ESF measures 

28.  To what extent are the effects of the 

184. 

Scope of long-term effects 

186. 

Net effects at micro level of 

EU support likely to continue after the 
end of the funding, at participant, 

from CIEs (micro-level) and macro 

the ESF measures  

187. 

Macro effects of the ESF 

153 

T3 Consultation: 
•  National stakeholder 

T6 Case studies  
• 

Desk analysis of national 
evaluations/documents 
(including AIRs, National 
Partnerships agreements, 
National Social Strategies, 
CSR and relevant literature 
as relevant) 
T7 Focus group 

•  Managing Authorities and 

stakeholders 

T3 Consultation: 
• 

Stakeholders 

T6 Case studies 
Interviews 
• 
Desk analysis of national 
• 
evaluations/documents 
(including AIRs as relevant) 

T5 Impact analysis 
• 
• 

Counterfactual evaluations  
RHOMOLO estimations 

 
 
 
 
 
 
 
 
Criteria 

Evaluation questions 

organizational and/or policy levels? Is 
there a trend of transfer towards new 
funds such as the JTF and RRF? 

Judgement criteria 
impacts of the ESF RHOMOLO results 
(macro-level). 

185. 

Extent to which the 

changes occurred at organisational and 
policy levels are enduring 

29.  To what extent has the EU support 

189. 

Extent to which systems 

contributed to sustainable structural 
changes in national systems 
supporting employment, social 
inclusion and education and training? 

supporting employment, social inclusion, 
education and training have 
implemented new organisational 
solutions and addressed new objectives 
and new targets 

Indicators 

Tasks and sources 

188. 

Number of practices and 

solutions transferred to new funds     

T3 Consultation: 
• 

Targeted survey 

190. 

Number of (and qualitative 
evidence on) systemic changes brought 
about by the ESF at the level of: 

191. 
192. 
193. 

tools 

194. 
195. 
196. 

organisation  
type of services  
approach, methods and 

skills 
administrative capacity 
policy (e.g., structural 

reforms supported for instance in terms 
of curricular reforms, funding scheme of 
schools, organisation of teacher 
education and training quality 
development of ECEC)  

197. 
199. 

Evaluative judgement on the 

T6 Case studies 
Interviews 
• 
Desk analysis of national 
• 
evaluations/documents  

T3 Consultation: 
•  National stakeholder 

T6 Case studies (in this context 
of case studies, the extent to 
which ESF contributed to 
implement reforms will be 
assessed) 
• 
• 

Interviews 
Desk analysis of national 
evaluations/documents 

T7 Focus groups 
•  Managing Authorities and 

stakeholders 

T3 Consultation: 
• 
Participant survey 
•  National stakeholder 
Targeted survey 
• 

T6 Case studies 
Interviews 
• 
Desk analysis of national 
• 
evaluations/documents  

T6 Case studies 
Interviews 
• 

Sustainability, 

visibility and 

simplification 

30.  How visible to the public (in general 
and per MS/region/stakeholder 
group in particular) are ESF funded 
operations and YEI? 

198. 

Visibility of the ESF among 

national/regional institutions and 
different groups of stakeholders 

visibility and external knowledge of the 
ESF in terms of: 

200. 

ESF 

201. 

Role and objective of the 

Scope and volume of the 

support provided  

202. 
203. 

Results achieved 
Extent to which supported 
individuals/organizations are aware that 
the support received is co-financed by 
the EU via ESF 

31.  To what extent were processes in MS 
(e.g., selection processes, governance 

204. 

Extent to which the 

206. 

Share of resources unspent 

simplification solutions have been 
utilised 

154 

 
 
 
 
 
 
Criteria 

Evaluation questions 
processes etc.) be simplified? (to 
complement questions on efficiency) 

Source: Supporting study 

Tasks and sources 
Desk analysis of national 
evaluations/documents 
(including AIRs as relevant 

T7 Focus groups 
•  Managing Authorities and 

stakeholders 

Judgement criteria 

Indicators 

205. 

Extent to which the ESF 

207. 

Number and types of 

• 

entailed administrative burden 
exceeding the capacity of both 
administrations and stakeholders  

problems encountered by the Managing 
Authorities  

208. 

Number and types of 

problems encountered by the 
beneficiaries in reporting the 
expenditures and in monitoring the 
intervention  

155 

 
 
ANNEX II.D:  ADMINISTRATIVE COSTS  

Administrative  costs  were  calculated  on  the  basis  of  a  2018  study251  on  ESIF  administrative  costs  and 
burden. The data was obtained from the survey of programme authorities and beneficiaries. In total, 114 
ESF programme authorities were surveyed including 17 authorities involved in programme preparation, 35 
managing authorities, 25 certifying authorities and paying agencies and 38 audit authorities and certification 
bodies. The data gaps (data on the administrative costs of authorities who did not reply) were filled with 
data from similar respondents252.  

The sample used to estimate the administrative burden for beneficiaries was relatively small. Only 66 ESF 
beneficiaries  responded  to  the  survey,  and  the  variance  was  high.  No  YEI  beneficiaries  were  included 
among the respondents; therefore, it was assumed that the cost per million euros spent was the same for 
YEI beneficiaries as for ESF beneficiaries. The sample nevertheless covered most EU Member States and 
all thematic objectives. 

The  study’s  underlying  dataset  was  used  to  calculate  the  total  administrative  costs  associated  with 
implementing  the  ESF  and  YEI.  The  total  costs  for  the  entire  period  for  programme  authorities  were 
obtained by multiplying the average cost estimates per million euros spent (from the 2018 study) by the 
total eligible spending up to the end of 2023. Since the calculation of total costs here is based on eligible 
spending, whereas the 2018 study relied on allocations, the updated figures presented in this Staff Working 
Document differ from those in the published study.  

For beneficiaries, the cost range was constructed by applying the estimated average and median costs per 
million  euros  spent  (from  the  2018  study)  to  the  total  eligible  spending  up  to  the  end  of  2023.  For 
beneficiaries, two types of costs were considered: the costs of preparing a funding application (application 
costs) and project management costs. The project management costs consist of costs related to gathering 
information, submitting payment claims, fulfilling information and publicity requirements, keeping records, 
preparation and submission of data, preparing information required for project evaluation and other tasks. 
However, not all of these costs stem from cohesion policy regulatory requirements, and it was not possible 
to distinguish them from those arising under the national regulatory framework. 

However, the administrative cost estimates should be treated with caution. They are based on a 2018 study, 
which  does  not  take  into  account  the  regulatory  changes  introduced  thereafter  or  the  post-2020  crisis 
context.  In  addition,  the  estimates  are  not  derived  from  actual  costs  incurred  by  the  end  of  the  period. 
Instead, the 2018 cost estimates per million euros spent were extrapolated to obtain the total costs for the 
whole period. Finally, the sample size for the beneficiary survey is small and highly variable.  

The 2018 study also compared the administrative costs of the 2014–2020 programming period with those 
of  the  previous  period,  using  the  administrative  cost  data  from  the  2012  study253.  The  comparison  was 
possible only for the administrative costs of programme authorities. The 2012 study survey covered the 
programme  authorities  of  12  programmes  in  seven  Member  States.  Due  to  the  low  response  rate  from 
beneficiaries in the 2012 study (covering only four operational programmes out of 130), a comparison of 
beneficiary  costs  was  not  feasible.  Both  the  2018  and  2012  studies  applied  the  same  definition  of 
administrative costs; however, their approaches to data collection and analysis differed slightly254. 

251  European  Commission:  Directorate-General  for  Regional  and  Urban  Policy,  Spatial  Foresight  and  t33,  New 
assessment of ESIF administrative costs and burden, 2018. https://data.europa.eu/doi/10.2776/36663 
252 For more details see section 2.2. of the study. 
253 EPEC/COWI (2012) Study Measuring Current and Future Requirements on Administrative Cost and Burden of 
Managing the ESF, 2012. 
254 For more details see section 4 of the 2018 study. 

156 

 
 
 
ANNEX III:  OVERVIEW OF BENEFITS AND COSTS  

The complete analysis of the benefits and costs is available in the Supporting Study Annex V255.  

ANNEX IV:  STAKEHOLDERS CONSULTATION - SYNOPSIS REPORT  

This annex provides a synopsis of the stakeholder consultations conducted for the study supporting the ex-
post evaluation of the 2014-2020 European Social Fund and Youth Employment Initiative. 

Outline of the consultation strategy 

The consultation strategy for the study and respective consultation activities aimed to collect relevant data 
and/or opinions from key stakeholders to gather meaningful evidence to answer the evaluation questions 
under  all  evaluation  criteria.  The  consultations  complemented  the  mapping  of  ESF-related  investment 
priorities, target populations and types of operations, as well as the analysis of the socio-economic context, 
and addressed gaps in the documentary evidence base of the study.  
EU  and  national  level  stakeholders  were  consulted,  including  EU  level  policymakers  and  national  and 
regional  authorities  involved  in  the  management  and  coordination  of  the  ESF  and  YEI,  national  and 
regional authorities involved in employment, training, education and labour market policy, representatives 
of ESF beneficiaries and participants of the ESF and YEI, academic and research bodies, as well as EU 
citizens. Different consultation activities and methods were used, including a public consultation (PC), 
alongside a call for evidence with feedback submitted by organisations; targeted interviews with EU and 
national  level  stakeholders,  an  internet-based  survey  targeting  ESF  Managing  Authorities;  in-depth 
interviews conducted within the context of the case studies; and online thematic focus groups. 

Overview of consultation activities 

The public consultation aimed to provide an open channel for all interested stakeholders to provide their 
input to the study. The consultation took the form of an e-survey, with closed and open-ended questions 
addressing the key evaluation criteria and tailored to the different stakeholder groups.  
The targeted interviews were conducted with EU and national level stakeholders directly and indirectly 
linked to employment, training, education and labour market policy and management and delivery of ESF, 
including  EU  policymakers,  Ministries  of  Employment  and/or  Social  Affairs  and  Public  Employment 
Services, ESF Managing Authorities, as well as national-level socio-economic partners and civil society 
organisations representing groups relevant to ESF and YEI support.  
The internet-based survey targeted ESF MAs in EU27 Member States and the UK.  
As  part  of  the  27  case  studies,  380  additional  in-depth  interviews  were  conducted  with  country  level 
representatives involved in the implementation and monitoring of the ESF.  
The thematic focus groups targeted national authorities, beneficiaries, and other stakeholders at EU and 
national level with the aim of filling remaining knowledge gaps and validating the findings of the study. 
Six focus groups were carried out. 

Summary and reflection on challenges 

The consultation methods were implemented as planned, in line with the agreed consultation strategy. The 
targeted  consultations  posed  some  challenges,  such  as  delays  in  the  interview  programme  due  to  the 
unresponsiveness of several national level stakeholders. The impact of this on the consultation strategy 

255 Available at: Annexes. 

157 

 
 
 
 
 
 
 
 
was minimal, however, as the other consultations conducted allowed the research team to still gather in-
depth views of a range of stakeholders across the EU27. 

Information on consultation activities and stakeholder groups consulted 

Table A9 – Overview of stakeholders consulted in all consultation activities 

Stakeholder 
category 

Public 
Consultation 

Targeted Interviews 

MA 
survey 

Case 
studies 

Focus 
groups 

EU  level  policymakers  responsible  for designing  and  coordinating  the  ESF  and 
YEI 

National and regional authorities involved in the management and coordination of 
the ESF operational programmes 
National  and  regional  authorities  and  stakeholders  involved  in  employment, 
training, education and labour market policy 
Representatives of beneficiaries and participants of the ESF and YEI, including 
civil society organisations and social partners 
Academic and research bodies focusing on employment, training, education and 
labour market policy 

EU citizens and any other interested parties 

x 

x 

x 

x 

x 

x 

x 

x 

x 

x 

x  x 

x 

x 

x  x 

x  x 

x 

Public consultation and call for evidence 

The public consultation ran from 16 October 2023 to 23 January 2024. It targeted ESF MAs, beneficiaries 
of  ESF,  CSOs,  social  and  economic  partners,  academic  and  research  bodies  focusing  on  employment, 
training, education and labour market policy, audit/certifying authorities, Youth Guarantee coordinators, 
and EU citizens. The PC was disseminated through the European Commission’s website and social media 
channels, the Ecorys’ LinkedIn page, as well as using multipliers. The consultation was available in all 
official EU languages and hosted on EU Survey. It gathered input through closed multiple choice questions, 
with  open-text  questions  to  provide  additional  information.  Questions  were  grouped  by  evaluation 
criterion.  Respondents  were  routed  to  questions  based  on  their  profile,  including  the  nature  of  their 
engagement with ESF. 
704 respondents took part in the PC. The vast majority of respondents (99%; 697) came from within the 
EU, with all 27 Member States represented. The highest number of responses came from Poland (43%; 
305). Responses were also received from four non-EU countries256 (1% of all responses; 7). 
The PC received replies from 607 organisations (86%) and 97 individuals (14%), of which 96 were EU 
citizens  and  one  non-EU  citizen.  The  highest  number  of  responses  (47%;  330)  came  from  public 
authorities. Academic/research institutions and NGOs respectively accounted for 10% (72) and 9% (65) 
of respondents. Companies/businesses, business associations and trade unions together represented 6.5% 
of  total  respondents  (46).  Among  the  607  organisational  respondents,  36%  (216)  represented  large 
organisations, 31% (188) represented medium-sized organisations, 27% (165) small-sized 6% (38) micro-
sized organisations. 
Organisational respondents were asked about their role in the delivery of the ESF. Overall, 585 respondents 
had a direct role in delivery of the ESF (96% of all organisational respondents), whereas 4% of respondents 
had no direct role. Two thirds were beneficiaries of ESF funding for the implementation of a project (67%; 
405), while one fifth of respondents represented an ESF MA or Intermediate Body (20%; 121). A smaller 
proportion  of  organisational  respondents  (10%)  were  members  of  a  Monitoring  Committee  (3%;  19), 
representatives from an EU funds Coordinating Body (3%; 18), certifying or audit authorities (2%; 12), 

256 UK, Zimbabwe, Russia and Serbia 

158 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
and  referral  or  advocacy  organisations  including  civil  society  organisations  that  were  not  beneficiaries 
(2%; 10).  
The 22 organisational respondents with no direct role in the delivery of the ESF were asked about their 
familiarity with the ESF. 59% (13) were familiar with the goals and scope of the ESF as well as with at 
least one activity funded whilst 18% (4) had only a general idea of the goal and scope of the ESF, without 
any knowledge of specific activities funded. 9% (2) indicated that they had never heard of the ESF. 

There were 17 responses to the call for evidence document accompanying the PC. The responses came 
from Belgium (6), Sweden (2), France (2), Spain (2), Slovenia (1), Poland (1), the Netherlands (1), Ireland 
(1), and Czechia (1)257. 

The contributions highlight the importance of the ESF and YEI in addressing social exclusion, promoting 
employment, and supporting vulnerable groups. 

The  most  repetitive  arguments  or  topics  raised  by  the  contributors  include  the  need  to  simplify  the 
administrative  procedures and  reduce  the  bureaucratic  burden  associated  with  accessing  and  managing 
ESF and YEI funds (including NGOs, public authorities, and EU citizens). Many contributors emphasised 
the importance of targeting specific vulnerable groups, such as the Roma, people with disabilities, and 
young  people,  and  providing  tailored  support  to  address  their  unique  needs  (including  NGOs,  public 
authorities). The need to ensure the sustainability and long-term impact of ESF and YEI-funded projects 
was also a common theme, with many contributors stressing the importance of focusing on achieving long-
term  results  rather 
just  short-term  outcomes  (including  NGOs,  public  authorities,  and 
academic/research institutions). Additionally, several contributors suggested that the ESF and YEI should 
be  more  flexible  and  adaptable  to  respond  to  changing  social  and  economic  contexts  and  to  address 
emerging needs (including NGOs and public authorities). 

than 

In  terms  of  relevance  to  the  funds,  the  contributions  highlight  the  importance  of  the  ESF  and  YEI  in 
promoting employment and social inclusion, particularly for vulnerable groups (including NGOs, public 
authorities, and EU citizens). The funds support education and training initiatives that help individuals 
acquire  skills  and  improve  their  employability.  The  YEI  has  been  instrumental  in  supporting  youth 
employment initiatives,  particularly in  rural  areas and  disadvantaged  urban settings.  The  ESF  and  YEI 
have  also  supported  social  entrepreneurship  and  innovation,  particularly  in  the  context  of  the  social 
economy and circular economy. 

NGOs, such as Fundación Secretariado Gitano, ENSIE, and Eurodiaconia, highlighted the importance of 
the  ESF  and  YEI  in  supporting  vulnerable  groups  and  promoting  social  inclusion.  Public  authorities 
emphasised the need to simplify administrative procedures and to focus on achieving long-term impact. 
EU  citizens  (2  contributions)  highlighted  the  importance  of  the  ESF  and  YEI  in  supporting  local 
development and promoting social inclusion. 

However, not all contributions were positive, and some dissenting views were expressed. For example, 1 
contribution expressed disagreement with the EU's actions and policies, implying that the ESF and YEI 
funds are not being used efficiently. Some contributors, such as a citizen and a public authority criticised 
the bureaucratic obstacles and administrative burdens associated with accessing and managing ESF and 
YEI funds. Another noted that while the ESF projects have been beneficial for individuals, they have not 
led to structural changes or had a significant impact on the labour market. Additionally, some contributors 
argued that the ESF and YEI do not provide sufficient support for rural areas, disadvantaged urban settings, 
and individuals with disabilities, migrant, and refugee backgrounds. 

257European Commission: https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives/13450-European-
Social-Fund-and-the-Youth-Employment-Initiative-2014-2020-evaluation-/feedback_en?p_id=32356638 

159 

 
 
 
 
 
 
 
 
 
Despite  these  dissenting  views,  the  overall  assessment  of  the  ESF  and  YEI  is  positive,  with  many 
contributors  highlighting  their  importance  in  addressing  social  exclusion,  promoting  employment,  and 
supporting vulnerable groups. The contributions emphasise the need to simplify administrative procedures, 
ensure sustainability and long-term impact, and provide tailored support to address the unique needs of 
specific vulnerable. By addressing these challenges and limitations, the ESF and YEI can continue to play 
a crucial role in promoting social inclusion and employment across Europe. 

Interviews 

The  interviews  targeted  the  following  stakeholder  groups:  EU  level  policymakers  and  stakeholders 
working on ESF, YEI and employment; national/regional authorities managing ESF; national and regional 
ministries  and  government  bodies  responsible  for  employment,  training,  education  and  labour  market 
policy; ESF Monitoring Committees, PES; CSOs; social partners; and Youth Guarantee coordinators. The 
interviews included questions on all evaluation criteria as well as overarching questions to investigate the 
characteristics and objectives of the ESF. 
150  interviews  were  conducted  between  February  and  June  2023.  Of  these,  17  were  with  EU  level 
stakeholders and 133 were with national-level stakeholders in EU Member States and the UK. 

Table A10 – Targeted interviews completed 

Stakeholder 

DG EMPL (Units G.1, B.2, B.5, D.3, Geodesk for 
Italy,  Poland,  Slovakia, 
Czechia,  Denmark, 
Spain/Malta, Sweden)  
DG EAC (Unit A.2) 
BusinessEurope 
European Federation of Education Employers 
European Confederation of Trade Unions (CESI) 
European Youth Forum 
Lifelong Learning Platform (LLP) 
European Women’s Lobby 
Caritas Europe 
ERGO Network 
European Association for the Education of Adults 
(EAEA) 
Expert  group  on  quality  investment  in  education 
and training 
Total EU level interviews 
ESF Managing Authorities 

National  and  regional  ministries  and  other  public 
institutions  responsible  for  employment,  training, 
education and labour market policy 
ESF Monitoring Committees 
Public Employment Services (PES) 

Civil society organisations and networks 

Social  partners  (incl.  trade  unions,  employer 
organisations, chambers of commerce) 
Youth Guarantee coordinators 
National ESF partner organisations 
Total national level interviews 

Source: Ecorys, 2024 

Member States 

Number  of 
interviews 
conducted 
6 

1 
1 
1 
1 
1 
1 
1 
1 
1 
1 

1 

17 
34 

51 

3 
13 

14 

9 

7 
2 
133 

AT, BE, BG, CZ, DE, DK, EE, EL, ES, FI, 
FR, HR, HU, IE, IT, LT, LU, LV, MT, NL, 
PL, PT, RO, SE, SK, UK 
AT,  BG,  CY,  EE,  EL,  IE,  IT,  LT,  LU,  LV, 
MT, NL, PL, PT, RO, SE, SK, UK  

BG, DK, IE 
BE, BG, ES, FI, FR, HU, IE, IT, LT, LU, MT, 
PL, SE 
BE, CY, DE, DK, EE, ES, HR, IE, LU, PT, 
SE, SK 
BG, CY, DE, DK, ES, LU, MT, PL 

EE, FI, LT, LV, PT, SK 
RO 

Online survey of Managing Authorities 

The  internet-based  survey  targeted  all  ESF  MAs.  The  survey  focused  on  all  evaluation  criteria, 
investigating  MAs’  experience  with  their  operational  programme  and  perceived  challenges  in 

160 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
implementation. The survey ran for 10 weeks, until 2 June 2023 and had a total of 114 responses (61% 
response rate, as seen in the Table below).  

Table A11 – Response rate of MAs to the survey, by country 

Member State  Number of OPs 

Completed survey responses 

Response rate 

AT 

BE 

BG 

CY 

CZ 

DE 

DK 

EE 

FI 

FR 

GR 

HR 

HU 

IE 

IT 

LT 

LU 

LV 

MT 

NL 

PL 

PT 

RO 

SI 

SE 

SK 

ES 

UK 

Total responses 

Case studies 

1 

4 

3 

1 

3 

17 

1 

1 

2 

33 

17 

1 

5 

1 

29 

1 

1 

1 

1 

1 

17 

10 

2 

1 

2 

2 

23 

6 

187 

1 

1 

2 

1 

3 

9 

1 

1 

2 

15 

6 

1 

3 

1 

16 

1 

1 

1 

1 

1 

13 

5 

1 

1 

2 

2 

18 

4 

114 

100% 

25% 

67% 

100% 

100% 

53% 

100% 

100% 

100% 

45% 

35% 

100% 

60% 

100% 

55% 

100% 

100% 

100% 

100% 

100% 

76% 

50% 

50% 

100% 

100% 

100% 

78% 

67% 

61% 

The in-depth interviews carried out in the framework of the case studies focused on gathering additional 
insights on the delivery of OPs, and examples of good practices and/or potential misuses. A total of 380 
national stakeholder interviews were conducted for the case studies  from July to October 2023, with 
stakeholders from ESF MAs, implementing bodies, ESF beneficiary organisations, national and regional 
ministries, governmental departments, and other public authorities responsible for employment, training, 
education and labour market policy, social partners and CSOs. 

Thematic focus groups 

The  online  focus  groups targeted  ESF  MAs,  national authorities  responsible  for employment,  training, 
education and labour market policy; ESF beneficiaries or project partners; PES; EU-level civil CSOs and 
networks  representing  groups  relevant  to  ESF  and  YEI;  academic  and  research  bodies;  and  European 
Commission representatives. Six focus groups were conducted in June 2024 with a total of 62 attendants 

161 

 
 
 
 
 
 
Table A12 - List of participants in focus groups, by country and stakeholder type 

Country  
TO 8 focus group (12 participants) 
AT 
BE 
CY 
FI 
FR 
DE 
IE 
PL 
PT 
SK 
ES 
EU 
TO 9 focus group (15 participants) 
CZ 
FR 
FR 
DE 
EL 
HU 
HU 
LV 
LT 
MT 
MT 
PT 
ES 
EU 
EU 
TO 10 focus group (13 participants) 
BE 
EE  
EL 
EL 
EL 
EL 
HU 
IE 
LT 
LU 
EU 
EU 
EU 
TO 11 focus group (6 participants) 
HR 
HR 
CY 
CY 
MT 
PL 
Crisis Response tools (6 participants) 
CY 
IT 
LT  
LT 
MT  

Stakeholder type  

Managing Authority 
Managing Authority 
Ministry 
Ministry 
Representatives of beneficiaries and participants 
Ministry 
Implementing body 
Representatives of beneficiaries and participants 
National authority 
Managing Authority 
Managing Authority 
EU social partner 

Ministry 
Representatives of beneficiaries and participants 
Representatives of beneficiaries and participants 
Representatives of beneficiaries and participants 
Ministry 
Ministry 
Ministry 
Implementing body 
Ministry 
Representatives of beneficiaries and participants 
Representatives of beneficiaries and participants 
Implementing body 
Representatives of beneficiaries and participants 
EU NGO  
EU social partner 

Representatives of beneficiaries and participants 
Ministry 
Ministry 
Ministry 
Ministry 
Ministry 
Implementing body 
Implementing body 
Implementing body 
Representatives of beneficiaries and participants 
EU social partner 
EU social partner 
EU NGO 

Representatives of beneficiaries and participants 
Representatives of beneficiaries and participants 
Implementing body 
Implementing body 
Ministry 
Managing Authority 

Managing Authority 
Representatives of beneficiaries and participants 
Ministry 
Managing Authority 
Representatives of beneficiaries and participants 

162 

 
 
SE 
YEI (10 participants) 
BG  
BG 
IE 
LV 
ES 
ES 
HR 
HR 
HR 
EU 
 Source: Ecorys 2024 

Managing Authority 

Representatives of beneficiaries and participants 
Representatives of beneficiaries and participants 
Representatives of beneficiaries and participants 
National authority 
Representatives of beneficiaries and participants 
Managing Authority 
Ministry 
Ministry 
Ministry 
EU NGO 

Methodology for data processing 

Quantitative analysis of results of the public consultation included analysis of frequency distribution for 
each variable related to the closed-ended questions, and cross-tabulations between specific variables and 
characteristics of respondents and between specific variables, though the low number of responses has 
limited the possibilities for this. For the qualitative data analysis, data was classified by related variable 
and analysed to identify additional information.  
For the targeted interviews with EU and national level stakeholders, the write-ups were collected and 
exported into analytical grids, broken down by question and evaluation criterion. The research team used 
the analytical grids to carry out in-depth analysis to inform the relevant sections and annexes of the interim 
and final reports. 
The MA survey was analysed using both quantitative and qualitative methods (for the open questions). 
The  responses  to  the  open-ended  questions  provided  by  the  respondents  in  their  native  language  were 
translated into English. Survey findings were used by the research team for all sections of the interim and 
final reports.  
Information from the case studies interviews was used in the analysis of each case study report, which 
informed the analysis carried out for all sections of the final report.  
Discussions in the focus groups were summarised in a report drafted by the research team and  used to 
finalise the findings and fill gaps in all sections of the final report. 

Overview of results of consultation activities 

Effectiveness 

Achievement of programme milestones and enabling/hindering factors: Stakeholders overall agreed 
that ESF achieved its programme milestones to a large extent. MAs frequently highlighted the importance 
of  effective  governance  and  implementation  approaches,  sufficient  ex-ante  and  ongoing  analysis  of 
priorities at the Member State and regional levels, and synergies with other policy instruments and funding 
sources.  Beneficiaries,  CSOs  and  social  partners  reported  that  effective  target  group  identification  and 
reach drawing on the expertise of actors ‘on the ground’ and well-developed networks were key factors 
supporting the achievement of milestones. By contrast, hindering factors reported included difficulties in 
monitoring participants’ development or results after the completion of interventions. This was particularly 
raised  by  stakeholders  in  Member  States  that  were  reliant  on  surveys  or  re-contacting  participants  as 
opposed  to  administrative  data.  Administrative  burdens  or  costs  were  also  frequently  reported  as  a 
hindering factor by beneficiaries, particularly smaller or less experienced organisations for which capacity 
emerged as an issue.  
Contribution to CPR thematic objectives and wider EU policies:  In terms of  employment, 87% of 
respondents to the MA survey felt that the ESF and YEI had contributed to a ‘large’ or ‘moderate’ extent 
to the European Pillar of Social Rights principle on equal opportunities and access to the labour market. 
Over 9 in 10 of the 114 respondents to the MA survey reported a positive contribution to the EU2020 
employment rate target. These views were echoed in interviews at the EU and national levels across all 

163 

 
 
 
stakeholder groups. The only contrasting view was from a small number of CSOs at the EU and national 
level  who  felt  that  the  ESF  could  still  do  more  to  support  the  employability  of  disadvantaged  or 
marginalised groups in particular.  
While  remaining  positive  overall,  stakeholder  views  on  the  contribution  to  social  inclusion  and 
combatting poverty were slightly more nuanced. While 65% of MA survey respondents felt that Thematic 
Objective  (TO)  8  operations  met their  objectives to a  large extent,  the  equivalent figure for  TO  9  was 
slightly lower at 59%. A number of CSOs interviewed felt that operations struggled to effectively support 
the most marginalised groups in some specific contexts.  
Stakeholders consulted overall felt that the ESF and YEI made a positive contribution to expanding access 
to education and training. Case study interviews conducted for the four IPs within TO 10 show this, as 
do the results of the MA survey, where around 8 in 10 respondents felt that TO 10 operations had met their 
objectives to a large (63%), moderate (15%) or small (2%) extent.  
There  is  less  evidence  from  stakeholder  consultations  on  the  contribution  of  the  ESF  to  institutional 
capacity building. Most EU and national stakeholders consulted had very limited or no insights into TO 
11  operations.  Two-thirds  of  respondents  to  the  MA  survey  reported  that  they  did  not  know  if  TO  11 
operations had met their objectives.  
Role in implementing Country Specific Recommendations: All stakeholder groups expressed positive 
views on the contribution of the ESF and YEI to implementing CSRs. However, stakeholders were less 
able to  assess the  extent  of  this  contribution,  often  citing the difficulty of  disaggregating  it  from  other 
influencing factors. MAs in particular felt that ESF made a positive contribution to implementing the CSRs, 
but that other national political priorities also influenced the focus of programming.  
Effectiveness in reaching intended target groups: Overall, the perception from stakeholders is that ESF 
and  YEI  actions  reached  the  intended  target  groups,  though  the  extent  of  this  depends  heavily  on  the 
national/regional context and the specific intervention itself. 93% of MAs responding to the MA survey 
held the view that operations had reached unemployed, 92% that they had reached women and 88% that 
they had reached the long term unemployed. However, these shares were notably lower for older workers 
(46%), the inactive and unemployed aged 55+ (53%), those with disabilities (63%), Roma (22%) and the 
homeless  (22%).  Similarly,  ESF  beneficiaries  interviewed  for  the  case  study  on  IP.8ii-YEI  operations 
highlighted challenges in reaching the most vulnerable. However, interviewees across stakeholder groups 
and MA survey respondents felt that targeting harder to reach groups had been positively supported by the 
requirement to allocate a minimum 20% of the ESF allocation to TO 9. 
Changes to the target groups from ESF support: All stakeholder types overall held the view that the 
ESF brought positive changes to target groups. Beneficiaries, MAs, CSOs and social partners interviewed 
held  the  view  that  TO  8  increased  opportunities  for  participants  to  find/maintain  a  job,  improve  their 
working  conditions  and  earn  more  or,  in  the  case  of  companies,  to  improve  their  productivity  and 
profitability. Similarly, all stakeholder groups held the view that operations under TO 9 contributed to 
improved  employment  rates,  including  for  the  most  disadvantaged,  although  at  different  degrees. 
Stakeholder  interviews  confirmed  that  operations  providing  vocational  and  post-secondary  education, 
under  TO  10  had  a  positive  contribution  to  improved  educational  environments  for  both  students  and 
teachers, setting the basis for better educational achievements as well as limiting early school leaver rates. 
In general, people taking part in ESF-supported operations declared that projects activities helped them in 
both improving their competencies and qualifications and in making their career choices.  
Structural changes from ESF: Stakeholders generally held the view that structural changes had taken 
place, but that the presence and scale of such contributions was relatively limited against other national 
drivers. MA and government representatives commonly reported in interviews that significant changes in 
employment, social inclusion and education policies and systems tend to be implemented in response to 
national  needs  (rather  than  being  caused  by  the  ESF).  Stakeholders  consulted  from  all  groups  also 
commonly  noted  that  other  cohesion/structural  funds,  alongside  national  policy reforms  and  strategies, 
were  equally  or  more  significant  drivers  of  change.  Beneficiaries  held  the  view  that  structural  issues 
underlying the disadvantages some groups faced could not be significantly addressed by EU support, as it 
formed a small part of overall funding for reform in Member States.  
Effectiveness in implementing horizontal principles:  Stakeholders held mixed views of the extent to 
which horizontal principles were implemented and influenced implementation. EU level stakeholders were 
164 

 
often unsatisfied with the extent of the use of horizontal principles to inform ESF operations. Conversely, 
national and regional level stakeholders, particularly MAs, argued that horizontal principles were taken on 
board in ESF operations in their countries. For example, 93% of MA respondents reported that the principle 
of gender equality, equal opportunities and non-discrimination was applied to a large or moderate extent. 
By contrast, interviews with beneficiaries highlighted more limited experience and institutional capacity 
on how to concretely support gender mainstreaming.  

Efficiency 

than  half  of  MAs 

Overall, stakeholders, particularly EU policymakers and MAs noted that administrative procedures had 
improved and simplified during the programming period, thanks to simplified cost options, digitalisation, 
and online tools. Stakeholders viewed simplification as a means to reduce administrative burdens in ESF 
procedures:  89%  of  MAs  (101  of  114)  in  the  survey  found  that  simplified  cost  options  improved 
operational  efficiency,  and  61%  (358  of  585)  of  PC  respondents  involved  in  ESF  management  and 
coordination considered these aspects adequate. However, CSOs, and beneficiaries interviewed pointed 
out that beneficiaries' lack of knowledge and familiarity with ESF procedures hindered implementation of 
these  simplifications.  National  and  regional  stakeholders  recommended  ensuring  flexibility  in  these 
options to adapt to external shocks, tailoring services to participants' needs, and simplifying reporting and 
auditing requirements. Digitalisation of application and reporting procedures, commonly cited by national 
MA and beneficiary stakeholders, was another improvement. 
Despite  these  improvements,  administrative  burdens  were  widely reported  by stakeholders  across  all 
groups  as  hindering  efficiency.  More 
the  survey  noted  significant 
administrative/regulatory burdens affecting beneficiaries (61%; 69 of 114)  and MAs (51%; 58 of 114). 
Audit requirements were reported as a hindrance by 43% of MAs in the survey (49 of 114) and identified 
as  an  unnecessary  administrative  burden  by  stakeholders  consulted  through  case  study  research  and 
national-level  interviews.  Other  common  issues  included  beneficiaries'  lack  of  administrative  capacity, 
mentioned by various stakeholders, and strict data protection rules, reported by beneficiary organisations 
and  MAs.  Nonetheless,  some  MAs  and  public  authorities  interviewed  found  the  administrative  level 
necessary and justified for transparent fund management. In addition, positive feedback was noted by MAs 
interviewed  for  the  case  studies  regarding  clear  guidance  and  support  initiatives,  and  by  beneficiaries 
regarding straightforward administrative and application processes. 
MA stakeholders reported successful efforts by ESF monitoring committees at the national level to enhance 
monitoring  and  evaluation  efficiency  through  digitalisation  and  online  tools.  89%  of  MA  survey 
respondents (101 of 114) and 69% of PC respondents (406 of 585) reporting that evaluation requirements 
did not significantly hinder efficiency. However, 45% of MA survey respondents (51  of 114) indicated 
that the large number of indicators hindered efficiency, and some beneficiary stakeholders criticised their 
effectiveness  in  measuring  the  success  of  operations.  Other  monitoring  challenges reported  in  the  case 
study  interviews  included  the  lack  of  detailed  linkage  between  expenditure  and  indicators  and  overly 
quantitative indicators.  

in 

Coherence 

Internal  coherence  of  dedicated  EU  support:  Stakeholders  overall  held  the  view  that  dedicated  EU 
support delivered through the ESF and YEI demonstrated strong internal coherence, with complementarity 
both within and between TOs and their respective IPs. Interviews conducted for the case studies indicated 
that  centralised  mechanisms  to  ensure  appropriate  funding  and  operational  objectives,  using  EU-level 
policies as a reference point, helped reduce overlap and reinforced complementarities. Regarding REACT-
EU,  MAs  noted  in  case  study  interviews  that  its  role  as  a  crisis  reaction  tool,  facilitating  project 
continuation, contributed to good internal coherence. Some isolated examples of potential overlap were 
provided by stakeholders, who noted instances where mechanisms to ensure coherence were absent.  
Coherence with other EU Funds: Stakeholders generally found dedicated EU support to be coherent with 
other  EU  funds.  EU-level interviewees,  MAs  and public  authorities  consulted through the case  studies 
noted strong coherence between ESF and ERDF and FEAD unlike the interviewees in the frame of the 
2014-2020 ERDF ex-post evaluation. Case study interviewees predominantly highlighted coherence with 

165 

 
 
ERDF, reflected in PC results where 61% of respondents reported ESF complemented or reinforced ERDF 
(386  of  638).  Fewer  respondents  indicated  coherence  with  Erasmus+  (48%;  309)  and  the  European 
Agriculture Fund for Rural Development (27%; 171). However, MA and public authority stakeholders 
reported  practical  difficulties  in  creating  complementarity  in  practice,  such  as  differing  administrative 
requirements  across  funds  and  the  lack  of  coordinated  announcements  or  calls  for  co-financing 
opportunities.  
Coherence with EU policies: The ESF and YEI programmes were seen as strongly coherent with other 
EU policies by stakeholders consulted. EU and national stakeholders in the interviews shared that Member 
States  often  used  key  EU-level  policies  as  the  main  reference  when  designing  interventions.  EU 
stakeholders particularly noted that ESF programming related to employment was most coherent with EU 
policy objectives. The Europe 2020 Strategy, the European Pillar of Social Rights, and the European Skills 
Agenda were the most frequently referenced policies in national interviews.  
Coherence at a national level: National and regional stakeholders interviewed largely felt that ESF and 
YEI support aligned well with other national and regional initiatives. This view was echoed in the MA 
survey, where 84% of respondents (96 of 114) reported that ESF operations were largely coherent with 
national/regional  policies.  However,  some  MA  and  beneficiary  stakeholders  mentioned  challenges  in 
aligning local needs with ESF TOs.  

EU added value 

Volume effects: Stakeholders consulted in the interviews, PC and case study consultations held the view 
that ESF/YEI funding significantly increased the volume of activities and support, surpassing what could 
be achieved with national resources alone. 94% of MA survey respondents (107 of 114) and 86% of PC 
respondents (568 of 660) reported that the ESF generated significant volume effects, through increasing 
the quantity and scale of activities and projects. EU and national stakeholders interviewed corroborated 
this  view,  emphasising  that  ESF/YEI  funding  enabled  more  extensive  activities  than  would  have  been 
possible with only national or local resources.  
Scope effects: ESF and YEI interventions were widely recognised by all stakeholders interviewed for their 
positive  impact  in  broadening  the  scope  of  support,  reaching  previously  underserved  groups  and 
introducing  new  measures.  This  extended  to  target  groups  such  as  migrants,  Roma,  NEETs,  and  other 
vulnerable populations, which were not adequately covered in the previous programming period. The MA 
survey supported these findings, with 87% of respondents (99 of 114) reporting significant scope effects 
in target group reach and policy areas. Several examples mentioned by MA and beneficiaries consulted 
through the interviews and case studies referred to the re-orientation of national policies and introduction 
of new types of support for NEETs258.  
Role effects: Stakeholders reported mixed outcomes across different TOs on the contribution of the ESF 
and YEI to innovation, idea transfer, and changes in national systems and structures. There was stronger 
consultation evidence of role effects in areas such as gender equality, labour market policies, and company 
competitiveness. 58% of respondents to the MA survey (69 of 114) reported observing role effects due to 
ESF and YEI, indicating some level of innovation and structural changes. Role effects were recognised by 
PC  respondents,  though  less  emphatically,  with  41%  of  respondents  (269)  acknowledging  that  ESF 
supported innovative service delivery methods.  
Process effects: Stakeholders consulted through the MA survey, case studies, and interviews held the view 
that ESF/YEI led to process effects through the development of new competencies and tools, as well as the 
professionalisation of implementing bodies. 61% of respondents to the MA survey (69 of 114) reported 
process  effects,  such  as  enhanced  collaboration  among  stakeholders  toward  long-term  objectives  and 
increased  capacity  of  government  administrations  and  beneficiaries.  Stakeholders  consulted  in  the 
interviews  and  case  study  research  expressed  that  ESF  management  and  implementation  have  led  to 
cumulative learning and professionalisation. 
Sustainability of added value: Stakeholders widely expect the positive effects from ESF and YEI support 
to continue beyond the end of EU funding. 88% of MA survey respondents (100 of 114) believed that the 

258 BG, FR, LT, LV 

166 

 
 
 
results  for  participants  are  likely  to  be  sustained  after  funding  ends.  Similarly,  many  beneficiaries 
interviewed  highlighted  ESF’s  long-lasting  impact  on  individuals,  particularly  through  upskilling  and 
training measures, noting improvements in personal confidence and wellbeing. On the other hand, MA and 
public authority stakeholders raised concerns about over-reliance on EU funding. 

Relevance 

The relevance of ESF and YEI operations to specific target groups varied across different stakeholder 
groups. MA survey respondents perceived ESF and YEI operations as relevant to many target groups, such 
as women, which 88% of MAs found to be adequately prioritised (110), the unemployed (87%; 99), and 
the long-term unemployed (82%; 94). However, there were concerns about the adequacy of support for 
specific groups like Roma, found by only 27% of MAs surveyed (21) to be adequately prioritised. Lower 
relevance was reported for TO 11 (19% of respondents). IP8.ii-YEI operations were generally very relevant 
for  young  NEETs,  particularly  in  Member  States  most  affected  by  the  2008-2012  economic  crisis. 
Different stakeholder types expressed the view that the involvement of stakeholders in the design and 
implementation of ESF/YEI operations supported relevance. According to respectively 32% and 39% of 
MA survey respondents, NGO/CSOs and social partners were properly involved in the implemented ESF 
operations at the national/regional level. 
Finally, stakeholders consulted had mixed views on the flexibility and adaptability of ESF/YEI OPs to 
changes in the implementation context. 53% of MA survey respondents viewed the changes and measures 
introduced under REACT EU as relevant to their Member State’s needs to react quickly to crises, while 
this was only the case for 17% in the case of CARE operations. Nonetheless, as captured in the case study 
stakeholder consultations, all stakeholder types particularly in countries with a high influx of refugees259, 
found CARE relevant in directly addressing the needs of refugees. 

Visibility 

Stakeholders directly involved in the management and delivery of ESF operations had a positive view of 
visibility. According to the PC, 79% of these respondents (461) rated communication as an adequate aspect 
of  the  administrative  arrangements.  90%  of  respondents  to  the  MA  survey  (113)  indicated  that  public 
visibility had increased to a large or moderate extent. Furthermore, 79% of MA survey respondents (90) 
noted that more emphasis was placed on visibility actions compared to the previous programming period. 
Successful visibility actions reported by stakeholders included dedicated websites, the display of the EU 
logo, and specific communication campaigns and events. Moreover, PC respondents highlighted that the 
multiplier effect was significant, with ESF recipients learning about the support through their employer 
(49%), educational or training institution (32%), or employment services (19%). 

259 BG, HU, PL 

167 

 
 
 
 
 
 
STUDY SUPPORTING THE EX-POST EVALUATION OF THE 2014-2020 EUROPEAN SOCIAL FUND AND 
YOUTH EMPLOYMENT INITIATIVE  

ANNEX V: KEY DATA AND ADDITIONAL INFORMATION 

Legal requirements 

During  the  2014-2020  period,  the  ESF  was  governed  by  the  European  Parliament  and  Council  Regulation 
1303/2013260 (Common Provisions Regulation) and by the Regulation 1304/2013 on the European Social Fund 
(ESF)261. The  evaluation was carried out in accordance with the  Art 57 of the Regulation (EU) No 1303/2013 
which requires the Commission to carry out and complete by 31 December 2024, with the assistance of external 

experts, an ex-post evaluation, to assess the effectiveness and efficiency of the Fund and contribution to the Union 

strategy for smart, sustainable and inclusive growth in accordance with specific requirements established in the 
Fund-specific rule262. 

Table A13: Europe 2020 strategy headlines indicators - EU27 (2013-2022) 

Topic 

Headline 
indicator 

2013  2014  2015  2016  2017  2018  2019 

2020 

2021 

2022 

Target 
2020 

Target 
2030 

Employment 

Education 

Employment 
rate age 
group 20-64, 
total (% of 
population) 

Employment 
rate age group 
20-64, females 
(% of 
population) 

Employment 
rate age group 
20-64, males 
(% of 
population) 
Early leavers 
from 
education and 
training, total 
([*]) (% of 
population 
aged 18-24)  
Early leavers 
from education 
and training, 
females (*) (% 
of population 
aged 18-24) 
Early leavers 
from education 
and training, 

66.8 

67.5  68.5  69.6  70.9  71.9  72.7 

71.7 

73.0  74.6 

75.0 

≥78 

61.2 

62.0  63.0  64.1  65.2  66.3  67.1 

66.2 

67.6  69.2 

72.4 

73.1  74.1  75.2  76.5  77.6  78.4 

77.2 

78.5  80.0 

: 

: 

11.8  11.1*  11.0  10.6  10.5  10.5  10.1 

10.0 

9.8** 

9.6 

<10.0 

≤9.0 

10 

9.4* 

9.4 

9.1 

8.9 

8.7 

8.4 

8.1 

8** 

8.0 

13.6  12.7*  12.5  12.1  12.1  12.1  11.8 

11.9 

11.5**  11.1 

: 

: 

260 Regulation (EU) No 1303/2013 of the European Parliament and of the Council of 17 December 2013 laying 
down common provisions on the European Regional Development Fund, the European Social Fund, the Cohesion 
Fund, the European Agricultural Fund for Rural Development and the European Maritime and Fisheries Fund and 
laying  down  general  provisions  on  the  European  Regional  Development  Fund,  the  European  Social  Fund,  the 
Cohesion  Fund  and  the  European  Maritime  and  Fisheries  Fund  and  repealing  Council  Regulation  (EC)  No 
1083/2006. Regulation - 1303/2013 - EN - EUR-Lex 
261 Available at: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32013R1304 
262 Regulation (EU) No 1304/2013 of the European Parliament and of the Council of 17 December 2013 on the 
European Social Fund and repealing Council Regulation (EC) No 1081/2006. Available at: https://eur-
lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2013:347:0470:0486:EN:PDF 

168 

 
 
 
 
 
 
 
 
 
 
 
 
 
STUDY SUPPORTING THE EX-POST EVALUATION OF THE 2014-2020 EUROPEAN SOCIAL FUND AND 
YOUTH EMPLOYMENT INITIATIVE  

males (*) (% of 
population 
aged 18-24) 
Tertiary 
educational 
attainment, 
total (*) (% of 
population 
aged 30-34) 
Tertiary 
educational 
attainment, 
females (*) (% 
of population 
aged 30-34) 
Tertiary 
educational 
attainment, 
males (*) (% of 
population 
aged 30-34) 

People at risk 
of poverty or 
social 
exclusion 
(Thousand 
persons) 

35.6  36.5*  37.3  37.8  38.6  39.5  40.4 

41.2 

41.9**  42.8 

> 40.0 

≥45.0 
(25-34 
years old) 

40.1 

41*  42.2  42.9  43.9  44.8  45.7 

46.3 

47.3**  48.4 

: 

31.2 

32*  32.5  32.7  33.2  34.2  35.2 

36.1 

36.6**  37.4 

: 

: 

104 
946 

103 
556 

98 
142 

95 
066 

92 
201 

94 
771*** 

95 581 

95 
324 

Lift 20 
million 
people 
out of 
poverty or 
social 
exclusion: 

Lift 15 
million 
people 
out of 
poverty or 
social 
exclusion: 

Poverty or 
Social 
Exclusion 

: 

People at risk 
of poverty or 
social 
exclusion (% of 
population)  
Source: Supporting study 
([*]) Break in time series in 2014 (switch from ISCED 97 to ISCED 2011).   
** Break in time series due to changes in EU-Labour Force Survey definition; 
*** data 2013-2019 estimated; in 2020 break in time series due to changes in EU-SILC survey; 

:  24.0  23.7  22.4  21.7  21.1  21.6*** 

21.7  21.6 

169 

 
 
 
  
 
 
  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
STUDY SUPPORTING THE EX-POST EVALUATION OF THE 2014-2020 EUROPEAN SOCIAL FUND AND 
YOUTH EMPLOYMENT INITIATIVE  

Table A14: Distribution of EU and national planned funds by EU fund, country and category of region 
(national co-financing included). Top-up funding of REACT-EU and YEI by country in 2022 (million 
EUR, %). 

MS 

ESF 

Total ESF  REACT-EU 

YEI 

Total 

Total (%) 

Less dev. 

Transition  More Dev. 

AT 

BE 

BG 

CY 

CZ 

DE 

DK 

EE 

ES 

FI 

FR 

GR 

HR 

HU 

IE 

IT 

LT 

LU 

LV 

MT 

NL 

PL 

PT 

RO 

SE 

SI 

SK 

UK 

42.2 

833.6 

875.7 

114.2    

681.2 

1 478.5 

2 159.7 

1 735.6 

176.8 

4 500.7    

176.8 

495.5 

164.0 

266.0 

65.0 

989.9 

2 516.8 

2 122.0 

281.3 

4 530.3 

193.2 

120.3 

39.5 

29.6 

4 151.0 

8 398.9 

12 549.9 

896.8    

13 446.7 

76.2 

334.6 

410.8 

682.2 

55.6    

12.8    

466.5 

695.0 

1 735.6    

4 005.2    

682.2    

0.7% 

1.7% 

1.4% 

0.2% 

3.0% 

8.9% 

0.3% 

0.5% 

694.5 

4 688.6 

4 905.7 

10 288.8 

5 937.3 

3 030.2 

19 256.3 

12.8% 

1 380.3 

2 397.0 

5 753.8 

9 531.2 

1 219.4 

1 141.8 

11 892.4 

1 036.5 

1 036.5 

89.9    

1 126.5 

2 579.3 

1 276.1 

1 180.9 

5 036.2 

1 664.4    

5 167.2    

1 664.4 

5 724.2 

832.7 

557.1 

832.7 

100.0 

530.0 

346.5 

141.6 

587.4 

224.5 

108.3 

204.4 

5 723.6 

2 418.9 

6 179.0 

1 178.8 

0.7% 

7.9% 

3.8% 

1.6% 

4.1% 

0.8% 

8 062.7 

972.8 

7 739.6 

16 775.1 

8 053.3 

2 362.4 

27 190.9 

18.0% 

1 412.2    

762.0    

1 412.2 

46.0 

69.2 

1 527.4 

39.1 

39.1 

762.0 

168.3 

73.2    

22.5 

63.1 

111.2    

168.3    

1 020.6 

1 020.6 

292.0    

112.3 

847.6 

279.5 

1 312.6 

1.0% 

0.1% 

0.6% 

0.2% 

0.9% 

13 770.5    

1 378.5 

15 149.1 

7 947.0 

122.0 

5 093.9    

474.3    

2 794.5    

638.3 

330.8 

8 707.3 

5 424.7 

1 436.6 

1 436.6 

424.1 

182.1 

898.5 

2 976.6 

1 279.2 

1 843.3 

5 204.3 

8 326.8    

80.1 

431.8 

234.0 

293.1 

13.2 

491.1 

586.9 

15 816.1 

10.5% 

501.8 

330.0 

132.5 

20.7 

187.5 

532.0 

9 640.9 

5 988.6 

1 862.2 

932.4 

3 655.1 

8 858.8 

6.4% 

4.0% 

1.2% 

0.6% 

2.4% 

5.9% 

Total 

59 505.1 

16 418.8 

44 378.5 

120 302.4 

20 080.6 

10 465.4 

150 848.4 

100.0% 

Source: Supporting study 

170 

 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
 
 
 
 
 
STUDY SUPPORTING THE EX-POST EVALUATION OF THE 2014-2020 EUROPEAN SOCIAL FUND AND 
YOUTH EMPLOYMENT INITIATIVE  

Table A15: Total outputs and results of ESF, YEI and REACT-EU reported by the Member States totals 
at the end of 2022 

OUTPUTS 

RESULTS 

Countr
y 

CO01 - 
unemployed
, including 
long-term 
unemployed 

CO03 - 
inactive 

CO05 - 
employed, 
including 
self-
employed 

AT 
BE 
BG 
CY 
CZ 
DE 
DK 
EE 
ES 
FI 
FR 
GR 
HR 
HU 
IE 
IT 
LT 
LU 
LV 
MT 
NL 
PL 
PT 
RO 
SE 
SI 
SK 
UK 

94 780 
979 885 
230 476 
15 124 
247 827 
751 442 
12 920 
71 581 
3 983 058 
102 714 
3 838 286 
500 955 
144 222 
704 254 
130 679 
4 165 124 
238 636 
6 011 
173 614 
7 182 
490 042 
1 842 585 
1 206 841 
334 460 
154 336 
83 246 
495 395 
1 234 463 

Total 

22 240 138 

174 818 
388 554 
1 487 591 
26 
86 220 
1 276 518 
74 796 
36 401 
3 044 045 
147 237 
1 603 563 
171 682 
141 681 
606 720 
136 758 
7 100 435 
539 650 
8 856 
97 111 
14 153 
195 887 
4 200 695 
1 122 361 
1 052 493 
52 421 
48 113 
580 724 
583 716 
24 973 
225 

28 698 
290 696 
594 674 
136 421 
556 385 
1 410 386 
37 711 
145 471 
1 768 415 
186 102 
948 627 
761 290 
216 489 
1 332 439 
62 752 
1 347 331 
358 611 
55 367 
187 223 
193 469 
171 363 
3 198 296 
1 364 653 
579 921 
259 657 
156 410 
239 270 
768 883 
17 357 
010 

CR01 - 
inactive 
participant
s engaged 
in job 
searching 
upon 
leaving 

CR02 - 
participant
s in 
education 
or training 
upon 
leaving 

CR03 - 
participants 
gaining a 
qualificatio
n upon 
leaving 

CR04 - 
participants 
in 
employment
, including 
self-
employment
,  upon 
leaving    

CR06 - 
participants 
in 
employment
, including 
self-
employment
, 6 months 
after leaving 

CR07 - 
participant
s with an 
improved 
labour 
market 
situation 6 
months 
after 
leaving 

14 412 
11 888 
18 470 
23 
3 899 
64 705 
1 133 
1 319 
307 807 
2 220 
535 317 
44 084 
5 172 
13 244 
4 785 
195 138 
5 995 
247 
1 635 
362 
16 342 
110 748 
75 131 
36 453 
3 582 
172 
750 
73 602 

16 091 
148 750 
502 622 
735 
21 305 
335 032 
2 149 
19 108 
398 777 
10 855 
389 031 
48 135 
3 478 
45 997 
81 563 
324 953 
27 556 
0 
19 861 
3 814 
16 922 
66 518 
81 565 
101 742 
24 645 
4 835 
125 744 
209 190 

93 789 
142 366 
219 549 
2 282 
247 031 
1 594 554 
9 400 
60 676 
2 742 729 
7 073 
595 516 
361 900 
31 156 
631 419 
116 862 
527 807 
159 897 
1 465 
42 512 
15 362 
25 034 
1 211 027 
365 887 
133 651 
44 885 
51 152 
26 745 
825 110 

22 515 
226 504 
36 791 
5 558 
131 053 
272 269 
3 739 
18 120 
1 506 625 
21 387 
1 418 080 
74 615 
43 606 
415 540 
25 761 
801 370 
115 923 
826 
39 284 
3 888 
160 234 
1 146 398 
351 099 
150 164 
36 808 
40 074 
56 246 
313 073 

29 067 
436 647 
74 612 
4 498 
113 961 
119 595 
39 453 
22 329 
516 559 
24 770 
2 158 390 
184 154 
77 749 
116 042 
14 120 
1 405 332 
33 931 
2 566 
37 755 
1 822 
151 192 
763 389 
176 063 
20 773 
34 695 
34 
138 729 
156 554 

3 044 
57 774 
244 240 
1 957 
16 431 
117 594 
3 679 
4 380 
30 662 
19 502 
278 340 
35 200 
59 945 
72 284 
5 225 
100 873 
11 426 
0 
14 374 
2 007 
77 995 
70 035 
44 033 
1 670 
91 003 
0 
2 101 
70 866 

1 548 635 

3 030 973 

10 286 836 

7 437 550 

6 854 781 

1 436 640 

Source: Supporting study. Elaboration based on SFC2014 data up to end of 2022, extracted in September 2023 

171