COMMISSION STAFF WORKING DOCUMENT Synthesis of the findings of the evaluations of European Structural and Investment Funds Programmes 2014-2020 Accompanying the document REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS European Structural and Investment Funds 2014-2020 2020 Summary report of the programme annual implementation reports covering implementation in 2014-2019
Date: 2021-04-28
Institutions: Directorate-General for Regional and Urban Policy, European Commission
Subjects: Cohesion Fund, EAFRD, EU Member State, EU programme, European Maritime and Fisheries Fund, European Regional Development Fund, European Social Fund, European Structural and Investment Funds, activity report, investment promotion

EN 

  EN 

   EUROPEAN COMMISSION  Brussels, 28.4.2021  SWD(2021) 86 final  COMMISSION STAFF WORKING DOCUMENT Synthesis of the findings of the evaluations of European Structural and Investment Funds Programmes 2014-2020 Accompanying the document REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS European Structural and Investment Funds 2014-2020  2020 Summary report of the programme annual implementation reports     covering implementation in 2014-2019 {COM(2021) 213 final}  
 
 
STAFF WORKING DOCUMENT ON THE 

SYNTHESIS OF THE FINDINGS OF THE EVALUATIONS OF 

EUROPEAN STRUCTURAL AND INVESTMENT FUNDS  

PROGRAMMES 2014-2020 

1. 

INTRODUCTION .............................................................................................................. 2 

2.  COHESION POLICY PROGRAMMES ........................................................................... 2 

2.1.  Evaluations completed by the Member States ............................................................. 3 

2.1.1.  General overview ................................................................................................. 3 

2.1.2.  Main findings of evaluations ................................................................................ 7 

2.1.3.  Support to Member States .................................................................................. 15 

2.2.  Evaluation work by the Commission ......................................................................... 16 

2.2.1.  ERDF/CF ............................................................................................................ 16 

2.2.2.  ESF/YEI ............................................................................................................. 18 

3.  EUROPEAN AGRICULTURAL FUND FOR RURAL DEVELOPMENT (EAFRD) .. 20 

3.1.  Evaluations completed by the Member States ........................................................... 20 

3.1.1.  General overview ............................................................................................... 20 

3.1.2.  Main findings of evaluations .............................................................................. 23 

3.1.3.  Support to Member States .................................................................................. 25 

3.2.  Evaluation work by the Commission ......................................................................... 25 

4.  EUROPEAN MARITIME AND FISHERIES FUND (EMFF) ....................................... 28 

4.1.  Evaluations completed by the Member States ........................................................... 28 

4.2.  Evaluation work by the Commission ......................................................................... 28 

5.  FINAL REMARKS .......................................................................................................... 29 

1 

 
 
 
 
 
 
 
1.  INTRODUCTION 

the  programme  annual 

This  staff  working  document  provides  an  overview  of  the  evaluations  carried  out  by  the 
Member States and the Commission on the programmes supported by the European Structural 
and  Investment  (ESI)  Funds1  and  implemented  under  shared  management  to  pursue  the 
objectives of the EU strategy for smart, sustainable and inclusive growth. It accompanies the 
2020  summary  report  of 
implementation  reports  covering 
implementation in 2014-2019.  
The Common Provisions Regulation for 2014-20202 requires that evaluations are carried out 
to  improve  the  quality  of  the  design  and  implementation  of  programmes,  as  well  as  to 
assess  their  effectiveness,  efficiency  and  impact.  Member  States  are  required  to  carry  out 
evaluations for each programme and ensure that each evaluation is duly followed up. At least 
once  during  the  programming  period,  an  evaluation  must  assess  how  support  from  the  ESI 
Funds  has  contributed  to  the  objectives  for  each  priority.  Evaluation  is  therefore  one  of  the 
main  tools  to  measure  how  the  respective  policies  have  performed.  Its  findings  add  to  the 
evidence  on  the  investments  supported  in  order  to  improve  the  policy  and  ensure  that  the 
actions supported deliver the best possible results. 

The  evaluations  carried  out  by  the  Member  States  may  assess  whether  targets  have  been 
achieved  (i.e.  monitoring/progress-oriented  evaluations),  how  funding  has  been  managed, 
how  projects  have  been  selected  and  run  (i.e.  procedure/implementation-oriented 
evaluations), or the outcome and effects of interventions (impact-oriented evaluations). Some 
of the above-mentioned parts could be combined in a single evaluation. 

Impact evaluations are carried out later on in the programming period, once the projects and 
measures implemented have produced enough results to be assessed. Their number has been 
growing  at  a  faster  rate  in  the  past  year.  However,  given  the  broad  thematic  and  territorial 
scope of the measures supported by the ESI Funds, Member State evaluations are designed to 
address  local  context  and  evaluation  needs.  As  a  result,  the  reported  findings  are  mostly 
specific  to  the  context  of  the  analysis.  Thus,  in  many  cases  the  findings  cannot  be 
aggregated at the EU level and do not allow for cross-cutting conclusions. 

This  document  synthesises  the  main  elements  of  the  findings  from  all  types  of  evaluations. 
This can lead to better implementation of the current programmes and provide useful insights 
for policy learning, also for helping prepare the programmes for 2021-2027.  

The following sections present an overview of the main findings from the evaluations of the 
ESI Funds programmes carried out by the Member States and the Commission. These focus 
mostly  on  the  past  year,  organised  by  policy  area.  Section  2  covers  cohesion  policy 
programmes,  Section  3  is  devoted  to  rural  development  policy  supported  by  the  European 
agricultural  fund  for  rural  development  (EAFRD),  and  Section  4  focuses  on  the  European 
maritime and fisheries fund (EMFF). Section 5 contains some concluding remarks.  

2.  COHESION POLICY PROGRAMMES 

Evaluation is key to measure how cohesion policy is performing. Cohesion Policy is  one of 
the most evaluated EU policies. This is thanks to the combined effort of the Member States 
and the Commission to monitor and analyse the results of the policy under the rules of shared 
management and thanks to the long-established importance of evidence for policy learning.  

https://cohesiondata.ec.europa.eu/ 

1 
2  Regulation (EU) No 1303/2013 of the European Parliament and of the Council of 17 December 2013 (OJ L 

347 20.12.2013, p. 320). 

2 

 
                                                 
This  section  provides  an  overview  of  the  evaluations  of  actions  funded  by  operational 
programmes supported by the European Regional Development Fund (ERDF), the Cohesion 
Fund (CF) and the European Social Fund (ESF) including the Youth Employment  Initiative 
(YEI). 

2.1.  Evaluations completed by the Member States 

2.1.1.  General overview 

The evaluation systems across the Member States differ in terms of maturity. Where there was 
no established and systematic use of evaluation to support national policies, cohesion policy 
has helped develop a culture of evaluation, starting from the planning and implementation 
of operational programmes. Participation in cohesion policy has helped evaluation propagate 
to national legislation and policies, increasing the efforts of the Member States to pursue an 
evidence-based approach to policy-making.  

To  help  the  Member  States  comply  with  the  regulatory  requirements  on  the  evaluation  of 
cohesion  policy, the Commission  has provided considerable  evaluation  guidance3 and many 
other  forms  of  support  to  Member  States  since  the  start  of  the  programming  period,  as 
detailed in Section 2.1.3. As a result, more than 1 400 evaluations carried out by Member 
States have been published since January 2015, of which over 1 000 referred to the 2014-
2020  programming  period.  This  represents  the  great  efforts  and  commitment  of  the 
Member  States  to  generate  evidence  for  policy  learning  in  line  with  the  stronger  result 
orientation  of  the  policy.  There  has  been  a  material  increase  in  the  evaluation  effort 
compared  to  the  2007-2013  programming  period,  when  around  800  evaluations  were 
produced  within  a  comparable  timeframe.  The  increase  in  the  number  of  completed 
evaluations has  generated valuable knowledge about  the implementation of the programmes 
and their effectiveness.  

Many  of  the  completed  evaluations  mostly  focus  on  finding  out  what  the  outcome  of  an 
intervention  was  rather  than  on  developing  an  understanding  of  how  and  why  it  was 
achieved.  For  a  fully-fledged  impact  evaluation,  this  is  a  crucial  element  to  investigate  in 
order to improve the way the intervention works and its effectiveness and to exploit the full 
potential of evaluation for policy learning. 

This  reflects  the  complexity  of  the  task  of  evaluating  the  impact  of  cohesion  policy, 
especially when trying to disentangle the effects of the measures funded from those of other 
things happening  at  the  same time. These include changes in  the socio-economic context  in 
which the measures are implemented or other policy measures in operation at the same time – 
not only in the area concerned, but also in other areas that can affect this. They also include 
spontaneous changes in behaviour, which by their nature may be hard to observe.  

Taking  into  account  all  the  relevant  factors  is  extremely  challenging,  especially  in  a  non-
experimental context where data on these factors and on how they interact with each other are 
imperfect and, in some cases, do not exist – not only because of a failure to set up the means 
to collect them, but also because a lot of information are intangible. 

It  is  therefore  important  to  have  realistic  expectations  of  evaluations  and  to  focus  on  the 
lessons learned for further improvement.  

Impact-oriented  evaluations  are  well  designed  in  general,  and  some  apply  advanced 
techniques, e.g. counterfactual methods, to distinguish the effects of the measures examined 
from  other  factors.  Case  studies  are  frequently  used  to  investigate  the  effects  of  specific 

3 

https://ec.europa.eu/regional_policy/en/policy/evaluations/guidance/  

3 

 
                                                 
instances  of  the  measures  supported  and  their  outcomes.  While  the  majority  of  evaluations 
make  serious  efforts  to  go  beyond  the  data  readily  available  (such  as  data  from  monitoring 
systems  or  official  statistics),  the  lack  of  adequate  data  and  particularly  access  to 
administrative data remains a common challenge faced by evaluators. 

While  impact  evaluations  are  very  important,  process  and  monitoring  evaluations  carried 
out  during  implementation  provide  evidence  that  can  be  used  immediately  in  programme 
management  in  order  to  steer  operations  in  the  right  direction.  They  can  also  be  used  to 
supplement the data available from local monitoring systems. 

Completed evaluations 

Cohesion policy evaluations completed and published since January 2015 relate either to the 
2014-2020 programming period or to the previous one (2007-2013), or in a small number of 
cases to both. While the majority of evaluations refer to 2014-2020, those covering 2007-2013 
programmes can still provide useful findings that are relevant for the managing authorities of 
the ongoing programmes. Overall, the number of evaluations has continued to increase in 
the past year at a faster rate than in previous years, up from less than 1 000 to over 1 400. 
The distribution of the evaluations across the funds has not changed in the past  year, with a 
slight  prevalence  of  ERDF/CF  over  ESF/YEI  and  around  20%  of  the  evaluations  covering 
both. 

Table  1:  Evaluations  published  since  January  2015  on  cohesion  policy  programmes  — 
breakdown by year, fund and programming period 

Funds 

ERDF+CF 

ESF+YEI 

Multi-fund 

Total 

2007-2013  Both periods 

2014-2020 

Total 

202 

82 

62 

346 

19 

3 

1 

23 

24 

12 

10 

46 

10 

3 

4 

17 

407 

439 

192 

633 

533 

264 

1 038 

1 430 

176 

173 

85 

434 

205 

179 

90 

474 

of which in the past year  ERDF+CF 

ESF+YEI 

Multi-fund 

Total in the past year 

Source: Commission evaluation helpdesk for cohesion policy 

The number of evaluations completed so far varies a lot across the Member States. However, 
the number of evaluations alone may not be representative of the evaluation effort made or of 
evidence  collected,  since  evaluations  vary  widely  in  terms  of  their  scale  and  scope  (e.g. 
number of operational programmes covered, priorities, themes, etc.). For example, more than 
20  evaluations  have  already  been  carried  out  in  Poland  on  the  Smart  Growth  operational 
programme,  many  of  them  very  narrowly  focused  on  support  for  particular  sectors.  This 
contrasts with the situation in other countries such as Portugal, where sectoral evaluations in 
many  cases  cover  all  support  across  multiple  programmes  and  over  both  programming 
periods. 

4 

 
 
 
Figure 1: Evaluations published since January 2015 on cohesion policy programmes  — 
breakdown by Member State 

Source: Commission evaluation helpdesk for cohesion policy  

Types of evaluations 

Around  one  third  of  the  evaluations  are  impact-oriented,  i.e.  they  aim  to  assess  the 
outcomes  and  results  of  the  programmes,  while  the  others  look  into  procedural  aspects  or 
progress  towards  achieving  the  targets  set.  The  overall  number  of  evaluations  aimed  at 
assessing 
implementation-oriented  and  progress-oriented 
evaluations. This is mainly because impact evaluations are normally carried out later in the 
programme cycle. These evaluations aim to analyse the effectiveness, efficiency and impact 
of the action supported and should be undertaken only once a critical number of projects have 
been completed and have produced results. 

is  smaller 

impact 

than 

Among  impact-oriented  evaluations,  only  one  third  assess  exclusively  the  impact  of  the 
interventions,  while  the  other  two  thirds  also  look  at  how  the  actions  were  implemented 
and/or monitor progress towards the targets.  

As  expected,  the  number  of  impact  evaluations  has  continued  to  rise.  In  the  past  year  there 
were  161  impact  evaluations,  the  highest  value  recorded  in  a  year  since  systematic 
collection started in 2015 and more than twice as many as in the previous year.  

For 2014-2020, the share of impact-oriented evaluations reached 20% and is expected to 
grow  further  when  more  results  emerge  following  implementation.  There  is  significant 
variation between the funds, with a higher share of impact-oriented evaluations for ESF/YEI 
(26%) than for ERDF/CF (18%) and multi-fund programmes (12%). This is mostly due to the 
different types of interventions supported by the different funds and their respective time for 
completion. 

5 

36016616414474726045433926252320191918161614141212119621050100150200250300350400PLITDEESFRCZUKHUROLTSKELSELVEENLATIESIBGDKCYHRPTFIBEMTLUPrevious yearsPast year 
 
Figure  2:  New  impact  evaluations  published  since  January  2015  on  cohesion  policy 
programmes by year — breakdown by programming period 

Source: Commission evaluation helpdesk for cohesion policy 

Most  evaluations  combine  relatively  simple  qualitative  methods  with  quantitative  ones.  In 
some cases, more sophisticated methods are used in order to assess the results and outcomes 
of  the  programmes.  Around  half  of  impact  evaluations  use  more  advanced  research 
methods,  such  as  counterfactual  evaluation  techniques,  theory-based  techniques  and  cost-
benefit analysis. Theory-based methods are more frequently applied in ERDF/CF evaluations, 
while counterfactual techniques are more often used for ESF/YEI and multi-fund evaluations. 

Thematic distribution 

The distribution of the evaluations is  spread unevenly across thematic objectives because 
of the different nature  of the actions funded and their financial weight. Figure 3 below 
shows  the  thematic  coverage  of  evaluations  for  the  2014-2020  programming  period  and 
impact-oriented  evaluations  referring  to  the  previous  period.  The  next  section  summarises 
the  main  findings.  More  than  half  of  the  evaluations  cover  multiple  thematic  objectives, 
which are reported under each of them. However, the chart does not include evaluations that 
broadly address a large number of thematic objectives. 

6 

 
 
Figure  3:  Thematic  coverage  of  evaluations  published  since  January  2015  on  cohesion 
policy  programmes  (all  2014-20  evaluations  and  impact-oriented  2007-13  evaluations) 

Source: Commission evaluation helpdesk for cohesion policy 

The  largest  number  of  evaluations  are  on  the  thematic  objectives  (TOs)  related  to 
inclusive  growth,  i.e.  employment  (TO8),  social  inclusion  (TO9)  and  education  (TO10), 
where  operations  supported  by  the  ESF  and  YEI  are  typically  implemented  faster  and 
generate results in a relatively short time.  

2.1.2.  Main findings of evaluations 

The  findings  summarised  below  are  from  the  evaluations  of  2014-2020  programmes  that 
assessed impact or procedural aspects, with the results likely to be more robust depending on 
the  information  on  the  data  used  and  the  methods  applied.  They  also  include  findings  from 
impact  evaluations  referring  to  the  2007-2013  period,  where  they  are  still  relevant  for  the 
current programmes. 

  Smart  growth: 

research  & 

innovation 

(R&I), 

information  and 

communication technology (ICT) and SME competitiveness 

For  the  2014-2020  programming  period,  the  ERDF  is  allocating  €135.8  billion  to  smart 
objectives  covering  research,  technological  development  and  innovation  (TO1),  ICT  (TO2) 
and  SME  competitiveness  (TO3).  Close  to  40%  of  the  allocation  had  already  been  paid  to 
project  beneficiaries  by  the  end  of  2019,  against  around  25%  in  2018.  This  rapid  increase, 
which  is  set  to  continue  in  2020,  is  reflected  in  the  change  in  the  number  of  evaluations 
focused on the impact of TO1 and TO3 investments over the past year. This shows that R&I 
projects and support to small and medium-sized enterprises (SMEs) are now producing 
noticeable  results  on  the  ground  in  many  EU  regions.  The  number  of  TO2  evaluations 
remains  low,  as  implementation  progress  is  slower  than  for  other  TOs  (27%  in  2019)  and 
funding is comparatively smaller. 

  Thematic Objective 1: Research & innovation 

In 2019, investments under TO1 made significant progress towards reaching their targets on 
the 
new  cooperation  between  enterprises  and  research 
commercialisation of new products by supported enterprises. Evaluations provide detailed 
insights into the performance of programmes and projects in these domains. In Tuscany (IT), 

institutions  and  on 

7 

TO1; 256TO2; 106TO3; 209TO4; 163TO5; 45TO6; 142TO7; 91TO8; 390TO9; 292TO10; 271TO11; 108 
 
the success rate of projects is high, with around 90% of them resulting in the production of 
prototypes  ready  to  be  sold.  Evaluations  also  show  that  projects  that  aim  to  step  up 
cooperation between enterprises and research institutions contribute to overarching objectives. 
In  Friuli-Venezia  Giulia  (IT)  for  example,  EU-funded  projects  have  helped  increase  R&D 
expenditure relative to GDP. TO1 investments also help improve research infrastructure, as 
in Finland, and to expand research areas in supported research centres, as is the case in Italy.  

In  some  Member  States,  the  manufacturing  sector  and  knowledge-intensive  services 
predominate in the areas where TO1 investments are undertaken, with the effect of directing 
substantial support towards large firms (especially in Poland and Italy). However, while EU 
support  in  Italy  contributed  to  a  broader  scheme  also  funded  from  the  national  budget,  in 
Poland  this  led  to  some  deadweight  costs,  which  is  also  the  case  when  SMEs  already 
undertaking research are given funding. 

  Thematic Objective 2: Information and Communication Technology 

Investments in ICT cover various intervention fields. Although the overall implementation of 
TO2 objectives is lagging behind other thematic objectives, significant progress has already 
been made in the specific field of broadband coverage, with implementation reaching 42% 
of the target by the end of 2019. Evaluations covering both 2007-2013 and 2014-2020 analyse 
in  detail  the  tangible  outcome  of  this  intervention  field.  In  Poland,  broadband  coverage  has 
clearly helped young people and households in difficult socio-economic situations to acquire 
digital skills. In Spain however, large differences remain between the main urban centres and 
rural  areas,  despite  a  considerable  increase  in  broadband  coverage.  This  demonstrates  that 
broadband  coverage  alone  is  not  always  sufficient  to  close  the  digital  gap.  Evaluations  in 
Poland and Sweden have shown that the prior level of digital skills on the ground is a major 
factor in the success or failure of certain ICT-related projects.  

Evaluations of other TO2 investment types show that funded projects have increased the use 
and quality of e-services in the public sector and helped develop  new  ones.  In  Italy, new 
digital  services  addressed  at  enterprises  have  also  helped  reduce  administrative  costs.  In 
Czechia, the introduction of ICT in enterprises has improved workers’ skills and has made it 
easier to access new markets and partners. 

  Thematic Objective 3: SME competitiveness 

Implementation  reports  show  a  rapid  increase  in  the  number  of  supported  companies,  from 
40%  of  the  planned  achievement  in  2018  to  55%  by  the  end  of  2019.  Evaluations  have 
confirmed that most of them either have grown in terms of  employment, productivity and 
market  share  or  consolidated  their  level  of  activity,  also  thanks  to  the  use  of  financial 
instruments. Supported SMEs have boosted their internationalisation and increased their range 
of  services  and  products.  For  instance,  business  incubators  in  Romania  have  helped  firms 
strengthen their position on the market, diversify products and improve their quality.  

Business creation interventions in France have also led to a marginally higher survival rate of 
the firms supported, despite implementation delays.  

In Poland, support under the Smart Growth operational programme seems to have benefited 
micro-enterprises  more  than  other  SMEs  and  helped  promote  innovation,  although  the 
allocation  of  resources  could  be  improved  with  different  selection  criteria  for  the  projects. 
Under  regional  operational  programmes,  EU  support  for  Polish  SMEs  led  to  employment 
growth 4 percentage points higher than that of unsuccessful applicants.  

Nevertheless, the length of the assessment procedure, together with the administrative burden, 
are reported to be common obstacles to a broader take-up of support. 

8 

 
  Sustainable  growth:  low-carbon  economy,  climate  change,  environment, 

transport and energy networks 

The  ERDF  and  the  Cohesion  Fund  are  allocating  €171.2  billion  to  sustainable  growth, 
covering investments in the low-carbon economy (TO4), climate change adaptation and risk 
prevention  (TO5),  environment  protection  and  resources  efficiency  (TO6)  and  in  network 
infrastructures in transport and energy (TO7). By the end of 2019, spending amounted to over 
€57  billion  (about  33%  of  the  planned  total,  against  22%  in  2018).  By  the  end  of  2019, 
implementation  progress  was  particularly  advanced  for  investments  under  TO7  (network 
infrastructure), while it remained below 30% for the other TOs. Nevertheless, in many cases, 
the results of investments do not appear immediately after their implementation, leading to a 
limited number of impact evaluations at this stage. 

  Thematic Objective 4: Low-carbon economy 

In  2014-2020,  25%  of  the  ESIF  must  be  spent  on  projects  with  climate  action  objectives 
(27% for the Cohesion Fund and 17% for the ERDF). With €24.1 billion allocated to climate 
action, investments under TO4 contribute the most to these objectives. Funded projects tackle 
energy supply or energy efficiency. Evaluations review the different types of action that help 
realise those objectives on the ground and show the difficulties and opportunities that each of 
them entail. Projects aimed at reducing the CO2 emissions of companies, besides indisputable 
environmental  benefits,  have  led  to  long-term  cost  savings  and  increased  energy 
efficiency. However, the overall impact has been limited because of the small proportion of 
firms  supported,  especially  in  Italy  and  Germany.  Likewise,  energy  efficiency  measures  for 
public buildings generally proved to be successful in Germany, Czechia and France. The large 
scale  of  the  projects,  which  makes  it  easier  to  manage  administrative  costs,  partly  explains 
these positive results.  

In terms of implementation issues, one evaluation found that financial instruments under the 
Polish infrastructure and environment operational programme have been an appropriate form 
of support for increasing both the renewable energy supply and energy efficiency. However, 
other  evaluations  in  Germany,  France  and  Czechia  found  that  complex  and  evolving 
regulations, 
in  project 
implementation.  For  example,  in  Czechia  the  inability  to  combine  the  ERDF  with  other 
support  for  renewable  production  limited  the  number  of  projects  implemented.  In  Austria, 
access  to  credit  remains  an  obstacle  for  small  firms  investing  in  projects  to  reduce  CO2 
emissions. 

in  particular  for  housing  renovations, 

to  difficulties 

lead 

  Thematic Objective 5: Climate change adaptation and risk prevention 

Comparatively  smaller  amounts  allocated  to  this  theme,  together  with  a  relatively  slow 
implementation  rate,  limited  the  number  of  TO5  impact  evaluations  focused  on  2014-2020 
(the  lowest  number  of  all  thematic  objectives).  However,  investments  in  flood  protection 
measures were already showing tangible results by the end of 2019, with 7.5 million people 
seeing  their  exposure  to  flood  risks  decrease  thanks  to  ERDF/CF  investments  (representing 
45% of the EU target). 

Evaluations  show  that  flood  protection  measures  included  in  particular  strengthened 
cooperation between services to increase preparedness and ensure sufficient capability for 
when  disasters  strike,  particularly  in  cross-border  areas.  Continued  collaboration  between 
Member  States  has  led  to  increased  capacity  to  plan  and  implement  measures  on  climate 
change  adaptation.  Likewise,  evaluations  in  Italy  and  Hungary  showed  that  EU  support 
focused  not  only  on  building  infrastructure,  but  also  helped  to  improve  monitoring  and 

9 

 
control,  notably  through  the  implementation  of  ICT  tools  (particularly  in  the  2007-2013 
programming period). 

  Thematic Objective 6: Environment protection and resource efficiency 

Rapid  progress  between  2018 and 2019 in  the implementation of key indicators such as the 
number  of  people  benefiting  from  an  improved  water  supply  or  habitat  areas  supported  to 
achieve a better conservation status indicates that TO6 investments are now being completed 
on the ground in many EU regions.  

In many evaluations, investments that aim to improve wastewater treatment were found to be 
successful in  preserving natural  habitats, promoting  biodiversity and  creating jobs. This 
demonstrates the virtuous circle that investments in resource efficiency measures can trigger. 
EU-funded  interventions  related  to  cultural  facilities  and  natural  heritage  have  also  helped 
increase visits to the sites concerned in Poland, France and Germany, even though substantial 
funds were directed in some cases to areas, where the tourism potential was already high.  

Evaluations in Austria and France highlighted that the nature of TO6 investments require the 
involvement  of  many  organisations  and  authorities,  which  has  helped  increase  cooperation 
between  them.  Nevertheless,  delays  in  programme  implementation  have  been  reported  by 
evaluations  in  different  Member  States.  These  are  often  linked  to  the  governance  of  the 
measures concerned. 

  Thematic Objective 7: Network infrastructure in transport and energy  

TO7  is  the  thematic  objective  that  benefits  from  the  highest  share  of  ERDF/CF  support. 
Evaluations  have  drawn  lessons  from  the  2007-2013  period  and  identified  preliminary 
findings  for  the  2014-2020  period.  They  go  beyond  the  amounts  invested  in  the  Trans-
European  Transport  Network  (TEN-T)  or  in  urban  or  maritime  transport  by  showing  the 
positive side effects and the challenges of TO7 interventions. 

TEN-T investments often rely heavily on ERDF and CF funding, as is the case in Romania, 
demonstrating the EU added value in major transport projects. Investment in the rail network 
has helped reduce travel time and increase the number of passengers, sometimes getting 
them off the road like in Spain, despite declining ERDF support for transport over the years. 
Similarly,  TEN-T  investments  in  the  motorway  network  in  Poland  have  led  to  the 
development  of  an  integrated  transport  system,  increased  accessibility  and  reduced  air 
pollution and road noise. It has also improved road safety, with at least a 50% reduction in 
road accidents on the sections targeted by the investment. From an economic point of view, 
estimates  show  that  it  only  takes  3  to  5  years  to  recoup  investment  costs  in  the  shape  of 
benefits for businesses. However, core road and rail links often fail to cover rural areas and 
secondary cities, as in Czechia. It is worth noting that robust findings on TEN-T investments 
during  the  2014-2020  period  remain  limited  so  far,  as  the  concrete  results  of  these  projects 
often take years to emerge. 

Investments  in  urban  areas  in  Italy,  Poland  and  Romania  have  made  connected  territories 
more  attractive,  which  has  benefited  the  businesses  located  there.  In  Italy,  businesses’ 
financial gains from new transport connections were found to compensate for the losses they 
had suffered during construction. 

  Inclusive growth: employment, social inclusion and education 

Cohesion policy funds, together with the Youth Employment  Initiative (YEI), plan to invest 
over  €152.5  billion  in  intervention  fields  related  to  inclusive  growth,  with  the  bulk  of 
funding coming from the ESF. Inclusive growth investments target employment (TO8), social 
inclusion (TO9) and education (TO10). By the end of 2019, overall Cohesion Policy spending 

10 

 
was close to €62.1 billion and reached about 41% of the planned total (around 29% in 2018). 
This rapid increase continued in 2020. Moreover, preliminary results of investments related to 
inclusive  growth,  and  in  particular  investments  in  quality  employment,  can  be  observed  as 
early as a few months after implementation, making it easier to assess their effects and impact 
earlier in the programming period.  

  Thematic Objective 8: Sustainable and quality employment 

By  the  end  of  2019,  4.5  million  people  had  found  a  job  after  receiving  ESF  and  YEI 
support.  

A  large  number  of  evaluations  analyse  the  implementation  and  impact  of  investments  that 
support sustainable and quality employment in order to provide a detailed picture of what this 
number means in practice.  

In  Italy,  Portugal  and  the  United  Kingdom,  evaluation  findings  have  shown  a  strong 
correlation  between  traineeships  and  increased  employability.  Project  participation, 
especially  in  cases  where  long-term  and  personalised  support  is  offered,  as  in  Germany, 
correlates with increased personal and job-related skills. There is also evidence of EU added 
value  as  the  ESF  contributes  to  the  sustainability  of  services,  which  would  be  financially 
unstable otherwise. 

However,  there  is  also  evidence  from  evaluations  in  Italy  and  Lithuania  that  employment 
found  after  participating  in  ESF-funded  projects  is  not  always  stable  or  remunerated  at  the 
average  national  level.  In  some  regions,  differences  between  the  employment  levels  of 
participants  and  non-participants  seem  to  dissipate  over  time,  which  indicates  that  positive 
outcomes may have a limited duration. In a small number of cases, interventions suffered 
from  communication  gaps,  reaching  out  to  participants  and  keeping  them  engaged  can  be  a 
challenge,  as  can  be  the  creation  of  effective  networks  for  all  organisations  involved  in  a 
project.  

  Thematic Objective 9: Social inclusion 

By  the  end  of  2019,  2.5  million  participants  with  disabilities,  5.6  million  migrants, 
participants  with  a  foreign  background  or  minorities  and  6.5  million  other  disadvantaged 
people  had  received  help  from  TO9  interventions  to 
improve  their  employment 
opportunities and develop the right skills for the jobs  market.  In addition, ERDF support 
allowed 39 million people to benefit from improved health services across the EU. 

The primary focus of TO9 evaluations for 2014-2020 is on labour market integration and the 
social inclusion of disadvantaged people. 

Notable  positive  results  are  from  projects  in  which  there  is  an  emphasis  on  regular  contact 
between coach and participant. In Germany and the United Kingdom, evaluations demonstrate 
that assessments of individual  needs  tend to produce better  results both in  terms  of soft 
skills developed and hard outcomes related to employment. In Germany, the need to organise 
and  coordinate  TO9  interventions  has  been  a  reason  for  forming  collaborations  between 
organisations  and  services,  which  has  led  to  the  creation  of  networks  of  organisations 
previously  not  in  contact.  Evaluation  findings  also  highlight  increased  engagement  –  the 
learning  opportunities  has  encouraged  first-time  social 
availability  of  funding  and 
entrepreneurs to create ventures and has made some people more proactive in terms of local 
policy-making  and  volunteering.  In  France,  the  complementarity  of  TO9  interventions 
funded  by  the  ESF  with  actions  funded  by  the  Asylum,  Migration  and  Integration  Fund 
(AMIF)  is  ensured  via  regular  and  structured  exchange  of  information  between  services. 
AMIF  support  helped  orientate  migrants  towards  services  funded  by  the  ESF  for  their 
integration. 

11 

 
On the other hand, many actions continue to focus on social inclusion through employment; 
while  this  is  in  itself  helpful,  it  does  not  always  address  other  social  barriers  faced  by 
participants.  In  addition,  a  German  evaluation  showed  that  some  measures  aimed  to  higher 
target  achievement  rates  excluded  those  who  were  hardest  to  reach  and  help,  leading  to 
‘creaming’  effects  (i.e.  selection  of  participants  with  higher  potential  rather  than  those  in 
higher need).  

  Thematic Objective 10: Education and vocational training 

By the end of 2019, 17.8 million  low-skilled  people were helped, 5.5 million had gained a 
qualification and 1.8 million were in education and training. In addition, ERDF investments in 
education  infrastructure  had  allowed  more  than  15  million  schoolchildren  and  students  to 
benefit from an improved studying environment by the end of 2019. 

TO10  evaluations  in  the  current  period  mainly  focus  on  early  school  leaving,  general 
education,  continuing  vocational  training,  and  the  employment  results  following  training. 
Evaluations of TO10 interventions in Germany, Poland and the United Kingdom have found 
that  practical  forms  of  education  such  as  traineeships  and  job  counselling  have  a  clear 
positive impact on the employment prospects of participants. There were further positive 
results in  Austria, where promoting communication networks that include students,  teachers 
and  families  can  reduce  education  dropout  rates.  Language  classes  and  social  counselling 
offered to migrants can improve the rates at which they integrate in their local community, as 
in Germany. Evaluations have shown that the courses provided are generally in line with the 
needs of the local labour markets. However, some groups have lower access to education and 
training  interventions,  such  as  learners  with  special  needs  in  Estonia  and  rural  dwellers  in 
Lithuania.  Service  provision  is  also  more  challenging  in  regions  with  very  heterogeneous 
participant  groups,  as  highlighted  in  one  evaluation  in  Germany.  Moreover,  some 
interventions  in  Hungary  were  perceived  as  being  brief  and  not  providing  sufficient 
knowledge to make a lasting impact on participant skillsets.  

  Strengthening institutional capacity and efficient public administration 

  Thematic Objective 11: Efficient public administration 

With  €6.2  billion  allocated  under  the  ESF  and  the  ERDF,  interventions  in  efficient  public 
administration  benefit  from  less  funding  than  other  TOs.  By  the  end  of  2019,  over  2 300 
projects targeting public administrations or public services at national, regional or local levels 
had  received  support  from  the  ESF.  TO11  evaluations  underline  the  –  often-intangible  – 
results  that  such  actions  bring  up.  They  show  that  TO11  interventions  have  increased 
awareness among public authorities on good governance principles, such as the exchange of 
information,  transparency,  coherence  of  interventions  and  prevention  of  conflicts  of  interest 
during  project  selection.  In  Latvia,  investment  in  public  administration  has  helped  public 
authorities better understand how to promote entrepreneurship and the sustainability of SMEs, 
and the training supported has strengthened inter-institutional cooperation in investigating 
complex corruption, economic and financial crime cases.  

In  Slovenia,  TO11  investments  have  helped  strengthen  non-governmental  organisation 
(NGO)  networks  and  connect  them  with  public  authorities.  However,  results  have  been 
affected by the limited capacity of NGOs to absorb funding and a lack of coherence between 
projects and NGO needs. 

An assessment of the evaluation function of cohesion policy in central and eastern European 
countries shows that evaluation results are not made sufficiently accessible and that the effects 

12 

 
of  the  measures  evaluated  should  be  disseminated  better.  In  addition,  evaluation  units  are 
often constrained by limited resources. 

  Cross-cutting instruments 

  Territorial instruments: ITIs, CLLD and sustainable urban development 

In  the  current  period,  Member  States  have  produced  42  evaluations  covering  integrated 
territorial development approaches: 18 focused on integrated territorial investments (ITIs), 6 
on  community-led  local  development  (CLLD),  3  on  both  ITIs  and  CLLD,  and  15  on 
sustainable urban development (SUD) carried out under a specific operational programme or 
priority  axis4.  The  evaluations  have  mainly  examined  the  governance,  procedures  and 
implementation  aspects  of  the  instruments.  They  are  based  on  a  mix  of  quantitative  and 
qualitative methods and most of them include case studies.  

Evaluations have highlighted positive findings. In general,  ITIs seem in  line with the needs 
identified at local level  – they favour the development of bottom-up approaches, facilitating 
the exchange of information, knowledge and experience. In Poland, ITIs have benefited small 
municipalities  partnering  with  larger  ones,  helping  them  overcome  their  financial  and 
organisational  limitations.  Cooperation  between  different  regional  and  local  interest  groups 
and public participation in local development strategies was a key success factor of CLLD in 
Austria, Poland and Romania; in the latter, they could be even more effective if concentrated 
in  specific  neighbourhoods  and  communities  concerned.  In  Sweden,  SUD  strategies  have 
focused  on  job  creation  and  innovation,  with  collaboration  between  various  stakeholders  in 
the projects a key factor for success. 

On  the  other  hand,  managing  and  monitoring  the  contributions  of  different  funds  and 
programmes has proved to be difficult and burdensome in terms of administrative capacity in 
Czechia and the Netherlands. Too many organisations were involved at the different levels in 
Italy  and  Hungary,  with  unclear  allocation  of  tasks  at  local,  regional  and  national  level. 
Estonia  experienced  difficulties  in  integrating  projects  and  different  types  of  measures  in  a 
specific territory. Evaluations have also identified a recurring need for timely preparation and 
adoption  of  strategies;  in  addition,  they  have  emphasised  that  the  authorities  implementing 
projects under the SUD strategies need to have staff with administrative, legal and managerial 
skills adapted to the types of actions undertaken. 

  Financial instruments  

Several evaluations have addressed the use of financial instruments. They show that these can 
be  used  to  help  companies  grow  in  the  early  phases  of  their  development.  In  Germany, 
financial instruments helped strengthen business competitiveness and increase production and 
profits. Similarly, under the Polish Smart Growth operational programme, a vast majority of 
beneficiaries  considered  that  such  support  was  a  significant  factor  in  encouraging  them  to 
innovate. However, for this operational programme, it was also observed that restrictions on 
eligible expenditure limited the applications for financing. 

While  the  diversity  of  financial  instruments  available  should  constitute  an  opportunity, 
evaluations found that in some cases it led to increased complexity that affected the take-up 
of the instrument. This was the case in Czechia, where the use of financial instruments to fund 

4  According  to  Article  7  of  Regulation  (EU)  No  1301/2013,  the  ERDF  should  support  sustainable  urban 
development through strategies that encompass integrated measures. Such support can be provided through 
ITIs,  a  specific  operational  programme,  or  a  specific  priority  axis  of  an  operational  programme.  SUD  is 
therefore also covered in some evaluations focusing on ITIs. 

13 

 
                                                 
cohesion policy expenditure is fragmented due to the high number of instruments available at 
EU, national and regional level. In addition, there are sometimes overlaps with other types of 
funding, such as grants. 

  European territorial cooperation  

The  vast  majority  of  the  evaluations  carried  out  under  the  European  territorial  cooperation 
(ETC)  goal cover multiple thematic objectives,  although they may not  necessarily  represent 
the themes in which Interreg programmes invest most.  

Positive outcomes are reported in the area of cooperation where stakeholders, especially 
those  pertaining  to  the  category  of  public  authorities,  are  expected  to  have  a  long-lasting 
collaboration even after the project has ended. This is also confirmed by the findings of some 
evaluations,  which  point  to  the  fact  that  partnerships  formed  during  the  2007-2013 
programming  period  also  applied 
in 
environmental projects tend to become permanent networks, as observed for example in 
the Interreg V-A Sweden-Denmark-Norway programme. There is also  evidence that the use 
of the small projects fund to boost social integration across borders under the Interreg V-A 
Poland-Germany/Saxony  programme  strengthened  cross-border  local  communities  and 
cooperation  among  partners.  On  the  other  hand,  some  evaluations  report  that  there  is  still 
some  room  for  improving  cooperation  between  the  bodies  involved  in  implementing 
cooperation programmes (managing authorities and joint secretariats). 

in  2014-2020.  Partnerships 

financing 

for 

In  general,  ETC  programmes  help  create  a  specific  framework  that  makes  collaboration 
easier between stakeholders across countries, which would otherwise be more difficult to 
attain. The most frequent topics where cooperation has improved are on environmental issues, 
management of maritime areas and management of natural and cultural heritage.  

The exchange of knowledge and good practices (via training activities, events, networking 
platforms etc.) remains an important and successful activity in cooperation programmes.  

While  the  evaluations  point  the  fact  that  ETC  programmes  helped  reduce  cultural 
barriers,  the  impact  of  the  programmes  so  far  in  terms  of  language  barriers  is  less  visible, 
especially for programmes where the languages of participating countries belong to different 
etymological  backgrounds.  Another  finding  put  forward  by  the  evaluations  relates  to  the 
simplification  measures  implemented  in  the  2014-2020  programming  period,  the  most 
notable  ones  being  those  on  the  electronic  exchange  of  data  for  project  application  and 
monitoring. While there has been a positive impact in several programmes, the conclusion of 
the evaluations indicate  that there are still improvements to  be made,  especially in  terms of 
user-friendliness and interoperability of the systems in place. 

There  were  also  positive  outcomes  for  the  tourism  sector,  especially  in  the  maritime  areas 
covered by the cooperation programmes, as reported for example by the Interreg V-A Italy-
France programme. 

Positive outcomes were also reported in the area of research and innovation. By the end of 
2019,  17 500  enterprises  and  3 600  research  institutions  had  participated  in  cross-border, 
interregional  or  transnational  research  projects.  For  example,  the  impact  evaluation  of  the 
Northern  Periphery  and  Arctic  Programme  shows  that  projects  have  achieved  closer 
cooperation  between  SMEs  and  R&D  centres,  increased  skills  in  SMEs  and  improved  the 
capacity for digital service delivery. 

14 

 
 
 
  Horizontal findings on implementation 

Besides assessing the effects of the actions supported, reported above by thematic objectives 
and  cross-cutting  instruments,  the  largest  share  of  evaluations  aim  to  assess  progress  and 
implementation.  These  evaluations  provide  valuable  evidence  on  the  way  that  programmes 
are being carried out, their consistency with the initial objectives and relevance in respect of 
development or social needs, as well as issues which they are encountering. They ultimately 
help  managing  authorities  identify  and  correct  the  weaknesses  that  might  emerge  during 
implementation.  

Implementation-oriented  evaluation  carried  out  in  the  Member  States  were  found  to  have 
clear  and  simple  objectives.  Their  underlying  evaluation  questions  are  relatively 
straightforward,  as  is  the  evaluation  design,  and  they  generally  rely  on  multiple  sources 
(surveys,  monitoring  data)  to  ensure  that  findings  are  robust.  They  mostly  confirm  the 
strategic nature of the projects selected and progress in line with targets, considering that 
they were undertaken before the pandemic.  

However, some difficulties emerged from evaluations that investigated the processes behind 
programme  implementation.  Several  evaluations  reveal  significant  delays  in  programme 
implementation. This  is  due to  a number of different  factors, including  uneven capacity  or 
limited  stability  within  regional  and  local  authorities  in  Estonia  and  Italy,  long  tendering 
processes  in  Czechia  and  Slovakia  and  difficult  cooperation  among  major  stakeholders  in 
Bulgaria.  Delays  are  also  related  to  the  implementation  of  complex  projects,  especially 
infrastructure ones, that caused unexpected difficulties. 

Other difficulties observed include inadequacy of funding and lack of cooperation between 
the  bodies  involved  in  implementation.  Overlap  with  national  policies  could  produce 
displacement effects and limit the take-up because of more favourable alternative measures.  

According to many evaluations, administrative issues can limit potential beneficiaries from 
participating  in  the  actions  funded  by  the  programmes  as  well  as  outcomes.  The  most 
common  issues  are  overly  complicated  and  lengthy  application  procedures,  changes  in 
management  procedures,  difficulties  in  understanding  and  applying  the  simplified  cost 
method, and lack of adequate information and sufficient financial incentives. In some cases, 
the demanding nature of innovative projects deterred potential applicants.  

A  number  of  evaluations  reported  issues  with  project  selection  procedures  –  sometimes 
unclear and too complex for applicants, or too long to complete in the case of some support 
schemes for enterprises. In a limited number of cases, the selection criteria were deemed as 
not suitable for selecting the best projects. 

In a few cases, weaknesses were identified in  monitoring and information systems, which 
are  a  primary  source  of  data  for  evaluations  as  well  as  for  tracking  progress  in 
implementation.  These  range  from  the  timeliness  of  data  collection  to  incompleteness  or 
inadequacy of the data collected. 

2.1.3.  Support to Member States 

The  Commission  is  committed  to  raising  awareness  of  the  importance  of  evidence-based 
policy implementation  and design within Member States. This  will help them improve their 
knowledge and administrative capacity in the area of cohesion policy evaluation. It manages 
the  evaluation  network  and  the  evaluation  partnership,  which  are  platforms  to  discuss 
cohesion policy evaluation with the Member States on ERDF/CF and ESF/YEI respectively. 
Topics include methodological guidance, data and indicators, progress on ongoing evaluations 
and evaluation results and their use. On these topics, Member States exchange experience and 
good practice to strengthen evaluation capacity throughout the EU. 

15 

 
The  Commission’s  cohesion  policy  evaluation  helpdesk  is  a  dedicated  service  that  helps 
Member States with evaluation issues. The evaluations carried out in the Member States since 
2015  on  programmes  supported  by  ERDF/CF  and  ESF/YEI  programmes  are  systematically 
collected, analysed and summarised. Summaries of the main findings of the Member States’ 
evaluations,  including  a  description  of  the  data  and  methodologies  used,  as  well  as  the  full 
evaluation studies, are available in the evaluation library published on the Inforegio website5. 
The  ESI  Funds  open  data  platform6  also  publishes  structured  data  on  all  the  evaluations, 
together with their summaries. 

The evaluation helpdesk has also strengthened its support to the Member States on specific 
methodological  issues,  following  periodic  calls  for  interest  launched  by  the  Commission. 
Expert  reviews  have  provided  valuable  feedback  on  the  evaluations  carried  out  by  national 
and  regional  authorities.  They  can  involve  assessing  full  evaluation  dossiers  (terms  of 
reference, consultation strategies, data collection and use, interim reports, final reports, etc.) 
or  be  limited  to  one  or  more  parts  of  dossiers.  Evaluations  do  not  need  to  be  completed  in 
order to be reviewed. This expert review scheme allows the Commission to provide for timely 
and tailor-made methodological support to the Member States.  

In  addition,  the  Commission  organises  summer  schools  for  managing  authorities  and  other 
public bodies involved in the evaluation of ERDF/CF programmes, with practical -training on 
topics selected by the Member States ahead of the training. The latest edition was held online 
and used concrete examples of evaluations provided by participants on research, technological 
development and innovation (TO1), support to SMEs (TO3) and low-carbon economy (TO4).  

As  evaluations  depend  on  good  monitoring  systems,  the  Commission  established  the  ESF 
Data  Support  Centre  in  March  2020.  This  centre  provides  a  forum  for  expert  discussions 
and mutual learning between managing authorities on monitoring issues. It is also a point of 
contact  for  Member  States  to  submit  methodological  queries  related  to  monitoring  and 
reporting  requirements  for  ESF+  programmes  for  the  2021-2027  programming  period.  A 
searchable  database  with  methodological  questions  and  their  corresponding  answers  is 
available in all EU official languages, under the Shared Fund Management Common System 
(SFC). This database is updated with new questions on an ongoing basis in order to facilitate 
programme negotiations.  

2.2.  Evaluation work by the Commission 

2.2.1.  ERDF/CF 

In February 2020, the Commission published two staff working documents presenting the ex 
post  evaluation  of  major  projects  supported  by  the  ERDF  and  the  CF  between  2000  and 
20137.  The  first  document  focused  on  transport8  and  the  second  on  environmental 
infrastructure  projects9.  The  two  evaluations  confirmed  the  importance  of  EU  support  for 
these  projects  in  achieving  EU  objectives.  Project  quality  at  entry,  project  governance  and 
project  management  proved  to  be  the  main  determinants  of  success,  while  forecasting 
capacity  emerged  as  a  problematic  determinant  of  performance.  By  supporting  large-scale 
transport  projects,  cohesion  policy  has  made  a  significant  contribution  to  improving 

https://ec.europa.eu/regional_policy/en/policy/evaluations/member-states/ 
https://cohesiondata.ec.europa.eu/dataset/OP-Evaluations/iz3t-u7bv/data  

5 
6 
7  https://ec.europa.eu/regional_policy/sources/docgener/evaluation/pdf/ex_post_env_2000_2013_exec_en.pdf  
8 

https://ec.europa.eu/regional_policy/sources/docgener/evaluation/pdf/swd_ex_post_transport_2000_2013_e
n.pdf  
https://ec.europa.eu/regional_policy/sources/docgener/evaluation/pdf/swd_ex_post_env_2000_2013_en.pdf  

9 

16 

 
                                                 
connectivity  at  EU  and  national  level  and  has  encouraged  the  development  of  sustainable 
transport. Investments have improved accessibility in the countries and regions concerned and 
paved  the  way  for  increased  trade  with  the  rest  of  the  EU,  which  is  vital  for  economic 
development.  Large-scale  environmental 
infrastructure  projects  have  made  a  major 
contribution to compliance with EU law and broader EU sustainable development strategies. 
The  staff  working  documents  also  highlighted  the  EU  added  value  of  these  projects  –  the 
availability of a significant proportion of EU funding was critical for enabling or accelerating 
implementation, and therefore achieving the objectives.  

The  Commission  also  made  significant  progress  on  its  ex  post  evaluation  of  investment 
projects  in  research  and  technological  development  (RTD)  infrastructures,  competence 
centres and activities financed by the ERDF in 2007-2013, where €17 billion was invested 
during  that  period.  The  evaluation  analyses  the  factors  that  contributed  to  the  success  or 
failure  of  the  actions  and  investigates  the  mechanisms  of  change  under  different  socio-
economic  conditions.  Its  methodology  is  based  on  the  use  of  detailed  data  on  projects  and 
beneficiaries, which helps provide a better view of the impact of the funds and will also serve 
as a model for the 2014-2020 ex post evaluation. Project- and beneficiary-level data collected 
during the evaluation are published on the ESI Funds open data platform. The aim is to reach 
the wider public10 and for the data be freely available for further analysis and research. The 
final report of the evaluation is expected in the second half of 2021. 

In 2020, the Commission launched an evaluation of e-Cohesion, the electronic data exchange 
system  to  make  it  easier  to  exchange  documents  and  data  between  ESI  Funds  beneficiaries 
and  the  relevant  authorities,  in  place  for  2014-2020.  To  allow  the  managing  authorities  to 
incorporate  any  changes  into  their  systems,  the  Commission  aims  to  deliver  the  findings  of 
this evaluation at an early stage of implementation of the 2021-2027 operational programmes.  

Under Article 57 of the Common Provision Regulation, the Commission will carry out an ex 
post  evaluation  of  the  ESI  Funds,  to  be  completed  by  the  end  of  2024.  At  this  stage,  it  is 
currently considering how best to shape the exercise. It has launched a preparatory study that 
will explore this option. This study will also identify potential shortcomings in the collection 
of data, as data quality issues can have an impact on the cost of the evaluation.  
The Commission is  also preparing the  9th evaluation conference on cohesion policy under 
the auspices  of the Portuguese Presidency of the EU Council. The conference will focus on 
the  evaluations  carried  out  by  the  Member  States  and  the  Commission,  and  will  provide  a 
platform to discuss the methodologies used to assess the role of the policy under the different 
thematic objectives.  

2.2.2.  ESF/YEI 

Four thematic evaluations on ESF support were initially planned as studies. They were then 
upgraded to evaluations to pave the way for the comprehensive ex post evaluation and to feed 
the upcoming programme negotiations: 

- 
- 

- 

evaluation of ESF/YEI support to youth employment11;  
evaluation  of  ESF  support  to  employment  and  labour  mobility  (TO8,  excluding 
support to youth employment)12;  
evaluation of ESF support to social inclusion (TO9)13;  

10  https://cohesiondata.ec.europa.eu/d/kkj2-8bik  
11  https://ec.europa.eu/social/main.jsp?catId=738&langId=en&pubId=8307&furtherPubs=yes 
12  https://ec.europa.eu/social/main.jsp?catId=738&langId=en&pubId=8330&furtherPubs=yes 
13  https://ec.europa.eu/social/main.jsp?catId=738&langId=en&pubId=8346&furtherPubs=yes 
https://ec.europa.eu/social/main.jsp?catId=738&langId=en&pubId=8350&furtherPubs=yes 

17 

 
                                                 
 
- 

evaluation of ESF support to education and training (TO10)14.  

They  cover  the  implementation  of  ESF  interventions  from  2014  to  2018  and  are  now 
completed and published. The findings of the youth employment evaluation support the new 
Youth Employment Package15, in particular the need to reach out to inactive/vulnerable/low-
skilled groups and to extend the guarantee to those up to 29 years of age. It also confirms that 
work experience or vocational training is key to increasing employability. As regards design, 
an  integrated  pathway  still  seems  to  be  the  best  approach  to  youth  employment  and 
individuals furthest away from the labour market who benefit greatly from tailored guidance 
and  support.  It  has  also  been  found  that  a  too  narrow  definition  of  target  groups  may  pose 
problems when adapting to specific situations or to changes in the context later on Adopting 
innovative  communication  approaches  helps  overcome 
reaching 
disadvantaged participants. Finally, embedded communication strategies would help increase 
the  visibility  of  EU  support,  such  as  making  wider  use  of  social  media  and  forging 
partnerships with schools, community organisations and frontline services. Increased visibility 
would also boost outreach. 

the  challenge  of 

The  evaluation  of  ESF  support  to  employment  and  labour  mobility  (TO8,  excluding 
youth) has shown that the net effects of active labour market policies are particularly strong 
for  women  –  who  are  at  a  certain  distance  from  the  labour  market  –  through 
integrated/individual-based approaches. One of the main policy findings has been that there is 
a need to better fight embedded gender stereotypes and strive for more integrated approaches 
to promote active ageing. On labour mobility and support to structural changes more data and 
evidence will be crucial.  

In the field of ESF support to social inclusion (TO9), the thematic evaluation has found that  
the provision of personalised support is costly and means that providers require more training. 
Yet sufficient time and personalised support for participants is crucial to ensure needs are met 
and  to  achieve  the  desired  results.  A  participatory  approach  to  designing  and  implementing 
social inclusion interventions can also improve the provision of individualised support. Cross-
sectoral partnerships facilitate effective engagement with the target group and generate greater 
effectiveness.  Moreover,  more  inclusive  partnerships  and  outreach  strategies  promote  the 
recruitment of participants, making the intervention more relevant.  

Finally,  the  evaluation  focussing  on  ESF  support  to  education  and  training  (TO10) 
provides evidence on early school leaving and lifelong learning operations as having been the 
most successful ones, both in terms of direct results and the potential for longer-term systemic 
change.  The  relative  success  of  these  priorities  appears  to  relate  to  their  stronger  alignment 
with national and regional strategic priorities. Although support to education and training has 
effectively addressed the needs of young people in general, success has been more mixed for 
low-qualified adults, those not in education, employment or training and the hardest-to-reach 
disadvantaged groups. It calls for maintaining a strong focus on disadvantaged learners (and 
to increase it in particular for tertiary education). Also, there is a need to focus more on other 
skills that are increasingly relevant, such as digitalisation of learning and distance learning. 

The  results  of  the  three  thematic  evaluations  of  the  support  to  TO8  (excluding  support  to 
youth), TO9 and TO10 are presented in a single Commission’s staff working document16. 

14  https://ec.europa.eu/social/main.jsp?catId=738&langId=en&pubId=8346&furtherPubs=yes 
15  https://ec.europa.eu/social/main.jsp?langId=en&catId=89&newsId=9793&furtherNews=yes 
16  Commission  Staff  Working  Document  -  Evaluation  of  the  2014-2018  ESF  support  to  employment  and 
labour  mobility,  social  inclusion  and  education  and  training  -  SWD(2021) 10  final:  https://eur-
lex.europa.eu/legal-content/en/TXT/?uri=CELEX:52021SC0010 .   

18 

 
                                                 
 
The ex post evaluation of the ESF to be completed by the end of 2024 will build on these four 
thematic evaluations and the study carried out on TO11 (Progress assessment of ESF support 
to  public  administration).  The  Common  Provisions  Regulation  requires  the  funds  to  be 
examined  in  terms  of  their  effectiveness  and  efficiency  and  their  contribution  to  the  EU 
priorities of smart, sustainable and inclusive growth. The  ex post evaluation will also assess 
the other evaluation criteria of the Better Regulation Guidelines – relevance, added value and 
coherence. As in the past, the ex post evaluation will be carried out in close cooperation with 
the Member States and managing authorities. It will be launched in 2022. 

Also, with a view to better understanding possible ways of simplification in data collection, 
DG EMPL is planning a study to examine the legal issues related to access to administrative 
data for evaluation purposes.  

3.  EUROPEAN AGRICULTURAL FUND FOR RURAL DEVELOPMENT 

(EAFRD) 

3.1.  Evaluations completed by the Member States 

3.1.1.  General overview 

Overall,  the  Member  States  made  considerable  progress  in  their  evaluation  activities, 
particularly  compared  to  the  last  programming  period.  They  have  drawn  lessons  from  the 
evaluation  activities  for  the  enhanced  annual  implementation  reports  (AIRs)  in  2019,  with 
clear  action  points:  closing  data  gaps,  making  better  use  of  existing  data,  and  putting  more 
effort into quantifying complementary result indicators and impact indicators. These lessons 
are  frequently  addressed  in  the  reported  follow-up  activities  from  2020  and  will  guide  the 
evaluation efforts until the ex post evaluation.  

The examples of reported evaluation findings show that Member States have a broad range of 
evaluation outcomes at their disposal in the meantime that they increasingly use and feed into 
the  policy  design  process.  The  findings  are  being  used  to  optimise  the  technical 
implementation  of  the  current  rural  development  programmes  (RDPs)  –  e.g.  in  terms  of 
adapting  selection  criteria,  speeding  up  the  uptake,  providing  capacity  –  and  also  to  detect 
areas where there are still weaknesses in terms of uptake.  

The  reported  data  collection  efforts  have  been  impressive  –  environmental  monitoring  in 
particular received a lot of attention by the Member States in 2020. It is still a major challenge 
in  view  of  the  ex  post  evaluation  of  the  current  programming  period,  but  also  in  terms  of 
preparing  the  future  monitoring  and  evaluation  system  and  measuring  how  the  common 
agricultural policy contributes to the European Green Deal targets.  

In terms of assessing the quantified effects of RDPs, further efforts are needed in the Member 
States  to  credibly  establish  the  link  between  the  implemented  RDP  interventions  and  the 
actual impacts and to base it less on assumptions and more on clear quantitative links.  

The European Evaluation Helpdesk for Rural Development has also analysed specific lessons 
from  the  2019  evaluation  exercise  in  the  ‘Synthesis  of  the  Evaluation  Components  of  the 

Executive summary: 
https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52021SC0011&rid=5 
Communication package: 
https://ec.europa.eu/social/main.jsp?langId=en&catId=89&furtherNews=yes&newsId=9900 . 

19 

 
 
                                                                                                                                                         
 
 
 
Enhanced  AIRs  2019’17,  the  ‘Assessment  of  the  Progress  in  Implementing  the  Evaluation 
Plans  of  RDPs  2014-2020’18,  and  the  Good  Practice  Workshop  No  1219.  Against  this 
background,  a  specific  thematic  working  group20  was  set  up  in  2020  to  address  remaining 
challenges,  e.g.  on  understanding  and  methodologies  for  calculating  complementary  result 
indicators. 

For the 113 AIRs submitted in  2020, the amount and quality of the information  reported in 
Chapter  2  of  the  AIRs  in  2020  markedly  increased  overall,  in  particular  on  completed 
evaluations, their findings, and in the follow-up actions to evaluation results.  

More  than  90  AIRs  included  completed  evaluations  and  their  summaries,  compared  to 
around  75  in  2019.  The  number  of  follow-up  actions  has  tripled.  This  trend  reflects  the 
advanced  stage  of  the  programmes  within  the  evaluation  cycle  and  explains  the  increasing 
availability  of  evaluation  results  not  only  on  rural  development  programme  (RDP)  delivery 
and monitoring, but also on rural development achievements and impacts.  

The number of completed evaluations reported in the AIRs in 2020 (285) increased by almost 
25%  compared  to  2019.  Among  these,  23%  were  related  to  common  agricultural  policy 
(CAP)  objective  2  (Environment),  12%  to  objective  3  (Territorial  balance),  5%  to  CAP 
objective  1  (Competitiveness),  and  5%  to  horizontal  priority  1  (Knowledge  transfer  and 
innovation).  However,  the  vast  majority  of  completed  evaluations  (33%)  covered  multiple 
RDP  priorities  and  objectives.  This  can  be  explained  by  the  fact  that  many  of  the  reported 
evaluations  were  still  closely  related  to  the  enhanced  AIRs  submitted  in  2019  (e.g. 
background  evaluations,  studies),  which  by  their  very  nature  focus  on  all  RDP  priorities.  A 
large  share  of  completed  evaluations  were  related  to  RDP  governance,  implementation, 
monitoring and evaluation (9%), whereas the evaluation of technical assistance and national 
rural networks was specifically addressed in 5% of the completed evaluations. 

17  https://enrd.ec.europa.eu/evaluation/publications/summary-report-synthesis-evaluation-components-

enhanced-airs-2019-chapter-7_en  

18  https://enrd.ec.europa.eu/evaluation/publications/summary-report-assessment-progress-implementing-

evaluation-plans-rdps-2014_en  

19  https://enrd.ec.europa.eu/evaluation/good-practice-workshops/how-demonstrate-rdp-achievements-and-

impacts-lessons-learned_en  

20  https://enrd.ec.europa.eu/evaluation/thematic-working-groups/thematic-working-group-8-ex-post-

evaluation-rdps-2014-2020_en 

20 

 
                                                 
Figure 4: Completed EAFRD evaluations across main topics 

Source: European Evaluation Helpdesk for Rural Development (2020) 

Modifications  to  the  RDP  evaluation  plan  fell  markedly  by  37%  compared  to  2019. 
Modifications  mostly  concerned  the  update  of  information  on  governance  and  coordination 
arrangements  for  evaluation,  the  evaluation  topics  and  the  adjustments  of  resources  for 
evaluation. In total, 26 modifications were mentioned in 8 AIRs.  

The  types  of  evaluation  activities  reported  indicate  that  most  RDPs  have  shifted  from  the 
planning and preparation phase of the evaluation cycle to actually conducting, disseminating 
and  following  up  on  evaluations.  Different  strategies  to  coordinate  the  generation  of 
evaluation results (e.g. working groups, evaluation units, outsourcing) and ensure their quality 
(e.g.  manuals,  capacity  building  etc.)  have  become  apparent  as  part  of  preparing  the 
evaluation inputs for the AIRs.  

21 

 
 
 
Figure 5: Evaluation activities in relation to the main phases of the evaluation cycle 

Follow-up activities 
6% 

Planning & 
Coordinating 
15% 

Reporting & 
Disseminating 
27% 

Preparing & 
Structuring 
15% 

Conducting 
37% 

Source: European Evaluation Helpdesk for Rural Development (2020) 

As for the activities related to the provision and management of data for evaluations, they 
increased slightly by 9% compared to 2019. They focused on agreements with data providers, 
improving the governance of data management and data collection from application forms, as 
well as on additional data needed for quantitative analysis. In total, 357 data-related activities 
were reported in  the AIRs in  2020, most of which concerned the processing/analysis phase. 
The highest number of data-related activities (49%) dealt with processing and analysing data 
as well as preparing the data systems (32%).  

As  for  communication  activities  used  to  publicise  evaluation  findings,  a  substantial 
number  (372)  were  reported  by  the  RDPs  in  2020.  Five  million  stakeholders  have  been 
reached,  mostly  through  websites  (4.6  million),  but  also  through  a  combination  of  different 
communication channels, i.e. newsletters, social media, evaluation reports, as well as internal 
and  open  meetings,  workshops,  seminars,  etc.  In  terms  of  target  groups,  most  of  the 
communication activities targeted selected groups, such as RDP monitoring committees, local 
action  groups,  national/regional  authorities,  researchers  and  thematic  experts,  evaluators, 
national  rural  networks,  rural  networks  and  associations.  In  addition,  28%  of  the  reported 
communication activities addressed the general public.  

22 

 
 
 
 
Figure 6: Number of communication activities by type and stakeholders reached 

Source: European Evaluation Helpdesk for Rural Development (2020) 

Follow-ups  on  evaluation  results  have  been  reported  to  a  much  higher  extent  than  in 
previous  years,  and  show  that  the  use  of  evaluation  findings  is  becoming  increasingly 
important. In 2020, the AIRs contained 1195 follow-ups, which is three times more than in 
the AIRs submitted in 2019. Most of these relate to improving the RDP delivery mechanism 
(36%), while 27% are already directly related to using evaluation results to prepare the CAP 
strategic  plans  for  the  post-2020  period.  Adaptation  of  the  RDP  monitoring  and  evaluation 
system is addressed by 17% of the reported activities. 

Figure 7: Reported follow-up activities given to evaluation results 

Improving the RDP delivery mechanism and
implementation

Preparing the CAP Strategic Plan for the new
period

36% 

27% 

Adapting the RDP Monitoring and Evaluation
system

17% 

Improving the RDP intervention logic

7% 

Other

13% 

Source: European Evaluation Helpdesk for Rural Development (2020) 

3.1.2.  Main findings of evaluations 

Distribution of the evaluation findings across evaluation topics follows one of the completed 
evaluations shown above. However, the types of evaluation  findings vary considerably. The 
reported findings not only concern RDP results and impacts, but also relate to a much broader 
range of evaluation outcomes.  

23 

 
 
 
The  largest  share  (42%)  of  findings  concerned  RDP  results  and  impacts  (24%  of  them 
reporting clearly positive impacts/results). A considerable share of findings (31%) concerned 
the  delivery  mechanism  of  rural  development  programmes  (assessment  of  selection  criteria, 
budget,  communication  activities  etc.).  18%  related  to  monitoring  progress  against  the 
objectives/targets  (e.g.  uptake  under  measures,  progress  in  monitoring  indicators  etc.).  The 
rest  (9%)  concerned  other  aspects  (e.g.  update  of  SWOT  analysis,  strategic  environmental 
assessment findings, concrete recommendations of evaluations etc.).  

Evaluation findings were reported by 99 RDPs, otherwise cross-references to AIR Chapter 7 
from the previous year were included. In fact, the findings linked to the assessment of RDP 
achievements  and  impacts  had  been  reported  in  detail  in  the  previous  year  by  providing 
detailed answers to the 30 common evaluation questions. These findings were presented in the 
last  staff  working  document  and  are  not  repeated  here.  Nevertheless,  some  complementary 
evaluation findings reported in 2020 in relation to the three CAP objectives and the horizontal 
priority are depicted below. These show the range of findings in relation to assessing delivery, 
monitoring progress and quantifying the RDP results and impacts. 

  CAP objective 1 – Fostering the competitiveness of agriculture  

A  number  of  RDPs  have  assessed  delivery  of  the  measures  under  the  related  rural 
development  priorities  2  (farm  viability  and  competitiveness)  and  3  (food  chain 
organisations). For example, Piedmont  (Italy) analysed the selection criteria for investments 
in agriculture and for operations related to the setting up of young farmers. It showed that the 
quality  of  projects  has  increased  overall.  Positive  contributions  had  also  been  found  for 
example by the RDP Croatia in relation to improving the economic performance of all farms 
(Focus  Area  2A),  notably  because  there  are  strong  synergies  between  the  contributing 
measures. Moreover, despite the overall negative trend, the share of young managers joining 
the  agricultural  sector  has  increased.  In  Flanders  (Belgium),  an  evaluation  of  the  economic 
impact  of  the  aid  showed  that  it  helped  make  enterprises  more  resilient  and  increased  their 
gross added value. In the RDP Hungary, the average agricultural family income has increased 
more rapidly in supported farms (32.5%) than in the control group (16.9%). The RDP Cyprus 
noted  a  significant  RDP  contribution  to  strengthen  cattle  breeding  but  still  marginally  to 
improve the competitiveness in sheep and goat farming.  

  CAP objective 2 – Ensuring the sustainable management of natural resources 

and climate actions  

Many  Member  States  reported  on  findings  stemming  from  their  environmental  monitoring 
(e.g.  of  farmland  birds  in  Flanders,  Slovenia,  Castile-La  Mancha  (Spain)  and  also  from  the 
German  RDP  on  Saxony,  where  a  vegetation  survey  and  plant-ecological  evaluation  was 
carried  out).  The  RDP  Estonia  was  able  to  find  evidence  that  its  average  indicators  of 
bumblebees per census track were higher in farms that received support for organic farming 
or  more  environmentally  friendly  management  (agri-environment  climate  payments). 
Nevertheless,  some  RDPs  report  that  the  effects  on  the  environment  have  still  been  rather 
limited  in  certain  areas  (e.g.  Marche  Italy,  Greece,  Croatia)  or  that  they  had  issues  in 
quantifying  some  of  the  related  indicators  (e.g.  in  the  RDP  Cyprus,  the  impact  indicator  on 
energy  savings  was  difficult  to  calculate).  Still,  the  RDP  Cyprus  also  found  evidence  for  a 
significant  RDP  contribution  to  improving  biodiversity  as  reflected  in  the  Farmland  Bird 
Index, together with a positive contribution to improving water quality. Croatia found that its 
RDP  measures  have  provided  a  significant  contribution  to  maintaining  and  improving 
biodiversity  in  areas  with  natural  constraints  as  well  as  contributions  to  conserving 
endangered native and protected breeds of domestic animals.  

24 

 
  CAP objective 3 – Achieving a balanced territorial development  

The delivery of measures under this CAP objective has in many cases been assessed with the 
main focus  on the implementation of  LEADER/CLLD.  Beyond  LEADER, the RDP  Cyprus 
found  that  its  programme  only  made  a  marginal  contribution  to  creating  employment, 
increasing GDP and reducing poverty. This is explained by the limited implementation of the 
measures  under  the  related  focus  areas.  While  some  RDPs  still  report  a  too  low  number  of 
completed  projects  to  properly  estimate  the  impact,  positive  results  and  impacts  have  been 
reported for example by the German RDP on Hessen, where the case studies on the support of 
rural infrastructures demonstrated that the competitiveness of agricultural holdings had been 
strengthened. In the Spanish RDP on Castile-La Mancha, the assessment carried out by local 
action  groups  helped  demonstrate  the  added  value  of  LEADER,  in  terms  of  better  results, 
social  capital  and  local  governance.  With  the  help  of  case  studies  on  its  support  to  rural 
infrastructures,  the  RDP  on  Hessen  found  that  the  routes  supported  are  multifunctional  and 
are  therefore  not  only  used  by  agriculture,  but  also  by  the  local  population.  This  helped 
strengthen  the  competitiveness  of  agricultural  holdings  and  at  the  same  time  increased  the 
quality of life of the local population, while avoiding negative environmental impacts. 

  Rural development priority 1 (fostering knowledge transfer and innovation in 

agriculture) 

Delivery of the interventions related to this priority has been assessed for example by the RDP 
Estonia.  It  found  that  the  change  to  the  advisory  system  and  the  conditions  of  advisory 
support negatively affected the use of counselling and the functioning of the advisory system. 
In  the  German  RDP  on  Saxony,  an  evaluation  of  advisory  services  in  the  field  of  nature 
conservation showed that consultants in some areas are reaching their capacity limits and that 
too few new stakeholders have ‘grown up’.  

The RDP Hungary reports on the positive trend in a steady increase in the share of qualified 
farm managers (highly skilled and secondary skilled). Looking at the period of available data, 
the share of qualified farm managers increased from 17.8% to 21.4% between 2013 and 2016. 
Bretagne (France) found in its initial results that the RDP contributes through some European 
Innovation  Partnership  projects  to  the  development  of  the  knowledge  base  in  rural  areas, 
notably the forest sector. In fact, RDP interventions have encouraged stronger links between 
agriculture,  food  production,  research  and  innovation,  including  the  improvement  of 
environmental management and performance. 

3.1.3.  Support to Member States 

In 2020, the European Evaluation Helpdesk21 continued to help Member States improve their 
evaluation capacities. Examples of specific activities included: 

-  providing Yearly Capacity Building events for the Member States; 
-  organising ‘Good practice’ workshops to discuss lessons and experiences on selected 

topics 

-  publishing guidance documents and tools; 
- 

collecting  good  practices  and  publishing  them  on  webpages,  in  newsletters  and  in 
factsheets issued by the European Evaluation Helpdesk (e.g. on the impact of RDPs on 
soil, animal welfare, ex ante evaluation). 

21  https://enrd.ec.europa.eu/evaluation_en 

25 

 
                                                 
3.2.  Evaluation work by the Commission 

On the evaluation work of the Commission, specific thematic evaluations are also being used 
to assess the effects of rural development programmes, supported by external studies financed 
by the Commission. The evaluations assess the effectiveness, efficiency, relevance, coherence 
and EU added value of all relevant CAP instruments. 

The  study  that  supports  the  evaluation  of  the  impact  of  the  CAP  measures  towards  the 
general objective of viable food production was published in November 2020. It provides a 
generally  positive  assessment  of  the  effectiveness,  efficiency,  relevance,  coherence  and  EU 
added value of the relevant measures, but points to room for improvement in the effectiveness 
of  coupled  payments,  decoupled  direct  payments  and  price  stability  tools,  as  well  as  the 
administrative  and  management  efficiency  of  the  greening  payment  and  the  active  farmer 
clause. The support study shows that CAP measures that address viable food production have 
strongly supported farm income and that market measures have helped limit the volatility of 
domestic prices for most agricultural products. It is important to highlight that the study was 
subject  to  considerable  limitations  due  to  the  short  observation  period  and  resulting  lack  of 
accurate,  detailed  and  homogenous  data.  More  recent  observations  and  complementary 
analysis will be added. 

In January 2021, the Commission published a staff working document on the  evaluation of 
the  impact  of  the  EU  agricultural  promotion  policy  in  internal  and  third  country 
markets.  It  deems  the  EU  agri-food  promotion  policy  to  boost  competitiveness  and 
consumption of EU products in and outside the EU to be effective and efficient. In particular, 
activities funded under the promotion policy help increase the sales and consumption of EU 
products  promoted  in  target  markets,  although  the  quantitative  extent  of  this  is  difficult  to 
measure. The objectives of the policy are relevant to stakeholders’ needs and address market 
challenges.  There  are  no  major  inconsistencies  with  other  EU  policies,  including  health, 
climate,  environmental  and  development  policies.  This  evaluation  will  feed  into  an  overall 
policy review planned for 2021, which aims to boost its contribution to sustainable production 
and consumption in line with the Commission’s Farm to fork strategy’s objectives.  

The  Commission  published  a  staff  working  document  on  the  evaluation  of  the  CAP 
measures applicable to the wine sector in October 2020. The evaluation concluded that the 
national support programmes have played a key role in improving the competitiveness of EU 
wine  producers  and  products,  in  particular  increasing  demand  from  third  countries.  The 
measures in the national support programmes are generally relevant to the needs of the sector, 
especially  because  they  offer  a  range  of  tools  that  can  be  adapted  to  the  various  levels  of 
development  of  EU  local  supply  chains  (i.e.  restructuring  and  conversion,  investment,  and 
promotion). The needs not sufficiently addressed by the EU’s wine policy include survival or 
adaptation  of  the  smallest  operators,  the  need  for  a  better-trained  workforce,  renewal  of 
businesses between  generations, environmental issues, and adaptation  to  market  demand for 
lower  alcohol  wines  and  sustainable  products.  While  there  could  also  be  greater  coherence 
between the EU’s wine policy and EU public health objectives, overall the EU’s wine policy 
is fully consistent with EU economic, social and CAP objectives.  

The  evaluation  of  the  CAP’s  impact  on  biodiversity,  water  and  soil  is  supported  by  a 
number of different studies. The first one, on the CAP’s impact on habitats, landscape and 
biodiversity, published in March 2020, concluded that the presence of the CAP has increased 
the Member States’ ambition to address biodiversity objectives as well as the level of funding. 
However,  the  CAP’s  contribution  and  benefits  are  highly  dependent  on  Member  States’ 
implementation choices and priorities. For certain Member States, their biodiversity priorities 
are  not  well  reflected  in  their  CAP  decisions.  The  most  effective  measures  for  conserving 
semi-natural habitats are agri-environment climate schemes, particularly tailored and targeted 

26 

 
to higher-level schemes, and the Natura 2000 measure, which can compensate land managers 
for  the  costs  of  mandatory  conservation  protection,  particularly  within  Natura  2000  areas. 
However, the impacts of agri-environment climate schemes are often constrained by limited 
budgets  and  farmer  uptake,  and  the  Natura  2000  measure  has  not  been  used  enough  by 
Member States. It is worth noting that the study outcomes were limited due to the absence of 
suitable monitoring data, where it makes further recommendations.  

The second study, on the  CAP’s impact on water, also  published in  March 2020, assessed 
the CAP framework as being effective in maintaining minimum practices beneficial for water 
quality;  however,  its  effects  on  the  quantitative  aspects  of  water  are  rather  contrasted.  The 
effects  of  CAP  instruments  and  measures  on  water  were  often  difficult  to  assess  due  to  the 
lack of data. The analysis highlighted the importance of implementation choices by Member 
States, as only a few measures actually supported operations that directly target water quality 
and quantity issues.  

The  third  study,  on  the  CAP’s  impact  on  sustainable  soil  management,  published  in 
February 2021, recognised that CAP instruments and measures introduced provisions on the 
use of plant protection products and fertilisers on a significant share of EU arable land. The 
CAP also contributed to lasting changes in farming practices by helping introduce the use of 
catch-,  cover-  and  nitrogen-fixing  crops.  The  analysis  suggests  that  efforts  to  reduce  soil 
erosion need to be strengthened, in particular in hotspots. The efficiency of the policy varies 
depending on the measures considered. Payment rates under support for activities in forests, 
the  environmental  measures  of  fruit  and  vegetable  operational  programmes  and  support  for 
organic  farming  were  generally  sufficient  to  encourage  farmers  and  forest  holders  to  apply. 
While the relevant rules address the need to limit erosion, increase carbon content in mineral 
soils,  protect  grasslands  and  ensure  that  their  carbon  content  is  maintained,  they  are 
considered not very ambitious and comprehensive. However, the EU regulations seem to have 
raised the level of ambition on soil protection. 

The  study  that  supports  the  evaluation  of  the  CAP’s  impact  on  knowledge  exchange  and 
advisory activities was published in February 2021. It concluded that the CAP measures are 
generally effective, efficient, relevant, coherent and offer value added in creating knowledge 
exchange  and  promoting  advisory  activities  and  innovation  in  the  agricultural  and  forestry 
sectors and in rural areas. The main drawback of the policy is that the latter results reach only 
around  10%  of  EU  farm  holdings  and  20%  of  CAP  beneficiaries.  Furthermore,  CAP 
instruments  and  measures  contribute  to  Member  States’  Agricultural  Knowledge  and 
Innovation Systems  (AKIS), as shown in  all case studies in Member States.  However, CAP 
support only concerns a small part of Member States’ AKIS. The increased budget (compared 
to  2007-2013)  allocated  by  rural  development  programmes  to  knowledge  transfer  and 
information actions (M1), advisory services (M2) and innovation (M16) suggests a growing 
interest in using these means and activities, in particular to achieve CAP objectives. 

The  study  that  supports  the  evaluation  of  the  socioeconomic  aspects  of  CAP  impacts  on 
territorial  development  of  rural  areas  was  also  published  in  February  2021.  The  overall 
conclusion  from  the  study  is  that  CAP  support  to  balanced  territorial  development,  which 
aims to improve socioeconomic and social inclusion aspects, is generally positive. However, 
the  impacts  tend  to  vary  considerably  depending  on  the  characteristics  of  the  rural  region 
analysed  and  the  policy  mix  applied.  CAP  funding  is  shown  to  be  especially  helpful  in 
developing highly rural, remote and/or agriculturally-dependent areas. Pillar I is found to be 
efficient  overall,  but  Pillar  II  is  considered  less  efficient  due  to  the  application  of  rules  on 
public  funding,  reducing  their  accessibility  to  socially  and  economically  disadvantaged 
beneficiaries.  There  is  an  overall  level  of  consistency  between  the  various  measures  and 
instruments,  although  separate  planning  of  the  interventions  weaken  their  complementarity. 
By  focusing  significant  resources  on  remote  and  very  marginal  rural  areas,  the  CAP  helps 

27 

 
address needs that national or regional policy instruments may struggle to meet on their own. 
The  evaluation  will  feed  into  the  design  of  a  long-term  vision  for  rural  areas,  which  the 
Commission has committed to develop together with all its stakeholders. 

4.  EUROPEAN MARITIME AND FISHERIES FUND (EMFF) 

4.1.  Evaluations completed by the Member States 

EMFF programme implementation advanced significantly in 2018 and 2019.  Member States 
had  committed  €3.24  billion  of  the  available  €5.7  billion  in  the  fund  by  the  end  of  2019. 
During this time, a total of 41 evaluation operations were conducted on the 27 Member State 
programmes,  with  €4.17  million  committed  to  support  this  work.  While  the  focus  and 
maturity of evaluations across the Member States are wide ranging, Member States evaluation 
capacity for the EMFF has been boosted by the Commission by providing practical support to 
Managing  Authorities,  which  helped  to  enrich  the  scope  and  strength  of  evaluations 
conducted.  

The  evaluations  have  the  effect  of  providing  evidence  to  improve  programme  and  policy 
decisions. According to information provided in their Annual Implementation Reports (AIR), 
several  Member  States  undertook  Operational  Programme  (OP)  modifications  as  a  result  of 
recommendations  arising  from  their  evaluations  (Austria,  Belgium,  Bulgaria,  France, 
Slovakia,  United  Kingdom).  Changes  were  based  on  evaluation  findings  including  the 
identification  of  delays  or  programme  objectives  that  were  not  optimised.  Nevertheless,  the 
evaluations  found  that  progress  towards  objectives  was  satisfactory.  Improvement  of 
administrative  capacity  and  communication  to  the  beneficiaries  to  simplify  administrative 
procedures were identified as common areas to be improved. 

In many Member States, evaluations also made specific reference to preparations for the next 
programming period. 

Most Member States outsourced their evaluation tasks. Only Sweden and Slovakia mentioned 
that external evaluations were used with internal evaluations. 

The  most  frequent  type  of  evaluations  conducted  by  the  Member  States  are  process 
evaluations  (implemented  by  Cyprus,  Czechia,  Spain,  France,  Malta,  Portugal,  United 
Kingdom). 

Most  evaluations  were  structured  to  address  effectiveness  and  efficiency  at  operational 
programme level as well as specific objectives and measures (Cyprus, Czechia, Spain, France, 
Latvia, Malta, Croatia, Portugal). 

Specific evaluations include:  

- 

- 

- 

- 

- 

- 

- 

implementation  of  Community  Led  Local  Development  (Estonia,  Ireland,  Croatia, 
Portugal, Romania); 

achievement of indicators (Austria, Slovakia); 

cost/benefit analysis of decommissioning scheme (Ireland); 

evaluation of lobster v-notching scheme (Ireland); 

evaluation of sustainable fisheries scheme (Ireland); 

recirculating aquaculture systems (Finland); 

impact assessment of the ‘blue economy’ sectors (Latvia); 

28 

 
 
- 

ex ante assessment of financial instruments  (Czechia, Ireland). 

Detailed  information  summarising  the  evaluations  of  each  Member  State  based  on 
information provided in AIRs is available in Section 7 of the EMFF implementation report for 
2019, published in December 202022.  

4.2.  Evaluation work by the Commission 

A  mid-term  evaluation  of  the  direct  management  portion  of  the  EMFF  was  conducted.  The 
staff working document was published in 202023. The evaluation found that the investments 
made had delivered value for money and recommended a framework of indicators compatible 
with  shared  management  in  the  future.  The  agreement  of  co-legislators  on  the  European 
Maritime Fisheries and Aquaculture Fund for 2021–2027 has enshrined this in the text of the 
fund regulation. 

5.  FINAL REMARKS  

Following  the  progress  made  on  implementation,  the  evaluation  effort  has  increased 
significantly and has outperformed the corresponding effort made in a comparable time of the 
previous  period.  The  majority  of  evaluations  are  still  assessing  implementation  issues. 
However, the proportion of impact evaluations has been growing steadily. While it is difficult 
to  establish general  findings, there is  evidence that ESI  Funds support has been effective in 
many  areas  in  supporting  different  types  of  beneficiaries  –  both  in  the  public  and  private 
sectors  and  individuals.  Evidence  of  EU  added  value  is  more  pronounced  in  the  area  of 
inclusive growth, which has the largest number of impact evaluations.  

While the COVID-19 pandemic has led to a partial reorientation of programmes, increasing 
support  to  the  health  sector,  SMEs  and  vulnerable  populations,  this  shift  has  not  yet  been 
reflected in the evaluations of managing authorities. The effects will have to be factored into 
future analyses. 

The  Commission  will  continue  to  provide  support  to  the  Member  States  through  dedicated 
networks  and  services.  The  main  issues  reported  in  the  evaluations  will  be  tackled  with 
increased support, tailored to the specific needs. At the same time, the managing authorities 
should  ensure  that  they  strengthen  their  evaluation  capacity.  This  should  build  on  the 
important improvements already observed as the new programming period starts. 

In 2021-2027,  an intervention logic  at  the level  of the specific objective  and a performance 
framework  includes  output  and  result  indicators,  together  with  simplified  evaluation 
requirements. This will help the Member States focus their evaluation efforts on investigating 
the mechanisms of EU support and on whether it actually helped strengthen economic, social 
and territorial cohesion. 

Evaluation  findings  on  the  2014-2020  programmes  will  provide  valuable  input  in  their 
respective areas for preparing the programmes for 2021-2027. They will be complemented by 
an  even  larger  number  of  impact  evaluations  in  the  final  years  of  the  programming  period, 
when more results are expected to emerge. 

22  https://ec.europa.eu/fisheries/sites/fisheries/files/emff-implementation-report-2019_en.pdf  
23  SWD(2020) 221 final. 

29