COMMISSION STAFF WORKING DOCUMENT Accompanying the document REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS European Structural and Investment Funds 2014-2020 2018 Summary report of the programme annual implementation reports covering implementation in 2014-2017
Date: 2019-04-01
Institutions: Directorate-General for Regional and Urban Policy, European Commission
Subjects: EU aid, EU investment, European Structural and Investment Funds, economic and social cohesion, economic growth, job creation, productivity, single market, structural adjustment, sustainable development

EN 

  EN 

   EUROPEAN COMMISSION  Brussels, 1.4.2019  SWD(2019) 145 final  COMMISSION STAFF WORKING DOCUMENT Accompanying the document REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS  European Structural and Investment Funds 2014-2020 2018 Summary report of the programme annual implementation reports  covering implementation in 2014-2017  {COM(2018) 816 final}  
 
 
STAFF WORKING DOCUMENT 

ACCOMPANYING THE 2018 SUMMARY REPORT 

ON THE EUROPEAN STRUCTURAL AND INVESTMENT FUNDS 2014-2020 

Table of Contents 

1. 

INTRODUCTION ....................................................................................................... 2 

2.  ERDF AND COHESION FUND ................................................................................ 3 

2.1. Synthesis of evaluation work completed by the Member States .......................... 3 
2.1.1.  Findings from evaluations on the 2014-2020 period ........................... 5 
2.1.2.  Findings from evaluations on the 2007-2013 period ........................... 6 
2.2. Synthesis of ERDF/CF evaluation work by the Commission ............................ 11 

3.  ESF AND YEI ........................................................................................................... 13 

3.1. Synthesis of ESF/YEI evaluations completed by the Member States ................ 14 
3.1.1.  Findings from evaluations on the 2014-2020 period ......................... 17 
3.1.2.  Findings from evaluations on the 2007-2013 period ......................... 18 
3.2.  Synthesis of ESF evaluation work by the Commission .................................. 20 
3.2.1.  Update of the 2007-2013 ex-post data .............................................. 20 
3.2.2.  Launch of two thematic evaluations .................................................. 21 

4.  EAFRD 

-  EUROPEAN  AGRICULTURAL  FUND  FOR  RURAL 

DEVELOPMENT ..................................................................................................... 21 

4.1. Synthesis of evaluation work by the Member States ......................................... 21 
4.1.1.  Period 2014-2020 .............................................................................. 21 
4.1.2.  Ex-post evaluations 2007-2013 ......................................................... 24 
4.2. Synthesis of EAFRD evaluation work by the Commission ............................... 25 

5.  EMFF - EUROPEAN MARITIME AND FISHERIES FUND (EMFF) .................. 26 

5.1. Period 2014-2020 ............................................................................................... 26 
5.2. Period 2007-2013 ............................................................................................... 26 

6.  CONCLUSION ......................................................................................................... 27 

1 

 
 
 
 
 
 
 
 
1. 

INTRODUCTION 

This  Staff  Working  Document  provides  a  complementary  synthesis  of  the  evaluation  work 
undertaken  by  the  Member  States  and  the  Commission  in  order  to  assess  the  impact  of  the 
programmes supported by ESI Funds1. 

The legislation for the period 2014-2020 sets out a stronger emphasis on the need to evaluate 
the  contribution  to  growth,  sustainable  development  and  job  creation  of  the  policies  co-
financed  through  the  ESI  Fund  programmes.  The  programmes  define  objectives  that  are 
specific  and  articulate  the  change  sought  in  particular  areas.  Evaluations  are  essential  to  try 
establishing whether those changes have taken place  and  whether and how the programmes 
have contributed. 

The  Staff  Working  Document  accompanying  the  2017  Summary  Report2  synthesised  the 
evaluation work expected under the national and programme evaluation plans, highlighting a 
good  level  of  completeness  and  coherence  with  regard  to  coordination,  management  and 
communication,  with  improvements  needed  on  skills,  methods  and  data  requirement.  An 
updated analysis, focused on the revised evaluation plans received by the Commission  until 
June 2018, shows that there are still shortcomings in the latter areas. 

The number of completed evaluations published by Member States has increased significantly 
since last year, although at a lower level than planned both for implementation- and progress-
oriented  evaluations  and,  to  a  larger  extent,  for  impact  evaluations.  Further  increase  of 
completed  evaluations  is  expected  in  2018-2019,  although  their  number  will  be  probably 
revised  from  what  was  initially  planned  in  order  to  take  into  account  the  actual  pace  of 
implementation. 

The  impact  evaluations  that  have  been  undertaken  so  far  are  aimed  not  only  to  assess  the 
effects  of  programmes  but  also  their  implementation  and  progress  towards  achieving  the 
targets  set.  Overall,  if  compared  to  planned  evaluations,  they  show  on  average  clearer 
evaluation design and more appropriate use of data. 

However,  an  in-depth  analysis  of  the  reliability  of  impact  evaluations,  related  to  both 
programming periods, showed that only a limited number of them can be considered as fully 
reliable, which highlights the need for improvement in the quality of the works produced by 
the Member States, to which the Commission is committed to provide necessary support.  

First  evidence  is  available  from  the  Member  States  on  the  period  2014-2020,  mostly  for 
ESF/YEI and show positive impact on employment of the various measures evaluated so far, 
while impact evaluations on the other Funds are still very limited, due to the scope and nature 
of  the  actions  funded.  For  the  2007-2013  period,  impact  evaluations  show  that  most 
investments  would have not  been carried out  without EU support and their success  requires 
strong coordination of different measures and cooperation among stakeholders. There is also 
evidence that investments have led to a greater attractiveness of the interested areas, although 

1   https://cohesiondata.ec.europa.eu/  
2   SWD(2017) 452 final - https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=SWD:2017:452:FIN  

2 

 
 
 
 
                                                 
with some exceptions. Support to SMEs has increased employment, productivity and exports 
and can be more effective when complemented with non-financial support. 

In  the  sections  that  follow  an  update  and  synthesis  of  the  evaluations  concluded  by  the 
Member States and the Commission are provided for each of the ESI Funds. 

2.  ERDF AND COHESION FUND 

2.1. Synthesis of evaluation work completed by the Member States 

The  national  and  regional  programmes  devoted  considerable  attention  to  the  topic  of 
evaluation.  The  majority  of  the  evaluations  identified  since  January  2015  are  ex-post 
evaluations  of  the  2007-2013  programming  period,  although  the  number  of  evaluations 
concerning 2014-2020 is now increasing faster, as almost 60% identified in the last 12 months 
until June 2018 are referring to 2014-2020, including those referring to both periods. 

Table  1:  Evaluations  published  since  January  2015  on  Cohesion  policy  interventions; 
breakdown by programming period 

  ERDF+CF  Multi-Fund  Total 

2007-2013 

2014-2020 

Both periods 

Total 

171 

90 

4 

265 

58 

87 

4 

229 

177 

8 

149 

414 

The  evaluation  effort  is  concentrated  in  some  Member  States.  In  particular,  Poland  (158), 
Spain (60), Hungary (26), UK (25), Lithuania (20), Italy (19), Slovakia (15), Czech Republic 
(14), Germany (12) and Romania (12) make up almost 90% of all evaluations identified. The 
high  variability  in  the  number  of  published  evaluations  among  Member  States  depends  not 
only on the different intensity of evaluations planned on the various thematic objectives but 
also on the different timing of the planned evaluations. 

Figure 1: Evaluations published since January 2015 by Member State 

3 

158602625201915141212109665552111110000020406080100120140160180 
 
 
 
 
 
 
the  2014-2020  period,  most  of 

Concerning 
identified  are  either 
process/implementation oriented or look mostly at the progress of the programmes and do not 
yet  allow the assessment of the contribution  of the programmes to  the expected results. For 
this reason, the following paragraphs will dwell mostly on the 2007-2013 evaluations.  

the  evaluations 

Although the number of evaluations identified has increased substantially in the last year, it is 
still behind the level initially planned. Overall, the number of evaluations identified until June 
2018 is approximately 50% of the planned value for the end of 2017, for impact evaluations, 
which require a more advanced level of implementation, the figures are even lower. From the 
evaluation  plans,  a  more  conspicuous  number  of  impact  evaluations  related  to  2014-2020 
programmes is expected in 2018-2019, although these figures will be probably revised to take 
into account the actual pace of implementation. 

As the goal of this document is to bring to light the effects of the interventions supported and 
the extent to which cohesion policy has contributed to economic and social development, the 
following  synthesis  is  based  on  the  findings  of  impact-oriented  evaluations.  These  include 
evaluations  dealing  partly  with  process/implementation  or  monitoring/progress  issues  in 
addition to those focussed on impact alone. 

In order to draw messages that can be relevant to the policy, an in-depth analysis was carried 
out to take into account the extent to which the findings of the reports appear to be reliable. 
The evaluations were inspected on the basis of a set of quality criteria, such as the clarity and 
suitability  of  the  evaluation  design,  the  appropriateness  and  correctness  of  the  techniques 
applied, data quality and availability, the validity of the findings and of the conclusions3.  

Overall,  if  compared  to  the  quality  rating  of  planned  impact-oriented  evaluations,  the 
evaluations reviewed show on average clearer evaluation design and more appropriate use of 
data. The subset of the most reliable impact evaluations includes approximately 30% of those 
scrutinised.  This  proportion  is  based  on  a  limited  number  of  cases  and  therefore  cannot  be 
generalised,  however  it  highlights  the  need  for  improvement  in  the  quality  of  the  works 
produced by the Member States. 

3   Each  criterion  is  assigned  a  score  ranging  from  ‘4’  strong/good  or  ‘3’  fairly  strong/good  to  ‘2’  fairly 
weak/poor or ‘1’ weak/poor (i.e. many weaknesses and limitations are identified therefore findings are not 
considered reliable). 

4 

 
 
 
 
 
 
 
                                                 
Table 2: Quality rating of planned impact-oriented evaluations – Breakdown by Fund, 
including ESF/YEI 

Fund 

Background 
information 

Objectives 

Design & 
Methods 

Data & 
Source 

Budget & 
Timing 

Overall 
average 

ERDF/CF 

ESF/YEI 

multi-Fund 

Average 

2,9 

2,7 

2,6 

2,7 

2,6 

2,6 

2,6 

2,6 

2,0 

2,0 

2,1 

2,1 

2,3 

1,8 

2,1 

2,2 

2,8 

2,7 

2,3 

2,5 

2,5 

2,4 

2,3 

2,4 

Table 3: Quality rating of impact-oriented evaluations reviewed, including ESF/YEI 

2007-2013 

2014-2020 

-
s
s
e
r
g
o
r
p
/
g
n
i
r
o
t
i
n
o
m
d
n
a

t
c
a
p
m

I

d
e
t
n
e
i
r
o
-
t
c
a
p
m

I

d
n
a

t
c
a
p
m

I

d
e
t
n
e
i
r
o

n
o
i
t
a
t
n
e
m
e
l
p
m

i
/
s
s
e
c
o
r
p
d
n
a

t
c
a
p
m

I

d
e
t
n
e
i
r
o
-
s
s
e
r
g
o
r
p
/
g
n
i
r
o
t
i
n
o
m
d
n
a

d
e
t
n
e
i
r
o
-
n
o
i
t
a
t
n
e
m
e
l
p
m

i
/
s
s
e
c
o
r
p

2,8 

2,6 

2,8 

2,7 

l
a
t
o
T

2,8 

-
s
s
e
r
g
o
r
p
/
g
n
i
r
o
t
i
n
o
m
d
n
a

t
c
a
p
m

I

d
n
a

t
c
a
p
m

I

d
e
t
n
e
i
r
o

n
o
i
t
a
t
n
e
m
e
l
p
m

i
/
s
s
e
c
o
r
p
d
n
a

t
c
a
p
m

I

d
e
t
n
e
i
r
o
-
s
s
e
r
g
o
r
p
/
g
n
i
r
o
t
i
n
o
m
d
n
a

d
e
t
n
e
i
r
o
-
n
o
i
t
a
t
n
e
m
e
l
p
m

i
/
s
s
e
c
o
r
p

d
e
t
n
e
i
r
o
-
t
c
a
p
m

I

l
a
t
o
T

l
l
a
r
e
v
O

2,8 

2,8 

3,1 

3,0 

2,9 

2,8 

2,5 

2,4 

2,4 

2,5 

2,5 

2,8 

2,0 

2,6 

2,2 

2,5 

2,5 

2,7 

2,6 

2,6 

2,5 

2,6 

3,0 

2,8 

3,0 

1,6 

2,7 

2,6 

Clarity of 
evaluation design 

Appropriateness of 
evaluation design  

Data quality and 
availability 

Validity of findings  2,6 

2,4 

2,3 

2,5 

2,5 

2,4 

2,5 

2,7 

1,8 

2,4 

2,5 

Validity of 
conclusions 

2,4 

2,5 

2,5 

2,6 

2,4 

2,5 

2,5 

3,1 

2,0 

2,6 

2,5 

Overall 

2,6 

2,5 

2,5 

2,6 

2,5 

2,7 

2,5 

2,9 

2,1 

2,6 

2,6 

5 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
The following summary of findings is based on a sample of 42 reliable impact evaluations, of 
which 41 related to the period 2007-2013 and one to the period 2014-2020. 

2.1.1.  Findings from evaluations on the 2014-2020 period 

So  far,  the  number  of  impact  evaluations  completed  on  ERDF/CF  is  extremely  limited  and 
only the findings of one evaluation can be reported entirely on this period4. Such evaluation 
spans across nine Swedish OPs  and is  focussed on the shift  towards a low-carbon economy 
(TO4).  The  main  findings  show  that  while,  in  overall  terms,  projects  were  in  line  with  the 
intervention  logic,  there  was  limited  coordination  with  relevant  actors,  an  excessive 
fragmentation  of  funding  on  small  projects  and,  in  some  cases,  the  use  of  inadequate 
indicators.  

2.1.2.  Findings from evaluations on the 2007-2013 period 

The 41 reliable impact evaluations of 2007-2013 programmes cover all Thematic Objectives 
except  TO5  “Promoting  climate  change  adaptation,  risk  prevention  and  management”,  TO8 
“Promoting  sustainable  and  quality  employment,  supporting  labour  mobility”  and  TO11 
“Enhancing  institutional  capacity  of  public  authorities  and  stakeholders  and  efficient  public 
administration”. They have been grouped into three main policy areas: 

  R&I, ICT and SME competitiveness:  

o  TO1-strengthening research, technological development and innovation; 
o  TO2-enhancing access to, and use and quality of, ICT; 
o  TO3-enhancing the competitiveness of SMEs. 

  Low  carbon  economy,  climate  change,  environment  and  transport  and  energy 

networks:  

o  TO4-supporting the shift towards a low-carbon economy in all sectors; 
o  TO6-preserving 
efficiency; 

the  environment  and  promoting  resource 

/  protecting 

o  TO7-promoting sustainable transport and removing bottlenecks in key network 

infrastructures. 

  Social inclusion and education: 

o  TO9-promoting social inclusion, combating poverty and any discrimination; 
o  TO10  -investing  in  education,  training  and  vocational  training  for  skills  and 

lifelong learning by developing education and training infrastructure. 

In addition, a quarter of the evaluations were aimed at estimating the effects of the 2007-2013 
programmes on the economy by using econometric models which implicitly cover all TOs but 
which do not  provide any  specific findings in  relation to  these. Such measures are included 
under a separate area ‘All/Multi TOs’. The reliability of the findings in these cases depends 
on  the  validity  of  the  model  used  and  its  ability  to  represent  the  workings  of  the  economy, 
which remains uncertain.  

4   Two evaluations are related to both programming periods 2007-2013 and 2014-2020 (housing renovation in 
Lithuania and transport in Estonia) and their findings are reported in Section 2.1.2 because they mainly refer 
to the earlier period. 

6 

 
 
 
 
                                                 
R&I, ICT and SME competitiveness 

The Polish evaluation on the effects of interventions co-funded by the national OP Innovative 
Economy and aimed at increasing technical  R&D and innovation (TO1) capacity of SMEs 
by  upgrading  research  infrastructure,  found  that  overall  the  support  was  one  of  the  main 
elements which enabled modernisation of research infrastructure on a large scale and helped 
international research cooperation to be established and intensified. Nevertheless, this did not 
translate into a significant increase in the transfer of knowledge within the economy, although 
effects  of  this  kind  are  expected  only  after  some  time.  Equally,  the  maintenance  of  the 
infrastructure created is perceived as being potentially problematic in the long run.  

The Scottish  Co-investment  Fund (SCF) supported by the regional investment bank and the 
OP  ERDF  Lowlands  and  Uplands,  invested  EUR  52  million  in  139  business  between  2009 
and 2013 with the aim of promoting business investment in R&I. The evaluation assessing the 
effects of the investment fund found that it produced significant economic effects at regional 
and national level because of a high level of additionality, a low displacement rate and strong 
export performance.  

Another  UK  evaluation  focused  on  the  support  provided  by  the  Welsh  project  “High 
Performance Computing (HPC)” to strengthen R&I and ICT. It found that the Special Purpose 
Vehicle provided an appropriate system on which to build a long-term sustainable model for 
increasing technological infrastructure, in-house expertise in applications and domain-specific 
support. It was also successful in providing services to businesses, students, researchers and 
academics. However, the project suffered an initial delay which affected the achievement of 
the targets set and arrangements had to be put in place to ensure a smooth transition between 
the first and the second phase.  

Among  evaluations  which  focused  strictly  on  ICT  (TO2),  the  evaluation  of  activities  co-
funded by the ERDF to  develop  this TO in  Poland found that measures  supported provided 
access to the internet for around 9 million people as a result of the creation of over 29 000 km 
of networks, increased ICT skills (90 000 young people enrolling in ICT studies), supported 
social  inclusion  (100 000  household  in  difficult  situations  receiving  basic  ICT  training  and 
8 000  residents  in  Eastern  Poland  taking  part  in  e-inclusion  activities)  and  increased 
commercial opportunities for businesses. From the study, it was evident that it was important 
for these measures to be coordinated at central level and additional support to be provided for 
the  development  of  e-skills  and  e-inclusion  as  well  as  of  entrepreneurship.  Another  Polish 
evaluation  of  the  Podkarpackie  regional  OP  assessed  24  projects  with  a  total  investment  of 
EUR 47 million, aimed at extending broadband and developing a number of ICT products and 
services.  It  found  that  the  given  support  helped  to  create  a  number  of  e-services  and  to 
construct 59 km of broadband network, 206 km of new local-area networks (LANs) and the 
modernisation of 240 km of existing LANs. Around 12 300 households (accounting for 4% of 
the working-age population in the region) were connected to broadband, most of them in rural 
areas.  E-government  and  e-health  services  increased  the  attractiveness  of  the  region  for 
investment and improved the quality and efficiency of healthcare facilities.  

In  the  UK,  the  evaluation  carried  out  on  the  “Next  generation  Broadband  Programme” 
(Superfast Cymru), co-funded by both ERDF OPs in Wales, found that after the completion of 
the project around 520 000 premises/facilities were served by a superfast (24Mbps+) service. 
Take-up amounted to 23% (which was greater than similar interventions in the rest of the UK) 

7 

 
 
 
 
and  beneficiaries  surveyed  reported  a  high  level  of  satisfaction  in  terms  of  time  saved  and 
increased  access  to  educational  opportunities.  In  addition,  the  projects  were  estimated  to 
generate a net present value of EUR 1,3 million and to have a benefit-cost ratio of 6,7. It also 
led to the net creation of an estimated 350 jobs by 2015 and 1 050 by 2024 and to a reduction 
in CO2 equivalent emissions of 33 kilotons/year by 2024. 

As regards SME competiveness (TO3), the evaluation of the projects Customer Engagement 
(CE) and New Business Start-up Support (NBSS) funded by the ERDF OPs in Wales found 
that  the  support  helped  to  create  or  safeguard  jobs,  increase  turnover,  improve  profitability 
and  introduce  new  products  or  services.  Allowing  for  some  deadweight,  displacement  and 
multiplier  effects  which  were  identified,  an  estimated  75  net  jobs  were  attributed  to  the 
support provided.  

A  Polish  evaluation  which  used  a  macroeconomic  model  to  assess  the  effects  of  business-
support measures implemented by the Agency for Enterprise Development and funded by the 
Innovative Economy OP, estimated that the given support added PLN 92 billion (EUR 21,5 
billion) to the GDP in 2007-2017, increased exports by PLN 38 million (EUR 9 million) and 
created 45 000 additional jobs, as well as raising productivity and earnings. 

An evaluation of 45 projects supporting SMEs in the Southern Denmark (Syddanmark) region 
between  2010  and  2012  estimated,  through  counterfactual  techniques,  that  these  projects 
increased employment in the firms concerned by around 1 600 in full-time terms. 

An  evaluation  of  measures  in  Romania  to  support  investment  in  business  infrastructure  of 
regional  and  local  importance  (TO3)  and  the  reconversion  of  old  industrial  sites  (IP6.5) 
concluded that these measures were consistent with the objectives, were effective overall and 
would not have taken place without EU funding. However, delays were experienced because 
of the nature of public procurement regulations, problems relating to property insurance and 
lack of companies specialised in soil decontamination.  

In  the  case  of  evaluations  on  activities  carried  out  by  the  Institute  for  Sustainable  Design 
(ISD),  co-funded  by  the  West  Wales  and  the  Valleys  OP  and  combining  TO1-2-3,  the 
measures were found to have helped develop a design-led cluster in the region, the creation of 
new products and services and new design attitudes and skills.  

In  Estonia,  an  evaluation  of  the  schemes  implemented  by  Enterprise  Estonia  (EAS),  co-
financed  by  the  ERDF,  found  that  these  schemes  helped  to  create  business  start-ups  and  to 
increase  innovation  and  exports.  In  some  areas,  however,  such  as  product  development, 
deadweight was significant, and it was important for the future to combine financial support 
with  mentoring,  training  initiatives  and  guidance  to  enterprises  at  the  application  stage  in 
order to expand the effects of the schemes.  

In Latvia, an evaluation of the impact of  EU support on business development found that  it 
was  insufficient  to  have  a  perceptible  effect  on  regional  development  problems.  Projects 
aimed  at  boosting  cooperation  between  businesses  and  research  centres  did  not  have  a 
significant effect, except on small and recently established enterprises.  

Low carbon economy, climate change, environment and transport end energy networks 

An evaluation in Lithuania of the measures supported for housing renovation over the 2007-
2013  and  the  2014-2020  periods  (though  mainly  in  the  earlier  period)  concluded  that  these 

8 

 
 
 
 
measures helped to increase significantly the number of renovation projects but did not ensure 
that the largest possible energy savings were achieved at the lowest cost. In fact, construction 
costs  increased  to  the  point  that  the  actual  savings  of  heat  energy  during  all  the  loan 
repayment  period  were  lower  than  the  total  costs  of  renovation,  thus  reducing  the 
attractiveness of the investment and lengthening the payback period. To improve this kind of 
support,  the  study  suggested  that  calls  for  proposals  should  be  better  coordinated  and  more 
attention should be given to publicity and consultation.  

In Poland, another evaluation in the same sectoral area, which assessed the contribution of EU 
funding  in  the  2007-2013  period  to  increasing  energy  efficiency  and  the  use  of  renewables 
found  that  the  supported  measures  helped  increase  the  share  of  renewables  in  electricity 
consumption,  but  that  the  impact  on  the  use  of  renewables  in  transport  (e.g.  biofuels)  was 
marginal. However, there were no significant effects on cogeneration or the modernisation of 
heating  and  energy  networks.  Interventions  reduced  the  impact  of  energy  production  on  the 
environment  but  did  not  have  a  significant  effect  on  the  development  of  low-carbon 
technologies.  

In  Italy,  an  evaluation  of  the  impact  of  a  new  tramway  system  in  Florence  concluded  that 
there were gains to the local community who increased their use of public transport. This lead 
to an improvement in environmental quality (and an increase in the value of housing).   

As  regards  environmental  protection  and  preservation  (TO6),  water  supply  projects  in 
Poland led to the construction and/or modernisation of 7 100 km of the supply network and to 
354 100  people  being  connected  to  this,  while  waste  management  projects  led  to  the 
construction,  expansion  and/or  modernisation  of  111  municipal  treatment  plants  and  186 
waste disposal facilities. Projects rehabilitated 38 km of the sea coastline and 339 municipal 
waste landfill sites and overall there was a significant reduction in emissions of air pollutants. 

As  regards  conservation  and  promotion  of  the  natural  and  cultural  heritage  (IP6.3),  an 
evaluation  in  the  Podlaskie  region  in  Poland  found  that  projects  in  this  sector  helped  to 
increase the attractiveness of the region for tourists only slightly, largely because of the wide 
dispersion of projects, poor transport connections and a lack of a tourist management system 
at local level, as well as a lack of cooperation between stakeholders. 

In  respect  of  transport  (TO7),  an  evaluation  of  projects  implemented  in  Estonia  over  the 
2007-2013  and  2014-2020  periods  (though  it  largely  covered  projects  undertaken  in  the 
earlier period, as relatively few projects in the later period had been completed) indicates that 
construction prices fell following the crisis and many projects were completed at lower cost 
than  originally  planned.  The  quality  of  rail  transport  improved  significantly  but  overall  the 
development  of  the  transport  infrastructure  was  not  found  to  be  a  driver  of  local  business 
activity.  

In the UK, an evaluation of two rail projects implemented in Wales and funded by the ERDF 
West Wales and the Valleys OP (the intermodal facility at Energlyn and a passing loop at Tir-
Phil  station)  found  a  reduction  in  car  use,  while  the  number  of  passenger  rail  journeys 
increased  in  line  with,  or  above,  forecasts,  though  the  increase  in  passenger  kilometres 
travelled was below expectations.  

In Polska Wschodnia (Eastern Poland), an evaluation of EU-funded transport projects found 
that most projects achieved or exceeded their targets in terms of reduction in travel time for 

9 

 
 
 
 
both  passengers  and  freight.  Effects  on  the  economic  development  of  the  region,  however, 
were not yet evident at the time the evaluation was carried out.  

Another  Polish  evaluation  of  transport  projects  co-funded  by  the  Infrastructure  and 
Environment  OP  concluded that the road network across the  country had been substantially 
developed, but the infrastructure at sea ports was still inferior to that in other Member States.   

In  Latvia,  an  evaluation  of  EU  funding  of  infrastructure  concluded  that  it  had  contributed 
significantly to the overall improvement in the of transport network. In particular, the railways 
had  been  modernised,  resulting  in  a  high-quality,  competitive  system  for  both  freight  and 
passenger transport and the conditions and safety of roads had been improved as had ports in 
the country as well as Riga airport. 

Social inclusion and education 

This sector includes thematic objectives that are only marginally funded by the ERDF. 

As  regards  evaluations  on  social  inclusion  (TO9),  an  evaluation  in  the  Lubuskie  region  in 
Poland  assessed  the  effects  of  urban  regeneration  projects  and  their  complementarity  with 
similar  projects  funded  in  2014-2020,  finding  that  supported  projects  involved  culture, 
tourism  and  roads  and  to  a  lesser  extent  to  social  infrastructure  and  increased  the 
attractiveness  of  the  areas  concerned  as  places  to  live  and  work.  Few  projects  would  have 
been implemented without support or would have been on a smaller scale. Projects  planned 
for the 2014-2020 period are in line with the main socio-economic and environmental needs 
of the region. 

As regards education and training (TO10), a counterfactual analysis in Romania of support 
for  investment  in  education  infrastructure  found  that  the  consequent  improvement  in  the 
quality of school buildings and equipment increased the standard of education (more in poorer 
localities than richer ones) and increased access for those with disabilities.  

All/Multiple Thematic Objectives 

As  some  evaluations  focused  on  more  than  one  TO  or  all  TOs,  their  results  are  presented 
separately.  

found 

to  have 

In Poland, an evaluation of the impact  of measures supported by the regional  Wielkopolska 
OP  on employment, which partly involved the use of financial instruments,  found that such 
measures helped to increase the number of permanent jobs created. Overall, the recipients of 
support were more resilient to the economic crisis and other market shocks. In the Pomorskie 
region, the support provided by the regional OP was estimated to have increased employment, 
GDP  growth,  SME  development  and  innovation.  In  the  Warmia  i  Mazury  region,  measures 
regional  socio-economic 
increased  business  competitiveness, 
were 
development  and  employment,  although  the  region  remained  less  developed  than  others  in 
Poland.  In  Kujawsko-Pomorskie,  Cohesion  Policy  support  is  estimated  to  have  increased 
employment,  income  and  living  standards,  as  well  as  to  reduce  disparities  in  development 
within the region.  In Podlaskie, it is  estimated that  Cohesion  Policy support increased GDP 
growth and the regional employment rate between 2004 and 2015 as well as raising household 
income  significantly.  An  evaluation  of  equal  opportunity  and  non-discrimination  measures 
supported by the Podlaskie regional OP found that these measures led to more equal access to 
the labour market and social services. They also helped to reduce intra-regional inequalities in 

10 

 
 
 
 
access  to  tourist,  sports  and  cultural  facilities  as  well  as  to  good  quality  education  and 
healthcare services in both urban and rural areas. 

A horizontal evaluation on the impact of EU funds on poverty and social exclusion found that 
overall  there  was  a  general  reduction  of  people  using  community  social  aid,  the  scale  of 
relative  and  absolute  poverty,  the  proportion  of  young  people  not  continuing  education 
beyond compulsory schooling, the proportion of NEETs among young people and the scale of 
severe material deprivation. 

In  Italy,  an  evaluation  of  measures  supported  by  the  Lombardia  regional  OP,  using  a 
counterfactual approach, found that these measures increased RDI expenditure and enterprise 
competitiveness,  as  well as the business  survival rate and  employment. They  also  improved 
energy efficiency and increased the share of renewables in energy consumption.  

An  evaluation  of  urban  development  measures  co-funded  by  the  Regional  OP  in  Romania 
concluded that these measures led to an improvement in the quality of life and an increase in 
job  creation,  a  reduction  of  unemployment  and  an  increase  in  the  attractiveness  of  cities  as 
well as high school enrolment rates.  

In Hungary, an evaluation of integrated urban development policies in the regional OPs found 
that cities started to use more integrated approaches to development but there remains a lack 
of strategy and limited integration with other policies and involvement of stakeholders.  

Evaluations using macroeconomic models to assess the effects of ESI Fund support, estimated 
that in Hungary, this support helped to improve education, the healthcare system and human 
capital,  to  increase  the  efficiency  of  public  administration,  to  develop  transport  and 
environment infrastructure and to stimulate RDI. Also in Hungary, investment in construction 
and  community  services  were  estimated  to  have  contributed  most  to  GDP  growth  and 
employment,  particularly  of  less-skilled  workers,  though  it  was  also  estimated  that  the 
additional  demand  generated  led  to  increased  costs  of  production,  thus  compromising  the 
competitiveness of domestic producers and leading to a decline in exports and an increase in 
imports.  

In  Lithuania,  the  projects  supported  over  the  period  2007-2020  are  estimated  to  have 
increased GDP by twice the amount of the expenditure involved.  

In Slovakia, the funding provided is estimated to have increased economic development and 
convergence and to have helped alleviate the effects of the crisis. In the Visegrad countries, it 
is estimated that there were significant spill-over effects on entrepreneurship and innovation, 
transport infrastructure, research, energy and environmental protection. 

The  main  findings  of  the  evaluations  undertaken  by  Member  States  (on  ERDF/CF  and 
ESF/YEI),  including  a  description  of  the  data  used  and  the  adopted  methodologies  are 
available in the Evaluation Library published on the Inforegio website.5 

5   The Evaluation Library is available here: http://ec.europa.eu/regional_policy/en/policy/evaluations/member-

states/  

11 

 
 
 
 
                                                 
2.2. Synthesis of ERDF/CF evaluation work by the Commission 

The ERDF and Cohesion Fund 2007-2013 ex post evaluation was finalised in late 2016 and 
was extensively presented6 at the time, and included a major communication campaign on the 
results of the Cohesion Policy on the ground. 

In 2017 the Commission launched an evaluation to complement previous ex-post evaluations. 
The objective of the latest evaluation of major projects, supported by ERDF and the Cohesion 
Fund between 2000 and 2013, is to analyse the long-term contribution (direct and indirect, as 
expected  and  unexpected)  of  major  investment  projects  on  economic  and  environmental 
development, quality of life and well-being of citizens. The first study under the evaluation, 
finalised  in  June  2018,  focused  on  10 ERDF  or  Cohesion  Fund  transport  projects.  A  Staff 
Working Document based on the findings of the study will be available in the first semester of 
2019.  

The  findings  show  that  major  transport  projects  can  positively  contribute  to  increasing  the 
transport  efficiency  within  EU  regions,  supporting  the  shift  to  cleaner  urban  mobility  and 
facilitating  cross-border  transport  connections.  Two  key  success  factors  were  identified:  a 
sound  forecasting  capacity  driving  the  project  design  and  selection  process  and  excellent 
managerial  capacity  to  keep  the  project  on  track.  There  is  also  evidence  that  major 
investments in the transport sector would benefit from strategic guidance by the Commission 
on the prioritisation and selection process towards projects that maximise EU objectives and 
are implemented according to well defined quality standards (e.g. environmental measures).  

This evaluation is now continuing to cover major projects in the policy area of environment.  

An  ex  post  evaluation  on  investment  projects  in  Research  and  Technological  Development 
(RTD)  infrastructures,  competence  centres  and  activities  financed  by  the  ERDF  during  the 
period 2007-2013 is currently under design. It follows and complements the work packages of 
the ex post evaluation of the period 2007-2013 and is aimed to obtain a deep understanding of 
the rationale for and effectiveness of ERDF support for research infrastructure and activities 
and  to  identify  factors  contributing  to  the  success  or  failure  of  these  investments  under 
different socio-economic conditions. 

During  the  2014-2020  programming  period,  the  main  evaluation  work  undertaken  by  the 
Commission  has  been  related  to  the  accumulation  of  evidence  from  Member  States' 
evaluations.    To  this  end,  the  Commission  has  launched  an  "Evaluation  Helpdesk"  contract 
(covering  ERDF/CF  and  ESF/YEI)  to  identify  and  synthesise,  among  other  tasks,  the 
evaluations  carried  out  by  Member  States.  Such  evaluations  are  collected  in  the  above-
mentioned Evaluation Library that includes also a summary of the main findings for 75% of 
them (100% of those identified in the last year).  

To provide support to the Member States, the Commission, through the Evaluation Helpdesk, 
organises evaluation training for the Member States through "summer schools" for managing 
authorities  and  others  involved  in  the  evaluation  of  programmes.  It  also  organises  peer 

6   The  summary  and  full  details  of  the  ERDF/CF  2007-2013  ex  post  evaluation  package  is  available  here: 

http://ec.europa.eu/regional_policy/en/policy/evaluations/ec/2007-2013/  

12 

 
 
 
 
                                                 
reviews  of  selected  evaluations  (final  reports  or  whole  evaluation  processes)  in  order  to 
discuss the existing strengths and weaknesses in the practice of evaluation of cohesion policy 
and to  help  Managing Authorities in  their work of designing and conducting evaluations. A 
targeted  methodological  support  on  issues  concerning  the  planning  and  implementation  of 
evaluations  was  also  provided  to  selected  Managing  Authorities  in  Latvia  (2016),  Greece 
(2017), and Croatia (2018). 

3.  ESF AND YEI 

By August 2017, evaluation plans have been submitted for all OPs. Nevertheless, evaluation 
planning has continued with a view to further specifying the already planned evaluations or 
catering for additional evidence needs, e.g. due to OP modifications. Since the last quarter of 
2017 until the last quarter of 2018, 24 ESF evaluation plans have been received, including 21 
revised substantially7 and including 22 multi-fund plans. 

With the  completion  of the  initial  phase  of  evaluation  planning,  the  focus  is  moving  on  the 
planning  of  impact  evaluations,  i.e.  the  evaluation  work  that  contributes  most  in  the 
assessment of the fund's achievements. The impact evaluations of the latest evaluation plans 
have been assessed according to five standard quality criteria8. The pattern that emerges from 
these  assessments  is  similar  to  the  main  findings  about  evaluation  planning  in  general. 
Namely, the planning of the impact evaluations is relatively strong in organisational aspects, 
such  as  budgeting  and  timing.  They  also  score  relatively  well  in  terms  of  defining  the 
objectives  of  the  evaluation  and  the  evaluation  questions.  However,  the  specification  of  the 
evaluation  design  and  methods  as  well  as  of  data  sources  is  less  articulated.  The  main 
shortcomings in respect of the former are the lack of detail on the approach to be used, such as 
theory-based, counterfactual or cost-benefit analysis, and on the rationale for the adoption of 
the  evaluation  approach.  In  a  number  of  cases,  there  are  also  question  marks  over  the 
appropriateness of the approach indicated for answering the evaluation questions.  

For the data and sources, the main shortcomings are the lack of detail about data sources to be 
used, the specific data to be taken from these, the quality of the data available and whether or 
not there is a need to take action in respect of missing data. Where a counterfactual approach 
is suggested, in the majority of cases, insufficient consideration is given to the data required to 
measure  the  performance  of  the  control  group.  As  Table  2  in  Section  2.1  shows  above,  the 
weakness  in  the  identification  of  data  sources  is  particularly  challenging  in  case  of  ESF 
planned evaluations. 

Following  the  plans,  the  quality  of  the  published  evaluations  is  assessed  through  criteria9 
similar to those used in analysing the information about planned evaluations. That assessment 

7  Less significant, on-going adjustments in the evaluation plans are not considered as reviewed plans. 
8   The criteria have been a) the clarity and pertinence of general background information, b) the objective of the 
evaluation and the evaluation questions, c) the evaluation design and methods, d) the data and data sources, 
e) the coherence of the budget indicated and the time allowed for the evaluation with the objective and scope. 

9   a)  The  clarity  and  suitability  of  the  evaluation  design,  b)  the  appropriateness  and  correctness  of  the 
techniques, c) data quality and availability, d) the validity of findings and e) the validity of conclusions. 

13 

 
 
 
 
                                                 
confirms  to  what  extent  the  shortcomings  in  evaluation  planning  translate  to  weaknesses  in 
the evaluations themselves. The results show that especially the validity  of the findings and 
the  appropriateness  of  evaluation  design  are  weak,  both  of  which  are  closely  related  to 
problems in the planning stage (see Table 3). 

These problems compromise the reliability of the findings and thus the use that can be made 
from evaluations. So far, altogether 22 ESF evaluations for the 2007-2013 and 7 for the 2014-
2020 period have been regarded reliable, i.e. rated above an average score of 3 according to 
the five quality criteria. 

Concerning  methodological  choices  and  the  quality  of  the  evaluations,  direct  feedback  is 
provided to the Managing Authorities in the evaluation peer reviews of selected evaluations 
with  the  support  of  the  Evaluation  Helpdesk.  The  reviews  appraise  the  evaluation  dossier 
from  the  terms  of  reference  to  the  final  report  with  a  particular  view  on  methodological 
choices. The main general findings of the reviews will be summarised in a guidance. 

Moreover, the Commission continues promoting the use of counterfactual impact evaluations 
by  Member  States,  notably  through  periodic  evaluation  partnership  and  Community  of 
Practice  meetings  and  the  data  fitness  initiative  carried  out  jointly  with  JRC  since  2017. 
Concerning better access to data sources, DG Employment is carrying out a feasibility study: 
the aim is to identify ways of accessing existing data sources, administrative registers, thereby 
enabling  the  execution  of  counterfactual  impact  evaluations.  With  a  view  to  the  future,  the 
Commission's  draft  proposal  for  the  ESF+  Regulation  enables  the  access  by  Managing 
Authorities to data registers for monitoring and evaluation purposes. 

3.1. Synthesis of ESF/YEI evaluations completed by the Member States 

As shown last year in the Staff Working Document accompanying the 2017 Strategic Report 
on the implementation of the ESIF (SWD(2017) 452), the number of evaluations planned to 
be  completed  is  showing  a  sharp  increase  compared  to  2016-2017.  The  same  applies  to  the 
specific  group  of  impact  evaluations.  The  data  about  the  period  between  August  2017  and 
August 2018 confirm that expectation.  

14 

 
 
 
 
Table 4: Evaluations published since January 2015 – Breakdown by programming period, type, year of publication and Fund 

2015 

ESF-
YEI 

multi-
Fund 

% 

2016 

ESF-
YEI 

multi-
Fund 

% 

2017 

ESF-
YEI 

2018 (1st semester) 

Cumulative 

multi-
Fund 

% 

ESF-
YEI 

multi-
Fund 

% 

ESF-
YEI 

multi-
Fund 

total  % 

2007-2013 

Impact 

Impact and process/implementation 
and/or monitoring/progress 

18 

22 

9 

2 

44%  7 

39%  9 

7 

7 

33%  1 

37%  0 

10 

11 

50% 

50% 

0 

0 

Total 

40 

11 

84%  16 

14 

70%  1 

21 

100%  0 

100%  26 

0% 

31 

28 

20 

54 

51 

42% 

40% 

100%  57 

48 

105 

82% 

Process/implementation 

Monitoring/progress 

Process/implementation and 
monitoring/progress 

Total 

Total 2007-2013 

2014-2020 

Impact 

Impact and process/implementation 
and/or monitoring/progress 

Total 

Process/implementation 

Monitoring/progress 

Process/implementation and 
monitoring/progress 

Total 

Total 2014-2020 

2007-2013 & 2014-2020 

3 

3 

2 

8 

48 

0 

4 

4 

2 

0 

3 

5 

9 

2 

0 

0 

2 

13 

0 

1 

1 

13 

3 

1 

17 

18 

8% 

5% 

3% 

1 

1 

3 

16%  5 

100 

21 

0% 

0 

19%  4 

19%  4 

56%  10 

11%  2 

15%  8 

81%  20 

100 

24 

4 

1 

3 

8 

22 

1 

0 

1 

10 

1 

3 

14 

15 

0 

0 

0 

0 

21 

2 

2 

4 

27 

6 

5 

38 

42 

12%  0 

5% 

0 

14%  0 

30%  0 

100 

1 

3% 

4 

10%  3 

13%  7 

51%  13 

8% 

3 

28%  10 

87%  26 

100 

33 

15 

2 

0 

2 

0 

0 

0 

0 

2 

0 

0 

0 

9 

1 

2 

0% 

0% 

0% 

0% 

100 

8% 

7% 

15% 

53% 

12% 

20% 

0 

0 

0 

0 

0 

1 

2 

3 

1 

1 

6 

0% 

0% 

0% 

0% 

100 

4% 

9% 

13% 

43% 

9% 

4 

4 

5 

13 

70 

5 

13 

18 

26 

6 

35% 

27 

6 

1 

3 

10 

58 

3 

3 

6 

59 

11 

11 

81 

87 

10 

5 

8 

8% 

4% 

6% 

23 

18% 

128 

100 

8 

16 

24 

85 

17 

38 

5% 

10% 

15% 

52% 

10% 

23% 

140 

85% 

164 

100 

85% 

8 

100 

11 

12 

12 

87% 

100 

59 

77 

 
 
  
  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Impact 

Impact and process/implementation 
and/or monitoring/progress 

Total 

Process/implementation 

Monitoring/progress 

Process/implementation and 
monitoring/progress 

Total 

Total both periods 

Overall 

Impact 

Impact and process/implementation 
and/or monitoring/progress 

Total 

Process/implementation 

Monitoring/progress 

Process/implementation and 
monitoring/progress 

Total 

Overall total 

0 

0 

0 

0 

0 

0 

0 

0 

18 

26 

44 

5 

3 

5 

13 

57 

0 

0 

0 

0 

0 

0 

0 

0 

9 

3 

12 

15 

3 

1 

19 

31 

0 

0 

0 

0 

0 

0 

0 

0 

31%  7 

33%  13 

64%  20 

23%  11 

7% 

7% 

3 

11 

36%  25 

100 

45 

0 

0 

0 

0 

0 

0 

0 

0 

8 

7 

15 

14 

2 

6 

22 

37 

2 

1 

3 

0 

0 

0 

0 

3 

14 

14 

28 

27 

6 

5 

38 

66 

75% 

25% 

0 

1 

100%  1 

0% 

0% 

0% 

0% 

100 

20% 

17% 

37% 

40% 

9% 

15% 

0 

0 

0 

0 

1 

1 

3 

4 

1 

1 

6 

63% 

8 

100 

12 

0 

0 

0 

1 

0 

0 

1 

1 

2 

0 

2 

10 

1 

2 

13 

15 

0% 

50% 

50% 

50% 

0% 

0% 

50% 

100 

11% 

11% 

22% 

41% 

7% 

30% 

1 

1 

2 

0 

0 

0 

0 

2 

32 

45 

77 

30 

10 

32 

2 

1 

3 

1 

0 

0 

1 

4 

33 

24 

57 

66 

12 

14 

3 

2 

5 

1 

0 

0 

1 

6 

50% 

33% 

83% 

17% 

0% 

0% 

17% 

100 

65 

69 

22% 

23% 

134 

45% 

96 

22 

46 

32% 

7% 

15% 

78% 

72 

92 

164 

55% 

100 

149 

149 

298 

100 

1 

0 

1 

0 

0 

0 

0 

1 

18%  6 

24%  3 

43%  9 

30%  13 

6% 

3 

21%  10 

57%  26 

100 

35 

16 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
evaluations 

Unsurprisingly,  most 
are 
relating 
process/implementation-oriented,  aimed  at  assessing  the  way  the  programmes  are  operated 
and implemented. Of the 164 ESF (and multi-fund) evaluations identified for the 2014-2020 
period, 52% are ‘pure’ process/implementation-oriented evaluations, 23% are also concerned 
with  assessing  progress  in  achieving  targets  and  10%  with  assessing  first  results  and 
outcomes. 

2014-2020 

period 

the 

to 

Most  evaluations  identified  apply  a  combination  of  relatively  simple  qualitative  and 
quantitative methods. Of the 298 ESF (and multi-fund) evaluations identified 227 combine at 
least two methods. Among impact-oriented evaluations, 32 use counterfactual evaluation and 
48 use theory-based evaluation techniques (alone or combined with other techniques).  

It  is  evident  that  the  share  of  evaluations  using  counterfactual  techniques  is  larger  for  ESF 
OPs and OPs using mixed funding than for ERDF OPs.  

Contrary to 2017 when the findings of all completed evaluations were synthesised, this year's 
synthesis uses findings only from evaluations deemed reliable10. This selection has resulted in 
22 ESF evaluations for the 2007-2013 and 7 for the 2014-2020 programming period. 

Table 5 - Impact evaluation identified, reviewed and considered reliable 

2007-2013 

2007-2013 & 2014-2020 

ESF-
YEI 
57 

38 

12 

multi-
Fund 
48 

41 

10 

Total 

105 

79 

22 

ESF-
YEI 
2 

2 

0 

multi-
Fund 
3 

3 

3 

Total 

5 

5 

3 

ESF-
YEI 
18 

18 

5 

2014-2020 
multi-
Fund 
6 

6 

2 

Overall 

Total  Total 

24 

24 

7 

134 

108 

32 

32% 

24% 

28% 

0% 

100% 

60% 

28% 

33% 

29% 

30% 

Identified 

Reviewed 
Considered 
reliable 
Reliable  as 
% 
of 
reviewed 

3.1.1.  Findings from evaluations on the 2014-2020 period 

Six of those seven evaluations concerned mainly employment related findings and one looked 
at results in institutional capacity building. 

 An evaluation of the effects  of  Le Chéile’s mentoring services for  young people in  Ireland 
found that the model put  in  place was robust and worked efficiently  with  positive results, a 
social return of 4 times the cost of the project being estimated from cost-benefit analysis. 

In Germany, an evaluation of the progress achieved and effects of 7 ESF-supported projects in 
the city of Hamburg to increase the employability of disadvantaged people and to help women 
into employment found that a close relationship between coaches and participants was created 
and  the  capabilities  of  participants  improved.  The  main  reasons  for  success  lay  in  the 

10   The criteria have been the same as those listed in Footnote 11. The evaluation has been considered reliable in 
case  the  overall  score  of  the  five  criteria  has  been  above  3  in  a  range  between  1  ('weak/poor')  to  4 
('strong/good'). 

17 

 
 
 
  
  
 
                                                 
adaptable  and  integrated  approach  adopted,  the  well-developed  network  of  coaches  and  the 
relatively long duration of projects.  

In Italy, an evaluation of the “Employment and Labour Policies Integrated Plan” (PIPOL), co-
funded  by  the  Friuli  Venezia  Giulia  regional  OP  concluded  that  participants  had  a  higher 
probability  of  finding  a  job  than  non-participants  and  that  overall,  traineeships  were  more 
effective  than  training  programmes.  The  measures  were  also  more  effective  for  women, 
young people, the low-educated and people from abroad.  

Another  Italian  evaluation  of  measures  co-funded  by  the  Piemonte  regional  OP  found  that 
31% of participants   were employed one  year  after the completion of training, an  estimated 
difference  of around 12-20 percentage points percentage as compared with non-participants. 

In  Estonia,  the  "My  First  Job"  initiative  to  support  the  (re)entry  of  young  people  into  the 
labour  market  through  wage  and  training  subsidies  and  training  was  found  to  have  had  a 
significant  effect  in  reducing  youth  unemployment,  though  there  were  large  numbers  of 
dropouts and relatively few who continued working for their original employer over the long-
term. It was recommended awareness-raising campaigns among young people and employers 
should  be  carried  out  and  the  selection  criteria  for  participation  broadened  to  include 
unemployed parents, in particular.  

An evaluation of the Curriculum and Career Guidance Programme, also in Estonia, aimed at 
implementing  the  Estonian  Lifelong  Learning  Strategy  2020,  indicated  that,  while  the 
operations  supported  were  in  line  with  the  objectives,  more  attention  needed  to  be  paid  to 
children with special needs. It also indicated that the sustainability of the programme was not 
guaranteed and effective financial plans needed, therefore, to be formulated. 

An evaluation of four specific activities under the ESF Knowledge, Education, Growth OP in 
Poland,  aimed  at  strengthening  institutional  capacity  and  efficient  public  administration 
(TO11),  found  that  they  did  not  appear  to  have  a  significant  effect  on  international  good 
governance indicators. 

3.1.2.  Findings from evaluations on the 2007-2013 period 

Employment 

In  this  theme,  a  number  of  evaluations  reviewed  carried  out  counterfactual  analysis.  In  the 
UK, ESF supported measures by both ESF OPs in Wales were considered to have increased 
access  to  employment  and raised skill  levels  of those in  work (both in the convergence and 
competitiveness  regions).  The  training  provided  had  a  positive  impact,  increasing  the 
likelihood of participants  (both unemployed and inactive) finding a job  afterwards  (by 26% 
for  the  unemployed  and  50%  for  the  inactive  as  compared  with  non-participants).  Positive 
effects  were  also  estimated  for  those  participating  in  training  to  increase  their  skills  (7%  of 
whom indicated that improvements in their employment situation were directly attributable to 
the  training).  Another  Welsh  evaluation  was  based  on  five  surveys  carried  out  on  those 
completing  both  ESF  OPs,  in  particular  those  participating  in  training  aimed  at  increasing 

18 

 
 
 
skill levels and employability. Results show that the people concerned acquired a wide range 
of skills. Half of the participants had a positive transition from unemployment to paid work. 

The evaluation of active labour market measures supported in Croatia between 2010 and 2013 
found  that  participants  in  vocational  training  increased  their  medium-term  employability  by 
40% as compared with unemployed non-participants. Measures were particularly positive for 
those  covered  by  the  Employment  Promotion  Act.  Positive  effects  were  also  visible  for  the 
long-term  unemployed  as  a  result  of  public  works  measures  and  labour  cost  subsidies  for 
small employers. Nevertheless, a sixth of the vocational training interventions did not provide 
participants with adequate work experience. 

In  Ireland,  the  evaluation  of  the  Job  Bridge  Activation  Programme  co-funded  by  the  ESF 
estimated that the support had significantly increased the likelihood of participants finding a 
job and more than other labour market activation programmes.  

In  Poland,  the  evaluation  of  the  measures  aimed  at  older  workers  and  elderly  in  the  Lublin 
region found that interventions had a positive effect on their employment though there was a 
relatively  large  deadweight.  Another  Polish  evaluation  of  activities  designed  to  increase  the 
employability  of  young  people  in  the  Wielkopolskie  region  found  that  it  improved  the  job 
search  capability  of  participants  and  increased  their  motivation  to  look  for  a  job.  From  the 
social  point of view, participation in  projects  helped beneficiaries to  gain independence and 
be more receptive to others.  

The  evaluation  of  the  Employment,  Human  Capital  and  Social  Cohesion  OP  2007-2013  in 
Cyprus  found  that  in  general  the  OP  was  effective  in  achieving  its  objectives.  Advisory 
guidance to enterprises and training of women were more efficient - in terms of unit costs - in 
2007-2013 than the previous period, but projects for integrating participants into employment 
and getting socially vulnerable groups into work were less efficient.  

Social inclusion 

In  this  field,  ESF-supported  measures  in  the  UK  were  found  to  have  been  successful  in 
reducing  social  isolation  and  increasing  work-related  activity  of  disadvantaged  families. 
Positive  factors  were  the  ongoing  support  to  participants  and  the  focus  on  a  broad  range  of 
limitations which disadvantaged families experienced. 

In  Lithuania,  the  evaluation  of  Cohesion  policy  found  that  the  Structural  Funds  helped  to 
improve  the  quality  of  healthcare  services  across  the  country,  as  evidenced  by  a  range  of 
indicators,  such  as  the  mortality  rate,  the  suicide  rate,  infant  mortality,  early  diagnosis  and 
oncological  treatment.  The  healthcare  system,  however,  still  needed  to  be  improved,  (as 
regards,  for  example,  training  of  staff,    e-health  systems  and  refurbishment  of  buildings)  in 
order to ensure future sustainability.  

Evaluation  of  the  economic  and  social  urban  regeneration  measured  undertaken  by  the 
regional  Lubuskie  OP  in  Poland  found  that  few  projects  would  have  been  implemented 
without support or would have been on a smaller scale. Measures were  mainly implemented 
in major urban centres in the South West of the province of Lublin and involved support for 

19 

 
 
 
culture,  tourism  and  road  improvement  and,  to  a  lesser  extent,  for  social  infrastructure. 
Overall, support helped to increase the attractiveness of the areas concerned as places to live 
and work. 

Education and training 

In the area of education and training, the need for ICT improvements was identified in many 
regions.  The  evaluation  of  training  activities  in  universities,  co-funded  by  the  ESF  Human 
Capital OP in Poland, found an overall high level of satisfaction among students, though also 
more  difficulty  for  small  firms  than  for  larger  ones  in  establishing  cooperation  with 
universities.  

Evaluation  of  the  effectiveness,  coherence  and  relevance  of  interventions  co-funded  by  the 
federal ESF OP in Germany highlighted the overall coherence of the measures supported with 
other  regional  labour  market  policies  and  with  specific  local  needs.  Regions  with  high 
unemployment,  therefore,  focused  mainly  on  employability  and  transition  from  school  to 
work, while those in which it was low, focused mainly on lifelong learning and disadvantaged 
groups.  The  decentralised  management  and  control  model  used  was  successful  in  enabling 
active  participation  of  employment  agencies,  job  centres  and  municipal  offices  providing 
social services and in ensuring effective coordination with federal authorities.  

Another German evaluation carried out  a  counterfactual analysis of the impact  of a specific 
action  funded  by  the  same  federal  ESF  OP,  Civil  Work  (Bürgerarbeit).  It  found  that  the 
activation phase increased access to employment, participants having an integration rate 35% 
higher than non-participants 6 months after the beginning of the programme, though after two 
years,  there  was  no  difference.  In  the  employment  phase,  however,  participation  in  the 
programme was estimated to have a negative effect, in that the integration rate was only half 
that of non-participants, which is partly explained by a locked-in effect, those participating in 
an employment programme not being able to look for a job.  

A Latvian evaluation of measures implemented under the Human Resource Development OP 
to promote equal opportunities and combat discrimination found that the inclusion of gender 
equality as a requirement in calls for tender is not sufficient in itself and that there was a need 
for a proper assessment of compliance during project selection.   

3.2.  Synthesis of ESF evaluation work by the Commission  

3.2.1.  Update of the 2007-2013 ex-post data  

The implementation of the programmes was not completed at the time of the Commission ex-
post  evaluation,  and  relied  on  the  2014  Annual  Implementation  Report  data  and  national 
evaluations from the Member States conducted until 30 June 2015. As a result, the ESF 2007-
2013 ex-post evaluation did not present the full picture of the ESF during that programming 
period.  

Therefore the Commission carried out  a study supporting the update of  the data reported in 
the 2007-2013 ESF ex-post evaluation based on the Final Implementation Reports which were 
submitted to the Commission in the course of 2017 (without reviewing evaluation findings). 

20 

 
 
 
The study will very soon be complemented by fact sheets for the entire EU as well as each of 
the 28 Member States to be used for dissemination to the public. 

3.2.2.  Launch of two thematic evaluations  

The Commission has recently launched two thematic evaluations; the first one focuses on the 
ESF and YEI support to youth employment and the second one covers Thematic Objective 8 
with  the  exception  of  support  to  youth  employment  and  mobility.  Both  of  them  will  be 
completed in 2019. They will be followed by two evaluations on Thematic Objectives 9 and 
10. 

These  studies  will  provide  the  Commission  with  evidence  and  analysis  that  will  allow  it 
building up its evaluation of the ESF support to employment and mobility. The results of the 
studies  will  be  integrated  into  the  staff-working  document  produced  by  the  Commission 
annually.  

On the one hand, these thematic evaluations aim at taking stock of the results of ESF and YEI 
related  support  until  now,  building  in  particular  on  the  relevant  evaluations  carried  out  by 
Member  States.  On  the  other  hand,  the  results  of  the  evaluations  should  feed  into  the  next 
programming  period,  by  providing  lessons,  notably  as  regards  cost-effectiveness,  outreach 
and target groups most in need, thus supporting the negotiation of the operational programmes 
for  the  ESF+.  It  will  also  pave  the  way  for  the  ex-post  evaluation  of  the  ESF  by  the 
Commission to be completed by December 2024. 

4.  EAFRD - EUROPEAN AGRICULTURAL FUND FOR RURAL DEVELOPMENT 

4.1. Synthesis of evaluation work by the Member States 

4.1.1.  Period 2014-2020  

According  to  the  information  submitted  by  the  Member  States  in  June  2018  in  115  Annual 
Implementation Reports (AIRs), the number of completed evaluations (192) has increased 
by 28% compared to 2017 and the  overall majority (89%) was published online. More 
than  half  of  the  completed  evaluations  (66%)  were  linked  to  the  major  EU  evaluation 
milestones  and  requirements.  Furthermore,  self-standing  evaluations  on  other  specific  Rural 
Development Programme (RDP) aspects, such as delivery mechanisms, financial instruments, 
and  technical  assistance  were  reported,  as  well  as  evaluations  on  other  rural  issues  such  as 
rural tourism and biodiversity.  

The  highest  share  of  completed  evaluations  covered  several  of  the  six  RDP  priorities 
(40%),  followed  by  evaluations  covering  only  RDP  Priority  4  ‘Restoring,  preserving  and 
enhancing ecosystems’ (14%). Relatively few evaluations have been completed in relation to 
Priority  1  ‘Knowledge 
innovation‘,  Priority  2  ‘Farm  viability  and 
competitiveness’,  Priority  3  ‘Food  chain  organisation  and  risk  management’,  Priority  5 
‘Resource-efficient, climate-resilient economy’ and Priority 6 ‘Social inclusion and economic 
development’.  Completed  evaluations  were  reported  also  on  RDP  cross-cutting  objectives, 

transfer  and 

21 

 
 
 
 
such as innovation (e.g. BE Flanders), environment (e.g. ES Castilla-Mancha, FI Mainland), 
and climate change (e.g. UK England).  

The extent of reporting on evaluation findings has overall increased:  79 AIRs contained 
summaries  of  completed  evaluations  (compared  to  65  in  2017).  Within  these  summaries, 
471 distinct  evaluation  findings  could  be  identified  covering  different  topics,  as  shown  in 
Figure 2. 

Figure 2: Reported evaluation findings across topics (N=471) 

Source: European Evaluation Helpdesk for Rural Development (2018) 

Following the same trend observed in the completed evaluations above, 67% of the evaluation 
findings was linked to one or more RDP priorities and focus areas. Among these evaluation 
findings,  the  highest  share  concerned  the  RDP  Priority  4  ‘Restoring,  preserving  and 
enhancing  ecosystems’  (21%),  followed  by  Priority  2  ‘Farm  viability  and  competitiveness’ 
(11%), Priority 6 ‘Social inclusion and economic development’ (10%), Priority 1 ‘Knowledge 
transfer  and  innovation‘  (7%),  Priority  5  ‘Resource-efficient,  climate-resilient  economy’ 
(6%), and Priority 3 ‘Food chain organisation and risk management’ (4%). Moreover, 14% 
of the evaluation  findings  was connected to  other specific thematic areas, such as European 
Innovation Partnerships on Agricultural Productivity and Sustainability (EIP-AGRI), National 
Rural  Networks,  innovation,  or  RDP  synergies  with  other  ESI  Funds.  To  a  smaller  extent, 
evaluation  findings  also  covered the RDP  management and delivery mechanism (8%), RDP 
monitoring and evaluation system (6%), and other topics (5%), such as ex post evaluation of 
RDPs 2007-2013.  

The analysis of the evaluation findings related to the six RDP priorities grouped according to 
the three CAP Objectives can be summarised as follows:  

22 

 
 
 
 
 
  The evaluation findings linked to CAP Objective 1 of fostering the competitiveness 
(15% of the total findings), covered both the implementation and the achievements of 
RDP  Priorities  2  and  3.  Member  States  were  able  to  capture  the  trends  in  farm 
productivity and modernisation in supported farms (e.g. ES Navarra, FR Bourgogne), 
the changes in the farm net value added (e.g. AT, BG, ES Galicia), the contributions to 
the  generational  renewal  (e.g.  ES  Galicia,  ES  La  Rioja,  IT  Bolzano),  as  well  as 
observations  on the level  of programme implementation  registered in  2017 (e.g. CZ, 
ES Extremadura, FR Auvergne). 

  The  findings  concerning  the  CAP  Objective  2  of  sustainable  management  of 
natural  resources  and  climate  action  (27%)  addressed  the  implementation  and 
achievements of RDP Priorities 4 and 5. In relation to RDP Priority 4, several Member 
States  reported  on  the  trends  of  the  Farmland  Bird  Indicator  (e.g.  CY,  EE,  SI),  the 
changes  in  the  content  and  dynamics  of  soil’s  nitrates  and  sulphates  (e.g.  EE,  DE 
Saxony,  IT  Piedmont),  and  the  implementation  of  agri-environmental  schemes  (UK 
England, FR Mayotte, ES La Rioja). For RDP Priority 5, the evaluations showed the 
RDP  support  to  carbon  conservation  and  capture  (e.g.  ES  Galicia,  ES  Navarra,  SK), 
the reduction of carbon emissions (e.g. IT Trento, PL), renewable energy (e.g. LV, ES 
La  Rioja),  and  investments  in  energy  efficiency  and  water  management  (e.g.  ES 
Murcia, FR Bourgogne, IT Bolzano). 

  The  findings  linked  to  CAP  Objective  3  of  ensuring  balanced  territorial 
development (10%) mainly described the implementation of Priority 6,  with a focus 
on LEADER/Community-Led Local Development (CLLD) (e.g. PT Madeira, BG, EL, 
ES  Valencia)  and  the  support  given  to  Local  Action  Groups  for  implementing  the 
LEADER/CLLD  strategies,  monitoring  and  evaluation  (e.g.  PT  Azores,  IT  Puglia, 
HR, CY).  

  The  findings  related  to  the  horizontal  RDP  Priority  1:  knowledge  transfer  and 
innovation  described  the  level  of  uptake  (e.g.  IT  Puglia,  ES  Galicia),  the  selection 
criteria  and  aspects  of  the  RDP  delivery  mechanism  (IT  Bolzano,  ES  Navarra,  IT 
Trento),  as  well  as  the  positive  contributions  towards  all  RDP  priorities  (e.g.  PT 
Mainland).   

Evaluation  findings  covering  multiple  RDP  priorities  and  focus  areas  (8%)  concerned  more 
general  aspects  of  the  RDP  2014-2020,  such  as  the  internal  and  external  coherence  of  the 
RDP  intervention  logic  (ES  Madrid),  good  practices  observed  across  the  supported  projects 
(ES Aragon), and the challenges in achieving the targets set up in the RDP (SI).  

The  communication  activities  in  relation  to  publicising  evaluation  findings  have 
increased  by  23%  compared  to  the  previous  reporting  period.  344  communication 
activities  were  reported  in  2018,  reaching  out  to  3,97  million  stakeholders.  These  were 
reached  mainly  through  websites  (3,04  million  stakeholders  reached),  publications  and 
evaluation reports (0,23 million), but also through other channels (e.g. written consultations, 
newsletters etc.)  

Follow-ups  given  to  evaluation  results  have  increased  by  more  than  three  times 
compared to those reported in the AIRs submitted in 2017 (from 169 to 544). The follow-
up activities concerned different areas as shown in Figure 3.  

23 

 
 
 
Figure 3: Reported follow-up activities given to evaluation results (N=544) 

Source: European Evaluation Helpdesk for Rural Development (2018) 

It  is  important  to  keep  in  mind  that  the  enhanced  AIRs  submitted  in  2019  will  contain  a 
significant update on the evaluations conducted and their findings as the programmes will 
have reported for the first time on the impacts of the 2014-2020 funds and assessed progress 
towards the objectives of the programme and its contribution to achieving the Union strategy 
for  smart,  sustainable  and  inclusive  growth  based  on  the  relevant  result  and  impact 
indicators11.  

4.1.2.  Ex-post evaluations 2007-2013  

For the EAFRD, Member States had to carry out by 31 December 2016 an ex-post evaluation 
of  their  RDPs12.  Member  States  examined  the  degree  of  utilisation  of  resources,  the 
effectiveness  and  efficiency  of  the  programming  of  the  EAFRD,  its  socio-economic  impact 
and its impact on the EU priorities. They had to cover the goals of the programme and aim to 
draw  lessons  concerning  rural  development  policy.  They  had  to  identify  the  factors  which 
contributed to the success or failure of the programmes’ implementation, including as regards 
sustainability, and identify best practice.  Based on these evaluations, a summary of these ex 
post evaluations was carried out by Commission services by the end of 2017. Furthermore, a 
synthesis  of  the  ex-post  evaluations  was  conducted,  analysing  the  effectiveness,  efficiency, 
coherence, relevance and EU added value of the 2007-2013 RDPs. 

This  work  provided  important  insights  into  the  contribution  to  policy  objectives  made  by  the 
2007-2013  programmes.  Regarding  effectiveness,  the  evidence  base  allows  to  state  that  the 
implementation  of  EAFRD  funding  via  RDPs  has  been  effective  in  achieving  EU-level 
objectives  and  supporting  key  EU  priorities  to  a  variable  extent  (albeit  with  the  caveat  of  a 
better  evidence  base  for  Axis  I  and  II  than  for  Axis  III  objectives).    Based  on  the  overall 
judgments  on  the  three  criteria  (effectiveness  in  achieving  objectives,  coherence  with  EU 

 Commission Implementation Regulation (EU) No 808/2014, Annex I, Point 9, and Annex VII, Point 7. 

11  
12   Article 86 of Regulation (EC) No 1698/2005.  

24 

 
 
 
 
 
 
                                                 
priorities and complementarity with other instruments, and subsidiarity), the evaluation found 
that EAFRD funding via the RDPs ensured EU added value to a moderate and variable extent, 
depending on the aspect considered. However, taking into consideration the limitations of the 
data,  it  was  not  possible  to  provide  an  overall  judgment  of  proportionality  of  EAFRD 
expenditure with regards to benefits achieved at the EU level (efficiency).  As to the extent to 
which  RDPs  have  contributed  to  addressing  the  social,  economic  and  environmental  needs 
within the programmes'  area, it can be concluded that RDPs  have contributed to  addressing 
the needs – across the social / socio-economic, economic and environmental dimensions – to 
an overall moderate extent. 

4.2. Synthesis of EAFRD evaluation work by the Commission 

As regards the evaluation work of the Commission, RDPs' effects are being also assessed via 
specific  thematic  evaluations,  in  which  the  Commission  has  to  ensure  that  the  combined 
impact of all CAP instruments, i.e. the first pillar (direct payments and market measures) and 
the  second  pillar  (rural  development),  is  measured  and  assessed13.  To  date,  the  following 
actives were conducted: 

  The  report  of  the  evaluator  of  the  "Evaluation  study  of  the  forestry  measures  under 
rural  development"14  was  published  in  March  2018.  It  covered  16  evaluation  study 
questions  relating  to:  the  drivers  influencing  the  implementation  choices  taken  by 
Member  States/regions  and  beneficiaries,  as  well  as  the  effects  of  the  measures 
concerning  production,  geographical  distribution;  effectiveness  (in  relation  to 
economic,  environmental  and  climate  objectives);  efficiency;  coherence;  relevance; 
and  EU  added  value.  The  EU  Forest  Strategy  is  implemented  through  a  variety  of 
policies (both at EU and national level) but the main EU-level funding to implement 
the Forest Strategy has been and remains the co-financing of forest measures under the 
Rural Development Regulation. Furthermore, as stated in the EU Forest Strategy, the 
Commission considers that rural development funding in particular should be used to 
support  the  implementation  of  sustainable  forest  management.  Given  that  the 
evaluation covers a limited period of time (2014-2017), in some cases the 2007-2013 
period  was  also  examined  to  assess  the  level  of  implementation  of  the  forest  sub-
measures  and to  capture their mid-term  effects.  This  approach was facilitated by the 
high  degree  of  similarity  of  the  Rural  Development  forest-related  measures  in  these 
two programming periods. 

From the analysis of effectiveness, it can be concluded that the forest measures (representing, 
at EU-28 level, 4,8 % of the total planned public expenditure of RDPs) and horizontal RDP 
measures implemented in forests, have a generally very positive effect, even if often difficult 
to separate from other factors such as state aids and the operations funded by foresters on their 
own.  The  evaluation  identified  also  some  possible  improvements  for  a  more  effective 
implementation of the forest measures. 

13   Article 110 of Regulation (EU) No 1306/2013.  
14    https://publications.europa.eu/s/ilDj 

25 

 
 
 
                                                 
 
The evaluation  process  will be finalised by the  publication of a Commission  Staff Working 
Document. 

  "Evaluation  of  the  impact  of  the  CAP  on  climate  change  and  greenhouse  gas 
emissions"  and  the  "Evaluation  of  the  impact  of  the  CAP  measures  towards  the 
general  objective  of  viable  food  production"  covering  policy  instruments  under  both 
first (EAGF) and second pillar (EARDF). 

  "Evaluation  of  the  impact  of  the  CAP  measures  on  generational  renewal,  local 
development and jobs in rural areas" was launched with the aim to assess how various 
forms of support for young farmers as well as other support measures available under 
the CAP Pillar I and II affect the generational renewal in rural areas.  

Moreover, "Evaluation on the impact of the CAP on water" was launched with the view to be 
published in 2020. 

CMEF Report 

By December 2018, an initial report on the implementation of the Common Monitoring and 
Evaluation  Framework  (CMEF)  and  first  results  on  the  performance  of  the  CAP  will  be 
published. The framework covers both CAP pillars as well as horizontal measures (e.g. cross 
compliance) of the CAP. The report will describe the CAP overall achievements on the basis 
of  the  data  and  evaluation  information  available  so  far  within  the  CMEF  framework.  The 
available evaluation information will also be taken into account. 

5.  EMFF - EUROPEAN MARITIME AND FISHERIES FUND (EMFF) 

5.1. Period 2014-2020 

The national European Maritime and Fisheries Fund  (EMFF) programmes experienced a late 
start  due  to  late  adoption.  In  most  Member  States,  the  first  operations  started  in  2017,  with 
Managing Authorities intending to begin evaluations in 2018. Two Member States, the Czech 
Republic and Sweden performed process evaluations during 2017.   

To  support  Member  States  with  process  and  impact  evaluations  of  the  EMFF,  the 
Commission  provided  a  working  paper  on  EMFF  evaluation15  and  the  EMFF  Evaluation 
toolbox16. 

5.2. Period 2007-2013 

National interim evaluation reports on the European Fisheries Fund (EFF) (2007-2013) were 
conducted  during  the  period  2007-2013.  These  were  summarised  in  a  Staff  Working 
Document17 on the ex-post evaluation of the EFF . 

15   https://ec.europa.eu/fisheries/sites/fisheries/files/fame-working-paper-emff-evaluation_en.pdf  
16   https://ec.europa.eu/fisheries/sites/fisheries/files/fame-working-paper-emff-evaluation-toolbox_en.pdf  

26 

 
 
 
                                                 
The EFF was launched in 2007 at the onset of the global economic crisis. The ensuing decline 
in  demand  for  fisheries  products  and  drop  in  prices,  as  well  as  reduced  access  to  private 
financing  put  the  sector  under  high  economic  pressure.  Despite  this  difficult  situation,  the 
following results were achieved: 

  Support  provided  by  the  EFF  for  the  local  development  of  fishing  depending 
communities  was  a  considerable  success  despite  a  slow  start.  It  allowed  for  the 
creation and maintenance of a large number of jobs and tangibly improved the quality 
of life in these areas. This included the creation of some 10 000 jobs in the processing 
and marketing sectors. 

  Competitiveness  of  the  fleet  was  enhanced,  through  modernisation  and,  most 
importantly, support for changing fishing  gears and methods. Ports and landing sites 
were  improved,  and  investments  in  marketing  and  processing  of  fish  increased  its 
added value. 

  The  EFF  contributed  to  reducing  overcapacity  in  many  fleets,  although  imbalances 

still exist between fleet capacity and fishing opportunities. 

  The  EFF  contributed  to  the  economic  resilience  of  the  aquaculture  beneficiaries, 
especially in the shellfish sector. Due to lack of strategic orientation however, only a 
marginal impact was identified on finfish aquaculture, whose production increased but 
at  a  slower  pace  than  production  at  the  world  scale,  which  has  been  an  extremely 
rapidly growing sector since the beginning of the EFF period. 

  For small scale coastal fisheries, EFF funds were made available for the management 
of  fishing  areas,  the  reduction  of  fishing  effort,  the  organisation  of  production  and 
innovation.  However,  uptake  of  this  measure  was  low  as  it  was  unfocused  and 
competing with other measures encompassing all fleet segments. 

  The  contribution  of  the  EFF  to  broader  conservation  objectives,  as  well  as  weak 
complementarities and synergies with other funds (such as LEADER) were identified 
as areas to be strengthened.  

Several shortcomings identified for the EFF were addressed by the EMFF which succeeded it 
for the 2014-2020 period. 

6.  CONCLUSION 

An overview of the implementation progress shows that the reported expenditure of projects 
selected  at  the  end  of  2017  in  the  programmes  supported  by  the  European  Structural  and 
Investment Funds amounts to 15% of the total planned. Although it has more than doubled in 
12  months,  it  still  represents  a  low  execution  rate,  especially  if  compared  to  the  previous 
programming period.  

The low pace of implementation has determined a limited availability of evaluations for the 
period 2014-2020 so far, behind the level initially planned. The evaluations mostly deal with 

17   SWD(2017) 274 final 

27 

 
 
 
 
                                                                                                                                                         
process/implementation  and  monitoring/progress  issues,  and  those  oriented  to  evaluate  the 
impact of the programmes and the policy are only a minor share of the total. 

However,  the  number  of  2014-2020  evaluations  completed  by  the  Member  States  is  now 
increasing, faster than those referred to in 2007-2013 (the number of evaluations published in 
the last 12 months and referring to the period 2014-2020 nearly doubles the number of those 
referring  to  2007-2013).  There  are  positive  signals  also  on  the  implementation,  like  the 
increase of the selection rate of the projects of 25 percentage points in 2017 alone: this is the 
largest annual rise so far, indicating that Member States are turning the planned investments 
into concrete projects.  

Based on the forecasts of the evaluation plans and the fact that implementation is accelerating 
to  reach  its  full  pace,  the  Commission  expects  that  a  more  conspicuous  number  of  impact 
evaluations related to 2014-2020 programmes will be available in the next 12 months, so that 
more evidence from the Member States will be reported in the 2019 edition of this document. 

28