COMMISSION STAFF WORKING DOCUMENT on the Ex Post Evaluation of the European Refugee Fund 2011-2013 and 2007-2010 Accompanying the document Report from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions Ex post evaluation reports for the period 2011 to 2013 of actions co-financed by the four Funds under the framework programme 'Solidarity and Management of Migration Flows'
Date: 2018-06-12
Institutions: Directorate-General for Migration and Home Affairs, European Commission
Subjects: EU migration policy, EU programme, European Union Agency for Asylum, aid to refugees, fund (EU), migratory movement, refugee, right of asylum

EN 

  EN 

   EUROPEAN COMMISSION  Brussels, 12.6.2018  SWD(2018) 334 final  COMMISSION STAFF WORKING DOCUMENT on the Ex Post Evaluation of the European Refugee Fund 2011-2013 and 2007-2010 Accompanying the document Report from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions  Ex post evaluation reports for the period 2011 to 2013 of actions co-financed by the four Funds under the framework programme 'Solidarity and Management of Migration Flows' {COM(2018) 456 final}  - {SWD(2018) 333 final}  - {SWD(2018) 335 final}  - {SWD(2018) 336 final}  
 
 
1  Contents 

2 

3 

4 

5 

Introduction ..................................................................................................................................... 2 

Background to the initiative ............................................................................................................ 3 

The ERF as funding instrument ....................................................................................................... 4 

Evaluation questions ....................................................................................................................... 6 

6  Method ............................................................................................................................................ 6 

6.1 

Limitations ................................................................................................................................... 7 

7  Management modes of the RF ........................................................................................................ 8 

7.1 Shared management: Multiannual and Annual Programmes of the Member States ........... 8 

7.2 Direct management: Annual Work Programmes of the Commission .................................. 10 

8 

9 

Implementation – state of play ..................................................................................................... 10 

Answers to the evaluation questions ............................................................................................ 15 

9.1 

9.2 

9.3 

9.4 

9.5 

9.6 

10 

11 

Relevance .............................................................................................................................. 15 

Effectiveness ......................................................................................................................... 18 

Efficiency .............................................................................................................................. 28 

Sustainability ......................................................................................................................... 31 

Complementarity and coherence ........................................................................................... 32 

EU added value ..................................................................................................................... 33 

Conclusion ................................................................................................................................. 34 

ANNEX 1 — Procedural information ......................................................................................... 37 

Section 7.1 ‘Shared management: Multiannual and Annual Programmes of the Member States’ and 
the explanations related to the allocation key .................................................................................... 37 

12 

13 

14 

15 

ANNEX 2 — Stakeholders’ consultation .................................................................................... 39 

ANNEX 3 — Methodology ......................................................................................................... 46 

ANNEX 4 — List of evaluation questions ................................................................................... 49 

ANNEX 5 — List of abbreviations and country codes ................................................................ 51 

1 

 
 
 
 
 
 
2 

Introduction 

This  staff  working  document  reports  on  the  ex  post  evaluation  assessing  the  results  and 
impacts  of  actions  co-financed  by  the  European  Refugee  Fund  (ERF)  under  the  annual 
programmes  implemented  by  the  27  participating  Member  States1  (including  emergency 
measures), and of the Community actions, in the period 2011-20132. Actions under national 
programmes financed during this period were implemented from 1 January 2011 to 30 June 
2015.  

The  ex  post  evaluation  assessed  ERF  in  the  light  of  its  relevance  (whether  its  objectives 
matched  with  societal  needs),  effectiveness  (to  what  extent  objectives  have  been  achieved), 
efficiency  (to  what  extent  costs  were  proportionate  the  achieved  benefits),  sustainability  of 
effects after the intervention ended, coherence between the actions financed by the instrument 
and complementarity to other interventions, and the added value of intervening at EU level. 

The  results  of  this  evaluation  will  complement  the  interim  evaluation  of  the  Asylum, 
Migration  and  Integration  Fund  (AMIF)3,  the  successor  of  the  Return  Fund,  Refugee  Fund 
and Integration Fund for the period 2014-2020. The interim evaluation of the AMIF is due in 
20184. The Commission will submit an interim evaluation report of the Fund by 30 June 2018 
to the European parliament, the Council, the European Economic and Social Committee and 
the  Committee  of  the  Regions.  This  interim  evaluation  report  will  be  based  on  the  interim 
evaluation reports provided by the Member States on the implementation of the actions and 
progress towards achieving the objectives of their national programmes to be submitted to the 
Commission by 31 December 2017. The interim evaluation report which the Commission will 
submit  will  also  include  an  assessment  of  the  mid-term  review  carried  out  by  the  Member 
States and of the implementation of the emergency assistance grants and Union actions. The 
results  of  the  interim  evaluation  together  with  the  results  of  the  present  evaluation  will 
contribute to the shaping of the future policies under DG HOME responsibility, especially to 
the preparation of the new funding instruments in the framework of the Multiannual Financial 
Framework (MFF) post 2020. 

1   Denmark  does  not  participate  in  accordance  with  Articles  1  and  2  of  the  Protocol  on  the  position  of 
Denmark,  annexed  to  the  Treaty  on  European  Union  and  to  the  Treaty  establishing  the  European 
Community. Croatia participated as from 2013 only, upon accession to the EU. 

2   The annual programmes implemented by the Member States under the ERF 2008-2010 annual programmes 
are  already  covered  by  the  ERF  2008-2010  ex  post  evaluation.  The  evaluation  also  covers  the  2008-2010 
Community actions. 

3   Regulation (EU) No 516/2014 of the European Parliament and of the Council of 16 April 2014 establishing 

the Asylum, Migration and Integration Fund (OJ L 150, 20.5.2014, p. 168). 

4   Part  of  the  interim  evaluation  is  the  mid-term  review  of  the  AMIF  national  programmes  of  the  Member 
States (Regulation (EU) No 514/2014, Article 15) which will take place in 2017 and 2018. The purpose of 
the mid-term review is for the Commission and the Member States to review the national programmes and 
assess the need for a possible revision of the programme, in the light of developments in Union and national 
policies  through  a  questionnaire  and  bilateral  dialogues  with  the  Member  States.  In  addition  if  the  need 
arises,  the  results of  the  mid-term  review  of  the national programmes  may support requests  for  additional 
funding made by the Commission to the budgetary authorities for the remaining implementation period.   

2 

 
 
 
 
                                                            
3  Background to the initiative 

In  the  years  2000,  the  European  Union  had  on  average  received  around  300  000  asylum 
applications a year. In this light a first tangible expression of solidarity in the field of asylum 
at EU level was the creation of the ERF in 20005, which has been instrumental in laying the 
foundations of collective action by the European Union for the reception of asylum applicants 
and persons requiring international protection as part of a comprehensive approach. The first 
ERF ended in 2004 and was followed by a second phase6 (ERF II) in 2005 - 2007.  

A number of crucial requirements in the field of EU asylum policy were identified in 2005 in 
an impact assessment7. The main problem to be addressed was to translate into practice the 
principle of solidarity8 between Member States in the EU asylum policy and was defined as 
follows:   

How  to  implement  the  principle  of  solidarity  in  managing  people  flows  by  ensuring  a  fair 
share  of  responsibilities  between  Member  States  as  regards  the  burden  arising  from  the 
implementation  of  common  policies  on  asylum  and  immigration  and  on  the  management  of 
the external borders? 

The 2008-2013 ERF9 (the subject of the current ex post evaluation and hereinafter referred to 
as  'ERF')  is  the  third  phase  of  solidarity  in  the  field  of  asylum  and  is  one  of  the  four  EU 
funding instruments10 established in 2007 as part of the framework programme on solidarity 
and migration flows11 and of a wider EU policy framework which also included the European 
Asylum Support Office Agency (EASO), the Dublin Regulation12, the EURODAC system13 

5   Council  Decision  2000/596/EC  of  28  September  2000  establishing  the  European  Refugee  Fund  for  the 

period 2000-2004 - OJ L 252, 6.10.200, p.12., with a EUR 216 million budget. 

6   Followed by ERF II, for 2005-2007, with a budget of EUR 114 million (Council Decision 2004/904/EC) 
7    Annex  to  the  General  Programme  Solidarity  and  Management  of  Migration  Flows  –  Extended  impact 

assessment 2005 – SEC(2005) 435 of 6/04/2005. 
Including share of financial responsibility. 

8  
9  Established  by  Decision  No  573/2007/EC  of  the  European  Parliament  of  the  Council  of  23  May  2007 
establishing  the  European  Refugee  Fund  for  the  period  2008  to  2013  as  part  of  the  General  programme 
'Solidarity and Management of Migration Flows'. 

10   The  General  Programme  “Solidarity  and  Management  of  Migration  Flows”  (SOLID)  consisted  of  four 
instruments: External Borders Fund (EBF), European Return Fund (RF), European Refugee Fund (ERF) and 
European Fund for the Integration of third-country nationals (EIF). 

11  Commission  Communication  establishing  A  framework  programme  on  solidarity  and  the  management  of 

migration flows for the period 2007-2013 (COM(2005) 123 final). 

12   Dublin II Regulation - Council Regulation (EC) No. 343/2003 of 18 February 2003 establishing the criteria 
and mechanisms for determining the Member State responsible for examining an asylum application lodged 
in  one  of  the  Member  States  by  a  third-country  national  &  Dublin  III  Regulation  -  Regulation  (EU)  No 
604/2013  of  the  European  Parliament  and  of  the  Council  of  26  June  2013  establishing  the  criteria  and 
mechanisms  for  determining  the  Member  State  responsible  for  examining  an  application  for  international 
protection lodged in one of the Member States by a third-country national or a stateless person. 

13  Established by Council Regulation (EC) No 2725/2000 of 11 December 2000 concerning the establishment 
of  ‘Eurodac’  for  the  comparison  of  fingerprints  for  the  effective  application  of  the  Dublin  Convention 
(OJ L 316, 15.12.2000, p. 1) and Council Regulation (EC) No 343/2003 of 18 February 2003 establishing 
the  criteria  and  mechanisms  for  determining  the  Member  State  responsible  for  examining  an  asylum 
application lodged in one of the Member States by a third-country national (OJ L 50, 25.2.2003, p. 1). 

3 

 
 
                                                            
and  emergency  measures14.  Since  1999,  the  EU  has  been  working  to  create  a  common 
European asylum system (CEAS)15 affecting the obligations of Member States with regards to 
the reception of asylum applicants and asylum procedures, the integration of refugees and the 
beneficiaries  of  subsidiary  protection.  Moreover,  the  EU  has  adopted  several  legislative 
measures16  harmonising  common  minimum  standards  for  asylum.  These  measures  (the 
asylum acquis) aimed at ensuring an equal treatment of asylum applicants in an open and fair 
system. 

One of the characteristic of the period17 was the sudden and unforeseen increase of migratory 
pressure which started already in 2012 and led to a drastic increase in the number of asylum 
applications in the European Union, with great variation across Member States. The number 
of  asylum  applicants  in  2014  reached  600  000  persons  and  were  mainly  concentrated  in 
Germany  (173  100),  followed  by  Sweden  (75  100),  Italy  (63  700),  France  (59  000)  and 
Hungary (41 300).  

Figure 1: Number of asylum applicants in the 28 Member States (period 2008-2016) 

Source: Eurostat data as of 07.03.2017 
http://appsso.eurostat.ec.europa.eu/nui/show.do?dataset=migr_asyappctza&lang=en  

4  The ERF as funding instrument  

The general objective reflected into the Impact Assessment18 underpinning the SOLID Funds 
was to  support  the development and  implementation  of the CEAS on the basis of solidarity 
between  Member  States,  and  to  promote  balance  in  Member  States’  efforts  to  bear  the 
consequences  of  receiving  refugees  and  displaced  persons,  taking  account  of  the  relevant 

14  Taken  pursuant  to  Council  Directive  2001/55/EC  to  provide  temporary  protection  in  the  event  of  a  mass 

influx of refugees. 

15   The  CEAS  provides  common  minimum  standards  for  the  treatment  of  all  asylum  applicants  and 
applications.  It  consists  of  a  legal  framework  covering  all  aspects  of  the  asylum  process  and  a  support 
agency, the European Asylum Support Office (EASO). 

16   The  revised  Asylum  Procedures  Directive,  the  revised  Reception  Conditions  Directive,  the  revised 

Qualification Directive, the revised Dublin Regulation and the revised Eurodac Regulation. 

17   Implementation period from 1 January 2011 to 30 June 2015. 
18   Annex  to  the  General  Programme  Solidarity  and  Management  of  Migration  Flows  –  Extended  impact 

assessment 2005 – SEC(2005) 435 of 6/04/2005. 

4 

 
 
 
 
                                                            
Community  legislation.  The  general  objective  of  the  ERF19  does  not  fully  capture  this  idea 
and limits its expression "to support and encourage the efforts made by the Member States in 
receiving  and  bearing  the  consequences  of  receiving  refugees  and  displaced  persons  taking 
into  account  of  Community  legislation  on  those  matters".  The  Fund  acted  as  a  financial 
solidarity mechanism to support Member States that, on behalf of the EU, bear a substantial 
long-term financial burden under the CEAS. 

The ERF's general objective was not translated further into specific and operational objectives 
in  the  legal  base.  Instead,  the  legal  base  identified  five  policy  areas  and  within  each  policy 
area  defined  concrete  eligible  actions.  The  five  policy  areas  are  the  following:  i)  reception 
conditions  and  asylum  procedures,  ii)  integration  of  target  groups  iii)  enhancement  of  MS' 
capacity  to  develop,  monitor  and  evaluate  their  asylum  policies,  iv)  resettlement  and  v) 
transfer  of  target  groups.  The  intervention  logic  of  the  ERF's  legal  base  is  therefore  not 
consistent with the wider policy objectives. 

The definition and scope of the policy areas was linked to the scope of the Common European 
Asylum  System.  Therefore  the  evaluation  assimilated  the  five  identified  policy  areas  to 
specific objectives and the eligible actions within the policy areas as operational objectives. In 
consequence, the effectiveness of the ERF was evaluated against these five policy areas.  

Furthermore, the absence of clearly defined objectives in the legal base led to the lack of EU 
targets linked to operational objectives to measure the results of the ERF. Therefore, to assess 
the effectiveness of the Fund, the evaluation had to rely mainly on programme targets set by 
Member  States  in  their  annual  programmes  which  reflect  first  and  foremost  national  needs 
and are assessed in the evaluation against interviews with stakeholders. This, combined with 
the absence of a baseline20, proved to be strong limitation in terms of methodology. 

To  compensate  for  the  absence  of  clearly  defined  objectives  and  targets  and  in  accordance 
with  the  Decision  establishing  the  ERF21,  the  Commission  adopted  Strategic  Guidelines22 
which  identified  three  priorities23  and  seven  specific  priorities  (between  2  to  3  specific 
priorities per priority). The three main priorities were:  

  Priority  1:  Implementation  of  the  principles  and  measures  set  out  in  the 
Community  acquis  in  the  field  of  asylum,  including  those  related  to  integration 
objectives. 

  Priority 2: Development of reference tools and evaluation methodologies to assess 
and  improve  the  quality  of  procedures  for  the  examination  of  claims  for 

19      Article  2(1)  of  Decision  No  573/2007/EC:  "The  general  objective  of  the  Fund  shall  be  to  support  and 
encourage  the  efforts  made  by  the  Member  States  in  receiving,  and  in  bearing  the  consequences  of 
receiving, refugees and displaced persons, taking account of Community legislation on those matters, by co-
financing the actions provided for in this Decision". 

20   Due to strong opposition from MSs. 
21    Article 17 of Decision 573/2007/EC. 
22   Commission  Decision  of  29  November  2007  implementing  Decision  No  573/2007/EC  of  the  European 
Parliament  and  of  the  Council  as  regards  the  adoption  of  the  strategic  guidelines  2008  to  2013  (notified 
under document number C(2007) 5738), OJ L 326, 12.12.2007, p. 29. 

23   Commission  Decision  of  29  November  2007  implementing  decision  No  573/  2007/  EC  of  the  European 

Parliament and of the Council as regards the adoption of the strategic guidelines 2008 to 2013).   

5 

 
 
                                                            
 
international  protection  and  to  underpin  administrative  structures  in  an  effort  to 
respond to the challenges brought forward by enhanced practical cooperation with 
other Member States. 

  Priority  3:  Actions  helping  to  enhance  responsibility  sharing  between  Member 

States and third countries (optional) 

The specific priorities were defined to target particular actions for which the Member States 
could request an EU co-financing rate of 75%24 (See Section 7). Each action planned in the 
annual  programme  should  be  programmed  under  one  of  the  priorities  set  in  the  Strategic 
Guidelines. As the Strategic Guidelines did not set any threshold per priorities, Member States 
could  distribute  their  annual  allocation  as  they  wished  provided  that  the  MSs  target  the 
available resources to the first two priorities, the third one being optional.  

However, the funding priorities contributed to the confusion of the intervention logic as the 
first  priority  had  a  very  wide  scope,  equivalent  more  or  less  to  the  general  objective  of  the 
Fund.  This  issue  was  addressed  under  AMIF  which  shows  a  clear  intervention  logic  and 
simpler  articulation,  with  one  general  objective,  translated  into  four  specific  objectives, 
accompanied  by  examples  of  actions.  Under  AMIF,  also  a  common  monitoring  framework 
has been developed to enable tracking of progress25. 

5  Evaluation questions 

This evaluation assessed the ERF against the five mandatory evaluation criteria laid down in 
the Commission Better Regulation Guidelines (effectiveness, efficiency, relevance, coherence 
and  EU  added  value).  In  addition,  the  co-financed  actions  were  also  assessed  for  their 
sustainability  and  complementarity.  To  assess  these  seven  criteria,  thirteen  evaluation 
questions were defined (see Annex 6). 

6  Method 

This evaluation relies on an external study carried out between April 2016 and March 2017. 
The  methodology  combined  desk  research,  on-site  visits,  and  qualitative  and  quantitative 
analysis. It required a systemic summary of the evidence regarding the implementation of the 
Fund,  most  of  the  information  being  provided  by  the  Member  States’  national  evaluation 
reports  (NERs)  and  annual  programmes  (APs).  The  APs  set  out  the  operational  objectives 
through actions for each year and the NERs provided information on outputs relating to the 
operational  objectives,  results  on  the  specific  objectives  and  impacts  as  regards  the  general 
objective. 

24   Article 14.4 of Decision 573/2007/EC. 
25   Commission  Delegated  Regulation  (EU)  2017/207  of  3  October  2016  on  the  Common  monitoring  and 
evaluation framework provided for in Regulation (EU) No 514/2014 of the European Parliament ad of the 
Council  laying  down  general  provisions  on  the  asylum,  migration  and  integration  Fund  and  on  the 
instrument  for  financial  support  for  police  cooperation,  preventing  and  combating  crime,  and  crisis 
management, OJ, 8.2.2017, L 33, p.1. 

6 

 
 
 
                                                            
Consultations  took  place  with  a  broad  range  of  stakeholders  (more  than  120),  including  27 
responsible  authorities  (RAs),  88  beneficiaries,  5  EU  officials  and  case-study  respondents. 
The  methodology  combined  interviews  and  case  studies26  (each  involving  a  site  visit).  The 
case studies illustrate an in-depth analysis of Member States' national programmes and have 
facilitated  the  understanding  of  success  stories  and  failures.  The  results  of  the  case  studies 
were integrated in the replies to the evaluation criteria. An internet based public consultation 
was also carried out (a detailed analysis of its outcome is attached in Annex 2).  

Information  was  triangulated  to  ensure  validity  and  robustness.  The  financial  data  extracted 
from SFC27 and ABAC28 and presented in the document was updated in January 2017. 

6.1  Limitations 

The  evaluation  process  has  encountered  some  difficulties.  In  particular,  the  assessment  of 
effectiveness and efficiency was hindered, to a certain extent, by the absence of comparable 
quantitative data. For annual programmes, it is due to the fact that the reporting requirements 
in  the  Fund  Regulation  did  not  stipulate  that  RAs  should  collect  the  data  that  would  be 
required to conduct an assessment of effectiveness and efficiency29. Furthermore, it has been 
difficult  to  measure  efficiency,  as  no  clear  trends  could  be  established  due  to  the  large 
diversity  of  projects,  which  could  not  be  compared  and/or  the  lack  of  ex  ante  targets  to 
measure efficiency. Therefore, the assessment of efficiency had to reply on more qualitative 
data collected through the stakeholder consultations and the national evaluation reports.   

Indicators  and  quantitative  targets  were  set  by  Member  States  through  their  annual 
programme when selecting the actions supported by the programme for each year. For each 
action  proposed,  Member  States  defined  independently  the  expected  quantified  results.  In 
consequence, there was a multitude of indicators across Member States with no harmonization 
and  a  limited  assessment  on  the  side  of  the  Commission  representing  a  limitation  for  the 
evaluation.  

Most of the Member States achieved their quantitative targets related to a number of asylum 
seekers and refugees to be received and/or integrated. On the contrary, what was missing was 
the  qualitative  analysis  of  the  results.  For  instance,  under  Priority  1,  how  the  reception  and 
integration  and  asylum  procedures  would  be  improved.  Furthermore,  given  the  absence  of 
agreed  benchmarks,  Member  States  had  different  understandings  of  what  to  record  in  the 
NERs30. It is likely that Member State evaluation experts dealing with NERs misunderstood 
the exact meaning of some indicators with the result that sometimes the data reported was not 
comparable  between  Member  States.  To  mitigate  this  problem  to  the  extent  possible,  RAs 
were provided with guidance on the interpretation of the indicators on an ad hoc basis at their 

26   AT, BE, BG, FR, IT, SE and UK. 
27   System  for  Fund  Management  2007  (SFC)  is  an  online  monitoring  tool  managed  by  the  Commission 
services and used for certain funding programmes, where Member States can upload implementation reports 
and financial information. 

28   ABAC is the Commission’s accrual-based online accounting system. 
29   This  is  explained  by  the  fact  that  the  Better  Regulation  agenda,  which  formalised  the  criteria  of  the 

evaluations to be carried out by the Commission, was adopted after the ERF was established. 

30   Some Member States recorded the total compound number relating to an indicator in a given year (i.e. the 
total for the year in question and all the previous ones), while others provided only the additional number for 
that year (i.e. the change in the number). Some were not consistent in the units used for the indicators. 

7 

 
 
                                                            
request  during  the  implementation  of  the  annual  programmes,  and  this  was  then  made 
accessible to all RAs. 

To partly overcome these limitations, different approaches/mitigation strategies were taken by 
the evaluator. For the effectiveness and efficiency, the analysis was restricted to being based 
primarily on stakeholders’ assessments, gathered through interviews and a phone survey with 
beneficiaries and RAs. The potentially positive stakeholder bias was corrected by looking at 
past achievement of predecessor programmes and the judgment of the evaluators. Overall, the 
availability, robustness and reliability of the data were sufficient to generate findings. 

To  improve  transparency  and  accountability  in  the  future,  the  Commission  has  developed  a 
specific common monitoring and evaluation framework31 and prepared a document to guide 
RAs in their AMIF evaluation exercise. Further details on the methodology can be found in 
Annex 3 and in the relevant parts of the document. 

7  Management modes of the RF  

The actions co-financed under the ERF were implemented through two different management 
modes: 

7.1 Shared management: Multiannual and Annual Programmes of the Member States 

Programming: Under the shared management mode, MSs hold the primary responsibility for 
the implementation and management and control of the interventions of the Fund. Under this 
management  mode,  the  Fund  is  implemented  on  the  basis  of  strategic  multiannual 
programmes adopted for each Member State covering the 2008-2013 programming period. In 
addition, annual programmes  were negotiated each  year  with each Member State within the 
framework  set  by  the  multiannual  programme  to  implement  the  annual  financial  allocation 
granted to each of them.  
Calculation  of  the  annual  financial  allocation32  for  each  Member  State:  to  express 
solidarity  with  the  Member  States  who  bear,  for  the  benefit  of  the  EU,  a  heavier  financial 
burden, the Fund basic act lays down the two objective criteria to be taken into consideration 
for the calculation of the annual allocation, i.e. the total number of refugees, TCNs enjoying 
subsidiary  protection  and  resettled  persons  admitted  in  a  given  MS  over  the  previous  three 
years  and  the  total  number  of  asylum  applicants,  persons  who  have  applied  for  subsidiary 
protection  and  TCNs  enjoying  temporary  protection  in  a  given  MS  over  the  previous  three 
years. 

The provisional allocations were communicated to the Member States the year before (by 1st 
July  each  year33)  the  adoption  of  the  annual  programme  in  question  and  were  based  on 

31   Commission  Delegated  Regulation  (EU)  2017/207  on  the  common  monitoring  and  evaluation  framework 
provided for in Regulation (EU) No 514/2014 of the European Parliament and of the Council laying down 
general  provisions  on  the  Asylum,  Migration  and  Integration  Fund  and  on  the  instrument  for  financial 
support for police cooperation, preventing and combating crime, and crisis management. 

32  Decision No 573/2007/EC, Article 13.  
33   Decision No 573/2007/EC, Article 20.2 

8 

 
 
 
 
                                                            
average  figures  over  the  previous  three  years  (e.g.  allocation  for  the  2013  AP  were 
communicated in 2012 and were based on data of 2009, 2010 and 2011).  

Content  of  the  Annual  Programme:  The  annual  programme  sets  out  the  actions  to  be 
implemented  in  the  Member  State  and  indicates  their  purpose,  scope,  beneficiaries  (public 
authorities  in  charge  of  return,  NGO's,  international  organisations  etc),  expected  results  and 
financial allocation (EU and national co-financing). Each action is then implemented by one 
or more project(s) on the ground. In the shared management context, the projects are selected 
by the competent national authority (i.e. the Responsible Authority (RA)). Member States are 
therefore  independent  and  only  ex  post  assessment  is  carried  out  by  the  Commission  at  the 
time of submission of the closure report of the AP in question. The target group for the Fund 
comprises any third-country national (TCN) or stateless person having the status defined by 
the  Geneva  Convention  and  who  is  permitted  to  reside  as  a  refugee  in  one  of  the  Member 
States, any third-country national or stateless person enjoying a form of subsidiary protection 
within  the  meaning  of  Directive  2004/83/EC,  any  third-country  national  or  stateless  person 
who has applied for one of the forms of protection described in points (a) and (b), any third-
country  national  or  stateless  person  enjoying  temporary  protection  within  the  meaning  of 
Directive 2001/55/EC and any third-country national or stateless person who is being or has 
been resettled in a Member State34. 

During  the  eligibility  period  of  each  annual  programme,  Member  States  were  entitled  to 
request  a  revision  of  their  annual  programme  to  adjust  the  actions  to  the  changing 
migration  flows  and/or  emergency  situations.  Member  States  could  not  however  request 
additional funding for the implementation of a specific AP.  

The  EU  co-financing  rate  is  set  at  50%.  This  rate  may  be  increased  to  75  %  for  projects 
addressing  specific  priorities  identified  in  the  Strategic  Guidelines  as  well  as  for  projects 
implemented by the Member States covered by the Cohesion Fund35.  

Technical  assistance:  Member  States  were  entitled  to  use  the  funds  allocated  to  the  AP  to 
finance  "preparatory  measures,  management,  monitoring,  evaluation,  information  and 
control measures as well measures for the reinforcement of the administrative capacity for the 
implementation of the Fund"36. For the AP 2011 to 2013, the technical assistance was set at 
4% of the total annual amount of funding allocated plus EUR 30,000. 

Closure  of  the  Annual  Programme:  The  eligibility  period  of  each  annual  programme  was 
set  at  two  years  and  a  half  (e.g.  2013  annual  programme  ran  from  1  January  2013  until  30 
June  2015).  After  the  end  of  eligibility  period,  the  Member  States  submitted  a  final  report, 
which included an audit report. The part of the annual allocation not spent – if any – after the 
end of the eligibility period was lost for the Member States and for the Commission. 
Emergency  assistance37:  In  addition,  to  top  up  the  allocation  provided  to  their  annual 
programmes,  the  Commission  was  entitled  to  provide  "assistance  to  Member  States  for  the 
implementation of emergency measures aimed at addressing situations of particular pressure". 
According  to  the  basic  act  of  the  Fund,  "Such  situations  are  characterised  by  the  sudden 
arrival  at  particular  points  on  the  borders  of  a  large  number  of  third  country  nationals  who 

34  
 Decision No 573/2007/EC, Article 6. 
35  Decision No 573/2007/EC, Article 14.4.  
36    Decision No 573/2007/EC, Article 16. 
37   Decision No 573/2007/EC, Article 5. 

9 

 
 
                                                            
may  be  in  need  of  international  protection,  which  place  exceptionally  heavy  and  urgent 
demands  on  the  reception  facilities,  the  asylum  system  or  infrastructure  of  the  Member 
State(s) concerned and may give rise to risks to human life, wellbeing or access to protection 
provided  under  Community  legislation."  The  duration  of  these  actions  could  not  exceed  6 
months.  

Over the period, the Commission allocated EUR 66 385 316 to emergency assistance which is 
equal to 18% of the total programmed budget dedicated to annual programmes (EUR 22 803 
735 in 2011, EUR 10 202 335 in 2012 and EUR 33 379 246 in 2013). Overall, 51 emergency 
measures  were  implemented  between  2011  and  2013  in  11  Member  States  faced  with 
particular pressure. 

7.2 Direct management: Annual Work Programmes of the Commission 

Under  this  management  mode,  up  to  10%  of  the  RF's  available  resources  could  be  used  to 
finance transnational actions or actions of interest to the EU as a whole, which were managed 
by the Commission (Community actions). 

Each year the Commission adopted an annual work programme (AWP) specifying how the 
objectives of the ERF were to be pursued for the implementation period, taking into account 
the  policy  context.  The  AWPs  were  mainly  implemented  via  grants,  for  which  the 
Commission  then  launched  calls  for  proposals  defining  the  eligible  actions.  Calls  for 
proposals  were  addressed  to  public  authorities  from  the  Member  States  participating  in  the 
Fund,  International  Organisations  and  NGOs  registered  in  one  of  the  Member  States.  The 
ERF  Community  Actions  were  co-financing  (up  to  90%  of  the  total  eligible  costs  of  the 
action)  transnational  projects  aiming  at  developing  methodological  tools  (i.e,  facilitating 
integration  and  identifying  alternatives  to  detention),  addressing  the  needs  of  vulnerable 
people  (in  particular  unaccompanied  minors  and  victims  of  torture),  and  strengthening 
responsibility  sharing  between  Member  States 
through  resettlement  and  relocation 
mechanisms. Furthermore, two grants aiming at reforming and setting up a new Greek asylum 
system were also directly awarded to UNHCR to support the emergency situation related to 
the asylum system in Greece.   

8 

Implementation – state of play 

The ERF was launched in 2008 with an available financial envelope of EUR 628 million. The 
allocation  for  2011-2013  was  EUR  386.2  million,  of  which  EUR  373.5  million  was  under 
shared  management  (including  emergency  measures  with  EUR  68  million  funded  to  10 
Member States) and EUR 12.7 million under direct management for Community Actions. The 
table below shows the EU distribution of financing to the different management modes during 
the period under assessment. 

10 

 
 
 
 
Table 1:   Total programmed EU contribution for ERF, 2011-2013 (EUR)38 

Shared  management  (via  national 
programmes)  

2011 

2012 

2013 

Total 

116 722 757 

111 259 303 

145 463 497 

373 445 557 

Direct 
Community actions) 

management 

(via 

3 953 032 

4 214 040 

4 545 211 

12 712 283 

Total ERF EU budget 

120 675 789 

115 473 343 

150 008 708 

386 157 840 

A) Implementation through shared management 

Finding 1: Absorption rates are good and remain stable at 81% 

As  illustrated  by  figure  2,  the  programmed  EU  contribution  rose  gradually  by  24 %,  from 
EUR 117 million in 2011 to EUR 145 million in 2013 due to new challenges to be addressed 
in the area of asylum. The average absorption rate remained stable across the period, at 81%.   

Figure 2:   Programmed  and  net  EU  contributions  and  absorption  rates  by  year,  ERF 

shared management, 2011-2013 

Source: European Commission data as of 31.12.2016 (ABAC/SFC) 

Under the shared management mode, Member States hold the primary responsibility for the 
implementation and management and control of the interventions of the Fund. Member States 
implement programmes at national level and allocate these funds to end recipients which are 
public authorities, NGOs or international organisations. 

Actions under national programmes were implemented from 1 January 2011 to 30 June 2015. 
It  should  be  underlined  that  some  national  programmes  were  not  yet  closed  when  the 
evaluation was carried. Member States demonstrated through the level of their absorption rate 
the  ability  to  perform  effective  programming  and  involve  national  stakeholders  in  the 

38   The Fund’s contribution to projects in Member States is set at maximum 50% of the total eligible costs of an 
action  and  at  75%  for  Member  States  covered  by  the  Cohesion  Fund  and  for  projects  reflecting  specific 
priorities  mentioned  in  the  strategic  guidelines.  For  projects  carried  out  under  direct  management,  the 
contribution may amount to 90% of the total eligible costs of a project. 

11 

 
 
 
 
                                                            
implementation of projects. The ERF absorption rate ranged from 72 % to 84 % in 2007-2013 
and  was  81 %  overall  for  2011-2013.  No  particular  correlation  could  be  identified  between 
absorption rate and geographical location, country size or share of the ERF. Nevertheless, as 
highlighted in  Figure 3, the absorption rate  of most Member States is above  70 %, with the 
exception  of  the  Czech  Republic  (60 %)  and  Ireland  (with  an  under-performing  rate  of 
41 %39). According to these Member States, these low absorption rates were due to the need 
for beneficiaries to co-finance 25 % of the projects themselves as it was not financed by the 
national  budget  of  the  Member  State,  which  appeared  to  be  a  constraint  for  most 
stakeholders40.  

Figure 3:   Programmed  and  net  EU  contributions  and  absorption  rates  by  Member 
State, ERF shared management, 2011-2013 (including emergency assistance) 

Source: European Commission data as of 31.12.2016 (ABAC/SFC) 

In the NERs and case studies, stakeholders cited the following obstacles to achieving a 100 % 
absorption rate: 

  secondary  impacts  of  the  financial  crisis  affecting  the  capacities  of  beneficiaries  to 

co-finance projects themselves; 

  Member  States  with  relatively  underdeveloped  asylum  systems  having  to  build 

capacity; 

  greater migratory pressure and related conflicts which became protracted; and 
  administrative delays, which resulted in agreed deadlines being missed.  

Finding 2: 'Sharing the burden' – the distribution of ERF funding follows partially the trends 
in asylum applications across Member States because of a delay of three years compared to 
the trend in asylum applications  
The total ERF contribution varied significantly across Member States. Five Member States41 
account for almost 59 % of the total programmed and the total final EU contributions. Funds 
were  allocated  on  the  basis  of  the  methodology  set  out  in  Article  13  of  Decision 
No 573/2007/EC.  The  annual  allocation  for  each  Member  State  took  account  of  the  target 

39   Ratio between the final EU contribution and the Programmed EC contribution. 
40   NGOs, International Organisations. 
41   IT, SE, FR, UK and DE. 

12 

 
 
 
 
                                                            
groups (asylum applicants and resettled persons) and the Eurostat statistics available for the 
last three years.  
Because of  this  method, the allocated budget  for the Member  States  shows a delay of three 
years  compared  to  the  trend  in  asylum  applications.  This  weakness  was  mitigated  to  some 
extent  by  the  allocation  for  emergency  measures  to  support  Member  States  subject  to 
particular and urgent pressure. The countries most affected by the sudden arrival of migrants42 
saw their allocations increase by 97 % over the 2011-2013 period thanks to EU funding for 
emergency measures and partly to the annual calculation of their allocation (See section 7). 

Finding 3: Due to the absence of a threshold, EU funding was concentrated on Priority 1 

In the period 2011-2013, funds were allocated to the three funding priorities43, plus technical 
assistance,  emergency  measures  and  resettlement  activities.  In  2011-2013,  over  90 %  of 
projects  were  implemented  under  priority  1  (including  emergency  measures),  8 %  under 
priority  2  and  less  than  2 %  under  priority  3.  The  countries  that  focused  more  on  the 
implementation of field and concrete support for asylum applicants44 addressed urgent day-to-
day issues45.  

42   BG, CY, DE, EL, FR, HU, IT, MT and NL. 
43   (1) Implementation of the principles and measures set out in the Community acquis in the field of asylum, 
including  those  related  to  integration  objectives;  (2)  Development  of  reference  tools  and  evaluation 
methodologies  to  assess  and  improve  the  quality  of  procedures  for  the  examination  of  claims  for 
international  protection  and  to  underpin  administrative  structures  in  an  effort to  respond  to  the  challenges 
brought  forward  by  enhanced  practical  cooperation  with  other  Member  States;  (3)  Actions  helping  to 
enhance responsibility sharing between Member States and third countries (optional priority). 

44   EL, IT, LU, SV and HR only implemented projects under priority 1. 
45   Improvement of the accommodation facilities, psychological and social support, legal assistance. 

13 

 
 
 
 
                                                            
Figure 4:   Programmed and final EU contribution by priority, 2011-2013 

80% 

72% 

250

200

150

100

50

0

U
E

t
e
N
/
d
e
m
m
a
r
g
o
r
P

€

s
n
o
i
l
l
i

M
n
o
i
t
u
b
i
r
t
n
o
C

95% 

91% 

93% 

100%

94% 

80%

60%

40%

20%

0%

Priority 1 Priority 2 Priority 3 Resettl.

EM

TA

Programmed EU Contribution

Final EU Contribution

Absorption rate

Source: European Commission data as of 27.02.2017 (SFC2007)46 

The number of emergency measures (EM) programmed between 2011 and 2013 (accounting 
for  18 %  of  the  ERF  envelope)  varied  greatly  among  Member  States  depending  on 
geographical location and/or the extent of asylum pressure faced by the Member State. Italy 
was the main beneficiary of emergency measures between 2011 and 2013, with 45 % of the 
total allocation. 

In  total,  9 058  persons  were  resettled  by  12  Member  States  in  2011-2013  using  ERF 
assistance. Numbers varied significantly between Member States, ranging from six in Czech 
Republic  to  over  4 400  in  Sweden.  In  comparison  to  the  period  2008-201047,  the  overall 
number increased slightly over 2011-2013. Of those resettled, 5 133 belonged to particularly 
vulnerable groups48. 

B) Implementation through direct management 

Community  actions  were  implemented  through  direct  management.  Around  3.2 %  of 
available ERF funding was used to finance these transnational actions or actions of interest to 
the  European  Union.  The  total  programmed  EU  contribution  under  direct  management  was 
EUR 12.7 million,  of  which  EUR  11.5  million  was  allocated  to  Community  actions  and 
EUR 1.2 million  spent  on  procuring  studies49.  The  envelope  was  40 %  smaller  than  that  for 
the previous (2008-2010) period, due to the creation of EASO in 2010.  

Of  the  15  projects  selected50  between  2011  and  2013,  14  closed51  projects  were  chiefly 
concentrated in 2011 and 2012. Absorption rates ranged between 17 % and 99 %. The average 

46   Source: SFC2007 (27.02.2017). These data were aggregated manually. 
47   4 378 persons were resettled from 2008 to 2010. 
48   Women at risk and unaccompanied minors. 
49   Study  on  the  feasibility  of  developing  Eurodac  into  a  supporting  tool  for  the  CEAS,  Study  on  the 

implementation of the Qualification Directive and Study on the implementation of the CEAS. 

50   Grant agreements signed between the European Commission and awarded beneficiaries. 
51   Projects are regarded as closed/completed once the technical implementation report has been received and 

approved and final payment (and de-commitment, if applicable) has been made. 

14 

 
 
 
                                                            
 
 
 
 
 
absorption  rate  per  project  (87 %52)  is  higher  than  for  shared  management.  Rates  increased 
over  time  for  projects  under  direct  management,  suggesting  that  stakeholders  developed 
greater expertise in the course of the ERF cycle. 

9  Answers to the evaluation questions 

9.1  Relevance 

EQ5: To what extent did the ERF objectives correspond to the needs related to receiving and 
bearing the consequences of receiving refugees and displaced persons? 

EQ6:  To  what  extent  did  the  ERF  actions  correspond  to  the  needs  related  to  receiving  and 
bearing the consequences of receiving refugees and displaced persons? 

Main  conclusions:  Overall,  Member  States  confirmed  that  the  formulation  of  ERF  funding 
priorities and eligible actions reflected their needs. In the context of greater asylum flows, the 
emergency  measures  were  particularly  relevant  to  address  immediate  emergency  situations. 
However,  the  need  for  greater  solidarity  and  responsibility-sharing  between  Member  States 
could have been better translated into clear objectives and priorities at EU and national level. 
Furthermore, ERF annual programmes sometimes lacked a strategic approach. The design of 
the  allocation  system  (based  on  historic  inflows)  did  not  always  properly  reflect  Member 
States’  actual  needs  in  view  of  fast  changing  refugee  trends.  However,  Member  States  that 
had to face the highest migration pressure, such as EL, BG, IT and MT, counter balanced this 
lack of flexibility of the Fund by making use  of the Emergency  Measures mechanism. This 
lack of flexibility of the distribution key is an issue which would need to be addressed in the 
future Multiannual Financial Framework53. 

Relevance of ERF funding priorities 

The underlying purpose of the ERF was to improve solidarity between Member States, help 
them to  share the burden of receiving asylum applicants  and address the need  to  harmonise 
asylum systems in line with the development of the CEAS.  

During  the  period  under  evaluation,  most  Member  States  had  pressing  needs  to  improve 
reception conditions for asylum applicants and to facilitate the integration of beneficiaries of 
international  protection.  The  need  to  develop  reception  conditions  with  higher  standards, 
fairer  and  more  effective  asylum  procedures  and  the  importance  of  raising  EU  citizens’ 
awareness of beneficiaries of international protection and asylum applicants was underlined in 
the 2005 impact assessment and constitutes an important element of the CEAS.  

The ERF priorities responded to most of the needs during the implementation period. One of 
the major characteristics of the period was the sudden and unforeseen increase of migratory 
pressure, which led to a drastic increase in the number of asylum applications in Europe, with 
great  variation  across  Member  States.  The  ERF  responded  well  to  this  crisis.  However,  the 
evaluation showed that the lack of harmonised asylum systems, in line with the development 

52   See Annex 6 for more details on programmed and awarded contributions for ERF direct management. 
53   The issue seems to be still valid under the AMIF, although specific actions and emergency assistance have 

helped to introduce more flexibility. 

15 

 
 
                                                            
of the CEAS, emerged as a key problem. National differences in legal frameworks related to 
asylum  remain,  despite  the  CEAS,  and  continue  to  lead  to  secondary movements54  between 
Member States, of asylum applicants.  

The  solidarity  dimension  (the  need  for  a  ‘balance  of  effort’  between  Member  States)  could 
have  been  strengthened  through  clearly  formulated  priorities  of  the  ERF.  As  a  result,  these 
dimensions were insufficiently translated into the Member States’ annual programmes which 
focussed  on  the  national  needs;  for  instance,  few  actions  related  to  the  transfer  of  persons 
(relocation) and implementation of the Dublin Regulation. 

Relevance of ERF-eligible actions 

All Member States confirmed that the ERF actions met their identified needs in the field of 
asylum.  The  diversity  of  eligible  actions55  ensured  that  almost  all  needs,  at  both  EU  and 
national  level,  were  addressed  adequately  and  that  new  challenges  could  be  met.  Member 
States56 were also provided with financial incentives to design and implement a resettlement 
strategy (see Effectiveness of the ERF at results level under section 9.2 for more details about 
these incentives). The evaluation study assessed that these additional financial incentives were 
to some extent effective in fostering responsibility-sharing. This can be evidenced in the fact 
that the number of persons resettled in this way doubled in the 2011-2013 period compared to 
the  2008-2010  period57.  The  United  Kingdom  and  Finland  particularly  underlined  in  their 
national  evaluation  reports  the  positive  impact  of  these  forms  of  support  on  their 
implementation  of  resettlement.  Such  incentives  could  have  also  been  used  for  priority  2 
measures and the transfer of persons (relocation); for the latter, the voluntary mechanism was 
only partially relevant as very few actions related to relocation were implemented 

Overall,  the  limitation  of  resources  was  a  problem  for  Member  States  –  especially  those 
facing  a  high  influx  of  refugees,  for  which  the  new  challenges  were  not  matched  by  an 
increase  in  the  allocated  budget.  The  allocation mechanism  was  based  on  historic  (previous 
three  years’)  inflows  and  did  not  reflect  Member  States’  current  and  evolving  needs.  As 
showed on the graph below, Member States that received, generally speaking and in absolute 
terms,  the  smallest  number  of  asylum  seekers  tended  to  have  nonetheless  high  levels  of 
programmed budget.  

54   According to the Dublin system, asylum applicants cannot choose the EU country where their application 
will be processed. However, some migrants seek to avoid registration and fingerprinting and then move on 
to  the  country  where  they  wish  to  get  asylum  and  settle  which  create  an  incentive  for  asylum  shopping. 
These  secondary  movements  create  imbalances  in  the  distribution  of  asylum  applicants  and  place 
disproportionate pressure on the favoured destination countries. 

55   31 eligible actions grouped into five categories relating to the ERF’s specific objectives:  

reception conditions and asylum procedures;  
integration;  

(i)  
(ii) 
(iii)   enhancement of Member States’ capacity to develop, monitor and evaluate their asylum policies;  
(iv)   resettlement; and  
(v)   transfer of people (relocation). 

56  UK and SE were the main beneficiaries of the resettlement programme. 
57   9 058 persons were resettled in 2011-2013 compared to 4 378 resettled in 2008-2010. 

16 

 
 
 
                                                            
Number of Asylum applicants and Programmed EU contribution per 
MSs 
(period 2011-2013) 

1.000.000

900.000

800.000

700.000

600.000

500.000

400.000

300.000

200.000

100.000

0

5,07 

5,02 

3,87 

4,27 

3,84 

2,40 

2,17 

1,80 

0,15 0,16 0,17 0,01 

0,21 0,17 0,14 0,25 

0,76 

0,45 

0,24 

0,15 

0,99 

1,03 

0,79 

0,51 

0,80 

1,41 

0,49 

EE LV SI HR SK LT PT CZ IE RO CY LU MT ES BG FI EL PL NL BE UK AT IT HU FR SE DE

Number of Asylum Applicants (period 2011-2013)

Programmed EU Contribution (period 2011- 2013)

EMAS Net Contribution

6

5

4

3

2

1

0

)
€
s
n
o

i
l
l
i

m

(

s
S
M

r
e
p
n
o
i
t
u
b
i
r
t
n
o
C
U
E
d
e
m
m
a
r
g
o
r
P

)
3
1
0
2
-
1
1
0
2
d
o
i
r
e
p
(

s
t
n
a
c
i
l

p
p
a
m
u
l
y
s
A
f
o
r
e
b
m
u
N

For  instance,  HU  and  IT  received  the  same  number  of  asylum  applicants  (around  240  000 
applications  in  each  Member  State)  whereas  the  programmed  EU  contribution  was  much 
higher  in  IT.  As  another  example,  DE  was  the  country  with  the  higher  number  of  asylum 
applicants,  namely  around  936  000  applications  were  submitted,  but  the  programmed  EU 
contribution  in  DE  and  UK  was  the  same,  whereas  the  UK  received  only  160  000  asylum 
applicants.  

The  allocation  system  was  a  problem  of  the  Fund's  legal  base.  The  Fund  thus  lacked 
flexibility in adapting quickly to changing Member State needs. However, Member States that 
had to face the highest migration pressure counter-balanced this lack of flexibility by making 
use of the Emergency Measures mechanism. Indeed, as it can be seen from the graph above, 
the countries most  affected by the sudden arrival of migrants, such as EL, BG, IT and MT, 
saw their allocations increase by 97 % over the 2011-2013 period. 

Flexibility  was  also  ensured  through  the  possibility  for  the  Member  States  to  modify  their 
national programmes at any time and to reallocate Funds in the area which required a stronger 
financial support.  

Under the AMIF, the distribution key for the basic allocation to Member States is based on 
the 2011-2013 allocation for the ERF, the EIF and the RF. A certain flexibility of the funding 
instrument  can  be  assured  by  the  resources  allocated  to  specific  actions58  and  emergency 

58   Article 16 and Annex II of Regulation (EU) No 516/2014. An additional amount can be granted to Member 
States provided that the additional amount is earmarked as such in the national programme and that it is used 
to implement the specific actions set in Annex II to the Regulation. These specific actions mainly include 
actions implemented by more than one Member State (joint action) in the field of integration, joint return 
operations, joint initiative in the field of legal migration, joint reintegration projects etc.  

17 

 
 
 
                                                            
 
 
 
 
 
 
 
 
 
 
 
 
 
assistance.  However,  during  the  2015/2016  crisis,  the  lack  of  flexibility  to  modify  the 
distribution  key  or  to  shift  funds  from  one  Member  State  to  another  was  apparent.  The 
2015/2016 migrant crisis showed that at a political level, several Member States have largely 
rejected  the  idea  of  responsibility-sharing  of  asylum  applicants  developed  by  the  European 
Commission. For the future multiannual financial framework, the allocation key must be more 
flexible,  for  example  by  providing  the  legal  basis  for  regular  updates  or  limiting  the  basic 
allocation and keeping a 'reserve' that can be distributed later during the implementation of the 
Multiannual Financial Framework.  

Relevance of priorities in the annual programmes 

Overall,  Member  States  considered  the  annual  programmes  as  reflecting  their  immediate 
national needs and some of them based their own priorities on the ERF priorities.  

Although  all  three  priorities  were  of  interest  to  all  Member  States,  91 %  of  the  total 
programmed budget was allocated to priority 1, which corresponds to the ‘implementation of 
the principles and measures set out in the Community acquis in the field of asylum, including 
those related to integration objectives. This was the priority best aligned with national needs. 
Central and Eastern Europe Member States59, where reception capacities were less developed, 
used priority 1 to improve reception capacities and  conditions. Member States with existing 
reception  capacities  focused  their  budget  on  priority  1  due  to  the  high  number  of  asylum 
applicants with basic needs.  

Five Member States made significant use of priority 260. Only the UK dedicated over 10 % of 
its budget to projects under priority 361. While the proportion of funds allocated to priorities 2 
and 3 was relatively small, Member States still found them to be of relevance.  

9.2  Effectiveness 

EQ1:  To  what  extent  did  the  ERF  actions  contribute  to  the  achievement  of  the  objectives 
defined in Articles 2 and 3 of Decision No 2007/573/EC and to the priorities defined by the 
strategic guidelines (Decision No 2007/815/ЕС)? 

Main conclusions: Overall, the ERF was evaluated as having been effective at national level, 
especially  for  newer  Member  States,  where  it  enhanced  the  capacity  to  establish  national 
asylum  systems.  The  evaluation  showed  that,  in  the  EU-15  Member  States,  the  ERF  was 
particularly  effective  in  improving  reception  conditions  for  asylum  applicants  and  the 
integration  of  beneficiaries  of  international  protection  but  with  some  difficulties  in 
implementation.  The  progress  registered  by  the  Fund  in  the  areas  of  relocation  and 
resettlement was marginal.  

Some limitations emerged when it came to assessing the overall effectiveness of the ERF at 

59   SI, EE, HU, LT, CZ, SK, HR, CY and BG. 
60   Development  of  reference  tools  and  evaluation  methodologies  to  assess  and  improve  procedures  for 
examining  claims  for  international  protection  and  to  underpin  administrative  structures  in  response  to  the 
challenges of greater cooperation between Member States. 

61   Actions helping to enhance responsibility-sharing between Member States and non-EU countries (optional 

priority). 

18 

 
 
                                                            
EU level.  The ERF monitoring and evaluation system did not allow for a detailed follow-up 
and  assessment  of  the  performance  of  the  Fund62.  Set  indicators  were  rather  weak  and 
monitoring data were not fully reliable. 

Complications  as  regards  the  construction  and  implementation  of  a  CEAS  also  remain.  The 
ERF had only moderate impact in terms of improving solidarity and responsibility-sharing. A 
better cooperation between the Member States with a centralised and harmonized EU asylum 
system should, in the future, lead to a more effective and coherent EU framework in the area 
of asylum. 

Effectiveness of the ERF at output level 

The  evaluation  concluded  that  all  Member  States,  with  some  minor  exceptions63,  achieved 
their quantitative targets64. In some cases, targets were over-achieved due to the rise in asylum 
flows and the fact that a wider target group benefited from the projects’ outcomes.  

However,  the  effectiveness  and  impact  of  the  ERF  projects  was  difficult  to  measure  in 
qualitative  terms  since  Member  States  took  different  approaches  (see  section  6  on 
methodology).  

Several sources  suggest that the projects in the new Member States focused more on filling 
gaps  and  addressing  asylum  applicants'  basic  humanitarian  needs,  mainly  accommodation. 
Conversely, in the EU-15 Member States, projects mainly involved activities to improve the 
well-being of those in the target groups. 

Activities to encourage participation by target groups and to improve the professionalism of 
project beneficiaries were organised. However, the effectiveness of the Fund was hampered 
when  the  implementation  of  projects  faced  certain  internal  and  external  difficulties,  namely 
the  long  pre-financing  period  and  insufficient  capacity  among  ERF  beneficiaries  to  handle 
delays in receiving the funding. The lack of interest among target groups themselves was also 
a  problem  in  seven  Member  States65.  Finally,  the  economic  crisis  (which  accompanied  the 
asylum  crisis  between  2011  and  2013)  with  cuts  to  public  resources,  higher  unemployment 
and discrimination against refugees all had a negative impact on project implementation and 
generated among stakeholders the feeling of a degree of under-achievement. 

Effectiveness of the ERF at priorities and policy areas level 

Effectiveness as regards reception conditions and asylum procedures 

EQ1A: To what extent did the ERF 2011-2013 actions contribute to the reception conditions 

62   Under  the  AMIF,  the  European  Commission  is  making  considerable  efforts  to  address  this  issue  by 
developing a common monitoring and evaluation framework with active cooperation of the Member States. 

63   UK and LU. 
64   Indicators mostly related to a number of asylum seekers and refugees to be received and/or integrated, and 
did not detail qualitative aspects under Priority 1 (how the reception and integration and asylum procedures 
would be improved) or expected progress under Priority 2 (what is expected from the development of tools 
and methodologies and from the improvement of administrative structures). 

65   BG, FI, FR, LU, PT, RO and UK. 

19 

 
 
                                                            
and asylum procedures? 

Main conclusion: In the NERs, all Member States confirmed that the ERF provided a high 
level of support for the improvement of reception conditions of asylum applicants and asylum 
procedures. On the other hand, the sharp increase in asylum flows in some cases reduced the 
quality of the services provided.  

The  improvement  of  reception  conditions  in  Member  States  was  one  of  the  main  areas  of 
focus  of  the  ERF.  Of  the  projects  co-financed  by  the  ERF,  47 %  concerned  reception 
capacities,  reaching  almost  two  thirds  of  asylum  applicants66.  All  Member  States  stated 
clearly in their NER that the ERF provided strong support in this area. The main results of the 
improvements to reception conditions and asylum procedures were as follows: 

  measures  relating  to  accommodation,  i.e. the  creation  and  enlargement  of  reception 
centres,  had  a  visible  impact,  with  a  total  of  87 000  beneficiaries.  ERF  funding  was 
used to increase accommodation capacities in eight Member States67, improve safety 
and modernise accommodation centres; 

 

improved social assistance and legal procedures (measures that directly facilitated the 
processing  of  asylum  claims  through  social,  legal  and  administrative  counselling  for 
around  662 000  beneficiaries).  Emergency  measures  reduced  the  backlog  in  asylum 
applicants procedures in Greece; and 

  medical and psychological care for asylum applicants proved to be a strong source of 
additional support for health insurance systems, reaching around 269 000 people. 

Overall, quantitative targets (set in the multiannual programmes by the Member States at the 
start  of  the  Fund)  were  over-achieved  in  all  areas  (total  achievement  rates  amounted  to 
140 %),  notably  because  of  the  increase  in  migration  flows.  Moreover,  the  migration  crisis 
presented a stronger impact in 2014, when it was no longer possible to modify the targets set 
in the NPs (the latest adoption was in 2013), hence the overachievement mainly refers to the 
last year of implementation of the Fund. Complementary data and information were collected 
by  the  evaluators  through  interviews  with  the  national  authorities  and  key  stakeholders,  to 
complete the picture and carry out also a qualitative assessment.  

With  the  overachievement  some  difficulties  arose,  since  the  planning  phase  of  the  project 
cycle  management  proved  to  be  more  complicated  and  the  values  of  qualitative  indicators 
may  have  been  reduced  (i.e. in  AT,  it  was  the  case  of  projects  aiming  at  providing  social 
counselling as the number of asylum seekers was higher than planned: while the number of 
persons assisted increased, the number of hours of assistance delivered per person could not 
always be offered as planned).  

Effectiveness as regards integration of people in the target groups 

66   Over a million new migrants entered the 27 Member States between 2011 and 2013. 
67   EL, FR, LT, RO, BE, BG, CY and LU. 

20 

 
 
                                                            
EQ1B: To what extent did the ERF 2011-2013 actions contribute to the integration of persons 
referred in Art 6 (target group)68? 

Main  conclusions:  Stakeholders  assessed  the  ERF’s  contribution  to  integration  as  positive. 
Several highlighted that refugees encountered difficulties in entering the labour and housing 
markets due to a deteriorating social and economic environment. 

Improving the process of integrating refugees was another main priority of the ERF. The Fund 
supported  42 %  of  all  integration  projects  supported  by  the  Fund,  which  involved  social 
assistance  with  administrative/judicial  formalities,  counselling  and  legal  aid,  and  language 
training;  these  reached  more  than  95 000  people69.  Member  States’  NERs  gave  a  mostly 
positive  assessment  of  the  ERF  contribution  to  changes  and  improvements  in  the  field  of 
integration. 

As regards integration, the evaluation highlights projects with a focus on: 

 

recognising refugees’ competencies and skills; 

  organising  common  activities  with  local  communities  and  raising  awareness  among 

EU citizens; and 

 

facilitating refugees’ entrepreneurship.  

The main outputs of integration measures were as follows: 

  social counselling contributed to the integration of over 95 000 refugees in 21 Member 
States. Such measures were particularly relevant in PL, FR and AT, with 45 000 final 
beneficiaries (asylum applicants) assisted through activities such as cultural dialogue 
with civil society, language training, sport events, cooking and cultural events; 

  accommodation  projects  benefiting  over  61 000  refugees  were  very  positive 

considering the difficulty for them to enter the housing market; and  

  a  Community  action  was  aimed  at  developing  an  UNHCR  (the  United  Nations 
Refugee  Agency)  online  integration  tool  containing  data  pertaining  to  refugee 
integration and designed to facilitate the exchange of information and raise awareness. 

On each topic, the number of target group people reached exceeded what was planned (overall 
achievement rates: 138 %), except in FI, PO, LU and the UK. 

Some Member States stressed the possible existence of specific obstacles to integration linked 
to  the  social  and  economic  environment,  including  access  for  beneficiaries  of  international 

68   The  target  groups  comprise  the  following  categories:  (a)  any  third-country  national  or  stateless  person 
having the status defined by the Geneva Convention and who is permitted to reside as a refugee in one of the 
Member States; (b) any third-country national or stateless person enjoying a form of subsidiary protection 
within  the  meaning  of  Directive  2004/83/EC;  (c)  any  third-country  national  or  stateless  person  who  has 
applied for one of the forms of protection described in points (a) and (b); (d) any third-country national or 
stateless person enjoying temporary protection within the meaning of Directive 2001/55/EC; (e) any third-
country national or stateless person who is being or has been resettled in a Member State. 

69  To  be  compared  with  73,000  new  asylum  applications  accepted  during  2011-2013.  France,  Greece, 

Germany, and Hungary were the MS having the higher number of beneficiaries in absolute data. 

21 

 
 
                                                            
protection to the labour (AT, DE, LV, PT, SK) and housing markets (NL), so that integration 
measures were considered as particularly relevant by those Member States.  

Effectiveness as regards enhancing Member States’ capacity to develop, monitor and evaluate 
their asylum policies 

EQ1C:  To  what  extent  did  the  ERF  2011-2013  actions  contribute  to  the  enhancement  of 
Member States’ capacity to develop, monitor and evaluate their asylum policies in the light of 
their  obligations  under  existing  and  future  Community  legislation  relating  to  the  common 
European  asylum  system  (in  particular  practical  cooperation  activities  between  Member 
States)? 

Main  conclusions:  The  Fund  did  not  prove  to  be  widely  effective  as  regards  enhancing 
Member States’ capacity to  develop,  monitor  and evaluate their asylum policies. Successful 
projects  strengthened  cooperation  between  Member  States  in  the  field  of  country  of  origin 
information  (COI).  Community  actions  could  have  been  more  often  used  as  a  tool  to 
strengthen the cooperation between Member States, although the creation of EASO in  2011 
led to a progressive transfer of these activities.  

In most Member States, projects focused on improving the quality of asylum procedures (103 
projects),  evaluating  asylum  policies  and  procedures  (40)  and  measures  to  collect/share 
country  of  origin  information  (32).  These  projects  were  those  mentioned  most  often  by 
Member States70as having been the most effective. They aimed to improve access to and the 
sharing  and quality  of  information  in  Member  States  and  typically  encouraged  stronger  and 
new  forms  of  cooperation  between  Member  States.  For  example,  under  the  MedCOI 
Community  action,  three  Member  States71  developed  a  sustainable  medical  database.  The 
ERF supported Finland’s Tellus project, which contributed in an initial phase to the launch of 
the EASO COI portal, which is now accessible to all Member States. The EASO Agency was 
set up in 2011, its main priority being to enhance practical cooperation among Member States 
on  asylum  by  facilitating  the  exchange  of  information.  This  implies  that  some  activities 
financed  under  the  ERF  Community  Actions,  in  particular  support  of  transnational 
cooperation between Member States on asylum related issues, were progressively transferred 
to EASO. 

A  number  of  Member  States  also  focused  efforts  on  elaborating  new  methodologies72  and 
monitoring  and  improving  procedures73.  The  ERF  contributed  to  improve  Member  States’ 
capacity  to  evaluate  their  asylum  policies  and  to  elaborate  new  methodologies,  replacing 
individual  practices,  for  working  with  the  most  vulnerable  asylum  applicants.  It  also 
contributed  to  improve  their  capacity  to  adapt  national  legislation  more  rapidly  to  the 
Community  acquis.  As  an  example,  FR’s  project  to  modernise  the  EURODAC  system  was 
considered  a  time-saving  investment  which  led  to  FR  adapting  promptly  to  the  Dublin  II 

70   AT, BG, CY, DE, EE, FR, IT, LT, LU, MT, NL, PL, PT and SI. 
71   AT, BE and NL. 
72   DE, IT, LT, LU, NL and SI. 
73   MT, PT and RO. 

22 

 
 
 
                                                            
Regulation74 and helped improve the productivity and stability of its national asylum system. 
BG,  DE,  BE,  DE  and  RO  were  of  the  opinion  that  seminars  and  training  courses  had 
improved the skills and knowledge of a wide range of staff. 

Notwithstanding the above positive outcomes, action to improve Member States’ capacity to 
develop, monitor and evaluate their asylum policies could have been more effective: priority 2 
accounted  for  only  7%  of  total  programming.  As  regards  the  shortcomings  identified,  all 
interviews  done  during  the  evaluation  highlighted  that  the  significant  and  unpredictable 
increase in the number of asylum applicants led Member States to focus on short-term effects 
at  the expense  of  long-term  impact.  They  also  pointed  out  that  Member  States’  cooperation 
was  insufficiently  developed  to  ensure  effectively  the  development  of  future  legislation 
pertaining  to  the  CEAS75.  National  projects  and  Community  actions  were  perceived  by  the 
persons interviewed as potentially strong tools for cooperation and methodological progress, 
but insufficiently used.  

Stakeholders'  consultations  also  showed  that  Community  actions  could  have  been  more 
effective.  In  particular,  over  half  the  respondents  lacked  knowledge  of  them  and  some 
beneficiaries considered that the funding was somewhat low in the light of the high level of 
competition in the project management area and in the organisation of projects. 

Effectiveness as regards the resettlement of persons (Article 6(e)) — priority 3 

EQ1D:  To  what  extent  did  the  ERF  2011-2013  actions  contribute  to  the  resettlement  of 
persons referred to in Article 6(e)? 

Main conclusions: During the 2011-2013 period, the ERF better contributed to resettlement 
activities  compared  to  the  previous  period  (2008-2010),  mainly  because  the  financial 
allocation was higher. However, it emerged that resettlement operations remained marginal at 
EU level, with only a third of Member States participating. The European Commission, under 
AMIF Regulation, took specific measures with the aim of promoting the resettlement, such as 
increasing the lump sum granted per resettled person.  

Under  the  ERF  several  aspects  of  resettlement  have  been  agreed  at  Union  level  for  the 
purpose  of  providing  targeted  financial  incentives,  namely  through  common  resettlement 
priorities and lump sums for each resettled person. Between 2011 and 2013, a total of 9 058 
people  were  resettled  by  12  Member  States76  using  the  ERF,  with  four77  using  the  ERF 
exclusively to  implement  such  operations, and with  a number of resettled people increasing 
over time78. Compared with 2008 - 2010 the number of resettled people doubled. One of the 

74   Council  Regulation  (EC) No.  343/2003  of 18  February  2003  establishing  the  criteria and  mechanisms  for 
determining  the  Member  State  responsible  for  examining  an  asylum  application  lodged  in  one  of  the 
Member States by a third-country national. 

75   The CEAS foresees the establishment of a fully efficient, fair and humane asylum policy which functions 
effectively both in times of normal and in times of high migratory pressure. It ensures a fair allocation of 
asylum applications among Member States and provides for a common set of rules at EU level to simplify 
and  shorten  the  asylum  procedure,  discourage  secondary  movements  and  increase  the  prospect  of 
integration. 

76   BE, CZ, DE, FR, FI, ES, HU, IE, NL, PT, SE and UK. 
77   BE, DE, HU and PT. 
78   2 463 in 2011, 2 786 in 2012 and 3 809 in 2013. 

23 

 
 
                                                            
main reason being that, for Member States receiving the fixed amount for resettlement for the 
first time, EUR 6 000 was allocated per resettled person instead of EUR 4 00079. Also, EUR 
5 000 per resettled person was allocated for Member States that had received the fixed amount 
for resettlement once in the previous year. The UK and FI which have actively implemented 
the  resettlement,  underlined  the  increase  in  the  fixed  amount  per  resettled  person  as  a 
significant incentive. 

In  addition  to  the  funding  of  resettlement,  ERF  projects  in  some  Member  States80  funded 
preparations for resettlement or aimed to establish and develop resettlement programmes. 

Although  the  ERF  provided  incentives  for  some  Member  States  to  conduct  resettlement 
operations, it also appeared that these were too marginal at EU level. A significant number of 
Member States81 did not implement any resettlement projects. This is also true of Community 
actions, under which only one resettlement project was financed. For several Member States, 
resettlement was not considered as a priority, with therefore a lack of interest to engage in or 
to develop resettlement activities further. Increased migratory pressure might also have been 
considered by Member States as a reason not to engage more in resettlement.   

The  European  Commission  has  promoted  the  principle  of  resettlement  in  the  AMIF 
Regulation82  with  the  progressive  establishment  of  a  Union  Resettlement  Programme.  The 
Fund  should  provide  targeted  assistance  in  the  form  of  financial  incentives:  EUR  6  000  or 
EUR  10  000  for  each  resettled  person.  In  addition,  the  AMIF  legal  basis  provides  that  the 
budget  allocated  to  each  national  programme  for  the  purpose  of  resettlement  could  not  be 
used for other purposes83, i.e. the MS could not request to move this budget to other regular 
actions84.  The  European  Commission,  in  cooperation  with  the  EASO  should  monitor  the 
effective implementation of resettlement operations supported under the Fund. For the period 
2014-2016, Member States reported the resettlement of over 16 000 people.  

Effectiveness as regards the transfer of persons under international protection (relocation) 

EQ1E:  To  what extent did  the ERF 2011-2013 actions  contribute to  the transfer of  persons 
falling within the categories referred to in Article 6(a), (b) and (c)? 

Main  conclusions:  The  ERF  had  a  very  limited  impact  as  regards  the  transfer  of  persons 

79   Decision  No 281/2012/EU  (revising  Decision  No 573/2007/EC)  increased  the  fixed  amount  for  Member 

States resettling people for the first time. 

80   BE, BG, HU, NL, RO, SE and UK. 
81   AT, CY, EE, EL, HR, IE, IT, LT, LU, LV, MT, PL, SI and SK. 
82   According  to  Article  17  (3)  of  Regulation  (EU)  No  516/2014  establishing  the  Asylum,  Migration  and 

Integration Fund. 

83  Regulation  (EU)  No  516/2014,  Article  17.9,  last  sentence  of  the  paragraph:  "Those  amounts  shall  not  be 

transferred to other actions under the national programme". 

84   In  2016,  the  Commission  proposed  a  regulation  establishing  a  Union  Resettlement  Framework  and 
amending Regulation (EU) No 516/2014 . The aim of the legislative proposal is to create a more structured, 
harmonised,  and  permanent  framework  for  resettlement  across  the  Union  with  the  view  to  create  a  more 
robust and collective approach. The objective is to reduce the divergences among the national resettlement 
practices  and  put  the  Union  in  a  stronger  position  to  also  achieve  its  policy  objectives  globally.  It  would 
allow  the  Union  to  make  a  single  pledge  to  contribute  to  global  resettlement  initiatives.  The  proposal 
foresees  that  Member  States  will  be  entitled  to  EUR  10  000  from  the  Union's  budget  for  each  resettled 
person. 

24 

 
 
                                                            
benefitting from international protection (relocation). At the level of Member States through 
the  ERF,  no  transfer  of  persons  under  international  protection  took  place,  and  only  one 
Community  action  has  been  financed  over  the  period.  As  a  consequence  of  this 
underperformance under ERF, the European Commission took specific measures under AMIF 
with the aim of responsibility-sharing between Member States on relocation. 

Member  States’  projects  involving  the  transfer  of  persons  benefitting  from  international 
protection  were  under-developed  and  the  evaluation  clearly  assessed  them  as  marginal.  The 
main objective of responsibility-sharing between Member States was not achieved. Part of the 
reasons which explain the limited impact is the considerable increase in arrivals of the number 
of refugees and the newer Member States’ relative lack of experience of managing projects in 
the field of relocation. However, there was also very limited political interest from the side of 
Member  States  to  engage  in  transferring  beneficiaries  of  international  protection.  In  the 
absence of any obligation, this made it very difficult to achieve meaningful results. More, the 
ERF did not foresee any additional financial incentive for intra-EU transfers of persons under 
international protection as it was the case for resettlement.  

This  explains  why  the  European  Commission  financed  through  ERF  Community  action  the 
EUREMA project, which provided an organised framework for preparing and implementing 
relocation and thus partially achieved planned outputs. It was innovative in that it fitted into 
the  framework  of  a  previous  pilot  project  and  relocated  264 people.  Unexpected  positive 
results  were  observed  enabling  Member  States  to  learn  lessons  for  the  future  as  regards 
improving and facilitating relocation. 

In the AMIF Regulation85, the European Commission has promoted the principle of relocation 
by  fixing  financing  resources  which  could  not  be  transferred  to  other  actions  under  the 
national  programmes  of  the  Member  States.  By  blocking  financial  resources  specifically 
dedicated to relocation, the system will probably be more effective than under the ERF.  

Effectiveness as regards emergency measures 

EQ2: To what extent did the ERF emergency measures contribute to the achievement of the 
Fund’s objectives and priorities? 

Main  conclusions:  All  stakeholders  agreed  that  the  emergency  measures  were  highly 
effective  in  meeting  the  specific  urgent  needs  of  Member  States  facing  high  migration 
pressure,  which  arose  mainly  from  external  factors  (increase  in  the  number  of  asylum 
applicants)  and  so  could  not  be  addressed  in  full  in  the  national  programmes.  The  rapid 
reaction of the European Commission with fast awarding of EU funding was highlighted as 
positive. 

In  2011-2013,  a  total  of  11  Member  States  facing  urgent  needs  implemented  51  emergency 
measures  using  a  programmed  budget  of  EUR  67.1  million.  The  aim  was  to  provide 
assistance in fast-moving circumstances with a focus on immediate results.  

85   According  to  Article  18  (3)  of  Regulation  (EU)  No  516/2014    establishing  the  Asylum,  Migration  and 

Integration Fund. 

25 

 
 
                                                            
As first countries of arrival in the Mediterranean, Italy and Greece were the main beneficiaries 
of  emergency  measures.  Malta  and  Cyprus  also  received  assistance  following  an  important 
influx  of  migrants  into  these  Member  States  in  order  to  complement  their  small  national 
allocations,  which  were  too  low  to  address  this  sudden  arrival  of  migrants.  The  additional 
assistance  represented  71 %  of  Cyprus’s  2013  budget  and  76 %  of  Malta’s.  Most  of  the 
measures  were  implemented  in  2013,  in  response  to  the  EU’s  asylum  crisis,  and  concerned 
eight Member States86 with immediate needs.  

Funding for these types of action was assessed case by case. The main actions focused on: 

 
 

improving accommodation capacities and renovating reception centres;  
recruiting  additional  staff  in  the  main  institutions  dealing  with  reception  and  asylum 
procedures; 

  medical and psychological assistance; 
  providing for basic needs; and 
 

legal assistance and translation services. 

in  a  short  (six-month) 

Generally,  emergency  measures  were  perceived  as  very  effective  and  described  by  some 
Member  States87  as  success  stories.  This  is  explained  by  the  fact  that  the  measures  are 
designed  to  cover  specific  needs  that  are  easily  quantifiable  (e.g. the  number  of  additional 
places in reception centres or the procurement of new medical tools/instruments/equipment), 
allowing  some  Member  States  to  extend  economies  of  scale  with  greater  effectiveness  and 
efficiency 
the  rapid 
timespan.  All 
implementation  of  the  emergency  measures  as  having  been  essential  in  meeting  the  urgent 
needs.  The  evaluation  also  pointed  out  that  the  European  Commission  responded  well  by 
being very reactive to this type of difficult situation. Respondents underlined that the situation 
would  have  been  worse  without  the  additional  assistance  provided  through  emergency 
measures.  On  the  other  hand,  three  Member  States89  considered  the  short  timeframe  for  the 
implementation of measures as having reduced their effectiveness as the requirement to spend 
a large amount of money within a short timeframe proved difficult. …. 

interviewees88  saw 

the 

Even  if  the  political  context  was  difficult,  the  AMIF  instrument  through  its  emergency 
assistance has provided as from 2015 strong support to the Member States most affected by 
the  2015  migrant  crisis.  With  the  arrival  in  the  second  half  of  2015  of  more  than  1  million 
third-country nationals via the Western Balkan route, the mechanism was flexible enough to 
provide at short notice financial assistance to address urgent and specific needs in the event of 
an emergency situation90. For the period 2014-2016, EUR 498 million have been awarded to 
14  Member  States91,  2  EU  agencies92  and  2  international  organisations93  through  AMIF 

86   IT, EL, MT, CY, FR, DE, HU and BG. 
87   BE, CY, EL and MT. 
88   Responsible authorities and beneficiaries. 
89   BG, EL and HU. 
90  

'Emergency situation' means a situation resulting from a heavy migratory pressure in one or more Member 
States  characterised  by  a  large  and  disproportionate  inflow  of  third-country  nationals,  which  places 
significant and urgent demands on their reception and detention facilities, asylum systems and procedures.  

91   AT, BE, BG, HR, CY, FI, FR, DE, EL, HU, IT, NL, SI and SE. 
92   EASO and EUROPOL. 
93   UNHCR and IOM. 

26 

 
 
                                                            
emergency assistance94. Greece, the country most affected by the migrant crisis, is the main 
beneficiary of emergency assistance.    

Effectiveness of the ERF for the development of a common approach for asylum  

The ERF supported Member States in the field of asylum and contributed to the development 
of  reception  conditions  for  asylum  applicants  and  the  integration  of  beneficiaries  of 
international  protection.  It  has  had  a  direct  effect  on  the  field,  supporting  over  a  million 
people  through  over  1 500  projects,  with  an  estimated  coverage  of  56 %  of  all  asylum 
applicants and beneficiaries of international protection.  

However,  while  EU  funding  was  effective  in  achieving  progress  under  the  various  ERF 
specific  objectives,  its  implementation  was  not  accompanied  by  the  development  of  a 
common approach of Member States in the processing asylum applications across Europe. As 
described  earlier,  for  instance,  Central  and  Eastern  Europe  Member  States95  used  the  ERF 
mainly  to  increase  and  improve  reception  conditions,  since  these  were  insufficiently 
developed  (due  to  relatively  low  numbers  of  asylum  applicants  in  the  past).    The  ERF 
contributed  to  a  certain  extent  to  the  implementation  of  the  asylum  'acquis'  in  the  Member 
States.  The main  focus  was  on  the implementation  of  the  Reception  Conditions  Directive96. 
The ERF had little direct influence as regards responsibility-sharing between Member States, 
especially regarding the transfer of persons under international protection (relocation). 

Although  the  results  and  achievements  of  the  ERF  were  assessed  differently  from  one 
Member State to another, the evaluation identified several trends at EU level: 

  The ERF played a crucial role in the context of the asylum crisis that started in 2012 
and provided essential support in situations that called for the adoption of emergency 
measures.  Community  actions  helped  in  urgent  situations  requiring  prompt  and  fast 
intervention  (in  Greece).  In  that  regard,  it  was  also  mentioned  during  the  interviews 
that  the  Fund  contributed  to  the  development  of  a  European  approach  and  to  the 
enhancement  of  cooperation,  for  example  by  developing  constant  dialogue  amongst 
experts.  

  The  ERF  played  a  key  role  in  encouraging  decision-makers  to  involve  the  Member 
States in resettlement operations, as confirmed by Member States that received a fixed 
amount per resettled person. 

  The ERF gave valuable support to Member States’ efforts in financing operational and 
locally based projects focusing on reception conditions for asylum applicants and the 
integration  of  beneficiaries  of  international  protection.  Through  concrete  actions,  the 
ERF contributed to a certain extent to the implementation of the asylum package and 
raised reception standards and living conditions for asylum applicants; 

94  https://ec.europa.eu/home-affairs/sites/homeaffairs/files/what-we-do/policies/european-agenda-

migration/press-material/financial_support_to_mms_under_amif_and_isf_en.pdf 

95  SI, EE, HU, LT, CZ, SK, HR, CY and BG. 
96  Council Directive 2003/9/EC of 27 January 2003 laying down minimum standards for the reception of asylum 
seekers. In the meantime this Directive has been repealed and replaced by DIRECTIVE 2013/33/EU of the 
European  parliament  and  of  the  Council  of  26  June  2013  laying  down  standards  for  the  reception  of 
applicants for international protection (recast). 

27 

 
 
                                                            
  The  ERF  had  a  limited  impact  on  the  development  of  interaction  and  practical 
cooperation  between  Member  States.  The  ERF  funded  a  few  large  transnational 
projects  that  brought  clear  results  but  practical  cooperation  remained  overall  limited 
following the creation of EASO which was entrusted with some practical cooperation 
tasks  from  2010  (leading  to  the  reduction  of  the  CAs’  envelope  in  order  to  free  up 
resources  for  funding  the  EASO),  as  well  as  by  the  emphasis  put  on  emergency 
measures  as  from  2013.  In  addition,  the  results  of  Community  actions  were  not 
sufficiently disseminated at EU level and among national stakeholders.  

  The ERF had a very limited impact in involving Member States in actions involving 

the transfer of people under international protection. 

 Effectiveness of the principle of solidarity 

The  principle  of  solidarity,  common  to  the  four  SOLID  Funds  has  two  components:  the 
solidarity between Member States and the financial solidarity.  

In  terms  of  solidarity  between  Member  States,  while  the  EC  made  significant  efforts  to 
advocate  for  solidarity  and  a  common  EU  approach  in  response  to  present  and  future 
migration challenges, this vision was not always shared by all EU Member States to the same 
extent.  The  objective  of  the  ERF  was  to  support  and  encourage  through  a  solidarity 
mechanism  the  efforts  made  by  the  Member  States  in  receiving  and  in  bearing  the 
consequences of receiving refugees and displaced persons. The ERF had a slow start due to 
Member States willing to protect their sovereignty on managing asylum related issues. When 
the priorities were set, the enhancement of responsibility sharing between Member States and 
third countries remained optional following bitter discussion with Member States. Whilst the 
experience under ERF showed the need to increase the solidarity amongst Member States, this 
remains  a  difficult  subject:  the  current  discussion  on  the  proposal  for  a  revised  Dublin 
regulation shows that consensus on solidarity matters is far from being achieved. 

As explained above (see section 9.1 on Relevance), in terms of financial solidarity, the initial 
allocation  key,  which  is  the  first  means  to  achieve  solidarity,  showed  its  limitation  in  the 
context of the crisis. For this reason, the use of emergency measures proved to be crucial for 
those Member States which were subject to acute and urgent migration pressure. Emergency 
measures partially counter-balanced the lack of flexibility of the distribution key. Indeed, the 
countries most affected by the sudden arrival of migrants saw their allocations increase by 97 
% over the 2011-2013 period thanks to EU funding for emergency measures and partly to the 
annual recalculation of the distribution key. This illustrates how the financial  solidarity was 
achieved. 

9.3  Efficiency 

EQ 3: To what extent were the effects of the ERF achieved at a reasonable cost in terms of 
financial and human resources? 

Main  conclusions:  Although  the  inconsistency  of  monitoring  data  restricted  the  extent  to 
which  it  was  possible  to  assess  the  cost-effectiveness  of  the  ERF,  the  evaluation  concluded 
that  the  actions  were  implemented  efficiently  for  the  most  part.  Management  and  control 
systems, such as internal organisation and a clear distribution of tasks within the responsible 
administrations  for  managing  the  Fund,  contributed  greatly  to  this  result  and  their  benefits 

28 

 
 
were acknowledged by the Member States. 

However,  the  evaluation  showed  that  the  annual  programming  requirements  and  the 
complexity relating  to the existence of separate funds97 led to a high  administrative burden. 
Small Member States had to deal with a combination of high administrative costs and small 
programmes.  Lack  of  experience  and  resources  also  contributed  to  the  high  administrative 
burden. 

Efficiency of ERF programme management 

Overall, RAs considered the ERF costly to manage, although the most experienced Member 
States  were  able  to  reduce  the  administrative  burden.  The  evaluation  estimated  that  the 
administration of national programmes required an average of 0.32 full-time equivalent (FTE) 
per  EUR 1 million  of  ERF  programmed  funding.  The  figure  varies  a  lot  between  Member 
States though (from 0.06 FTE in FR to 2.97 FTE in LV), and is lower for the Member States 
with  the  largest  ERF  programmes.  This  average  is  consistent  with  the  estimate  for  the 
cohesion policy98 (0.44 FTE).  

The EU contribution was also a factor explaining the varying proportions of the ERF budget 
dedicated to management. The Member States with larger ERF programmes were also those 
where the smallest percentage of the EU contribution was dedicated to fund management due 
to a critical mass effect. The main time-consuming tasks (56 % of the RAs’ workload) were 
those  relating  to  ERF  management,  i.e. project  selection,  control  and  verification  of 
deliverables. The workload relating to the preparation of the programmes was estimated at a 
fifth of the total. 

All  Member  States  pointed  to  the  high  administrative  burden  of  ERF  management  (i.e. the 
burden linked to multi-annual and annual programming, and addressing asylum and migration 
issues via three different funds which under the current AMIF has been overcome) but RAs 
regarded  most  of  the  requirements  as  legitimate,  in  that  they  ensured  the  robustness  and 
reliability of the ERF management and control systems. The evaluation showed that, for half 
of the Member States99, the administrative burdens, especially verifications by the RAs, were 
the  most  problematic.  Some  Member  States  tried  to  use  the  same  human  resources  for  the 
ERF as  for the other SOLID funds, in  order to reduce the administrative burden and secure 
efficiency gains. In Member States with less experience of managing EU funding, many costs 
arose from the need to build technical and administrative capacities. 

In some cases, the perception of a high administrative burden applied more at national than at 
EU  level.  For  instance,  FR  and  SE  faced  financial  difficulties  (gaps  in  EU  reimbursement) 

97   For  the  period  2007-13,  almost  EUR  4  billion  was  allocated  for  the  management  of  the  Union's  external 
borders  and  for  the  implementation  of  common  asylum  and  immigration  policies  through  the  General 
Programme “Solidarity and Management of Migration Flows” (SOLID). This General Programme consisted 
of  four  instruments:  External  Borders  Fund  (EBF),  European  Return  Fund  (RF),  European  Refugee  Fund 
(ERF) and European Fund for the Integration of third-country nationals (EIF). 

98   The  governance  and  administrative  structures  and  costs  at  national  and  regional  levels  for  European 
Regional  Development  Fund  (ERDF)  and  Cohesion  Fund  (CF)  programmes  over  the  2007-2013 
programming period were reviewed in 2010. 

99   AT, BE, BU, CY, CZ, DE, ES, FI, LT, PT, SE, SI and UK. 

29 

 
 
                                                            
following  a  misleading  interpretation  of  ‘eligible  costs’.  Other  Member  States100  had 
difficulties  due  to  the  long  pre-financing  phase,  which  caused  cash-flow  problems  for 
beneficiaries  waiting  for  EU  reimbursement,  or  the  lack  of  a  clear  distribution  of 
responsibilities (BE). 

Efficiency of the ERF for beneficiaries 

Time-consuming arrangements for (bi-annual) reporting and payment delays were mentioned 
by some countries as affecting the efficiency of the Fund. On the other hand, administrative 
burden is often linked to institutional learning. The evaluation showed that beneficiaries' size, 
nature  and  experience  of  managing  EU  finding  influenced  their  ability  to  meet  the  ERF 
requirements.  Member  States  and  RAs  convey  the  European  Commission’s  requirements  to 
national beneficiaries, including small organisations, and to build capacity to respond properly 
to  the  requirements.  Overall,  the  estimated  cost  of  meeting  the  European  Commission’s 
administrative requirements is evaluated as high, but reasonable as compared with other EU 
funding. 

Cost-effectiveness of projects 

The  quantitative  data  on  ERF  reported  in  the  NERs  do  not  adequately  reflect  the  actual 
performance of ERF actions in the Member States, but 22 Member States considered the costs 
incurred  as  reasonable.  The  data  cannot  support  any  clear  conclusion,  given  the  lack  of 
comparability across projects and the shortcomings of the available indicators when it comes 
to measuring results and impacts. 

Each  project  targeting asylum  applicants  or  persons  benefiting  from  international  protection 
(priority 1)  reached  an  average  of  around  700  final  project  beneficiaries,  with  an  average 
budget  of  EUR 150 000.  The  average  committed  cost  per  beneficiary  (including  emergency 
measures  varied  widely  between  Member  States  (from  EUR 9 065  in  the  UK  to  EUR 37  in 
SL). This is probably related to the range and the nature of the activities carried out under the 
projects. For instance, UK policy in 2011-2013 was focused on resettlement and integration. 
By nature, this involves high-value projects benefiting a relatively small number of people.  

For  priority 2,  the  average  cost  of  a  project  was  EUR 123 553  and  for  priority  3  it  was 
EUR 301 037. 

Member  States  receiving  a  high  number  of  asylum  applicants  and  beneficiaries  of 
international protection have relatively low budgets per project beneficiary (e.g. only EUR 44 
for  EL).  This  is  linked  to  the  nature  of  the  projects:  these  Member  States  target  projects 
enhancing  reception  conditions  or  meeting  the  basic  needs  of  asylum  applicants.  The 
discrepancies  put  a  question  mark  over  the  reasonability  of  costs  and  the  relevance  of  the 
allocation  system.  In  HU,  for  example,  the  national  budget  allocated  to  asylum  issues  was 
reduced for political reasons and the ERF became more important in terms of meeting asylum 
applicants’ needs. 

100   AT, BE and LT. 

30 

 
 
                                                            
9.4  Sustainability 

EQ4:  To  what  extent  did  the  positive  effects  of  the  ERF  actions  last  after  the  interventions 
were terminated? 

Main  conclusions:  The  evaluation  concluded  that  the  sustainability  of  the  projects'  results  
funded under the 2011-2013 ERF depended on the type of action funded. In general, the ERF 
led  to  the  consolidation  of  sustainable  relationships  with  stakeholders,  but  the  most 
sustainable  actions  (presenting  structural  effects)  were  Member  States’  investments  in 
reception capacities. For other types of project, certain conditions regarding the selection of 
sustainable  projects  could  have  been  set  in  order  to  produce  sustainable  effects.  As  for  the 
sustainability  of  projects  after  the  end  of  ERF  funding,  this  was  more  questioned.  The 
selection phase of AMIF projects involved sustainability as criterion for awarding a project.  

Sustainability of results and impacts 

The  sustainability  of  a  project  is  linked  to  its  nature,  impacts  and  results.  The  creation  of 
reception  capacities  is  highly  sustainable,  especially  for  Member  States  that  lacked  them 
previously.  Analysis  of  the  NERs  and  interviews  with  RAs  and  beneficiaries  reveal  that 
actions  aimed  at  empowering  target  groups,  those  resulting  in  greater  capacities  to  receive 
asylum applicants and those resulting in a concrete change of practice are considered among 
the most sustainable. Actions to improve the drawing-up of studies and to develop platforms, 
websites and new tools or methodologies have also proven to be sustainable, since the outputs 
and materials can continue to be used, provided they are updated and maintained. 

Projects focusing on the integration of beneficiaries of international protection are considered 
less sustainable in transit Member States, where beneficiaries of international protection like 
asylum applicants tend to seek to move on to another country. 

Sustainability of projects 

implementation  of  ERF  projects  produced  side-effects 

The 
improved  project 
sustainability.  Thirteen  Member  States101  highlighted  the  importance  of  capacity-building 
among  stakeholders  in  the  field  of  asylum  as  a  key  factor,  since  expertise  favours  actions’ 
long-term  effectiveness.  Some  Member  States102  commented  that  the  implementation  of 
ERF-funded projects favoured a ‘culture of cooperation’ among stakeholders. 

that 

However,  the  sustainability  of  projects  after  the  end  of  ERF  funding  is  uncertain. 
Beneficiaries’  capacity  to  sustain  the  projects’  effects  is  not  ensured  without  the  support  of 
EU funds. A total of 13 Member States103 stressed that national beneficiaries rely heavily on 
EU  funds  to  implement  projects.  Moreover,  some  RAs  replicate  projects  over  the  years  in 
order to ensure a type of sustainability of their effects.  

For  the  AMIF  during  the  selection  phase  of  projects  by  the  responsible  authority,  the 
sustainability  is  now  included  as  a  criteria  leading  to  the  award  decision.  Through  the 

101   AT, FR, DE, EL, ES, IE, IT, LT, LV, MT, PL, RO and SE. 
102   BG, DE, IE, EL, ES, IT, MT, NL, PL and UK. 
103   AT, CZ, EE, EL, ES, FI, FR, HU, IE, LU, LV, SK and UK. 

31 

 
 
                                                            
guidance document set by the common monitoring and evaluation framework, Member States 
will have to report on the measures adopted to ensure sustainability. 

9.5  Complementarity and coherence 

EQ7: To what extent were the ERF actions coherent with and complementary to other actions 
related  to  asylum,  financed  by  other  EU  financial  instruments  –  including  the  activities  of 
EASO – and from national resources of Member States? 

Main  conclusions:  The  evaluation  concluded  that,  in  most  Member  States,  the  ERF  was 
implemented in coherence with, and complemented, other actions relating to asylum, although 
there  may  have  been  some  overlaps  between  actions  or  EU  agencies.  There  was  however  a 
general  lack  of  awareness  at  national  level  of  the  outputs  of  Community  actions  and  this 
might have led to a certain risk of incoherence. 

Coherence and complementarity at national level 

Of the 27 Member States concerned,  26 affirmed that the ERF was used in coherence with, 
and complemented, other sources of funding, thanks to active cooperation between the ERF 
RAs and the government authorities responsible for managing other EU funds. Also, in most 
Member  States,  coordination  between  the  ERF  and  other  SOLID  funds  was  ensured  by  the 
fact that they were managed by a single RA. 

However, some risks of duplication were identified. The EIF, the European Social Fund and a 
significant number of organisations that operate in the field of asylum at both EU and national 
level  may  have  created  overlaps.  Nevertheless,  RAs  addressed  these  risks  through 
organisational  measures,  e.g. adopting  national  asylum  strategies  and  close  cooperation 
between the stakeholders responsible for managing various funds.  

Lastly, many Member States pointed out that Member States’ coordination arrangements were 
subject to demanding control requirements from the Commission. 

Coherence and complementarity at EU level 

At  EU  level,  the  coherence  of  ERF  interventions  with  those  implemented  by  other 
organisations  (e.g. EASO)  and  the  extent  to  which  ERF  Community  actions  and  national 
projects  complemented  each  other  was  not  raised  as  an  issue.  As  Community  actions  and 
national  actions  are  not  supposed  to  target  the  same  objectives,  the  risk  of  overlaps  was 
limited.  Nonetheless,  stakeholders104  consulted  underlined  their  lack  of  awareness  about  the 
outputs  of  Community  actions.  This  was  a  limit  to  RAs’  ability  to  create  transnational 
networks  and  capitalise  on  previous  outputs  when  selecting  national  projects,  which  in  turn 
could lead raising a potential risk of incoherence between Community and national actions. 

Although  it  remained  a  theoretical  concern  as  the  evaluation  could  not  confirm  it,  overlaps 
and duplications with EASO might have occurred since parts of the agency's mandate were at 
the time still covered by Community actions. However, the risk of inconsistency in terms of 
mandate  and  action  remained  low,  as  EASO  operated  from  a  ‘continuous  activity’ 

104   Respondents to the e-survey. 

32 

 
 
                                                            
perspective, while the ERF was project-based. It would have been desirable to involve EASO 
in the selection of priorities for Community actions. 

9.6  EU added value 

EQ8: What is the additional value of the ERF actions compared to what the Member States 
would have been able to carry out through investments necessary for the implementation of 
EU policies in the field of asylum without the support of the Fund? 

Main conclusions: The evaluation concluded that all Member States considered that the EU 
funds  allowed  for  funding  innovative  projects  or  projects  aimed  at  specific  target  groups 
which, without the ERF, would not have been implemented. In addition, the added value of 
the  ERF  was  crucial  for  Member  States  with  less  developed  asylum  systems.  For  some 
priorities,  it  is  questionable,  however:  there  was  no  added  value  as  regards  Member  States’ 
cooperation and exchange of best practices. 

According to the ERF Decision105, the expected added value of the Fund was ‘to promote a 
balance of efforts between Member States in receiving refugees and displaced persons’, since 
this  objective  could  not  be  sufficiently  achieved by  the  Member  States  acting  alone.  Added 
value  brought  in  by  the  Fund  was  therefore  assessed  as  regards  the  financial  impact  of  the 
funding ("financial added value") and the added value resulting from greater cooperation and 
more effective practices within Member States. 

Financial added value 

The financial added value of the ERF was widely recognised by beneficiaries and RAs. All 27 
Member  States  confirmed  that  it  would  not  have  been  possible  to  implement  additional 
projects  and  actions  without  the  support  of  the  Fund.  Three  quarters  of  the  beneficiaries 
involved in the ex post evaluation survey confirmed this. A total of 13 Member States stated 
that the ERF was used to target types of service that go beyond what the state is responsible 
for  or  able  to  deliver.  In  some  Member  States106  with  less  developed  or  recent  asylum 
systems, where asylum issues were not a priority and national resources were limited107, the 
ERF contributed significantly to the establishment of national asylum systems. In most other 
contexts,  it  provided  an  opportunity  to  finance  innovative  projects  by  testing  new 
methodologies  or  facilities108,  to  enlarge  the  scope  of  activities  by  financing  additional 
services outside the remit of the state or to focus more on specific target groups. 

Finally,  the  importance  of  ERF  funding  as  a  proportion  of  a  Member  State’s  total  asylum 
budget  tended  to  follow  two  trends.  In  Western  and  Northern  Member  States  with  mature 
asylum systems, it was very low (e.g. less than 3 %) as it only financed a small fraction of the 
overall costs for Member States of running their asylum systems. For other Member States, it 
reached up to 50 %, indicating clear and vital added value. In these Member States, the ERF 
played a crucial role in the modernisation of the asylum system: it allowed the introduction or 
improvement of structures in compliance with the EU asylum acquis and it is reasonable to 

105   Established by Decision No 573/2007/EC of 23 May 2007. 
106  BG, HR, EE, HU, LV and SK. 
107   This was due to budget cuts and financial constraints. 
108   NL, UK and IT particularly insisted on the experimental aspect of ERF-funded projects. 

33 

 
 
                                                            
assume  that  much  less  or  nothing  would  have  happened  without  the  ERF.  In  a  system  of 
'perfect' solidarity, the ERF should probably have to finance all costs related to maintaining 
the asylum systems, however, this would mean that the annual budget of the ERF should have 
been much more important. 

Added value in terms of cooperation and exchange of best practices 

The assessment of the added value of the ERF in terms of development of deeper cooperation 
between  stakeholders  in  the  field  of  asylum  was  mixt109.  According  to  the  interviews  with 
RAs, only a few networks were created or sustained through ERF funding and the sharing of 
best practices was not as widespread or intensive as expected. Some Member States110 did not 
see the ERF as a tool to foster cooperation and responsibility-sharing. Nevertheless, some said 
that the ERF had influenced the design of their national policies. 

On  the  other  hand,  according  beneficiaries,  the  ERF  brought  added  value  to  project 
beneficiaries  in  terms  of  professionalisation111  of  organisations,  improved  knowledge  of 
asylum  issues,  and  recognition  and  legitimacy  due  to  the  European  dimension  of  the  ERF. 
According to 19 Member States112, the Fund enabled national project beneficiaries to improve 
their internal processes, enhance their projects and budget management skills, and implement 
new evaluation methodologies. 

10  Conclusion 

Throughout  this  evaluation,  the  ERF  2011-2013  actions  have  been  assessed  on  the  basis  of 
their contribution to the establishment of the burden sharing and solidarity system required by 
Member  States  for  receiving  and  bearing  the  consequences  of  receiving  refugees  and 
displaced persons. 

Over this period, the ERF was implemented in a rapidly changing context which posed many 
challenges, and was shortly followed by a severe and unexpected migratory and humanitarian 
crisis in the Member States. In this context, the ex post evaluation provides evidence that the 
ERF responded effectively to those challenges and delivered a wide range of results. The ERF 
has  been  effective  at  Member  States  level  to  implement  the  operational  aspects  of  the 
European asylum system by increasing the capacities of the Member States. In some Member 
States  with less developed or recent asylum systems, where asylum issues were not a priority 
and national resources were limited, the ERF contributed significantly to the establishment of 
national asylum systems. The Fund provided an opportunity to finance innovative projects by 
testing  new  methodologies  or  facilities,  to  enlarge  the  scope  of  activities  by  financing 
additional  services outside the remit of the state or  to  focus  more on specific target  groups. 
However, the Fund contributed only partially to the development and implementation of EU 
policy  and  legislation  in  the  field  of  asylum  due  to  the  fact  that  following  the  unexpected 

109   Based on assessment by 18 Member States. 
110   BE, CY, EE, ES, HU, IT and MY. 
111   The benefit for beneficiaries that was cited most often. 
112   AT, BE, BG, HR, CY, CZ, EE, FI, FR, EL, HU, IE, IT, LV, LT, MT, NL, PL and ES. 

34 

 
 
                                                            
migratory  crisis,  Member  States  focussed  their  efforts  on  the  basic  needs  of  the  asylum 
applicants.  The  allocation  system  was  also  not  relevant  to  the  EU  needs  of  convergence, 
solidarity and development of the CEAS. It relied on historical flows of asylum applicants. It 
should be pointed out that the effectiveness of the Emergency Measures mechanism mitigated 
this inconsistency. They played a key role in providing support to those Member States facing 
the greatest migration pressure.  

Furthermore,  the  ERF  contributed  to  the  increase  of  the  Member  States'  capacities  and 
appropriate  infrastructures,  especially  in  Member  States,  which  lagged  behind  in  terms  of 
asylum policy development. 

The  evaluation  helped  to  identify  some  lessons  for  the  future,  most  of  them  have  been 
addressed under AMIF.  

First  of  all,  the  formulation  of  Priority  1  was  broad  and  covered  potentially  all  types  of 
needs. This created an unbalanced uptake at national level for the other priorities of the Fund. 
Under  the  AMIF  Regulation,  the  article  related  to  Reception  and  Asylum  Systems  is  still 
broad.  However,  Member  States  indicated  their  priorities  in  the  National  Programmes  that 
were  adopted  after  policy  dialogue  with  the  European  Commission.  Compared  to  the  ERF 
programming period, in AMIF the setting-up of the national programmes is the result of the 
outcomes of a policy dialogue between the European Commission and the Member States.     

Another  issue  identified  during  the  evaluation  was  the  absence  in  the  ERF  of  effective 
monitoring  and  evaluation  mechanisms.  The  European  Commission  has  addressed  this 
issue  for  the  2014-2020  programming  period  by  developing  a  common  monitoring  and 
evaluation framework.  

Regarding  efficiency,  the  evaluation  identified  the  lack  of  efficiency  of  funding  to  support 
less  experienced  Member  States  or  those  facing  difficulties  in  the  implementation  of  their 
management  and  monitoring  systems.  In  consequence,  in  the  context  of  AMIF,  more 
emphasis  on  strategic  programming  was  put  through  a  closely  follow-up  of  those  Member 
States facing difficulties in the implementation of the funds. Furthermore, the AMIF foresees 
the  organisation  of  a  mid-term  review  of  the  multiannual  national  programs  where  the 
situation of each Member State is reviewed in the light of the interim evaluation reports and in 
the light of developments in Union policies in the Member State concerned. 

Finally, the administrative burden of the ERF was perceived as high. Improvements were 
made for the 2014-2020 programming period by the creation of only one fund that covers all 
migration  matters,  the  adoption  of  one  single  multiannual  programme  and  the  alignment  of 
the eligibility rules on the national rules of the Member State. These latter are also encouraged 
to use the simplified costs options. 

Another lesson from the evaluation was the  solidarity dimension. When the ERF priorities 
were  set,  the  enhancement  of  responsibility  sharing  between  Member  States  and  third 
countries  remained  optional.  For  this  reason,  certain  issues  related  to  resettlement  and 

35 

 
 
relocation  were  already  addressed  under  AMIF  by  removing  the  optionality  and  by  putting 
this specific objective at the same level as the others. However, the current discussion on the 
proposal  for  a  revised  Dublin  regulation  shows  that  consensus  on  solidarity  is  difficult  to 
achieve  and  that  improvements  are  still  needed.  This  issue  should  be  addressed  where 
possible during the current programming period or during the next MFF. 

Lessons learnt  

As stated above, the allocation mechanism of the ERF was based on historic inflows and did 
not  take  enough  into  account  evolving  needs  of  the  Member  States.  Under  AMIF,  the 
distribution  key  is  still  based  on  the  2011-2013  allocation  for  the  ERF,  the  EIF  and  the  RF 
although  more  flexibility  is  ensured  by  a  much  more  important  envelop  allocated  for 
emergency assistance.  

Moreover, as explained in section 5, the logical framework of the ERF was not well designed: 
the articulation between general objective and eligible actions, on the one hand, and between 
them and the priorities (and specific priorities) on the other hand, remained confused. Whilst 
the logical framework has been better defined with the AMIF, under each specific objective 
there is still the possibility for Member States to avoid investments in areas less "attractive". 
This issue should be addressed in the new generation of funds post 2020. 

Finally,  as  enounced  above,  AMIF  successor  should  pronounce  and  fund  increasingly  the 
activities that promote solidarity within the EU to jointly stem the challenges resulting from 
high migration. 

The interim evaluation of AMIF will be completed in 2018 and will inform if the identified 
weaknesses  have  been  addressed  properly.  The  interim  evaluation  is  expected  to  have  an 
impact on the implementation of the AMIF in  the remaining spending period  2019-2020. If 
not, this input will be taken up in the design of the post 2020 instruments. 

36 

 
 
 
11  ANNEX 1 — Procedural information 

Leading DG 
Participating Units of DG HOME 

Participating DGs  

Roadmap approval 
External consulting firm 
specialised in evaluation 

DG HOME 
A2 – Legal Affairs 
C3 – Asylum 
C4 –  Migration management support  
E1 – Union Actions 
E2 – National programmes for South and East 
Europe, evaluation, AMF/ISF Committee 
E3 - National programmes for North and West 
Europe, budget, MFF, agencies 
Secretariat General 
DG BUDG 
DG JUST 
Legal service (SJ) 
December 2015 
Contract signed in April 2016 with consortium of: 
Ernst & Young Special Business Services (lead 
partner)  
Center for International Legal Cooperation 
Rand Europa Community Interest Company  

Number of steering group meetings  6 
Last deliverable handed in 
Approval of the final report by 
Steering Group 
RSB consultation 
Agenda planning  

March 2017 
March 2017 

2016/HOME/081 

Changes 
(Ares(2017)287965 of 8/06/2017)  

response 

to 

in 

the  Regulatory  Scrutiny  Board's 

recommendations 

 Main considerations 

Explicitly acknowledge that the intervention 
logic is weak, and that this makes it harder to 
evaluate the programme's results. 

Clarify how the evaluation assessed the 
Fund's effectiveness in the absence of 
specific EU-level objectives and targets.  

Demonstrate how the ERF applied the 
solidarity principle in practice.  

Section 4 ‘The ERF as funding 
instrument 

Section 6 ‘Method’ and Section 9.2 
‘Effectiveness’ 

Section 7.1 ‘Shared management: 

Multiannual and Annual 
Programmes of the Member States’ 
and the explanations related to the 

37 

 
 
 
 
 
 
 
 
 
allocation key 

 Show the extent to which emergency 
measures contributed to the Fund's flexibility. 

Section 9.1 

Streamline and harmonise the conclusions, 
and indicate lessons that are relevant for 
future action 

Conclusion 

38 

 
 
 
 
 
 
 
12  ANNEX 2 — Stakeholders’ consultation 

Overview 

In  order  to  inform  the  ex  post  evaluation  of  the  European  Refugee  Fund  (ERF)  2011-2013, 
several  stakeholders  were  consulted  through  open  public  consultations  conducted  by  the 
European  Commission  and  other  stakeholder  consultations  carried  out  by  the  external 
evaluator, in the framework of its contract with the European Commission.  

This  annex provides an overview of the consultation  processes and the type of stakeholders 
consulted,  and  presents  the  results  of  these  consultations.  These  results have  been  compiled 
and  summarised  from  the  responses  received  by  the  European  Commission  to  its  public 
consultation,  and  from  the  final  evaluation  report  of  the  ERF  submitted  to  the  European 
Commission  by  the  external  evaluator.  The  stakeholder  consultations  presented  below  have 
met  the  European  Commission's  general  principles  and  minimum  standards  for  stakeholder 
consultation113. 

The public consultation conducted by the European Commission 

Between 10 May 2016 and 9 August 2016, the European Commission held an Internet-based 
open public consultation on the ex-post evaluation of the ERF 2011-2013 in the form of an 
online  questionnaire.  Contributions  were  sought  in  particular  from  individuals  (experts, 
beneficiaries),  local  and  national  Member  State  authorities,  intergovernmental  and  non-
governmental  organisations,  social  partners  and  civil  society,  academic  institutions, 
international  organisations,  and  EU  Institutions  and  Agencies.  The  final  number  of 
respondents  was  very  small:  12  participants  from  eight  Member  States114.  This  is  clearly  a 
limitation  of  the  public  consultation.  Eight  representatives  of  a  non-profit  organisation,  two 
representatives  of  a  public  authority,  one  representative  of  an  academic/research  institution 
and one representative of a private company. 
A  limitation  to  take  into  consideration  is  the  fact  that  the  information  available  on  the 
identities of the participants is based on self-reported values which cannot be verified. 

The consultations conducted by the external evaluators 

The external evaluator conducted several types of stakeholder consultations, including: 8 case 
studies, 4 types of face-to-face or telephone interviews, and 2 e-surveys targeted at 2 specific 
types of stakeholders. This section provides an overview of each type of consultation, as well 
as  possible  difficulties  or  limitations  encountered.  The  design  of  these  consultations  was 
informed  by  previous  desk  research  carried  out  by  the  external  evaluator.  When  relevant, 
findings  from  this  desk  research  will  also  be  presented,  particularly  those  of  the  National 
Evaluation Reports (NERs). 

113  See European Commission Better Regulation guidelines (SWD(2015)111). 
114  AT, BG, CY, HU, IE, LV, MT and SK. 

39 

 
 
 
 
 
 
 
                                                            
1.  Case Studies 

In order to gain an in-depth understanding of the ERF's implementation at the national level, 8 
case studies were completed on Austria, Belgium, Bulgaria, France, Hungary, Italy, Sweden 
and  the  United  Kingdom.  As  part  of  these  case  studies,  interviews  with  organisations 
benefitting from the ERF were carried out. The following activities were carried out for each 
case study country: 

  Austria: 10 interviews conducted during a field visit which took place in July 2016; 
  Belgium: 10 interviews conducted during a field visit which took place in September 

2016; 

  Bulgaria: 12 interviews conducted during a field visit which took place in July 2016; 
  France: 11 interviews conducted during a field visit which took place in July and 

September 2016; 

  Hungary: 11 interviews conducted during a field visit which took place in July 2016; 
 
Italy: 9 interviews conducted during a field visit which took place in July 2016; 
  Sweden: 9 interviews conducted during a field visit which took place in August 2016; 
  United Kingdom: 11 interviews conducted during a field visit which took place in July 

2016. 

2.  Interviews 

Interviews  with  Responsible  Authorities  (RAs):  19  telephone  interviews  with  RAs  were 
initially  planned,  but  only  17  of  these  were  conducted.  In  addition,  two  Member  States115 
provided written feedback. 

Interviews  with  EU-level  stakeholders:  3  interviews  were  conducted  in  September  2016 
with officers from the Directorate-General for Regional and Urban Policy (DG REGIO), the 
Directorate-General  Migration  and  Home  Affairs  (DG  HOME)  and  the  European  Asylum 
Support  Office  (EASO).  Two  other  stakeholders  initially  contacted  declined  interview 
requests or suggested another stakeholder, and there were no replies to interview requests sent 
to members of the LIBE Committee, ECRE and the European Union (EU) Council. 

Interviews with other International Organisations: field visits were carried out to Member 
States  selected  for case studies,  which provided an opportunity for the external evaluator  to 
meet  local  branches  of  UNHCR  and/or  IOM.  A  complementary  interview  was  carried  out 
with the UNHCR representation in Brussels. 

Beneficiaries of the 5 Selected Community Actions (CAs): Following the initial selection 
of  5  CAs,  interviews  were  conducted  with  13  organisations:  all  the  selected  CA  project 
leaders were interviewed as well as a selection of co-beneficiaries in 8 Member States chosen 
for the case study visits. 

115  MT and RO. 

40 

 
 
 
 
                                                            
Difficulties  encountered  regarding  the  availability  of  key  stakeholders:  the  external 
evaluator encountered challenges with the scheduling of interviews and case study field visits, 
due  to  the  fact  that  interviews  took  place  during  the  vacation  period.  Some  Member  States 
also  faced  work  overload,  or  had  particular  national  contexts  which  limited  interview 
availabilities  of  RAs.  The  external  evaluation  team  had  to  postpone  some  case  study  field 
visits and phone interviews with RAs until after the summer break, between the end of August 
and  the  beginning  of  September.  In  addition,  given  high  staff  turnover  rates  in  national 
administrations  and  beneficiary  organisations,  the  external  evaluation  team  sometimes 
encountered difficulties finding the appropriate. 

3.  E-Surveys 

Two specific e-surveys were launched targeting two different stakeholder groups: 

1) E-survey targeting beneficiaries of CAs (leading partners) for 2008-2013: this e-survey 
was open between 11 July and 1 September 2016 and sent to 56 leading partners. Responses 
from 17 participants were received, 13 of which were exploitable for analysis. 

2) E-survey targeting all beneficiaries of the Annual Programmes for 2011-2013 in the 8 
case study countries: this e-survey was open between 29 September and 20 October 2016. 64 
participants provided responses out of the total 276 who received the e-survey invitation. 55 
of these responses were exploitable for analysis. 

Results 

The most  relevant results of the consultations carried out by the European Commission and 
the  external  evaluator  are  grouped  in  this  section  and  presented  according  to  the  following 
evaluation  criteria:  relevance,  effectiveness,  efficiency,  sustainability,  coherence  and 
complementarity, and EU added value. 

Relevance 

Generally, the RAs and beneficiaries consulted found that the actions funded by the ERF were 
relevant  (ERF  objectives  corresponded  to  their  needs)  both  at  EU  level  and  for  individual 
Member  States.  Question  10  of  the  Commission's  public  consultation  asked  participants 
whether, in their opinion, the projects and activities supported or financed by the ERF in their 
country  of  residence  addressed  the  needs  of  the  potential  beneficiaries.  50%  of  respondents 
replied "Yes" and the other 50% replied "Generally yes, but with some problems". Some of 
the problems highlighted by these respondents for this period were: 

  small associations were said to have difficulty fulfilling all the requirements or "rules" 

 

of the projects; 
there was thought to be a lack of financial resources at times, for example, sometimes 
resources were insufficient in relation to the number of applicants or potential 
beneficiaries; 

41 

 
 
 
 

 

 

limiting the eligibility for funding to TCNs was thought to exclude naturalized citizens 
of immigrant origin; 
it was also mentioned that the limited implementation period in relation to a large gap 
time period resulted in the development of a belief from the side of the target group 
that the projects did not make any difference to their lives; 
the amount of resources available was limited compared to the increasing number of 
asylum-seekers arriving in Member States from 2013 onwards. 

The problem highlighted by Member States was the restrictive definition of the target group 
of  the  fund,  which  was  found  to  create  difficulties  for  beneficiaries116  in  implementing 
projects.  An  additional  problem  mentioned  by  national  authorities  was  the  fact  that  the 
amount  of  resources  available  was  limited  compared  to  the  increasing  number  of  asylum-
seekers arriving in Member States from 2013 onwards. 

Effectiveness 

In  the  area  of  effectiveness,  the  two  most  relevant  questionnaire  questions  in  the  European 
Commission's  public  consultations  were  questions  2  and  5.  Question  2  asked  participants 
whether  the  result(s)  sought  in  the  general  objective  of  the  ERF  had  been  achieved  in  the 
respondent's country. Question 5 asked whether, based on respondents' experience, the actions 
financed by the ERF in their country were consistent with the general and specific objectives 
of the fund.  

For question 2, the vast majority of participants considered that the general objective of the 
ERF had been achieved in their country at least to some extent. For question 5, the majority of 
participants found that actions financed under the funds were indeed consistent with the fund 
objectives. However 50% of respondents thought this was so to a limited extent. 

Results from other stakeholder consultations provide additional details regarding whether the 
results  sought  in  the  general  objective  of  the  ERF  have  been  achieved  in  the  respondent's 
country.  One  of  the  main  specific  objectives  of  the  ERF  was  the  improvement  of  reception 
conditions. In the NERs, Member States reported that the ERF brought strong support to the 
improvement  of  reception  conditions  of  asylum  applicants  or  other  persons  seeking 
protection. This support was mainly dedicated to providing social assistance, counselling and 
legal  aid  and  language/interpretation  assistance  (38%  of  operations),  as  well  as  bringing 
medical  and  psychological  care  (19%)  and  providing  accommodation  infrastructures  and 
services (15%). 

Regarding integration, which was the second main area of focus, in Member State NERs, the 
ERF  contribution  to  changes  and  improvements  in  the  field  of  integration  was  mostly 
positively assessed. The ERF supported 904 operations in this field, i.e. 42% of all operations 
co-financed by the ERF in the Member States from 2011 to 2013, which reached more than 

116  FI, NL. 

42 

 
 
 
                                                            
95,000  persons  in  total.  Except  for  Finland,  Poland,  Luxembourg  and  the  UK,  all  Member 
States reached their targets. 

Regarding resettlement, according to the NERs, resettlement measures increased from being 
implemented by 9 Member States in 2008-2010 to being implemented by 12 Member States 
in  2011-2013117.  The  total  number  of  people  resettled  doubled  from  4,378  in  2008-2010  to 
9,058 over 2011-2013. 

Efficiency 

Regarding  efficiency,  the  most  relevant  question  in  the  Commission's  public  consultations 
was question 11, which asked whether participants considered that the effects of the actions 
carried out under the ERF were achieved at a reasonable cost in terms of financial and human 
resources  deployed.  The  vast  majority  of  respondents  found  that  the  effects  of  the  actions 
carried  out  under  the  fund  were  achieved  at  reasonable  cost  at  least  to  some  extent.  They 
found  that  the  effects  of  the  actions  carried  out  under  the  fund  were  achieved  at  reasonable 
cost at least to some extent.  

The  results  of  other  stakeholder  consultations  provide  additional  insights  to  the  responses 
provided  by  the  respondents  to  the  public  consultation  (see  above).  The  costs  incurred  to 
achieve  national  programmes  and  project  outputs  were  assessed  by  the  vast  majority  of 
Member  States  (22)  as  reasonable  compared  to  observed  results  or  to  the  costs  engaged  to 
implement projects (according to NERs). 

All Member States claimed that the administrative burden related to the management of the 
ERF was high, but they also found that it was justified to ensure reliable management. Issues 
highlighted by several Member States as contributing to the administrative burden of the ERF 
were the burden linked to multi-annual and annual programming, and the burden linked to the 
division  of  asylum  and  migration  issues  into  three  separate  funds,  which  was  said  to 
encourage the multiplication of administrative requirements. These issues have been under the 
Asylum, Migration and Integration Fund (AMIF) (2014-2020). 

Sustainability 

Regarding  sustainability,  the  most  relevant  question  from  the  Commission's  public 
consultations was question 16. Under this question participants were asked whether, after the 
end  of  the  project  supported  by  the  ERF,  the  services/assistance  had  remained  available. 
There  were  very  few  responses  to  question  16.  The  majority  of  respondents  reported  that 
services/assistance ended after the end of the project in their country. 

Interviews  with  RAs  and  beneficiaries  revealed  that  the  actions  viewed  as  being  more 
sustainable were:  actions  resulting  in  the increase  of  capacities  for  the reception  of  asylum-
seekers,  particularly  for  Member  States  with  low  capacity  beforehand  (e.g.  BG);  projects 

117  BE, CZ, DE, FR, FI, ES, HU, IE, NL, PT, SE, UK. 

43 

 
 
 
 
 
                                                            
aimed  at  the  elaboration  of  studies,  the  development  of  platforms,  IT  tools  and 
methodologies118;  and  actions  aimed  at  empowering  target  groups,  such  as  educational  or 
language  training,  workshops,  or  counselling  sessions  to  help  refugees  find  employment  or 
accommodation;  actions  resulting  in  a  concrete  change  of  practice,  such  as  improvement  in 
asylum procedure119. Projects focused on the integration of refugees were reported as having 
the most fluctuating sustainability, and being highly dependent on the socio-economic context 
of Member States120. 

Coherence and Complementarity 

The most relevant question from the Commission's public consultations regarding coherence 
and complementarity is question 12. This question asked participants whether they considered 
that  projects  and  actions  supported  by  the  fund  were  coherent  and  complementary  to  other 
actions in the same field funded by other EU financial instruments and national resources of 
the Member States. Over 50% of respondents considered that only few actions were coherent 
and complementary to others. One of the respondents which considered this also reported that 
in his/her experience, in comparison with TCNs, refugees have higher needs and face stronger 
barriers to access the job market and integrate into Irish society. He/she thus considered that 
specialised  and  tailored  intervention  strategies  should  be  designed  for  this  particular  target 
group so that they may successfully secure employment. 

Regarding coherence and complementarity at national level, in their NERs, 26 Member States 
out of the total 27121 that participated in this fund reported that the ERF 2011-2013 was used 
in  coherence  and  complementarity  with  other  mechanisms  and  actions  implemented. 
Regarding risks of duplication and overlaps, some Member States thought that this was more 
likely for the European Integration Fund (EIF) and the European Social Fund (ESF), and they 
sought  to  address  these  risks  through  organisational  corrective  actions.  At  EU  level, 
coherence  between  Community  Actions  and  national  actions  was  not  raised  as  an  issue  by 
stakeholders, since these actions do not target the same objectives. However, interviews with 
RAs  and  beneficiaries  revealed  a  lack  of  knowledge  of  the  outputs  of  ERF  Community 
Actions, which reduces the possibilities for synergies. 

EU Added Value 

Regarding  EU  added  value,  Question  1  of  the  Commission's  public  consultations  asked 
participants whether they considered that the implementation of the ERF in their country had 
affected positively the work of the public administrations in the field of asylum. Question 13 
asked participants whether they considered that the contribution of the fund had been crucial 
to  the  implementation  of  EU  policies  in  their  country  in  this  field.  50%  of  respondents  to 
question 1 thought that this was to a great extent. Regarding question 13, 50% of respondents 

118  IT, BG, CZ, ES, FR, DE, NL, PL, AT, BE, SE. 
119  PT, EE, DE, FI. 
120  AT, DE, LV, PT, SK, NL. 
121  AT is the exception. 

44 

 
 
 
 
 
                                                            
thought  that  the  contribution  of  the  ERF  "made  a  huge  difference".  Some  respondents 
provided specific reasons for this:  

  A  participant  considered  that  changes  had  been  made  and  new  ideas  implemented 
thanks to the fund's contribution, but that the results and the level of change had not 
been assessed at national level, or there were no relevant available publications. 

Overall, the responses of Member States (NERs) were very positive regarding the EU added 
value of the ERF. Reports from 27 Member States (NERs) and the majority of responses to 
interviews conducted at national level by the external evaluators suggest that the ERF brought 
a more or less significant financial added value, as it enabled the implementation of additional 
projects  that  would  probably  not  have  been  financed  by  national  public  resources  or  other 
sources  of  financing.  The  additional  funding  provided  by  the  ERF  to  national  beneficiaries 
was the most cited nature of added value resulting from the EU intervention during interviews 
with RAs and beneficiaries. Beneficiaries of Community Actions also found the added value 
to be strong (only 1/13 respondents believed that projects could be implemented without the 
ERF). In many Member States facing budget constraints and with relatively little experience 
with  asylum  issues,  the  ERF  was  found  to  have  helped  compensate  the  lack  of  national 
resources,  either  because  of  budget  cuts  and  financial  constraints,  or  because  asylum  issues 
remained at the bottom of the countries' list of priorities. This was the case in “new” Member 
States,  such  as  Bulgaria,  Croatia,  Estonia,  Hungary,  Latvia,  and  Slovakia,  where  the  ERF 
contributed extensively to the development of an asylum system and corresponding facilities, 
and where nothing was previously needed or foreseen.  

Regarding  cooperation  and  exchange  of  best  practices,  18  Member  States  declared  in  their 
NERs that the ERF allowed for the development of deeper cooperation between the different 
stakeholders  working  in  the  field  of  asylum.  But  this  finding  is  contradicted  by  interview 
responses  from  RAs,  who  considered  that  few  networks  were  created  or  sustained  through 
ERF  funding.  Member  States  also  reported  in  their  NERs  that  the  ERF  brought  some 
unexpected  "soft"  added  value  to  international  organisations  and  beneficiaries  in  terms  of 
professionalization, expertise and recognition. 

45 

 
 
 
 
13  ANNEX 3 — Methodology 

Study conducted by external evaluators   

The  evaluation  relied  on  a  supporting  study  conducted  by  an  external  company.  It  was 
decided  to  rely  on  an  external  study  so  as  to  obtain  a  robust  and  impartial  overview  of  the 
Fund. A number of elements ensured the high quality of the study: 

  A regular and transparent dialogue took place between the European Commission sand 

the external evaluator; 

  The terms of reference of the contract were clearly set out and respected; 
  The Fund's management modes were clearly distinguished in the methodology; 
  The evaluation applied a mixed-methods approach in order to address the evaluation 

questions more fully and to ensure all relevant stakeholders were consulted; 
  All data sources were assessed for validity and presented data are clearly labelled. 
  Triangulation analysis was elaborated. 

Communication between the European Commission and the external company 

The  study's  progress  was  followed  by  an  Inter-service  Steering  Group  (ISSG)  comprised  of 
officials  from  the  SG,  SJ,  DG  BUDG,  DG  REGIO,  DG  HOME  (particularly  Units  A2,  C3, 
C4,  E1,  E2  and  E3).  Four  meetings  took  place  between  the  contractors  and  the  ISSG  and 
structured  feedback  (in  both  directions)  was  provided  on  a  weekly  basis  throughout  the 
duration of the contract. This two-way dialogue was enriched by the active participation in the 
ISSG of policy and implementation units and shadowed by horizontal units and the Secretariat 
General. The quality assessment of the overall process (external contractor's work and report) 
took place with the participation of the ISSG (A2, C3, E2 and SG); the other members were 
consulted by email.   

Phases 

The  structuring  feature  of  the  external  evaluation  was  the  segmentation  of  the  tasks  into 
clearly  defined  phases  which  were  closely  observed  by  all  parties.  These  phases  had  been 
determined in the Terms of Reference. Adherence to the Terms of Reference made the study 
itself more efficient and transparent. 

Data sources and quality 

The information analysed by the contractors through desk research can be grouped into two 
categories: 

1.  Documentation  relating  to  implementation  –  legal  acts;  high-level  contextual 
documents;  programme  documents  (Annual  Work  Programmes)  and  addition 
evaluation  documents  (National  evaluation  reports  for  shared  management  and 
technical implementation reports for direct management);  

2.  Statistical data – this includes: 

o  Official statistics from Eurostat, EASO and UNHCR; 

46 

 
 
 
 
 
o  Financial data extracted from ABAC for ERF and all SOLID Funds in order to 

present Programmed and Net EU contributions and absorption rates;   

o  Financial data extracted from SFC2007 to present the breakdowns by Priority 

and Specific Priority;  

o  Financial  data  extracted  from  SFC2007  to  present  Programmed  and  Net  EU 

contributions and absorption rates for Annual Programmes; 

o  Financial  data  for  direct  management  were  provided  by  the  Eurpean 
Commission.  Figures  presented  are  the  sum  of  the  cost  claims  minus  the 
ineligible costs. 

Open Public Consultation 

Between 10 May 2016 and 9 August 2016, the European Commission also held an Internet-
based  public  consultation  on  the  ERF  2011-2013  in  the  form  of  an  online  questionnaire. 
Contributions  were  particularly  sought  from:  individuals  (experts,  beneficiaries),  local  and 
national  Member  State  Authorities,  intergovernmental  and  non-governmental  organisations, 
social  partners  and  civil  society,  academic  institutions,  international  organisations,  and  EU 
Institutions  and  Agencies.  The  final  number  of  respondents  was  small:  twelve  participants 
from  eight  Member  States122.  This  means  that  contributions  are  not  representative  of  the 
targeted  stakeholders,  but  the  results  provide  additional  insights.  Eight  representatives  of  a 
non-profit  organisation,  two  representatives  of  a  public  authority,  one  representative  of  an 
academic/research institution and one representative of a private company. 

Analysis 

Three different levels of analysis were undertaken by the external company: 

1.  Descriptive analysis, at two levels, to provide context and a basis for the development 

of other types of analysis: 

  EU  level:  Different  official  documentation,  such  as  the  Decision  establishing 
the  ERF,  as  well  as  interviews  with  DG  HOME  were  analysed  and  used  to 
describe the context of the ERF, such as the objectives it was aiming to achieve 
and the type of actions eligible for funding under the ERF.  

  Case studies: The analysis of the data collected relating to the national actions 
as  part  of  the  case  studies  (all  programmatic  documents,  national  evaluation 
reports  and  interview  notes)  was  carried  out  and  described  according  to  the 
intervention  logic  elements,  as  well  as  all  the  evaluation  criteria  in  the  case 
study reports. The case studies were selected according to four fixed criteria to 
improve  representativeness  (Objectives,  Priorities,  types  of  intervention); 
relevance  (external  borders  and  migratory  pressure);  solidarity  (where 
investments  exceed  input)  and  coverage  of  the  evaluation  questions  in  the 
national evaluation reports. 

2.  Thematic  analysis:  For  this  analysis,  N-vivo,  a  qualitative  analysis tool, was  used  to 
encode and subsequently analyse the information from the national evaluation reports 

122   AT, BG, CY, HU, IE, LV, MT and SK. 

47 

 
 
 
 
                                                            
and relevant interviews (EU level and interviews with RAs). The encoding was done 
according to a Coding Framework which included all the different intervention logic 
elements,  as  well as  all the evaluation criteria, and allowed for the creation of “sub-
nodes” as further themes emerged from the analysis. Through the encoding, trends and 
themes  emerged  across  the  participating  countries  under  the  different  evaluation 
criteria and ERF objectives and priorities. In addition, quantitative data collected was 
analysed,  such  as  the  context  and  effectiveness  indicators  from  the  final  closure 
reports submitted by the Member States (as presented in the SFC 2007 database) and 
national evaluation  reports. This  quantitative analysis also allowed for key messages 
to emerge from the data (such as type of priority receiving the most funding).  

3.  Comparative  analysis:  Building  on  the  descriptive  and  thematic  analysis,  a 
comparative  analysis  was  undertaken,  comparing  the  findings  from  different 
participating countries under each of the evaluation criteria. The comparative analysis 
allowed  the  study  team  to  assess  the  extent  to  which  the  research  findings  were 
coherent.  The  case  studies  were  also  included  in  the  analysis  and  used  to  illustrate 
certain findings.  

4.  Interviews:  Over  120  interviews  were  organized  by  telephone  and  in  person  by  the 
contractor  to  gather  additional  input  from  several  stakeholder  groups  (European 
Institutions, EU Agencies, national responsible authorities and other stakeholders).   

Assessing the Impacts of the ERF 

Evaluating  the  impacts  of  the  ERF  at  national  level  is  more  complicated  as  it  requires 
experiment  conditions  with  a  matched  control  where  ERF  was  not  utilised.  In  order  to 
mitigate this, the study used information from the case studies and interviews, as well as the 
evaluators’ own judgment. 

48 

 
 
 
 
 
 
 
14 

ANNEX 4 — List of evaluation questions 

Theme 1 Effectiveness  

1.  To  what  extent  did  the  ERF  2011-2013  (or  2008-2010)  actions  contributed  to  the 
achievement of the objectives defined in Articles 2 and 3 of Decision No 2007/573/EC and to 
the priorities defined by the Strategic guidelines (Decision No 2007/815/EC)?  

a) To what extent did the ERF 2011-2013 (or 2008-2010) actions contribute to the 
reception conditions and asylum procedures?  

b) To what extent did the ERF 2011-2013 (or 2008-2010) actions contribute to the 
integration of persons referred to in Art 6 (target group)? 

c) To what extent did the ERF 2011-2013 (or 2008-2010) actions contribute to the 
enhancement of Member States’ capacity to develop, monitor and evaluate their 
asylum policies in the light of their obligations under existing and future Community 
legislation relating to the Common European Asylum System (in particular practical 
cooperation activities between Member States)? 

d) To what extent did the ERF 2011-2013 (or 2008-2010) actions contribute to the 
resettlement of persons referred to in Article 6(e)? 

e) To what extent did the ERF 2011-2013 (or 2008-2010) actions contribute to the 
transfer of persons falling within the categories referred to in Article 6(a) and 6(b) and 
6(c)? 

2. To what extend did the ERF 2011-2013 (or 2008-2010) emergency actions contributed to 
the achievement of these same objectives and to the priorities? 

Theme 2 Efficiency  

3. To what extent were the effects of the ERF 2011-2013 (or 2008-2010) actions achieved at a 
reasonable cost in terms of financial and human resources deployed?  

Theme 3 Sustainability 

4.  To  what  extent  have  the  positive  effects  of  the  ERF  2011-2013  (or  2008-2010)  actions 
lasted after the interventions were terminated? 

Theme 4 Relevance  

5. To what extent did the ERF objectives correspond to the needs related to receiving, and in 
bearing the consequences of receiving, refugees and displaced persons by the Member States? 

49 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6.  To  what  extent  did  the  ERF  actions  correspond  to  the  needs  related  to  receiving,  and  in 
bearing the consequences of receiving, refugees and displaced persons by the Member States? 

Theme 5 Coherence and complementarity  

7.  To  what  extent  were  the  ERF  2011-2013  (or  2008-2010)  actions  coherent  with  and 
complementary to other actions related to asylum, financed by other EU financial instruments 
and  from  national  resources  of  the  Member  States,  including  the  activities  of  the  European 
Asylum Support Office (EASO), supporting EU Member States on asylum? 

Theme 6 EU added value 

8.  What  is  the  additional  value  resulting  from  the  ERF  2011-2013  (or  2008-2010)  actions 
compared  to  what  the  Member  States  would  be  able  to  carry  out  through  investments 
necessary for the implementation of the EU policies in the field of asylum without the support 
of the ERF 2011-2013 (or 2008-2010) actions? 

50 

 
 
 
 
15 

ANNEX 5 — List of abbreviations and country codes 

ABAC  
AMIF   
AP 
AWP   
CA 
CEAS   
COI 
EAC 
EASO  
EBF 
EC 
EIA 
EIF 
EQ 
ERF 
IOM 
ISF 
ISSG 
MPI 
MS 
NGO 
RA 
RF 
SFC 2007 
SOLID  

Accrual Basic Accounting 
Asylum, Migration and Integration Fund 
Annual Programme 
Annual Work Programme 
Community Action 
Common European Asylum System 
Country of Origin Information 
European Asylum Curriculum 
European Asylum Support Office 
European Borders Fund 
European Commission 
Extended Impact Assessment 
European Integration Fund  
Evaluation Question 
European Refugee Fund 
International Organisation for Migration 
Internal Security Fund 
Inter Service Steering Group 
Migration Policy Institute 
Member State(s) 
Non-Governmental Organisation 
Responsible Authority 
European Return Fund 
System for Fund Management 2007 
Solidarity and Management of Migration Flows 

List of country codes 

AT 
BE 
BG 
CH 
CY 
CZ 
DE 
DK 
EE 
EL 
ES 
FI 
FR 

Austria  
Belgium 
Bulgaria 
Switzerland 
Cyprus 
Czech Republic 
Germany 
Denmark 
Estonia 
Greece 
Spain 
Finland 
France 

51 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
HR 
HU 
IE 
IS 
IT 
LT 
LU 
LV 
MT 
NL 
NO 
PL 
PT 
RO 
SE 
SI 
SK 
UK 

Croatia 
Hungary 
Ireland 
Iceland 
Italy 
Lithuania 
Luxembourg 
Latvia 
Malta 
Netherlands 
Norway  
Poland 
Portugal 
Romania 
Sweden 
Slovenia 
Slovakia 
United Kingdom  

52