EUROPEAN COMMISSION Brussels,12.6.2018 SWD(2018) 334 final COMMISSION STAFF WORKING DOCUMENT on the Ex Post Evaluation of the European Refugee Fund 2011-2013 and 2007-2010 Accompanying the document Report from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions Ex post evaluation reports for the period 2011 to 2013 of actions co-financed by the four Funds under the framework programme 'Solidarity and Management of Migration Flows' {COM(2018) 456 final}



{SWD(2018) 333 final}



{SWD(2018) 335 final}



{SWD(2018) 336 final}



Contents 2 Introduction 3 Background to the initiative 4 The ERF as funding instrument 5 Evaluation questions 6 Method 6.1 Limitations 7 Management modes of the RF 7.1 Shared management: Multiannual and Annual Programmes of the Member States 7.2 Direct management: Annual Work Programmes of the Commission 8 Implementation – state of play 9 Answers to the evaluation questions 9.1 Relevance 9.2 Effectiveness 9.3 Efficiency 9.4 Sustainability 9.5 Complementarity and coherence 9.6 EU added value

10 Conclusion 11 ANNEX 1 — Procedural information Section 7.1 ‘Shared management: Multiannual and Annual Programmes of the Member States’ and the explanations related to the allocation key 12 ANNEX 2 — Stakeholders’ consultation 13 ANNEX 3 — Methodology 14 ANNEX 4 — List of evaluation questions 15 ANNEX 5 — List of abbreviations and country codes



1Introduction This staff working document reports on the ex post evaluation assessing the results and impacts of actions co-financed by the European Refugee Fund (ERF) under the annual programmes implemented by the 27 participating Member States (including emergency measures), and of the Community actions, in the period 2011-2013. Actions under national programmes financed during this period were implemented from 1 January 2011 to 30 June 2015. The ex post evaluation assessed ERF in the light of its relevance (whether its objectives matched with societal needs), effectiveness (to what extent objectives have been achieved), efficiency (to what extent costs were proportionate the achieved benefits), sustainability of effects after the intervention ended, coherence between the actions financed by the instrument and complementarity to other interventions, and the added value of intervening at EU level. The results of this evaluation will complement the interim evaluation of the Asylum, Migration and Integration Fund (AMIF), the successor of the Return Fund, Refugee Fund and Integration Fund for the period 2014-2020. The interim evaluation of the AMIF is due in 2018. The Commission will submit an interim evaluation report of the Fund by 30 June 2018 to the European parliament, the Council, the European Economic and Social Committee and the Committee of the Regions. This interim evaluation report will be based on the interim evaluation reports provided by the Member States on the implementation of the actions and progress towards achieving the objectives of their national programmes to be submitted to the Commission by 31 December 2017. The interim evaluation report which the Commission will submit will also include an assessment of the mid-term review carried out by the Member States and of the implementation of the emergency assistance grants and Union actions. The results of the interim evaluation together with the results of the present evaluation will contribute to the shaping of the future policies under DG HOME responsibility, especially to the preparation of the new funding instruments in the framework of the Multiannual Financial Framework (MFF) post 2020. 2Background to the initiative In the years 2000, the European Union had on average received around 300 000 asylum applications a year. In this light a first tangible expression of solidarity in the field of asylum at EU level was the creation of the ERF in 2000, which has been instrumental in laying the foundations of collective action by the European Union for the reception of asylum applicants and persons requiring international protection as part of a comprehensive approach. The first ERF ended in 2004 and was followed by a second phase (ERF II) in 2005 - 2007. A number of crucial requirements in the field of EU asylum policy were identified in 2005 in an impact assessment. The main problem to be addressed was to translate into practice the principle of solidarity between Member States in the EU asylum policy and was defined as follows: How to implement the principle of solidarity in managing people flows by ensuring a fair share of responsibilities between Member States as regards the burden arising from the implementation of common policies on asylum and immigration and on the management of the external borders? The 2008-2013 ERF (the subject of the current ex post evaluation and hereinafter referred to as 'ERF') is the third phase of solidarity in the field of asylum and is one of the four EU funding instruments established in 2007 as part of the framework programme on solidarity and migration flows and of a wider EU policy framework which also included the European Asylum Support Office Agency (EASO), the Dublin Regulation, the EURODAC system and emergency measures. Since 1999, the EU has been working to create a common European asylum system (CEAS) affecting the obligations of Member States with regards to the reception of asylum applicants and asylum procedures, the integration of refugees and the beneficiaries of subsidiary protection. Moreover, the EU has adopted several legislative measures harmonising common minimum standards for asylum. These measures (the asylum acquis) aimed at ensuring an equal treatment of asylum applicants in an open and fair system.

One of the characteristic of the period was the sudden and unforeseen increase of migratory pressure which started already in 2012 and led to a drastic increase in the number of asylum applications in the European Union, with great variation across Member States. The number of asylum applicants in 2014 reached 600 000 persons and were mainly concentrated in Germany (173 100), followed by Sweden (75 100), Italy (63 700), France (59 000) and Hungary (41 300). Figure 1: Number of asylum applicants in the 28 Member States (period 2008-2016) Source: Eurostat data as of 07.03.2017 http://appsso.eurostat.ec.europa.eu/nui/show.do?dataset=migr_asyappctza&lang=en 3The ERF as funding instrument The general objective reflected into the Impact Assessment underpinning the SOLID Funds was to support the development and implementation of the CEAS on the basis of solidarity between Member States, and to promote balance in Member States’ efforts to bear the consequences of receiving refugees and displaced persons, taking account of the relevant Community legislation. The general objective of the ERF does not fully capture this idea and limits its expression "to support and encourage the efforts made by the Member States in receiving and bearing the consequences of receiving refugees and displaced persons taking into account of Community legislation on those matters". The Fund acted as a financial solidarity mechanism to support Member States that, on behalf of the EU, bear a substantial longterm financial burden under the CEAS. The ERF's general objective was not translated further into specific and operational objectives in the legal base. Instead, the legal base identified five policy areas and within each policy area defined concrete eligible actions. The five policy areas are the following: i) reception conditions and asylum procedures, ii) integration of target groups iii) enhancement of MS' capacity to develop, monitor and evaluate their asylum policies, iv) resettlement and v) transfer of target groups. The intervention logic of the ERF's legal base is therefore not consistent with the wider policy objectives. The definition and scope of the policy areas was linked to the scope of the Common European Asylum System. Therefore the evaluation assimilated the five identified policy areas to specific objectives and the eligible actions within the policy areas as operational objectives. In consequence, the effectiveness of the ERF was evaluated against these five policy areas. Furthermore, the absence of clearly defined objectives in the legal base led to the lack of EU targets linked to operational objectives to measure the results of the ERF. Therefore, to assess the effectiveness of the Fund, the evaluation had to rely mainly on programme targets set by Member States in their annual programmes which reflect first and foremost national needs and are assessed in the evaluation against interviews with stakeholders. This, combined with the absence of a baseline, proved to be strong limitation in terms of methodology. To compensate for the absence of clearly defined objectives and targets and in accordance with the Decision establishing the ERF, the Commission adopted Strategic Guidelines which identified three priorities and seven specific priorities (between 2 to 3 specific priorities per priority). The three main priorities were: ·Priority 3: Actions helping to enhance responsibility sharing between Member States and third countries (optional) The specific priorities were defined to target particular actions for which the Member States could request an EU co-financing rate of 75% (See Section 7). Each action planned in the annual programme should be programmed under one of the priorities set in the Strategic Guidelines. As the Strategic Guidelines did not set any threshold per priorities, Member States could distribute their annual allocation as they wished provided that the MSs target the available resources to the first two priorities, the third one being optional. However, the funding priorities contributed to the confusion of the intervention logic as the first priority had a very wide scope, equivalent more or less to the general objective of the Fund. This issue was addressed under AMIF which shows a clear intervention logic and simpler articulation, with one general objective, translated into four specific objectives, accompanied by examples of actions. Under AMIF, also a common monitoring framework has been developed to enable tracking of progress. 4Evaluation questions This evaluation assessed the ERF against the five mandatory evaluation criteria laid down in the Commission Better Regulation Guidelines (effectiveness, efficiency, relevance, coherence and EU added value). In addition, the co-financed actions were also assessed for their sustainability and complementarity. To assess these seven criteria, thirteen evaluation questions were defined (see Annex 6). 5Method This evaluation relies on an external study carried out between April 2016 and March 2017. The methodology combined desk research, on-site visits, and qualitative and quantitative analysis. It required a systemic summary of the evidence regarding the implementation of the Fund, most of the information being provided by the Member States’ national evaluation reports (NERs) and annual programmes (APs). The APs set out the operational objectives through actions for each year and the NERs provided information on outputs relating to the operational objectives, results on the specific objectives and impacts as regards the general objective.

Consultations took place with a broad range of stakeholders (more than 120), including 27 responsible authorities (RAs), 88 beneficiaries, 5 EU officials and casestudy respondents. The methodology combined interviews and case studies (each involving a site visit). The case studies illustrate an in-depth analysis of Member States' national programmes and have facilitated the understanding of success stories and failures. The results of the case studies were integrated in the replies to the evaluation criteria. An internet based public consultation was also carried out (a detailed analysis of its outcome is attached in Annex 2). Information was triangulated to ensure validity and robustness. The financial data extracted from SFC and ABAC and presented in the document was updated in January 2017. 5.1Limitations The evaluation process has encountered some difficulties. In particular, the assessment of effectiveness and efficiency was hindered, to a certain extent, by the absence of comparable quantitative data. For annual programmes, it is due to the fact that the reporting requirements in the Fund Regulation did not stipulate that RAs should collect the data that would be required to conduct an assessment of effectiveness and efficiency. Furthermore, it has been difficult to measure efficiency, as no clear trends could be established due to the large diversity of projects, which could not be compared and/or the lack of ex ante targets to measure efficiency. Therefore, the assessment of efficiency had to reply on more qualitative data collected through the stakeholder consultations and the national evaluation reports. Indicators and quantitative targets were set by Member States through their annual programme when selecting the actions supported by the programme for each year. For each action proposed, Member States defined independently the expected quantified results. In consequence, there was a multitude of indicators across Member States with no harmonization and a limited assessment on the side of the Commission representing a limitation for the evaluation. Most of the Member States achieved their quantitative targets related to a number of asylum seekers and refugees to be received and/or integrated. On the contrary, what was missing was the qualitative analysis of the results. For instance, under Priority 1, how the reception and integration and asylum procedures would be improved. Furthermore, given the absence of agreed benchmarks, Member States had different understandings of what to record in the NERs. It is likely that Member State evaluation experts dealing with NERs misunderstood the exact meaning of some indicators with the result that sometimes the data reported was not comparable between Member States. To mitigate this problem to the extent possible, RAs were provided with guidance on the interpretation of the indicators on an ad hoc basis at their request during the implementation of the annual programmes, and this was then made accessible to all RAs. To partly overcome these limitations, different approaches/mitigation strategies were taken by the evaluator. For the effectiveness and efficiency, the analysis was restricted to being based primarily on stakeholders’ assessments, gathered through interviews and a phone survey with beneficiaries and RAs. The potentially positive stakeholder bias was corrected by looking at past achievement of predecessor programmes and the judgment of the evaluators. Overall, the availability, robustness and reliability of the data were sufficient to generate findings. To improve transparency and accountability in the future, the Commission has developed a specific common monitoring and evaluation framework and prepared a document to guide RAs in their AMIF evaluation exercise. Further details on the methodology can be found in Annex 3 and in the relevant parts of the document. 6Management modes of the RF The actions co-financed under the ERF were implemented through two different management modes: 7.1 Shared management: Multiannual and Annual Programmes of the Member States Programming: Under the shared management mode, MSs hold the primary responsibility for the implementation and management and control of the interventions of the Fund. Under this management mode, the Fund is implemented on the basis of strategic multiannual programmes adopted for each Member State covering the 2008-2013 programming period. In addition, annual programmes were negotiated each year with each Member State within the framework set by the multiannual programme to implement the annual financial allocation granted to each of them. Calculation of the annual financial allocation for each Member State: to express solidarity with the Member States who bear, for the benefit of the EU, a heavier financial burden, the Fund basic act lays down the two objective criteria to be taken into consideration for the calculation of the annual allocation, i.e. the total number of refugees, TCNs enjoying subsidiary protection and resettled persons admitted in a given MS over the previous three years and the total number of asylum applicants, persons who have applied for subsidiary protection and TCNs enjoying temporary protection in a given MS over the previous three years. The provisional allocations were communicated to the Member States the year before (by 1st July each year ) the adoption of the annual programme in question and were based on average figures over the previous three years (e.g. allocation for the 2013 AP were communicated in 2012 and were based on data of 2009, 2010 and 2011). Content of the Annual Programme: The annual programme sets out the actions to be implemented in the Member State and indicates their purpose, scope, beneficiaries (public authorities in charge of return, NGO's, international organisations etc), expected results and financial allocation (EU and national co-financing). Each action is then implemented by one or more project(s) on the ground. In the shared management context, the projects are selected by the competent national authority (i.e. the Responsible Authority (RA)). Member States are therefore independent and only ex post assessment is carried out by the Commission at the time of submission of the closure report of the AP in question. The target group for the Fund comprises any third-country national (TCN) or stateless person having the status defined by the Geneva Convention and who is permitted to reside as a refugee in one of the Member States, any third-country national or stateless person enjoying a form of subsidiary protection within the meaning of Directive 2004/83/EC, any third-country national or stateless person who has applied for one of the forms of protection described in points (a) and (b), any third-country national or stateless person enjoying temporary protection within the meaning of Directive 2001/55/EC and any third-country national or stateless person who is being or has been resettled in a Member State . During the eligibility period of each annual programme, Member States were entitled to request a revision of their annual programme to adjust the actions to the changing migration flows and/or emergency situations. Member States could not however request additional funding for the implementation of a specific AP. The EU co-financing rate is set at 50%. This rate may be increased to 75 % for projects addressing specific priorities identified in the Strategic Guidelines as well as for projects implemented by the Member States covered by the Cohesion Fund . Technical assistance: Member States were entitled to use the funds allocated to the AP to finance "preparatory measures, management, monitoring, evaluation, information and control measures as well measures for the reinforcement of the administrative capacity for the implementation of the Fund" . For the AP 2011 to 2013, the technical assistance was set at 4% of the total annual amount of funding allocated plus EUR 30,000.

Closure of the Annual Programme: The eligibility period of each annual programme was set at two years and a half (e.g. 2013 annual programme ran from 1 January 2013 until 30 June 2015). After the end of eligibility period, the Member States submitted a final report, which included an audit report. The part of the annual allocation not spent – if any – after the end of the eligibility period was lost for the Member States and for the Commission. Emergency assistance: In addition, to top up the allocation provided to their annual programmes, the Commission was entitled to provide "assistance to Member States for the implementation of emergency measures aimed at addressing situations of particular pressure". According to the basic act of the Fund, "Such situations are characterised by the sudden arrival at particular points on the borders of a large number of third country nationals who may be in need of international protection, which place exceptionally heavy and urgent demands on the reception facilities, the asylum system or infrastructure of the Member State(s) concerned and may give rise to risks to human life, wellbeing or access to protection provided under Community legislation." The duration of these actions could not exceed 6 months. Over the period, the Commission allocated EUR 66 385 316 to emergency assistance which is equal to 18% of the total programmed budget dedicated to annual programmes (EUR 22 803 735 in 2011, EUR 10 202 335 in 2012 and EUR 33 379 246 in 2013). Overall, 51 emergency measures were implemented between 2011 and 2013 in 11 Member States faced with particular pressure. 7.2 Direct management: Annual Work Programmes of the Commission Under this management mode, up to 10% of the RF's available resources could be used to finance transnational actions or actions of interest to the EU as a whole, which were managed by the Commission (Community actions). Each year the Commission adopted an annual work programme (AWP) specifying how the objectives of the ERF were to be pursued for the implementation period, taking into account the policy context. The AWPs were mainly implemented via grants, for which the Commission then launched calls for proposals defining the eligible actions. Calls for proposals were addressed to public authorities from the Member States participating in the Fund, International Organisations and NGOs registered in one of the Member States. The ERF Community Actions were co-financing (up to 90% of the total eligible costs of the action) transnational projects aiming at developing methodological tools (i.e, facilitating integration and identifying alternatives to detention), addressing the needs of vulnerable people (in particular unaccompanied minors and victims of torture), and strengthening responsibility sharing between Member States through resettlement and relocation mechanisms. Furthermore, two grants aiming at reforming and setting up a new Greek asylum system were also directly awarded to UNHCR to support the emergency situation related to the asylum system in Greece. 7Implementation – state of play The ERF was launched in 2008 with an available financial envelope of EUR 628 million. The allocation for 2011-2013 was EUR 386.2 million, of which EUR 373.5 million was under shared management (including emergency measures with EUR 68 million funded to 10 Member States) and EUR 12.7 million under direct management for Community Actions. The table below shows the EU distribution of financing to the different management modes during the period under assessment. Table 1: Total programmed EU contribution for ERF, 2011-2013 (EUR) 2011 2012 2013 Total Shared management (via national programmes) 116 722 757 111 259 303 145 463 497 373 445 557 Direct management (via Community actions) 3 953 032 4 214 040 4 545 211 12 712 283 Total ERF EU budget 120 675 789 115 473 343 150 008 708 386 157 840 A) Implementation through shared management Finding 1: Absorption rates are good and remain stable at 81% As illustrated by figure 2, the programmed EU contribution rose gradually by 24 %, from EUR 117 million in 2011 to EUR 145 million in 2013 due to new challenges to be addressed in the area of asylum. The average absorption rate remained stable across the period, at 81%. Figure 2: Programmed and net EU contributions and absorption rates by year, ERF shared management, 2011-2013 Source: European Commission data as of 31.12.2016 (ABAC/SFC) Under the shared management mode, Member States hold the primary responsibility for the implementation and management and control of the interventions of the Fund. Member States implement programmes at national level and allocate these funds to end recipients which are public authorities, NGOs or international organisations. Actions under national programmes were implemented from 1 January 2011 to 30 June 2015. It should be underlined that some national programmes were not yet closed when the evaluation was carried. Member States demonstrated through the level of their absorption rate the ability to perform effective programming and involve national stakeholders in the implementation of projects. The ERF absorption rate ranged from 72 % to 84 % in 20072013 and was 81 % overall for 2011-2013. No particular correlation could be identified between absorption rate and geographical location, country size or share of the ERF. Nevertheless, as highlighted in Figure 3, the absorption rate of most Member States is above 70 %, with the exception of the Czech Republic (60 %) and Ireland (with an underperforming rate of 41 %). According to these Member States, these low absorption rates were due to the need for beneficiaries to cofinance 25 % of the projects themselves as it was not financed by the national budget of the Member State, which appeared to be a constraint for most stakeholders.

Figure 3: Programmed and net EU contributions and absorption rates by Member State, ERF shared management, 2011-2013 (including emergency assistance) Source: European Commission data as of 31.12.2016 (ABAC/SFC) In the NERs and case studies, stakeholders cited the following obstacles to achieving a 100 % absorption rate: ·secondary impacts of the financial crisis affecting the capacities of beneficiaries to cofinance projects themselves; ·Member States with relatively underdeveloped asylum systems having to build capacity; ·greater migratory pressure and related conflicts which became protracted; and ·administrative delays, which resulted in agreed deadlines being missed. Finding 2: 'Sharing the burden' – the distribution of ERF funding follows partially the trends in asylum applications across Member States because of a delay of three years compared to the trend in asylum applications The total ERF contribution varied significantly across Member States. Five Member States account for almost 59 % of the total programmed and the total final EU contributions. Funds were allocated on the basis of the methodology set out in Article 13 of Decision No 573/2007/EC. The annual allocation for each Member State took account of the target groups (asylum applicants and resettled persons) and the Eurostat statistics available for the last three years. Because of this method, the allocated budget for the Member States shows a delay of three years compared to the trend in asylum applications. This weakness was mitigated to some extent by the allocation for emergency measures to support Member States subject to particular and urgent pressure. The countries most affected by the sudden arrival of migrants saw their allocations increase by 97 % over the 2011-2013 period thanks to EU funding for emergency measures and partly to the annual calculation of their allocation (See section 7). Finding 3: Due to the absence of a threshold, EU funding was concentrated on Priority 1 In the period 2011-2013, funds were allocated to the three funding priorities, plus technical assistance, emergency measures and resettlement activities. In 20112013, over 90 % of projects were implemented under priority 1 (including emergency measures), 8 % under priority 2 and less than 2 % under priority 3. The countries that focused more on the implementation of field and concrete support for asylum applicants addressed urgent day-to-day issues. Figure 4: Programmed and final EU contribution by priority, 2011-2013 Source: European Commission data as of 27.02.2017 (SFC2007) The number of emergency measures (EM) programmed between 2011 and 2013 (accounting for 18 % of the ERF envelope) varied greatly among Member States depending on geographical location and/or the extent of asylum pressure faced by the Member State. Italy was the main beneficiary of emergency measures between 2011 and 2013, with 45 % of the total allocation.

In total, 9 058 persons were resettled by 12 Member States in 2011-2013 using ERF assistance. Numbers varied significantly between Member States, ranging from six in Czech Republic to over 4 400 in Sweden. In comparison to the period 2008-2010, the overall number increased slightly over 2011-2013. Of those resettled, 5 133 belonged to particularly vulnerable groups. B) Implementation through direct management Community actions were implemented through direct management. Around 3.2 % of available ERF funding was used to finance these transnational actions or actions of interest to the European Union. The total programmed EU contribution under direct management was EUR 12.7 million, of which EUR 11.5 million was allocated to Community actions and EUR 1.2 million spent on procuring studies. The envelope was 40 % smaller than that for the previous (2008-2010) period, due to the creation of EASO in 2010. Of the 15 projects selected between 2011 and 2013, 14 closed projects were chiefly concentrated in 2011 and 2012. Absorption rates ranged between 17 % and 99 %. The average absorption rate per project (87 %) is higher than for shared management. Rates increased over time for projects under direct management, suggesting that stakeholders developed greater expertise in the course of the ERF cycle. 8Answers to the evaluation questions 8.1Relevance EQ5: To what extent did the ERF objectives correspond to the needs related to receiving and bearing the consequences of receiving refugees and displaced persons? EQ6: To what extent did the ERF actions correspond to the needs related to receiving and bearing the consequences of receiving refugees and displaced persons? Main conclusions: Overall, Member States confirmed that the formulation of ERF funding priorities and eligible actions reflected their needs. In the context of greater asylum flows, the emergency measures were particularly relevant to address immediate emergency situations. However, the need for greater solidarity and responsibility-sharing between Member States could have been better translated into clear objectives and priorities at EU and national level. Furthermore, ERF annual programmes sometimes lacked a strategic approach. The design of the allocation system (based on historic inflows) did not always properly reflect Member States’ actual needs in view of fast changing refugee trends. However, Member States that had to face the highest migration pressure, such as EL, BG, IT and MT, counter balanced this lack of flexibility of the Fund by making use of the Emergency Measures mechanism. This lack of flexibility of the distribution key is an issue which would need to be addressed in the future Multiannual Financial Framework. Relevance of ERF funding priorities The underlying purpose of the ERF was to improve solidarity between Member States, help them to share the burden of receiving asylum applicants and address the need to harmonise asylum systems in line with the development of the CEAS. During the period under evaluation, most Member States had pressing needs to improve reception conditions for asylum applicants and to facilitate the integration of beneficiaries of international protection. The need to develop reception conditions with higher standards, fairer and more effective asylum procedures and the importance of raising EU citizens’ awareness of beneficiaries of international protection and asylum applicants was underlined in the 2005 impact assessment and constitutes an important element of the CEAS. The ERF priorities responded to most of the needs during the implementation period. One of the major characteristics of the period was the sudden and unforeseen increase of migratory pressure, which led to a drastic increase in the number of asylum applications in Europe, with great variation across Member States. The ERF responded well to this crisis. However, the evaluation showed that the lack of harmonised asylum systems, in line with the development of the CEAS, emerged as a key problem. National differences in legal frameworks related to asylum remain, despite the CEAS, and continue to lead to secondary movements between Member States, of asylum applicants. The solidarity dimension (the need for a ‘balance of effort’ between Member States) could have been strengthened through clearly formulated priorities of the ERF. As a result, these dimensions were insufficiently translated into the Member States’ annual programmes which focussed on the national needs; for instance, few actions related to the transfer of persons (relocation) and implementation of the Dublin Regulation. Relevance of ERFeligible actions All Member States confirmed that the ERF actions met their identified needs in the field of asylum. The diversity of eligible actions ensured that almost all needs, at both EU and national level, were addressed adequately and that new challenges could be met. Member States were also provided with financial incentives to design and implement a resettlement strategy (see Effectiveness of the ERF at results level under section 9.2 for more details about these incentives). The evaluation study assessed that these additional financial incentives were to some extent effective in fostering responsibility-sharing. This can be evidenced in the fact that the number of persons resettled in this way doubled in the 2011-2013 period compared to the 2008-2010 period. The United Kingdom and Finland particularly underlined in their national evaluation reports the positive impact of these forms of support on their implementation of resettlement. Such incentives could have also been used for priority 2 measures and the transfer of persons (relocation); for the latter, the voluntary mechanism was only partially relevant as very few actions related to relocation were implemented Overall, the limitation of resources was a problem for Member States – especially those facing a high influx of refugees, for which the new challenges were not matched by an increase in the allocated budget. The allocation mechanism was based on historic (previous three years’) inflows and did not reflect Member States’ current and evolving needs. As showed on the graph below, Member States that received, generally speaking and in absolute terms, the smallest number of asylum seekers tended to have nonetheless high levels of programmed budget. For instance, HU and IT received the same number of asylum applicants (around 240 000 applications in each Member State) whereas the programmed EU contribution was much higher in IT. As another example, DE was the country with the higher number of asylum applicants, namely around 936 000 applications were submitted, but the programmed EU contribution in DE and UK was the same, whereas the UK received only 160 000 asylum applicants. The allocation system was a problem of the Fund's legal base. The Fund thus lacked flexibility in adapting quickly to changing Member State needs. However, Member States that had to face the highest migration pressure counter-balanced this lack of flexibility by making use of the Emergency Measures mechanism. Indeed, as it can be seen from the graph above, the countries most affected by the sudden arrival of migrants, such as EL, BG, IT and MT, saw their allocations increase by 97 % over the 2011-2013 period. Flexibility was also ensured through the possibility for the Member States to modify their national programmes at any time and to reallocate Funds in the area which required a stronger financial support.

Under the AMIF, the distribution key for the basic allocation to Member States is based on the 2011-2013 allocation for the ERF, the EIF and the RF. A certain flexibility of the funding instrument can be assured by the resources allocated to specific actions and emergency assistance. However, during the 2015/2016 crisis, the lack of flexibility to modify the distribution key or to shift funds from one Member State to another was apparent. The 2015/2016 migrant crisis showed that at a political level, several Member States have largely rejected the idea of responsibility-sharing of asylum applicants developed by the European Commission. For the future multiannual financial framework, the allocation key must be more flexible, for example by providing the legal basis for regular updates or limiting the basic allocation and keeping a 'reserve' that can be distributed later during the implementation of the Multiannual Financial Framework. Relevance of priorities in the annual programmes Overall, Member States considered the annual programmes as reflecting their immediate national needs and some of them based their own priorities on the ERF priorities. Although all three priorities were of interest to all Member States, 91 % of the total programmed budget was allocated to priority 1, which corresponds to the ‘implementation of the principles and measures set out in the Community acquis in the field of asylum, including those related to integration objectives. This was the priority best aligned with national needs. Central and Eastern Europe Member States, where reception capacities were less developed, used priority 1 to improve reception capacities and conditions. Member States with existing reception capacities focused their budget on priority 1 due to the high number of asylum applicants with basic needs. Five Member States made significant use of priority 2. Only the UK dedicated over 10 % of its budget to projects under priority 3. While the proportion of funds allocated to priorities 2 and 3 was relatively small, Member States still found them to be of relevance. 8.2Effectiveness EQ1: To what extent did the ERF actions contribute to the achievement of the objectives defined in Articles 2 and 3 of Decision No 2007/573/EC and to the priorities defined by the strategic guidelines (Decision No 2007/815/ЕС)? Main conclusions: Overall, the ERF was evaluated as having been effective at national level, especially for newer Member States, where it enhanced the capacity to establish national asylum systems. The evaluation showed that, in the EU-15 Member States, the ERF was particularly effective in improving reception conditions for asylum applicants and the integration of beneficiaries of international protection but with some difficulties in implementation. The progress registered by the Fund in the areas of relocation and resettlement was marginal. Some limitations emerged when it came to assessing the overall effectiveness of the ERF at EU level. The ERF monitoring and evaluation system did not allow for a detailed followup and assessment of the performance of the Fund. Set indicators were rather weak and monitoring data were not fully reliable. Complications as regards the construction and implementation of a CEAS also remain. The ERF had only moderate impact in terms of improving solidarity and responsibilitysharing. A better cooperation between the Member States with a centralised and harmonized EU asylum system should, in the future, lead to a more effective and coherent EU framework in the area of asylum. Effectiveness of the ERF at output level The evaluation concluded that all Member States, with some minor exceptions, achieved their quantitative targets. In some cases, targets were overachieved due to the rise in asylum flows and the fact that a wider target group benefited from the projects’ outcomes. However, the effectiveness and impact of the ERF projects was difficult to measure in qualitative terms since Member States took different approaches (see section 6 on methodology). Several sources suggest that the projects in the new Member States focused more on filling gaps and addressing asylum applicants' basic humanitarian needs, mainly accommodation. Conversely, in the EU-15 Member States, projects mainly involved activities to improve the well-being of those in the target groups. Activities to encourage participation by target groups and to improve the professionalism of project beneficiaries were organised. However, the effectiveness of the Fund was hampered when the implementation of projects faced certain internal and external difficulties, namely the long pre-financing period and insufficient capacity among ERF beneficiaries to handle delays in receiving the funding. The lack of interest among target groups themselves was also a problem in seven Member States. Finally, the economic crisis (which accompanied the asylum crisis between 2011 and 2013) with cuts to public resources, higher unemployment and discrimination against refugees all had a negative impact on project implementation and generated among stakeholders the feeling of a degree of underachievement. Effectiveness of the ERF at priorities and policy areas level Effectiveness as regards reception conditions and asylum procedures EQ1A: To what extent did the ERF 2011-2013 actions contribute to the reception conditions and asylum procedures? Main conclusion: In the NERs, all Member States confirmed that the ERF provided a high level of support for the improvement of reception conditions of asylum applicants and asylum procedures. On the other hand, the sharp increase in asylum flows in some cases reduced the quality of the services provided. The improvement of reception conditions in Member States was one of the main areas of focus of the ERF. Of the projects co-financed by the ERF, 47 % concerned reception capacities, reaching almost two thirds of asylum applicants. All Member States stated clearly in their NER that the ERF provided strong support in this area. The main results of the improvements to reception conditions and asylum procedures were as follows: ·measures relating to accommodation, i.e. the creation and enlargement of reception centres, had a visible impact, with a total of 87 000 beneficiaries. ERF funding was used to increase accommodation capacities in eight Member States, improve safety and modernise accommodation centres; ·improved social assistance and legal procedures (measures that directly facilitated the processing of asylum claims through social, legal and administrative counselling for around 662 000 beneficiaries). Emergency measures reduced the backlog in asylum applicants procedures in Greece; and ·medical and psychological care for asylum applicants proved to be a strong source of additional support for health insurance systems, reaching around 269 000 people. Overall, quantitative targets (set in the multiannual programmes by the Member States at the start of the Fund) were over-achieved in all areas (total achievement rates amounted to 140 %), notably because of the increase in migration flows. Moreover, the migration crisis presented a stronger impact in 2014, when it was no longer possible to modify the targets set in the NPs (the latest adoption was in 2013), hence the overachievement mainly refers to the last year of implementation of the Fund. Complementary data and information were collected by the evaluators through interviews with the national authorities and key stakeholders, to complete the picture and carry out also a qualitative assessment.

With the overachievement some difficulties arose, since the planning phase of the project cycle management proved to be more complicated and the values of qualitative indicators may have been reduced (i.e. in AT, it was the case of projects aiming at providing social counselling as the number of asylum seekers was higher than planned: while the number of persons assisted increased, the number of hours of assistance delivered per person could not always be offered as planned). Effectiveness as regards integration of people in the target groups EQ1B: To what extent did the ERF 2011-2013 actions contribute to the integration of persons referred in Art 6 (target group)? Main conclusions: Stakeholders assessed the ERF’s contribution to integration as positive. Several highlighted that refugees encountered difficulties in entering the labour and housing markets due to a deteriorating social and economic environment. Improving the process of integrating refugees was another main priority of the ERF. The Fund supported 42 % of all integration projects supported by the Fund, which involved social assistance with administrative/judicial formalities, counselling and legal aid, and language training; these reached more than 95 000 people. Member States’ NERs gave a mostly positive assessment of the ERF contribution to changes and improvements in the field of integration. As regards integration, the evaluation highlights projects with a focus on: ·recognising refugees’ competencies and skills; ·organising common activities with local communities and raising awareness among EU citizens; and ·facilitating refugees’ entrepreneurship. The main outputs of integration measures were as follows: ·social counselling contributed to the integration of over 95 000 refugees in 21 Member States. Such measures were particularly relevant in PL, FR and AT, with 45 000 final beneficiaries (asylum applicants) assisted through activities such as cultural dialogue with civil society, language training, sport events, cooking and cultural events; ·accommodation projects benefiting over 61 000 refugees were very positive considering the difficulty for them to enter the housing market; and ·a Community action was aimed at developing an UNHCR (the United Nations Refugee Agency) online integration tool containing data pertaining to refugee integration and designed to facilitate the exchange of information and raise awareness. On each topic, the number of target group people reached exceeded what was planned (overall achievement rates: 138 %), except in FI, PO, LU and the UK. Some Member States stressed the possible existence of specific obstacles to integration linked to the social and economic environment, including access for beneficiaries of international protection to the labour (AT, DE, LV, PT, SK) and housing markets (NL), so that integration measures were considered as particularly relevant by those Member States. Effectiveness as regards enhancing Member States’ capacity to develop, monitor and evaluate their asylum policies EQ1C: To what extent did the ERF 2011-2013 actions contribute to the enhancement of Member States’ capacity to develop, monitor and evaluate their asylum policies in the light of their obligations under existing and future Community legislation relating to the common European asylum system (in particular practical cooperation activities between Member States)? Main conclusions: The Fund did not prove to be widely effective as regards enhancing Member States’ capacity to develop, monitor and evaluate their asylum policies. Successful projects strengthened cooperation between Member States in the field of country of origin information (COI). Community actions could have been more often used as a tool to strengthen the cooperation between Member States, although the creation of EASO in 2011 led to a progressive transfer of these activities. In most Member States, projects focused on improving the quality of asylum procedures (103 projects), evaluating asylum policies and procedures (40) and measures to collect/share country of origin information (32). These projects were those mentioned most often by Member Statesas having been the most effective. They aimed to improve access to and the sharing and quality of information in Member States and typically encouraged stronger and new forms of cooperation between Member States. For example, under the MedCOI Community action, three Member States developed a sustainable medical database. The ERF supported Finland’s Tellus project, which contributed in an initial phase to the launch of the EASO COI portal, which is now accessible to all Member States. The EASO Agency was set up in 2011, its main priority being to enhance practical cooperation among Member States on asylum by facilitating the exchange of information. This implies that some activities financed under the ERF Community Actions, in particular support of transnational cooperation between Member States on asylum related issues, were progressively transferred to EASO. A number of Member States also focused efforts on elaborating new methodologies and monitoring and improving procedures. The ERF contributed to improve Member States’ capacity to evaluate their asylum policies and to elaborate new methodologies, replacing individual practices, for working with the most vulnerable asylum applicants. It also contributed to improve their capacity to adapt national legislation more rapidly to the Community acquis. As an example, FR’s project to modernise the EURODAC system was considered a time-saving investment which led to FR adapting promptly to the Dublin II Regulation and helped improve the productivity and stability of its national asylum system. BG, DE, BE, DE and RO were of the opinion that seminars and training courses had improved the skills and knowledge of a wide range of staff. Notwithstanding the above positive outcomes, action to improve Member States’ capacity to develop, monitor and evaluate their asylum policies could have been more effective: priority 2 accounted for only 7% of total programming. As regards the shortcomings identified, all interviews done during the evaluation highlighted that the significant and unpredictable increase in the number of asylum applicants led Member States to focus on shortterm effects at the expense of longterm impact. They also pointed out that Member States’ cooperation was insufficiently developed to ensure effectively the development of future legislation pertaining to the CEAS. National projects and Community actions were perceived by the persons interviewed as potentially strong tools for cooperation and methodological progress, but insufficiently used.

Stakeholders' consultations also showed that Community actions could have been more effective. In particular, over half the respondents lacked knowledge of them and some beneficiaries considered that the funding was somewhat low in the light of the high level of competition in the project management area and in the organisation of projects. Effectiveness as regards the resettlement of persons (Article 6(e)) — priority 3 EQ1D: To what extent did the ERF 2011-2013 actions contribute to the resettlement of persons referred to in Article 6(e)? Main conclusions: During the 2011-2013 period, the ERF better contributed to resettlement activities compared to the previous period (2008-2010), mainly because the financial allocation was higher. However, it emerged that resettlement operations remained marginal at EU level, with only a third of Member States participating. The European Commission, under AMIF Regulation, took specific measures with the aim of promoting the resettlement, such as increasing the lump sum granted per resettled person. Under the ERF several aspects of resettlement have been agreed at Union level for the purpose of providing targeted financial incentives, namely through common resettlement priorities and lump sums for each resettled person. Between 2011 and 2013, a total of 9 058 people were resettled by 12 Member States using the ERF, with four using the ERF exclusively to implement such operations, and with a number of resettled people increasing over time . Compared with 2008 - 2010 the number of resettled people doubled. One of the main reason being that, for Member States receiving the fixed amount for resettlement for the first time, EUR 6 000 was allocated per resettled person instead of EUR 4 000. Also, EUR 5 000 per resettled person was allocated for Member States that had received the fixed amount for resettlement once in the previous year. The UK and FI which have actively implemented the resettlement, underlined the increase in the fixed amount per resettled person as a significant incentive. In addition to the funding of resettlement, ERF projects in some Member States funded preparations for resettlement or aimed to establish and develop resettlement programmes. Although the ERF provided incentives for some Member States to conduct resettlement operations, it also appeared that these were too marginal at EU level. A significant number of Member States did not implement any resettlement projects. This is also true of Community actions, under which only one resettlement project was financed. For several Member States, resettlement was not considered as a priority, with therefore a lack of interest to engage in or to develop resettlement activities further. Increased migratory pressure might also have been considered by Member States as a reason not to engage more in resettlement. The European Commission has promoted the principle of resettlement in the AMIF Regulation with the progressive establishment of a Union Resettlement Programme. The Fund should provide targeted assistance in the form of financial incentives: EUR 6 000 or EUR 10 000 for each resettled person. In addition, the AMIF legal basis provides that the budget allocated to each national programme for the purpose of resettlement could not be used for other purposes, i.e. the MS could not request to move this budget to other regular actions. The European Commission, in cooperation with the EASO should monitor the effective implementation of resettlement operations supported under the Fund. For the period 2014-2016, Member States reported the resettlement of over 16 000 people. Effectiveness as regards the transfer of persons under international protection (relocation) EQ1E: To what extent did the ERF 2011-2013 actions contribute to the transfer of persons falling within the categories referred to in Article 6(a), (b) and (c)? Main conclusions: The ERF had a very limited impact as regards the transfer of persons benefitting from international protection (relocation). At the level of Member States through the ERF, no transfer of persons under international protection took place, and only one Community action has been financed over the period. As a consequence of this underperformance under ERF, the European Commission took specific measures under AMIF with the aim of responsibility-sharing between Member States on relocation. Member States’ projects involving the transfer of persons benefitting from international protection were underdeveloped and the evaluation clearly assessed them as marginal. The main objective of responsibility-sharing between Member States was not achieved. Part of the reasons which explain the limited impact is the considerable increase in arrivals of the number of refugees and the newer Member States’ relative lack of experience of managing projects in the field of relocation. However, there was also very limited political interest from the side of Member States to engage in transferring beneficiaries of international protection. In the absence of any obligation, this made it very difficult to achieve meaningful results. More, the ERF did not foresee any additional financial incentive for intra-EU transfers of persons under international protection as it was the case for resettlement. This explains why the European Commission financed through ERF Community action the EUREMA project, which provided an organised framework for preparing and implementing relocation and thus partially achieved planned outputs. It was innovative in that it fitted into the framework of a previous pilot project and relocated 264 people. Unexpected positive results were observed enabling Member States to learn lessons for the future as regards improving and facilitating relocation. In the AMIF Regulation, the European Commission has promoted the principle of relocation by fixing financing resources which could not be transferred to other actions under the national programmes of the Member States. By blocking financial resources specifically dedicated to relocation, the system will probably be more effective than under the ERF. Effectiveness as regards emergency measures EQ2: To what extent did the ERF emergency measures contribute to the achievement of the Fund’s objectives and priorities? Main conclusions: All stakeholders agreed that the emergency measures were highly effective in meeting the specific urgent needs of Member States facing high migration pressure, which arose mainly from external factors (increase in the number of asylum applicants) and so could not be addressed in full in the national programmes. The rapid reaction of the European Commission with fast awarding of EU funding was highlighted as positive. In 2011-2013, a total of 11 Member States facing urgent needs implemented 51 emergency measures using a programmed budget of EUR 67.1 million. The aim was to provide assistance in fast-moving circumstances with a focus on immediate results. As first countries of arrival in the Mediterranean, Italy and Greece were the main beneficiaries of emergency measures. Malta and Cyprus also received assistance following an important influx of migrants into these Member States in order to complement their small national allocations, which were too low to address this sudden arrival of migrants. The additional assistance represented 71 % of Cyprus’s 2013 budget and 76 % of Malta’s. Most of the measures were implemented in 2013, in response to the EU’s asylum crisis, and concerned eight Member States with immediate needs. Funding for these types of action was assessed case by case. The main actions focused on: ·improving accommodation capacities and renovating reception centres; ·recruiting additional staff in the main institutions dealing with reception and asylum procedures; ·medical and psychological assistance;

·providing for basic needs; and ·legal assistance and translation services. Generally, emergency measures were perceived as very effective and described by some Member States as success stories. This is explained by the fact that the measures are designed to cover specific needs that are easily quantifiable (e.g. the number of additional places in reception centres or the procurement of new medical tools/instruments/equipment), allowing some Member States to extend economies of scale with greater effectiveness and efficiency in a short (sixmonth) timespan. All the interviewees saw the rapid implementation of the emergency measures as having been essential in meeting the urgent needs. The evaluation also pointed out that the European Commission responded well by being very reactive to this type of difficult situation. Respondents underlined that the situation would have been worse without the additional assistance provided through emergency measures. On the other hand, three Member States considered the short timeframe for the implementation of measures as having reduced their effectiveness as the requirement to spend a large amount of money within a short timeframe proved difficult. …. Even if the political context was difficult, the AMIF instrument through its emergency assistance has provided as from 2015 strong support to the Member States most affected by the 2015 migrant crisis. With the arrival in the second half of 2015 of more than 1 million third-country nationals via the Western Balkan route, the mechanism was flexible enough to provide at short notice financial assistance to address urgent and specific needs in the event of an emergency situation. For the period 2014-2016, EUR 498 million have been awarded to 14 Member States, 2 EU agencies and 2 international organisations through AMIF emergency assistance. Greece, the country most affected by the migrant crisis, is the main beneficiary of emergency assistance. Effectiveness of the ERF for the development of a common approach for asylum The ERF supported Member States in the field of asylum and contributed to the development of reception conditions for asylum applicants and the integration of beneficiaries of international protection. It has had a direct effect on the field, supporting over a million people through over 1 500 projects, with an estimated coverage of 56 % of all asylum applicants and beneficiaries of international protection. However, while EU funding was effective in achieving progress under the various ERF specific objectives, its implementation was not accompanied by the development of a common approach of Member States in the processing asylum applications across Europe. As described earlier, for instance, Central and Eastern Europe Member States used the ERF mainly to increase and improve reception conditions, since these were insufficiently developed (due to relatively low numbers of asylum applicants in the past). The ERF contributed to a certain extent to the implementation of the asylum 'acquis' in the Member States. The main focus was on the implementation of the Reception Conditions Directive. The ERF had little direct influence as regards responsibility-sharing between Member States, especially regarding the transfer of persons under international protection (relocation). Although the results and achievements of the ERF were assessed differently from one Member State to another, the evaluation identified several trends at EU level: ·The ERF played a crucial role in the context of the asylum crisis that started in 2012 and provided essential support in situations that called for the adoption of emergency measures. Community actions helped in urgent situations requiring prompt and fast intervention (in Greece). In that regard, it was also mentioned during the interviews that the Fund contributed to the development of a European approach and to the enhancement of cooperation, for example by developing constant dialogue amongst experts. ·The ERF played a key role in encouraging decision-makers to involve the Member States in resettlement operations, as confirmed by Member States that received a fixed amount per resettled person. ·The ERF gave valuable support to Member States’ efforts in financing operational and locally based projects focusing on reception conditions for asylum applicants and the integration of beneficiaries of international protection. Through concrete actions, the ERF contributed to a certain extent to the implementation of the asylum package and raised reception standards and living conditions for asylum applicants; ·The ERF had a limited impact on the development of interaction and practical cooperation between Member States. The ERF funded a few large transnational projects that brought clear results but practical cooperation remained overall limited following the creation of EASO which was entrusted with some practical cooperation tasks from 2010 (leading to the reduction of the CAs’ envelope in order to free up resources for funding the EASO), as well as by the emphasis put on emergency measures as from 2013. In addition, the results of Community actions were not sufficiently disseminated at EU level and among national stakeholders. ·The ERF had a very limited impact in involving Member States in actions involving the transfer of people under international protection. Effectiveness of the principle of solidarity The principle of solidarity, common to the four SOLID Funds has two components: the solidarity between Member States and the financial solidarity. In terms of solidarity between Member States, while the EC made significant efforts to advocate for solidarity and a common EU approach in response to present and future migration challenges, this vision was not always shared by all EU Member States to the same extent. The objective of the ERF was to support and encourage through a solidarity mechanism the efforts made by the Member States in receiving and in bearing the consequences of receiving refugees and displaced persons. The ERF had a slow start due to Member States willing to protect their sovereignty on managing asylum related issues. When the priorities were set, the enhancement of responsibility sharing between Member States and third countries remained optional following bitter discussion with Member States. Whilst the experience under ERF showed the need to increase the solidarity amongst Member States, this remains a difficult subject: the current discussion on the proposal for a revised Dublin regulation shows that consensus on solidarity matters is far from being achieved. As explained above (see section 9.1 on Relevance), in terms of financial solidarity, the initial allocation key, which is the first means to achieve solidarity, showed its limitation in the context of the crisis. For this reason, the use of emergency measures proved to be crucial for those Member States which were subject to acute and urgent migration pressure. Emergency measures partially counter-balanced the lack of flexibility of the distribution key. Indeed, the countries most affected by the sudden arrival of migrants saw their allocations increase by 97 % over the 2011-2013 period thanks to EU funding for emergency measures and partly to the annual recalculation of the distribution key. This illustrates how the financial solidarity was achieved. 8.3Efficiency EQ 3: To what extent were the effects of the ERF achieved at a reasonable cost in terms of financial and human resources? Main conclusions: Although the inconsistency of monitoring data restricted the extent to which it was possible to assess the cost-effectiveness of the ERF, the evaluation concluded that the actions were implemented efficiently for the most part. Management and control systems, such as internal organisation and a clear distribution of tasks within the responsible administrations for managing the Fund, contributed greatly to this result and their benefits were acknowledged by the Member States. However, the evaluation showed that the annual programming requirements and the complexity relating to the existence of separate funds led to a high administrative burden. Small Member States had to deal with a combination of high administrative costs and small programmes. Lack of experience and resources also contributed to the high administrative burden. Efficiency of ERF programme management

Overall, RAs considered the ERF costly to manage, although the most experienced Member States were able to reduce the administrative burden. The evaluation estimated that the administration of national programmes required an average of 0.32 fulltime equivalent (FTE) per EUR 1 million of ERF programmed funding. The figure varies a lot between Member States though (from 0.06 FTE in FR to 2.97 FTE in LV), and is lower for the Member States with the largest ERF programmes. This average is consistent with the estimate for the cohesion policy (0.44 FTE). The EU contribution was also a factor explaining the varying proportions of the ERF budget dedicated to management. The Member States with larger ERF programmes were also those where the smallest percentage of the EU contribution was dedicated to fund management due to a critical mass effect. The main timeconsuming tasks (56 % of the RAs’ workload) were those relating to ERF management, i.e. project selection, control and verification of deliverables. The workload relating to the preparation of the programmes was estimated at a fifth of the total. All Member States pointed to the high administrative burden of ERF management (i.e. the burden linked to multi-annual and annual programming, and addressing asylum and migration issues via three different funds which under the current AMIF has been overcome) but RAs regarded most of the requirements as legitimate, in that they ensured the robustness and reliability of the ERF management and control systems. The evaluation showed that, for half of the Member States, the administrative burdens, especially verifications by the RAs, were the most problematic. Some Member States tried to use the same human resources for the ERF as for the other SOLID funds, in order to reduce the administrative burden and secure efficiency gains. In Member States with less experience of managing EU funding, many costs arose from the need to build technical and administrative capacities. In some cases, the perception of a high administrative burden applied more at national than at EU level. For instance, FR and SE faced financial difficulties (gaps in EU reimbursement) following a misleading interpretation of ‘eligible costs’. Other Member States had difficulties due to the long pre-financing phase, which caused cashflow problems for beneficiaries waiting for EU reimbursement, or the lack of a clear distribution of responsibilities (BE). Efficiency of the ERF for beneficiaries Time-consuming arrangements for (bi-annual) reporting and payment delays were mentioned by some countries as affecting the efficiency of the Fund. On the other hand, administrative burden is often linked to institutional learning. The evaluation showed that beneficiaries' size, nature and experience of managing EU finding influenced their ability to meet the ERF requirements. Member States and RAs convey the European Commission’s requirements to national beneficiaries, including small organisations, and to build capacity to respond properly to the requirements. Overall, the estimated cost of meeting the European Commission’s administrative requirements is evaluated as high, but reasonable as compared with other EU funding. Cost-effectiveness of projects The quantitative data on ERF reported in the NERs do not adequately reflect the actual performance of ERF actions in the Member States, but 22 Member States considered the costs incurred as reasonable. The data cannot support any clear conclusion, given the lack of comparability across projects and the shortcomings of the available indicators when it comes to measuring results and impacts. Each project targeting asylum applicants or persons benefiting from international protection (priority 1) reached an average of around 700 final project beneficiaries, with an average budget of EUR 150 000. The average committed cost per beneficiary (including emergency measures varied widely between Member States (from EUR 9 065 in the UK to EUR 37 in SL). This is probably related to the range and the nature of the activities carried out under the projects. For instance, UK policy in 2011-2013 was focused on resettlement and integration. By nature, this involves highvalue projects benefiting a relatively small number of people. For priority 2, the average cost of a project was EUR 123 553 and for priority 3 it was EUR 301 037. Member States receiving a high number of asylum applicants and beneficiaries of international protection have relatively low budgets per project beneficiary (e.g. only EUR 44 for EL). This is linked to the nature of the projects: these Member States target projects enhancing reception conditions or meeting the basic needs of asylum applicants. The discrepancies put a question mark over the reasonability of costs and the relevance of the allocation system. In HU, for example, the national budget allocated to asylum issues was reduced for political reasons and the ERF became more important in terms of meeting asylum applicants’ needs. 8.4Sustainability EQ4: To what extent did the positive effects of the ERF actions last after the interventions were terminated? Main conclusions: The evaluation concluded that the sustainability of the projects' results funded under the 2011-2013 ERF depended on the type of action funded. In general, the ERF led to the consolidation of sustainable relationships with stakeholders, but the most sustainable actions (presenting structural effects) were Member States’ investments in reception capacities. For other types of project, certain conditions regarding the selection of sustainable projects could have been set in order to produce sustainable effects. As for the sustainability of projects after the end of ERF funding, this was more questioned. The selection phase of AMIF projects involved sustainability as criterion for awarding a project. Sustainability of results and impacts The sustainability of a project is linked to its nature, impacts and results. The creation of reception capacities is highly sustainable, especially for Member States that lacked them previously. Analysis of the NERs and interviews with RAs and beneficiaries reveal that actions aimed at empowering target groups, those resulting in greater capacities to receive asylum applicants and those resulting in a concrete change of practice are considered among the most sustainable. Actions to improve the drawingup of studies and to develop platforms, websites and new tools or methodologies have also proven to be sustainable, since the outputs and materials can continue to be used, provided they are updated and maintained. Projects focusing on the integration of beneficiaries of international protection are considered less sustainable in transit Member States, where beneficiaries of international protection like asylum applicants tend to seek to move on to another country. Sustainability of projects The implementation of ERF projects produced side-effects that improved project sustainability. Thirteen Member States highlighted the importance of capacitybuilding among stakeholders in the field of asylum as a key factor, since expertise favours actions’ long-term effectiveness. Some Member States commented that the implementation of ERFfunded projects favoured a ‘culture of cooperation’ among stakeholders. However, the sustainability of projects after the end of ERF funding is uncertain. Beneficiaries’ capacity to sustain the projects’ effects is not ensured without the support of EU funds. A total of 13 Member States stressed that national beneficiaries rely heavily on EU funds to implement projects. Moreover, some RAs replicate projects over the years in order to ensure a type of sustainability of their effects. For the AMIF during the selection phase of projects by the responsible authority, the sustainability is now included as a criteria leading to the award decision. Through the guidance document set by the common monitoring and evaluation framework, Member States will have to report on the measures adopted to ensure sustainability.

8.5Complementarity and coherence EQ7: To what extent were the ERF actions coherent with and complementary to other actions related to asylum, financed by other EU financial instruments – including the activities of EASO – and from national resources of Member States? Main conclusions: The evaluation concluded that, in most Member States, the ERF was implemented in coherence with, and complemented, other actions relating to asylum, although there may have been some overlaps between actions or EU agencies. There was however a general lack of awareness at national level of the outputs of Community actions and this might have led to a certain risk of incoherence. Coherence and complementarity at national level Of the 27 Member States concerned, 26 affirmed that the ERF was used in coherence with, and complemented, other sources of funding, thanks to active cooperation between the ERF RAs and the government authorities responsible for managing other EU funds. Also, in most Member States, coordination between the ERF and other SOLID funds was ensured by the fact that they were managed by a single RA. However, some risks of duplication were identified. The EIF, the European Social Fund and a significant number of organisations that operate in the field of asylum at both EU and national level may have created overlaps. Nevertheless, RAs addressed these risks through organisational measures, e.g. adopting national asylum strategies and close cooperation between the stakeholders responsible for managing various funds. Lastly, many Member States pointed out that Member States’ coordination arrangements were subject to demanding control requirements from the Commission. Coherence and complementarity at EU level At EU level, the coherence of ERF interventions with those implemented by other organisations (e.g. EASO) and the extent to which ERF Community actions and national projects complemented each other was not raised as an issue. As Community actions and national actions are not supposed to target the same objectives, the risk of overlaps was limited. Nonetheless, stakeholders consulted underlined their lack of awareness about the outputs of Community actions. This was a limit to RAs’ ability to create transnational networks and capitalise on previous outputs when selecting national projects, which in turn could lead raising a potential risk of incoherence between Community and national actions. Although it remained a theoretical concern as the evaluation could not confirm it, overlaps and duplications with EASO might have occurred since parts of the agency's mandate were at the time still covered by Community actions. However, the risk of inconsistency in terms of mandate and action remained low, as EASO operated from a ‘continuous activity’ perspective, while the ERF was project-based. It would have been desirable to involve EASO in the selection of priorities for Community actions. 8.6EU added value EQ8: What is the additional value of the ERF actions compared to what the Member States would have been able to carry out through investments necessary for the implementation of EU policies in the field of asylum without the support of the Fund? Main conclusions: The evaluation concluded that all Member States considered that the EU funds allowed for funding innovative projects or projects aimed at specific target groups which, without the ERF, would not have been implemented. In addition, the added value of the ERF was crucial for Member States with less developed asylum systems. For some priorities, it is questionable, however: there was no added value as regards Member States’ cooperation and exchange of best practices. According to the ERF Decision, the expected added value of the Fund was ‘to promote a balance of efforts between Member States in receiving refugees and displaced persons’, since this objective could not be sufficiently achieved by the Member States acting alone. Added value brought in by the Fund was therefore assessed as regards the financial impact of the funding ("financial added value") and the added value resulting from greater cooperation and more effective practices within Member States. Financial added value The financial added value of the ERF was widely recognised by beneficiaries and RAs. All 27 Member States confirmed that it would not have been possible to implement additional projects and actions without the support of the Fund. Three quarters of the beneficiaries involved in the ex post evaluation survey confirmed this. A total of 13 Member States stated that the ERF was used to target types of service that go beyond what the state is responsible for or able to deliver. In some Member States with less developed or recent asylum systems, where asylum issues were not a priority and national resources were limited, the ERF contributed significantly to the establishment of national asylum systems. In most other contexts, it provided an opportunity to finance innovative projects by testing new methodologies or facilities, to enlarge the scope of activities by financing additional services outside the remit of the state or to focus more on specific target groups. Finally, the importance of ERF funding as a proportion of a Member State’s total asylum budget tended to follow two trends. In Western and Northern Member States with mature asylum systems, it was very low (e.g. less than 3 %) as it only financed a small fraction of the overall costs for Member States of running their asylum systems. For other Member States, it reached up to 50 %, indicating clear and vital added value. In these Member States, the ERF played a crucial role in the modernisation of the asylum system: it allowed the introduction or improvement of structures in compliance with the EU asylum acquis and it is reasonable to assume that much less or nothing would have happened without the ERF. In a system of 'perfect' solidarity, the ERF should probably have to finance all costs related to maintaining the asylum systems, however, this would mean that the annual budget of the ERF should have been much more important. Added value in terms of cooperation and exchange of best practices The assessment of the added value of the ERF in terms of development of deeper cooperation between stakeholders in the field of asylum was mixt. According to the interviews with RAs, only a few networks were created or sustained through ERF funding and the sharing of best practices was not as widespread or intensive as expected. Some Member States did not see the ERF as a tool to foster cooperation and responsibilitysharing. Nevertheless, some said that the ERF had influenced the design of their national policies. On the other hand, according beneficiaries, the ERF brought added value to project beneficiaries in terms of professionalisation of organisations, improved knowledge of asylum issues, and recognition and legitimacy due to the European dimension of the ERF. According to 19 Member States, the Fund enabled national project beneficiaries to improve their internal processes, enhance their projects and budget management skills, and implement new evaluation methodologies. 9Conclusion Throughout this evaluation, the ERF 2011-2013 actions have been assessed on the basis of their contribution to the establishment of the burden sharing and solidarity system required by Member States for receiving and bearing the consequences of receiving refugees and displaced persons.

Over this period, the ERF was implemented in a rapidly changing context which posed many challenges, and was shortly followed by a severe and unexpected migratory and humanitarian crisis in the Member States. In this context, the ex post evaluation provides evidence that the ERF responded effectively to those challenges and delivered a wide range of results. The ERF has been effective at Member States level to implement the operational aspects of the European asylum system by increasing the capacities of the Member States. In some Member States with less developed or recent asylum systems, where asylum issues were not a priority and national resources were limited, the ERF contributed significantly to the establishment of national asylum systems. The Fund provided an opportunity to finance innovative projects by testing new methodologies or facilities, to enlarge the scope of activities by financing additional services outside the remit of the state or to focus more on specific target groups. However, the Fund contributed only partially to the development and implementation of EU policy and legislation in the field of asylum due to the fact that following the unexpected migratory crisis, Member States focussed their efforts on the basic needs of the asylum applicants. The allocation system was also not relevant to the EU needs of convergence, solidarity and development of the CEAS. It relied on historical flows of asylum applicants. It should be pointed out that the effectiveness of the Emergency Measures mechanism mitigated this inconsistency. They played a key role in providing support to those Member States facing the greatest migration pressure. Furthermore, the ERF contributed to the increase of the Member States' capacities and appropriate infrastructures, especially in Member States, which lagged behind in terms of asylum policy development. The evaluation helped to identify some lessons for the future, most of them have been addressed under AMIF. First of all, the formulation of Priority 1 was broad and covered potentially all types of needs. This created an unbalanced uptake at national level for the other priorities of the Fund. Under the AMIF Regulation, the article related to Reception and Asylum Systems is still broad. However, Member States indicated their priorities in the National Programmes that were adopted after policy dialogue with the European Commission. Compared to the ERF programming period, in AMIF the setting-up of the national programmes is the result of the outcomes of a policy dialogue between the European Commission and the Member States. Another issue identified during the evaluation was the absence in the ERF of effective monitoring and evaluation mechanisms. The European Commission has addressed this issue for the 2014-2020 programming period by developing a common monitoring and evaluation framework. Regarding efficiency, the evaluation identified the lack of efficiency of funding to support less experienced Member States or those facing difficulties in the implementation of their management and monitoring systems. In consequence, in the context of AMIF, more emphasis on strategic programming was put through a closely follow-up of those Member States facing difficulties in the implementation of the funds. Furthermore, the AMIF foresees the organisation of a mid-term review of the multiannual national programs where the situation of each Member State is reviewed in the light of the interim evaluation reports and in the light of developments in Union policies in the Member State concerned. Finally, the administrative burden of the ERF was perceived as high. Improvements were made for the 2014-2020 programming period by the creation of only one fund that covers all migration matters, the adoption of one single multiannual programme and the alignment of the eligibility rules on the national rules of the Member State. These latter are also encouraged to use the simplified costs options. Another lesson from the evaluation was the solidarity dimension. When the ERF priorities were set, the enhancement of responsibility sharing between Member States and third countries remained optional. For this reason, certain issues related to resettlement and relocation were already addressed under AMIF by removing the optionality and by putting this specific objective at the same level as the others. However, the current discussion on the proposal for a revised Dublin regulation shows that consensus on solidarity is difficult to achieve and that improvements are still needed. This issue should be addressed where possible during the current programming period or during the next MFF. Lessons learnt As stated above, the allocation mechanism of the ERF was based on historic inflows and did not take enough into account evolving needs of the Member States. Under AMIF, the distribution key is still based on the 2011-2013 allocation for the ERF, the EIF and the RF although more flexibility is ensured by a much more important envelop allocated for emergency assistance. Moreover, as explained in section 5, the logical framework of the ERF was not well designed: the articulation between general objective and eligible actions, on the one hand, and between them and the priorities (and specific priorities) on the other hand, remained confused. Whilst the logical framework has been better defined with the AMIF, under each specific objective there is still the possibility for Member States to avoid investments in areas less "attractive". This issue should be addressed in the new generation of funds post 2020. Finally, as enounced above, AMIF successor should pronounce and fund increasingly the activities that promote solidarity within the EU to jointly stem the challenges resulting from high migration. The interim evaluation of AMIF will be completed in 2018 and will inform if the identified weaknesses have been addressed properly. The interim evaluation is expected to have an impact on the implementation of the AMIF in the remaining spending period 2019-2020. If not, this input will be taken up in the design of the post 2020 instruments.

10ANNEX 1 — Procedural information Leading DG DG HOME Participating Units of DG HOME A2 – Legal Affairs

C3 – Asylum

C4 – Migration management support

E1 – Union Actions

E2 – National programmes for South and East Europe, evaluation, AMF/ISF Committee

E3 - National programmes for North and West Europe, budget, MFF, agencies Participating DGs Secretariat General DG BUDG DG JUST Legal service (SJ) Roadmap approval December 2015 External consulting firm specialised in evaluation Contract signed in April 2016 with consortium of: Ernst & Young Special Business Services (lead partner) Center for International Legal Cooperation Rand Europa Community Interest Company Number of steering group meetings 6 Last deliverable handed in March 2017 Approval of the final report by Steering Group March 2017 RSB consultation Agenda planning 2016/HOME/081



Changes in response to the Regulatory Scrutiny Board's recommendations (Ares(2017)287965 of 8/06/2017) Main considerations Explicitly acknowledge that the intervention logic is weak, and that this makes it harder to evaluate the programme's results. Section 4 ‘The ERF as funding instrument Clarify how the evaluation assessed the Fund's effectiveness in the absence of specific EU-level objectives and targets. Section 6 ‘Method’ and Section 9.2 ‘Effectiveness’ Demonstrate how the ERF applied the solidarity principle in practice. Section 7.1 ‘Shared management: Multiannual and Annual Programmes of the Member States’ and the explanations related to the allocation key Show the extent to which emergency measures contributed to the Fund's flexibility. Section 9.1 Streamline and harmonise the conclusions, and indicate lessons that are relevant for future action Conclusion





11ANNEX 2 — Stakeholders’ consultation Overview In order to inform the ex post evaluation of the European Refugee Fund (ERF) 2011-2013, several stakeholders were consulted through open public consultations conducted by the European Commission and other stakeholder consultations carried out by the external evaluator, in the framework of its contract with the European Commission. This annex provides an overview of the consultation processes and the type of stakeholders consulted, and presents the results of these consultations. These results have been compiled and summarised from the responses received by the European Commission to its public consultation, and from the final evaluation report of the ERF submitted to the European Commission by the external evaluator. The stakeholder consultations presented below have met the European Commission's general principles and minimum standards for stakeholder consultation. The public consultation conducted by the European Commission Between 10 May 2016 and 9 August 2016, the European Commission held an Internet-based open public consultation on the ex-post evaluation of the ERF 2011-2013 in the form of an online questionnaire. Contributions were sought in particular from individuals (experts, beneficiaries), local and national Member State authorities, intergovernmental and non-governmental organisations, social partners and civil society, academic institutions, international organisations, and EU Institutions and Agencies. The final number of respondents was very small: 12 participants from eight Member States. This is clearly a limitation of the public consultation. Eight representatives of a non-profit organisation, two representatives of a public authority, one representative of an academic/research institution and one representative of a private company. A limitation to take into consideration is the fact that the information available on the identities of the participants is based on self-reported values which cannot be verified. The consultations conducted by the external evaluators The external evaluator conducted several types of stakeholder consultations, including: 8 case studies, 4 types of face-to-face or telephone interviews, and 2 e-surveys targeted at 2 specific types of stakeholders. This section provides an overview of each type of consultation, as well as possible difficulties or limitations encountered. The design of these consultations was informed by previous desk research carried out by the external evaluator. When relevant, findings from this desk research will also be presented, particularly those of the National Evaluation Reports (NERs). 1.Case Studies In order to gain an in-depth understanding of the ERF's implementation at the national level, 8 case studies were completed on Austria, Belgium, Bulgaria, France, Hungary, Italy, Sweden and the United Kingdom. As part of these case studies, interviews with organisations benefitting from the ERF were carried out. The following activities were carried out for each case study country: ·Austria: 10 interviews conducted during a field visit which took place in July 2016; ·Belgium: 10 interviews conducted during a field visit which took place in September 2016;

·Bulgaria: 12 interviews conducted during a field visit which took place in July 2016; ·France: 11 interviews conducted during a field visit which took place in July and September 2016; ·Hungary: 11 interviews conducted during a field visit which took place in July 2016; ·Italy: 9 interviews conducted during a field visit which took place in July 2016; ·Sweden: 9 interviews conducted during a field visit which took place in August 2016; ·United Kingdom: 11 interviews conducted during a field visit which took place in July 2016. 2.Interviews Interviews with Responsible Authorities (RAs): 19 telephone interviews with RAs were initially planned, but only 17 of these were conducted. In addition, two Member States provided written feedback. Interviews with EU-level stakeholders: 3 interviews were conducted in September 2016 with officers from the Directorate-General for Regional and Urban Policy (DG REGIO), the Directorate-General Migration and Home Affairs (DG HOME) and the European Asylum Support Office (EASO). Two other stakeholders initially contacted declined interview requests or suggested another stakeholder, and there were no replies to interview requests sent to members of the LIBE Committee, ECRE and the European Union (EU) Council. Interviews with other International Organisations: field visits were carried out to Member States selected for case studies, which provided an opportunity for the external evaluator to meet local branches of UNHCR and/or IOM. A complementary interview was carried out with the UNHCR representation in Brussels. Beneficiaries of the 5 Selected Community Actions (CAs): Following the initial selection of 5 CAs, interviews were conducted with 13 organisations: all the selected CA project leaders were interviewed as well as a selection of co-beneficiaries in 8 Member States chosen for the case study visits. Difficulties encountered regarding the availability of key stakeholders: the external evaluator encountered challenges with the scheduling of interviews and case study field visits, due to the fact that interviews took place during the vacation period. Some Member States also faced work overload, or had particular national contexts which limited interview availabilities of RAs. The external evaluation team had to postpone some case study field visits and phone interviews with RAs until after the summer break, between the end of August and the beginning of September. In addition, given high staff turnover rates in national administrations and beneficiary organisations, the external evaluation team sometimes encountered difficulties finding the appropriate. 3.E-Surveys Two specific e-surveys were launched targeting two different stakeholder groups: 1) E-survey targeting beneficiaries of CAs (leading partners) for 2008-2013: this e-survey was open between 11 July and 1 September 2016 and sent to 56 leading partners. Responses from 17 participants were received, 13 of which were exploitable for analysis. 2) E-survey targeting all beneficiaries of the Annual Programmes for 2011-2013 in the 8 case study countries: this e-survey was open between 29 September and 20 October 2016. 64 participants provided responses out of the total 276 who received the e-survey invitation. 55 of these responses were exploitable for analysis. Results The most relevant results of the consultations carried out by the European Commission and the external evaluator are grouped in this section and presented according to the following evaluation criteria: relevance, effectiveness, efficiency, sustainability, coherence and complementarity, and EU added value.

Relevance Generally, the RAs and beneficiaries consulted found that the actions funded by the ERF were relevant (ERF objectives corresponded to their needs) both at EU level and for individual Member States. Question 10 of the Commission's public consultation asked participants whether, in their opinion, the projects and activities supported or financed by the ERF in their country of residence addressed the needs of the potential beneficiaries. 50% of respondents replied "Yes" and the other 50% replied "Generally yes, but with some problems". Some of the problems highlighted by these respondents for this period were: ·small associations were said to have difficulty fulfilling all the requirements or "rules" of the projects; ·there was thought to be a lack of financial resources at times, for example, sometimes resources were insufficient in relation to the number of applicants or potential beneficiaries; ·limiting the eligibility for funding to TCNs was thought to exclude naturalized citizens of immigrant origin; ·it was also mentioned that the limited implementation period in relation to a large gap time period resulted in the development of a belief from the side of the target group that the projects did not make any difference to their lives; ·the amount of resources available was limited compared to the increasing number of asylum-seekers arriving in Member States from 2013 onwards. The problem highlighted by Member States was the restrictive definition of the target group of the fund, which was found to create difficulties for beneficiaries in implementing projects. An additional problem mentioned by national authorities was the fact that the amount of resources available was limited compared to the increasing number of asylum-seekers arriving in Member States from 2013 onwards. Effectiveness In the area of effectiveness, the two most relevant questionnaire questions in the European Commission's public consultations were questions 2 and 5. Question 2 asked participants whether the result(s) sought in the general objective of the ERF had been achieved in the respondent's country. Question 5 asked whether, based on respondents' experience, the actions financed by the ERF in their country were consistent with the general and specific objectives of the fund. For question 2, the vast majority of participants considered that the general objective of the ERF had been achieved in their country at least to some extent. For question 5, the majority of participants found that actions financed under the funds were indeed consistent with the fund objectives. However 50% of respondents thought this was so to a limited extent. Results from other stakeholder consultations provide additional details regarding whether the results sought in the general objective of the ERF have been achieved in the respondent's country. One of the main specific objectives of the ERF was the improvement of reception conditions. In the NERs, Member States reported that the ERF brought strong support to the improvement of reception conditions of asylum applicants or other persons seeking protection. This support was mainly dedicated to providing social assistance, counselling and legal aid and language/interpretation assistance (38% of operations), as well as bringing medical and psychological care (19%) and providing accommodation infrastructures and services (15%). Regarding integration, which wa