# SWD_2017_0602_FIN.ENG.xhtml.6_EN_autre_document_travail_service_part1_v3.docx

european flagEUROPEAN COMMISSION

Brussels, 15.12.2017

SWD(2017) 602 final

COMMISSION STAFF WORKING DOCUMENT

Evaluation of the European Neighbourhood Instrument

Accompanying the document

REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL

Mid-term review report of the External Financing Instruments

{COM(2017) 720 final}{SWD(2017) 463 final}{SWD(2017) 600 final}{SWD(2017) 601 final}{SWD(2017) 604 final}{SWD(2017) 605 final}{SWD(2017) 606 final}{SWD(2017) 607 final}{SWD(2017) 608 final}{SWD(2017) 609 final}

Contents

Executive summary

Purpose of the evaluation

Scope of the evaluation

2. Background of the initiative

Description of ENI and its objectives

Baseline

3. Method

4. Implementation state of play

5. Answers to the evaluation questions

6. Conclusions

Annex 1. Evaluation Questions

Annex 2. Procedural information

Annex 2bis Response to the opinion of the Regulatory Scrutiny Board

Opinion of the Regulatory Scrutiny Board

Title: Evaluation / European Neighbourhood Instrument

Overall opinion:
     1st submission: NEGATIVE

2nd submission: 28/09/2017 - POSITIVE

Annex 3. Synopsis report of the stakeholders' consultation

Consultation strategy

Stakeholder mapping

Stakeholder consultation strategy

Stakeholder consultation framework

Timeframe

Stakeholder statistics

Open Public Consultation (07/02/17 - 05/05/17)

Annex 4. Acronyms

Annex 5. External evaluators' report, including its annexes and methods and analytical models used

Executive summary

This Staff Working Document evaluates the European Neighbourhood Instrument (ENI) at its mid-term, covering the period January 2014 to June 2017. With a budget of EUR 15.4 billion, the ENI is the largest financing instrument within the EU budget supporting the implementation of the European Neighbourhood Policy (ENP) in the 16 Neighbourhood countries. Its overall objective is to advance further towards an area of shared prosperity and good neighbourliness by developing a special relationship founded on cooperation, peace and security, mutual accountability and a shared commitment to the universal values of democracy, the rule of law and respect for the human rights. The main findings are as follows:

The ENI remains relevant and fit for purpose. It has allowed the EU to implement the reviewed Neighbourhood Policy. It has proven its flexibility, in line with the impact assessment, by reacting to the multiple crises and new challenges in the Neighbourhood, in particular in Ukraine, Tunisia and in relation to the migration crises. The implementation of the principle of differentiation has allowed the EU to adapt its support to partner countries’ needs and ambitions (e.g. by more than doubling EU assistance to Tunisia), as envisaged in the ENI impact assessment. However, the response capacity of the instrument has been stretched to its limits.

As regards effectiveness, ENI presents a mixed picture. In the field of human rights, democracy and governance, the political context in many countries has made it difficult to pursue comprehensive support strategies and impeded the achievement of significant results. However, EU support in the area of economic governance and trade has contributed to the improved business environment in several countries in the Eastern Neighbourhood, but also Morocco, as well as to the increased trade potential between the Neighbourhood countries and the EU. Budget support under the ENI has contributed to macroeconomic stabilisation in important EU partners such as Jordan, Moldova, Tunisia and Ukraine. The incentive-based approach has been only partly successful in promoting deep and sustainable democracy; it has been more effective in supporting those partners committed to reforms (in particular Georgia, Morocco, Tunisia and Ukraine). Overall, the prospects for ensuring the sustainability of reforms are limited in several countries because of the unfavourable political environment.

Within the Commission, the ENI is delivering aid efficiently in terms of procedures, processes and the ratio of administrative costs, with an adequate organisational set-up and quality in-house expertise (although concentrated in only some key priority areas). The ENI has a clear EU added-value resulting from its ability to provide substantial funding in the form of grants, its capacity to mobilise other EU instruments and loans from financial institutions, its capacity to mobilise EU Member States' expertise and its policy leverage through dialogue with national authorities and civil society. It offers a unique portfolio of aid modalities which can be used in a coherent way and adjusted to the partner country’s needs and capacity. None of these features can be achieved individually by EU Member States.

Significant efforts are made to coordinate ENI and other external financing instruments programmes, although there is scope to increase the coordination between country and regional programmes. Coordination with the Member States has increased, in particular through the development of joint programming approaches (e.g. in Moldova and Palestine). The blending of ENI grants with loans from other financial institutions (in particular those of European Financial Institutions), attracted significant additional resources to achieve ENI objectives. Policy dialogue and cooperation leveraged the implementation of agreed reforms in a number of countries (e.g. public administration reform in Ukraine). This leverage is stronger when budget support is used as implementing modality, because of the greater depth of policy dialogue with the beneficiary throughout the life of the operation.

1. Introduction

Purpose of the evaluation

This document sets out the results of a mid-term evaluation of the European Neighbourhood Instrument 2014-2020 (ENI)
  1
  . The evaluation assesses whether the ENI is fit for purpose, based on its performance to-date, to deliver on its objectives of advancing further towards an area of shared prosperity and good neighbourliness by developing a special relationship founded on cooperation, peace and security, mutual accountability and a shared commitment to the universal values of democracy, the rule of law and respect for human rights
  2
  .

Its purpose is to inform future work on the instrument and its actions. In particular, this evaluation, which is part of a set of ten evaluations covering all the EU external financing instruments
  3
  , informs the Mid-Term Review Report
  4
  , which draws conclusions across these instruments.

This document is largely based on an external evaluation carried out by independent consultants, presented in Annex 5, complemented by internal assessments, other evaluations and a broad consultation process.

Scope of the evaluation

This evaluation covers the period from 1 January 2014 to 1 June 2017. However, due to the length of the implementation cycle of the ENI, the availability of data and results are limited. Therefore the evaluation also looks at the previous instrument for the Neighbourhood (ENPI, 2007-2013)
  5
  for some of the evaluation criteria (e.g. efficiency and effectiveness). When using the previous instrument as a source of data, it is important to note that the overall objective of the ENI has broadly remained the same as for the former instrument.

Consistent with the requirements of the Common Implementing Regulation, and in view to usefully feed into the Mid-Term Review Report, the evaluation is set at instrument level. As a consequence, it focuses, to the extent possible, on the elements contained in the ENI Regulation (e.g. on its principles, scope, flexibility and complementarity with other instruments) rather than on the projects that have been put in place on the basis of the instrument. However, some information on programming has also been included to show progress on how the instrument has been implemented (see section 4).

The countries covered by the evaluation are those eligible under the ENI Regulation (see annex 1 of the Regulation)
  6
  .

In accordance with the EU Better Regulation Agenda
  7
  and the Common Implementing Regulation, the following evaluation criteria are used: relevance; effectiveness, impact and sustainability; efficiency; EU added value; coherence, consistency, complementarity and synergies, and leverage.

2. Background of the initiative

The External Financing Instruments make up a major part of the Multiannual Financial Framework's
  8
  Heading IV "Global Europe"
  9
  which provides the EU with the tools necessary to reinforce its role on the world stage and to ensure that it is able to live up to its ambitions in promoting its interests and universal values and principles such as democracy, human rights, peace, solidarity, stability and poverty reduction and to help safeguard global public goods.

Adopted in early 2014, the External Financing Instruments were designed to facilitate and support policy implementation, with the intention of remaining relevant for the entire duration of the Multi-annual Financial Framework, thereby enabling the EU to implement external action policy as needed within the defined principles and objectives.

Description of ENI and its objectives

The European Neighbourhood Instrument (ENI) is the main financing instrument to support the implementation of the European Neighbourhood Policy (ENP). The European Neighbourhood Policy (ENP) was conceived in 2003 to develop closer relations between the enlarged EU and its neighbouring countries. The ENP was substantially reviewed in 2011, notably in the context of the EU's response to the events of the Arab Spring. It was further reviewed in 2015 to take account of the significant political developments in the neighbourhood.

The ENI Regulation, consistent with Article 8 of the Treaty on European Union (TEU), aims to advance further towards an area of shared prosperity and good neighbourliness involving the Union and the Neighbourhood countries and territories by developing a special relationship founded on cooperation, peace and security, mutual accountability and a shared commitment to the universal values of democracy, the rule of law and respect for the human rights (see the Intervention Logic below). The EU provides its support to partner countries in the Neighbourhood region mainly through the ENI, with its initial envelop of over EUR 15.4 billion for 2014-2020, replenished on a number of occasions since 2014. The ENI replaced the European Neighbourhood and Partnership Instrument (ENPI), which covered the period 2007-2013. It covers the same countries with the exception of Russia, which is only eligible to multi-country and Cross-Border Cooperation programmes under the ENI with a particular focus on people to people contacts and with the principle of co-financing by EU and Russia.

Union support under the ENI Regulation is used for the benefit of partner countries and the areas involved in cross-border cooperation, as well as for the common benefit of the Union and partner countries. It is programmed through: bilateral, multi-country and Cross-Border Cooperation programmes.

The main objectives for assistance under the ENI are the promotion of human rights, good governance and the rule of law and the strengthening of civil society, market access and free trade areas, the management of migration, sustainable and inclusive development, environment and climate change, security and prevention and management of conflicts as well as regional and cross-border cooperation.

The reconstructed ENI Intervention Logic (IL) was designed based on the ENI Regulation and the evolution of the ENI strategic documents, notably the revised ENP of November 2015.

The ENI is not the only instrument in place to implement the ENP. In addition to other contributing instruments (such us the DCI, EIDHR and IcSP) and EU's external crisis response tools (such us Macro-Financial Assistance and Humanitarian assistance) the EU also conducts policy dialogue promotes participation of partner countries in EU programmes and sectoral policies (agreements on trade, aviation, Research and Development, SMEs, migration and mobility etc.). These activities contribute to the achievements of ENI objectives, but are not part of this evaluation. Their contributions to ENI objectives cannot be disentangled form contributions of ENI programmes.

Baseline

As this is a mid-term evaluation, the baseline has been set at January 2014 when the ENI 2014-2020 entered into force. Therefore the evaluation compares, to the extent possible, the current situation with that of January 2014. For some evaluation criteria, where data is unavailable for this reference date, earlier baselines have been used, as described in the document (see Evaluation questions on effectiveness and efficiency), considering that the overall objectives of ENI are in line with the ones of the ENPI.

The ENI Regulation doesn't include indicators at instrument level. Indicators are defined for each country at sector level in multi-annual programming documents and specific programmes. Indicators are linked to specific country situations and reform objectives and progress can only be assessed in these sectors at country level. Therefore aggregation at instrument level is often not possible, in particular in sectors such as public administration reforms, judiciary reform or public finance management reforms. This means that progress and impact cannot be assessed at the level of the instrument; assessment of progress is made at country or regional level for the different sectors.

The ENI has been implemented in all countries, despite the difficult and often worsening political, social, and economic situation in many of them in 2014, because non-engagement was not an option for the EU
  10
  , given the close relationship with these countries and the immediate impact on the EU itself of major crises at its borders. Supporting the sometimes difficult political and economic transition processes in these countries was an imperative for the EU.

Supporting partner countries which have embarked on ambitious agreements with the EU, the most advanced ones being the Association Agreements including Deep and Comprehensive Free Trade Areas (AA/DCFTA) with Moldova, Georgia and Ukraine is one of the key objectives of the ENI. To that end capacity development is a major component of reforms programmes under the ENI.

Figure 1 – Reconstructed Intervention Logic

Positive incentives and reward for progress in democratic reforms were introduced in the aftermath of the Arab Spring ("more for more" approach). This has been formally translated in the ENI Regulation with the introduction of the incentive-based approach (with the so-called "umbrella programmes"), with financial incentives decided on an annual basis for partner countries committed to achieve progress towards deep and sustainable democracy and implementation of agreed reform objectives contributing to the attainment of that goal. Progress is assessed on the basis of the country annual progress reports and other relevant documents. The Regulation also provides for a higher degree of differentiation among partners, including in the definition of country allocations.

The objectives of the instrument have also been streamlined with a focus on policy priorities and a more limited number of specific objectives. A better link with relevant internal polices and instruments was promoted, in particular with the extension of the possibility for countries to participate in some EU programmes and in the work of EU agencies.

In line with the EU commitments in Busan to increase the coherence, complementarity and effectiveness of EU action abroad, the ENI Regulation encourages moves towards joint programming.

3. Method

This evaluation is supported by an external evaluation carried out from August 2016 to May 2017. The external evaluation of the ENI was managed by the Inter-Service Group through the following steps: an inception report (which explained how the evaluation design would deliver the information required); a desk report (providing initial responses to evaluation questions); visits to Egypt, Georgia, Lebanon, Tunisia and Ukraine to meet key interlocutors to obtain first-hand view in-country; a survey to EU delegations covering all instruments; an Open Public Consultation
  11
  on the draft report which comprised of a 12 week online survey and targeted meetings with Member States in March 2017; and a final report.

Overall, the quality of the collected evidence (data, documentation, interviews and survey results) for this evaluation can be assessed as good, within the limitations mentioned below. Beside the limited amount of information available on implementation, there was a limited feedback from the Open Public Consultation and the Commission services had to provide substantial comments and revisions to the text provided by the external evaluators in order to address some shortcomings, notably related to the understanding of the policy framework and articulation of the different financing instruments. This Staff Working Document largely concurs with the findings and conclusions of the external evaluation.

Organisation

As part of a wider set of evaluations covering the eight External Financing Instruments, the European Development Fund (EDF) and the Common Implementing Regulation (CIR), all the evaluations were carried out in an interlinked and co-ordinated manner. To ensure coherence of the different evaluations, relevant Commission services have worked closely together from the beginning of the process in June 2015. The external contractor delivered its final report at the end of June 2017. All reports prepared by the external contractor were discussed, reviewed and approved by the dedicated Inter Service Group set up by the Commission for the ENI evaluation.

The draft final report prepared by the external contractor was placed, with the other instruments draft reports, on the web for the open public consultation (OPC) from 7 February to 3 May 2017. Forty four contributions were received. During that period targeted face to face meetings were organised with representatives of civil society organisations (CSOs), EU Member States and the European Parliament
  12
  .

Evaluation Design

The intervention logic (IL) forms the backbone of the analytical framework. It is visualised in an IL diagram, which brings together the most relevant elements (from inputs to impacts, but also assumptions) in a single framework. To take into account the revised ENP policy of 2015 that put a focus on the stabilisation and the ability of responding to emergencies in the neighbourhood, a reconstructed IL was prepared by the external consultants
  13
  .

Covering the different levels of the IL, the evaluation questions (EQ) structure the analysis to gather evidence. Each question was structured into judgement criteria (JC) and indicators required to provide an answer based on a synthesis of evidences.

Methodology and data

The EU evaluation criteria (relevance, effectiveness, efficiency, coherence, and added value, complemented by impact, sustainability, consistency, complementarity and synergies and leverage) were applied as an underlying basis. The evaluation questions gave rise to a number of judgement criteria and associated indicators. The external evaluation used a mixed methods approach blending quantitative and qualitative methods, with a focus on the latter. The main analytical tools consisted of rigorous assessment of documentation and consultation of stakeholders (semi-structured interviews, group consultations), and the Open Public Consultation; key issues were also addressed by a survey of beneficiaries and stakeholders organised by the Commission.

For all EQs, data collection included a mixture of desk review of documents, semi-structured interviews in Brussels by phone or face-to-face, as well as four field missions that took place to Egypt, Georgia, Tunisia, and Ukraine.

Challenges and limitations

The process of this evaluation is robust and the evidence reasonably solid. However, external evaluators were faced with four main challenges:

- the implementation of ENI was still in its early stages when the collection of evidence took place. Therefore, effectiveness/sustainability/impact criteria could not be assessed based on outputs and outcomes at this stage;

- the timeframe in which the evaluation took place was tight given the requirements imposed by the Common Implementing Regulation. This resulted in difficulties related to access to data, documentation and availability of key respondents;

- the ENI impact assessment had not provided an estimation of the expected impact in quantitative terms
  14
  ;

- The ENPI is still under implementation, and there is no ex-post evaluation yet at this stage and it was not possible to build on that.

The multiple rounds of commenting also mean that facts were verified and this facilitated the cross verification of data from multiple sources. The evaluators were provided access to all relevant information and colleagues within the Commission services and in EU Delegations. The Commission services also ensured that the evaluators would have access to representatives of the beneficiaries.

As indicated by the Intervention Logic, the ENI instrument can only be seen as a contributing factor towards any results achieved, especially in the context of the geo-political changes in the Neighbourhood regions. Many factors, both internal and external, affect developments in the beneficiaries. The reform agenda for stabilising the neighbourhood, economic integration of partners in the EU market and political association is highly demanding and requires a strong political commitment from partners. Ultimately, it is for the countries to adopt and implement the necessary reforms and policies for ensuring that they can reap the benefits of economic integration and political association.

4. Implementation state of play

Specific objectives and priorities and indicative financial allocations for EU support are set in multi-annual programming documents, whose duration is adapted to the situation prevailing in each country or region (from two to four years) and which are developed through a wide consultation process, involving the authorities, civil society organisations and all relevant stakeholders and the Member States. These priorities at country or regional level are then translated into annual action programmes detailing the different actions.

The vast majority of funding is channelled through bilateral programmes, tailored to the needs of each partner country, in line with the principles of differentiation and ownership of the Neighbourhood policy. In addition, the ENI also supports multi-country and Cross Border Cooperation programmes (15 ENI-CBC programmes adopted for the period 2014-2020, totalling around EUR 1 billion). The ENI also contributes to the Erasmus+ programme with a planned amount of more than EUR 700 million for 2014-2020.

In response to the recent developments in the Neighbourhood and in order to implement the Global Strategy for the EU's Foreign and Security Policy of 2016 and the ENP Review of 2015, some multi-annual programming documents were revised by anticipation in 2017 instead of 2018.

Joint programming is developing in the neighbourhood and a full-fledged joint programming document will be adopted in the course of 2017 for Palestine, covering the period 2017-2020, which will replace a Single Support Framework.

Under the ENI, the EU continued to increase its support to civil society, through mainstreaming but also through specific complementary allocations to support the development of their capacities, helping them become more professional and reliable partners in the policy-making and reform processes.

Cooperation under the ENI is implemented through a range of modalities. Support to large reforms is often implemented through sector budget support programmes, when conditions in the country allow, or sector programmes. Technical assistance is provided to accompany reform programmes and support partner countries in the implementation of Association Agreements, including by sharing expertise from Member States through TAIEX
  15
  , Twinnings
  16
  and the SIGMA
  17
  programme.

Under the ENI, the blending approach has been further developed with the Neighbourhood Investment Facility (NIF) which leverages loans from European Financial Institutions to finance investments in sustainable energy, transport infrastructure, environment and private sector development, targeting small and medium-sized enterprises.

The ENI incentive-based mechanism ("Umbrella Programme") is providing, on an annual basis, additional funding to countries demonstrating progress in building deep and sustainable democracy for an amount of around 10% of the annual ENI budget (average amount of EUR 200 million a year). The main beneficiaries from 2014 to 2016 were Tunisia, Ukraine and Georgia (see table below).

Since the adoption of the ENI, the EU has been faced with the need to respond to the multiple crises and unforeseen events which unfold in its neighbourhood (see the following table, pointing out the main negative events affecting several countries, with related impact.

Table 1 - Major events having negatively affected some ENI beneficiary countries and related impact

| Country | Major events affecting the country | Impact |
| --- | --- | --- |
| Neighbourhood East | | |
| Armenia | In 2015 Armenia officially joins the Russian-led Eurasian Customs Union, having decided against signing an EU Association Agreement. | Reduced willingness to comply with the EU agenda |
| Ukraine (59% of the population and 51% of the total GDP of Neighbourhood East before the crisis) | Maidan event in 2014


  Annexation of Crimea by Russia


  Conflict in eastern Ukraine | Political instability (Indicator on Political stability and absence of violence (WB) from 45.5 in 2010 to 5.71 in 2014)


  Losses of human lives (about 10,000 reported killed since the start of the crisis)


  Substantial interference of Russia in national politics and security


  Substantial losses in economic terms (growth rate from 5.5% in 2010 to -6.6% in 2104 and -9.8% in 2015)


  Macroeconomic imbalances (Fiscal balance -4.14% of GDP in 2014; public debt 70% in 2014, as opposed to a 40% of GDP the year before) |
| Neighbourhood South | | |
| Egypt (38% of the population and 17% of the total GDP of Neighbourhood South in the last year for which statistics are available for all countries - 2006) | Revolution (2011) and political unrest (2013). Terrorism | Deteriorated macroeconomic environment and macroeconomic imbalances (fiscal balance -12.9% in 2014 of GDP; public debt around 90%)


  Worsening conditions of Rule of law (RoL indicator (WB) down from 49.76 in 2010 to 31.25 in 2015)


  Increased insecurity (Indicators on Political Stability and absence of violence (WB) from 19.43 in 2010 to 8.57 in 2015)


  No budget support payments since 2012. |
| Jordan (3.3% of the population and 2.5% of the total GDP of Neighbourhood South in the last year for which statistics are available for all countries - 2006) | Civil war in Syria


  Terrorism | Major inflow of refugees from Syria (at least 660,000)


  Macroeconomic imbalances (Fiscal balance at least -11% of GDP during the last years; public debt around 90% of GDP) |
| Lebanon (4% of the population and 3.6% of the total GDP of Neighbourhood South in the last year for which statistics are available for all countries - 2006) | Civil war in Syria


  Terrorism | Major inflow of refugees from Syria (about 1.1 million)


  Macroeconomic imbalances (fiscal balance at – 7.5% of GDP in 2015; public debt around 140% of GDP) |
| Libya (3% of the population and 9% of the total GDP of Neighbourhood South in the last year for which statistics are available for all countries - 2006) | Fall of the Kaddafi regime in 2011 following the Arab Spring


  Terrorism | Deep political instability and deteriorated security (Indicator on Political stability and absence of violence (WB) from 47.39 in 2010 to 3.35 in 2015


  Collapsing economy (IMF estimates a growth rate of -53% of DGP in 2014; a fiscal balance of – 73% of GDP in the same year) |
| Syria (10% of the population and 5.3% of the total GDP of Neighbourhood South in the last year for which statistics are available for all countries - 2006) | Civil war following the Arab Spring


  Creation of ISIL in 2014


  Military interventions of NATO and Russia | Major political disruption (Indicator on Political stability and absence of violence (WB) from 22.27 in 2010 to 0 in the last years)


  Major macroeconomic crisis (no data available)


  Losses of human lives (estimates vary, but go up to a level of 475,000)


  More than 5 millions of people escaping


  About 6.5 million internally displaced |
| Tunisia (5.3% of the population and 5.6% of the total GDP of Neighbourhood South in the last year for which statistics are available for all countries - 2006) | Revolution and political instability following the Arab Spring. Three major terrorist attacks in 2015. | Macroeconomic imbalances (Fiscal balance -5.2% of GDP in 2015; public debt above 57% in the same year)


  Negative impact, in terms of security and attractiveness for investments and tourism due the deteriorated situation in Libya (Indicator on Political stability and absence of violence from 43.60 in 2010 to 18.47 in 2014, and still deteriorating)


  Social unrest |

Country

Major events affecting the country

Impact

Neighbourhood East

Armenia

In 2015 Armenia officially joins the Russian-led Eurasian Customs Union, having decided against signing an EU Association Agreement.

Reduced willingness to comply with the EU agenda

Ukraine (59% of the population and 51% of the total GDP of Neighbourhood East before the crisis)

Maidan event in 2014

Annexation of Crimea by Russia

Conflict in eastern Ukraine

Political instability (Indicator on Political stability and absence of violence (WB) from 45.5 in 2010 to 5.71 in 2014)

Losses of human lives (about 10,000 reported killed since the start of the crisis)

Substantial interference of Russia in national politics and security

Substantial losses in economic terms (growth rate from 5.5% in 2010 to -6.6% in 2104 and -9.8% in 2015)

Macroeconomic imbalances (Fiscal balance -4.14% of GDP in 2014; public debt 70% in 2014, as opposed to a 40% of GDP the year before)

Neighbourhood South

Egypt (38% of the population and 17% of the total GDP of Neighbourhood South in the last year for which statistics are available for all countries - 2006)

Revolution (2011) and political unrest (2013). Terrorism

Deteriorated macroeconomic environment and macroeconomic imbalances (fiscal balance -12.9% in 2014 of GDP; public debt around 90%)

Worsening conditions of Rule of law (RoL indicator (WB) down from 49.76 in 2010 to 31.25 in 2015)

Increased insecurity (Indicators on Political Stability and absence of violence (WB) from 19.43 in 2010 to 8.57 in 2015)

No budget support payments since 2012.

Jordan (3.3% of the population and 2.5% of the total GDP of Neighbourhood South in the last year for which statistics are available for all countries - 2006)

Civil war in Syria

Terrorism

Major inflow of refugees from Syria (at least 660,000)

Macroeconomic imbalances (Fiscal balance at least -11% of GDP during the last years; public debt around 90% of GDP)

Lebanon (4% of the population and 3.6% of the total GDP of Neighbourhood South in the last year for which statistics are available for all countries - 2006)

Civil war in Syria

Terrorism

Major inflow of refugees from Syria (about 1.1 million)

Macroeconomic imbalances (fiscal balance at – 7.5% of GDP in 2015; public debt around 140% of GDP)

Libya (3% of the population and 9% of the total GDP of Neighbourhood South in the last year for which statistics are available for all countries - 2006)

Fall of the Kaddafi regime in 2011 following the Arab Spring

Terrorism

Deep political instability and deteriorated security (Indicator on Political stability and absence of violence (WB) from 47.39 in 2010 to 3.35 in 2015

Collapsing economy (IMF estimates a growth rate of -53% of DGP in 2014; a fiscal balance of – 73% of GDP in the same year)

Syria (10% of the population and 5.3% of the total GDP of Neighbourhood South in the last year for which statistics are available for all countries - 2006)

Civil war following the Arab Spring

Creation of ISIL in 2014

Military interventions of NATO and Russia

Major political disruption (Indicator on Political stability and absence of violence (WB) from 22.27 in 2010 to 0 in the last years)

Major macroeconomic crisis (no data available)

Losses of human lives (estimates vary, but go up to a level of 475,000)

More than 5 millions of people escaping

About 6.5 million internally displaced

Tunisia (5.3% of the population and 5.6% of the total GDP of Neighbourhood South in the last year for which statistics are available for all countries - 2006)

Revolution and political instability following the Arab Spring. Three major terrorist attacks in 2015.

Macroeconomic imbalances (Fiscal balance -5.2% of GDP in 2015; public debt above 57% in the same year)

Negative impact, in terms of security and attractiveness for investments and tourism due the deteriorated situation in Libya (Indicator on Political stability and absence of violence from 43.60 in 2010 to 18.47 in 2014, and still deteriorating)

Social unrest

The flexibility of the instrument has been stretched to the limit. The initial budged proved to be insufficient, which was partially addressed through transfers from other external instruments. Faced with the illegal annexation of Crimean peninsula by the Russian Federation and the conflict in the eastern Ukraine when the negotiation of the ENI Regulation had just been finalised, the Commission mobilised all the possibilities of the external cooperation regulations tool box and adopted in April 2014 a major State Building Contract of EUR 355 million to support the political and economic stabilisation of Ukraine, the largest bilateral programme ever for Ukraine. This was also the first large programme adopted under the new ENI Regulation, showing its flexibility to respond to major crises. It was accompanied by a EUR 10 million programme in favour of civil society to help civil society organisations (CSOs) monitor national reforms.

The same year, the scale of the Syrian crisis and its enormous impact on neighbouring countries led the Commission to establish EU Regional Trust Fund in Response to the Syrian Crisis (the "Madad" fund)
  18
  . This Trust Fund brings a more coherent and integrated EU response to the crisis and primarily addresses longer term economic, educational and social needs of Syrian refugees in neighbouring countries such as Jordan, Lebanon, Turkey and Iraq, as well as helping overstretched host communities and their administrations. As of end of June 2017, contributions from the EU budget to the Madad fund amounted to EUR 956.20 million, of which EUR 523.5 million from ENI for a total amount of around EUR 1300 million. The establishment of the Madad fund aimed at responding to the challenges around the Syria crisis in a more integrated and coherent way and leveraging additional funds from the Member States, alleviating EU budget constraints.

The ENI also contributed to the EU emergency trust fund for Africa
  19
   for stability and addressing root causes of irregular migration and displaced persons in Africa for a total of EUR 194 million since 2015, in order to address the root causes of migration and the different aspects of migration management in the five beneficiary countries of the North Africa window (Algeria, Egypt, Libya, Morocco and Tunisia).

A number of special measures has also been used to ensure quicker reaction to evolving needs.

Since 2014, the initial ENI budget set at EUR 15.4 billion in 2014 has been substantially increased (by EUR 1.1 billion) through reinforcements from Heading IV margin, redeployments in Heading IV and reinforcement from Contingency Margin in support of UNRWA and in response to the consequences of Syrian Crisis.

In total, until June 2017, EUR 7.0 billion have been committed, and EUR 5.3 billion disbursed. Out of the first amount, EUR 4.4 billion were committed to the partners in the South (63 %), and EUR 2.0 billion to the East (28 %), with EUR 3.3 billion and EUR 1.5 billion disbursed so far.

The implementation of ENI assistance is continuously monitored, in particular through Results-Oriented Monitoring (ROM) of projects and programmes
  20
  , internal monitoring implemented by EU Delegation staff and evaluations at policy, thematic, as well as at country or regional level
  21
  .

No systemic monitoring of ENI at instrument level is taking place, also in the absence of a specific monitoring system set out at instrument level. Results reporting has so far taken place in the framework of the end of project results reporting exercise, put in place by DEVCO based on the EU Results Framework
  22
  , but focused so far only on projects related to the previous Multi-annual Financing Framework (MFF).

| Table 2 -ENI 2014-2017commitments(as of July 2017) | | | | | ENI + ENPI 2014-2016 payments | | |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Country/


  Programme | 2014 | of which


  umbrella | 2015 | of which


  umbrella | 2016 | of which


  umbrella | 2017


  indicative | | Country/


  Programme | 2014 | 2015 | 2016 |
| South | | | | | |
| Algeria | 26,3 | | 25,0 | | 25,0 | | 30,0 | | Algeria | 33,0 | 33,2 | 42,2 |
| Egypt | 115,0 | | 105,0 | | 105,0 | | 100,0 | | Egypt | 91,1 | 64,6 | 93,2 |
| Israel | - | | 2,0 | | 1,8 | | 1,8 | | Israel | 4,4 | 6,6 | 4,2 |
| Jordan | 174,5 | 15,0 | 100,0 | | 80,0 | | 88,0 | | Jordan | 56,3 | 101,3 | 148,4 |
| Lebanon | 143,3 | 15,0 | 40,0 | | 40,0 | | 44,0 | | Lebanon | 78,2 | 100,3 | 66,9 |
| Libya | 8,0 | | 3,0 | | 10,0 | | 12,5 | | Libya | 11,0 | 7,7 | 6,2 |
| Morocco | 218,0 | 20,0 | 210,0 | 30,0 | 190,0 | | 189,5 | | Morocco | 76,2 | 134,5 | 224,0 |
| Palestine | 307,5 | | 325,3 | | 290,0 | | 310,0 | | Palestine | 303,2 | 317,5 | 330,6 |
| Syrian crisis * | 61,3 | | 409,0 | | 170,0 | | 201,0 | | Syrian crisis*** | 2,0 | 35,1 | 78,6 |
| Tunisia | 169,0 | 50,0 | 186,8 | 71,8 | 213,5 | 80,5 | 300,0 | | Tunisia | 178,5 | 142,0 | 108,2 |
| Reg. & other


  multi-country | 229,1 | | 196,4 | | 126,0 | | 181,0 | | Reg. & other


  multi-country | 92,8 | 118,9 | 173,9 |
| Africa TF ** | | | | | 55,0 | | 139,0 | | | | | 30 |
| Total ENI South | 1.452,0 | 100 | 1.602,5 | 101,8 | 1.306,3 | 80,5 | 1.596,8 | | Total ENI South | 926,8 | 1.061,8 | 1.306,4 |
| EAST | | | | |
| Armenia | 34,0 | | 30,0 | | 28,0 | | 28,0 | | Armenia | 23,2 | 36,7 | 49,2 |
| Azerbaijan* | 21,0 | | 14,5 | | 13,5 | | 19,1 | | Azerbaijan | 6,8 | 22,1 | 21,5 |
| Belarus | 19,0 | | 14,5 | | 29,0 | | 29,0 | | Belarus | 22,3 | 17,9 | 23,7 |
| Georgia | 131,0 | 30,0 | 100,0 | 10,0 | 110,0 | 21,0 | 110,0 | | Georgia | 41,1 | 71,9 | 96,4 |
| Moldova | 131,0 | 30,0 | 90,0 | | 89,0 | | 89,0 | | Moldova | 93,7 | 42,2 | 82,8 |
| Ukraine | 242,0 | 40,0 | 200,0 | 90,0 | 200,0 | 92,0 | 200,0 | | Ukraine | 314,1 | 80,0 | 206,2 |
| Reg. & other


  multi-country | 152,4 | | 150,1 | | 144,5 | | 144,5 | | Reg. & other


  multi-country | 47,0 | 88,8 | 111,0 |
| Total ENI East | 730,4 | 100 | 599,1 | 100 | 614,0 | 113 | 619,6 | | Total ENI East | 548,1 | 359,6 | 590,8 |
| TOTAL ENI | 2182.4 | 200 | 2201,6 | 201,8 | 1920,3 | 193,5 | 2189,4 | | | 1.623,2 | 1.573,2 | 2078,00 |

Table 2 -ENI 2014-2017commitments(as of July 2017)

ENI + ENPI 2014-2016 payments

Country/

Programme

2014

of which

umbrella

2015

of which

umbrella

2016

of which

umbrella

2017

indicative

Country/

Programme

2014

2015

2016

South

Algeria

26,3

25,0

25,0

30,0

Algeria

33,0

33,2

42,2

Egypt

115,0

105,0

105,0

100,0

Egypt

91,1

64,6

93,2

Israel

-

2,0

1,8

1,8

Israel

4,4

6,6

4,2

Jordan

174,5

15,0

100,0

80,0

88,0

Jordan

56,3

101,3

148,4

Lebanon

143,3

15,0

40,0

40,0

44,0

Lebanon

78,2

100,3

66,9

Libya

8,0

3,0

10,0

12,5

Libya

11,0

7,7

6,2

Morocco

218,0

20,0

210,0

30,0

190,0

189,5

Morocco

76,2

134,5

224,0

Palestine

307,5

325,3

290,0

310,0

Palestine

303,2

317,5

330,6

Syrian crisis *

61,3

409,0

170,0

201,0

Syrian crisis***

2,0

35,1

78,6

Tunisia

169,0

50,0

186,8

71,8

213,5

80,5

300,0

Tunisia

178,5

142,0

108,2

Reg. & other

multi-country

229,1

196,4

126,0

181,0

Reg. & other

multi-country

92,8

118,9

173,9

Africa TF **

55,0

139,0

30

Total ENI South

1.452,0

100

1.602,5

101,8

1.306,3

80,5

1.596,8

Total ENI South

926,8

1.061,8

1.306,4

EAST

Armenia

34,0

30,0

28,0

28,0

Armenia

23,2

36,7

49,2

Azerbaijan*

21,0

14,5

13,5

19,1

Azerbaijan

6,8

22,1

21,5

Belarus

19,0

14,5

29,0

29,0

Belarus

22,3

17,9

23,7

Georgia

131,0

30,0

100,0

10,0

110,0

21,0

110,0

Georgia

41,1

71,9

96,4

Moldova

131,0

30,0

90,0

89,0

89,0

Moldova

93,7

42,2

82,8

Ukraine

242,0

40,0

200,0

90,0

200,0

92,0

200,0

Ukraine

314,1

80,0

206,2

Reg. & other

multi-country

152,4

150,1

144,5

144,5

Reg. & other

multi-country

47,0

88,8

111,0

Total ENI East

730,4

100

599,1

100

614,0

113

619,6

Total ENI East

548,1

359,6

590,8

TOTAL ENI

2182.4

200

2201,6

201,8

1920,3

193,5

2189,4

1.623,2

1.573,2

2078,00

- including ENI contribution to the EU Madad fund ** ENI contribution to the EU Africa TF ***excluding TF payments

5. Answers to the evaluation questions

Evaluation Question 1: Relevance

To what extent do the overall and the specific objectives and the design
  23
  of the ENI respond to: (i) EU priorities and beneficiary needs identified at the time the instrument was adopted (2014)?; (ii) Current EU priorities, in particular emerging from the 2015 ENP Review such as stabilisation, and beneficiary needs, given the evolving challenges and priorities in the international context (2017)?

Overall, the ENI instrument appeared fit for purpose at the moment it was set out, properly addressing the most significant needs identified, in line with the policy framework set out by the revised European Neighbourhood Policy (ENP) and as such it ensured credibility to the EU's intention to deepen political cooperation and pursue gradual economic integration with its neighbours.

The relevance of the ENI instrument remains rooted in the existence of an EU policy for its neighbourhood. This is reflected in the Treaty (Article 8: "The Union shall develop a special relationship with neighbouring countries") which calls for a special relation with the neighbours, and in the strong political will of EU member States to have a dedicated policy for the European Neighbourhood and related instruments. The Council conclusions on the ENP of 2015 are quite clear in that respect ("Underlining the importance of a special relationship with the EU's neighbours, the stabilisation of the neighbourhood in political, economic and security terms will be the main political priority for the EU in the next years").

During the public consultation launched in 2015 on the occasion of the review of the ENP, many stakeholders reaffirmed that the EU must have a special and more effective relationship with its neighbours with dedicated tools
  24
  . Most ENP partners with only a few exceptions – and not taking into account those countries that are at war like Syria – want to build special relations with the European Union. On its part, the EU has very strong interest to engage in special relations with neighbours also in order to promote and defend EU values and common interests. This is key for building resilience in the neighbourhood. The ENI is only part of a wider set of tools to ensure that the EU delivers on the commitment of Article 8 of the Treaty.

The EU priorities with regard to the European Neighbourhood Instrument (ENI) stem from its policy framework, as defined in Article 3(1), and are not set out in a single specific document but rather in a set of bilateral agreements, Communications, conclusions and declarations of EU institutions (European Parliament and Council).

Though the strategic framework is complex and defined by a plurality of documents (as pointed out by the external evaluators, who consider that this complex set somewhat hinders (its) clear understanding and visibility
  25
  ), the Commission services consider that the strategic framework for the implementation of the ENI is clearly set out in the recently adopted Global Strategy for the EU's Foreign and Security Policy
  26
  (and its predecessor, the 2003 EU Security Strategy
  27
  ) and the 2015 Joint Communication on the review of the European Neighbourhood Policy
  28
  . However, what is somewhat missing is a clear description about how the EU intends to mobilise its various instruments in complement of the ENI. Though the articulation between geographic and thematic instruments makes sense in principle, the lack of clear, operational document explaining the articulation of all EU instruments makes it complicated for beneficiaries and external stakeholders to easily grasp it. That being said, the ENI is the main financial instrument to support partners in the region with grants, and the existence of other much smaller instruments provides a broader range of available tools, allows for greater flexibility and is thus serving the principle of differentiation, as also acknowledged by the external evaluators
  29
  , though several interventions for human rights actions, for security, for the promotion of EU interests were also funded from ENI, from which the large majority of financial resources was coming
  30
  .

The objectives (Article 1) and thematic priorities (Article 2) of the ENI Regulation are congruent with the six core pillars of the ENP as laid out in the 2011 Joint Communication on the ENP “A new response to a Changing Neighbourhood”
  31
  . The latter is the main policy document that framed the preparation of the ENI Regulation. Even after the adoption of the ENP Review in November 2015
  32
  , the broad and comprehensive nature of priorities listed in the ENI Regulation, as well as the framework for programming and implementation, has allowed the EU to respond to new priorities. As a result, the Regulation ensures the responsiveness of the ENI to the challenges of the Neighbourhood in line with ENP priorities, which has been a proof of flexibility.

The EU has responded to considerable challenges in the neighbourhood since 2014, in particular to the consequences of the Syrian Crisis, the illegal annexation of the Crimean peninsula by Russia and the conflict in eastern Ukraine, civil war in Libya, the emergence of the Islamic State, and high youth unemployment in most partner countries fuelling resentment, disenfranchisement and pressures to migrate towards Europe. The response capacity, budget and flexibility of the ENI instrument has been stretched to its limits. In fact, external evaluators
  33
  and a few stakeholders
  34
  have considered that the response brought under the ENI has not been proportionate to the needs and challenges.

Commission services agree that facing the challenges in the neighbourhood now and in the future (e.g. reconstruction of war-torn regions in Ukraine and Syria, state building in Libya, tackling rising inequalities in most countries, pressing demographics in Egypt and Palestine, boosting sustainable and inclusive growth in the partner countries) requires increased EU political and financial investment as well as leveraging additional funding, including by further enhancing cooperation with major Financial Institutions and participation of the private sector though blending facility and guarantees.

Despite the numerous efforts to adapt to circumstances, the ENI instrument was not specifically conceived to contribute to crisis prevention, thus enhancing broad EU interests (as pointed out by external evaluators). This would require that strategy and programming documents, which do not usually include a risk assessment analysis that goes beyond projects or sectors
  35
  , include geostrategic analyses (both conflict analysis and early warning) aiming at identifying and anticipating possible discontinuities, social/political changes and risk of conflict, better assessing the potential for violent and radical changes in the neighbourhood countries and ensuring early and coherent EU action. Currently, these documents which are shared with the partner governments broadly reflect the political status quo.

In countries where budget support is deployed, a detailed country risk assessment is compiled and updated every year based on a list of pre-defined criteria which allow comparison to be made over the years for a particular country and also among different countries. The Commission services acknowledge that interventions on the ground shall be based on good anticipation of possible developments. Nevertheless, some of the changes in the neighbourhood in 2011-2015 were radical and violent, and challenging for all development actors present on the ground. In such difficult circumstances the implementation of the instrument was flexible and the Commission adjusted the financial allocations in accordance with the scope of the challenges
  36
  .

Nevertheless, the Commission services acknowledge that linkages between the political and geopolitical risk analysis already conducted within the EU institutions (notably in the EEAS) and the services in charge of programming financial assistance under ENI were not adequate. An integrated, comprehensive approach was not in place and human rights and gender are crucial in this context as well. Moreover the analysis of agents and factors of changes in programming documents was not adequately developed. The Commission services do not consider that there is a need however to set a specific component on crisis prevention under the ENI, nor to create dedicated structures to crisis-stricken countries. Many of the existing interventions by ENI and other instruments (in particular the IcSP) are already contributing to crisis prevention (e.g. providing budget support and macro financial assistance to Tunisia in 2014-15 helped prevent macroeconomic crisis). Rather, flexibility, though present, was probably not sufficient, and the necessary synergies with the instrument contributing to peace and stability which works on crisis prevention were not sufficiently developed.

The ENI Regulation sets out solid foundations for enhanced alignment with EU and partners’ needs and priorities, which was emphasised by the ENP Review in 2015. In comparison to ENPI, the ENI strengthened the principle of differentiation and the focus on country ownership and civil society
  37
  . As evidenced by face to face consultations, partners value this principle
  38
  . With the exception of the magnitude, the ENI interventions look in general well in line with on-going challenges, emerging needs and limited capacities/political will for reform in most of the ENI partner countries. Policy dialogue is an inherent feature of the ENI framework that is implemented, thus ensuring that strategy and programming documents as well as ENI programmes are aligned to the priorities of partner countries’ governments and their citizens' needs (especially through the inclusion of civil society representatives).

Evaluation Question 2: Effectiveness, impact, sustainability

To what extent does the ENI deliver results against the instrument's objectives, and specific EU priorities?

It is still too early to assess with absolute certainty the effectiveness, impact and sustainability of the instrument given the short period of implementation
  39
  . Only some strategic, high level indicators
  40
  can give an initial evidence of whether or not the countries are moving in the expected direction, knowing that the EU can only indirectly influence their evolution. However, considering the continuity of interventions from the previous instrument
  41
  , and despite the negative impact of exogenous factors (Arab Spring, wars, refugee crisis, economic slowdown in Europe), significant reforms were accompanied and progress was registered in those countries that were willing to engage in a reform process and in those areas where there was ownership by the national authorities and the external conditions were conducive. For example, there has been a clear trend of improved business environment in some Neighbourhood East partners such as Georgia; Jordan has demonstrated economic resilience despite the huge strains from the Iraq war and refuges crisis, notably through budget support grants provided by the ENI and loans under the EU macro-financial assistance; important public administration reform programmes in Georgia, Morocco and Ukraine supported by the EU have allowed improved service delivery and public financial management
  42
  ; Tunisia has made progress in its democratic transition with hugely improved press freedom and voice and accountability. This can be shown by some international indicators in the table below (in green cases of significant improvement; in red cases of significant deterioration).

Table 3 – Evolution of selected indicators in some countries

| Indicators | Region/Country | 2010 | 2013 | 2016 |
| --- | --- | --- | --- | --- |
| Distance to frontier (WB) (range 1-100) |
| | N East | 59.73 | 67.19 | 72.10 |
| | Georgia | 76.61 | 82.09 | 80.20 |
| | Moldova | 59.21 | 32.26 | 72.75 |
| | Ukraine | 44.21 | 58.14 | 63.90 |
| | N South | 58.64 | 56.97 | 54.69 |
| | Morocco | 60.04 | 64.38 | 67.50 |
| | Jordan | 57.17 | 57.88 | 57.30 |
| World Competitiveness Index (WEF) (range 1-7) |
| | N East | 4 | 4.1 | 4.2 |
| | Georgia | 3.8 | 4.1 | 4.32 |
| | N South | 4.1 | 4.2 | 4.1 |
| | Morocco | 4.1 | 4.1 | 4.2 |
| Corruption perception index (Transparency International) (range 1-100) |
| | N East | 2.8* | 33.7 | 36.8 |
| | Georgia | 3.8* | 49 | 57 |
| | N South | 3.5* | 34.7 | 34.8 |
| Regulatory Quality (WB) (range 1-100) |
| | N East | 43.04 | 43.30 | 48.48 |
| | Georgia | 65.05 | 73.68 | 81.25 |
| | N South | 44.50 | 36.04 | 32.59 |
| Press Freedom (Reporters without Borders (range 100-1)$ |
| | N East | 39 | 38.7 | 38.5 |
| | N South | 50.2 | 37.4 | 46.1 |
| | Tunisia | 72.5 | 39.93 | 31.6 |
| Voice and Accountability (WB) (range 1-100) |
| | N East | 29.94 | 30.49 | 32.51 |
| | Georgia | 42.18 | 54.50 | 53.69 |
| | N South | 22.96 | 29.02 | 29.50 |
| | Tunisia | 9.95 | 44.08 | 56.65 |

Indicators

Region/Country

2010

2013

2016

Distance to frontier (WB) (range 1-100)

N East

59.73

67.19

72.10

Georgia

76.61

82.09

80.20

Moldova

59.21

32.26

72.75

Ukraine

44.21

58.14

63.90

N South

58.64

56.97

54.69

Morocco

60.04

64.38

67.50

Jordan

57.17

57.88

57.30

World Competitiveness Index (WEF) (range 1-7)

N East

4

4.1

4.2

Georgia

3.8

4.1

4.32

N South

4.1

4.2

4.1

Morocco

4.1

4.1

4.2

Corruption perception index (Transparency International) (range 1-100)

N East

2.8*

33.7

36.8

Georgia

3.8*

49

57

N South

3.5*

34.7

34.8

Regulatory Quality (WB) (range 1-100)

N East

43.04

43.30

48.48

Georgia

65.05

73.68

81.25

N South

44.50

36.04

32.59

Press Freedom (Reporters without Borders (range 100-1)$

N East

39

38.7

38.5

N South

50.2

37.4

46.1

Tunisia

72.5

39.93

31.6

Voice and Accountability (WB) (range 1-100)

N East

29.94

30.49

32.51

Georgia

42.18

54.50

53.69

N South

22.96

29.02

29.50

Tunisia

9.95

44.08

56.65

*: For this year the range was 1-10.

$: For this indicator a decrease in the score means an improvement.

Recent reports from the European Court of Auditors for the operations in Tunisia and Ukraine have demonstrated that EU actions under ENI and its predecessor ENPI were effective
  43
  . On the economic governance side, also the recently-concluded evaluation is rather positive regarding interventions in Georgia and Morocco notably on Public Financial Management
  44
  . The strategic evaluation on Azerbaijan, currently in its final stage, tends to conclude in very positive terms about the role played by EU assistance in enabling the country to approximate the EU acquis, especially thanks to the twinning and TAIEX instruments. This is also one of the conclusions of the strategic evaluation on Jordan
  45
  , which also notes that the EU, together with other actors, instigated, through its policy dialogue, significant reforms in the energy and social protection sector. Overall, many of the above-mentioned evaluations point to the distinct EU contribution on institutional strengthening. Also, Cross Border Cooperation programmes have had a positive impact on people to people contract and good neighbourly relations between EU regions and those of neighbourhood countries, although they have had less impact on stability and security at the border
  46
  .

However, in countries where there is no strong political commitment to comprehensive reforms or the willingness to have closer relations with the EU, the effectiveness of the ENPI and the ENI has been less strong.

Since 2014, progress of the Neighbourhood countries regarding respect of EU fundamental values has been uneven. In the field of human rights, democracy and governance, the Commission has targeted activities that are relevant and important, but the countries and regional contexts have made it difficult to pursue comprehensive support strategies and hence impeded the achievement of significant results. While the Results Oriented Monitoring (ROM) Annual report (2014) on the ENI
  47
  estimates that actions under the ENI and the ENPI have effectively delivered well targeted outputs of good technical quality in the field of rule of law and good governance (e.g. by strengthening civil society in various countries), the prospects for sustainability are limited in several countries. The unfavourable political environment (look at the previous table) in most Neighbourhood countries remains the main cause. Nevertheless, limited improvements and success stories exist in specific contexts or areas
  48
  . For example, Ukraine made significant steps forwards with reforms in the areas of anti-corruption, public procurement and public administration.

In the context of the EU efforts at promoting deep and sustainable democracy (one of the objective of the ENI), the incentive-based approach has been partly successful
  49
  . Indeed, first of all, it is already the case that those partners most keen to establish a close relation with the EU are offered more developed incentives (e.g. Association Agreement - AA/Deep and Comprehensive Free Trade Agreement - DCFTA) for three partners in the East and two in the South, access to EU programmes, visa facilitation or liberalisation agreements in place, etc. Second, the existing mechanism has been effective in supporting partners that had committed to reforms, most notably Ukraine, Tunisia, and Georgia. This has allowed for example to double the annual allocation to Tunisia at a critical moment to support the transition of the country and to substantially increase the support to Ukraine’s political and economic reforms after the illegal annexation of the Crimean peninsula and the conflict in the eastern part of the country (in line with the expectations of the impact assessment
  50
  on the greater need for flexibility). This support has allowed effective macroeconomic stabilisation in these partners and/or avoiding economic recessions
  51
  . For example, after the large recession in 2014-2015 in Ukraine, growth rebounded to more than 2% in 2016 and international reserves doubled thanks in particular to the large EU and international support package (including ENI grants and loans from the macro-financial assistance instrument)
  52
  . Thirdly, the existing provisions of the incentives-based approach (the umbrella programmes and the ranges mechanism) ensure that EU interests and values are taken into consideration in the allocation mechanism. However, it should be noted that the incentive in form of additional umbrella funds has limited impact on reform processes in partner countries, and that providing indicative allocations in form of ranges is perceived as misleading by some partner countries. As mentioned in the external evaluation report, "the mechanism has allowed extending the scope of reforms already agreed but hardly incentivised partner countries for more or deeper reforms"
  53
  .

The results of ENI sector programmes and associated policy dialogue considerably differ between countries and within countries. Good preparation, political and sectoral commitment to reforms, administrative capacities and the mobilisation of national resources are key factors of success; when these conditions are not met, the ENI programmes cannot reach intended objectives. This also explains why the mix of implementation modalities has been adjusted to the absorption capacity, Public Financial Management discipline and sector priorities of each country. The choice of implementation modalities is strongly linked to the EU response to partners' needs and priorities. On the positive side, ENI actions have been readily available to support the dialogue and identified reforms with the Eastern partnership framework, leading, among others, to the identification of 20 very concrete deliverables to be achieved by 2020 in the four priority areas identified at the Riga Summit.
  54

Few ENI programmes have already been implemented on the ground. Most programmes launched so far inevitably do not yet fully address the latest EU priorities (as stated by the ENP Review of late 2015) and still focus mainly on addressing the long-term root causes of migration and poverty, although there are notable exceptions (e.g. the large budget support operations for Ukraine and Tunisia that were targeted at the stabilisation of the economic situation of these two important partners). Newly designed ENI programmes (but not yet implemented) focus more on the new orientations of the revised ENP. The most recent project formulations address ‘EU policy priorities’ adopted by the 2015 ENP Review, recent communications on security, migration, resilience and the Global Strategy for the EU's Foreign and Security Policy inter alia through the integrated approach in order to address crises. To note, the adopted Partnership Priorities for Lebanon and Jordan, and those in preparation for Algeria, Armenia, Azerbaijan, Belarus and Egypt indicate the new way forward.

Addressing the refugee crisis through the flexible use of resources coming from the ENI, other EU instruments (such as the Instrument for Pre-accession Assistance and Humanitarian Aid), and EU Member States by means of their contributions to the Trust Funds
  55
  has had a positive impact in reducing the flow of refugees in Europe and the associated tensions with unmanaged flows. The continuous flow of refugees on the central Mediterranean migration route demonstrates however that the scale of the EU response so far does not match the scale of the problem in countries of transit or origin
  56
  .

The preparation and implementation of ENPI/ENI programmes is extensively used to sustain a strong policy dialogue. According to the external evaluators, the comparative advantage of the EU in this field is fully acknowledged by the partner countries, EU Member States and other donors
  57
  . The degree of leadership of the partner country in the dialogue varies and depends on the expertise and the coordination capacity of the governmental body in charge of international cooperation. Reported progress is of different quality in each country and even across sectors in a single country. In this context, chronic political instability, tensions in the relations with the EU and weakening administrative capacity in some partner countries since 2011 have hampered the likelihood of ENI programmes to reach their objectives. This has been for example the case in countries such as Moldova and Egypt.

Though several countries in the Eastern Neighbourhood have significantly improved their performance recently including through strong EU support and conditionality, the ENI countries remain among those with a relatively poor business environment, in particular in the South
  58
  , Over the last decade, and through support provided by the ENPI and ENI, the EU has contributed to a significant extent to increased trade potential within the Southern Neighbourhood
  59
  , and between the Neighbourhood countries and the EU
  60
  , though increase in trade flows within each sub region has been limited in both Neighbourhood regions.

In both regions, limited available information as well as the parallel support of many donors does not permit for explicitly linking EU financed projects to economic progress or resilience. Moreover, as most of them are middle-income countries, the specific contribution of the ENI actions is difficult to measure against national public policies. However, as shown in the table 3 , it is likely that, in the absence of particular shocks, the continuous financial support provided under the ENPI and now the ENI, associated with the ongoing policy dialogue have contributed to improve the enabling environment and to some significant results in some partner countries (e.g. Georgia, Tunisia, and Morocco).

EU cross cutting priorities (including fundamental rights, gender equality, climate change) have been unevenly implemented, with difficulties where they are not seen as priorities by the partners (although there are exceptions such as the continuous work in Morocco on Gender
  61
  or on democracy building in Tunisia
  62
  ).

Evaluation Question 3: Efficiency

To what extent is the ENI delivering efficiently?

The ENI is delivering aid efficiently in terms of procedures, processes and the ratio of administrative costs
  63
  . Financial control systems are in place and are effective at all levels
  64
  , and monitoring and evaluation systems at project and at country level provide the required feedback on implementation. Financial management procedures have been largely kept identical to the ENPI, reducing the disruption and loss of efficiency associated to major changes. Often ENI programmes continue directly on the achievements made by the ENPI, for example the ENI Cross Border Cooperation programmes. One additional step that has been introduced in the multi-annual programming process, is the strategic dialogue with the European Parliament before the finalisation of multi-annual programming documents, which adds several weeks to the programming process. This represents a step forward in terms of transparency and inclusiveness, but clearly negatively impacts on efficiency.

The features of the ENI Regulation and the CIR have allowed flexibility – something in line with the impact assessment and valued both by the external evaluators and the stakeholders consulted, though with diverging views
  65
  - and offered new opportunities to achieve the policy objectives. However, the EU Budget Financial Regulation, the Common Implementing Regulation (CIR) and the Commission administrative procedures and practices have prevailed over simplification efforts, leaving the regulatory burden unchanged
  66
  ; this has undermined reaching one of the objectives of the ENI impact assessment. The cumbersome administrative procedures make it difficult for smaller civil society organisations and social partner organisations (trade unions and employers' organisations) to get involved in project implementation
  67
  or may distract from the core business of the association
  68
  .

In the case of cross border cooperation (CBC), the simplification of implementation rules was an important objective; a recent survey of programmes authorities managing CBC operations show that the ENI rules have brought an improvement compared to the previous ENPI rules
  69
  , as expected by the impact assessment.

Under the ENI Regulation no significant changes occurred in the time span required to finalise a decision (the benchmark is set to 47 weeks for multi-annual programming, despite the four weeks added by the Strategic Dialogue with the European Parliament before finalisation of the programming documents) and for the various steps of implementation (formulation, financial agreement, contracting). ). However, some partner countries still complain that the process is too long and bureaucratic and that it is difficult to see a link between a programming mission, when consultations with partner countries authorities take place, and beginning of implementation. However, EU Delegations indicated that the time span for decision-making is improving
  70
  , as expected under the Impact Assessment.

Special measures
  71
  made possible by the Financing Regulation and the CIR are contributing to the ENI’s efficiency in delivery by accelerating the implementation process for unforeseen needs. The recourse to special measures (used notably in the case of Ukraine to respond to the 2014 event) does provide flexibility to the EU response and can help accelerate the programme preparation process.

The external evaluation found that the ENI exhibits an adequate organisational set-up and high quality in-house expertise which are considered key factors of efficiency in general and of cost-efficiency in particular. In the present setting, cost-efficiency of ENI management is confirmed by an administrative cost ratio of 2.0% on 2015 commitments (2.5% in 2014). However, in the absence of special arrangements (like with the Support Group for Ukraine
  72
  , or the Madad fund to address the refugee crisis), the Commission faced serious human resources constraints when a strong increase in financial assistance volumes took place (e.g. Tunisia).

In-house expertise was strengthened with the creation of Centres of Thematic Expertise (CoTE) which enhance coherence in formulating sector and thematic ENI interventions, allowing drawing on the lessons of past projects and best practices from other countries/regions. The expertise is focused on the most important policy areas but in some cases internal capacities remain weak. This is compensated through closer interaction with line DGs.

Budget support operations can contribute to deliver efficiently and rapidly (as evidenced in the swift delivery of large operations to Ukraine in 2014 and Tunisia since 2011). However, the strict eligibility criteria linked to budget support, and the time required for the good preparation of the operations, restrict the use of this modality to only a limited number of countries which have clear strategies and the strong willingness to implement them. As noted by the Court of Auditors' special report on EU assistance to Moldova
  73
  , when the Commission's programmes are not fully aligned to the national strategies, and when conditionality is not interpreted strictly, they reduce the Commission's leverage.

The setting of +/-10% range in the multi-annual indicative budget is another mechanism used – even if not intended –to introduce more flexibility in annual allocations
  74
  among countries. However, given the multiple developments in a series of countries, both positive or negative, and the tight budget, the Commission adjusted the initially planned indicative envelopes beyond the 10% range; there were strong increase in Jordan, Ukraine and Tunisia (not counting umbrella programmes funding) and decrease in Azerbaijan and Algeria for example. This flexibility has been in line with the objectives set out in the impact assessment and proved useful to respond to fast evolving challenges.

The monitoring and evaluation (M&E) system for the ENI was not set out at the Regulation level, but rather at corporate level, through the EU Results Framework
  75
  , and then through the normal programming and planning stage (Single Support Framework or Country Strategy Papers and Multi-annual Indicative Programmes/ Annual Action Documents), but cannot rely on national systems for assessing results, due to the poor quality of statistics, sometimes, and to a still limited culture of monitoring and evaluation. Monitoring and reporting systems (including result-oriented monitoring; the Programme Statement; as well as the Authorising Officer by Sub-Delegation and the External Assistance Management Reports) represent an efficient way to inform the Management and steer the work. Projects’ activities are monitored in a robust and systematic way at the level of outputs; in the case of CBC programmes, for example, a number of common output indicators have been defined in order to improve communication on results. This should enable better reporting in the future as well as future evaluation. However, reporting on outcome and long term impact remains embryonic
  76
  . This is partly due to the lack of relevant indicators, partly to an unclear intervention logic (theory of change)
  77
  . In 2018, after three pilot years, the Commission will be able to start reporting on operational results achieved under the ENI. In addition, a new information management tool under development will also focus more on results reporting and should be gradually rolled out as of 2018. Work has started already at the Commission on defining standardised set of result indicators for specific sector interventions (such as for Public Financial Management or Public administration reform, thanks to SIGMA). Further work is ongoing in the framework of the updating of the EU Results Framework and in the perspective of the implementation of a new IT tool (OPSYS) with a dedicated dimension dedicated to results..

Evaluation Question 4: Added value

To what extent do the ENI programmes add value compared to interventions by Member States or other key donors?

As an European Union instrument, the ENI is unique in the sense that it federates the somehow diverging interests of the Member States around the common objectives of the European Neighbourhood policy. This is linked to the closeness of the ENI beneficiaries and their direct influence on domestic issues (through migration, security, trade). Through the ENI, the EU common perception of urgency to work with our Neighbours is more vivid than ever even though differences in means and solutions could be debated on. This dynamism overcomes the traditional EU Member States' vested interests or historical links with beneficiary countries.

Seen in a broad context, there is a positive EU added-value in ENI programming and implementation vis-à-vis EU Member States.

The EU added value can be seen through the ENI’s i) ability to provide substantial funding mainly through grants in such cases as the responses to the Syrian Crisis, the stabilisation of Ukraine and Tunisia, or the support to reformers such as Georgia and Morocco
  78
  , ii) capacity to coordinate with other EU instruments and mobilise EU Member States' parallel funding along commonly agreed objectives to address everything from long-term to short-term and emergency challenges
  79
  , iii) comparative advantages in mobilising expertise suited for the needs and priorities within a framework consistent with EU and partner countries’ mutual interests, and iv) political influence and policy leverage
  80
  through dialogue with national authorities and civil society
  81
  . Moreover, various aid modalities (like Twinning and TAIEX, enabling the mobilisation of key EU public expertise) can be used in a coherent way and adjusted to the partner country’s (absorption) capacity. None of these features can be achieved individually by EU Member States.

Moreover, the importance of keeping a specific instrument was emphasised by several contributors to the Open Public Consultation
  82
  . It is also unique to the EU to actively combine regional, CBC and ENI-wide programmes to promote confidence building and good neighbourly
  83
  relations among countries on both sides of the EU borders and in some cases with the involvement of the Russian Federation. The combination of intervention types and aid modalities is unique among donors, even more so when the ENI uses innovative instruments like blending
  84
  and EU Trust Funds to mobilise additional funding and implementation capacities. Trust Funds (the "Madad" fund and the North Africa Window of the EU Emergency trust fund for Africa) are another potential incentive for reducing EU Member States aid fragmentation by using their expertise and implementation capacities. To date the Madad fund has attracted resources from many EU Member States, although the level of these contributions could be higher given the magnitude of the challenges in the region. However, visibility of the EU in such operations is rather limited; more generally, strategic communication and visibility are areas where stakeholders have noted room for improvement
  85
  .

The added value of the EU in enhancing the approximation to its acquis differs according to the ambition of partners in their relations with the EU. Nevertheless, the adoption of EU legislation and norms and standards remains a key objective for many partners and derive from their contractual obligations with the EU (notably those having DCFTAs), but also from the interest to improve export perspectives in third countries. For example, the adoption of EU sanitary and phytosanitary standards is necessary for all countries to export agricultural products to the EU, but can also make the products more tradable in non EU countries. Approximation to the acquis is therefore taking place in the economic area and in technical areas related to Rule of law issues irrespective of the willingness of a country to implement more fundamental and critical reforms. This explains the success of the twinning instruments also in relatively less open minded governments.

Evaluation Question 5: Coherence, consistency, complementarity and synergies

To what extent does the ENI facilitate coherence, consistency, complementarity and synergies both internally between its own set of objectives and programmes and vis-à-vis other EFIs?

The internal coherence within the ENI programmes is good. The overall good internal coherence is based on the existence of, and compliance with, the programming instructions but also the continuous efforts of the different services to avoid overlaps
  86
  . This finding is also shared by the external (ROM) monitoring.

The ENI bilateral, multi-country and cross-border (CBC) programmes have each been designed with different cooperation areas in mind and the Commission services make full use of this comprehensive toolbox
  87
  . Coherence is mainly found in relation to the policy framework of the Regulation (e.g. the strategic intent) rather than in operational linkages or synergies. However, the extent to which coherence and complementarity issues are fully exploited during implementation is not always clear from management reporting. Also, some evaluations suggest that the coordination of regional and bilateral programmes is suboptimal. For instance, evaluations have shown that there is little coordination between bilateral programmes, regional programmes and investment facilities in relation to private sector development, an area in which EU aid is intended to come from a combination of bilateral aid, regional private sector development programmes and regional investment facilities.
  88
  According to another evaluation
  89
  , Palestine is also an example case of the lack of consistency between the implementation of financial cooperation, the EU's official discourse and actions taken by individual Member States.

There is coherence between the ENI Regulation and the actions implemented under ENI in the sense that actions are required to present expected results in line with the specific objectives of the Regulation. A bottom-up process (meaning that actions’ priorities have been based on the needs of partner countries) ensures this coherence. In addition, the quality review process and Quality Support Groups have guaranteed that inconsistencies are spotted and corrected during the programming cycle
  90
  .

Overall consistency between the instruments is good and is ensured through different coordination mechanisms and close dialogue. At the level of the design of the country or regional programmes, there are numerous coordination mechanisms among the Commission services and the EEAS that ensure complementarity. For example, the Commission elaborated a joint humanitarian and development framework for Syria, Jordan and Lebanon, supported by different instruments ENI, emergency assistance and IcSP; since 2011, there have been numerous examples when external financing instruments intervene at different stages of a crisis situation, for example in the case of Libya, Tunisia and Ukraine, with the humanitarian instrument intervening usually first, followed by the IcSP and then the ENI intervention. The same complementarity applies in the case of the Syrian refugees crisis where the humanitarian instrument focuses on life saving actions and the ENI is more geared towards longer term development issues (such as providing support for schooling of children in hosting communities in Jordan and Lebanon). Strong coordination also exists within the Eastern Partnership ensuring prioritisation including for investments in connectivity.

Also, country team meetings are an occasion to discuss the interventions of each instrument; for example, the respective activity of all instruments in Ukraine is coordinated by the Support Group for Ukraine the service invites representatives from other services in the quality review process of programming documents and then these programming documents from the ENI go through the formal inter-service consultation process of the Commission. The same process applies for the programming process of other instruments.

The fact that the vast majority of funds comes from the ENI also ensures that the ENI strategy documents are considered by other instruments as the main points of reference for their interventions. For example, in 2015, 88% of the total allocation to the ENP countries of North Africa was provided by the ENI instrument and 8.6% by the humanitarian instrument. In the South Caucasus, the ENI provided 85% of total EU funds. Thematic instruments have not had the resources required to match the challenges faced by most neighbourhood countries.

There are also instances when resources from the ENI and EU internal programmes are pulled together in a coherent manner. For example, resources from the ENI are transferred to the Erasmus + programme at the beginning of two multi annual periods and put at the disposal of EU national agencies; similarly, the resources from the ENI on CBC are combined with resources from the European Development Fund in a context of one single operational programme. This was one key expectation from the ENI impact assessment.

All key aspects for stabilisation and development of partner countries are covered by the set of External Financing Instruments: defenders of human rights, empowerment of civil society, institutional building (of national and local authorities), etc. For most of these aspects, related objectives can be found in the geographic instrument and in related thematic Instruments. Crisis prevention is indicated as an ENI objective
  91
  . In fact, most actions under ENI encompassing a crisis prevention or tensions alleviation component are related to acute crisis and post-crisis situations and rather contribute to mitigate a potential deepening of the crisis. This is certainly a key aspect for stabilisation, for instance, through short-term (budget) support to Tunisia to help the country maintain its macro-economic stability or the support to host and refugee communities in Lebanon and Jordan. However, the share of ENI resources dedicated to the actual prevention of crisis, acknowledged as being the most cost-effective approach to stabilisation, is rather low. This is not a big limitation per se, as there is a dedicated thematic instrument for crisis response and prevention, the IcSP, which works to complement ENI programmes. However, emerging needs linked to crisis prevention and response are only partly addressed (mainly by the IcSP)
  92
  . Nevertheless, the Commission does not consider, as suggested by the external evaluators that there is a need to establish a dedicated component on crisis prevention within the ENI as this would undermine flexibility. More focus on this aspect of EU interventions is needed in strategic and programming documents.

Coherence has not been sufficient between funding channelled through the CBC programmes with other funding provided in the context of various regional cooperation frameworks as well as the implementation of EU's macro-regional strategies.

In a different context, the Neighbourhood Investment Facility (NIF) is a primary instrument for coordination and alignment of European financial institutions, including, but by far not only, the European Investment Bank (EIB) and the European Bank for Reconstruction and Development (EBRD). The NIF supports (mainly) infrastructure projects that complement or follow the reform agenda implemented by partner countries and supported by Sector Reform Programmes (including where relevant with budget support). This sequencing is important as it guarantees that instruments are used to the best of their values: grants, also in the form of Budget support for the reform process and the associated legislative agenda (laws, regulations, policies) and loans (through the NIF and in partnership with European and international financing institutions) for the implementation of investments that these reforms have identified as priority and the viability of which is ensured by the reformed institutional and regulatory set up.

The trust funds for Syrian Crisis and Africa have been used successfully and so have been other EU instruments outside the ENI, such as Macro-Financial Assistance (MFA)
  93
  . Most coordination efforts between ENI and other instruments are made at the level of the EU Delegations. Programme managers in charge of one particular sector are able to properly sequence and coordinate the various instruments.

A telephone survey confirmed that coordination with EU Member States has been strengthened in recent years
  94
  , also through Joint Programming, even though they are reluctant to further advance programme complementarity and synergies. Most partner countries are now engaged in a way or another in this process, with different levels of ambitions; in the case of Palestine, a full joint programming document has been prepared in 2017, replacing the previous EU programming document.

However, evidence from field visits demonstrates that gaining strength in advocacy by joining hands with EU Member States (and beyond with IFIs and other donors) is differentiated between the East and the South.
  95
  In the East, the EU (including EIB and EBRD) is typically the main (up to 70% of ODA) donor, few EU Member States are active (mainly Germany, Poland and Sweden) and only loans for middle-income countries are available from IMF and the World Bank. In the South, the state of play is more balanced. Besides the EU institutions, EU Member States are more present and IFIs can propose concessional loans. Even if the potential for synergy is higher for consistency in dialogue on policy reforms, it is difficult to materialise due to the weakness of the coordination, notably by lack of leadership by beneficiary governments. Public financial management and sector policy dialogues are mostly fuelled by the EU and the World Bank, the other donors presenting a lower profile.

In general, there is room for improvement concerning the communication to the partner countries and the general public on the EU's assistance.

Evaluation question 6: Leverage

To what extent has the ENI/ENPI leveraged: further funds and/or political and policy engagement (for example to what extent has the incentive-based approach leveraged progress made by partner countries in building and consolidating deep and sustainable democracy and in implementing agreed political, economic and social reform objectives).

Though it is too early to assess the extent to which ENI has improved the leverage of EU resources on the capacity of partners to engage in structural reforms
  96
  , especially the experience with budget support and blending tends to support a positive contribution of EU interventions
  97
  .

Expected results of committed budget support programmes seem encouraging so far. This corroborates the findings of previous evaluations on budget support operations financed under the ENPI, according to which the experience of many countries demonstrates that budget support can achieve significant results within a diverse and often challenging set of contexts
  98
  . In particular, it has contributed in important ways to upgrading the capability of these governments to manage their public finances, to deliver services and to regulate economic activity, for the benefit of their citizens. For example, one evaluation noted that the coverage of health services and of secondary and tertiary education had improved in Morocco and Tunisia, and there were significant reductions in income poverty in both countries (although it is difficult to attribute this directly to budget support)
  99
  . This evaluation highlighted the limited scope of reforms addressing governance issues, which remains a current challenge under the ENI.

Blending of EU grants with resources (loans and grants) of EU and EU Member States’ financial institutions – and to a lesser extent with IFIs – proved to be an effective way to attract additional resources to achieve ENI objectives through infrastructure projects or by supporting private sector development
  100
  . Since 2008, ENPI/ENI instruments have contributed EUR 1 678 million to the Neighbourhood Investment Facility (NIF), leading to a total of close to EUR 15 billion from financial institutions. Also 15 EU Member States have committed a total of EUR 84 million to the NIF Trust Fund, complementing ENPI/EU resources. ENI funds brought into EU trust funds have induced EU Member States to contribute but not at the level proportionate to the challenges.

Through policy dialogue and cooperation, the ENI actions leveraged sustainable implementation of agreed reform priorities in a number of countries during the period under review, mainly Morocco, Georgia, Ukraine and Tunisia. Egypt is improving its up-take of ENI-sponsored reforms as well but these are limited to ‘technical’ areas (renewable energy, urban development). In the other countries of the Neighbourhood, many factors which remain beyond the reach of EU action (e.g. political weakness, oil revenues, on-going conflicts, diverging regional alliances etc.) represent stumbling blocks to leverage at present. ENI leverage is stronger in those cases where the budget support modality can be used as the amount of resources invested by the EU is much higher than for traditional technical assistance programmes, there is a stronger policy dialogue and focus on horizontal issues like public financial management, not to mention the possibility to attract other donors to participate in support packages for such reforms (e.g. Morocco and Tunisia)
  101
  .

The ENI provided the framework and the resources to mobilise additional funding, either from the pre-existing blending facility - NIF
  102
  , or from recent initiatives like the EU Trust Funds (for example resources leveraged from EU Member States amount to just over 10% of the contribution from the EU Budget).

Financial leverage is sought at project level, either by partnering with EU Member States and International Financial Institutions, or by promoting contributions from the national budgets. There are many successful reported cases of funds’ pooling at a project level
  103
  . Other types of funds’ pooling, such as public private partnerships or schemes using diaspora remittances, have not been used so far for various reasons (e.g. like the significant sovereign risk or the lack of an appropriate legal framework).

It should be also noted that, as appears in the external evaluation, it is difficult to assess to what extent the incentive-based mechanisms under the ENI have leveraged reforms since those countries that implemented more reforms were in general already convinced of their interest (as Georgia, Morocco, Jordan, Ukraine), whilst others did not find the incentive a sufficient reason to embark in new reforms (Egypt, Azerbaijan, among others).

6. Conclusions

The European Neighbourhood Instrument (ENI) is the main and by far the largest financial instrument supporting through grants the implementation of the EU policy towards the 16 countries in the Southern and Eastern Neighbourhood of the EU, and Russia for programmes where it is eligible.

It is yet difficult to assess the effectiveness of ENI as implementation started only in 2015 and most programmes adopted since then require time to be rolled out, let alone to produce outputs and outcomes. In many instances, actions financed under the previous instrument (ENPI) and ENI, together with other EU instruments, have been effective in supporting stabilisation and building the resilience of important partner countries, notably by allowing a swift recovery of growth in Ukraine after 2015 or by avoiding a recession in Tunisia which could have derailed the smooth transition to democracy. ENPI/ENI funds have also been helpful in the stabilisation of Jordan and Lebanon which have been confronted with the massive influx of Syrian refugees, notably in the efforts made to provide school opportunities for Syrian children. ENPI and ENI interventions have contributed to institutional and administrative strengthening in many partner countries, through the provision of high level of EU expertise. ENPI and ENI funds have also leveraged much higher funds (notably loans) from European and international financial institutions, and strengthened the policy leverage of the EU.

The evaluation finds that ENI is overall relevant and fit for purpose. It has allowed the EU to implement the reviewed Neighbourhood Policy. The ENI has also proven its capacity to respond in a flexible manner, in line with the impact assessment, to the multiple crises and new challenges in the Neighbourhood, in particular in Ukraine and Tunisia. This included the mobilisation of funding from other sources (in particular from the margin of heading IV and other external financing instruments), the mobilisation of other instruments, such as Humanitarian Aid and Macro-Financial Assistance, in parallel to ENI interventions, the Special measures to address the Ukrainian crisis, as well as Trust Funds in response to the refugee crisis.

The external evaluation, the broad consultation with stakeholders and the internal assessments highlight certain areas to be further considered:

·The current challenges and needs in the Neighbourhood have put serious strains on the ENI budget and human resources, and despite budget reinforcements and use of all flexibility mechanisms, the response capacities of the instrument have been stretched to their limits.

·The large share of programmed assistance has allowed to keep supporting structural reforms but at times limited the scope for adjusting the EU financial response to pressing needs. The lack of a single support framework for Ukraine in the first years of the ENI implementation has not prevented the Commission to provide relevant support.

·The complex political environment in some Neighbourhood countries means that the implementation of the ENI has not been equally effective in all countries, despite increasing differentiation.

·Neighbourhood countries have made varying degrees of progress in political and economic reforms. Nonetheless, both the political and the economic situation remain challenging in many of them, notably because of high unemployment, an unfavourable business climate and high corruption levels. ENI operations are addressing the root causes of these challenges but partner countries' commitment to reform varies greatly and limits the effectiveness of ENI interventions. In addition to partners' political commitment to reforms, their institutional capacity is key for the sustainability of reform and meaningful policy dialogue with the EU.

·The implementation of the incentive-based approach defined in the ENI provided significantly higher financial resources to those partners that have made the strongest progress on political reforms but its impact on leveraging further reforms, in particular in other countries, still needs to be demonstrated. The ENI has not been able to provide sufficient incentives to those countries reluctant to engage in political reforms.

·The overall coherence and complementarity of the different EU instruments has improved but was not always up to the high challenges the EU was confronted with in the Neighbourhood. Existing risk analyses and scenario building tools, already developed by the EEAS, have not been sufficiently used in the framework of joint action by the EEAS and Commission services.

·As it is the case for other external action instruments, reporting at outcome and impact remains embryonic both at the instrument and at country level. Work is ongoing to develop an efficient monitoring system at the Commission and to support data production by partner countries.

Overall, the existence of a dedicated financial instrument for the neighbourhood has been one of the most concrete evidence translating the political importance attached to the region by the EU, and has lent credibility to the EU's intention to deepen political cooperation and pursue gradual economic integration with its neighbours.

Annex 1. Evaluation Questions

Relevance

1. To what extent do the overall and the specific objectives (ENI Regulation, Article 1 and 2) and the design
  104
  of the ENI respond to:

(i) EU priorities and beneficiary needs identified at the time the instrument was adopted (2014)?

(ii) Current EU priorities, in particular emerging from the 2015 ENP Review such us stabilisation, and beneficiary needs, given the evolving challenges and priorities in the international context (2017)?

Information sought in this area includes:

· A timeline showing congruence/divergence of the instrument against the initial context (e.g. ENP review 2011, negotiations on Association Agreements and DCFTAs) and against the evolving context, including global challenges, and institutional policy changes e.g. to what extent does the ENI support the implementation of the revised ENP, how it responds to new policy/political situations in our partner countries and also respond to the demands of Agenda 2030, including the need to co-operate with emerging countries on implementing the SDGs.

Effectiveness, impact, sustainability

2. To what extent does the ENI deliver results against the instrument's objectives, and specific EU priorities?
  105

Information sought in this area includes:

·To what extent do ENPI/ENI programmes contribute towards shared prosperity and good neighbourliness, and towards the other general and specific objectives listed in the ENI Regulation, Article 1 and 2

·To what extent have ENI/ENPI programmes supported policy dialogue and implementation of reform objectives agreed with each partner countries?

·To what extent has the ENI/ENPI contributed to the European Union's priorities for smart, sustainable and inclusive growth?

·To what extent has the ENI/ENPI contributed to enhance sub-regional, regional and European Neighbourhood-wide collaboration as well as cross-border cooperation.

·To what extent does the ENI mainstream EU policy priorities (e.g. gender, climate change) and other issues highlighted for mainstreaming in the instrument, and, where relevant, deliver on the commitments including the financial allocations (ENI Regulation preamble, Article 1, 2 and 7)

·To what extent does the ENI/ENPI promote principles of aid effectiveness, such as ownership, joint programming (ENI Regulation, Article 5)

·To what extent are the processes conducive to programming, identification/formulation of effective actions (ENI Regulation, Article 6-17)?

·To what extent has the ENI/ENPI provided differentiated support?

·To what extent does the ENI use the incentive-based approach? (ENI Regulation, Article 4 and 7)

·To what extent is the ENI flexible enough to respond to changing needs? (e.g. in response to the ENP review of 2015, and rapidly evolving contexts in many countries)

Efficiency

3. To what extent is the ENI delivering efficiently?
  106

Information sought in this area includes:

·What is the ratio of administrative costs (as defined as “ENI Support Expenditure” in the Draft General Budget of the EU
  107
  ) to overall budget?

·How efficient is budget execution in terms of time taken from commitments to payments?

·Have the changes made to ENI 2014 – 2020 from the previous ENPI 2007 – 2013 brought efficiency gains? E.g. to what extent have changes in the programming process contributed to simplification?

·Are there areas, such as administrative/management procedures, where the ENI can be simplified to eliminate unnecessary burden?

·To what extent is the ENI in line with the implementing rules of the CIR? Specifically in terms of:

oImplementation

§Subject matter and principles

§Adoption of action programmes, individual measures and special measures

§Support measures

oProvisions on the Financing Methods

§General financing provisions

§Taxes duties and charges

§Specific financing provisions

§Protection of the financial interests of the Union

oRules on nationality and origin for public procurement, grant and other award procedures

oClimate action and biodiversity expenditure

oInvolvement of stakeholders of beneficiary countries

oCommon rules

§Eligibility under the ENI

oMonitoring and evaluation of actions

·To what extent are the following in place and functioning:

oappropriate monitoring processes and indicators for measurement of the performance of the ENI instrument

orelevant strategic and operational indicators to measure results achieved by the ENI?

Added value

4. To what extent do the ENI programmes add value compared to interventions by Member States or other key donors?

Information sought in this area includes:

·Where the ENI is operating in the same field as Member States or other key donors, does it offer added value in terms of size of engagement, particular expertise, and/or particular weight in advocacy?

Coherence, consistency, complementarity and synergies

5. To what extent does the ENI facilitate coherence, consistency, complementarity and synergies both internally between its own set of objectives and programmes and vis-à-vis other EFIs?

Information sought in this area includes:

·To what extent are the different ENI programmes coherent/complementing/overlapping with one another, including coherence between bilateral, multi-country and cross-border cooperation programmes?

·To what extent are the different ENI programmes aligned with the evolving ENP policy and, where relevant, the EU development policy?

·To what extent are the programmes consistent with EU external action policies?

·To what extent do the programmes complement/overlap/stimulate synergies with other external action financing instruments?
  108

·To what extent does the ENI complement/overlap with other EU instruments outside of development policy?

·To what extent does the ENI complement/overlap with interventions of other donors?

Leverage

6. To what extent has the ENI/ENPI leveraged

üfurther funds and/or

üpolitical and policy engagement (for example to what extent has the incentive-based approach leveraged progress made by partner countries in building and consolidating deep and sustainable democracy and in implementing agreed political, economic and social reform objectives).

7. To what extent could the ENI be enhanced to achieve its policy objectives more effectively and efficiently?

8. How can programming and implementation of ENI assistance be enhanced to improve the impact and sustainability of financial assistance?

Annex 2. Procedural information

Organisation

This evaluation assessed the European Neighbourhood Instrument (ENI) ahead of the Mid-Term Review Report, as set out in Article 17 of the Common Implementing Regulation (CIR 2014). It will mainly be used to generate information for the Mid-Term Review Report requested by the CIR due end 2017. The evaluation provides information on relevance, EU added value, coherence and complementarity, effectiveness, efficiency, sustainability, leverage and impact of the instrument.

The Staff working document is based on the independent assessment carried out by an external contractor and complemented by further internal analysis.

The lead DG to carry out and manage this evaluation has been the Directorate General for Neighbourhood and Enlargement Negotiations (DG NEAR). But as this evaluation was part of a wider set of evaluations covering the instruments under Heading IV of the Multiannual Financial Framework 2014-2020 as set out in the CIR, as well as the performance review of the European Development Fund, and in view of ensuring a consistent European external policy, all the evaluations were carried out in an interlinked and co-ordinated manner. To ensure this coherence of the different evaluation from the beginning DG NEAR, DEVCO, FPI and EEAS worked closely together.

An Umbrella Inter-service Steering group (ISG) was set up to oversee the mid-term review process of the external instruments. This ISG is chaired by the Directorate General for International Cooperation and development (DEVCO) and is composed of members from relevant Commission services.. In addition ISGs were set up for each individual evaluation. The process started in June 2015 and the Umbrella ISG met formally for the first time on 21 September 2015 and during the following weeks and months agreed on the joint elements which all the different evaluation should use in their specific roadmaps and Terms of reference.

The ISG for the ENI evaluation was set up in October 2015, including relevant services of the Commission, and met for the first time on 15 October 2015. The roadmap for ENI was published, like all the other instruments Road Maps, at the end of October 2015; no comments were received. The Terms of reference were approved by the ISG on 15 April 2016. A specific contract was awarded on 9 June 2016 to PARTICIP GmbH, using the framework contract set up for carrying out evaluations in the context of external relations, and the evaluators started their work on 30 June 2016.

All foreseen deliverables were discussed in depth by the ISG, under the coordination of the evaluation manager in DG NEAR. During the evaluation process from October 2015 onwards the ISG met 7 times, in addition the members were consulted through email several times on draft and revised draft reports.

Following the approval of the Draft Report by the ISG after the meeting of 11 January 2017, this report was then placed, with the other instruments draft reports, on the Commission's website for the web based open public consultation (OPC) on 7 February 2017. The OPC concluded on 3 May 2017. During the OPC period targeted face to face meetings were organised with representatives of CSOs, both from the beneficiary countries and Europe, with representatives of EU member states and European Parliament. Following the closure of the OPC, during which 44 responses were received, a revised report was discussed again by the ISG leading to the issuing of the final report.

Evaluation Design

Concerning the evaluation design, the intervention logic (IL) forms the backbone of the analytical framework. It is visualised through an overall reconstructed IL diagram, which brings together the most relevant elements (from inputs to impacts, but also assumptions) in a single framework.

In accordance with the reconstructed IL for ENI, the evaluation first focused mainly on the additional features and tools introduced with the ENI Regulation. The development of intervention logics for the ENP 2011 and its revised version of 2015 that successively constituted the pivot of the instrument’s policy framework led the consultant to pay additional attention to the ability of DG NEAR and EEAS to respond to emergencies and crises in the Neighbourhood.

The critical analysis of the (reconstructed) IL helped to focus the evaluative framework (EQs, JCs and indicators) on key issues, particularly the underlying theory of change and the assumptions that ensure the operationalisation of the regulatory provisions.

Evaluation questions, which were very similar to all the simultaneously on-going instruments evaluations, structure the analysis to gather evidence. The evaluation questions, as well as more instrument-specific sub-questions, were provided in the Terms of Reference. Considering the objective of the various instruments evaluations launched under the mid-term review, consistency among the respective EQ/JC structure was prioritised up to the JC level. Each question was structured into JC and indicators required to provide an answer based on a synthesis of evidences.

Methodology and data

Concerning data collection and analysis, the evaluation of ENI was evidence-based. The EU evaluation criteria (relevance, effective-ness, efficiency, EU added value, coherence, consistency, complementarity and synergies, as well as leverage) were applied as an underlying basis. The evaluation questions (EQs) from the Terms of Reference gave rise to a number of JCs and associated indicators. The evaluation was indicator-based.

The evaluation used a mixed methods approach blending quantitative and qualitative methods, with a focus on the latter. The main analytical tools consisted of rigorous assessment of documentation and consultation of stakeholders (via interviews, group consultations, and the Open Public Consultation- OPC; key issues have also been addressed by a joint-instruments survey managed by the chapeau contract).

For all EQs, data collection included various tools and methods. Priority was given to document review and interviews at HQ, in line with the methodological indications in the ToR and further guidance provided by DG NEAR and DEVCO. Data collection activities were carried out mainly during the desk phase, but also continued during subsequent phases. They included:

• Compilation and analysis of roughly 300 DG NEAR/EEAS documents;

• Interviews with approximately 60 staff (mostly heads of unit/division and key staff) in NEAR, EEAS, Line DGs, EU Member States representatives in ENI Committee;

• Questionnaire-based telephone survey with twelve (of 16) Heads of Cooperation in ENI countries;

• Analysis of EAMRs for the years 2013, 2014, and 2015;

• Analysis of the QSG2 quality reviews 2014 to-date;

• Results of the Joint Survey, notably the ENI-specific open questions.

The above sources were complemented during the validation phase, which focused on (i) validation of the hypotheses identified by JCs and EQs during the desk work and (ii) gathering the views of the stakeholders (national authorities, CSOs, EU Member States) and EU Delegations as key implementers/partners of ENI programmes.

The four field missions (Egypt, Georgia, Tunisia, and Ukraine) utilised the same interview guidelines and were reported upon based on a common template.

Interviews were organised during the desk phase with relevant units of DG NEAR and the EEAS, as well as other concerned EU institutions and Member States. To the extent possible, interviews were attended by two senior experts of the team. Brief interview guidelines were transmitted at least two days before the meetings, in particular for telephone interviews (with EU Delegations Heads of Cooperation, Centres of Thematic Expertise (CoTEs), EU Member States).

The combination of data collection methods and techniques varied according to the different EQs and JCs. Several methods and techniques were used to collect the necessary data to assess a given JC according to the nature of the set of indicators identified. Where possible, the Evaluation Team combined the use of qualitative and quantitative data and relied both on primary and secondary data sources (EU Budget, NEAR) while taking into account re-source and time constraints.

The synthesis phase was devoted to constructing answers to the evaluation questions, revising and strengthening the narrative of the intervention logic and formulating conclusions and recommendations on the basis of the data collected throughout the process. A draft final report was made available for an Open Public Consultation, which provided some inputs for the final report.

Challenges and limitations

The main challenge, as identified already in the ToR, has been the very tight timeline dictated by the CIR. The evaluation team managed to mitigate potential limitations by mobilising resources to carry out all tasks indicated in the methodology in parallel and analyse additional sources of information when available. The agreed tight schedule might have somewhat limited the depth of data treatment but was overall overcome effectively by the external evaluators' team.

The key limitation, which is inherent to any mid-term review, is that the outputs of the Instrument (i.e. ENI programmes under the SSF) have only recently entered the implementation stage. Therefore, effectiveness/sustainability/impact criteria could not be assessed based on actual results at this stage. The external evaluators' team was, however, able to gain a first impression on these criteria based on the quality of the action documents and, for the five case studies, the congruence of the analysis of the context and the EU response strategy.

The ISG as well as other staff from DG NEAR, EEAS and ENI delegations actively participated in providing missing data to the evaluators. The multiple rounds of commenting also mean that facts have been verified and this facilitated the triangulation of data.

Overall, the quality of the collected evidence (documentation, interviews, data and survey results) for this evaluation can be assessed as good, demonstrating a satisfactory degree of confidence regarding the various findings of this evaluation.

Annex 2bis Response to the opinion of the Regulatory Scrutiny Board

Opinion of the Regulatory Scrutiny Board

Title: Evaluation / European Neighbourhood Instrument

Overall opinion:
     1st submission: NEGATIVE

2nd submission: 28/09/2017 - POSITIVE

| Comments on 2nd submission


  (C) Further considerations and recommendations | NEAR Response |
| --- | --- |
| (1) The report should apply the intervention logic more systematically throughout. A new section summarising partner country contexts (see also point 3 below) could form a basis for assessing the extent to which the 'key contextual factors' have applied and the 'intervention assumptions' were met. The section on limitations of the report should discuss associated implications for ENI performance and draw any lessons for the future. | In view of keeping a coherent approach for the external financing instruments SWDs and not to overload the documents given the number of countries concerned, this recommendation is not taken on board.


  The Conclusions put emphasis on problematic issues to be tackled. A "fiche contradictoire" will be shortly prepared by the services to address issues to be corrected within the present instrument. Work has started, meanwhile, with respect to the future MFF, for which a Impact Assessment will be prepared. |
| (2) Annex 3 contains some useful material on stakeholder concerns, and the report might refer to this in its responses to evaluation questions. More generally, these are concerns that the evaluation could follow up on, triangulating across other data sources to provide policymakers with a clearer understanding of alleged shortcomings. The same holds for the recommendations of the external evaluation and the critical comments from the Court of Auditors. Commission services should account for these concerns and explain why they are not taken up in the report. | The revised version includes more reference to the consultation, both on line (OPC) and through ad hoc meetings with EU MS, the EP and civil society and local authorities' organisations.


  Some Court of Auditors reports are rather positive. Reference has been added to different CoA reports and to other thematic evaluations. |
| (3) In order to make the report more readable and more self-standing, as well as to better present the current context of the ENI, the report could summarise separately the situation and context of the 16 partner countries in question. This might take the form of either a summary table or a dedicated section. The objective would be to describe the context in which ENI has been operating and to give the reader an overview of the varied and often deteriorating circumstances in recipient countries during the review period. An overview table of the 16 countries might usefully set out the main challenges the countries have faced that have complicated ENI implementation. As programme effectiveness relies on partner country co-operation, this section should link with a section on limitations (see below). The limitations section could then address associated consequences for programme implementation and evaluation. The report should discuss how changes in political commitment of partner countries may have lowered ENI performance in this context.


  The executive summary is mostly easy to follow. The main report is less accessible, with language that often seems aimed at insiders. More could be done to communicate the main points in plain language, especially in the conclusions section. | The section on Conclusion has been reworked.


  This report is an evaluation of the ENI regulation and cannot go into the details of the situation in each of the 16 ENI countries, which would make the report very long. In addition, consistency with the other external financing instruments SWDs needs to be ensured, and the Commission cannot have in these reports a state of play on the countries it is intervening in. A table has been added making reference to the major events having taken place in some countries in the last few years with the impact in the concerned countries.


  . |
| Comments on 1st submission | |
| (1) Presentation, scope and lessons learnt The SWD should be revised to become a self-standing document, which a non-expert reader can understand without having to consult the external study. It should do more than summarise positive aspects and conclusions of the external study. In its analysis, findings and conclusions, the SWD currently appears to overlook critical views expressed in the external evaluation, European Court of Auditors' reports and by some stakeholders. The SWD should be transparent about these and explain where the Services may not agree with the assessment of the external evaluation study and why (e.g. crisis prevention, flexibility, involvement of national authorities, multiple strategy scenarios). Since the results of this mid-term evaluation will feed into developing the future instrument, the SWD should identify and cover aspects that are relevant to learning from past experience. This includes assessing what design features (e.g. priority setting, programmed vs flexible support, delivery mechanisms, incentive-based approach) of the instrument work well or not. It would be helpful to indicate any countries or sectors that receive more than sufficient funding (e.g. 'donor congestion') and where there are shortages. It would also be helpful to include an assessment of the degree of coherence and complementarity with other financing instruments. The report should critically assess the existing monitoring and reporting arrangements. It should present any lessons for future data collection that would facilitate a better-informed and more useful evaluation in the future. | The revised version put more emphasis on the critical aspects pointed out by the evaluation. These will be better addressed in the framework of the "fiche contradictoire" that DG NEAR is preparing on the follow-up. Whenever DG NEAR disagrees with the views expressed by the evaluators, this is clearly said, and will be mentioned in the "fiche contradictoire" if recommendations are not accepted. More substantial changes are likely to be proposed in the framework of the future instrument, which is outside the scope of the current mid-term review report. |
| 2) Effectiveness, Efficiency, Intervention Logic


  The SWD should explain its metrics for assessing effectiveness of the ENI. Firstly, it should clarify the ENI's objectives (e.g. contribute to stabilisation in the region, alignment with the EU values and rules, ...). Secondly, it should explain how ENI interventions are intended to work towards reaching those objectives (the 'intervention logic'). It should also incorporate relevant experience with the predecessor programme (ENPI) and expectations as set out in the impact assessment for the current ENI. In doing so, the SWD should specify which design features of the ENI make it effective or not (e.g. flexibility, budget support, conditionality). Using the intervention logic, the SWD should specify what benchmarks it will use to assess the performance of the instrument. The SWD should also describe what the ENI adds to other programmes, and compare against a country or set of countries where this instrument does not operate. Finally, it should explain how changes in the Neighbourhood policy have affected the effectiveness of the ENI.


  The SWD should identify any potential for simplifying processes and procedures. It should include basic information relevant to conclusions on ENI efficiency in terms of procedures, processes and the ratio of administrative costs. | The revised text has addressed, as much as possible these points. Reference is made to the ENI Impact Assessment and ENPI performance, whenever possible, to explain to which extent it was up to the expectations, mentioning areas where some important results were achieved (in streamlining public administration reforms (Georgia, Ukraine), keep focus on democratic transition despite a difficult environment and terrorism attacks (Tunisia), ensure economic resilience in difficult environments (Jordan, Ukraine, Tunisia). Reference to high level indicators is made to highlight progress, or lack of it, in relevant areas.


  Further elements of value added of EU support and complementarity with other instruments have been added.


  Areas of success in terms of simplification and flexibility are highlighted, together with areas where more work is required. |
| 3) Relevance and coherence


  The SWD should assess the continued relevance of the ENI as a geographic instrument covering countries in very diverse circumstances and with very different needs. It should discuss the trade-offs in using geographic versus thematic instruments in the countries covered by the ENI. The SWD should explain what design features make ENI complementary to other financing instruments, e.g. the European Instrument for Democracy and Human Rights, the Instrument contributing to Stability and Peace, the Development Cooperation Instrument, as well as instruments like Erasmus+. It should explain relevant sequencing considerations when there is overlap between the geographic and thematic focus (e.g. short-term crisis prevention by thematic instruments followed by long-term stability by geographic instruments). | This part has also been strengthened. The importance of the ENI, which provides most of the funding to Neighbourhood countries and its complementarity with specific, but very small, thematic instruments contributions is emphasised. |

Comments on 2nd submission

(C) Further considerations and recommendations

NEAR Response

(1) The report should apply the intervention logic more systematically throughout. A new section summarising partner country contexts (see also point 3 below) could form a basis for assessing the extent to which the 'key contextual factors' have applied and the 'intervention assumptions' were met. The section on limitations of the report should discuss associated implications for ENI performance and draw any lessons for the future.

In view of keeping a coherent approach for the external financing instruments SWDs and not to overload the documents given the number of countries concerned, this recommendation is not taken on board.

The Conclusions put emphasis on problematic issues to be tackled. A "fiche contradictoire" will be shortly prepared by the services to address issues to be corrected within the present instrument. Work has started, meanwhile, with respect to the future MFF, for which a Impact Assessment will be prepared.

(2) Annex 3 contains some useful material on stakeholder concerns, and the report might refer to this in its responses to evaluation questions. More generally, these are concerns that the evaluation could follow up on, triangulating across other data sources to provide policymakers with a clearer understanding of alleged shortcomings. The same holds for the recommendations of the external evaluation and the critical comments from the Court of Auditors. Commission services should account for these concerns and explain why they are not taken up in the report.

The revised version includes more reference to the consultation, both on line (OPC) and through ad hoc meetings with EU MS, the EP and civil society and local authorities' organisations.

Some Court of Auditors reports are rather positive. Reference has been added to different CoA reports and to other thematic evaluations.

(3) In order to make the report more readable and more self-standing, as well as to better present the current context of the ENI, the report could summarise separately the situation and context of the 16 partner countries in question. This might take the form of either a summary table or a dedicated section. The objective would be to describe the context in which ENI has been operating and to give the reader an overview of the varied and often deteriorating circumstances in recipient countries during the review period. An overview table of the 16 countries might usefully set out the main challenges the countries have faced that have complicated ENI implementation. As programme effectiveness relies on partner country co-operation, this section should link with a section on limitations (see below). The limitations section could then address associated consequences for programme implementation and evaluation. The report should discuss how changes in political commitment of partner countries may have lowered ENI performance in this context.

The executive summary is mostly easy to follow. The main report is less accessible, with language that often seems aimed at insiders. More could be done to communicate the main points in plain language, especially in the conclusions section.

The section on Conclusion has been reworked.

This report is an evaluation of the ENI regulation and cannot go into the details of the situation in each of the 16 ENI countries, which would make the report very long. In addition, consistency with the other external financing instruments SWDs needs to be ensured, and the Commission cannot have in these reports a state of play on the countries it is intervening in. A table has been added making reference to the major events having taken place in some countries in the last few years with the impact in the concerned countries.

.

Comments on 1st submission

(1) Presentation, scope and lessons learnt The SWD should be revised to become a self-standing document, which a non-expert reader can understand without having to consult the external study. It should do more than summarise positive aspects and conclusions of the external study. In its analysis, findings and conclusions, the SWD currently appears to overlook critical views expressed in the external evaluation, European Court of Auditors' reports and by some stakeholders. The SWD should be transparent about these and explain where the Services may not agree with the assessment of the external evaluation study and why (e.g. crisis prevention, flexibility, involvement of national authorities, multiple strategy scenarios). Since the results of this mid-term evaluation will feed into developing the future instrument, the SWD should identify and cover aspects that are relevant to learning from past experience. This includes assessing what design features (e.g. priority setting, programmed vs flexible support, delivery mechanisms, incentive-based approach) of the instrument work well or not. It would be helpful to indicate any countries or sectors that receive more than sufficient funding (e.g. 'donor congestion') and where there are shortages. It would also be helpful to include an assessment of the degree of coherence and complementarity with other financing instruments. The report should critically assess the existing monitoring and reporting arrangements. It should present any lessons for future data collection that would facilitate a better-informed and more useful evaluation in the future.

The revised version put more emphasis on the critical aspects pointed out by the evaluation. These will be better addressed in the framework of the "fiche contradictoire" that DG NEAR is preparing on the follow-up. Whenever DG NEAR disagrees with the views expressed by the evaluators, this is clearly said, and will be mentioned in the "fiche contradictoire" if recommendations are not accepted. More substantial changes are likely to be proposed in the framework of the future instrument, which is outside the scope of the current mid-term review report.

2) Effectiveness, Efficiency, Intervention Logic

The SWD should explain its metrics for assessing effectiveness of the ENI. Firstly, it should clarify the ENI's objectives (e.g. contribute to stabilisation in the region, alignment with the EU values and rules, ...). Secondly, it should explain how ENI interventions are intended to work towards reaching those objectives (the 'intervention logic'). It should also incorporate relevant experience with the predecessor programme (ENPI) and expectations as set out in the impact assessment for the current ENI. In doing so, the SWD should specify which design features of the ENI make it effective or not (e.g. flexibility, budget support, conditionality). Using the intervention logic, the SWD should specify what benchmarks it will use to assess the performance of the instrument. The SWD should also describe what the ENI adds to other programmes, and compare against a country or set of countries where this instrument does not operate. Finally, it should explain how changes in the Neighbourhood policy have affected the effectiveness of the ENI.

The SWD should identify any potential for simplifying processes and procedures. It should include basic information relevant to conclusions on ENI efficiency in terms of procedures, processes and the ratio of administrative costs.

The revised text has addressed, as much as possible these points. Reference is made to the ENI Impact Assessment and ENPI performance, whenever possible, to explain to which extent it was up to the expectations, mentioning areas where some important results were achieved (in streamlining public administration reforms (Georgia, Ukraine), keep focus on democratic transition despite a difficult environment and terrorism attacks (Tunisia), ensure economic resilience in difficult environments (Jordan, Ukraine, Tunisia). Reference to high level indicators is made to highlight progress, or lack of it, in relevant areas.

Further elements of value added of EU support and complementarity with other instruments have been added.

Areas of success in terms of simplification and flexibility are highlighted, together with areas where more work is required.

3) Relevance and coherence

The SWD should assess the continued relevance of the ENI as a geographic instrument covering countries in very diverse circumstances and with very different needs. It should discuss the trade-offs in using geographic versus thematic instruments in the countries covered by the ENI. The SWD should explain what design features make ENI complementary to other financing instruments, e.g. the European Instrument for Democracy and Human Rights, the Instrument contributing to Stability and Peace, the Development Cooperation Instrument, as well as instruments like Erasmus+. It should explain relevant sequencing considerations when there is overlap between the geographic and thematic focus (e.g. short-term crisis prevention by thematic instruments followed by long-term stability by geographic instruments).

This part has also been strengthened. The importance of the ENI, which provides most of the funding to Neighbourhood countries and its complementarity with specific, but very small, thematic instruments contributions is emphasised.

Annex 3. Synopsis report of the stakeholders' consultation

Consultation strategy

The evaluation of the European Neighbourhood Instrument (ENI) for the period 2014-20 was supposed to feed, together with parallel evaluations of other external financing instruments under the multiannual financial framework (MFF), the required mid-term review report (MTR). The objectives of all instruments evaluations, including the one on the ENI, are to a) provide the relevant external relations services of the European Union and the wider public with an assessment of the instruments, including complementarities and synergies among them; and b) inform the programming and implementation of the current instruments, as well as the next generation of instruments.

This consultation strategy provides an overview of the approach that was taken for consulting with the main stakeholders of this evaluation. It contains two elements. The first describes the overall setup of the strategy (i.e. the underlying stakeholder mapping, the framework and strategy as well as the timeframe). The second provides statistics on the consultation of stakeholders. An important component of the stakeholder strategy is the open public consultation (OPC) at the end of the synthesis phase of the evaluation to acquire feedback from all relevant parties on the main evaluation findings, which is presented shortly in its own separate section below.

One should also mention that the consultation had already started at the moment of the evaluation preparation, when the roadmap was published, together with the others related to the other external evaluations, to seek feedback from stakeholders. But no feedback was received on any of the roadmaps.

Stakeholder mapping

An important element of any consultation strategy is to identify or map the stakeholder groups to be consulted as illustrated in the figure below.

Figure 1
     Stakeholder Mapping

| |
| --- |

The table below was first introduced as a planning tool in the Inception Report and has been updated to provide information on the actual consultations that took place during the course of this evaluation. It provides details of who was consulted, on what issue, when and how.

Table 1
     Consultation process: Who, what, when and how?

| Who? (Type and group) | What? (Consultation issues) | When? (Stage) | How? (tool) |
| --- | --- | --- | --- |
| | EQ1 | EQ2 | EQ3 | EQ4 | EQ5 | EQ6 | | |
| Commission Services and EEAS |
| DG NEAR | ■ | ■ | ■ | ■ | ■ | ■ | All stages | Interviews |
| DG DEVCO, AGRI, TRADE, CLIMA, ECFIN, HOME, EUROSTAT... | ■ | ■ | | ■ | ■ | ■ | Desk and validation | Interviews |
| ERDF, EIB, EBRD | | | | | ■ | ■ | Validation | Interviews |
| EEAS | ■ | ■ | | ■ | ■ | ■ | All stages | Interviews |
| EU Delegations | ■ | ■ | ■ | ■ | ■ | ■ | Desk and validation | Interviews, EUD survey |
| Development partners |
| Agencies and ministries of EU Member States | | ■ | | | ■ | ■ | All stages | Interviews and OPC |
| Multilateral organisations (e.g. UN, World Bank) | | ■ | | | ■ | ■ | Validation | Interviews |
| Third-country donors (e.g., USAID) | | ■ | | | ■ | ■ | Validation | Interviews |
| Partner countries (national) |
| Aid coordination ministries, ministries involved in informal dialogues (e.g. EaP) | ■ | ■ | ■ | ■ | ■ | ■ | Validation and synthesis | Interviews and OPC |
| CSOs and authorities at national level | ■ | ■ | ■ | ■ | | ■ | Validation and synthesis | Interviews, group discussions and OPC |
| Regional organisations, partnerships, cooperation frameworks or networks |
| UfM, LAs, EaP, ND, BBS... | ■ | ■ | ■ | | | | Synthesis | OPC |
| CSO networks and regional platforms; grouping of LAs at regional level | ■ | ■ | ■ | | | | Synthesis | OPC |
| ENI/ENPI key programmes (only team leaders, for case studies) |
| National programmes | | ■ | ■ | | | | Validation | Interviews |
| Regional programmes (EUROMED projects, ENPARD, CKIS, TRACECA...) | | ■ | ■ | | | | Validation | Interviews |
| Private sector |
| Trade associations | | ■ | | | | ■ | Synthesis | OPC |
| European Chamber of Commerce | | ■ | | | | ■ | Synthesis | OPC |

Who? (Type and group)

What? (Consultation issues)

When? (Stage)

How? (tool)

EQ1

EQ2

EQ3

EQ4

EQ5

EQ6

Commission Services and EEAS

DG NEAR

■

■

■

■

■

■

All stages

Interviews

DG DEVCO, AGRI, TRADE, CLIMA, ECFIN, HOME, EUROSTAT...

■

■

■

■

■

Desk and validation

Interviews

ERDF, EIB, EBRD

■

■

Validation

Interviews

EEAS

■

■

■

■

■

All stages

Interviews

EU Delegations

■

■

■

■

■

■

Desk and validation

Interviews, EUD survey

Development partners

Agencies and ministries of EU Member States

■

■

■

All stages

Interviews and OPC

Multilateral organisations (e.g. UN, World Bank)

■

■

■

Validation

Interviews

Third-country donors (e.g., USAID)

■

■

■

Validation

Interviews

Partner countries (national)

Aid coordination ministries, ministries involved in informal dialogues (e.g. EaP)

■

■

■

■

■

■

Validation and synthesis

Interviews and OPC

CSOs and authorities at national level

■

■

■

■

■

Validation and synthesis

Interviews, group discussions and OPC

Regional organisations, partnerships, cooperation frameworks or networks

UfM, LAs, EaP, ND, BBS...

■

■

■

Synthesis

OPC

CSO networks and regional platforms; grouping of LAs at regional level

■

■

■

Synthesis

OPC

ENI/ENPI key programmes (only team leaders, for case studies)

National programmes

■

■

Validation

Interviews

Regional programmes (EUROMED projects, ENPARD, CKIS, TRACECA...)

■

■

Validation

Interviews

Private sector

Trade associations

■

■

Synthesis

OPC

European Chamber of Commerce

■

■

Synthesis

OPC

Stakeholder consultation strategy

The stakeholder mapping for ENI outlined the main institutions or groups that are considered as ‘stakeholders’. The developed stakeholder consultation strategy aimed at ensuring that the evaluation team could fully engage with all these stakeholders during the evaluation process. An important component of this consultation process was the open public consultation (OPC) done at the end of the synthesis phase of the evaluation to acquire feedback from all relevant parties on the main evaluation findings. Details on the implemented and completed approach are given below.

The approach taken by this evaluation to engaging with the scope of all these aforementioned stakeholders has been defined by their role in ENI and their relative importance and influence over it. The consultation approach for the principal stakeholders identified in the above table has been as follows:

Commission Services and EEAS

The evaluation team closely consulted all the relevant DG NEAR geographical and thematic units throughout the desk and validation phases and informed them of results in the evaluation. Other DGs and entities have been consulted during the desk and validation phases where specific instances required it and informed of evaluation results. EU Delegations have also been consulted and informed throughout all phases of the evaluation.

Development Partners

The development partners active in the Neighbourhood (EU Member States agencies, international organisations) have been consulted in-country in the validation phase as well as desk phase if judged necessary.

Partner countries (national)

Aid coordination ministries and other relevant line ministries of partner countries have been consulted in the case study countries during the validation phase. CSOs also have an active role in ensuring citizens are adequately represented in the formulation of ENI actions and in overseeing as well as implementing them. They have been consulted particularly during the validation phase and the OPC.

Stakeholder consultation framework

Consultation with stakeholders took place via the following means:

·Interviews (face-to-face and via phone) and group discussions with various stakeholders at HQ level as well as via field missions to four case study countries;

·Interviews with EU Delegations in the form of a phone survey (in general targeting the Heads of Cooperation) during the desk phase;

·Instruments-wide survey to EU delegations (coordinated by the chapeau team);

·Open Public Consultation (OPC) via web and face-to-face.

Timeframe

The timeframe for the delivery of the consultation strategy as follows:

| Consultation actions | Deadline |
| --- | --- |
| Desk Phase | Until 10th October 2016 |
| Validation Phase (incl. presentation of preliminary findings) | Until 24th November 2016 |
| Synthesis Phase (pre-OPC) | Until 16th January 2017 |
| Open Public Consultation | From 7th February 2017 until 5th May 2017 |
| Synthesis Phase (post-OPC) | Until mid-June 2017 |

Consultation actions

Deadline

Desk Phase

Until 10th October 2016

Validation Phase (incl. presentation of preliminary findings)

Until 24th November 2016

Synthesis Phase (pre-OPC)

Until 16th January 2017

Open Public Consultation

From 7th February 2017 until 5th May 2017

Synthesis Phase (post-OPC)

Until mid-June 2017

Stakeholder statistics

Interviews took place during the desk phase with all relevant units of DG NEAR, as well as other EU entities and EU Member States. At the same time, interviews with EU Delegations (in general with the Heads of Cooperation) were undertaken in form of a phone survey. During the validation phase, interviews took mainly place within the frame of field visits to four case study countries (Egypt, Georgia, Ukraine and Tunisia). Consultation in the synthesis phase mainly concentrated on the Open Public Consultation and the targeted face-to-face meetings (see separate chapter below) and reverting to previous interview partners for clarification following the comments on the report. During all phases, the evaluation team proactively reached out to the identified stakeholders and made sure that everyone was given an opportunity to provide inputs.

To the possible extent, interviews were structured around interview guidelines shared with the interview partners beforehand and were attended by two (senior) experts of the team. In total, 174 interview partners were consulted, the vast majority of which fit into the category of “EU entity”. The following graphs provide a more detailed overview of the persons interviewed.

Figure 2
     Overview of persons interviewed by category

| |
| --- |

Figure 3
     Overview of persons interviewed by country
  109

| |
| --- |

Open Public Consultation (07/02/17 - 05/05/17)

The Open Public Consultation (OPC) done at the end of the synthesis phase represented an important component of this consultation process. By collecting contributions through a web survey and through technical workshops, it allowed to acquire feedback from all relevant parties on the main evaluation findings. Even though the OPC was also aimed at the broader public and all relevant stakeholders were targeted, particular attention was paid to the contributions of EU Member States.

From the web OPC, a total of 44 contributors answered questions related to the ENI evaluation.

Figure 4
     Type of contributors from the web OPC

| |
| --- |

The very limited number of contributions received doesn't not allow the possibility to come up with relevant conclusions, also because the responses to the questions presented a very polarised perception, confirming the overall mixed picture to which the external evaluation came up with.

The instrument is generally considered as being relevant (with regards to the proximity of the Neighbourhood, human rights and the need to provide a differentiated approach for the partner countries) and as having an added value (in a sense that the EU is seen as a driving factor behind reforms in partner countries and that its assistance adds political weight to specific policies and issues).

Even though coherence and coordination is mentioned positively by some, it is also often raised as an area that still needs improvement, in particular with regards to EU Member States and other donors. Other mainly negative answers (but also several mixed contributions) frequently cite the increasing instability in the Neighbourhood as evidence that ENI has failed to achieve its primary objective of creating an area of “good neighbourliness”. However, some contributors criticize the instrument in particular for focusing too much on short-term stability issues and raise concerns about ENI funding increasingly being channelled to security, at the expense of a more sustainable support of democracy, human rights and civil society.

The mainly positive assessments, which were submitted in majority by public authorities, all praise the incentive-based approach for its results in the Eastern Partnership, underlining the belief that the “more for more” principle has managed to foster major reforms and transformation. Positive results of the mechanism in specific cases (e.g. Georgia, Ukraine, Tunisia) are also often mentioned in mixed answers, most of which highlight the relevance of the principle and the need to reward and encourage countries that have shown goodwill in establishing reforms that mirror EU values.

The contributors of both mixed and mainly negative assessments however call into question the effectiveness of the incentive-based approach: the financial volume is too small to have a real leverage effect (in comparison to the countries’ GDP) and the actual promotion of human rights and democracy has suffered from the application of a more pragmatic approach centred around ownership and common values following the recent crises (in particular related to migration). This is perceived as creating a reward mechanism mainly for “friendly governments” that are willing to go along a specific reform path, rather than offering an incentive to human rights and democracy reforms in the whole Neighbourhood. In this context, a very interesting complementary point has been put forward by one of the contributors: “When partner countries show their goodwill in establishing reforms that mirror EU values, it is only natural that they receive the support that matches their ambition. However, when central governments in partner countries are not complying and not willing to bring positive changes, the budget allocated through the ENI is in theory distributed to NGOs. Yet, EU Delegations find it difficult to identify and involve umbrella organisations that represent civil society at large. In this case, the ones penalised are first and foremost the citizens since they do not benefit from programmes that foster stability.” EU platform, network or associationWith regard to the contribution to the stabilisation process, as one could have expected, most contributors express their opinion that the current political situation in the Neighbourhood and its recent crises (repression and hardening of regimes in the aftermath of the “Arab Spring”, outbreak of conflicts in Libya, Syria and Ukraine) are evidence of the failing attempts of ENI (and to a larger scale EU) to stabilise the region. In the negative and mixed answers, several explanations are given as to why the stabilisation efforts have been unsuccessful; citing a lack of complementarity and flexibility, but also the lack of political will and the difficult situation of civil society in the Neighbourhood countries.

The main topic present in most of the answers is however the balance between short term efforts (crisis prevention/response, security policies) and long-term development. In that respect, some (mainly positive) answers tend to see the ENI’s focus on long-term development as an enabling factor for positive results (for example in Ukraine and Tunisia, but also the CBC Programmes were mentioned as a positive example of a long-term engagement which establishes networks and exchanges across borders, thus stabilising relationships). This view is also shared in some mixed or negative answers, stating that EU/ENI has to concentrate even further on long-term reforms and development in order to truly contribute to the stability of the region. But there are also other voices that identify the insufficient means of crisis prevention as the main problem of EU/ENI efforts of stabilisation. One of the more neutral statements in this respect summarizes the issue: “The key dilemma is to have a field-informed and adaptable policy that allows for cooperation on concrete problems needing responses in the immediate while not neglecting long-term support to regional integration, key for sustainable solutions. The ENI can be tweaked and improved but the EU needs to find a common long-term, strategic and comprehensive vision for the Neighbourhood, taking into account what is feasible, in light of interests, aspirations and opportunities on both sides. Otherwise the responses that ENI can offer are quite limited and their achievements can be quickly undermined by structural challenges and recurrent crises.” Research/academia

Beside the OPC, and besides bilateral meetings organised by the external evaluators in the field with different stakeholders, over 180 participants from the EU Parliament and EU Member States attended a technical workshop organised in Brussels in March. In addition, the draft evaluation report was also presented at the Policy Forum on Development and in the Council’s Working Parties on Eastern Europe and Central Asia (COEST) and Mashreq/Maghreb (MaMa) in April. The main feedback received was the following:

During each meeting, a number of issues were raised with regards to the evaluation findings, but also to ENI in general. Comments were made by different stakeholders from the Council of the European Union, European Parliament, Member States and Civil Society Organisations.

The following bullet points summarise the main issues raised during the face-to-face consultations:

·The principle of differentiation is generally appreciated as means for better and more targeted assistance. Nonetheless, if each country is treated specifically, the overall instrument is put into question;

·The incentive-based approach is an important tool, but clearly has room for improvement to make it more effective, e.g. by revising the allocation criteria;

·The point of the appropriate balance between crisis prevention and long-term development was raised;

·Coordination and coherence between different programmes (bilateral, multilateral) or instruments needs improving and operational linkages and synergies need to be created;

·Cooperation between EU Delegations and Member States needs strengthening, e.g. by applying Joint Programming in a systematic way;

·The lag between programming and implementation;

·The (low) absorption capacity of partner countries is seen as the main obstacle of an effective and fast implementation of the instrument and a key problem in the Neighbourhood;

·Insufficient (time) capacities of EU Delegations staff;

·Visibility aspects of ENI with regards to strategic communication (reaching out in a broader sense than only governments) and with regards to indirect management;

·Added value of Trust Funds over standard ENI programmes;

·Advantages and disadvantages of Budget Support;

·Involvement of CSOs especially with regards to the challenges of reaching out to smaller CSOs.

Annex 4. Acronyms

AA

| AA | Association Agreement |
| --- | --- |
| AFD | Agence Française de Développement |
| AOSD | Authorising Officer by Sub-Delegation |
| AP | Action Plan |
| ATA | Anti-Tax Avoidance Directive |
| BS | Budget Support |
| CBC | Cross-Border Cooperation |
| CC | Climate Change |
| CH | Switzerland |
| ECB | European Central Bank |
| CIR | Common Implementing Regulation |
| COEST | Working Party on Eastern Europe and Central Asia |
| COSCE | Council of Europe |
| CoTE | Centres of Thematic Expertise |
| CRIS | Common Relex Information System |
| CSO | Civil Society Organisation |
| DAC | Development Assistance Committee |
| DCFTA | Deep and Comprehensive Free Trade Areas |
| DCI | Development Co-operation Instrument |
| DG AGRI | Directorate-General for Agriculture and Rural Development |
| DG BUDG | Directorate-General for Budget |
| DG DEVCO | Directorate General for International Co-operation and Development |
| DG ECHO | Directorate-General for European Civil Protection and Humanitarian Aid Operations |
| DG ECFIN | Directorate-General for Economic and Financial Affairs |
| DG ECFIN | Directorate-General for Economic and Financial Affairs |
| DG ELARG | Former Directorate-General for Neighbourhood and Enlargement |
| DG HOME | Directorate-General for Migration and Home Affairs |
| DG NEAR | Directorate-General for Neighbourhood and Enlargement Negotiations |
| DG TAXUD | Directorate-General for Taxation and Customs Union |
| DG TRADE | Directorate-General for Trade |
| EAMR | External Assistance Management Report |
| EAMRs | External Assistance Management Reports |
| EBRD | European Bank for Reconstruction and Development |
| EC | European Commission |
| EDF | European Development Fund |
| EEA | European Economic Area |
| EEAS | European External Action Service |
| EEC | European Economic Community |
| EFI | External Financing Instrument |
| EFTA | European Free Trade Association |
| EIA | Environmental impact assessment |
| EIB | European Investment Bank |
| EIDHR | European Instrument for Democracy and Human Rights |
| EIF | European Investment Fund |
| ENI | European Neighbourhood Instrument |
| ENP | European Neighbourhood Policy |
| ENPI | European Neighbourhood and Partnership Instrument |
| EP | European Parliament |
| EQ | Evaluation Question |
| ERDF | European Regional Development Fund |
| EU | European Union |
| EUD | Delegation of the European Union |
| EUISS | European Union Institute for Security Studies |
| EUR | Euro |
| EU MS | EU Member States |
| EURATOM | European Atomic Energy Community |
| EUTF | EU Trust Fund |
| FAQ | Frequently Asked Questions |
| FEMIP | Facility for Euro-Mediterranean Investment and Partnership |
| FPI | Foreign Policy Instrument |
| FR | Financial Regulation |
| HoD | Head of Delegation |
| HQ | Headquarters |
| HR | Human rights |
| IcSP | Instrument contributing to Stability and Peace |
| IFI | International Financial Institution |
| INGO | International Non-Governmental Organization |
| IO | International Organisation |
| IPA | Instrument for Pre-accession Assistance |
| JC | Judgement Criterion |
| KFW | Kreditanstalt für Wiederaufbau (German development bank) |
| KPI | Key Performance Indicator |
| LAs | Local Authorities |
| LRRD | Linking Relief, Rehabilitation and Development |
| LS | Local Stakeholders |
| MFA | Macro-Financial Assistance |
| MFF | Multiannual Financial Framework |
| MICs | Middle-Income Countries |
| MIP | Multiannual Indicative Programme |
| MIS | Management information System |
| MS | Member State |
| MTR | Mid-Term Review |
| NGOs | Non-Governmental Organisations |
| NIF | Neighbourhood Investment Facility |
| ODA | Official Development Assistance |
| OECD | Organisation for Economic Co-operation and Development |
| PAR | Public Administration Reform |
| PFM | Public Finance Management |
| PPP | Public-Private Partnership |
| PRAG | Practical Guide |
| RACER | Relevance, acceptability, clarity, easiness, robustness indicators |
| RF | Result Framework |
| ROM | Results-Oriented Monitoring |
| SDGs | Sustainable Development Goals |
| SGUA | Support Group for Ukraine |
| SMEs | Small- and Medium-Sized Enterprises |
| SSF | Single Support Framework |
| TEU | Treaty on the European Union |
| TF | Trust Fund |
| TOR | Terms of Reference |
| UFM | Union for the Mediterranean |
| UN | United Nations |
| UK | United Kingdom |
| UNFCCC | Climate Change Framework Convention on Climate Change |
| USAID | United States Agency for International Development |
| VAT | Value-Added Tax |
| VET | Vocational Education and Training |
| WB | World Bank |

Association Agreement

AFD

Agence Française de Développement

AOSD

Authorising Officer by Sub-Delegation

AP

Action Plan

ATA

Anti-Tax Avoidance Directive

BS

Budget Support

CBC

Cross-Border Cooperation

CC

Climate Change

CH

Switzerland

ECB

European Central Bank

CIR

Common Implementing Regulation

COEST

Working Party on Eastern Europe and Central Asia

COSCE

Council of Europe

CoTE

Centres of Thematic Expertise

CRIS

Common Relex Information System

CSO

Civil Society Organisation

DAC

Development Assistance Committee

DCFTA

Deep and Comprehensive Free Trade Areas

DCI

Development Co-operation Instrument

DG AGRI

Directorate-General for Agriculture and Rural Development

DG BUDG

Directorate-General for Budget

DG DEVCO

Directorate General for International Co-operation and Development

DG ECHO

Directorate-General for European Civil Protection and Humanitarian Aid Operations

DG ECFIN

Directorate-General for Economic and Financial Affairs

DG ECFIN

Directorate-General for Economic and Financial Affairs

DG ELARG

Former Directorate-General for Neighbourhood and Enlargement

DG HOME

Directorate-General for Migration and Home Affairs

DG NEAR

Directorate-General for Neighbourhood and Enlargement Negotiations

DG TAXUD

Directorate-General for Taxation and Customs Union

DG TRADE

Directorate-General for Trade

EAMR

External Assistance Management Report

EAMRs

External Assistance Management Reports

EBRD

European Bank for Reconstruction and Development

EC

European Commission

EDF

European Development Fund

EEA

European Economic Area

EEAS

European External Action Service

EEC

European Economic Community

EFI

External Financing Instrument

EFTA

European Free Trade Association

EIA

Environmental impact assessment

EIB

European Investment Bank

EIDHR

European Instrument for Democracy and Human Rights

EIF

European Investment Fund

ENI

European Neighbourhood Instrument

ENP

European Neighbourhood Policy

ENPI

European Neighbourhood and Partnership Instrument

EP

European Parliament

EQ

Evaluation Question

ERDF

European Regional Development Fund

EU

European Union

EUD

Delegation of the European Union

EUISS

European Union Institute for Security Studies

EUR

Euro

EU MS

EU Member States

EURATOM

European Atomic Energy Community

EUTF

EU Trust Fund

FAQ

Frequently Asked Questions

FEMIP

Facility for Euro-Mediterranean Investment and Partnership

FPI

Foreign Policy Instrument

FR

Financial Regulation

HoD

Head of Delegation

HQ

Headquarters

HR

Human rights

IcSP

Instrument contributing to Stability and Peace

IFI

International Financial Institution

INGO

International Non-Governmental Organization

IO

International Organisation

IPA

Instrument for Pre-accession Assistance

JC

Judgement Criterion

KFW

Kreditanstalt für Wiederaufbau (German development bank)

KPI

Key Performance Indicator

LAs

Local Authorities

LRRD

Linking Relief, Rehabilitation and Development

LS

Local Stakeholders

MFA

Macro-Financial Assistance

MFF

Multiannual Financial Framework

MICs

Middle-Income Countries

MIP

Multiannual Indicative Programme

MIS

Management information System

MS

Member State

MTR

Mid-Term Review

NGOs

Non-Governmental Organisations

NIF

Neighbourhood Investment Facility

ODA

Official Development Assistance

OECD

Organisation for Economic Co-operation and Development

PAR

Public Administration Reform

PFM

Public Finance Management

PPP

Public-Private Partnership

PRAG

Practical Guide

RACER

Relevance, acceptability, clarity, easiness, robustness indicators

RF

Result Framework

ROM

Results-Oriented Monitoring

SDGs

Sustainable Development Goals

SGUA

Support Group for Ukraine

SMEs

Small- and Medium-Sized Enterprises

SSF

Single Support Framework

TEU

Treaty on the European Union

TF

Trust Fund

TOR

Terms of Reference

UFM

Union for the Mediterranean

UN

United Nations

UK

United Kingdom

UNFCCC

Climate Change Framework Convention on Climate Change

USAID

United States Agency for International Development

VAT

Value-Added Tax

VET

Vocational Education and Training

WB

World Bank

Annex 5. External evaluators' report, including its annexes and methods and analytical models used

"The external evaluation can be found here: https://ec.europa.eu/europeaid/public-consultation-external-financing-instruments-european-union_en"

Regulation (EU) No 232/2014 of the European Parliament and of the Council of 11 March 2014, OJ L77, p. 27

Art. 1.1 of the ENI.

11th European Development Fund (EDF), European Neighbourhood Instrument (ENI), European Instrument for Democracy and Human Rights (EIDHR), Greenland Decision (GD), Instrument contributing to Stability and Peace (IcSP), Instrument for Pre-accession Assistance (IPA II), Instrument on Nuclear Safety Cooperation (INSC), Overseas Countries and Territories Decision (OCT), Partnership Instrument (PI) and the Common Implementing Regulation (CIR). For the purpose of this exercise, the evaluation of the Overseas Countries and Territories Decision is included within the evaluation of the 11th European Development Fund.

As requested in Article 17 of the Common Implementing Regulation: Regulation (EU) No 236/2014 of the European Parliament and of the Council of 11 March 2014, OJ L77, p. 95

Regulation (EC) No 1638/2006 of the European Parliament and of the Council of 24 October 2006 laying down general provisions establishing a European Neighbourhood and Partnership Instrument (ENPI) of 9 November 2006, OJ L130

They are: Armenia, Azerbaijan, Belarus, Georgia, The Republic of Moldova and Ukraine, for the Neighbourhood East, and Algeria, Egypt, Israel, Jordan, Lebanon, Libya, Morocco, occupied Palestinian territory, Syria, Tunisia, for the Neighbourhood South.

Commission Communication Better regulation for better results – An EU Agenda, COM (2015) 215, and Commission Staff Working Document Better Regulation guidelines, SWD (2015) 111.

Council Regulation (EU, EURATOM) No 1311/2013 of 2 December 2013 laying down the Multiannual Financial Framework for the years 2014-2020, OJ L 347/884, p. 884.

The Multi-annual Financial Framework is divided into six broad groups of expenditure called "Headings". The EFIs make up the majority of Heading IV: Global Europe.

Relations with the government of Syria were suspended in 2011.

https://ec.europa.eu/europeaid/public-consultation-external-financing-instruments-european-union_en

See Annex 3.

See at page 6.

This was largely inevitable, considering the high number of sectors and countries involved, the fact that the programming had not taken place yet, that the amount of resources available was not known yet and that many interventions are of soft nature.

Technical Assistance and Information Exchange programme of the Commission. The programme, initially created for Enlargement countries, provides short term technical assistance coming from EU Member States to partner countries' institutions.

The programme foresees medium-term technical assistance provided by experts coming from EU MS institutions to their homologues in the partner countries.

SIGMA (Support for Improvement in Governance and Management) is a joint initiative of the Organisation for Economic Co-operation and Development and the European Union providing support in the area of public administration, rule of law.

Set up in December 2014 to face the Syrian refugee crisis, with contribution from ENI, Turkey, IPA and several Member States, the fund has already reached more than EUR 1.3 billion. See https://ec.europa.eu/neighbourhood-enlargement/neighbourhood/countries/syria/madad_en.

EU emergency trust fund for Africa for stability and addressing root causes of irregular migration and displaced persons in Africa, set-up end at the Valetta Summit in November 2015

An average number of 150 projects is subject to ROM per year. Related reports are not made public.

See DG NEAR evaluation plan and published evaluation reports on https://ec.europa.eu/neighbourhood-enlargement/news_corner/key-documents_en

This consists of a set of indicators at development, operational and organisational level on which DEVCO is gathering data and reporting since 2016. See https://ec.europa.eu/europeaid/sites/devco/files/swd-2015-80-f1-staff-working-paper-v3-p1-805238_en_0.pdf.

i.e. how it all fits together.

Joint Staff Working Document "Towards a new European Neighbourhood Policy". SWD (2015) 500 final of 18.11.2015

Page. 3 of the evaluation report.

http://europa.eu/globalstrategy/

https://europa.eu/globalstrategy/en/european-security-strategy-secure-europe-better-world

https://ec.europa.eu/neighbourhood-enlargement/sites/near/files/neighbourhood/pdf/key-documents/151118_joint-communication_review-of-the-enp_en.pdf

“An example of this coherence is that EIDHR should focus on civil society action, while geographic instruments should support projects undertaken by national governments”, Evaluation report, vol. II, page 324.

The external evaluation reports also refers to the fact that "there have been cases where ENI country programming does not always reflect sufficiently priorities indicated under other instruments or policy issues of high importance to other line DGs (e.g. the SSFs of Lebanon and Jordan did not consider migration even though it is a focal area of DCI and also several DGs (HOME, ECHO). When the Syrian crisis led to a massive influx of refugees to this country, this lack of consistency became problematic)” Evaluation report, vol. II, par. 5.3.2, page 289.

COM(2011) 303 of 25 May 2011

JOIN (2015) 50 final of 18 November 2015

See evaluation report, page 30.

See Annex 3.

See evaluation report, page 3.

"In total, 12.5% of the total value of ENI commitments were either reallocated or added after the initial allocations had been made in 2014", Evaluation report, vol. 2, par. 12.2.5, page 71.

This principle is clearly stated in the ENP review.

Evaluation report, vol. 3, Face to face consultation.

One of the contributors to the OPC says that "Results are limited, especially as it seems that the ENI was not ready to address major crisis happening relatively suddenly: the instrument, while flexible, lacks in terms of preparation, forecasting and situation assessment", Evaluation report, Annex, 3, Summary of OPC contribution, Question 1.

Examples of these indicators are those related to the business environment (Doing Business), political stability, rule law, regulatory quality, press freedom, mentioned in the table below,

Which is still under implementation, and for which no ex-post evaluation has taken place.

With regard to Georgia, the PEFA report issued in 2013 (concluded that "Georgia has advanced significantly its budgetary and financial managements systems since the previous PEFA assessment Report of 2008" (page 8), getting an A rating on 65% of sub areas assessed by the report, being very highly rated on areas like transparency and budget classification. In the case of Morocco, the comparison between the 2009 and 2016 PEFA demonstrated an improvement in many areas, with a 17% increase of indicators with an A or B score.

The special report of the European Court of Auditors on Tunisia (n. 3, 2017 confirms that "the money was generally well spent as it contributed significantly to the democratic transition and the economic stability of Tunisia after the revolution". It also stated that "EU actions were well coordinated with the main donors and within the EU institutions and departments, but joint programming with the Member States was not realised." (page 6).

The special report of Ukraine (n. 32, 2016) concludes that in the wake of the Maidan events in 2014 "EU assistance to Ukraine has been partially effective in supporting the transformation of Ukraine into a well governed state" (page 7). Overall, the report recognises the capacity of the EU to substantially increase the level of financial support to the country, also thanks to budget support and macro-financial assistance, while improving the policy dialogue. Steps forward were taken in critical/strategic areas, like public administration reform and energy, though "the economic and geopolitical challenges faced by Ukraine heavily affect government priorities and the activities of the public services" (page 14).

In visited stable countries (Georgia, Morocco), "their macro fiscal balances have been stable, governance indicators improving in time together with/and growth outcomes (employment and leaving standards) paving the way for further improvement of EG in general. Nevertheless, in countries in transition or affected from external shocks (Tunisia, Ukraine, Egypt and somehow Jordan), the interventions have been partially effective with the best positive outcomes in governance matters and mixed (positive and negative) outcomes in employment and unemployment levels" (page 11).

Evaluation of the EU's cooperation with the Hashemite Kingdom of Jordan, February 2015, available at https://ec.europa.eu/europeaid/sites/devco/files/strategic-evaluation-cooperation-ec-jordan-1340-main-report-201502_en.pdf

Ex-post Evaluation of 2007-2013 ENPI CBC Programmes; draft Desk report; July 2017.

These reports remain internal to the Commission.

Six out of 12 countries report improvement in the rule of law and better governance (Algeria, Jordan, Ukraine, Morocco, Georgia and Armenia), and in the areas of human rights (Algeria, Jordan, Morocco, Palestine, Moldova and Georgia), sub-regional cooperation (Jordan, Belarus, Georgia, Egypt, Moldova and Azerbaijan) and management of mobility of people and of legal migration (Jordan, Ukraine, Morocco, Georgia, Moldova and Azerbaijan). Five countries refer to an improvement in the management of irregular migration (Morocco, Belarus, Georgia, Moldova and Azerbaijan) and only 3 countries see progress in CBC cooperation (Belarus, Moldova and Armenia) – Evaluation report, vol. 2, page 81 and page 347.

Page viii of the evaluation report and Annex 3.

COM 2011 839 (final), 07/12/2011.

Jordan has continued growing at rates above 2% and is substantially decreasing the fiscal deficit, despite the difficult regional environment; IMF expects Tunisia to grow at 2.5% in 2017 and Ukraine at 2% in 2017.

IMF Article IV consultation with Ukraine, April 2017.

Evaluation report, vol. 1, par. 4.3.

SWD (2017)300 of 9.06.2017 https://ec.europa.eu/neighbourhood-enlargement/sites/near/files/eap_20_deliverables_for_2020.pdf

They represent a way by which funds coming from different sources are pooled together and managed in a more flexible way by the Commission, enabled by art. 4.1 of the Common Implementing Regulation.

" The protracted crisis with the steady increase of refugees requires an additional and urgent effort to address the recent developments and especially the growing tensions in the social fabric." Evaluation report, vol. 2, par. 1.2.4.

Page 10 of the Evaluation report.

According to the World Bank Doing Business report the distance to frontier indicator, representing the distance vis-à-vis the best performing country in terms of business environment, declined from 58.64 for the Neighbourhood South in 2010, just before the Arab Spring, to 54.69 in 2016; for the Neighbourhood East, on the contrary, the indicator improved from 59.73 in 2010 to 72.10 in 2016

Indeed, the commitments taken within the Association Agreements led to a gradual reduction of tariff barriers and to substantial investments made in the areas of standards and quality controls. In the period 2010-2014 exports form Neighbourhood South countries to the EU (excluding Syria and Libya) increased from 76 to 95 billion dollars despite the economic crisis (see also Evaluation report, vol. 2, pages 111-112).

Evaluation of the EU's support to two European Neighbourhood Policy regions – June 2013, available at https://ec.europa.eu/europeaid/strategic-evaluation-eu-cooperation-east-and-south-enpi-regions-2004-2010_en

A first EUR 48 million support programme was financed under the ENPI in 2011 and a second one of EU 35 million is tentatively planned for the annual action plan of 2017. Both programmes have a large budget support component.

JC 21, indicator 211, page 79 Volume 2 of the Evaluation report

Indeed, administrative costs represented in 2015 2% of the committed amounts. This percentage becomes 3.1% looking at actual expenditures. Overstretched human resources, on the one side, use of national systems, while using budget support, explain this overall positive figure. See evaluation report, Annex 2, page 141.

Par. 3.1.2 in Evaluation report, vol. 2, elaborates more on the controls performed.

Evaluation report, vol. 2, par. 1.2.2, and vol. 3, OPC, Question 3.

See evaluation report, page 16.

Evaluation report, vol. 3, par. 5.5.1;

See N. Cherif, " Improving Foreign Support to Tunisia’s Civil Society", C·A·Perspectives on Tunisia No. 07-2017.

Ex-post Evaluation of 2007-2013 ENPI CBC Programmes; draft Desk report; July 2017.

Evaluation report, page 16.

See evaluation report, vol. 2, par. 1.2.4, referring to the different special measures taken to address the crisis in Ukraine of the refugee crisis in Syria to support Jordan or Lebanon.

See evaluation report Volume 2, page 158

https://www.eca.europa.eu/en/Pages/DocItem.aspx?did=37235.

Annual allocations per country are a commission decision, as multiannual allocations are indicative and defined before the adoption of the actual annual budget.

https://ec.europa.eu/europeaid/sites/devco/files/swd-2015-80-f1-staff-working-paper-v3-p1-805238_en_0.pdf.

See evaluation report, page 21. See also

  https://ec.europa.eu/europeaid/eu-international-cooperation-and-development-first-report-selected-results-july-2013-june-2014_en

  , for DEVCO report on results including also ENI countries.

According to an analysis made in the framework of the 127 Results Oriented Monitoring reviews made in the region, in 72% of the cases in which the projects were found problematic (meaning unlikely to achieve the expected outcomes), which were 42% of the total) , this was related to the lack of relevant indicators.

Since 2014, ENI demonstrated (in particular in relation to Ukraine) a strong capacity to attract unprecedented envelops within a coordinated policy framework. Since the 2014 crisis, some EUR 8 billion were made available by the EU institutions, in a collective effort coordinated by the Support Group for Ukraine (SGUA).. Evaluation report, Annex 2, page 195.

"The capacity for reconciliation and confidence building (sometimes a long term perspective) is reinforced by the mix of instruments at the disposal of EUDs to answer contradicting needs and priorities. This flexibility of choice between instruments is not significant in financial terms (ENI provides on average some 90% of the resources availed to Neighbourhood partners), but is important to keep a diversified portfolio addressing long/short terms needs and conflicting priorities at country level", Evaluation report, Annex 2, page 195.

"Most significant achievements regarding policy reforms were conveyed by budget support preparation and implementation. BS (state building contracts and sector budget support) tranche indicators provide a sound framework for assessing progress in the reform agenda. EUDs without a BS have fewer opportunities to engage in policy dialogue – mainly during preparation and signature of multi-annual programming or financing agreements." Evaluation report, Annex 2, page 373.

JC 41, Evaluation report page 23

A public authority pointed out that "regardless of its final form, there is a need for a separate financial instrument supporting the implementation of the ENP"; also, an organisation expressed the view that "the ENI has the potential to contribute to stabilisation in the European Neighbourhood" (Evaluation report, vol. 3, page 159).

"The combination of regional, cross-border and Neighbourhood-wide programmes and initiatives, with resources that – though not sufficient compared to the felt needs of national authorities and civil society expectations regarding the EU – are proportionate to the objectives of the actions, and is already an EU added-value in itself.", Evaluation report, Annex 2, page 196.

Blending is a way by which different sources of funding (grants, loans, guarantees), also coming from the private sector, are mobilised to stimulate higher investment funding.

See JC 55, Volume 2 of the Evaluation report and annex 3 on the stakeholders' consultation.

Page 26 Evaluation report.

JC 51, page 26 Evaluation report

Thematic Evaluation of the EU's Support to Private Sector Development in Third Countries – Algeria, Jordan, Morocco and Ukraine:
  https://ec.europa.eu/europeaid/strategic-evaluation-eu-support-private-sector-development-third-countries-2004-2010_en
  ; and 2017 Draft report on Evaluation to Private sector development.

Strategic evaluation of the EU cooperation with the occupied Palestinian Territory and support to the Palestinian people (2008-2013): https://ec.europa.eu/europeaid/strategic-evaluation-eu-cooperation-occupied-palestinian-territory-and-support-palestinian-people_en

JC 52, page 26 Evaluation report

A table listing all the "potential crisis prevention projects" funded by ENI can be found in the evaluation report, Vol 2, p 291-295.

JC 54, pages 26-27 Evaluation report

Page 27, Evaluation report.

Page 33, Evaluation report.

See evaluation report Vol 2 page 208

Page 35, Evaluation report.

Synthesis of Budget Support Evaluations: analysis of the findings, conclusions and recommendations of seven country evaluations of Budget Support – ADE 2014

https://ec.europa.eu/europeaid/sites/devco/files/strategic-evaluation-synthesis-ec-budget-support-1335-main-report-201411_en.pdf
  , Summary of key findings.

JC 63, pages 37-38 Evaluation report

In these countries the African Development Bank has normally been involved in joint budget support operations. See the “Economic Governance” evaluation, Annexes on Morocco and Tunisia - https://ec.europa.eu/neighbourhood-enlargement/sites/near/files/170818_thematic_ev_of_economic_governance_-_final_report.pdf..

It is estimated that 1 euro committed by the EC originated an overall investment of 8.6 euros in 2014 and 6.9 in 2015. See page 38, Evaluation report.

Page 38, Evaluation report.

i.e. how it all fits together

Evaluators will need to look at both the current ENI 2014-2020 and the previous ENPI 2007-2013 to respond to this question. Evaluators should distinguish the findings between the two periods.

Evaluations will need to compare, where possible, information from the current ENI 2014-2020 with the previous ENPI 2007-2013.

See Title 22, item 22-01-04-02 of the latest, 2016 draft budget

  http://eur-lex.europa.eu/budget/data/DB/2016/en/SEC03.pdf

Note the respective mandates of DEVCO and FPI in EIDHR, PI and IcSP instruments

"HQ" stands for stakeholders interviewed at an EU-wide level and mostly refers to Commission staff in Brussels.