COMMISSION STAFF WORKING DOCUMENT Evaluation of the Instrument for Pre-accession Assistance (IPA II) Accompanying the document Report from the Commission to the European Parliament and the Council Mid-term Review Report on the External Financing Instruments
Date: 2017-12-15
Institutions: Directorate-General for International Cooperation and Development, European Commission
Subjects: EU aid, EU financial instrument, European integration, European neighbourhood policy, administrative reform, aid to refugees, economic reform, financial aid, political reform, pre-accession aid

EN 

  EN 

   EUROPEAN COMMISSION  Brussels, 15.12.2017  SWD(2017) 463 final  COMMISSION STAFF WORKING DOCUMENT Evaluation of the Instrument for Pre-accession Assistance (IPA II)   Accompanying the document Report from the Commission to the European Parliament and the Council Mid-term Review Report on the External Financing Instruments {COM(2017) 720 final} - {SWD(2017) 600 final} - {SWD(2017) 601 final} - {SWD(2017) 602 final} - {SWD(2017) 604 final} - {SWD(2017) 605 final} - {SWD(2017) 606 final} - {SWD(2017) 607 final} - {SWD(2017) 608 final} - {SWD(2017) 609 final}  
 
 
Contents 

Executive summary .................................................................................................................... 3 

1. Introduction ............................................................................................................................ 4 

Purpose of the evaluation ....................................................................................................... 4 

Scope of the evaluation .......................................................................................................... 4 

2. Background of the initiative ................................................................................................... 5 

Description of the initiative and its objectives ....................................................................... 5 

Baseline .................................................................................................................................. 5 

3. Method ................................................................................................................................... 6 

4. Implementation state of play .................................................................................................. 8 

5. Answers to the evaluation questions .................................................................................... 10 

6. Conclusions .......................................................................................................................... 19 

Annex 1. Intervention logic ................................................................................................... 22 

Annex 2. Evaluation Questions ............................................................................................. 23 

Annex 3. Procedural information ......................................................................................... 26 

Annex 4. Synopsis report of the stakeholders' consultation ............................................... 31 

3.1 Introduction .................................................................................................................. 31 

3.2 Stakeholder Mapping ................................................................................................... 31 

3.3 Stakeholder Consultation Strategy ............................................................................. 32 

3.4 Stakeholder Consultation Framework ....................................................................... 33 

3.5 Stakeholder statistics .................................................................................................... 33 

General ............................................................................................................................ 34 

Field missions .................................................................................................................. 34 

3.6 OPC Contribution ........................................................................................................ 35 

3.6.1 Introduction ............................................................................................................... 35 

3.6.2 Summary of OPC contributions .............................................................................. 36 

3.6.2.1.1 Question 1: Addressing IPA II objectives ..................................................... 36 

1 

 
 
3.6.2.1.1 Question 2: Addressing inclusive public consultation ................................. 37 

3.6.3.1.1 Question 3: Addressing fight against corruption ......................................... 39 

3.6.4.1.1 Question 4: Any other views on IPA II ......................................................... 40 

3.6.3 Face-to-face consultations ......................................................................................... 41 

3.6.3.1 Summary of contributions ................................................................................. 42 

3.6.3.2 Response of the Commission ............................................................................. 42 

Annex 5. Acronyms ................................................................................................................ 44 

Annex 6. External evaluators' report, including its annexes and methods and analytical 
models used ............................................................................................................................. 48 

2 

 
 
 
 
 
Executive summary 

This Staff Working Document evaluates the Instrument for Pre-Accession Assistance (IPA II) 
at its mid-term, covering the period January 2014 to June 2017. With a budget of EUR 11.7 
billion,  IPA  II  is  the  main  financial  instrument  supporting  the  implementation  of  the  EU's 
enlargement policy in the period 2014-2020. It is one of the External Financing Instruments of 
the  EU  Multiannual  Financial  Framework  2014-2020.  Its  main  objective  is  to  support 
candidate  countries  and  potential  candidates  in  carrying  out  the  reforms  required  for  EU 
membership. 

Though  this  assessment  comes  at  an  early  stage  of  implementation,  both  the  external 
evaluation and the broad consultation process carried out during 14 months confirm that the 
instrument is overall fit for purpose, and that it corresponds to EU priorities and beneficiary 
needs.  

Compared  to  its  predecessor,  IPA  II  has  become  more  strategic.  In  terms  of  priorities,  it  is 
increasingly  focused  on  promoting  key  reforms,  i.e.  the  three  "fundamentals"  of  the 
enlargement strategy (rule of law, public administration reform and economic governance). It 
has also demonstrated its capacity to react in a flexible manner to emerging crisis situations, 
such  as  the refugee crisis or  the floods in  the  Western Balkans, through the mobilisation of 
additional  funding  and  the  funding  of  new  initiatives  (e.g.  the  Regional  Trust  Fund  in 
response to the Syrian crisis or the Facility for Refugees in Turkey).  

The  introduction  of  the  sector  approach,  especially  when  implemented  through  budget 
support,  has  improved  the  strategic  focus  of  IPA  II  and  the  focus  on  reforms,  though  its 
implementation remains uneven across sectors and beneficiaries.  

The  external  evaluation  has  highlighted  a  mixed  feedback  about  indirect  management  with 
beneficiaries:  while  its  impact  in  terms  of  increased  ownership  is  considered  positive,  there 
have  been  poor  contracting  performance  and  longer  delays  of  implementation,  especially  in 
Turkey.  The  Commission's  response  consisted  in  adopting  a  more  cautious  and  tailor-made 
approach in the move to indirect management by beneficiaries.  

IPA  II  has  an  increased  focus  on  performance  and  results,  including  through  a  newly 
established performance framework. However, there is some room for improvement, resulting 
in  particular  from  the  weak  quality  of  indicators,  the  lack  of  a  proper  monitoring  and 
evaluation  frameworks  at  local  level  and  the  insufficient  quality  of  data  collection  in 
beneficiaries.  

Complementarity  of  IPA  II  with  the  actions  of  other  Instruments,  notably  the  European 
Instrument  for  Democracy  and  Human  Rights  (EiDHR)  and  the  Instrument  contributing  to 
Stability  and  Peace  (IcSP),  is  good  but  coordination  during  both  the  planning  and 
programming phase has not been sufficient.  

The findings and conclusions of the evaluation will feed into the reflection on how to improve 
the  implementation  of  IPA  II  for  the  remaining  period  until  2020,  and  on  the  future  set  of 
Instruments for the next Multiannual Financial Framework. 

3 

 
1. Introduction 
Purpose of the evaluation 

This  Staff  Working  Document  presents  an  assessment  of  the  Instrument  for  pre-accession 
assistance1 (IPA II) 2014-2020 at mid-term, and is largely based on an external evaluation, as 
well  as  on  a  broad  consultation  process  with  different  stakeholders.  It  also  sets  out  the 
Commission's  own views about the instrument and about the  conclusions of the evaluations 
report. The aim is to assess whether the IPA II is appropriate and well suited for its designated 
purpose, based on its performance to-date, to deliver on its objectives of supporting candidate 
countries and potential candidates ("beneficiaries") in implementing the political, institutional, 
legal, administrative, social and economic reforms required for future membership2. 

Its  purpose  is  to  inform  future  work  on  the  instrument  and  its  actions.  In  particular,  this 
evaluation, which is  part of a set of  ten evaluations covering all  the EU External Financing 
Instruments3,  informs  the  Mid-Term  Review  Report4,  which  draws  conclusions  across  the 
Instruments. 

This document is largely based on an external evaluation by independent consultants provided 
in Annex 5. 

Scope of the evaluation 

This evaluation covers the period from 1 January 2014 to 1 June 2017. However, due to the 
length  of  the  implementation  cycle  of  IPA  II,  the  availability  of  data  on  results  is  limited. 
Therefore the evaluation also looked at the previous Instrument for Pre-Accession Assistance 
covering  the  period  2007-2013  for  some  of  the  evaluation  criteria  (e.g.  efficiency  and 
effectiveness). 

In order to feed into the Mid-Term Review Report, the evaluation is set at instrument level. 
By  consequence  it  focuses,  to  the  extent  possible,  on  the  IPA  II  Regulation  (e.g.  on  its 
principles and flexibility) rather than the measures that have been put in place to implement 
the instrument. However, some information on programming has also been included to show 
progress in how the instrument has been implemented (see section 4). 

The beneficiaries covered under the evaluation are those eligible under the IPA II Regulation: 
Albania,  Bosnia  and  Herzegovina,  the  former  Yugoslav  Republic  of  Macedonia,  Iceland5, 
Kosovo, Montenegro, Serbia and Turkey. 

1 Regulation (EU) No 231/2014 of the European Parliament and of the Council of 15 March 2014, OJ L77, p. 11. 
2 Articles 1 and 2 of the IPA II regulation 
3 The  Development  Cooperation  Instrument,  the  11th  European  Development  Fund  which  is  outside  of  the  EU  budget,  the 
European Neighbourhood Instrument, the European Instrument for Democracy and Human Rights, the Greenland Decision, 
the  Instrument  contributing  to  Stability  and  Peace,  the  Instrument  for  Pre-accession  Assistance,  the  Instrument  on  Nuclear 
Safety  Cooperation,  the  Overseas  Countries  and  Territories  Decision,  the  Partnership  Instrument  and  the  Common 
Implementing Regulation. For the purpose of this exercise, the evaluation of the Overseas Countries and Territories Decision 
is included within the evaluation of the 11th European Development Fund. 
4 As  provided  for  in  Article  17  of  the  Common  Implementing  Regulation,  Regulation  (EU)  No  236/2014  of  the  European 
Parliament and of the Council of 11 March 2014, OJ L77, p. 95. 
5 Though there was no specific analysis for this country. 
 This designation is without prejudice to positions on status, and is in line with UNSCR 1244/1999 and the ICJ Opinion on 
the Kosovo declaration of independence. 

4 

 
 
 
 
 
 
 
 
 
                                                 
In accordance with the EU Better Regulation requirements6, the following evaluation criteria 
are used: relevance, effectiveness, efficiency, coherence, and added value, complemented by 
impact, sustainability, consistency, complementarity and synergies and leverage. 

2. Background of the initiative 

The  External  Financing  Instruments  make  up  a  major  part  of  the  Multiannual  Financial 
Framework's7 Heading IV "Global Europe"8 which provides the EU with the tools necessary 
to reinforce its role on the world stage and to ensure that it is able to live up to its ambitions in 
promoting its interests and universal values and principles such as democracy, human rights, 
peace, solidarity, stability and poverty reduction and to help safeguard global public goods.  

Adopted  in  early  2014,  the  Instruments  were  designed  to  facilitate  and  support  policy 
implementation, with the intention of remaining relevant for the entire duration of the Multi-
annual Financial Framework, thereby enabling the EU to implement external action policy as 
needed  within  the  defined  principles  and  objectives.  The  budget  of  IPA  II  amounts  to  EUR 
11.7 billion for the period from 2014 to 2020. 

Description of the initiative and its objectives 

IPA  II  assistance  is  provided  to  IPA  II  beneficiaries  in  accordance  with  the  general 
enlargement policy framework.  

Assistance  is  predominantly  focused  on  a  selected  number  of  policy  areas  to  help 
beneficiaries strengthen democratic institutions and the rule of law, reform the judiciary and 
public  administration,  respect  fundamental  rights  and  promote  gender  equality,  tolerance, 
social  inclusion  and  non-discrimination.  It  is  also  aimed  at  enhancing  their  economic  and 
social  development  with  a  view  to  attaining  the  targets  set  in  the  Europe  2020  strategy  for 
smart,  sustainable  and  inclusive  growth.  In  addition,  IPA  II  assistance  supports  the 
progressive alignment of beneficiaries with EU legislation and standards.  

The  reconstructed  IL  (Annex  1)  for  IPA  II  defines  the  impact,  outcomes,  outputs,  activities 
and inputs contributing to or resulting from the implementation of IPA II (through its actions). 
These correspond to the different sequential levels necessary to bring about the desired results 
as defined by the IPA II Regulation. The reconstructed IL takes into account the provisions of 
the  IPA  II  Regulation  and  the  main  policy  orientations  defined  in  the  EU  Enlargement 
strategies adopted in 2013-2015 (focus on fundamentals first in particular). 

Baseline 

As this is a mid-term evaluation, the baseline has been set at January 2014 when IPA II 2014-
2020 was adopted. Therefore the evaluation compares, to the extent possible, the situation on 
1  January  2014  with  the  current  situation.  For  some  evaluation  criteria,  where  data  is 
unavailable for this reference date, earlier baselines have been used, as described later in the 
document  (see  Evaluation  questions  on  effectiveness  and  efficiency),  considering  that  the 
main objectives of the instrument are still in line with those of the IPA Regulation. 

6 Commission Communication Better regulation for better results – An EU Agenda, COM (2015) 215, and Commission Staff 
Working Document Better Regulation guidelines, SWD (2015) 111. 
7 Council  Regulation  (EU,  EURATOM)  No  1311/2013  of  2  December  2013  laying  down  the  Multiannual 
Financial Framework for the years 2014-2020, OJ L 347/884, p. 884. 
8 The Multi-annual Financial Framework is divided into six broad groups of expenditure called "Headings". The 
External Financial Instruments make up the majority of Heading IV: Global Europe. 

5 

 
                                                 
At the beginning of 2014, accession status was quite diverse among IPA II beneficiaries, as 
shown by the table below.  

State of play of the EU enlargement process at the beginning of 2014 

Beneficiary 

Status at beginning of 2014 

Turkey 

Iceland 

Montenegro 

Serbia 

Candidate - Negotiations opened 3/10/2005 

Candidate - Negotiations opened 27/07/2010,  
suspended 5/2013. 

Candidate - Negotiations opened 29/06/2012 

Candidate - Negotiations opened 21/01/2014 

The former Yugoslav Republic of Macedonia  Candidate - Negotiations not yet opened 

Albania 

Bosnia and Herzegovina 

Kosovo 

Potential candidate 

Potential candidate 

Potential candidate 

In 2014, political stability, democracy, the respect of human rights and good governance still 
needed  to  be  strengthened  in  all  beneficiaries.  Weak  public  institutions  and  administrative 
capacity affected negatively the effective and sustainable deployment of EU assistance. The 
level of socio-economic development was well below EU average, with a need for substantial 
investments to align their legislation and institutions to EU standards.  

To  respond  to  these  challenges,  IPA  II  was  simplified,  in  particular  by  removing  barriers 
linked to accession status9 . IPA II was made more strategic and results-oriented through the 
introduction of a longer term planning and sector approach to address key reform challenges 
in a comprehensive way and with an increased focus on results and impact.  

The  IPA  regulation  does  not  include  indicators  at  instrument  level.  Indicators  with  agreed 
targets  are  defined,  for  each  beneficiary  in  the  Indicative  Strategy  Papers,  as  well  as  in 
specific programmes, in particular when implemented through budget support. Indicators are 
linked  to  the  specific  situation  in  each  country  and  progress  can  only  be  assessed  in  these 
sectors  at  country  level.  Therefore  it  is  difficult  to  assess  progress  at  the  level  of  the 
instrument;  assessment  of  progress  is  made  at  country  or  regional  level  for  the  different 
sectors.  

3. Method  

The present assessment is supported by an external evaluation carried out from August 2016 
to  May  2017.  The  external  evaluation  of  IPA  II  was  managed  by  the  Commission  services 
through the following steps: an inception report (which explained how the evaluation design 
would  deliver  the  information  required);  a  desk  report  (providing  initial  responses  to 
evaluation  questions);  field  visits  to  meet  key  interlocutors  to  obtain  first-hand  view  in-
country; a survey to EU delegations covering all instruments; an Open Public Consultation10 
on  the  draft  report  which  comprised  a  12  week  online  survey  and  targeted  meetings  with 
Member States in March 2017; and a final report.  

9 IPA  was  divided  into  five  components,  with  the  two  first  -  on  transition  assistance,  institution  building  and 
cross-border cooperation- open to all beneficiaries, the three others - on regional development, human resources 
development and rural development - only open to candidate countries. 
10 https://ec.europa.eu/europeaid/public-consultation-external-financing-instruments-european-union_en 

6 

 
                                                 
Overall,  the  quality  of  the  collected  evidence  (data,  documentation,  interviews  and  survey 
results) for this evaluation can be assessed as good. However, apart from the limited evidence 
on  implementation,  there  was  a  limited  feedback  from  the  Open  Public  Consultation.  This 
Staff  Working  Document  largely  concurs  with  the  findings  and  conclusions  of  the  external 
evaluation unless said otherwise.  

Organisation 

As part of a wider set of evaluations covering the eight External Financing Instruments (the 
instruments),  the  European  Development  Fund  (EDF)  and  the  Common  Implementing 
Regulation  (CIR),  all  the  evaluations  were  carried  out  in  an  interlinked  and  co-ordinated 
manner. To ensure coherence of the different evaluations, relevant Commission services have 
worked closely together from the beginning of the process in June 2015. All reports prepared 
by  the  external  contractor  were  discussed,  reviewed  and  approved  by  the  dedicated  internal 
discussions set up by the Commission for the IPA II evaluation. 

The  draft  final  report  prepared  by  the  external  contractor  was  placed,  with  the  other 
instruments draft evaluation reports, on the web for the open public consultation (OPC) from 
7  February  to  3  May  2017.  Thirty  contributions  were  received.  During  that  period  targeted 
face  to  face  meetings  were  organised  with  representatives  of  civil  society  organisations 
(CSOs), EU member states and the European Parliament. The external contractor delivered its 
final report at the end of June 2017. 

Evaluation Design 

The intervention logic (IL) forms the backbone of the analytical framework. It is visualised in 
an IL diagram, which brings together the most relevant elements (from inputs to impacts, but 
also assumptions) in a single framework. To take into account the Commission's Enlargement 
Strategy with a focus on 'fundamentals first', a reconstructed IL was prepared by the external 
consultants. 

Covering the different levels of the IL, the evaluation questions (EQ) structure the analysis to 
gather evidence. Each question was structured into judgement criteria (JC) and indicators to 
provide an answer based on a synthesis of evidences.  

Methodology and data 

The EU evaluation criteria (relevance, effectiveness, efficiency, coherence, and added value, 
complemented  by  impact,  sustainability,  consistency,  complementarity  and  synergies  and 
leverage) were applied as an underlying basis. The evaluation questions gave rise to a number 
of judgement criteria and associated indicators. The external evaluation used a mixed methods 
approach blending quantitative and qualitative methods, with a focus on the latter. The main 
analytical  tools  consisted  of  rigorous  assessment  of  documentation  and  consultation  of 
stakeholders (semi-structured interviews, group consultations). The Commission services also 
organised an Open Public Consultation and a survey of beneficiaries and stakeholders in order 
to address key issues. 

For all EQs, data collection included a mixture of desk review of documents, semi-structured 
interviews  in  Brussels  by  phone  or  face-to-face,  as  well  as  field  visits  to  all  seven  IPA  II 
beneficiaries.  

Challenges and limitations 

7 

 
The process of the external evaluation is robust and the evidence reasonably solid. However, 
evaluators were faced with three main challenges: 

-  the  implementation  of  IPA  II  was  still  in  its  early  stages  when  the  collection  of  evidence 
took place. Therefore, effectiveness/sustainability/impact criteria could not be assessed based 
on outcomes at this stage; 

- the timeframe in which the evaluation took place was tight given the requirements imposed 
by  the  Common  Implementing  Regulation.  This  resulted  in  risks  related  to  access  to  data, 
documentation and availability of key respondents; 

- IPA I is still under implementation, and no ex-post evaluation has taken place yet. 

The multiple rounds of comments by Commission services mean that facts were verified and 
this  facilitated  the  cross  verification  of  data  from  multiple  sources.  The  evaluators  were 
provided  access  to  all  relevant  information  and  colleagues  within  the  Commission  services 
and in EU Delegations and Office. The Commission services also ensured that the evaluators 
would  have  access  to  representatives  of  the  beneficiaries,  in  particular  the  National  IPA 
Coordinators (NIPACs).  

Overall,  the  quality  of  the  collected  evidence  (data,  documentation,  interviews  and  survey 
results)  for  this  evaluation  can  be  assessed  as  good.  The  evaluators  demonstrated  a  good 
understanding  of  the  instrument  and  took  a  very  positive  and  open  attitude  throughout  the 
whole process. However, there was limited feedback from the Open Public Consultation. 

As  indicated  by  the  Intervention  Logic,  the  IPA  II  instrument  can  only  be  seen  as  a 
contributing factor towards any results achieved. First, the EU enlargement process is a highly 
ambitious  transformative  agenda.  Second,  many  factors,  both  internal  and  external,  affect 
developments  in  the  beneficiaries.  Ultimately,  it  up  to  the beneficiaries  themselves  to  adopt 
and implement the necessary reforms and policies for meeting the accession criteria. 

4. Implementation state of play 

IPA  II  is  still  at  an  early  stage  of  implementation  as  reform  programmes  are  complex  to 
develop,  and  require  extensive  stakeholder  consultations.  The  first  actions  and  programmes 
started in 2015.  

Financial  assistance  is  planned  and  programmed  on  the  basis  of  indicative  strategy  papers 
setting  out  the  priorities  and  objectives  for  each  beneficiary  and  the  multi-beneficiary 
programmes  for  the  entire  period  up  to  2020 11  and  a  breakdown  of  indicative  financial 
allocations.  The  strategy  papers  define  the  results  to  be  achieved  by  2020  and  the  actions 
necessary  to  reach  them,  as  well  as  indicators  for  monitoring  progress  and  measuring 
performance.  Strategy  papers  were  adopted  in  2014,  and  their  mid-term  revision  has  been 
launched in 2017. Priorities from the strategy papers are translated into detailed actions in the 
annual or multi-annual action programmes. Cross-Border Cooperation Programmes set out the 
priorities for assistance in the area of territorial cooperation between IPA II beneficiaries. 

The implementation of IPA II is based on a sector approach. Reforms are promoted within the 
framework  of  pre-defined  sectors  closely  linked  to  the  enlargement  strategy,  such  as 
democracy and governance, rule of law or growth and competitiveness. At the same time, IPA 
II  assistance  is  tailored  to  the  specific  needs  of  each  individual  beneficiary,  and  takes  into 

11 For Bosnia and Herzegovina only for the 2014-2017 period. 

8 

 
                                                 
account  the  status  of  each  partner  in  the  overall  accession  process.  To  support  structural 
reforms  based  on  clear  strategies,  sector  budget  support  has  been  introduced  as  a  delivery 
modality. Sector reform contracts amounting to EUR 355 million are now in place in Albania, 
Kosovo, Montenegro and Serbia, in the field of public administration and/or public financial 
integrated  border 
management,  employment  and  skills,  anti-corruption, 
management  and  education.  Policies  supported  under  budget  support  must  have  targeted 
results and  a robust  monitoring  and evaluation system  for collecting, storing  and evaluating 
data in place. 

transport, 

More generally, a considerable share of the assistance provided so far, EUR 2.27 billion has 
been  focused  on  addressing  the  "fundamentals  first",  i.e.  strengthening  the  rule  of  law  and 
fundamental  rights,  public  administration  reform  as  well  as  economic  development  and 
competitiveness, in line with the priorities set out in the Commission's enlargement strategy.  

IPA II commitments and payments from 2014-2016 (in million Euro) 

Faced with the most important refugee inflows for decades, the EU reacted swiftly with the 
establishment of the Facility for Refugees in Turkey drawing funds from both Member States 
(EUR 2 billion), as well as from the EU budget – mainly IPA II and humanitarian aid (EUR 1 
billion). The Facility became operational in February 2016 and is designed to ensure that the 
needs  of  3  million  Syrian  refugees  and  host  communities  in  Turkey  are  addressed  in  a 
comprehensive and coordinated manner. Despite the absence of a clear reference to migration 
in the thematic list of priorities in the IPA II Regulation, the mobilisation of IPA II fund is a 
unique example of EU reactiveness and joined-up action. Out of the EUR 3 billion, EUR 2.9 
billion  has  been  allocated,  more  than  EUR  1.6  billion  contracted  and  EUR  811  million 
disbursed at the end of June 2017, with a focus on six priority areas: humanitarian assistance, 
migration  management,  education,  health,  municipal  infrastructure  and  socio-economic 
support. 

In  parallel,  despite  not  being  a  key  focus  of  IPA  II  assistance,  the  EU  provided  significant 
support to the countries along the Western Balkans migration route, in particular Serbia and 

9 

Beneficiary/Programme201420152016201420152016Albania68,788,982,474,286,681,1Bosnia and Herzegovina65,739,75098,884,950,2The former Yugoslav Republic of Macedonia81,767,264,565,852,278Kosovo66,182,173,989,968,358,7Montenegro39,536,435,428,523,620,7Serbia179216,1189,4142,3125,5101,4Turkey620,4626,7631,1488,1735,1926,8Multi-beneficiary programme including other regional and horizontal support348364,2393,6132,8131,2152,7Special measure25TOTAL 1.469,101.521,301.545,401.120,301.307,401.759,60CommitmentsPayments 
 
 
the  former  Yugoslav  Republic  of  Macedonia  to  help  these  countries  to  properly  manage 
migration flows.  

The implementation of financial assistance provided under IPA II is continuously monitored, 
in  particular  through  the  Results-Oriented  Monitoring  (ROM)12 of  projects  and  programmes 
by  external  experts,  in  addition  to  the  internal  monitoring  by  Commission  staff.  IPA  II 
programmes  are  also  subject  to  evaluations  at  policy,  thematic,  as  well  as  at  beneficiary  or 
regional level13.  

Facility for Refugees in Turkey: Amounts committed per priority area 

In addition, joint monitoring committees and sectoral monitoring committees, including joint 
monitoring  committees  for  cross-border  cooperation  programmes,  bring  together  the 
Commission,  beneficiaries  and  stakeholders  several  times  a  year  and  monitor  the 
implementation of financial assistance programmes. 

5. Answers to the evaluation questions 

Evaluation Question 1: Relevance  

1. To what extent do the overall objectives (IPA II Regulation, Article 1 and 2) and the design 
of the IPA II respond to: 

(i)  EU  priorities  and  beneficiary  needs  identified  at  the  time  the  instrument  was  adopted 
(2014)? 

(ii) Current EU priorities and beneficiary needs, given the evolving challenges and priorities 
in the international context (2017)?  

(iii) the need to set out a relevant system of indicators for programming? 
Overall,  as  confirmed  by  the  external  evaluation14,  IPA  II  appeared  fit  for  purpose  at  the 
moment  it  was  set  out.  The  overall  objectives  and  the  design  of  IPA  II  respond  to  EU 
priorities and beneficiary needs. IPA II is in line with the Global Strategy for the EU's Foreign 
and Security Policy, which sets out the EU's core interests and principles for engaging with 
the  wider  world.  In  addition,  IPA  II  supports  the  EU  Enlargement  strategy  and  policy 
frameworks  stipulated  in  the  IPA  II  Regulation.  The  Enlargement  strategy  is  focusing  on 

12 137 projects are subject to a ROM review per year, on average. 
13 See  https://ec.europa.eu/neighbourhood-enlargement/news_corner/key-documents_en  for  the  evaluation  plan 
and published report. 
14 Evaluation report, page i. 

10 

 
 
 
                                                 
those  areas  in  which  all  beneficiaries  face  severe  challenges,  with  a  strong  emphasis  on 
structural reforms as the basis for the accession process.  

The planning of IPA II assistance is reflected in the individual Indicative Strategy Papers that 
ensure a close link between the political priorities and the priorities for financial assistance are 
properly set in these documents. 

Strengthening  the  rule  of  law,  including  the  fight  against  corruption  and  organised  crime, 
good  governance  and  public  administration  reform,  as  well  as  economic  governance  and 
competitiveness are identified in the IPA II Regulation as key challenges in most of the IPA II 
beneficiaries, and have been identified as the three fundamental priorities of the Commission's 
enlargement strategy.15 The need to deal with these “fundamentals first” has been reflected in 
relevant programming documents, which form the basis for the allocation of IPA II funding.  
As  pointed  out  by  the  external  evaluation 16,  clear  links  between  the  actual  stage  of  EU 
accession and the IPA II Indicative Strategy Papers exist. Beneficiary-specific differences are 
clearly  spelled  out  in  the  related  Indicative  Strategy  Papers.  A  move  towards  sector 
approaches  had  already  been  launched  during  the  late  stage  of  IPA  notably  in  the  area  of 
agriculture and rural development, and more clearly under IPA II, though this process is still 
on-going.  The  Commission  services  consider  that,  overall,  the  introduction  of  the  sector 
approach has proved useful, having encouraged a more strategic orientation, also in budgetary 
terms, more sectoral focus and progress in the utilisation of budget support. This is clearly the 
case for example in the area of governance and public administration reform (with reinforced 
policy dialogue and budget support operations in Albania, Kosovo, Montenegro and Serbia). 
This  has  produced  complementary  benefits,  in  relation  to  public  financial  management 
reforms,  increased  policy  dialogue  and  reduced  transaction  costs.  However,  the  external 
evaluation highlights an uneven implementation. In some sectors, the move from the project-
based  programming  approach  towards  true  sector  programming  remains  to  be  completed  as 
many beneficiaries either do not have a proper understanding of the sector approach or the full 
capacities  and  willingness  to  develop  such  strategies 17.  In  this  framework  an  element  of 
insufficient clarity and quality is represented by the Sector Planning Documents, prepared by 
the beneficiaries, in line with the programming guidelines issued by the Directorate General 
for  Neighbourhood  and  Enlargement  Negotiations  of  the  Commission,  but  not  fully 
understood by them, and of uneven quality18.  

IPA  II  builds  directly  on  the  achievements  made  by  the  predecessor  programme  in  many 
cases.  The  Rural  Development  component  of  the  Instrument  for  Pre-accession  Assistance 
(IPARD) and, to a slightly lesser extent, IPA Cross Border Cooperation, demonstrate a strong 
degree  of  continuity19.  Thus,  there  is  good  potential  for  complementing  and  increasing  the 
sustainability  and  impact  of  IPA  I  interventions.  The  management  of  the  former  IPA 
Components  III  (regional  development)  and  IV  (human  resources  development)  has  been 
unified within one Commission department for the current programming period (2014-2020), 
ensuring  also  continuity  in  these  policy  areas.  Also,  efforts  are  made  to  continue  with  the 
achievements  made  under  IPA  I,  e.g.  by  promoting  the  same  or  similar  sector  policy 

15  JC  11,  indicator  111  of  the  Evaluation  report,  page  9.  See  also  https://ec.europa.eu/neighbourhood-
enlargement/countries/package_en for the Enlargement Package. 
16 Evaluation report, page 8. 
17 JC 13, indicators 132-133 of the Evaluation report, pages 9-10. A dedicated strategic evaluation is ongoing to 
better  apprehend  the  limitations  and  perspectives  related  to  the  implementation  of  sector  approaches  in  the 
region. 
18 Evaluation report, page 11. 
19 JC 13, indicator 133 of the Evaluation report, page 10. 

11 

 
                                                 
objectives  or  to  continue  with  local  beneficiary  institutions  that  developed  their  capacities 
under IPA I.  

The overall objectives and the design of IPA II have been able to respond to a large extent to 
evolving  challenges  and  priorities.  The  instrument  has  demonstrated  a  high  degree  of 
flexibility in cases of pressing emerging needs, such as the reaction to the 2014 floods in the 
Western  Balkans  and  the  refugee  crisis.  Examples  include  the  introduction  of  special 
measures  (10  in  total  between  2014  and  2017) 20  to  address  the  consequences  of  the 
devastating floods in the Western Balkans in 2014, as well as the setting up of the Facility for 
Refugees  in  Turkey  in  201521,  and  the  EU  Regional  Trust  Fund  in  response  to  the  Syrian 
Crisis (Madad fund)22. 

The  relevance  of  the  instrument  and  of  some  of  the  changes  introduced  were  overall 
acknowledged  by  the  stakeholders consulted,  both  during the  face  to  face  interviews  and  in 
the Open public consultation23. By contrast, stakeholders criticised above all the insufficient 
involvement of non-state actors in the design and implementation of actions24.  

Evaluation Question 2:  Effectiveness, impact, sustainability 

To what extent does IPA deliver results as compared to the instrument's objectives, and specific EU 
priorities?  
Though  no  definitive  conclusions  can  be  drawn  at  this  early  stage  of  implementation,  some 
positive  elements  seem  to  confirm  the  effectiveness  and  impact.  The  external  evaluation 
points to a higher concentration of funds in priority areas, in line with programme objectives 
and  the  Commission's  enlargement  strategy25,  in  particular  in  the  fields  of  political  reforms 
and economic, social and territorial development, which constitute around three quarters of all 
IPA II allocations. A substantial portion of the IPA II funds is devoted to institution building 
in the sectors of democracy and governance, and rule of law and fundamental rights, in line 
with  political  priorities.  For  example,  comprehensive  sector  budget  support  programmes  in 
the area of public administration reform were launched for Albania, Kosovo, Montenegro and 
Serbia and some of them are already being implemented. 

Another key element is the existence of mechanisms to support the programming and delivery 
of  assistance.  The  introduction  of  budget  support  offers  considerable  potential  for  a  more 
efficient and effective delivery of IPA II assistance and its focus on results. Budget support 
explicitly  addresses  key  elements  of  political  reforms  and  demands  a  complex  set  of 
preconditions  to  be  met.  Its  introduction  has  generated  greater  policy  dialogue  not  only 
between the Commission and IPA II beneficiaries26, but also among beneficiaries’ institutions 
involved in its delivery. This requires significant administrative and/ or legislative changes as 
well as governance reforms, a macroeconomic stabilisation programme and a credible public 

the  Republic  of  Serbia  and  Turkey:   

20 See for instance the Commission Implementing Decision C(2014)9797 final of 17.12.2014 adopting a special 
measure  on  flood  recovery  and  flood  risk  management  in  Albania,  Bosnia  and  Herzegovina,  the  former 
Yugoslav  Republic  of  Macedonia,  Kosovo*,  Montenegro, 
https://ec.europa.eu/neighbourhood-enlargement/sites/near/files/pdf/financial_assistance/ipa/2014/multi-
country/ipa_ii_2014-037-703_2014-037-788_2014-37-853_flood_recovery_and_risk_management.pdf 
21 Commission Decision C(2015)9500 of 24.11.2015 on the coordination of the actions of the Union and of the 
Member  States  through  a  coordination  mechanism  –  the  Refugee  Facility  for  Turkey,  as  amended  by  the 
Commission Decision C(2016)855 of 10.2.2016.  
22  Commission  Decision  C(2015)9691  final  of  21.12.2015  amending  Decision  C(2014)9615  final  on 
establishment of a European Union Regional Trust Fund in response to the Syrian crisis, "the Madad Fund". 
23 Evaluation report, vol. 1, Executive Summary, and vol. 2, par. 6.2.1 
24 Evaluation report, vol. 2, page 248 and 250. 
25 JC 21, indicators 211.1 and 211.2 (page 12), indicator 215.1 (page 14) of the Evaluation report.  
26 As evidenced by the intensification of the meetings of the special groups on Public Administration Reform.  

12 

 
                                                 
finance  management  reform  programme.  All  of  these  effects  can  be  observed  in  those  IPA 
beneficiaries  that  have  now  introduced  budget  support  and  they  can  be  considered  a  major 
indirect effect of the instrument27 in the Western Balkans. So far, Turkey has not considered 
appropriate to use the budget support modality. 

The sector approach has generally facilitated improved thematic coherence mostly evident in 
the areas of public administration reform and public financial management. It demands a new 
way  of  thinking  among  the  institutions  responsible  for  its  programming  and  delivery. 
Although the sector approach is still in a transition stage as regards its full implementation in 
all  beneficiaries  and  faced  resistance  at  the  beginning, evidence  confirms  that  this  approach 
has indeed moved IPA II towards a greater focus on reforms by concentrating funding on key 
sector  reform  priorities28.  The  sector  approach  has  changed  the  approach  of  many  of  those 
involved in programming in the Commission services, National IPA Coordinators (NIPACs) 
and sector lead institutions away from the project-based, input-output philosophy, very much 
focused on the harmonisation of EU legislation ("acquis") that often prevailed under IPA I29.  

It is difficult to assess the direct effects resulting from IPA II support. This is due to the fact 
that much of the assistance under the IPA II has only started to be implemented or is still in 
the  contracting  stage.  Likewise,  the  impact  of  IPA  II  at  this  stage  is  not  evident  and  any 
assessment of impact prospects would  be unduly speculative. A key condition for impact is 
the existence of strong policy dialogue between the EU and IPA II beneficiaries, particularly 
as regards political reforms. This is evident in most IPA II beneficiaries, in particular where 
budget support has been introduced30, but less so in Turkey where policy dialogue has been 
more challenging in the last few years. Regarding multi-beneficiary programmes, outputs and 
some preliminary outcomes have already been achieved because they started early, as they do 
not require the signature of a financing agreement with beneficiaries. This is for example the 
case of the programmes for public administration reforms (e.g. improved legislation on public 
procurement, civil service, administrative procedures, public internal financial control through 
the  support  of  the  OECD/SIGMA  programme)  or  public  financial  management  (e.g.  review 
and improvement of public investment management through the programme implemented by 
the International Monetary Fund). 

The  Commission  services  have  promoted  a  high  level  of  stakeholder  involvement  in  the 
programming  of  IPA  II.  The  extent  to  which  this  has  happened  in  practice  remains  uneven 
and  significant  differences  among  beneficiaries  exist  in  the  quality  of  the  programming 
documents.  Several stakeholders have criticised what they see as a lack of involvement and 
support  to  both  non-state  actors  and  local  authorities 31 .  Insufficient  involvement  of 
stakeholders is also seen as a way that prevents a successful fight against corruption, which is 
considered not adequately addressed32. The modest involvement of stakeholders may have a 
negative effect on impact and sustainability.  

Finally, Cross Border Cooperation (CBC) under IPA II has remained largely unchanged and 
is perceived in its focus and basic functioning. Also, CBC is recognised as being a complex 
instrument33 requiring continuity.  

27 JC 21, indicator 211.2 (page 12) of the Evaluation report. 
28 page 11 of the Evaluation report. 
29 A specific evaluation on sector approaches has been launched in 2017 by DG NEAR and is at the desk phase. 
30 page 11 of the Evaluation report, JC 21.  
31 Evaluation report, vol. 2, page 255. 
32 Evaluation report, vol. 2, page 252. 
33 JC 21.6, indicators 216.1 and 216.2 page 14 of the Evaluation report 

13 

 
                                                 
Evaluation Question 3: Efficiency 

To what extent is IPA delivering efficiently?  

Considering  the  changes  under  IPA  II  and  the  short  period  of  actual  implementation,  it  is 
difficult to come up with conclusive evidence on efficiency. The cost-effectiveness of IPA II 
is confirmed by an administrative cost ratio in a range of 3-5% for commitments made by the 
Commission. Where continuity from IPA I to IPA II has been fully ensured, as in IPARD34, 
efficiency  gains  are  more  evident.  The  available  indicators 35  suggest  progress  in  the 
Commission’s administrative efficiency as well when comparing 2014 with 2015. 

Overall, sound financial management has improved between 2014 and 2015 for most of the 
beneficiaries. The EU Delegations in Enlargement countries have performed well and without 
major differences between them. 

However,  the  operational  efficiency  is  difficult  to  assess  properly  at  this  stage.  There  is  a 
widespread  belief  among  the  stakeholders  that  time  lags  between  programming  and 
implementation  are  too  high 36 .  Operational  efficiency  is  also  related  to  the  applied 
management modes. Here a direct comparison with IPA is difficult as indirect management is 
no  longer  the  rule  under  IPA  II.  IPA  II  allows,  in  principle,  more  flexibility  in  applying 
different management modes. At the same time, compared to the IPA period, the number of 
beneficiaries operating  at  least partially under the indirect management  with  the beneficiary 
country (IMBC) mode has increased to five.  

In  recent  years,  the  stricter  conditions  set  by  the  Commission  in  order  for  beneficiaries  to 
move to an indirect management mode (notably in terms of management and control systems, 
frequency  of  controls)  have  slowed  down  implementation  of  the  programmes  in  those 
beneficiaries. For many policy areas, inefficiencies in the IMBC have generated chronic and 
cumulative delays (in particular in the former Yugoslav Republic of Macedonia and Turkey). 
This  is  adversely  affecting  IPA  II  programmes  and  overall  budget  execution,  notably  in 
Turkey,  where  the  majority  of  funding  is  implemented  through  IMBC  and  only  a  small 
percentage of funds have been contracted yet under the annual programmes 2014, 2015 and 
201637.  

For  the  indirect  management  mode  to  function  more  effectively,  significantly  greater 
resources are required (in particular in terms of institution building, as shown by IPARD)38. 
There  is  currently  no  reliable  data  available  to  assess  the  cost-effectiveness  of  the  indirect 
management  mode  applied  by  the  IPA  II  beneficiaries.  Lack  of  transparency  with  regard  to 
absorption  capacity  of  IPA  funds,  partly  related  to  visibility,  was  pointed  out  both  in  the 
OPC39 and in the specific consultation with stakeholders40. 

34 The rural development component managed, by DG AGRI. See Evaluation report, vol. 2, page 3 " As IPARD 
money  can  only  flow  after  management  powers  have  actually  been  conferred,  the  absorption  rate  has  initially 
been low. However, as management for some measures has now been conferred for all of the three beneficiary 
countries, the overall uptake of IPARD funds is moving in an upward direction.” 
35 JC 31, indicators 311, 312, page 19 of the Evaluation report 
3636 See evaluation report, vol. 2, page 255, 
37 Nonetheless,  available  statistics  do  not  show  a  significantly  different  path  in  terms  of  delays  of  payments 
between the two management modes (evaluation report, vol. 2, pages 114-115). 
38 page 18 of the Evaluation report and JC 32, indicators 323 and 333. 
39 Evaluation report, vol. 2, page 248. 
40 Evaluation report, vol. 2, page 255. 

14 

 
                                                 
In  terms  of  coherence  between  strategies,  instruments  and  procedures,  efficiency  is 
improving41. Improvements in the programming documents and a better appreciation of what 
the  sector  approach  looks  like  in  practice  are  materialising42.  With  the  introduction  of  the 
sector-based  approach,  a  major  shift  in  thinking  for  all  stakeholders  has  been  initiated 
although  some  line  ministries  among  beneficiaries  still  have  difficulties  to  fully  grasp  and 
apply the new approach43. In-house expertise was strengthened with the creation of Centres of 
Thematic  Expertise  which  enhance  coherence  in  formulating  sector  and  thematic  IPA  II 
interventions,  drawing  on  the  lessons  of  past  projects  and  best  practices  from  other 
beneficiaries/regions.  In  addition,  increased  attention  has  been  given  by  the  Commission, 
notably through a revised internal quality review process, to screening relevance and maturity 
of  proposals,  with  those  not  meeting  the  necessary  requirements  being  deferred  to  later 
programming years.  

The  use  of  sector  budget  support  has  been  actively  promoted  by  the  Commission  and  is 
expanding  across  the  Western  Balkans.  Four  beneficiaries  are  now  involved  in  such 
operations.  With  budget  support,  much  less  resources  are  required  for  the  contracting  and 
managing  of  contracts  compared  to  classic  technical  assistance  projects,  but  additional 
resources  need  to  be  allocated  to  policy  analysis  and  dialogue  with  the  beneficiaries  both 
during the programming and implementation phases. Also, there are first indications that the 
introduction of this modality in  itself is  likely  to improve the quality of the policy dialogue 
(e.g.  in  Albania,  Serbia,  Kosovo)44.  Intensified  policy  dialogue  works  best  with  the  various 
administrations  involved  in  the  preparation  of  the  support  –  as  effective  cooperation  and 
communication  is  a  requirement  already  during  the  planning  phase  –  but  also  between  the 
Commission and the national stakeholders. 

When an agreement on budget support is made, other positive processes take place: increased 
willingness to assess and follow-up closely public financial management issues, to engage in 
more in-depth dialogue on macroeconomic issues (in close link with the EU-supported efforts 
to  improve  economic  governance  through  Economic  Reform  Programmes)  but  also 
implementation  of  prior  actions  before  the  signature  of  the  Financing  Agreement.  Budget 
support  has  also  prompted  the  preparation  of  sector  reform  progress  reports  by  the 
government  (e.g.  in  Serbia  and  Albania),  demonstrating  increased  capacity  in  the  area  of 
monitoring  and  evaluation,  creating  space  for  more  informed  policy  dialogue  between 
stakeholders and constituting a tool for improving accountability and transparency45. 

A major potential driver for increased efficiency can be seen in the monitoring processes for 
the measurement of IPA II performance which are being put in place both in the Commission 
and  in  beneficiaries.  The  setting  up  of  IPA  II  Sector  Monitoring  Committees  in  the 
beneficiaries  involves  national  stakeholders  to  a  varying  extent,  in  line  with  the  dominant 
implementation modalities in the particular beneficiary. A set of common strategic indicators 
has been defined, though not all beneficiaries have agreed to define targets46. The assessment 

41 page 20 of the Evaluation report. 
42 This appeared also from comments in the framework of the OPC (evaluation report, vol. 2, page 250). 
43 page 18 and page 20-21 JC 33, indicator 332 of the Evaluation report 
44 page 19 of the Evaluation report. Moreover, “with the introduction of the modality of Sector Budget Support 
(SBS) (the Albanian authorities) have stopped requesting more indirect management, because the SBS under the 
sectoral approach provides more flexibility to the National Authorities to manage the EU funded interventions in 
a more coordinated way, servicing the National goals” (evaluation report, vol. 2, page 67. 
45 Evaluation report, pages iii-iv. 
46 These indicators address the impact dimension and are based on international sources, like World Bank, World 
Economic  Forum,  Transparency  International,  etc.  They  address  the  key  areas  of  support  (like  Rule  of  law, 
public administration reforms, business environment, etc. The beneficiaries had been requested to set out targets 

15 

 
                                                 
 
of  ‘operational’  performance  is  based  on  indicators  of  outputs  and  immediate  outcomes  put 
forward  in  the  action  documents.  The  quality  of  action  documents  and  the  indicators 
(especially at outcome level) contained therein still need improvement. The Commission has 
developed internal Guidelines on linking planning/programming, monitoring and evaluation47, 
which  are  meant  to  help  all  involved  parties  in  the  preparation  of  action  documents.  In 
addition,  the  Commission  has  organised  capacity  building  activities  both  for  Commission 
staff and the beneficiaries, and it has revised the in-house quality review process to increase 
scrutiny and the focus on results.  

Evaluation Question 4: Added value 

To what extent do the IPA II actions add value compared to interventions by Member States 
or other key donors?  

IPA II added-value is reflected in the IPA II Regulation through a reference to the principle of 
subsidiarity,  as  set  out  in  Article  5  of  the  Treaty  on  the  European  Union.  Overall,  the 
Commission  support  is  consistently  aligned  with  actions  by  other  donors,  including  EU 
Member  States  and  non-EU  donors,  usually  with  the  EU  representing  the  most  significant 
provider of grant funding. 

For EU Member States representatives and stakeholders in the beneficiaries, the EU’s unique 
supranational  nature  is  a  key  element  of  added  value.  Its  political  influence  and  leverage 
allows engagement with authorities or other donors with greater authority and legal certainty 
than individual EU Member States.48 
The  intense  use  of  both  the  Twinning 49 and  TAIEX 50 initiatives,  where  valued  European 
public  expertise  is  made  available  to  beneficiaries,  confirms  the  added  value  of  bringing 
particular EU Member States expertise into the IPA II beneficiaries to address specific needs. 
The  fostering  of  long-term  relations  with  a  similar  institution  in  an  EU  Member  State  is  an 
intangible benefit explicitly ascribed to Twinning.  

IPA  II  contributes  to  other  donors'  actions  only  to  a  limited  extent,  reflecting  the  lower 
engagement  of  other  donors  in  the  Western  Balkans  and  Turkey.  IPA  II  national  funds  are 
used  for  some  multi-donor  funds,  both  for  longer-term  programmes  and  for  emergency 
response actions. The dominant multi donor fund in IPA II is the Western Balkans Investment 
Framework  (WBIF),  which  provides  financing  and  technical  assistance  to  strategic 
investments  in  the  energy,  environment,  social,  and  transport  sectors,  and  also  supports 
private  sector  development  initiatives.  With  regard  to  added  value,  it  is  clear,  for  example, 
that  without  this  facility,  which  represents  a  relevant  and  important  response  to  the 

representing  the  level  of  progress  expected  in  the  different  areas.  On  01/09/2015  the  Turkish  authorities 
confirmed  the  intention  not  to  come  up  with  targets  with  reference  to  some  indicators  (like  the  one  on  Press 
Freedom). 
47

 https://myintracomm.ec.europa.eu/dg/near/whatwedo/monitoring-reporting-evaluation/Pages/Monitoring-

evaluation.aspx 
48 Evaluation report, vol. 2, page 166. 
49 Twinning is a specific tool set out in the Enlargement area to support beneficiary administrations by means of 
(https://ec.europa.eu/neighbourhood-
experts 
enlargement/tenders/twinning_en). 
50 Technical Assistance and Information Exchange instrument of the European Commission  

the  European 

administrations 

coming 

public 

from 

16 

 
 
                                                                                                                                                         
development needs of the Western Balkans region,51 the majority of funded projects would be 
difficult, or impossible, to finance solely from EU Member States' funds.  

It is also unique to the EU to actively promote territorial cooperation such as through IPA II 
Cross-Border  programmes,  transnational  and  interregional  cooperation  programmes,  as  well 
as  macro-regional  strategies.  The  added  value  is  obvious:  reconciliation  and  confidence 
building  in  the  Western  Balkans,  overcoming  of  geographical  and  mental  barriers  and 
developing  of  good  neighbourly  relations  –  all  these  remain  key  aspects  in  the  enlargement 
process, addressed solely by EU programmes and not by other donors52. 

Coordination and division of labour as promoted in the aid effectiveness agenda show mixed 
achievements in IPA II beneficiaries despite the fact that it has been strongly supported by the 
EU53. The departure of most international and bilateral donors and the dominating role of EU 
grant funding mean that division of labour is de facto increasingly seen as less of an issue. 

The  EU  Regional  Trust  Fund  in  Response  to  the  Syrian  Crisis  (Madad  fund)  is  a  good 
example of the EU’s capacity to mobilise large amounts of funding from different sources in 
response to an emerging crisis. The contributions, including from IPA II, to the Madad fund 
are  significant  and  allow  policy  dialogue  at a  higher  level  than  EU  Member  States  bilateral 
interventions.  The  Madad  fund  provides  also  an  illustration  for  effective  joint  programming 
undertaken by the EU and other donors, including EU Member States. 

Evaluation Question 5: Coherence, consistency, complementarity and synergies  

To what extent does IPA II facilitate coherence, consistency, complementarity and synergies 
both internally between its own set of objectives and actions and vis-à-vis other EFIs?  

As  the  main  EU  funding  instrument  for  pre-accession  support,  IPA  II  finances  actions 
required not only for aligning with the EU acquis, but also to promote the financial, social and 
institutional  development  of  beneficiaries.  However,  the  overall  available  funds  of  the 
Instrument are small compared to the actual needs required to comply with the obligations of 
EU membership54.  

Besides IPA II, the European Instrument for Democracy and Human Rights (EIDHR) and the 
Instrument  contributing  to  Stability  and  Peace  (IcSP)  also  provide  grants  for  the 
implementation of specific actions in the Western Balkans and Turkey. Although IPA II and 
the  EIDHR  provide  support  to  local  civil  society  organisations,  for  instance  through  a 
dedicated Civil Society Facility, many stakeholders consulted (notably from CSOs) during the 
Open Public Consultation and face to face interviews, noted the limited support of IPA II to 
civil  society  and  local  governments  and  the  need  to  provide  more  support  to  civil  society 
actors in beneficiaries55. 

IPA II procedures for planning, programming and implementation of programmes and actions 
foresee the coordination and the stimulation of synergies with the above mentioned thematic 

51  page  23  of  Evaluation  report,  and  Evaluation  of  the  Western  Balkans  Investment  Framework  (2015)  . 
https://ec.europa.eu/neighbourhood-
enlargement/sites/near/files/pdf/financial_assistance/phare/evaluation/2015/2014_352812_1_final_evaluation_re
port.pdf 
52 Evaluation report, vol. 2, page 77. 
53 Chapter 3.4.2, pages 24-25 of the Evaluation Report 
54 page 25 of the Evaluation report 
55 Volume 2, annex 6, page 253-254 of the Evaluation report 

17 

 
 
                                                 
Instruments,  but  there  is  still  room  for  improvement 56.  Coordination  of  the  ad-hoc  (non-
programmable)  actions  of  the  EIDHR  and  the  IcSP  with  similar  actions  of  IPA  II  requires 
programme  or  task  managers  in  EU  delegations  to  take  the  initiative.  When  this  is  not 
happening, there is a risk of duplication or a lack of flexible response to an emerging need57. 

Financial  support  provided  under  IPA  II  contributes  to  the  long-term  objectives  of  EU 
accession by funding the social development of the beneficiaries58 and urgent needs stemming 
from  emergencies  such  as  natural  disasters  (e  g.  the  floods  which  occurred  in  Serbia  and 
Bosnia and Herzegovina in 2014) or social problems (e.g. linked to the recent refugees crisis 
in 2015 and 2016) also through loans, co-financed where appropriate with loans provided by 
the International Financial Institutions. 

The  EIB  is  a  key  financing  partner  of  IPA  II  beneficiaries,  providing  its  banking  products 
(mainly loans and guarantees) either alone or in the framework of wider financing Facilities 
created together with other International Financial Institutions and Funds. One example is the 
Western  Balkan  Investment  Framework  (WBIF).  The  EBRD  and  the  Council  of  Europe 
Development Bank (CEB), although less active than the EIB, play a similar role. 

In  addition  to  the  bilateral  and  multi-country  programmes  and  actions,  IPA  II  also  finances 
cross-border  cooperation  (CBC)  programmes  implemented  with  EU  Member  States,  with 
neighbouring  beneficiaries  and  between  IPA  II  beneficiaries  and  contributes  to  the 
participation  of  the  IPA  II  beneficiaries  in  the  transnational  and  interregional  cooperation 
programmes  of  the  European  Regional  Development  Fund  (ERDF).  It  also  supports  actions 
implemented through macro-regional strategy frameworks. Its contribution complements the 
financing of the ERDF.  

Other donors active in the beneficiaries include a number of EU Member States, International 
Organisations  and  International  Financial  Institutions.  There  are  continuous  efforts  from  all 
donors,  as  well  as  beneficiaries  to  coordinate  activities  and  to  programme  and  implement 
actions in a way to further strengthen complementarity and synergies. Donor coordination in 
most of the IPA II beneficiaries is adequate59. It is ensured by a high-level National Authority, 
supported  by  the  EU  Delegation/Office  on  the  basis  of  the  National  Development  Strategy 
and  its  in-built  sectoral  priorities.  Donor  coordination  still  needs  to  be  improved  in  some 
beneficiaries (e.g. in Kosovo). As a consequence, there is also a considerable risk of overlaps 
and gaps60. 

Evaluation question 6: Leverage 

To what extent has IPA leveraged further funds and/or political or policy engagement?  

IPA  II  is  used  pro-actively  for  supporting  negotiations  with  the  governments  of  the 
beneficiaries for taking the necessary measures leading to reforms, although this applies more 
to  the  Western  Balkans  than  Turkey.  This  includes  rewards,  but  also  the  decommitment  of 
funds,  together  with  a  number  of  other  measures.  IPA  II  has  contributed  to  an  increasing 
political  and policy engagement by the respective beneficiaries,  in particular in the Western 
Balkans.  

56 page 26 and  JC 52 page 28 of the Evaluation report 
57 page 26 of the Evaluation report 
58 For example, the regional housing programme financed by the multi beneficiary programme 
59 page 25 and JC 54 of the Evaluation report 
60 page 25 of the Evaluation report 

18 

 
 
                                                 
The  policy  discussion  platforms  created  for  the  coordination  of  the  policies  of  the 
Commission,  the  EU  Member  States  and  other  donors  have  contributed  to  the  coordination 
and promotion of common policies and to the creation of greater policy leverage. 

IPA II has also managed to create considerable financial leverage. As the biggest donor in the 
beneficiaries, IPA II funds offer a reliable financial basis to leverage funds from other donors. 
Financial  leverage  is  mainly  ensured  by  special  financing  vehicles  which  pool  all  these 
different financing sources. Such vehicles are the WBIF61, EDIF62, GGF63, but also the Madad 
fund and the Facility for Refugees in Turkey. 

The  amounts  already  mobilised  by  other  donors  to  co-finance  IPA  II  projects  are 
considerable64. It was not possible to collect detailed data on the overall financial leverage in 
the  framework  of  this  evaluation,  however,  it  has  been  estimated  that:  (1)  under  IPARD  I, 
EUR  1.00  from  the  EU  has  attracted  at  least  EUR  1.60  from  other  sources;  (2)  to  date,  the 
WBIF has allocated EUR 1.3 billion to the Western Balkans, out of which EUR 473 million 
represent  technical  assistance  and  investment  grants  aimed  at  leveraging  EUR  13.5  billion  
investments65. The major part of these investment grants comes from IPA funds (the rest from 
EBRD,  EIB,  CEB  and  19  EU  Member  States).  The  leverage  of  the  grants  provided  is  very 
significant.  EU  Member  States  have  been  very  active  and  participated  in  many  of  these 
facilities and funds. 

6. Conclusions 

IPA  II  (2014-2020)  is  the  main  financial  instrument  supporting  the  implementation  of  the 
EU's enlargement policy.  

It  is  yet  difficult  to  assess  with  absolute  certainty  the  effectiveness  of  the  instrument  as 
implementation started only in 2015 and most programmes adopted since then require time to 
be rolled out, let alone to produce outputs and outcomes. 

The external evaluation finds that IPA II is overall relevant and well suited for its designated 
purpose. Its overall objectives and design are in line with EU priorities and beneficiary needs. 
The  instrument  has  also,  compared  to  IPA  I,  a  stronger  focus  on  key  reforms  required  to 
ensure that candidate countries and potential candidates successfully meet the EU accession 
criteria.  In  addition,  IPA  II  is  more  strategic  and  results-oriented,  and  has  allowed  greater 
leverage of other donors' funds. 

In  parallel  to  this  focus  on  long  term  reforms,  the  instrument  has  also  demonstrated  some 
flexibility to face emerging priorities. Since 2015, the EU has provided significant support to 
the  countries  along  the  Western  Balkans  migration  route  to  help  them  properly  manage  the 
influx  of  refugees  trying  to  reach  the  EU.  The  most  important  response  was  in  relation  to 
refugees in Turkey; the EU established the Facility for Refugees in Turkey which draws funds 
from  both  Member  States  (EUR  2  billion),  and  from  the  EU  budget  –  mainly  IPA  II  and 
humanitarian aid (EUR 1 billion). 

The introduction of the sector approach to programming has improved the strategic focus of 
IPA  II  over  its  predecessor.  This  more  conceptual  approach  to  programming,  prioritising 
sector  level  change  over  projects  is  already  observable  in  the  Commission  and  some 

61 Western Balkan Investment Framework 
62 Enterprise Development & Innovation Facility  
63 Green for Growth Fund 
64 JC 63, indicator 631 
65 Evaluation report, vol. 1, par. 3.6. 

19 

 
                                                 
Institutions  in  beneficiaries.  However,  the  implementation  of  the  sector  approach  remains 
uneven  across  sectors  and  many  beneficiaries  have  difficulties  in  the  proper  uptake  of  the 
approach,  as  well  as  in  understanding  the  proper  meaning  of  sector  planning  documents,  as 
revealed by the external evaluation.  

In support of the sector approach, the Commission has introduced sector budget support as a 
key  delivery  modality.  The  strict  eligibility  criteria  linked  to  this  approach  have  pushed  the 
IPA  beneficiaries  to  embark  on  new  reforms  in  important  areas  of  governance  such  as  the 
strengthening  of  democratic  institutions,  the  rule  of  law,  public  financial  management  and 
macroeconomic  stabilisation  (in  close  link  with  the  EU-supported  efforts  to  improve 
economic  governance  through  Economic  Reform  Programmes).  The  intensive  engagement 
between  the  Commission  and  IPA  II  beneficiaries  on  the  introduction  of  this  complex 
mechanism has enhanced the quality of policy dialogue between the Commission and IPA II 
counterparts.  

Evidence from IPA I shows that the introduction of the Decentralised Implementation System 
/  Indirect  Management  with  the  beneficiary  country  (IMBC)  has  a  mixed  impact.  It  offers 
improved  ownership  of  the  programme  among  beneficiaries,  but  in  most  cases  overall 
efficiency  suffers  compared  to  the  direct  management  mode.  In  some  beneficiaries,  notably 
Turkey,  inefficiencies  in  the  IMBC  are  evident  for  many  policy  areas  and  have  generated 
chronic  delays  that  have  accumulated  and  led  to  the  decommitment  of  funds.  The 
Commission takes now a much more prudent approach in the framework of IPA II, compared 
to the predecessor instrument, in view of the identified weaknesses. 

The  monitoring  and  evaluation  system  embedded  in  the  IPA  Regulation,  which  involves 
setting up a performance framework, is promising, but the monitoring and evaluation systems 
at  national  level  and  many  indicators  used,  especially  at  outcome  level,  are  still  weak.  This 
implies the need of further strengthening statistical capacities, further clarification with regard 
to the functioning of the monitoring system, and a solid system of data gathering, encoding 
and analysis. In this context, the Commission is developing a new information management 
system (OPSYS), which will improve the data collection processes and hence the reporting on 
results.  

Complementarity of IPA II with the actions of other Instruments, mainly the EIDHR and the 
IcSP is good, but coordination and cooperation during the planning and programming phases 
remains uneven. 

Donor  coordination  is  largely  satisfactory  in  most  of  the  IPA  II  beneficiaries.  The  EU  is 
promoting the coordination of donors to be fulfilled by a National Authority. IPA II is making 
considerable  efforts  in  strengthening  public  consultation  with  the  civil  society  and  other 
actors,  in  particular  at  the  planning  and  programming  stages.  However,  the  process  differs 
widely  among  beneficiaries  and  sectors,  and  the  involvement  of  civil  society  organisations 
during the implementation of IPA II projects remains limited. 

20 

 
 
 
 
21 

 
 
 
 
Annex 1. Intervention logic 

22 

 
 
 
 
 
 
 
 
 
 
 
 
Annex 2. Evaluation Questions 

Relevance  

1.  To  what  extent  do  the  overall  objectives  (IPA  II  Regulation,  Article  1  and  2)  and  the 
design66 of the IPA II respond to: 

(i)  EU  priorities  and  beneficiary  needs  identified  at  the  time  the  instrument  was  adopted 
(2014)? 

(ii) Current EU priorities and beneficiary needs, given the evolving challenges and priorities 
in the international context (2017)?  

(iii) the need to set out a relevant system of indicators for programming? 

Information sought in this area includes: 

 

 A timeline showing congruence/divergence of the instrument against an evolving context, 
including global challenges, and institutional policy changes e.g. to what extent does IPA 
II respond to the changes in the larger neighbourhood. 

  To  what  extent  has  the  new  programming  reflected  the  recent  shift  of  the  enlargement 
strategy towards the three fundamental pillars (i.e. rule of law, economic governance and 
competitiveness and public administration reform), in terms of priorities and modalities of 
intervention and planning of reforms (like Economic Reform Programmes)?  

Effectiveness, impact, sustainability 

2.  To  what  extent  does  IPA  deliver  results  as  compared  to  the  instrument's  objectives,  and 
specific EU priorities?67 

Information sought in this area includes: 

  To what extent do IPA  programmes contribute towards: 

o  Support for political reforms; 
o  Support  for  economic,  social  and  territorial  development,  with  a  view  to  smart, 

sustainable and inclusive growth; 

o  Strengthening  of  the  ability  of  the  beneficiaries  at  all  levels  to  fulfil  the  obligations 

stemming from Union membership; 

o  Strengthening  regional  integration  and  territorial  cooperation  (IPA  II  regulation 

Article 1 and 2). 
o  Institution building 
o  Improving cooperation and good neighbourly relations among CBC partners  

  To  what  extent  has  IPA  contributed  to  the  European  Union's  priorities  for  smart, 

sustainable and inclusive growth? 

  To what extent does IPA II mainstream EU policy priorities (e.g. gender, climate change) 
and  other  issues  highlighted  for  mainstreaming  in  the  instrument,  and,  where  relevant, 
deliver  on  the  commitments  including  the  financial  allocations  (IPA  II  Regulation, 
preamble)? 

  To what extent does IPA II promote principles of aid effectiveness, such as coordinating  

assistance, partnership and ownership (IPA II Regulation, Preamble and Article 5)?  

66 i.e. how it all fits together 
67 Evaluators  will  need  to  look  at  both  the  current  IPA  II  2014-2020  and  the  previous  IPA  I  2007-2013  to 
respond to this question. Evaluators should distinguish the findings between the two periods. 

23 

 
                                                 
  To what extent are the processes conducive to programming, identification/formulation of 

effective actions (IPA II Regulation, Article 6-15)?   

  How have the criteria for the performance reward been designed under IPA II?   

  To  what  extent  is  IPA  II  flexible  enough  to  respond  to  changing  needs  (e.g.  changed 

policy priorities, changed contexts)? 

Efficiency 
3. To what extent is IPA delivering efficiently?68 

Information sought in this area includes: 

  What is the ratio of administrative costs (as defined as “IPA II Support Expenditure” in 

the Draft General Budget of the EU69) to overall budget? 

  How efficient is budget execution in terms of time taken from commitments to payments? 

  Have  the  changes  made  to  IPA  II  2014  –  2020  from  the  previous  IPA  I  2007  –  2013 
brought  efficiency  gains?  E.g.  Has  the  introduction  of  sector  approach  and  new  aid 
modalities brought positive change in terms of efficiency of delivery? 

  Are  there  areas,  such  as  administrative/management  procedures,  where  IPA  II  could  be 

simplified to eliminate unnecessary burden? 

  To what extent is IPA II in line with the implementing rules of the CIR? Specifically in 

terms of :  

o  Implementation 

  Subject matter and principles 
  Adoption of action programmes, individual measures and special measures 
  Support measures 

o  Provisions on the Financing Methods 
  General financing provisions 
  Taxes duties and charges 
  Specific financing provisions 
  Protection of the financial interests of the Union 

o  Rules  on  nationality  and  origin  for  public  procurement,  grant  and  other  award 

procedures 

o  Climate action and biodiversity expenditure 
o  Involvement of stakeholders of beneficiary countries 
o  Common rules 

  Eligibility under IPA II 
o  Monitoring and evaluation of actions 

  To what extent are the following in place and functioning: 

o  appropriate  monitoring    processes  and  indicators    for  measurement  of  the 

performance of the IPA II instrument 

o  relevant  strategic  and  operational  indicators    to  measure  results  achieved  by  IPA 

II? 

68 Evaluations  will  need  to  compare,  where  possible,  information  from  the  current  IPA  II  2014-2020  with  the 
previous IPA I 2007-2013. 
69

the 

latest, 

2016 

draft 

budget 

http://eur-

Title 

 See 
of 
item 
lex.europa.eu/budget/data/DB/2016/en/SEC03.pdf 

22-01-04-01 

22, 

24 

 
                                                 
Added value 

4.  To what extent do the IPA II programmes add value compared to interventions by Member 
States or other key donors? 

Information sought in this area includes: 

  Where IPA II is operating in the same field as other donors, does it offer added value in 
terms of size of engagement, particular expertise, and/or particular weight in advocacy? 

Coherence, consistency, complementarity and synergies  

5.  To  what  extent  does  IPA  II  facilitate  coherence,  consistency,  complementarity  and 
synergies  both  internally  between  its  own  set  of  objectives  and  programmes  and  vis-à-vis 
other EFIs? 

Information sought in this area includes: 

  To what extent are the different IPA II programmes coherent/complementing/overlapping 
with one another, including coherence between bilateral and multi-country programmes? 
  To what extent are the different IPA II programmes aligned with EU enlargement policy? 
  To what extent are the programmes consistent with EU external action policies? 
  To  what  extent  do  the  programmes  complement/overlap/stimulate  synergies  with  other 

 

external action financing instruments? 
 To  what  extent  does  IPA  II  complement/overlap  with  other  EU  instruments  outside  of 
development  and  other  external  policies,  notably  the  EU  territorial  cooperation 
programmes and the regional macro strategies? 

  To what extent does the IPA II complement/overlap with interventions of other donors? 
 

 To  what  extent  policy  dialogue  and  implementation  of  financial  cooperation  act  in  a 
synergetic way? 

Leverage 

6. To what extent has IPA leveraged  

- further funds,  and/or 
-  political or policy engagement?  

7.  How  could  IPA  be  enhanced  to  achieve  its  policy  objectives  more  effectively  and 
efficiently?  

8. How can programming and implementation of IPA assistance be enhanced to improve the 
impact and sustainability of financial assistance? 

25 

 
 
 
 
 
Annex 3. Procedural information 
Organisation  
This evaluation assessed the Instrument for Pre-accession Assistance (IPA II)70 ahead of the 
Mid-Term Review Report, as set out in Article 17 of the Common Implementing Regulation 
(CIR 2014). It will mainly be used to generate information for the Mid-Term Review Report 
requested  by  the  CIR  due  end  2017.  The  evaluation  provides  information  on  relevance,  EU 
added  value,  coherence  and  complementarity,  effectiveness,  efficiency,  sustainability, 
leverage and impact of the instrument.  
The  Staff  working  document  is  based  on  the  independent  assessment  carried  out  by  an 
external contractor and complemented by further internal analysis.   
The  lead  DG  to  carry  out  and  manage  this  evaluation  was  the  Directorate  General  for 
Neighbourhood and Enlargement Negotiations (DG NEAR). But as this evaluation was part 
of  a  wider  set  of  evaluations  covering  the  instruments  under  Heading  4  of  the  Multiannual 
Financial Framework 2014-2020 as set out in the CIR, as well as the performance review of 
the  European  Development  Fund,  and  in  view  of  ensuring  a  consistent  European  external 
policy,  all  the  evaluations  were  carried  out  in  an  interlinked  and  co-ordinated  manner.  To 
ensure the coherence of the different evaluations from the beginning, the Directorates General 
in charge of external actions in the Commission worked closely together. 
An  Umbrella  Inter-service  Steering  group  (ISG)  was  set  up  to  oversee  the  mid-term  review 
process  of  the  instruments.  This  ISG  is  chaired  by  Directorate  General  for  International 
Cooperation  and  Development  and  is  composed  of  members  from  relevant  Commission 
services. In addition ISGs were set up for each individual evaluation. The process started in 
June 2015 and the Umbrella ISG met formally for the first time on 21 September 2015 and 
during the following weeks and months agreed on the joint elements which all the different 
evaluation should use in their specific roadmaps and Terms of reference. 
The ISG for the IPA II evaluation was set up in October 2015, including relevant services of 
the Commission, and met for the first time on 15 October 2015. The roadmap for IPA II was 
published, like all the other instruments roadmaps, at the end of October 2015, no comments 
were  received.    The  Terms  of  reference  were  approved  by  the  ISG  on  15  April  2016.    A 
specific  contract  was  awarded  on  9  June  2016  to  PARTICIP  GmbH,  using  the  framework 
contract  set  up  for  carrying  out  evaluations  in  the  context  of  external  relations,  and  the 
evaluators started their work on 28 June 2016. 
As part of their work the evaluators produced an Inception Report, a desk report and a draft 
Report.    All  these  reports  were  presented to  the  ISG  meeting.    ISG  members  then provided 
written comments to the evaluation manager in DG NEAR, who combined all the comments 
and communicated them to the evaluators.  Revised versions were checked by the evaluation 
manager and further comments from the ISG were requested where necessary.  Several ISG 
members  also  requested  comments  from  their  directorates/units  where  this  was  deemed 
necessary.  During the evaluation process from October 2015 onwards the ISG met 7 times, in 
addition the members were consulted through email several times on draft and revised draft 
reports.  
Following the approval of the Draft Report by the ISG after the meeting of 11 January 2017, 
this  report  was  then  placed,  with  the  other  instruments  draft  reports,  on  the  Commission's 
website  for  the  web  based  open  public  consultation  (OPC)  on  7  February  2017.    The  OPC 
concluded  on  3  May  2017.    During  the  OPC  period  targeted  face  to  face  meetings  were 

70 Regulation (EU) No 231/2014 of the European Parliament and of the Council of 15 March 2014, OJ L77. 

26 

 
                                                 
organised with representatives of CSOs, both from the beneficiary countries and Europe, with 
representatives of EU Member States and European Parliament. 
Following  the  closure  of  the  OPC  during  which  only  30  responses  were  received,  the 
evaluators then submitted a Final Draft Evaluation Report on 15 May, this was discussed by 
the ISG on 18 May leading to the issuing of the final report.  
Evaluation Design  
Concerning  the  evaluation  design,  the  intervention  logic  (IL)  forms  the  backbone  of  the 
analytical framework. It is visualised through an overall IL diagram, which brings together the 
most relevant elements (from inputs to impacts, but also assumptions) in a single framework.  
The critical analysis of the (reconstructed) IL helped to focus the evaluative framework (EQs, 
JCs  and  indicators)  on  key  issues,  particularly  the  underlying  theory  of  change  and  the 
assumptions that ensure the operationalisation of the regulatory provisions.  
Evaluation questions, which were very similar to all the simultaneously on-going instruments 
evaluation,  structure  the  analysis  to  gather  evidence.  The  evaluation  questions,  as  well  as 
more  instrument-specific  sub-questions,  were  provided  in  the  Terms  of  Reference. 
Considering the objective of the various instruments evaluations launched under the mid-term 
review, consistency among the respective EQ/JC structure was prioritised up to the JC level. 
Each question was structured into judgement criteria (JC) and indicators required to provide 
an answer based on a synthesis of evidences.  
Methodology and data 
The  evaluation  of  IPA  II  is  evidence-based.  The  EU  evaluation  criteria  (relevance, 
effectiveness,  efficiency,  added  value  of  the  EU,  coherence,  consistency,  complementarity 
and  synergies,  as  well  as  leverage)  were  applied  as  an  underlying  basis.  The  evaluation 
questions  (EQs)  from  the  Terms  of  Reference  gave  rise  to  a  number  of  JCs  and  associated 
indicators. The evaluation is indicator-based. The evaluation used a mixed methods approach 
blending quantitative and qualitative methods, with a focus on the latter. The main analytical 
tools consisted of rigorous assessment of documentation and consultation of stakeholders (via 
interviews, group consultations, and the Open Public Consultation- OPC; key issues were also 
been addressed by a joint-instruments survey managed by one of the contractors in charge of 
preparing an overall coherence report).   

For  all  EQs,  data  collection  included  various  tools  and  methods.  The  combination  of  data 
collection  methods  and  techniques  varied  according  to  the  different  EQs  and  JCs.  Several 
methods and techniques were used to collect the necessary data to assess a given JC according 
to  the  nature  of  the  set  of  indicators  identified.  Where  possible,  the  Evaluation  Team 
combined  the  use  of  qualitative  and  quantitative  data  and  relied  both  on  primary  and 
secondary data sources while taking into account resource and time constraints. Priority was 
given to document review and interviews at HQ, in line with the methodological indications 
in the ToR. Data collection activities were carried out mainly during the desk phase, but also 
continued during subsequent phases. 

The following analytical tools were applied in order to reply to the respective EQs:  

EQ1 on relevance included an assessment whether 1) IPA II did (or did not) correspond to the 
objectives of the EU Enlargement strategy; and whether 2) IPA II did (or did not) correspond 
to the needs of beneficiary countries related to their stated EU priorities, given also the recent 
fundamental shift in the way how the programming of IPA II is conducted since 2014 (sector-
based approach). 

27 

 
 
 
 
 
EQ2  on  effectiveness  examined  the  extent  to  which  the  conditions  were  in  place  for  the 
achievement of programme objectives once implementation moves forward.  

EQ3 on efficiency was also pitched at the instrument level and assessed the relative costs and 
outcomes  of  the  current  programming  period.  It  also  looked  at  process  efficiency  at  the 
implementation  level,  including  whether  the  recently  enforced  management  and  monitoring 
arrangements  are  designed  and  applied  in  a  manner  which  ensures  increased  results 
orientation and allows for improved performance measurement. 

EQ4 on value added was closely related to the idea of comparative advantage, particularly as 
regards  EU  MS.  Value  addition  was  also  assumed  to  derive  from  complementarity  and 
synergies with other donors, which were also assessed under EQ5. 

EQ5  on  the  issues  of  coherence, consistency, complementarity  and  synergy  was  assessed  at 
two  levels:  the  conceptual  level  (scope  and  rationale  of  the  instrument  versus  other 
instruments  and  EU  policies)  and  the  operational  level  (how  is  coherence,  consistency, 
complementarity and synergy ensured at programming and implementation level). 

EQ6 on leverage was assessed using two parameters: 1) IPA II financial assistance has been 
used for increasing political and policy engagement of the beneficiaries towards reforms but 
also in terms of better political coordination with EU MS and other donors; 2) the extent to 
which IPA II is able to leverage additional financial resources. 

The evaluation of IPA II comprised three phases: 

Presentation of the inception report to the EU in Brussels. 

Kick  off  meeting  and  consultation  with  the  EU  in  Brussels  and  consultations  with 

Desk phase: 
Inception: 
• 
other stakeholders, particularly in DG NEAR. 
• 
Desk: 
• 
Identifying  and  gathering  information  at  the  indicator  level.  Documents  reviewed 
included  policy  and  strategy  papers,  programme  documentation  from  the  CRIS/  MIS 
database, evaluations, studies and assessments. In addition to the document review, interviews 
were  conducted  by  phone  or  in  person  with  staff  members  from  the  European  Commission 
and  EU  Member  States  to  verify  information,  obtain  leads  to  documents/new  interviewees, 
and to discuss the Intervention Logic (IL). 
• 
the JCs were not revised.  
• 
finding and needs of the evaluation.  Elaborating pathways of change for further testing.  
• 
Conducting  the  first  step  of  the  IL  analysis  by  verifying  whether:  1)  the  intended 
activities  under  the  programme  are  actually  implemented;  2)  the  IL,  as  depicted  in  the 
inception report, is agreed to by consulted stakeholders.  
• 

Refining  the  methods  of  analysis  of  the  data  collected,  according  to  the  preliminary 

Reviewing the EQs/ JCs to assess whether or not a revision was needed: The EQs and 

Elaboration of a detailed validation phase work plan and a list of people to interview.  

Validation phase: 
• 
etc. 
• 

Further  interviews  with  staff  available  in  Brussels  for  DG NEAR,  DEVCO,  HOME, 

Interviews (phone) with EU MS members of the IPA II Committee.  

28 

 
 
 
 
 
 
 
 
Targeted  compilation  of  documentation  directly  related  to  each  JC,  in  particular 

Further  systematic  analysis  and  extractions  from  the  documentation  to  fully  answer 

Mission  to  all  seven  IPA  II  countries  to  conduct  semi-structured  interviews  and/  or 

• 
JCs and indicators.  
• 
documents collected during the field visits. 
• 
group discussions. 
Drafting a separate CIR input according to the guidelines provided by DG DEVCO. 
• 
Based on the success of data collection, assessing whether there was need for further 
• 
research  and  interviews  to  prepare  the  draft  final  report,  and  in  particular  the  conclusions 
chapter. 

Synthesis phase: 
• 
Preparing 
recommendations. 
• 

the  (draft)  final  evaluation  report, 

together  with  conclusions  and 

Implementation of the Open Public Consultation. Results feed into the final report. 

The main purpose of the field missions was to complete the data collection and to contribute 
to  answering  the  EQs.  The  visits  served  to  validate  or  revise  the  preliminary  findings  and 
hypotheses formulated during the desk phase and were in particular an opportunity to obtain 
more  information  on  the  actual  status  of  implementation  on  the  ground.  The  field  missions 
were conducted as follows: 

What  

When  

Who  

Mission to Albania 

Mission to Kosovo 

30/10 to 05/11 2016 
(half) 

30/10 to 05/11 2016 
(half) 

Mission to Serbia 

31/10 to 02/11 2016 

Mission to the 
former Yugoslav 
Republic of 
Macedonia 

Mission to 
Montenegro 

14/11 to 16/11 2016 

28/11 to 30/11 2016 

Mission to Turkey 

28/11 to 01/12 2016 

Mission to Bosnia 
and Herzegovina 

04/12 to 08/12 2016 

EUD/O senior management (Heads 
of Delegation/Office, Heads of 
Cooperation, Head of Operations) 
Key authorities (NIPAC, NAO, 
etc.) 
At least one key line ministry/ 
sector lead ministry 
International donors active in the 
country (e.g. UNDP) 
Bilateral donors/ Member State 
representatives in the beneficiary 
At least one key CSO in the 
beneficiary 

Issues  related  to  the  CIR  were  assessed  in  a  common  way  across  all  external  financing 
instrument evaluations. A questionnaire was developed and shared with the evaluation team, 
which  was  answered  to  feed  into  the  CIR  evaluation.  Besides  the  review  of  documentation 
and  interviews,  a  common  survey  was  conducted  for  the  different  instruments  evaluations. 
The purpose was to gather feedback from the EUDs on specific aspects of the CIR and on the 
coherence and complementarity of the instruments. All collected information was analysed in 
order to prepare a consolidated response to the CIR questionnaire. 

29 

 
 
 
 
 
 
Challenges and limitations 
The main challenge of the evaluation was the very tight timeline dictated by the CIR, which 
resulted in risks related to access to data, documentation and availability of key respondents. 
The  external  evaluation  team  counterbalanced  the  risks  by  being  highly  flexible  especially 
when  gathering  information.  Countering  the  risks  demanded  considerable  efforts  from  the 
external evaluation team, but the team managed to mitigate potential limitations by mobilising 
resources to carry out all tasks indicated in the methodology in parallel and analyse additional 
sources  of  information  when  available.  Ultimately  these  challenges  did  not  compromise  the 
foundations of the evaluation findings. The quality of the collected evidence (documentation, 
interviews,  data  and  survey  results)  for  this  evaluation  can  be  overall  assessed  as  good, 
demonstrating  a  satisfactory  degree  of  confidence  regarding  the  various  findings  of  this 
evaluation. 
The key limitation, which is inherent to any mid-term review, is that the outputs (i.e. IPA II 
funded programmes) of the Instrument have only recently entered the implementation stage. 
Therefore, effectiveness/sustainability/impact criteria could not be assessed based on results at 
this stage. The external evaluation team was, however, able to gain a first impression of these 
criteria  based  on  the  quality  of  the  action  documents  and,  for  the  five  case  studies,  the 
congruence of the analysis of the context and the EU response strategy. 
The  ISG  as  well  as  other  staff  from  DG  NEAR,  EEAS  and  IPA  II  delegations  actively 
participated  in  providing  missing  data  to  the  external  evaluators.    The  multiple  rounds  of 
commenting also mean that facts have been verified and have facilitated the triangulation of 
data. 
Overall, the quality of the collected evidence (documentation, interviews and survey results) 
for this evaluation can be assessed as good, demonstrating a satisfactory degree of confidence 
regarding the various findings of this evaluation. 

30 

 
 
 
 
Annex 4. Synopsis report of the stakeholders' consultation 

3.1 Introduction 
The  evaluation  of  IPA  II,  together  with  the  other  independent  evaluations  of  each  External 
Financing  Instrument,  that  of  the  Common  Implementing  Regulation  (CIR)  and  the 
Coherence Report, are some of the sources of information to feed into the Mid Term Review 
Report  (MTR)  of  the  instruments.  The  MTR  is  required  by  the  Common  Implementing 
Regulation (CIR) Article 17, by end December 2017.  
This  Consultation  strategy  provides  an  overview  of  the  approach  that  has  been  taken  for 
consulting  this  IPA  II  evaluation  with  its  main  stakeholders.  It  contains  two  elements.  The 
first is a stakeholder mapping for IPA II which outlines the main institutions or groups that 
are  considered  as  ‘stakeholders’.  The  second  is  a  stakeholder  consultation  strategy  that  the 
evaluation team has deployed, in consultation with the ISG, to engage with these stakeholders 
during  the  evaluation  process,  along  with  the  timeline  for  its  delivery.  An  important 
component of this consultation process has been the Open Public Consultation (OPC) done at 
the end of the synthesis phase of the evaluation. The OPC allowed acquiring feedback from 
all potentially relevant parties on the main evaluation findings, tough it was undermined by a 
low level of participation, which makes the feedback received not that significant.  

3.2 Stakeholder Mapping 
Below is a presentation of the main stakeholder groups in IPA II. They have been split into 
stakeholders operating at European level and those at national level.  

Figure 1 

Stakeholder mapping on European level 

31 

International Financing InstitutionsEU MS and other donors/ agencies•GIZ•USAID•Other governmentaldonors(e.g. SIDA, SDC)•OSF03Commission Services & EEAS01International non-state actors•REC•ERRC•Others06Other European Institutions02International Organisations•CoE•OSCE•UN AgenciesRegional•RCC. 05•DG NEAR•EEAS•DG AGRI•DG REGIO•DG HOME•DG EMPL•DG ECFIN•DG DEVCO•DG CLIMA•SGEuropean Level•European Parliament•ECoA•OLAF•EUROJUST/EUROPOL•European ChamberofCommerce. 
 
Figure 2 

Stakeholder mapping on national level 

3.3 Stakeholder Consultation Strategy 
The  stakeholder  mapping  for  IPA  II  outlined  the  main  institutions  or  groups  that  are 
considered  as  ‘stakeholders’.  The  developed  stakeholder  consultation  strategy  aimed  at 
ensuring  that  the  evaluation  team  could  fully  engage  with  all  these  stakeholders  during  the 
evaluation process. An important component of this consultation process was the open public 
consultation  (OPC)  done  at  the  end  of  the  synthesis  phase  of  the  evaluation  to  acquire 
feedback  from  all  relevant  parties  on  the  main  evaluation  findings.  Details  on  the 
implemented and completed approach are given below. 
The approach taken by this evaluation to engaging with the scope of all these aforementioned 
stakeholders  has  been  defined  by  their  role  in  IPA  II  and  their  relative  importance  and 
influence  over  it.  The  consultation  approach  for  the  principal  stakeholders  identified  in  the 
above table has been as follows: 
Commission Services and EEAS 
The  evaluation  involved  all  the  relevant  DG  NEAR  geographical  and  thematic  units 
throughout the desk and validation phases. DG AGRI has also been consulted throughout the 
evaluation  in  relation  to  IPARD  II.  Other  DGs  have  been  consulted  during  the  desk  and 
validation  phases  where  specific  instances  required  it  and  informed  of  evaluation  results. 
EUDs have also been consulted and informed throughout all phases of the evaluation.  
IFIs 
A major IFI presence in IPA II is the WBIF. Its Secretariat has been consulted during the desk 
and validation phases to understand complementarity of this instrument with other related IPA 
II  measures  and  has  also  been  consulted  in  the  synthesis  phase.  Other  IFIs  have  been 

32 

Governments (local)National Level.Partner countries -regionalCivil Society •National platforms•Regional networks05Partner countries -national•NIPACs•NAO•CFCU•Line Ministries•Operating Structures•Other governmentagencies03EU Delegations/ EU Office01Academia & Research•Think tanks•Universities& researchinstitutes07EU Funded Agencies02Private sector•Chambers ofCommerce•Professional associations. 06•EU Delegations•EU Offices•ReSPA•TACSO•Etc.Partner countries -regional•Regional governments•Municipalities•Associationsofmunicipalities04 
 
consulted in specific cases during the desk and validation phases, otherwise these have been 
informed of the evaluation preliminary results via the OPC. 
International Organisations 
Those  international  organisations  implementing  IPA  assistance  (CoE,  UN  agencies)  have 
been consulted in-country in the validation phase as well as desk phase if judged necessary. 
The RCC was consulted in the validation phase and in the synthesis phase when preparing the 
draft evaluation report.  
National Governments 
The  NIPAC,  NAO  and  OS  have  all  been  consulted  in  the  validation  phase,  as  they  have  a 
central role in the delivery of IPA II in the beneficiaries.  
Civil Society Organisations 
These stakeholders have an active role in ensuring citizens are adequately represented in the 
formulation  of  IPA  II  actions  and  in  overseeing  as  well  as  implementing  them.  They  have 
been consulted at all relevant levels, particularly during the validation phase and the OPC. 

3.4 Stakeholder Consultation Framework 
Consultation with stakeholders took place via the following means: 

 

Interviews  (face-to-face  and  via  phone)  and  group  discussions  with  various 
stakeholders at HQ level as well as via field missions to all seven IPA II beneficiaries; 
Instruments-wide survey to EU delegations (coordinated by the chapeau team); 

 
  Open Public Consultation (OPC) via web and face-to-face; 

Desk Phase 
The evaluators prepared the inception report and submitted it to the client for its approval. At 
this stage of the evaluation, no further stakeholder consultation took place. During the actual 
sub-phase  of  desk  work,  stakeholders  as  identified  above  have  been  closely  consulted  as 
appropriate,  preferably  via  phone/email/  face-to-face  /video-conference  discussions.  By  the 
end of the desk phase, a Desk Report has been prepared and discussed with the ISG. 
Validation Phase 
In  line  with  the  assessment  provided  in  the  table  above,  the  evaluation  team  has  consulted 
with those stakeholders identified as having a direct role in IPA II. The evaluators contacted 
these stakeholders with a view to acquiring information for the evaluation via the appropriate 
evaluation tools e.g. interviews, focus groups, questionnaires.  
Synthesis Phase 
Towards  the  end  of  the  synthesis  phase,  the  evaluation  team  has  prepared  a  set  of  key 
preliminary  findings  and  conclusions  based  on  its  analysis  from  the  validation  phase.  This 
formed a central part of the OPC process required by the ToR.  
Report dissemination 
Once the final evaluation report has been submitted, the ISG will decide on dissemination of 
the final report. 

3.5 Stakeholder statistics 
This section provides an overview of how many stakeholders have been consulted during the 
different  phases  of  the  evaluation,  where  they  were  consulted  (HQ  level  or  in  the  field),  as 
well as which stakeholder groups they belonged to. 

33 

 
 
General 
A total of 352 stakeholders have been interviewed to inform the results of this evaluation. 
58  interviews  took  place  at  HQ  level,  namely  in  Belgium,  Luxembourg  or  via  phone.  The 
remaining  294  interviews  were  conducted  during  the  field  missions  to  all  seven  IPA  II 
beneficiaries.  
The  interviews  conducted  covered  a  large  variety  of  stakeholders  on  EU  level  as  well  as 
international, regional and national level. The following stakeholder groups were consulted:  

Stakeholder group 
Commission Services & EEAS 
EU MS and other donors/agencies 
Other European Institutions 
EU Delegations/ EU Office 
Partner country institutions 
EU Funded Agencies 
International Organisations 
International Financing Institutions 
Civil Society 
Grand Total 

Number of stakeholders 

43 
29 
10 
50 
171 
1 
29 
5 
14 
352 

Field missions 
The  duration  of  the  different  field  visits  was  between  2,5  and  4  days.  The  following  table 
shows the number of stakeholders consulted per country: 

Country 
Albania 
Bosnia and Herzegovina 
The former Yugoslav Republic of Macedonia 
Kosovo 
Montenegro 
Serbia 
Turkey 
Grand Total 

Number of stakeholders 

61 
32 
90 
20 
23 
20 
48 
294 

34 

 
 
 
3.6 OPC Contribution  

3.6.1 Introduction 
The  draft  evaluation  report  on  the  Instrument  for  Pre-accession  Assistance  (IPA  II)71 was 
posted on the website of the European Commission for an Open Public Consultation (OPC) 
between  7  February  and  5  May  2017 72,  together  with  the  evaluation  reports  of  all  other 
External  Financing  Instruments  (instruments).  All  stakeholders  in  beneficiary  and  EU 
countries were welcome to participate in this process. The objective of the web-consultation 
was twofold: 

  To gather feedback from the broadest possible range of stakeholders, including those 
in beneficiaries and in the EU Member States, on the emerging conclusions from the 
evaluations. 

  To  gather  preliminary  ideas  on  the  future  external  financing  instruments  after  the 

current ones have expired by 31 December 2020. 

From  the  web  OPC,  a  total  of  30  contributions  were  received  which  related  specifically  to 
IPA  II.  Most  contributions  came  from  public  authorities  (12  in  total)  and  organisations  or 
associations 
(3), 
research/academia  (3)  and  EU  platforms,  networks  or  associations  (2),  as  illustrated  by  the 
graph below. 

(10).  Further  contributions  were  made  by  citizens/individuals 

Figure 3 

Type of contributors to IPA II via the web OPC 

In the framework of the web OPC, contributors were invited to respond to the following four 
IPA II-specific questions:  

  Question 1: How well do you think the IPA II has addressed its objectives? The main 
assessment  criteria  for  the  evaluation  are:  relevance;  effectiveness,  impact  and 
sustainability;  efficiency;  EU  added  value;  coherence,  consistency,  complementarity 
and  synergies;  and  leverage.  Feel  free  to  comment  on  the  findings,  conclusions  or 
recommendations for any/all of the criteria. 

71 Regulation (EU) No 231/2014 of the European Parliament and of the Council of 11 March 2014 establishing 
an Instrument for Pre-accession Assistance (IPA II). 
72 https://ec.europa.eu/europeaid/public-consultation-external-financing-instruments-european-union_en  

35 

233101202468101214EU platform, network, or associationResearch/academiaCitizen/individualOrganisation or associationPublic authority 
 
                                                 
  24 answers received 

  Question  2:  To  what  extent  do  you  think  the  strategic  orientation  of  IPA  II  and  the 
political  dialogue  between  the  EU  and  the  Governments  in  beneficiary  countries  is 
inclusive  public 
adequately  addressing 
consultation? 

improvement  of  a 

the  desired 

truly 

  15 answers received 

  Question  3:  To  what  extent  do  you  think  the  strategic  orientation  of  IPA  II  and  the 
political  dialogue  between  the  EU  and  the  Governments  in  beneficiary  countries  are 
adequately addressing the key issue of fight against corruption? 

  15 answers received 

  Question 4: If you have any other views on the IPA II you would like to share, they 

are welcome here. 

  22 answers received 

The following summaries synthesise the main contributions received for each question from 
the web OPC. 

3.6.2 Summary of OPC contributions 

3.6.2.1.1 Question 1: Addressing IPA II objectives  

Question  1:  How  well  do  you  think  the  IPA  II  has  addressed  its  objectives?  The  main 
impact  and 
assessment  criteria  for 
sustainability;  efficiency;  EU  added  value;  coherence,  consistency,  complementarity  and 
synergies;  and  leverage.  Feel  free  to  comment  on  the  findings,  conclusions  or 
recommendations for any/all of the criteria. 

the  evaluation  are:  relevance;  effectiveness, 

3.6.2.1.2 Summary of contributions 
Out  of  the  30  contributors  which  responded  to  the  IPA  II-specific  questions,  24  provided  a 
contribution to Question 1.  
The contributions were categorized as being positive, mixed, negative or neutral.  There is a 
balance  between  positive  (6)  and  negative  (5)  assessments,  with  mixed  (9)  being  the  most 
used category. Four contributions were neutral. 
The following bullet points provide a summary of the contributions received73:  
Positive 
 

IPA  II  is  well aligned  with  the  EU’s  enlargement  strategy and  increases  its  strategic 
relevance; 
IPA II has a better strategic focus than IPA I; 

 
  The priorities set for IPA II reflect the priorities for the region; 
 
IPA II objectives are well-defined; 
 
IPA II addresses its objectives well and its priorities and selected projects are relevant; 
  Security-related objectives and priorities appear in all relevant IPA II documents and 

are clearly defined. 

Negative 

73 Eight  of  the  24  contributions  were  marked  with  „cannot  be  directly  published  but  may  be  included  within 
statistical data”; their content is not included in the summary. 

36 

 
                                                 
  Lack of a suspension clause in the IPA II Regulation; 
  Lack of transparency about IPA’s absorption capacities; 
  Long administrative procedures; 
  Absence of joint programming between IPA II, EIDHR and IcSP; 
  The IPA Committee’s role is often too technical; 
  CSOs  have  difficulties  to  obtain  funding  (e.g.  co-financing  difficulties,  changing 
regulations  for  applying  to  IPA  funding,  procedures  creating  barriers  for  grassroots 
organisations, too little funding for CSOs); 

  Lack  of  comprehensive  horizontal  issues  (Roma,  gender,  child  rights,  disability 

aspects, climate action, security issues) in strategy papers and programming. 

Recommendations 

  Establishment  of  a  comprehensive  and  transparent  database  to  inform  the  MS  about 
budget execution, implementation of measures and projects, absorption and backlogs; 
  Better coordination and coherence between IPA II and the development cooperation of 

other MS and donors for more effectiveness; 

  The  integration  of  horizontal  themes  should  be  increased  (e.g.  for  gender:  gender 
analyses,  gender  disaggregated  data  for  all  sectors,  gender  responsive  budgeting, 
alignment of IPA II with the GAP II); 

  Coordination such as the WBIF and CoTEs should be well-resourced; 
 

IPA II after 2020 should not contain a stand-still clause, as this risks to become a tool 
for political pressure. 

In summary, the comments received are largely in line with the findings of the evaluation. 

3.6.2.1.1 Question 2: Addressing inclusive public consultation 

Question 2: To what extent do you think the strategic orientation of IPA II and the political 
dialogue  between  the  EU  and  the  Governments  in  beneficiary  countries  is  adequately 
addressing the desired improvement of a truly inclusive public consultation? 

3.6.2.1.2Summary of contributions 
Out  of  the  30  contributors  which  responded  to  the  IPA  II-specific  questions,  15  provided  a 
contribution to Question 2.  
The majority of contributions assessed the public consultation processes as negative (8). Only 
two  contributors  assessed  this  question  positively,  two  gave  a  mixed  assessment.  Three 
contributions remained neutral. 
The following bullet points provide a summary of the contributions received74:  
Positive 

  The strategic orientation of IPA II has improved over its predecessor due to the sector 

approach. 

Negative 

74 Five  of  the  15  contributions  were  marked  with  „cannot  be  directly  published  but  may  be  included  within 
statistical data”; their content is not included in the summary. 

37 

 
                                                 
 

 

Involvement of non-state actors in the implementation of IPA II significantly affects 
the level  of public consultations, but  there is  only  limited access to  IPA II funds for 
civil society; 
IPA II and the political dialogue between the EU and governments lacks structure for 
truly inclusive public consultations, as public consultations and dialogue are still done 
ad-hoc, based on personal connections and initiatives of individual EU representatives, 
or thanks to CSO initiatives, rather than through a systematic approach; 

  Processes are too complicated for a real inclusive public involvement; 
  CSOs are often excluded from governmental consultations; 
  The  participatory  processes  are  often  of  bad  quality,  e.g.  when  during  the  public 
consultation  on  IPA  II  SPDs  only  a  limited  number  of  NGOs  are  consulted  over  a 
short period of time; 

  At national level, public consultations are sometimes completely omitted; 
  As  the  subject  of  IPA  II  is  highly  political  and  has  low  impacts  on  the  public’s 
everyday life, the majority of the population is not in a position to express a concrete 
opinion on the policy. 

Recommendations 

  The  role  of  civil  society  in  the  beneficiaries  should  be  improved  to  counterbalance 

governments when implementing reform programmes; 

  The EU should reinforce its conditionalities; they should be concrete, measurable and 

realistic. They should be made public to the extent feasible; 

  The  political  dialogue  related  to  pre-accession  negotiations  should  take  into  account 

non-discrimination, disability and accessibility issues; 

  DG  NEAR  should  not  accept  SPDs  if  they  have  not  passed  a  truly  inclusive  public 
consultation and if they are not of sufficient quality and in line with EU policies and 
objectives; 

  The  EC  should  step  up  its  efforts  in  order  to  maximize  the  EU’s  support  for  civil 

society in Turkey via IPA II funds. 

3.6.2.1.3 Response of the Commission 
IPA  II  is  making  considerable  efforts  in  strengthening  public  consultation  with  the  civil 
society  and  other  actors.  Still,  the  process  is  at  the  beginning  and  differs  widely  among 
beneficiaries and sectors.  
A new performance framework is being set up for the instrument, which gradually should also 
improve transparency of budget executions and a true results-oriented reporting.  
With 
reward/penalise beneficiaries having regard to actual performance.  
Also based on past evaluations, consultation with CSOs has already improved. .  
The political dialogue with beneficiaries is strongly based on the European values, inter alia 
also taking into account non-discrimination, disability and accessibility issues.  
For Turkey, civil society support has been identified as one of the important areas of future 
IPA II funding (for instance increased support towards rights-based CSOs/ activists). 

instruments  already  foresees  a  mechanism 

the  performance  reward 

the 

to 

38 

 
3.6.3.1.1 Question 3: Addressing fight against corruption 

Question 3: To what extent do you think the strategic orientation of IPA II and the political 
dialogue  between  the  EU  and  the  Governments  in  beneficiary  countries  are  adequately 
addressing the key issue of fight against corruption? 

3.6.3.1.2 Summary of contributions 
Out  of  the  30  contributors  which  responded  to  the  IPA  II-specific  questions,  16  provided  a 
contribution to Question 3.  
The  above  graph  shows  that  the  opinions  of  the  contributors  are  almost  evenly  divided  in 
positive  (5)  and  negative  (7).  Only  one  contributor  gave  a  mixed  assessment  of  IPA  II’s 
contribution  to  the  fight  against  corruption.  Three  contributors  remained  neutral  in  their 
answers.  
The following bullet points provide a summary of the contributions received75:  
Positive 

  The  strategic  orientation  of  IPA  II  and  political  dialogue  largely  take  account  of  the 

key issues of the fight against corruption; 

  The  financial  support  via  PA  II  to  improve  the  legislation  against  corruption  is  an 

 

important step to reduce corruption; 
IPA II funding has made a positive contribution towards the fight against corruption, 
and some progress has been made in the beneficiaries; 

Negative 

  The fight against corruption is still not adequately addressed; 
  The fight against corruption is impeded by the fact that corruption is entrenched in the 

mentality in the beneficiary states; 

  The  increased  use  of  budget  support  does  not  help  fighting  corruption;  it  rather 

increases; 

  As  long  as  CSOs  are  generally  excluded  from  governmental  dialogues  and 
consultations & as long as governmental agencies are working with closed doors, it is 
difficult to fight against corruption in the candidate countries and potential candidates; 
  The complexity of the tools and processes at hand does not facilitate the fight against 

corruption; it rather facilitates corruption; 

  Weak  impact  on the  fight  against  corruption  in  the  field, including  almost  inexistent 

monitoring of corruption issues. 

Recommendations 

  All rule of law issues, including fight against corruption, should be a main focus of 

IPA II; 

  The IPA Committee should be informed about cases of corruption. Reports should be 
linked  to  actions  on  operational  level  (e.g.  amendments  of  programmes,  action 
programme, performance reward, etc.). 

75 Six  of  the  16  contributions  were  marked  with  „cannot  be  directly  published  but  may  be  included  within 
statistical data”; their content is not included in the summary. 

39 

 
                                                 
3.6.3.1.3 Response of the Commission 
In  line  with  revised  strategic  approach  for  the  instrument,  addressing  “fundamentals  first”, 
also  the  fight  against  corruption  is  now  more  in  the  focus  of  pre-accession  assistance 
compared to IPA I.  
Corruption is a systemic problem for many beneficiaries. However, pre-accession support is 
only one tool among others to effectively address this problem.  
Corruption is a not a particular problem for the actual delivery of pre-accession support, it is 
primarily  related  to  the  given  beneficiary  and  its  prevailing  culture  and  public  and  private 
structures and systems. To some extent, the instrument tries to address these systemic issues 
also through increased political and policy dialogue but results have still to materialise.  

3.6.4.1.1 Question 4: Any other views on IPA II 

Question  4:  If  you  have  any  other  views  on  the IPA  II  you would  like  to  share,  they  are 
welcome here. 

3.6.4.1.2 Summary of contributions 
Out  of  the  30  contributors  which  responded  to  the  IPA  II-specific  questions,  22  provided  a 
contribution to Question 4.  
The following bullet points provide a summary of the contributions received76: 

  Strategic  communication  and  public  diplomacy  should  be  increased  and  improved, 
regarding  the  values  of  the  EU  as  well  as  the  EU’s  added  value.  The  information 
strand projects under IPA II should be strengthened to make the IPA II activities of the 
EU more visible and to counterbalance the propaganda activities of third countries. A 
European  narrative  should  be  constructed  which  at  the  same  time  informs  of  the 
benefits  and  requirements  of  EU  membership.  Improving  the  visibility  and 
transparency  of  EU  measures  and  better  communication,  outreach  and  visibility  are 
central  points  for  all  operations  and  documents.  Visibility  could  also  be  improved 
through  the  establishment  of  “quick-impact  flagship  projects”  in  the  candidate 
countries and potential candidates. 

  Transparency  of  budget  execution  should  be  improved,  incl.  the  establishment  of  an 
adequate  database.  The  IPA  Committee  should  be  informed  more  timely  about  new 
policies, financial acts and draft action programmes. 

  EU funding awarded through IPA should shift from small investments in many areas 
towards  investing  into  systemic  changes  of  a  sector  in  order  to  build  sustainable 
impact and the EU should apply the lessons learned during the previous EU accession 
process and support its Member States to share their transition experience. 

  The  IPA  II  Regulation  lacks  sufficient  procedures  to  take  into  account  negative 
tendencies  in  relation  to  the  Copenhagen  criteria,  notably  on  democracy  and  human 
rights, because of the absence of a suspension clause. 

  Consultation  with  CSOs  is  an  important  component  of  IPA  assistance  programming 

which should continue and be developed even further. 

  Horizontal issues should be better addressed (e.g. disability issues, child protection) 
 

IPA  funding  to  women’s  CSOs  is  problematic,  as  it  does  not  reach  grassroots’ 
organisations  and  it  is  not  flexible  regarding  methods  and  expected  results;  EU 

76 Six  of  the  22  contributions  were  marked  with  „cannot  be  directly  published  but  may  be  included  within 
statistical data”; their content is not included in the summary. 

40 

 
                                                 
funding  falls  under  similar  smear  attacks  in  several  IPA  beneficiaries  and  for  that 
reason it is viewed as a problematic source; IPA administration requirements are too 
high/rigid;  and  some  of  the  funding  to  women  human  rights  defenders  /  women 
peacebuilders  is  channeled  through  UN  Women  in  a  re-granting  scheme,  but  UN 
Women is neither an expert of the specific country / conflict contexts, nor of practice 
on the ground, and is therefore highly criticized by those CSOs that are in most need 
of IPA-support. 
IPA  priorities  should  be  evidence-base  driven:  the  EU  should  support  the  candidate 
countries and potential candidates in areas where evidence shows that increased efforts 
are needed.  

 

  The  introduction  of  the  Centres  of  Thematic  Expertise  (CoTE)  in  DG  NEAR  to 

mainstream horizontal issues already in early planning phases is crucial. 

3.6.3.1.3 Response of the Commission 
Based on the introduced changes and new orientations, IPA II is expected to strengthen also 
communication and visibility.  
The instrument also recognises the need for effective policy and political dialogue in order to 
make pre-accession support fully successful.  
A new performance framework is being set up for the instrument, which gradually should also 
improve transparency of budget executions but also the results-oriented reporting.  
IPA  II  is  more  inclusive  than  its  predecessor  but  this  has  so  far  mostly  been  limited  to 
programming aspects.  
IPA  II  puts  emphasis  on  a  limited  number  of  key  sectors,  providing  concentrated  funding 
there, including sector budget support, to increase impact and sustainability.  
The  involvement  of  EU  MS  is  taking  place  for  instance  through  Twinning  or  TAIEX  or 
delegation of management tasks to accredited MS agencies but could still be increased.  
The need to strengthen the tackling of horizontal issues has been recognised by DG NEAR, 
particularly by introducing CoTEs and focal points. This process works already well in some 
sectors but not in all, including in areas like supporting women CSOs.  
In conclusion, IPA II recognises many of the critical comments raised above and some of the 
raised aspects in line with the instrument’s implementation in the field. 

3.6.3 Face-to-face consultations 
As part of the public consultation, the findings of the IPA II draft report were discussed face-
to-face on several occasions as follows: 

  23 March 2017, Civil Society Organisations (CSOs) and Local Authorities (LAs) from 
the  European  Union  and  partner  countries  in  the  framework  of  the  Policy  Forum  on 
Development Meeting; 

  24 March 2017, with the council working groups COWEB and COELA; 
  28  March  2017,  with  representatives  from  the  European  Parliament  and  the  EU 

Member States. 

41 

 
3.6.3.1 Summary of contributions 
During each meeting, a number of issues were raised with regards to the evaluation findings, 
but  also  to  IPA  II  in  general.  Comments  were  made  by  different  stakeholders  from  the 
Council  of  the  European  Union,  European  Parliament,  Member  States  and  Civil  Society 
Organisations.  
The  following  bullet  points  summarize  the  main  issues  raised  during  the  face-to-face 
consultations: 

  Visibility aspects of IPA II, such as communicating better the added value of the EU 

support within the beneficiaries; 

  Transparency issues related to  budget execution, absorption rates, allocated and paid 
amounts,  etc.  and  the  need  to  strengthen  administrative  data  collection,  monitoring 
systems and sector-based management information systems, 

  The lag between programming and implementation; 
  Coordination,  synergies  and  knowledge  sharing  between  IPA  and  other  instruments, 

IFIs, MS and other donors and development actors; 

  The need to take into account different actors in the field and to involve them into the 

implementation; 

  The limited support of IPA II to civil society and local governments and the need to 

provide more support to civil societal actors in the beneficiaries; 

  The lack of a suspension clause in the IPA II Regulation; 
  Possibilities of monitoring the success of projects also after their completion; 
  The response to low performance under the performance framework; 
 
  The  importance  to  focus  on  the  rule  of  law  and  key  reforms  in  economy  and 

Issues of corruption in the public sector and the private sector; 

administration; 

  Stability issues and how to build stability in the beneficiaries. 

3.6.3.2 Response of the Commission 
There  is  a  clear  requirement  now  under  IPA  II  to  strengthen  also  communication  and 
visibility,  revised  procedures  and  intensified  actions  are  being  launched  at  the  time  of  this 
evaluation.  
A new performance framework is being set up for the instrument, which gradually should also 
improve transparency of budget executions but also the results-oriented reporting.  
IPA  II  is  more  inclusive  than  its  predecessor  but  this  has  so  far  mostly  been  limited  to 
programming  aspects.  The  inclusiveness  of  the  implementation  process  still  needs  to 
materialise.  
Measures  to  tackle  low  performance  have  been  applied  in  the  past  and  might  be  used  even 
more in the future, based on objective performance data. Discussions about an increased use 
of  sanction  mechanisms  are  currently  ongoing.  Suspension,  if  it  is  to  be  more  strongly 
implemented in future, will also need an accompanying strong policy dialogue.  
Final/ ex-post evaluations are already important tools to supplement the performance review. 
Also, the Results-oriented Monitoring allows for the review of actions being at the final stage. 
The focus on the rule of law and other key reforms is a major feature of the new instrument. 
Stability is a basic requirement for effectively delivering pre-accession assistance but cannot 
be addressed solely by the instrument.  

42 

 
In conclusion, IPA II potentially offers many of the features required by the given comments. 
It  needs  still  to  be  seen  to  which  extent  these  requirements  can  be  addressed  in  the  given 
context of beneficiaries and their prioritised sectors. 

43 

 
 
 
 
Annex 5. Acronyms 

List of acronyms and abbreviations 

AAP 

AD 

BS 

CARDS 

CBC 

CEB 

CIR 

CoTE 

CSO 

CSP 

DCI 

DG 

Annual Action Programme 

Action Document 

Budget Support 

Community Assistance for Reconstruction, Development, and 
Stabilisation 

Cross-Border Cooperation 

Council of Europe Development Bank  

Common Implementation Regulation 

Centre of Thematic Experience 

Civil Society Organisations 

Country Strategy Paper 

Development Cooperation Instrument 

Directorate-General 

DG AGRI 

Directorate-General for Agriculture and Rural Development 

DG DEVCO 

Directorate-General for International Cooperation and Development 

DG EMPL 

Directorate-General for Employment, Social Affairs & Inclusion 

DG NEAR 

Directorate-General for Neighbourhood and Enlargement Negotiations 

DG REGIO 

Directorate-General for Regional Policy 

DIS 

EAMR 

EBRD 

EC 

EDIF 

EEAS 

EFI 

EIB 

EIDHR 

ENI 

Decentralised Implementation System 

External Assistance Management Report 

European Bank for Reconstruction and Development  

European Commission 

Enterprise Development & Innovation Facility 

European External Action Service 

External Financing Instrument 

European Investment Bank 

European Instrument for Democracy and Human Rights  

European Neighbourhood Instrument 

44 

 
EQ 

ERDF 

EU 

EUD 

EUR 

FPI 

Evaluation Question 

European Regional Development Fund  

European Union 

European Union Delegation 

Euro 

Service for Foreign Policy Instruments 

FRONTEX 

European Border and Coast Guard Agency 

GGF 

HQ 

HRD 

IB 

IFI 

IL 

IMBC 

INSC 

IOM 

IPA 

IPARD 

ISP 

ISPA 

IT 

JC 

KPI 

LA 

LGBT 

MAAP 

Green for Growth Fund 

Headquarters 

Human Resources Development 

Institution Building 

International Financial Institution 

Intervention Logic 

Indirect Management with the Beneficiary Country 

Instrument for Nuclear Safety Cooperation 

International Organisation for Migration 

Instrument for Pre-Accession Assistance 

Rural Development component of the Instrument for Pre-accession 
Assistance 

Indicative Strategy Paper 

Instrument for Structural Policy for Pre-Accession 

Information Technology 

Judgement Criterion 

Key Performance Indicator 

Local Authority 

Lesbian, Gay, Bisexual, Transgender 

Multi-annual Action Plan 

MADAD 

EU Regional Trust Fund in Response to the Syrian Crisis  

MAP 

MBP 

Multi-annual Programming 

Multi-Beneficiary Programme  

45 

 
MCSP 

Multi-country Strategy Paper 

MFF 

MIS 

MS 

MTR 

NAO 

Multi-annual Financing Framework 

Management Information System 

Member State 

Mid-term Review 

National Authorising Officer 

NIPAC 

National IPA Coordinator 

OPC 

PAR 

PFM 

Open Public Consultation 

Public Administration Reform 

Public Financial Management 

PHARE 

Poland and Hungary: Assistance for Restructuring their Economies 

PI 

QC 

QSG 

RACER 

ReSPA 

RHP 

ROM 

Partnership Instrument 

Quality Control 

Quality Support Group 

Relevance, Acceptability, Clarity, Easiness, Robustness 

Regional School of Public Administration 

Regional Housing Programme 

Results-oriented Monitoring 

SAPARD 

Special Accession Programme for Agriculture and Rural Development 

SIGMA 

Support for Improvement in Governance and Management 

SLI 

SMC 

SMSC 

SOP 

SPD 

TA 

TAIEX 

TEN-T 

TEU 

Sector Lead Institution 

Sector Monitoring Committee 

Sector Monitoring Sub-Committee 

Sector Operational Programme 

Sector Planning Document 

Technical Assistance 

Technical Assistance and Information Exchange instrument of the 
European Commission 

Trans-European Transport Network 

Treaty of the EU 

46 

 
WBIF 

WBT 

Western Balkans Investment Framework 

Western Balkans and Turkey 

47 

 
 
 
 
Annex  6.  External  evaluators'  report,  including  its  annexes  and  methods 

and analytical models used 

"The  external  evaluation  can  be 
consultation-external-financing-instruments-european-union_en" 

found  here:  https://ec.europa.eu/europeaid/public-

48