COMMISSION STAFF WORKING DOCUMENT EVALUATION of Macro-financial assistance to third countries (Meta-evaluation of operations for 2010-2020)
Date: 2023-01-13
Institutions: Directorate-General for Economic and Financial Affairs, European Commission
Subjects: EU aid, European neighbourhood policy, International Monetary Fund, economic stabilisation, financial aid, macro-financial assistance, macroeconomics, third country

EN 

  EN 

   EUROPEAN COMMISSION  Brussels, 13.1.2023  SWD(2023) 16 final  COMMISSION STAFF WORKING DOCUMENT  EVALUATION  of    Macro-financial assistance to third countries   (Meta-evaluation of operations for 2010-2020)  {SWD(2023) 17 final}  
 
 
Table of contents 

1. 

INTRODUCTION ................................................................................................................................. 5 

1.1 

Purpose and scope of the meta-evaluation ........................................................................ 5 

2.  WHAT WAS THE EXPECTED OUTCOME OF THE INTERVENTION? ........................................ 8 

2.1 

2.2 

Description of the intervention and its objectives ............................................................. 8 

Point(s) of comparison .................................................................................................... 11 

3.  HOW HAS THE SITUATION EVOLVED OVER THE EVALUATION PERIOD? ........................ 13 

4. 

EVALUATION FINDINGS (ANALYTICAL PART)........................................................................ 20 

4.1 

4.2 

4.3 

4.4 

To what extent was the intervention successful and why? .............................................. 20 

How did the EU intervention make a difference? ........................................................... 29 

Is the intervention still relevant? ..................................................................................... 30 

Is the current framework for evaluating interventions appropriate? ............................... 31 

5.  WHAT ARE THE CONCLUSIONS AND LESSONS LEARNED? .................................................. 34 

5.1 

5.2 

Conclusions ..................................................................................................................... 34 

Lessons learned ............................................................................................................... 35 

ANNEX I:   PROCEDURAL INFORMATION ........................................................................................... 37 

ANNEX II. METHODOLOGY AND ANALYTICAL MODELS USED.................................................... 40 

ANNEX III. EVALUATION MATRIX AND, WHERE RELEVANT, DETAILS ON ANSWERS 

TO THE EVALUATION QUESTIONS (BY CRITERION) .............................................................. 44 

ANNEX  IV.  OVERVIEW  OF  BENEFITS  AND  COSTS  AND,  WHERE  RELEVANT,  TABLE 

ON SIMPLIFICATION AND BURDEN REDUCTION. ................................................................... 59 

ANNEX V. STAKEHOLDERS CONSULTATION - SYNOPSIS REPORT .............................................. 60 

BIBLIOGRAPHY ......................................................................................................................................... 71 

1 

 
 
 
 
 
 
Glossary 

Term, abbreviation or 
acronym 

Meaning or definition 

BoPF 

BR(G) 

Comitology 

CPI 

DG ECFIN 

ECJ 

EEAS 

EFSM 

EIB 

ENI 

ENPI 

Financial programming 

Balance  of  Payments  Facility:  the  EU’s  facility  for 
providing  medium-term 
for 
Member  States’  balances  of  payments  pursuant  to 
Council Regulation (EC) No 332/2002. 

financial  assistance 

Better  Regulation  (Guidelines): 
the  Commission’s 
policy  framework  for  ensuring  evidence-based  and 
transparent  law-making  based  on  the  views  of  those 
who may be affected by the laws in question. 

to  Regulation 

Arrangements  pursuant 
(EU)  No 
182/2011  for  Commission  implementing  acts  to  be 
submitted 
committee  of  Member  State 
representatives  either  pronouncing  itself  or  voting  on 
the draft before it enters into force. 

to 

a 

Consumer Price Index. 

Directorate-General for Economic and Financial Affairs 
of the European Commission. 

European Court of Justice. 

European External Action Service. 

European Financial Stabilisation Mechanism: the EU’s 
facility  for  providing  financial  assistance  to  Member 
States  pursuant 
to  Council  Regulation  (EU)  No 
407/2010. 

European Investment Bank. 

European Neighbourhood Instrument. 

European Neighbourhood Policy Instrument. 

Determination  of  a  target  for  the  fiscal  and  monetary 
stance to produce a certain economic outcome (e.g. the 
observance  of  external  financing  constraints  given  a 
certain  exchange  rate 
the  basis  of  a 
quantitative  model  of  key  inter-relationships  in  an 
economy. 

level)  on 

Framework regulation 

A  regulation  containing  the  general  legal  provisions 

2 

 
 
governing  the  matter  concerned,  e.  g.  procedures, 
scope, purpose. 

GDP 

Gross domestic product. 

Global Financial Safety Net 

IFI 

IMF 

IPA 

Joint Declaration 

LFA 

MFA 

MFF 

MoU 

NDICI (Regulation) 

OLP 

OPC 

Public financial management 

self-insurance 

The  global  network  of  crisis-financing  instruments 
encompassing 
(reserves);  bilateral 
arrangements (swap lines between central banks during 
periods  of  stress);  regional  arrangements  whereby 
members  stand  ready  to  support  one  another;  and  a 
multilateral system with the IMF at its centre.  

International financial institution. 

International Monetary Fund. 

Instrument for Pre-Accession Assistance. 

Joint  Declaration  by  the  European  Parliament  and  the 
Council  adopted  together  with  the  decision  providing 
further macro-financial assistance to Georgia (Decision 
No 778/2013/EU (OJ L 218, 14.8.2013, p. 15). 

Loan Facility Agreement. 

Macro-financial assistance. 

the  seven-year 
Multiannual  Financial  Framework: 
framework that sets out the annual expenditure ceilings 
for various policy areas. 

Memorandum of understanding. 

Regulation (EU) 2021/947 of the European Parliament 
and of the Council of 9 June 2021 establishing the 
Neighbourhood, Development and International 
Cooperation Instrument – Global Europe (OJ L 209, 
14.6.2021, p. 1). 

Ordinary legislative procedure pursuant to Article 294 
TFEU. 

Open public consultation. 

The  set  of  all  components  of  a  country’s  budget 
process,  including  strategic  planning,  a  medium-term 
expenditure framework,  annual  budgeting (formulation 
and  execution),  as  well  as  revenue  management, 

3 

 
 
SDR 

TEU 

TFEU 

procurement, control, accounting, reporting, monitoring 
and evaluation, audit and oversight. 

Special  drawing 
reserve  asset  exchangeable  for 
currencies of IMF members. 

right: 

IMF-created 

international 
the  freely  usable 

Treaty on the European Union. 

Treaty on the Functioning of the European Union. 

4 

 
 
1. 

INTRODUCTION 

1.1 Purpose and scope of the meta-evaluation 

Macro-financial  assistance  (MFA)  is  a  form  of  EU  financial  aid  for  partner 
countries  experiencing  (or  threatened  by)  a  balance  of  payments  crisis,  helping  to 
restore their external stability and to return their economies to a sustainable path. Taking 
the form of medium- or long-term loans or grants (or an appropriate combination of the 
two),  and  normally  complements  financing  provided  by  the  international  community  in 
the  context  of  a  financial  arrangement  with  the  International  Monetary  Fund  (IMF). 
Support  can be provided only  if beneficiaries  respect effective democratic mechanisms. 
Disbursements are conditional on the implementation of policy reforms  agreed between 
the  Commission  on  the  EU’s  behalf  and  the  beneficiary  country  and  laid  down  in  a 
memorandum of understanding (MoU) (European Commission, 2021). 

Since  its  inception  30  years  ago,  MFA  has  been  increasingly  used by the  EU  in  its 
dealings with countries outside the EU and is an important part of its contribution 
to  the  Global  Financial  Safety  Net  under  G20  rules.  Initially  MFA  was  extended 
mostly to  countries of central and eastern Europe (some of which have since joined the 
EU), but it has progressively been extended to other countries in the former Soviet Union 
and  the  Mediterranean.  MFA  volumes  increased  substantially  following  the  global 
economic  and  financial  crisis  of  2008-2009,  which  profoundly  affected  the  emerging 
economies  of  EU  neighbourhood  countries.  Throughout  2014-2020,  MFA  has  been 
instrumental  in  supporting  partner  countries  in  the  eastern  and  southern  neighbourhood 
facing conflict and challenging political transitions. In 2020-2022, EUR 3 billion in MFA 
loans was  made available to  help  10 partner  countries limit  the economic fallout  of the 
COVID-19  pandemic.  More  recently,  in  2022,  with  the  escalation  of  the  conflict  and  the 
Russian  war  of  aggression  against  Ukraine,  Ukraine  requested  new  MFA  support.  It 
received EUR 1.2 billion in emergency MFA (disbursed in March 2022) and, at the time of 
writing, EUR 6 billion in exceptional MFA (EUR 1 billion disbursed in August and EUR 5 
billion disbursed from October to December 2022). 

The geopolitical, security-related and economic challenges beyond the EU’s borders 
require  a  strong  and  determined  action.  To  rise  to  new  (and  old)  challenges,  the 
current Commission has identified the need to strengthen the role of Europe in the world 
(Von der Leyen, 2019). To do this, the EU must act quickly and efficiently to support key 
partner countries and territories facing external financing difficulties. MFA is one way of 
doing this.  

The  COVID-19  pandemic  illustrates  this  perfectly.  The  pandemic  challenges  partner 
countries  and  their  financial  capacity  like  never  before.  Against  this  backdrop,  the  EU 
granted a total of EUR 3 billion in MFA to 10 partner countries in spring 2020 (Decision 
(EU) 2020/7011). This decision and the speed it was implemented with showed that MFA 
continues to be used extensively by the EU in its dealings with countries outside the EU, 
in line with its geopolitical ambitions.  

1 OJ L 165, 27. 5.2020, p. 31. 

5 

 
 
                                                           
MFA has also demonstrated its worth as a flexible crisis instrument enabling the EU 
to  swiftly  and  substantially  support  Ukraine  following  Russia’s  unprovoked  and 
unjustified  invasion  of  the  country  at  the  beginning  of  2022.  Already  since  2014, 
Ukraine has received an unprecedented  amount  of MFA.  In 2014-2019,  a total  of  EUR 
4.4 billion was made available to Ukraine in four MFA operations. This is the most EU 
MFA  ever  granted  to  a  non-EU  country  in  such  a  short  period  of  time.  Following 
Russia’s unprovoked and unjustified invasion of Ukraine, an additional EUR 7.2 billion 
of  MFA  loans  were  made  available  at  record  speed.  This  is  testament  to  the  EU’s 
unwavering support for Ukraine. 

Unlike  other  EU  external  action  instruments,  MFA  operations  are  adopted  and 
implemented directly on the basis of the treaties, without a specific legal framework. 
Since the Treaty on the European Union (TEU) and the Treaty on the Functioning of the 
European Union (TFEU) entered into force on 1 December 2009, Article 212 TFEU (and 
for developing countries Article 209 TFEU) are the legal basis for MFA and  govern it.. 
The European Parliament and the Council authorise each MFA operation on the basis of a 
dedicated  Commission  proposal,  in  accordance  with  the  ordinary  legislative  procedure 
(OLP,  as  set  out  in  Article  294  TFEU).  In  urgent  cases,  MFA  operations  can  be 
authorised by the Council alone, in a non-legislative procedure, always on the basis of a 
Commission proposal, pursuant to Article 213 TFEU. 

For a crisis instrument, the decision-making process for MFA is considered rather 
lengthy.  The  first  meta-evaluation  of  past  MFA  operations,  undertaken  in  20092, 
identified  delays  in  adopting  MFA  decisions  as  a  major  shortcoming  of  the  instrument. 
To remedy this, the Commission put forward a proposal for a Framework Regulation for 
MFA in July 2011, aimed at expediting decision-making and aligning the management of 
the  MFA  instrument  with  that  of  other  external  instruments  (European  Commission, 
2011). However, the proposed approach was not  acceptable to  the co-legislators,  which 
modified  the  proposal  to  the  extent  that  it  lost  its  “raison  d’être”,  and  the  Commission 
ultimately withdrew the proposal; lawfully, as the Court of Justice later held (ECJ, 2015). 
As a result, there is no framework regulation for MFA policy and legislative decisions on 
individual MFA operations continue to be adopted by the EP and the Council on a case-
by-case  basis  in  the  OLP.  Recent  experience  with  the  COVID-19  MFA  package  and 
Ukraine  have  shown  that  the  current  MFA  set-up  lends  itself  to  swift  adoption,  if  all 
institutions agree to make full use of the available procedural flexibilities in particularly 
urgent cases. 

In line with the applicable legal and political framework3, ex-post evaluations of each 
individual  MFA  operation  are  undertaken.  This  means  that  a  steady  stream  of 
evaluations, not necessarily in sync with the MFF cycle, is a distinct feature of MFA. By 
2020,  15  such  evaluations  had  been  completed  since  the  first  meta-evaluation  of  MFA 
operations in 2009. In 2020/21, the Commission’s Directorate-General for Economic and 
Financial  Affairs  (DG  ECFIN)  carried  out  a  new  meta-evaluation  covering  15  MFA 

2 (GHK Consulting Limited & European Policy Evaluation Consortium, 2009) 

3 MFA Decisions, the Financial Regulation applicable to the general budget of the European Communities 

and Better Regulation Guidelines. 

6 

 
 
                                                           
evaluations  completed  in  the  last  10  years.  The  exercise,  coinciding  with  the  MFA 
instrument’s  thirtieth  anniversary,  took  stock  of  previous  evaluation  results  and  aims  to 
support  organisational  learning  both  in  terms  of  the  design  and  implementation  of  the 
MFA instrument and the methodology used to evaluate individual operations. The meta-
evaluation will also support transparency and accountability and help lay the groundwork 
for  any  possible  improvements  in  the  instrument’s  design  and/or  implementation  that 
might be necessary in order to even better support the need for a stronger ‘Global Europe’ 
in line with President von der Leyen’s vision. 

An  external  evaluation  study  has  been  undertaken  to  help  inform  the  meta-
evaluation  (DG  ECFIN,  2020)4.  The  study  began  in  December  2020  and  ended  in 
October  2021.  Its  outcome  is  supplemented  by  input  from  a  high-level  seminar  on  the 
experience  of  30  years  of  MFA  that  took  place  in  November  2021  to  flesh  out  the 
evidence base for the meta-evaluation.  

Against this backdrop, and on the basis of the 15 evaluations mentioned above, the 
meta-evaluation: 
1.  assesses  the  principles  and  characteristics  of  the  MFA  instrument  in  line  with  the 
Better  Regulation  criteria  of  relevance,  effectiveness,  efficiency,  coherence,  value 
added,  and  taking  into  account  the  instrument’s  ability  to  respond  to  EU  external 
action priorities, as well as the day-to-day management of MFA interventions;  

2.  assesses  the  overarching  methodological  framework  applied  across  the  evaluations 

and identifies areas where there is room for improvement. 

The meta-evaluation’s methodological approach is threefold: 
1.  participatory  and  inclusive  data  collection  and  analysis  in  a  (key)  stakeholder 
consultation  programme  (semi-structured  interviews  and  a  public  consultation),  a 
validation workshop preceded by a roundtable with the Commission staff in charge of 
MFA, and a high-level seminar;  

2.  a mix of qualitative and quantitative input, with qualitative input consisting mainly of 
document  analysis  (chiefly  the  evaluation  reports  of  the  15  operations  mentioned 
above)  and  stakeholder  input,  and  quantitative  data  supporting  the  qualitative 
information as much as possible;  

3.  triangulation:  the  information  and  data  collected  from  a  range  of  different  sources 

collectively provide answers to DG ECFIN’s evaluation questions. 

A  number  of  limitations  were  experienced  during  the  completion  of  the  study  and 
consequently the preparation of the meta-evaluation, including data availability and 
access  to  stakeholders  (Valdani,  Vacari  e  Associati  &  Centre  for  Strategy  and 
Evaluation  Services,  Oxford Analytics, 2021). Such limitations have undoubtedly had a 
bearing on the quality of the exercise, but do not affect the robustness or reliability of the 
meta-evaluation’s  conclusions,  which  serve  as  a  useful  basis  for  reflections  on  the 
instrument. The limitations and mitigating factors are elaborated on in Annex II.  

4 (Valdani, Vacari e Associati & Centre for Strategy and Evaluation Services, Oxford Analytics, 2021) 

7 

 
 
 
                                                           
2.  WHAT WAS THE EXPECTED OUTCOME OF THE INTERVENTION? 

2.1 Description of the intervention and its objectives 

toolkit, 

Since  1990,  the  EU  (and  its  predecessors5)  has  been  providing  MFA  to  partner 
countries,  primarily  in  its  neighbourhood.  Although  the  EU  has  been  using  the 
instrument relatively regularly, it continues to be meant to be an ad hoc financing tool in 
the  EU’s  external  policy 
triggered  by  sudden  economic  and  financial 
developments  in  recipient  countries.  This  means  that  instead  of  having  a  specific  legal 
basis, operations have been based on the EU’s ‘flexibility clause’, requiring a unanimous 
decision  in  the  Council  without  a  formal  say  for  the  European  Parliament  (European 
Commission,  2021).  In  order  to  nonetheless  provide  a  framework  for  those  ad  hoc 
decisions, the Council formulated a set of criteria to determine the basic features of MFA 
operations  at  an  informal  meeting  in  Genval,  Belgium  (the  Genval  criteria)  in  1993.  It 
revised these criteria in 1996 and 2002 (European Commission, 2011).  

By  a  Joint  Declaration  in  20136,  the  European  Parliament  and  the  Council  has 
provided guidance on the MFA instrument. While not legally binding, it takes stock 
of the main features of MFA. It was drafted in the aftermath of the entry into force of the 
Lisbon Treaty in 2009, which gave the European Parliament full co-legislator powers on 
MFA. The Declaration, based on the Genval criteria, albeit clarified and adapted reflects 
the co-legislators’ shared understanding. In line with these criteria, MFA is an exceptional 
financing instrument of untied and undesignated balance-of-payments support for eligible 
third countries and territories. Its objectives are: 
1.  to restore a sustainable external finance situation for eligible countries and territories 

facing external financing difficulties; 

2.  to  underpin  the  implementation  of  a  policy  programme  that  contains  far-reaching 
adjustment  and  structural  reform  measures  designed  to  improve  countries’  and 
territories’ balance of payment positions, in particular over the programme period; 
3.  to strengthen the implementation of relevant EU programmes and agreements with the 

EU. 

Box: The principles of the Joint Declaration 

The Joint Declaration of the European Parliament and the Council sets out the principles to be followed for 
co-legislators  when  agreeing  on  MFA.  These  principles  are  based  on  the  earlier  Genval  criteria  and 
incorporate  some  parts  of  the  2011  Commission  proposal  (COM(2011)  396  final)  for  a  framework 
regulation. 

(1)  Aim  of  assistance:  This  principle  sets  out  the  underlying  context  for  MFA.  The  assistance  should  be 
exceptional,  with  the  aim  of  providing  balance-of-payments  support  to  help  establish  a  sustainable 
situation  for  countries  and  territories  facing  difficulties  in  this  area.  It  should  be  focused  on  the 
financing  gap  a  country  faces,  taking  into  account  the  assistance  received  from  other  international 
partners, in particular the IMF. MFA operations should be short-term. 

5 European Economic Community until 1993, then the European Community from 2010. 

6  Decision  No 778/2013/EU  of  the  European  Parliament  and  of  the  Council  of  12 August 2013  providing 

further macro-financial assistance to Georgia (OJ L 218, 14.8.2013, p. 15). 

8 

 
 
                                                           
(2)  Geographic  scope:  This  principle  indicates  which  countries  or  territories  are  eligible  to  apply  for 
financial assistance, namely candidate and potential candidate countries for membership of the EU, or 
countries  in  the  scope  of  its  neighbourhood  policy.  Other  countries  might  be  considered  eligible  in 
exceptional cases. 

(3)  Form  of  assistance:  Assistance  should  mostly  be  in  the  form  of  loans,  but  when  appropriate  can  be 

provided in the form of grants or a combination of the two. 

(4)  Financial provisions: This principle establishes the financial provisions and how they have to be in line 

with the EU Multiannual Financial Framework. 

(5)  Amount  of 

the  assistance:  The  amount  of  financial  assistance  should  be  based  on 

the 
country’s/territory’s  financing  needs  or  gaps,  consistent  with  a  sufficient  and  defendable  level  of 
foreign exchange reserves.  

(6)  Conditionality: MFA is conditional on the (prospective) recipient’s: 

a. 

respecting effective democratic mechanisms, including a multiparty parliamentary system and the 
rule of law, and on its guaranteeing respect for human rights (political precondition); 

c. 

b.  having in place a non-precautionary financial arrangement with the IMF and (in the disbursement 
phase)  a  satisfactory  track  record  of  implementing  the  relevant  IMF-supported  economic 
adjustment programme;  
implementing  specific  structural  reforms  (policy  conditions)  that  the  recipient  would  have  to 
undertake in a set timeframe, agreed between the Commission and the recipient in an MoU. 
(7)  Procedure: The  EU  decision-making  procedure  sets  out  the  details  of  the  different  roles  and  steps  in 
providing  financial  assistance.  These  consist  of  the  ordinary  legislative  procedure  (OLP)  for  the 
adoption of a decision on a proposal by the Commission, and comitology for the adoption of the MoU 
(without veto powers for Member States in the case of operations below EUR 90 million and with veto 
powers for operations above that), as well as the Commission Decision with respect to the grant and/or 
loan agreement.  

Principles  (8)-(11)  focus  on  operational  details  with  regard  to  implementation  and  financial  management, 
disbursements, support measures and the protection of the EU’s financial interests. 
Principles (12) and (13) make provision for regular reporting on and evaluation of MFA operations. 
The  combination  of  the  objectives  mentioned  above  makes  MFA  a  singular 
instrument.  
  On  the  one  hand,  it  is  a  traditional  instrument  for  addressing  temporary  external 
financing difficulties and crises. In that sense, it mirrors and is closely linked to IMF 
interventions  with  the same purpose, albeit  intended as  a complementary  measure to 
create financial stability.  

  On the other hand, unlike the IMF, the EU is an explicitly political actor, and in line 
with Article 205 TFEU and Article 21 TEU, MFA is ultimately a political action. As 
such, it is part and parcel of a framework of tools, instruments and policies designed to 
also achieve the EU’s broader objectives and promote its values.  

It  therefore  follows  that  MFA  combines  parts  of  a  balance-of-payments  financing 
tool  (such  as  the  IMF’s)  with  traditional  instruments  of  external  financial 
cooperation,  whilst  being  distinct  from  them  both.  In  contrast  to  IMF  support,  MFA 
explicitly  requires  the  fulfilment  of  political  preconditions.  In  contrast  to  other  EU 
external financial cooperation instruments, MFA operations continue to be authorised by 
means  of  the  OLP,  thereby  remaining  outside  of  the  scope  of  the  Commission’s 
implementing  powers  under  the  EU’s  Neighbourhood,  Development  and  International 
Cooperation Instrument – Global Europe (NDICI)7.  

7 This distinction is political rather than legal. The adoption of the 2013 Joint Declaration and the Council 
Conclusions from 2002 reflects the co-legislators’ shared understanding that MFA is not part of the EU’s 
standard external financial cooperation toolbox. 

9 

 
 
                                                           
MFA typically comes into play when an unsustainable mix of economic policies or an 
economic  shock  have  given  rise  to  external  financing  imbalances  that  result  (or  risk 
resulting)  in  an  undesired  currency  depreciation.  MFA  operations  therefore  follow  a 
traditional pattern of seeking to correct such an unsustainable policy mix (or other causes 
of a shock) whilst easing economic adjustment by means of financial assistance. It targets 
the  policy  areas  that  are  relevant  to  macroeconomic  stability  in  general  and  aims  to 
improve them, in line with the following diagram.  

Figure 1: MFA intervention logic 

  The  Commission  (on  behalf  of  the  EU)  provides  financial  assistance,  having  agreed 
with  the  beneficiary  on  a  set  of  (structural)  policy  conditions  underpinning  the 
macroeconomic  adjustment  taking  place.  Technical  assistance  may  also  be  agreed 
(Inputs). 

  The beneficiary begins  the process  of  implementing the reforms agreed on.  In the 15 
operations  evaluated,  reforms  tended  to  focus  on  public  sector  governance,  the  fight 
against corruption and  money laundering,  the  energy  sector,  the business  climate,  the 
financial sector, and the social safety net and labour market. 

  During and after an MFA operation, the policy reforms and financial assistance should 
help stabilise the beneficiary’s macroeconomic situation, including with regard to their 
balance-of-payment  position  and  (as  necessary)  public  finances.  Ideally,  these  effects 
should last for a long period, underpinned by structural improvements in the efficiency 
and  effectiveness  of  the  development  and  implementation  of  economically  relevant 
policies. 

intended  effects  depend  on  the  operation  and  the 
However,  the  specific 
country/economy concerned. For reasons already mentioned, the intervention logic and 
the operation’s operational objectives need to be placed in the context of the MFA’s ends 
and  purposes  as  a  way  for  the  EU  to  carry  out  external  action.  These  include,  among 
others,  safeguarding  EU  values  and  fundamental  interests,  consolidating  and  supporting 
democracy and the rule of law, encouraging the integration of all countries into the world 
economy, and promoting an international system based on closer multilateral cooperation 

10 

 
 
 
 
and  good  global  governance  (Article  21  TEU).  These  objectives  (except  in  individual 
cases)  are  not  part  of  the  operational  objectives  of  MFA  operations  and  therefore  of  the 
intervention  logic.  Rather,  they  are  preconditions  for  a  country’s  benefiting  from  MFA 
support. 

It  must  also  be  stressed  that  MFA  is  usually  part  of  a  stabilisation  operation 
incorporating,  alongside  the  EU,  at  least  the  IMF  and  most  often  also  the  World 
Bank  Group  and  in  some  cases  other  bilateral  and  multilateral  international 
partners. In the case of the IMF, the link is particularly close given the need for parallel 
IMF support (Principle 6 of the Joint Declaration) and the requirement for the amount of 
assistance  to  be  based  on  a  country’s  needs  (Principle  5). At  the  same  time,  given  the 
IMF’s  own  ceilings  (as  a  multiple  of  a  country’s  share  in  its  capital,  the  quota  (IMF, 
1944)), its interventions can equally be contingent on an MFA operation as it is necessary 
to ensure that the stabilisation operation is fully financed8.   

This link also results in a certain division of labour: Commission services rely on the 
IMF its core areas of expertise, most notably in agreeing with the beneficiary country the 
macroeconomic  stance  required  to  underpin  the  adjustment  to  external  financing 
constraints  (the  financial  programming  exercise).  The  IMF’s  own  policies  require  it  to 
comprehensively  cover all reform  requirements  (including structural  ones) necessary  for 
the  standalone  success  of  its  intervention  (IMF,  2005).  Since  MFA  operations  are 
conditional  on  a  continuously  satisfactory  track  record  in  implementing  IMF-supported 
policy programmes, IMF conditions (unless waived) are in practice incorporated into the 
MFA  requirements.  For  its  part,  the  Commission  provides  the  IMF  with  its  country-
specific expertise on structural reform priorities, as well as a set of draft reforms designed 
to support the implementation of the IMF’s conditions. 

Similarly,  MFA  usually  comes  into  play  alongside  other  EU  instruments.  Following 
on  from  the  European  Neighbourhood  Policy  Instrument  (ENPI)  and  the  European 
Neighbourhood  Instrument  (ENI  –  2014-2020),  the  Neighbourhood,  Development  and 
International Cooperation Instrument (NDICI) is to give neighbourhood partner countries 
support during the 2021-2027 period; in accession countries, the relevant instrument is the 
Instrument for Pre-Accession Assistance (IPA). The EU is also active regionally through 
EIB lending.  

2.2 Point(s) of comparison 

As  Section  1  explains,  MFA  operations  are  systematically  evaluated  after  they  end, 
notably  to  assess  (i)  the  impact  of  MFA  on  beneficiary  countries’  economies  and  in 
particular  on  the  sustainability  of  their  external  position  and  (ii)  the  operations’  added 
value. These evaluations are also supposed to provide the following insights into the EU’s 
financial assistance: 

8  The  IMF’s  Financing  Assurances  Policy  requires  a  recipient’s  projected  balance-of-payments  position, 
improved as needed through external assistance, to be compatible with an assumed exchange rate that does 
not  unduly  jeopardise  the  repayment  (repurchase)  of  IMF  support.  The  required  degree  of  certainty  of 
these projections varies with the time horizon. 

11 

 
 
                                                           
 

 

to  what  extent  the  ex  ante  considerations  determining  the  design  and  terms  of  each 
operation  have  been  appropriate,  taking  due  account  of  the  economic,  political  and 
institutional context; 
to  what  extent  the  outcome  of  the  operations  has  achieved  the  objectives  set  in  the 
legislative decisions. 

The  ex  post  evaluations  consist  of  an  ex  post  assessment  of  the  MFA  operations  in 
question,  focusing  on  the  macroeconomic  and  structural  adjustment  measures  that  the 
Commission, on the EU’s behalf, agreed with each beneficiary country in an MoU.  

The  MoU’s  policy  conditions  are  based  on  the  economic  stabilisation  and  reform 
programme endorsed by the authorities of the beneficiary country and are consistent 
with its agreements with the IMF. In line with the division of labour mentioned above 
that  has  evolved  over  time  between  the  Commission  and  the  IMF,  the  MoU  does  not 
include  specific  macroeconomic  indicators  and  targets.  This  is  because  the  Commission 
relies for these on the  IMF’s macro-framework agreed with the country receiving MFA. 
Instead, the MoUs specify that disbursement of MFA is conditional on a satisfactory track 
record  of  implementation  of  the  policy  programme  agreed  between  the  country  and  the 
IMF. 

Given  this  is  the  case,  a  precise  assessment  of  its  standalone  impact  is  difficult  to 
make. As explained above, MFA is an external financial cooperation instrument (serving 
both  economic  and  external  policy  objectives),  implemented  alongside  an  IMF 
intervention and the EU’s regular financial operations, typically as part of a stabilisation 
operation involving other international institutions. This means that the operations’ effects 
on macroeconomic variables over time cannot be attributed solely to MFA. In addition, as 
MFA is  a short-term  crisis-related  instrument  spanning a maximum  of 2 to 3  years, it is 
not possible to quantify its impact in terms of indicators/milestones beyond the timeframe 
of the MFA operations themselves. This makes it difficult to capture the impact of some of 
the medium-term reforms. 

Despite  these  constraints,  past  evaluations  have  traditionally  looked  at  a  certain 
number of macroeconomic indicators. These included: 
 

indicators  for  gauging  external  stability:  developments  of  international  reserves  and 
current account balances;  
indicators  for  gauging  internal  stability:  inflation  levels,  GDP  growth,  government 
structural balances and debt levels. 

 

Besides  the  constraints  outlined  above,  some  of  these  indicators  (such  as  GDP  growth, 
government  debt  levels,  current  account  balances)  also  raise  conceptual  questions. 
Assessing significant impacts on GDP growth and government debt is therefore ambitious 
even  for  a  combined  MFA/IMF  operation,  while  government  structural  balances  are 
beyond  the  scope  of  MFA.  Finally,  current  account  balances  also  behave  peculiarly, 
tending to exhibit some automatic self-correction over the cycle. This makes it difficult to 
assess  and  isolate  the  impact  of  MFA  operations  on  such  indicators  as  their  evolution 
depends on multiple factors. Further analysis may therefore be warranted to fine-tune the 
Guidelines for the Ex Post Evaluation of Macro-financial Assistance Operations that will 
be  included  in  the  new  ECFIN  framework  contracts  for  the  provision  of  evaluation  and 
evaluation-related services.  

12 

 
 
 
3.  HOW HAS THE SITUATION EVOLVED OVER THE EVALUATION PERIOD? 

Since  the  2009  meta-evaluation,  the  main  features  of  the  MFA  instrument  have 
remained unchanged, with some  limited  changes introduced following the adoption 
of  the  Lisbon  Treaty.  This  section  provides  an  overview  of  the  operations  evaluated 
since  2010,  the  applicable  decision-making  procedures,  and  the  amounts  and  financial 
terms  of  the  assistance.  It  also  describes  the  budgetary  arrangements  governing  the 
instrument as well as the conditions and compliance with them.   

MFA operations 2010-2020 

MFA amounts made available to non-EU countries  increased  from  EUR 1.2 billion 
in 2000-2008 to EUR 6.3 billion in 2009-2019. Political and economic upheaval in Arab 
Mediterranean  partner  countries  has  led  to  greater  demand  for  MFA  since  2012,  with 
operations implemented in Jordan and Tunisia. From 2014, the conflict in eastern Ukraine 
and the marked deterioration of the country’s economic and financial situation triggered 
an  unprecedented  level  of  MFA  support.  In  2014-2019,  a  total  of  EUR  4.4  billion  was 
made  available  to  Ukraine  under  four  MFA  operations,  the  largest  amount  of  financial 
assistance ever granted to a non-EU country in such a short period of time. All but two of 
these operations have already been evaluated and are included in the meta-evaluation.  

During  the  2010-2020  period,  the  bulk  of  MFA  funds  (71%)  was  used  to  support 
eastern neighbourhood countries, followed by southern neighbourhood countries (24%), 
(pre)-accession countries and other countries (5%). The bulk of MFA funds was provided 
in loans (95%), the rest in grants (5%).  

Of these 17 MFA operations, in total 15 have been completed and evaluated since the 
last meta-evaluation in 2009/10. 

Table 1: MFA operations evaluated over the 2010-2020 period 

Operation 

Timeframe↓ 

Year of evaluation 

Georgia 06  2006-08 
Kosovo 06  2006-10 
Moldova 07  2007-08 
Lebanon 07  2007-10 
Georgia 09  2009-10 
Serbia 09  2009-11 
2009-13 

Bosnia and 
Herzegovina 09 

Moldova 10  2010-12 
Armenia 09  2011-12 
Kyrgyz Republic 13  2013-16 
Georgia 13  2013-17 
Jordan 13  2014-15 
Ukraine 10 and 14  2014-15 
Tunisia 14  2014-17 
Ukraine 15  2015-18 
All  2006-18 

2010 
2012 
2010 
2012 
2012 
2013 
2015 

2013 
2013 
2019 
2019 
2017 
2017 
2019 
2020 
2010-20 

Many operations were also authorised in the wake of the COVID-19 pandemic, and 
more  recently  in  the  wake  of  Russia’s  unprovoked  and  unjustified  invasion  of 
Ukraine. In response to the pandemic, the Commission adopted a proposal for a EUR 3 

13 

 
 
billion MFA package  for 10 enlargement  and neighbourhood partner  countries in  spring 
2020. This was endorsed by the co-legislators in May 2020. At the time of its adoption, it 
represented the largest volume of MFA the EU ever made available to partner countries at 
a single time. In response to the 2022 invasion of Ukraine, an additional EUR 7.2 billion 
of MFA loans was made available to Ukraine at record speed. The COVID-19 package of 
operations  was  only  fully  completed  by  mid-2022  and  support  for  Ukraine  is  ongoing. 
Both sets of operations therefore remain to be evaluated. They provide valuable insights 
into, for instance, the decision-making procedure used to  initiate the package (discussed 
below).  

As  explained  above,  the  Genval  criteria  applied  to  operations  before  2013  and  the 
Joint Declaration to operations after that, with the following implications.  
  The change of regime increased the possible amount of assistance. The Genval criteria 
set a ceiling for the amount of assistance at 60% of the residual financing gap for (pre-
)  accession  and  association  countries  and  33%  of  that  gap  for  neighbourhood 
countries.  On  the  insistence  of  the  European  Parliament,  the  2013  Joint  Declaration 
removed these ceilings whilst introducing the principles of fair burden-sharing and EU 
value added. However, the Commission has continued to apply the 60%/33% ceilings 
as a rule of thumb, since they were considered to adequately reflect the Declaration’s 
principle of fair burden-sharing. 

  The  2013  Joint  Declaration  showed  a  clear  preference  for  assistance  in  the  form  of 
loans (instead of grants), but the Genval criteria were more neutral in this regard. As 
Table 3 shows, this has resulted in a marked decrease in the share of grants over time. 

Procedures and decision-making 

In  the  absence  of  a  framework  regulation  that  would  attribute  powers  to  the 
Commission,  operations  have  to  be    authorised  by  means  of  decisions  –  acts  of 
secondary  EU  law,  on  the  basis  of  Article  212  TFEU.  Until  31  November  2009,  the 
Council  adopted  such  acts  alone9.  Since  then,  the  ordinary  legislative  procedure  usually 
applies,  with  equal  involvement  of  the  European  Parliament  and  the  Council.  The 
procedure  follows  the  rules  set  out  in  Article  294  TFEU,  as  follows.  The  Commission 
triggers the procedure when it prepares and submits a proposal to the European Parliament 
and  the  Council,  who  reach  their  respective  positions  in  ensuing  internal  deliberations 
before  entering  interinstitutional  negotiations.  In  theory,  the  procedure  could  include  a 
first and second reading in the Parliament and the Council, a conciliation procedure and, 
possibly,  a  third  reading.  However,  in  practice  co-legislators  usually  reach  early 
agreements during the first reading. When the situation in a third country requires urgent 
financial assistance from the EU, Article 213 TFUE allows the Council, acting alone in a 
non-legislative  procedure,  to  adopt  the  necessary  decisions  on  a  proposal  from  the 
Commission. 

Since the adoption of the Joint Declaration, the Commission/ EEAS and its services 
must  complete  a  certain  number  of  steps  to  prepare  the  operation.  On  receiving  a 
potential beneficiary’s request, they carry out an ex ante evaluation to assess the amount 

9 This was also the case for Ukraine 15 (see table), pursuant to Article 213 TFEU. 

14 

 
 
                                                           
of  financing  needed  and  whether  or  not  the  country  in  question  fulfils  the  eligibility 
conditions  for  assistance,  including  the  political  preconditions.  The  Commission  then 
submits  a  proposal  to  the  co-legislators.  After  the  operation  is  authorised  by  the  co-
legislators, an MoU is negotiated and concluded between the beneficiary country and the 
European  Commission,  on  behalf  of  the  EU,  setting  out  the  reform  conditions  for  the 
operation.  The  financial  terms  are  set  out  in  a  loan  agreement  and,  if  the  operation  also 
comprises a grant, a grant agreement. The MoU is adopted as an implementing act under 
comitology, in line with the Joint Declaration. 

Figure 2: Process for granting and providing MFA 

The time lag between request and first disbursement varies a lot. In the 15 operations 
evaluated,  the  average  length  between  the  recipients’  request  for  assistance  and  the  first 
disbursement  was  roughly  28  months  and  the  median  26  months.  However,  since  these 
figures include operations authorised before the entry into force of the Lisbon Treaty and 
during the protracted negotiations on the Commission  proposal  for an MFA  Framework 
Regulation, putting some operations on hold, they do not give an accurate picture of the 
current state of play.  

This  is  why  it  is  necessary  to  adjust  the  sample  of  operations  in  order  to  get  a 
meaningful picture of the lags in the current decision-making procedures. Only 5 of 
the  15  operations  evaluated  took  place  within  the  current  steady  state  (after  the  Joint 
Declaration entered into force in 2013). Excluding the special case of Ukraine (with three 
subsequent and sometimes overlapping operations), this would leave only the Tunisia and 
Jordan operations. It would therefore make more sense to describe the state of play on the 
basis of all operations since 2013. 

On the basis of such an adjusted sample, the following observations can be made (see 
Table 2). 
  Whilst  remaining  somewhat  variable,  the  average/median  overall  period  between  a 
request and the first disbursement was roughly 17 months. In some cases there was an 
improvement over time, as both the EU institutions and beneficiary countries became 
more familiar with the procedures. On the basis of more recent cases since 2020, it can 
be seen that the standard procedures make it possible for the period between a request 
and the first disbursement to be less than 10 months. 

15 

 
 
 
  Thanks to the use of urgency procedures in the European Parliament, the Commission 
proposal for the COVID-19 package was adopted in a record 1 month. The one-time 
use of Article 213 TFEU for urgent cases (Ukraine 2014) made it possible for only the 
Council to decide, also expediting the process. 

  Entry  into  force  of  the  MoU’s  (Step  III)  sometimes  requiring  ratification  by  the 
beneficiary’s parliament was the main drag in the process, but the OLP (Step II) also 
sometimes added to the overall lag between a request and disbursement. 

Table 2: Time and steps (see Figure 3) from MFA request to (first) disbursement 

 Operation↓ 

Albania 20 
Bosnia and Herzegovina 20 
Jordan 20-2 
Kosovo 20 
Moldova 20 
Northern Macedonia 20 
Montenegro 20 
Tunisia 20 
Ukraine 20 
Jordan 20-1* 
Ukraine 18 
Moldova 17 
Jordan 16 
Tunisia 16 
Ukraine 15 
Ukraine 14 
Tunisia 14 
Jordan 13 
Moldova 10 
Ukraine 10 

Median w/o COVID-MFA, w/o Art. 213 
Average w/ COVID-MFA, w/o Art. 213 

Duration in months 

Step I 
0 
0 
0 
0 
0 
0 
0 
0 
1 
2 
3 
17 
4 
6 
4 

3 
5 
12 
8 

4 
3 

Step II 
1 
1 
1 
1 
1 
1 
1 
1 
1 
4 
4 
8 
6 
5 
3 
1 
5 
8 
4 
8 

5 
3 

Step III 
8 
12 
4 
3 
4 
2 
4 
12 
4 
9 
5 
4 
9 
13 
3 
1 
11 
5 
2 
45 

7 
7 

Step IV 
2 
4 
10 
1 
2 
3 
1 
1 
3 
2 
0 
21 
1 
2 
1 
1 
1 
9 
0 
1 

1 
4 

Total 
12 
18 
15 
6 
7 
6 
5 
13 
8 
16 
12 
50 
20 
27 
10 
3 
20 
26 
18 
63 

17 
17 

* Jordan 20-2 was a top-up of Jordan 20-1, part of the COVID-19 package and authorised in a multicountry 
(omnibus) decision. 

Amounts and financial terms 

The  15  operations  evaluated  enabled  around  EUR  4  billion  of  support  to  be  made 
available on very advantageous terms. In total, roughly EUR 3.3 billion were disbursed 
to the beneficiaries10. MFA most often takes the form of concessional medium-/long-term 
loans, the funds for which the Commission borrows on capital markets and lends on to the 
beneficiary country. MFA loans therefore had a typical maturity of around 15 years and 
were  provided  at  the  EU’s  own  borrowing  costs  over  the  same  period  –  the  costs  of  an 
AAA-rated debtor borrowing in their home currency.  

10  As  Table  3  shows,  the  difference  is  mainly  due  to  a  non-disbursement  of  assistance  under  operation 
Ukraine  14.  The  operation  expired  on  13  January  2018  without  the  third  and  final  disbursement  having 
been  released,  given  the  lack  of  sufficient  progress  in  fulfilling  some  of  the  required  conditions 
(Commission Decision EU/2018/405 final). 

16 

 
 
 
 
 
 
 
 
 
  
                                                           
In  some  cases,  MFA  took  the  form  of  grants  financed  by  the  EU  budget,  or  a 
combination of loans and grants, with the share of grants changing over time. Before 
the  adoption  of  the  2013  Joint  Declaration,  MFA  disbursements  tended  to  have  a 
significant  share  of  grants.  However,  reflecting  the  Joint  Declaration’s  stated  preference 
for  loans  and  the  limited  budget  available,  since  2013  MFA  grants  have  been  provided 
only for eligible countries with very high vulnerabilities and comparably low income per 
head.  The  Commission  is  currently  looking  into  whether  or  not  the  methodology  for 
deciding on the provision of grants needs to be updated to maximise their usefulness while 
respecting the limited budget available in the 2021-2027 MFF. 

Table 3: Overview of financial terms of support provided 

Country 
(in chronological order) 

Amount (EUR m) 
authorised   disbursed 

Share of 
grants 

Interest rate 

Maturity 
(months) 

Georgia 06 
Kosovo 06 
Moldova 07 
Lebanon 07 
Georgia 09 
Serbia 09 
Bosnia and  
Herzegovina 09 
Moldova 10 
Armenia 09 
Kyrgyz Republic 13 
Georgia 13 
Jordan 13 
Ukraine 10 and 14 
Tunisia 14 
Ukraine 15 
Total 

33.5 
50 
45 
80 
46 
200 
100 

90 
100 
30 
46 
180 
1000 
300 
1800 
4067 

22 
30 
45 
40 
46 
100 
100 

90 
100 
30 
46 
180 
1000 
300 
1200 
3329 

100% 
100 
100 
38 
50 
65 
0 

100 
25 
50 
50 
0 
0 
0 
0 

Budgetary arrangements 

n/a 
n/a 
n/a 
3.1% 

3.4% 
2.0% 

n/a 
3.4% 
1.0% 
1.0% 
0.8% 
1.4% 
0.9% 
0.7% 

n/a 
n/a 
n/a 
60 
177 
96 
120 

n/a 
179 
176 
175 
176 
156 
161 
131 

As  a  crisis  instrument  and  unlike  other  EU  external  financing  instruments,  MFA 
does not provide financial support on a regular, programmable basis. When provided 
in  the  form  of  loans,  its  budgetary  impact  is  limited  to  the  provisioning  of  the  External 
Action Guarantee in the Common Provisioning Fund financed from geographical budget 
lines  under  NDICI11  to  cover  the  EU  budget  in  the  event  of  potential  defaults.  At  a 
provisioning  rate  of  9%,  this  mechanism  creates  a  significant  multiplier  between 
budgetary expenses and the headline amounts provided12.   

To  streamline  its  external  financing  architecture13,  the  EU  changed  the  budgetary 
set-up for  MFA in 2021.  In the 2021-2027 MFF, MFA  grants continue to  be budgeted 
separately  (under  budget  line  14  20  03  01  Macro-financial  assistance  (MFA)  –  grants). 

11 The  9% provisioning is paid from the EU budget into the  Common Provisioning Fund  after the loan is 

disbursed to the beneficiary country.  

12 To date, no calls on the External Action Guarantee Fund have been recorded for MFA operations. 

13 (European Commission, 2018) 

17 

 
 
 
 
 
 
                                                           
However,  the  provisioning  of  the  guarantee  for  MFA  loans  is  now  part  of  the  budget 
allocated  to  NDICI  in  the  External  Action  Guarantee,  where  the  Regulation  sets  certain 
ceilings for the use of the budget for guarantees  in general.  Previously, MFA operations 
were  provisioned  in  the  Guarantee  Fund  for  External  Action  (GFEA),  which  had  a 
dedicated budget line. The assets constituted in the GFEA have now been transferred to 
the Common Provisioning Fund. 

In  recent  years,  the  EU  has  increased  the  budgetary  means  available  for  the 
instrument. Faced with increasing demand for MFA support in the context of the 2014-
2020  mid-term  review  of  the  MFF,  the  Commission  proposed  (and  co-legislators 
endorsed) an increase in the volume of lending capacity from EUR 0.5 billion to EUR 2 
billion annually for the remainder of the programming period. Similarly, the Commission 
proposed  that  the  volume  of  MFA  lending  remain  consistent  with  these  agreed  yearly 
volumes – at EUR 14 billion – in preparation for the 2021-2027 MFF14. During the NDICI 
negotiation process, the overall volume of guaranteed operations  was reduced compared 
to the Commission proposal, with implications for the MFA budget as well.  The adopted 
NDICI Regulation does not feature a specific envelope for MFA loans under the External 
Action  Guarantee.  However,  as  result  of  the  overall  ceiling  for  the  external  action 
guarantee  established  in  the  adopted  NDICI  Regulation,  the  maximum  volume  of  MFA 
loans  was  fixed  at  EUR  11.6  billion.  In  addition,  outside  of  the  NDICI  Regulation,  a 
budget  of  close  to  EUR  350  million  is  envisaged  for  MFA  grants.  The  budget  for 
provisioning  the External  Action Guarantee  is  EUR 10 billion, covering  European  Fund 
for Sustainable Development+ operations, including MFA operations and Euratom loans 
granted during the period 2021-27.   

MFA conditions and compliance 

The  15  operations  that  form  the  basis  of  the  meta-evaluation  contained  a  total  of  185 
conditions, with compliance varying per operation as follows. 

Table 4: MFA conditions compliance rate by operation 

 Operations↓   

Armenia 09  
Bosnia and Herzegovina 09  

Fully 
compliant  
 78%  
 63%  

Partly 
compliant  
22%  
38%  

Georgia 06 
Georgia 09  
Georgia 13  
Jordan 13  
Kosovo 06 
Kyrgyz Republic 13  
Lebanon 07*  

  91%  
100%  
100%  
 78%  
100%  
100%  
 29%  

 9%  
 0%  
 0%  
11%  
 0%  
 0%  
43%  

Not compliant  

 0%  
 0%  

 0%  
 0%  
 0%  
11%  
 0%  
 0%  
29%  

of 

Total 
conditions  
9  
8  

11  
6  
8  
9  
7  
7  
7  

14 The Commission Proposal (COM(2018) 460 final) states: […] the External Action Guarantee will support 
EFSD+  operations,  macro-financial  assistance  and  loans  to  third  countries  on  the  basis  of  Council 
Decision 77/270/Euratom, for a proposed amount of up to EUR 60 billion, out of which EUR 14 billion 
are envisaged to cover macro-financial assistance loans. 

18 

 
 
                                                           
Moldova 07 
Moldova 10  
Serbia 09*  
Tunisia 14  
Ukraine 10  
Ukraine 14  
Ukraine 15  

100%  
 70%  
100%  
 67%  
 69%  
 82%  
 81%  

 0%  
30%  
 0%  
22%  
24%  
 9%  
 6%  

 0%  
 0%  
 0%  
11%  
 8%  
 9%  
14%  

12  
10  
9  
9  
26  
11  
36  

* Given the non-disbursement of the second tranche, compliance was not officially recognised. 

The  large  majority  of  conditions  covered  budgetary  reforms,  including  public 
financial management reforms. Almost a quarter of all conditions related to this policy 
area,  followed  by  internal  and  external  auditing  (14%),  social  and  healthcare  policies 
(11%) and public procurement (10%). 

Chart 1: Distribution of MFA conditions across policy areas 

Finance, Investment & Business environment

Public procurement

Energy sector

Social and Healthcare policies

Public Financial Management & Budget*

Public Administration & Statistics

Privatisation & state-owned enterprises

Auditing (internal and external)

Trade & Competition

Anti-corruption

Taxation

9% 

10% 

7% 

11% 

23% 

6% 

3% 

14% 

9% 

4% 

5% 

Anti-corruption

Trade & 
Competition

Auditing (internal 
and external)

Privatisation & state-
owned enterprises

Taxation

Finance, Investment 
& Business 
environment

Public procurement

Energy sector

Social and 
Healthcare policies

Public 
Administration & 
Statistics

Public Financial 
Management & 
Budget*

*Covers  the  financial  management  of  the  government,  for  example  fiscal  impact  analysis,  and  everything  related  to  the 
government budget. It also includes monetary policy and the organisation of the central bank, which is actually a separate 
topic but is taken into account here because there are too few reforms in this monetary policy area alone. 

Evaluations and methodology applied 

The ex post evaluations of MFA have drawn on the 2010 and 2015 Guidelines for the 
Ex  Post  Evaluation  of  Macro-financial  assistance  Operations  drawn  up  by  DG 
ECFIN. The  guidelines  take account  of the very specific nature of MFA  operations and 
the resulting challenges for assessing them. The 2015 Guidelines ushered in a number of 
changes in the way the ‘core questions’ for MFA ex post evaluations were formulated. The 
2010 Guidelines formulated the core questions in terms of how effective MFA had been in 
relation  to  macroeconomic  stabilisation,  supporting  structural  reform,  and  returning 
external debt to a sustainable path,  as well as examining indirect/unexpected effects and 
how  the  MFA’s  design  affected  its  effectiveness  or  efficiency.  The  2015  Guidelines 
adopted  a  different  approach,  focusing  on  the  broader  Better  Regulation  themes  of 
relevance, efficiency, effectiveness, coherence and EU added value. Of the 15 evaluations 

19 

 
 
 
 
that form the basis of this meta-evaluation, 8 were done using the 2010 Guidelines and 7 
done using the 2015 Guidelines.    

Whilst  all  the  ex  post  evaluations  examined  the  contribution  of  MFA  to  short-term 
macroeconomic  stabilisation,  the  impact  on  structural  reforms,  and  various  design 
and implementation issues, the precise mix of methodologies to do so varied from one 
evaluation to the next. In particular, there was a significant change between before and 
after 2015 in the  evaluations’ suggested approach to counterfactual analysis. Until 2015, 
an essentially  econometric modelling approach was suggested in  the ECFIN  Guidelines. 
However, it was judged that the economic models used for MFA did not produce reliable 
results. Since 2015, counterfactual analysis has therefore been carried out using relevant 
quantitative and qualitative techniques (Delphi technique and focus groups). 

4.  EVALUATION FINDINGS (ANALYTICAL PART) 

4.1 To what extent was the intervention successful and why?  

In  line  with  the  Better  Regulations  Guidelines,  the  performance  of  MFA  should  be 
analysed on the basis of its effectiveness, coherence and efficiency. Applied to MFA,  
  “effectiveness” describes the instrument’s track record in meeting its stated objectives, 
i.e.  contributing  to  macroeconomic  stabilisation  and  promoting  implementation  of 
important structural reforms; 

  “coherence” depicts the extent to which:  

o  MFA  operations  support  the  Union’s  own  broader  objectives  in  its  external 

policies (directly and indirectly) -> “external coherence”, 

o  a  common  framework  of  rules,  policies  and  practices  supports  the  instrument’s 

overall good and consistent functioning -> “internal coherence”: and 

  “efficiency”  relates  to  the  ambition  to  deliver  in  the  aforementioned  dimensions  at 
minimal  costs  to  the  overall  resources  available  to  the  Union  and  the  beneficiary 
country. 

Effectiveness in stabilising macroeconomic developments 

in  stabilising  beneficiaries’ 

MFA  has  generally  been  effective 
(external) 
macroeconomic  environment,  especially  in  the  short  and  medium  term.  Besides  the 
specific/individual  assessments  in  the  15  evaluations  previously  mentioned,  hard  data 
back this statement up in the following areas. 
  Despite  a  challenging  external  environment  at  the  start  of  the  respective  MFA 
operations, market access could be maintained or eventually regained in all cases. This 
includes one beneficiary’s (Ukraine’s) restructuring its debt in good faith negotiations 
with  private  bondholders  in  2015.  With  the  exception  of  one  strategic  default  (by 
Ukraine  on  a  Eurobond  of  EUR  3  billion  held  by  the  Russian  Federation  that  the 
Ukrainian authorities consider illegitimate), beneficiaries did not default. 

  Reserve adequacy and current accounts mostly improved during MFA operations, and 

inflation subsided.  

20 

 
 
 
Table 5: Selected macroeconomic indicators before/during/after MFA operations 

Current account 
balance (% GDP)  

 Inflation (CPI) 

                Foreign exchange reserves       

(in months of imports) 

Before  

MFA   After   Before   MFA   After   Before   MFA   After  

Georgia 06 
Kosovo 06 
Moldova 07 
Lebanon 07 
Bosnia and 
Herzegovina 09 
Georgia 09 
Serbia 09 
Moldova 10 
Armenia 09 
Georgia 13  
Kyrgyzstan 13 
Jordan 13 
Tunisia I 14 
Ukraine 10 and 
14 
Ukraine 15 

-8.1 
-7.9 
-7.9 
-13.3 
-12.9 

-19 
-15 
-7.9 
-14.8 
-12.3 
-8.5 
-12 
-8.2 
-7.9 

-15.6  
-9.2  
-13.1  
-16.8  
-7.2  

-10.4  
-6.7  
-8.1  
-10.2  
-10.3  
-14.6  
-8.1  
-9.8  
-1.1  

-12.3 
-7.5 
-0.3 
-23.2 
-5.2 

-10.2 
-7.4 
-5.7 
-5.8 
-6.5 
-8.3 
-1.3 
0.4 
-2.3 

-7.1 

-0.6  

-0.5 

9.5 
-0.7 
12.3 
1.4 
5 

9.5 
9.7 
8.3 
6.6 
4.9 
9.1 
4.5 
4.4 
2.7 

4.1 

9.4  
3.1  
12.5  
4.9  
1.5  

4.4  
8.5  
6.5  
5.1  
2.9  
5.2  
0.9  
4.5  
30.3  

5.8 
3.9 
5 
5.5 
-1.2 

2.3 
8.7 
6.4 
4.2 
4.2 
2 
2.5 
6.6 
13.1 

25.7  

7.3 

1.7 
4.6 
3.0 
11.5 
4.5 

1.7 
4.7 
7.9 
4.4 
3.3 
3.7 
6.6 
3.2 
2.3 

3.1 
2.9 
5.1 
16.6 
5.9 

3.1 
7.0 
5.2 
4.5 
3.2 
5.0 
8.3 
3.2 
3.1 

3.2 
2.5 
2.1 
17.7 
7.3 

3.2 
5.2 
5.2 
4.2 
3.2 
4.2 
8.0 
3.8 
3.5 

Note.  Before  – average of the 3 years  before  MFA.  MFA – average  during  MFA. After  –  average of  the 3  years after 
MFA.  
Source: IMF (April 2021), Worldbank 

These  findings  are  more  nuanced  for  timeframes  longer  than  the  medium  term, 
based on the external study. Some improvements in current account balances are visible 
during that time, but gains in reserve adequacy could usually not be extended further and 
current accounts remained in deficit (a few years after the completion of MFA operations). 
Participants at the validation workshop tended to confirm this interpretation of the data. 

However,  these  conclusions,  especially  insofar  as  they  concern  current  account 
balances,  need  to  be  assessed  carefully.  There  is  no  consensus  among  professional 
economists  on  the  right  current  account  position  for  a  given  country.  Clearly,  not  all 
countries  can  reduce  their  current  account  deficits  at  the  same  time  or  consistently  run 
current  account  surpluses.  Moreover,  for  potential  beneficiaries  of  MFA,  universally  in 
their  economic  transition,  standard  economic  theory  would  suggest  from  a  normative 
perspective  that  surpluses  are  undesirable,  whilst  the  size  of  sustainable  current  account 
deficits depends on a variety of country- and time-specific factors.15  
In  any  event,  it  must  again  be  stressed  that  the  beneficial  short-to-medium-term 
effects outlined above cannot be attributed to MFA alone, given its relatively limited 
size  and given  its operations  are undertaken in  conjunction  with  IMF operations.  In this 
vein, several stakeholders (including stakeholders at the validation workshop) underlined 
that  concomitance  with  IMF  operations  has  been  essential  in  producing  the  results 

15This is why analysts such as IMF staff and Commission services calculate structural – meaning sustainable 
current account – deficits for the economies of their interest, controlling for a certain set of econometric 
determinants (Phillips, et al., 2013).  

21 

 
 
 
 
 
                                                           
mentioned  before.  Interviewees  from  EU  institutions  have  also  pointed  out  that  the 
complementarity  of  expertise  and  policy  conditions  has  made  the  overall  stabilisation 
operation  more  impactful.  Interviews  with  stakeholders  also  indicate  that  cooperation 
between the EU, the IMF and the World Bank has increased political leverage for the EU 
to push for reforms. 

Table 6: Achievement of macroeconomic stabilisation according to the evaluation reports 

Effective for 
macroeconomi
c stabilisation  

If not, what 
was the 
justification 
for failure   

Armenia 09   Yes  

Not much  

Bosnia and Herzegovina 
09  
Georgia 06   Yes  
Georgia 09   Not much  
Georgia 13   Yes  
Jordan 13   Yes  
Kosovo 06  Yes  
Kyrgyz Republic 13   Yes  

Lebanon 07   Not much  
Moldova 07   Yes  
Moldova 10  Marginal  
Serbia 09   Marginal  

Tunisia 14   Not much  

Ukraine 10 and 14   Yes  
Ukraine 15   Yes  

Small size   

Small size   

Small size   

Small size   
External 
context  
Small size   

MFA 
size (% 
of 
current 
account 
deficit in 
the year 
before)  
11  
6  

6  
2  
3  
7  
20  
4  
5  
15  
26  
2.8  

9  
10  
10  

MFA size 
(% of 
GDP in 
the year 
before)  

Was 
macroeconomic 
stabilisation 
achieved 
through the 
combined effect 
of MFA and IMF 
support?  

1.4  
0.8  

0.6  
0.5  
0.4  
0.7  
1.7  
0.6  
0.5  
1.7  
2.3  
0.6  

0.9  
1.2  
1.0  

Yes  
Yes  

Yes  
Yes  
Yes  
Yes  
Yes  
Yes  
Yes  
Yes  
Yes  
Yes  

Yes  
Yes  
Yes  

At the same time, it appears that concomitance with IMF support can at times result 
in implementation difficulties, such as disbursement delays. The external study says that 
interviewees and some OPC respondents expressed this concern. Some even advocated a 
review  of  the  Joint  Declaration’s  Principle  (6)  on  this  requirement.  Some  stakeholders 
also  argued  that  the  scope  of  the  EU’s  leverage  in  an  economic  adjustment  programme 
predefined  by  the  IMF  was  not  clear.  In  this  context,  the  study  reports  that  some 
stakeholders in IFIs wondered whether the EU could be more influential, also considering 
the amount of financial capacity there is in MFA. Others expressed concern that the close 
link to  IMF support could  be detrimental to the perception of the EU as an autonomous 
global actor16.  

Effectiveness in incentivising reforms 

In contrast to the obligatory concomitance with IMF support, the link with political 
preconditions17 appears to be of less concern. In this regard, the external study focuses 

16 (European Parliament, 2012). 

17 A precondition is that the beneficiary respects effective  democratic  mechanisms, including a  multiparty 

parliamentary system and the rule of law, and that respect for human rights is guaranteed. 

22 

 
 
   
  
  
   
   
   
   
   
   
   
   
   
                                                           
interrupted  due 

to  non-compliance.  This 

on one case (Moldova) in which their implementation caused some difficulties as the MFA 
in  a 
operation  was 
desynchronisation  with  the  continuing  IMF  operation.  But  otherwise,  there  is  little 
reference in ex post evaluation reports to a (negative) impact of the political precondition 
on the outcomes, effectiveness and efficiency of MFA operations. OPC respondents also 
expressed little criticism in this regard, possibly because MFA is such an integral part of 
the EU’s external policy framework. For instance, the French senate confirmed this view 
in  a  2017  report  recommending  a  more  prominent  and  explicit  role  for  the  political 
preconditions  by including them, as well as the  consequences of non-compliance, in  the 
MoU (Marc, 2017). 

turn  resulted 

in 

That  said,  the  external  study  sees  some  room  for  improvement  with  regard  to  the 
way  the  political  preconditions  are  assessed  in  practice.  It  seems  that  ensuring 
compliance is at times very challenging, for two main reasons. 
1.  Countries that benefit from MFA have different political systems. As a result, political 
preconditions  might  be  more  easily  and  swiftly  fulfilled  in  some  countries  than  in 
others.  

2.  As  several  stakeholders  in  EU  institutions  and  IFIs  acknowledged,  the  political 
preconditions  are  rarely  specified. This  makes  it  hard  to  assess  their  fulfilment,  also 
because different Commission services/the EEAS may differ in their assessment.  

Like  the  link  with  political  preconditions,  the  requirement  to  implement  structural 
reforms  enjoys  relatively  strong  support  among  stakeholders.  Furthermore,  previous 
evaluations  found  that  81%  of  the  requirements  had  been  fully  met,  even  if  compliance 
rates  differed  between  operations  and  targeted  sectors.  Despite  this  broadly  strong  track 
record, key stakeholders noted the prominent example of Ukraine (15),  where  excessive 
ambition  led  to  a  proliferation  of  requirements  competing  with  each  other  in  terms  of 
implementation resources and political capital. The external study also identifies the 2014-
2017 operation in Tunisia as having suffered from the same problem. 

Table 7: MFA conditions compliance rate by thematic groups 

Policy areas  

Finance, Investment & 
Business environment  
Public procurement  
Energy sector  
Social and Healthcare 
policies  
Public Financial 
Management & Budget  
Public Administration & 
Statistics  
Privatisation & state-
owned enterprises  
Auditing (internal and 
external)  
Trade & Competition  
Anti-corruption   
Taxation  
Total  

Total number 
of conditions  
17  

Share of  
fully compliant 
94%  

Share of  
partly compliant 
 6%  

Share of  
not compliant  
 0%  

 6%  
 8%  
10%  

11%  

18%  

 0%  

20%  

13%  
14%  
50%  
14%  

 0%  
 0%  
 5%  

 5%  

 0%  

20%  

 4%  

13%  
29%  
10%  
 5%  

18  
13  
20  

42  

11  

5  

26  

16  
7  
10  

94%  
92%  
85%  

83%  

82%  

80%  

77%  

75%  
57%  
40%  
81%  

23 

 
 
 
  
Given the high compliance rates, the external study finds little conclusive evidence of 
backtracking. What evidence is there is mainly in the area of the fight against corruption 
and in cases in which there had already been problems during implementation.  

These  results  are  all  the  more  noteworthy  given  previous  analyses  in  this  area.  In 
2007, the Independent Evaluation Office of the IMF found a compliance rate of just 54% 
in  the  IMF’s financial  arrangements,  with  higher compliance by traditional  interlocutors 
(finance  ministries  and  central  banks)  in  its  core  area  of  expertise  (IEO,  2007).  In 
comparison  with  this  benchmark,  MFA  operations  appear  to  be  more  successful  in 
achieving implementation of the requested reforms. 

Several features support the effectiveness of MFA in incentivising meaningful reform 
based on the external study 

  The  instrument’s  geographical  focus  played  a  supportive  role.  Interviewees 
suggested  that  the  EU’s  strong  political  ties  with  countries  in  its  direct  vicinity  help 
ensure  that  the  MFA  conditions  are  met  and  that  the  instrument  achieves  its  goals. 
Conversely,  interviewees  stated  that  fewer  political  and  economic  connections  result 
in significantly lower leverage and that this potentially reduces the impact of the MFA. 
A majority of the OPC respondents (90%) likewise argued that Principle 2 of the Joint 
Declaration  in  its  current  formulation  suits  objectives  of  the  MFA  instrument  either 
very well or rather well.There is some evidence that the high share of loans rather 
than  grants  can  support  the  implementation  of  some  types  of  reforms.  This  is 
underpinned by the theoretical notion that a beneficiary country has more incentive to 
implement revenue-raising reforms because they improve its repayment capacity18. In 
this  context,  some  shareholders  support  the  Joint  Declaration’s  Principle  3  and  its 
emphasis on loans.  

  Similarly, there is broad support among interviewees from the EU institutions for 
the standard availability period of 2.5 years. This is considered sufficient to allow 
ambitious  reform  programmes  to  be  implemented  and  has  the  benefit  of  allowing 
alignment  with  IMF  arrangements  (typically  extending  over  3  years).  A  shorter 
availability period could also mitigate the risk of a change of government and hence 
the  loss  of  political  will  to  implement  a  given  reform  programme,  even  if 
parliamentary ratification should to some extent help to avert such a scenario. 

  The  effectiveness  of  the  MFA  operations  arguably  stems  at  least  in  part  from  the 
political  leverage  they  generate.  This  contributes  to  a  more  comprehensive  reform 
effort that typically reinforces the effectiveness of the individual reforms undertaken.  

Conversely, the external study also highlighted some features of the instrument that 
tend to reduce its effectiveness. 

  The 

long  procedure  for  authorising  MFA  operations  undermines  their 
effectiveness. As indicated in Section 3 above, the typical time lag between a recipient 
country’s  request  for  assistance  and  the  first  disbursement  was  roughly  17  months, 
including 5 months for concluding the OLP approval process in the EU and sometimes 
including ratification by national parliaments. In particular, the time lag until the MoU 

18 (Klein, et al., 2005) 

24 

 
 
                                                           
(and  therefore  the  MFA)  enters  into  force  may  allow  the  economic  situation  to 
deteriorate further in the absence of full external funding support in cases where such 
a  risk  is  typically  mitigated  by  the  timing  of  the  underlying  IMF  assistance. 
Furthermore,  the  external  study  points  to  one  case  (i.e. Armenia  09),  which  was  in 
urgent need of financial assistance and chose to seek front-loading from the IMF given 
MFA’s  lengthy  procedures.19  In  this  context,  only  half  of  the  OPC  respondents 
consider the procedure for granting MFA (Principle 7) to be very suitable. 

  Notwithstanding  the  relatively  high  compliance  rates,  the  MoU’s  inflexibility 
appears to have reduced MFA’s effectiveness in times of change. MFA instalments 
are  disbursed  in  full  or not  at  all  –  unlike  EU  budget  support,  which  is  disbursed  in 
increments as and when conditions are fulfilled. This is in line with IMF policies but, 
unlike  those  policies,  MFA  practice  is  that  policy  conditions  are  fixed  at  the 
operation’s inception and remain unchanged once the national parliament has ratified 
the text and the MFA has entered into force. The MoU may be amended upon mutual 
agreement,  but  the  time  that  would  be  needed  to  complete  the  respective  approval 
processes  within  the  EU  and  the  beneficiary  country  makes  it  almost  impossible  for 
the  amendment  to  enter  into  force  before  the  end  of  the  availability  period. 
Furthermore, 
to  reform 
implementation, MFA does not allow for partial disbursements. Flexibility is therefore 
limited  to  waivers  that  exceptionally  allow  disbursements  to  take  place  despite  non-
fulfilment of one or several conditions, but new conditions cannot be added  ‘ad hoc’  
in order to reflect changes in the economic environment or a change caused by other 
drivers to the priority or assessment of necessary policy reforms. 

to  encourage  a  comprehensive  approach 

in  order 

  Assuming all else is equal, (small) volumes also affect effectiveness, as the case of 
Kosovo shows20.  The  corresponding  ex  post  evaluation  report  from  2012 stated  that 
the  overall  contribution  of  MFA 
to  macro-fiscal  stability  and  structural 
reforms was limited  due  to  a  number  of  factors  such  as  its  volume.  A  total  of 
EUR 30 million  (0.6%  of  Kosovo’s  GDP)  was  disbursed.  Given  the relatively small 
amount disbursed, the ‘real financial difference’ was moderate and so was the impact 
of the structural conditions linked to the MFA. The reference points for the amounts of 
the assistance (Principle 5) therefore appear to be somewhat controversial, with 70% 
of respondents judging them to be only ‘somewhat’ or ‘not very’ suitable. 

In  any  event,  ownership  of  the  agreed  reforms  is  of  critical  importance  for  the 
MFA’s  effectiveness  in  promoting  important  reforms.  This  is  a  consistent  finding  of 
analyses  of  the  reform-based  financial  assistance  and  is  also  reflected  in  the  external 
evaluation21. Some factors (i.e. the lags in decision-making, and the relevance of reforms) 
therefore  have  an  impact  on  the  success  of  MFA  operations  through  their  impact  on 
ownership.  The  external  study  similarly  links  cases  where  reform  requirements  lacked 

19  This  particular  case  dates  from  before  the  Lisbon  Treaty  introduced  the  requirement  to  apply  the  OLP 
approval  procedure.  This  nevertheless  case  demonstrates  that  delays  (irrespective  of  their  cause)  can 
inhibit MFA’s effectiveness. 

20 (Ecorys, Wiener Institut für Internationale Wirtschaftsvergleiche, Case, 2012) 

21 Including (Boughton, et al., 2002), (Johnson, 2005). 

25 

 
 
                                                           
focus and were overly broad (Ukraine 10, 14) to  a loss of ownership by  the beneficiary 
country  and  therefore  its  compliance,  which  in  turn  reduces  the  MFA’s  effectiveness  in 
promoting reforms. 

Additional  analysis  is  required  in  order  to  properly  judge  how  the  choice  of  policy 
conditionality  affected  the  MFA’s  effectiveness.  The  country-specific  ex  post 
evaluations  as  well  as  stakeholder  consultation  generally  indicate  that  the  policy  areas 
covered  by  the  MFA  conditions  are  relevant.  However,  the  methodology  for  assessing 
whether  the  reform  areas  chosen  were  well-targeted,  well-sequenced  and  sufficiently 
SMART  (Specific,  Measurable,  Attainable,  Realistic  and  Time-bound)  to  support  the 
MFA’s overall objectives needs to be refined. Such analysis would not be straightforward, 
however, given the wide scope of the economic and structural MFA objectives on the one 
hand  and  the  instrument’s  contribution  to  the  EU’s  broader  objectives  (as  reflected,  for 
example, in the political preconditions) on the other. Further efforts to explore this issue 
could be useful.  

Efficiency: 

The external evaluation confirms that MFA is a relatively (cost-)efficient instrument 
for the EU. As explained above and in line with the provisioning policies of the External 
Action Guarantee (Article 31, NDICIR) and its predecessor the External Action Guarantee 
Fund,  provisions  are  set  for  MFA  loans  at  9%  of  the  principal  amount  to  be  disbursed. 
Given  that  no  beneficiary  has  defaulted  to  date  on  an  MFA  loan,  the  related  external 
guarantee provisioning has been routinely returned to the EU budget.  

Some  of  the  principles  mentioned  above  further  support  this  high  cost-efficiency. 
Concomitance with IMF interventions provides some assurance concerning the country’s 
repayment  capacity  (European  Commission,  2021)  because  the  IMF  does  not  usually 
provide  funding  to  countries  with  an  unsustainable  debt  (unless  it  is  taking  measures  to 
address  this  issue).  In  addition,  the  synergies  arising  from  relying  on  the  IMF’s  work 
generate important cost-savings on the EU side. The geographical focus similarly creates 
synergies  in  terms  of  expertise  and  human  resources  between  and  within  different 
Commission departments. 

MFA  is  also  cost-efficient  for  the  beneficiary  countries  because  its  loans  are  highly 
concessional. The  EU’s  high  credit  rating  allows  it  to  borrow  money at much  lower 
interest rates than beneficiary countries. The EU passes on these interest rates in its loan 
agreements,  so  the  beneficiary  countries  receive  a  financial  benefit  that  is  often 
substantial. This benefit can be expressed as the difference between the nominal value of a 
credit (on the basis of market rates) and the present value of the debt service on the date of 
disbursement  (calculated  at  a  discount  rate  applicable  to  the  currency  of  the  transaction 
and expressed as a percentage of the nominal). 22  

22 (OECD - DAC Working Party on Development Finance Statistics, 2018) 

26 

 
 
 
 
                                                           
Table 8: Concessionality of selected operations under the COVID-19 MFA package* 

Country  

Interest rate  

Amount (EUR)  

Concessionality (%)  

Kosovo  
North Macedonia  
Montenegro  
Jordan  
Georgia  
Moldova  
Ukraine  
* The COVID-19 MFAs are cited as an example here because they consisted entirely of loans. 

          50 000 000   
          80 000 000   
       30 000 000   
    150 000 000   
       75 000 000   
      50 000 000   
       600 000 000   

0.125%  
0.125%  
0.125%  
0.125%  
0.125%  
0.125%  
0.125%  

61.8%  
56.3%  
56.3%  
61.5%  
61.5%  
61.5%  
61.4%  

The  current  EU  decision-making  arrangements  can  cause  some  inefficiencies.  By 
reaching  agreement  on  a  template  for  MFA  decisions  as  a  follow-up  to  the  2013  Joint 
Declaration, the EU’s institutions have revealed relatively little divergence in terms of the 
substance of MFA operations during the OLP approval negotiations. In the case of MFA, 
the  main  purpose  of  legislative  negotiations  (to  resolve  disagreement)  was  therefore 
largely absent, but the co-legislators were still required to go through the process required 
for a standard OLP negotiation. The co-legislators generally needed 5 months to adopt the 
Commission’s  proposal  for  MFA  operations  under  review.  In  addition  and  before  MFA 
enters  into  force,  the  MoU  has  to  be  formally  agreed  (see  Step  III  in  Table  2)  –  which 
requires a Commission implementing act subject to Comitology (which is also generally 
used  to  decide  on  the  EU’s  budget  support  outside  of  emergencies).  It  took  roughly  7 
months to conclude the negotiations with the beneficiary country and adopt the MoU, but 
this can take less time if the negotiations are conducted largely in parallel with the OLP 
process.  Besides,  recent  experience  with  the  COVID-19  MFA  package  and  with  MFA 
support  to  Ukraine  in  the  context  of  Russia’s  unjustified  invasion  of  the  country  has 
shown that MFA’s current set-up can allow very swift adoption if all institutions agree to 
make full use of the available procedural flexibilities. 

Questions  can  also  be  raised  over  the  efficiency  of  the  decision-making  process  in 
beneficiary countries, which often also ratify the MoUs. This can create a stronger sense 
of ownership of the reforms to be implemented, but tends to delay the disbursement of the 
MFA and all but prevents MoUs’ subsequent adaptation (as discussed above). In addition, 
national  ratification  is  not  common  practice  for  IMF  programme  documents  or  the 
conditions attached to the EU’s budget support under ENPI/NDICI or IPA III.  

Coherence: 

1.  Coherence with other EU instruments 

As  indicated  above,  MFA  usually  intervenes  in  tandem  with  other  EU  instruments, 
such  as  currently  NDICI  and  IPA.  This  results  in  significant  synergies,  chiefly  with 
budget support under these instruments. As Table 7 shows, MFA conditions often include 
reforms  related  to  PFM/budget  and  auditing.  These  areas  are  also  critical  to  the 
effectiveness  of  the  budget  support  operations  under  these  instruments,  where  the  MFA 
operation  requires  implementation  of  all  conditions  before  disbursement  can  take  place, 
thus strengthening the incentives to carry out comprehensive reforms in these areas. MFA 
conditions therefore tend to be closely dovetailed with budget support conditions so that 
they can reinforce and complement them as much as possible. 

27 

 
 
In order to underpin such coherence with other instruments, coordination between 
Commission services (DG ECFIN, DG NEAR, DG INTPA and others) takes place on 
an  ongoing  basis.  This  reflects  the  need  to  address  specific  issues  related  to  the  MFA 
conditions, insofar as they are under the responsibility of the respective DGs. 

The  co-existence  of  intervention  through  MFA  with  intervention  through  other 
instruments  such  as  ENPI  has  nevertheless  produced  some  issues  at  times.  For 
instance,  the  ex  post  evaluation  of  Moldova  07 found  that  it  had  resulted  in  modest 
additional  administrative  costs  because  two  steering  committees  and  two  reporting 
mechanisms  were  run  at  the  same  time.  It  also  found  that  MFA  and  ENPI  policy 
conditions  had  overlapped,  and  that  this  had  resulted  in  some  synergies  but  also  had 
caused some ‘confusion’. The ex post evaluation does not specify the consequences of this 
confusion,  but  the  external  study  carried  out  for  this  meta-evaluation  argues  that  the 
overlap of conditions probably made it harder for both the EU and the beneficiary country 
to keep track of the reforms. 

By contrast, the example of Armenia stands out as an illustration of how a high level 
of  coherence  can  be  achieved  and  make  a  real  difference.  According  to  the 
corresponding evaluation23, the MoU sought to ensure consistency with the policy lines of 
the  IMF-supported  economic  adjustment  programme  and  the  priorities  of  the  ENPI  and 
the  Eastern  Partnership.  Alongside  the  ENPI,  MFA  conditions  targeted  socio-economic 
reforms and sustainable development, and supported good economic governance, efficient 
public  spending,  trade  and  investment.  Reforms  were  selected  which  were  likely  to  be 
achievable  during  the  MFA  operation  and  which  non-governmental  national  and 
international  stakeholders  also  identified  as  priority  issues.  In  fact,  those  reforms  also 
prepared  the  country  to  meet  preconditions  for  the  DCFTA  negotiations  between  May 
2012 and July 2013. 

All in all, MFA’s coherence with other EU instruments appears to be good.  

  As the stakeholders confirmed, the objectives, design and implementation of MFA are 
complementary  and  consistent  with  other  initiatives  providing  financial  support  to 
beneficiary  countries.  MFA  complements  the  budget  support  programmes  in  the 
beneficiary countries. The two instruments’ conditions are aligned as far as possible.  

  There appears to be some potential for the MFA to provide additional benefit to other 

EU external policies, chiefly in conjunction with budget support operations.  

It should nevertheless be noted that there are limits to the level of coherence that the 
MFA  can  deliver  without  losing  its  stand-alone  rationale  and  nature.    As  the  Joint 
Declaration  explicitly  states,  MFA  is  not  an  instrument  for  long-term  (‘development’) 
financing. It has a specific place and role in the EU’s external policy toolkit of focusing 
on relatively short-term  and repayable balance of payment support with  macroeconomic 
significance.  Coherence  must therefore not  be understood as  replicating the intervention 
model  of  budget  support  because  that  might  compromise  the  complementarity  of  those 
instruments with each other.  

23 (ICF International, GHK, Cambridge Economics, 2013) 

28 

 
 
 
                                                           
2.  Coherence with the interventions of international partners 

Another  dimension  of  the  MFA’s  coherence  with  other  instruments  concerns 
coordination between the Commission and the IMF. Synchronisation with the parallel 
IMF  reviews  was  sometimes  challenging  and  some  conditions  overlapped.  That  said, 
these  coordination  difficulties  did  not  undermine  the  overall  success  of  MFA 24.  Close 
coordination  with  the  IMF  and  occasional  interlinking  with  interventions  by  the  World 
Bank  enabled  the  Commission  to  draw  on  the  expertise  of  these  organisations. 
Coordination  has  therefore  been  reported  by  the  stakeholders  as  ‘good’  between  MFA, 
other EU support, the IMF and other sponsors. Complementarity between MFA and IMF 
support resulted in a joint impact that exceeded what both initiatives could have achieved 
alone.  

The stakeholder consultation shows that external coherence with IFI interventions is 
generally  seen  as  high  and  to  be  supporting  the  operation  of  the  instrument. 
Stakeholders mainly referred to complementarity and the absence of overlap between the 
conditions of the different actors once their different interests, mandates and approaches 
had been taken into account.  

Regarding  internal  coherence,  stakeholder  feedback  suggests  that  MFA  has  been 
consistently  and  clearly  developed,  so  that  its  implementation  follows  the  same  set  of 
principles and rules in all beneficiary countries. This internal coherence is founded on the 
Joint  Declaration,  which  provides  a  common  framework  despite  the  lack  of  a  dedicated 
legal act. 

However,  when  assessing  Better  Regulation  criteria,  the  external  study  finds  some 
variation  in  the  outcome  of  individual  operations.  This  variation  may  reflect  specific 
country  and/or  time  factors,  but  could  also  indicate  that  further  standardisation  work  is 
needed. 

When effectiveness,  coherence and efficiency are all taken  into account, the overall 
assessment  of  the  MFA  instrument’s  success  over  the  last  decade  is  positive.  The 
analysed  evidence  suggests  that  MFA  helped  stabilise  beneficiaries’  economies  and 
promoted the implementation of important policy reforms. The instrument’s external and 
internal  coherence  was  satisfactory  overall,  even  if  external  coherence  with  other  EU 
external policy instruments could be further enhanced. Finally, MFA is very cost-efficient, 
especially  when  support  is  provided  in  the  form  of  loans,  although  swifter  upstream 
decision-making procedures could produce additional resource savings. 

4.2 How did the EU intervention make a difference? 

From  the  Member  States’  perspective,  EU  MFA  operations  provide  financial  and 
political value added. The financing of MFA loans by means of EU borrowing does not 
affect Member States’ finances or debt ratios, but does allow them to provide  support in 
regions that they have interest in. Consistency with the EU’s external policies allows more 
economies of scope (within the Commission and the EEAS) than smaller Member States 

24 (Ecorys, Economisti Associati, Case, 2010), (ICF International, GHK, Cambridge Economics, 2013) 

29 

 
 
 
                                                           
in particular would otherwise be able to mobilise. This added value tends to increase with 
the volume of the operation. 

From  the  beneficiaries’  perspective,  all  the  consulted  stakeholders  recognised  the 
value added by MFA. According to them, a majority of the reforms implemented as part 
of  MFA’s  reform  requirements  would  not  have  taken  place  otherwise  (or  at  least  not  as 
quickly) –  e.g. reforms  in the areas of public financial management, anti-corruption and 
the judicial system. This is despite the fact that the scope and overall impact of the MFA 
operations in their countries were relatively small compared with the overall EU presence 
there. 

As some key stakeholders suggest,  MFA supports the EU’s unique role in fostering 
reforms to strengthen the rule of law in particular. Interviewed stakeholders from both 
the  Commission  and  the  IFIs  suggested  that  not  all  the  MFA  conditions  were  directly 
affecting  macro-economic  stability  in  the  beneficiary  countries.  Stakeholders  considered 
this  to  be  an  advantage,  however,  especially  because  MFA  in  the  beneficiary  countries 
works alongside the IMF and other EU instruments. Unlike the IMF, which is committed 
to strict neutrality, MFA promotes democratic values directly as well as other values that 
are of interest to the EU and the beneficiary countries. Rule of law and democratic values 
are not MFA operational objectives, however, but are rather pre-conditions for obtaining 
MFA. 

This  is  despite  the  observation  that  the  size  of  EU  added  value  is  perceived 
differently  across  operations,  both  by  external  evaluators  and  by  beneficiary 
countries. According  to  the  external  study,  this  may  reflect  different  levels  of 
involvement by the EU (compared with the IMF and the World Bank) in the perception of 
both the external evaluators and the beneficiary countries. This could be because (i) IMF 
support  has  to  be  in  place  before  the  authorisation  of  an  MFA  and  (ii)  according  to 
interviews with the IFIs, the IMF was usually ‘making a bit more noise’ about its support 
than the EU. 

The  interviewed  stakeholders  have  noted  in  this  context  that  MFA’s  visibility  was 
rather low in most beneficiary countries. This low visibility reduced the value added by 
the  MFA  because  the  instrument  was  under-utilised  for  the  promotion  of  EU  economic 
diplomacy  in  beneficiary  countries  and  their  geographical  surroundings.  The  meta-
evaluation  finds  that  the  EU  could  increase  the  value  it  adds  through  more  active 
engagement with political leaders, greater media coverage and engagement with a wider 
group  of  stakeholders,  including  the  private  sector  and  civil  society.  Some  stakeholders 
also mentioned the possibility of communicating more consistently on the whole package 
of  the  EU’s  assistance,  including  MFA.  Some  key  stakeholders  were  convinced  that 
increased visibility can help the EU to reach out to non-traditional interlocutors and thus 
provide new opportunities to form ‘coalitions for reform’, increase the sense of ownership 
in  the beneficiary  countries, and thus  make MFA more effective in  promoting important 
reforms. 

4.3 Is the intervention still relevant? 

Continuing  demand  for  MFA  operations  from  beneficiaries  demonstrates  the 
instrument’s ongoing relevance. During the evaluation period, MFA was requested and 
deployed  15  times.  Moreover,  10  operations  were  requested  and  authorised  at  the 
inception of the COVID-19 pandemic in 2020. The Commission received further requests 

30 

 
 
by  the  end  of  2021,  despite  the  increased  availability  of  global  liquidity  given  the  2021 
general  allocation  of  special  drawing  rights  (SDRs)  by  the  IMF.  The  emergency  and 
exceptional MFA provided to Ukraine in 2022 in the context of Russia’s unjustified war of 
aggression  is  further  important  testimony  of  the  usefulness  of  MFA  as  a  flexible  crisis 
instrument. 

On the basis of past operations, the external study also draws a positive conclusion 
on the relevance of policy reforms  promoted through MFA.  MFA had an operational 
and/or  political  reinforcing  effect  in  all  areas  in  which  conditions  were  formulated 
(Moldova).  It  often  therefore  acted  as  a  positive  contributing  factor  rather  than  the  sole 
cause of a reform (Ukraine 10.1425). MFA similarly promoted actions that might not have 
been  implemented  otherwise  while,  in  some  other  instances,  the  authorities  would 
probably have taken action even without the MFA conditions (e.g. Moldova). In terms of 
scope,  they  addressed  important  issues,  even  if  they  did  not  always  comprehensively 
cover all of the most pressing reform needs (e.g. Bosnia and Herzegovina26). Reforms in 
Public Financial Management (PFM) and the social safety net, which are often the subject 
of conditions (see Table 7) were particularly relevant (Ukraine 10.14). 

Against this backdrop, the external study concludes that the MFA instrument  has a 
high level of relevance in most countries. Structural reform conditions enshrined in the 
MoU  are  relevant  and  most  of  them  have  been  implemented  as  agreed.  Only  in  a  few 
cases  was  implementation  of  the  agreed  reforms  slow  and  ownership  of  the  reform 
programme limited (e.g. Tunisia 14). 

4.4 Is the current framework for evaluating interventions appropriate? 

With the 2015 ‘Guidelines for the Ex-Post Evaluation of Macro-Financial Assistance 
Operations’, the MFA ex post evaluations started to follow the Commission’s Better 
Regulation  Guidelines  (BRG).  Previous  evaluations  did  examine  the  five  criteria  of 
effectiveness, efficiency, coherence, relevance and EU added value but the BRG provide 
more  detailed  guidance  on  the  general  approach  to  be  used.  More  specifically,  BRG 
brought them together, gave them a more precise and standardised definition, and placed 
them  at  the  heart  of  the  evaluation  framework  for  EU-supported  interventions  including 
the MFA operations.  

The  post-BRG  ex  post  evaluation  reports  were  therefore  structured  more  closely 
around the five criteria after 2015 than had previously been the case according to the 
external  study.  The  evidence  demonstrates  that  the  MFA  evaluations  throughout  the 
period generally provided a good assessment of the key evaluation criteria set out in the 
2015  BRG.  These  key  criteria  featured  in  the  MFA  ex  post  evaluations  throughout  the 
period,  albeit  to  a  varying  degree.  Within  the  overall  BRG  framework,  each  evaluation 
was  meant  to  be  tailored  to  the  objectives  of  the  specific  programme  and  its  expected 
outputs, results and impact. Whereas earlier, pre-2015 ex post evaluation reports had been 
structured  around  the  main  MFA  objectives  (macroeconomic  impacts,  contribution  to 
structural  reforms,  and  the  design  and  implementation  of  the  MFA  operations),  the  later  

25  (ICF  International,  Cambridge  Econometrics,  Institute  for  Economic  Research  and  Policy  Consulting, 

2017) 

26 (ICF International, 2015) 

31 

 
 
                                                           
post-2015  ex  post  evaluation  reports  were  largely  structured  around  the  five  evaluation 
criteria. 

In  addition  to  the  five  evaluation  criteria,  the  BRG  encouraged  a  more  in-depth 
assessment  of  social  impacts  than  had  previously  been  evident.  The  Social  Impact 
Analysis  (SIA)  included  in  the  MFA  ex  post  evaluations  was  intended  to  improve  the 
understanding  of  the  extent  to  which  MFA  interventions,  which  primarily  had  an 
economic  objective,  also  had  effects  –  intended  or  otherwise  -  of  a  social  nature  and 
whether these were positive or negative, or a combination of these effects depending on 
geographical, social and other factors. This was a new type of analysis. In contrast, to the 
Debt Sustainability Analysis (DSA) introduced by the BRG was less novel to the ex post 
evaluations, because it was essentially an extension of the existing macroeconomic impact 
assessment. 

Overall, the use of SIAs and DSAs made a significant contribution to the assessment 
of MFA efficiency and effectiveness, based on the external study. With the DSAs the 
difference between the pre- and post-BRG level of analysis was less pronounced because 
this  type  of  analysis  was  already  a  key  element  in  the  earlier  ex  post  evaluations.  The 
DSAs improved the understanding not only of the macroeconomic impacts – which were 
central to all the ex post evaluations, whether pre- or post-BRG – but also of the longer-
terms effects in terms of debt sustainability. 

The  BRG  also  placed  increased  emphasis  on  the  systemic  involvement  of  key 
stakeholders (both in terms of impact assessments and ex post evaluations). There had 
already been quite extensive consultations with  stakeholders  in  Brussels,  the beneficiary 
countries  and  the  international  financial  institutions  as  part  of  the  earlier  MFA  ex post 
evaluations. However, the 2015 BRG encouraged evaluators to further develop this aspect 
of  their  research,  with  examples  including  the  Delphi  technique  successfully  used  to  a 
greater extent in engaging a wide range of stakeholders with a knowledge of MFA. The 
2015 DG ECFIN evaluation guidelines enshrined this principle further by de-emphasising 
econometric  modelling  for  the  purpose  of  counterfactual  analysis.  The  analysis  of  high-
frequency  economic  indicators  has  instead  been  supplemented  by  greater  stakeholder 
consultation in the construction of counterfactuals. This decision was in line with a 2012 
CEPS-report commissioned by the European Parliament 27 which concluded the following 
on  this  matter:  ‘The  meta-evaluation  report  [of  2010]  devoted  attention  to  the  question 
whether econometric modelling techniques could materially help the evaluation process. 
Their  conclusion,  which  this  reviewer  shares,  is  that  there  are  both  practical  and 
theoretical reasons why this could add little value to the evaluations.’ 

The methodological approach to MFA counterfactual modelling is in line with how 
DG  ECFIN  has  evaluated  euro  area  economic  adjustment  programmes 28  where  a 
crisis  makes  counterfactual  scenarios  underpinned  by  econometric  modelling  less 
reliable.  Such  an  approach  also  has  parallels  in  the  work  of  other  institutions  (e.g.  the 

27  Emerson  M.,  EU  Macro-Financial  Assistance:  A  Critical  Assessment,  Centre  for  European  Policy 

Studies - CEPS, Belgium, February 2012, and p. 39. 

28  Evaluation  reports  on  economic  and  financial  affairs  policies  and  spending  activities  |  European 

Commission (europa.eu) 

32 

 
 
 
                                                           
IMF’s Independent Evaluation Office (IEO) recognises  ‘significant empirical challenges 
involved  in  identifying  appropriate  counterfactuals  and  isolating  the  impact  of 
programmes’ 29).  

When undertaking individual MFA evaluations, DG ECFIN does not  prescriptively 
set  an  explicit  methodology  to  be  followed  by  external  contractors.  Instead,  an 
overarching  set  of  MFA  evaluation  methodological  guidelines  complements  the  Better 
Regulation  requirements.  This  promotes  competition  and  innovation  among  external 
contractors  in  devising  suitable  evaluation  methodology,  not  just  for  counterfactual 
analysis but also for the overall assessment of the evaluation criteria. Examples include (i) 
analysis of fiscal savings relating to favourable debt terms used by one contractor that has 
subsequently been applied in other MFA evaluations, and (ii) the innovative use of social 
media  analysis  by  a  contractor  in  the  assessment  of  relevance,  effectiveness,  efficiency 
and EU added value in the evaluation of MFA Ukraine I & II. 

Against this backdrop, it seems that the standard of individual evaluations was good 
and  therefore  that,  by  extension,  the  reliability  and  relevance  of  the  methodologies 
employed  was  appropriate.  Evaluations  followed  the  relevant  methodological 
guidelines and were further enhanced by the introduction of the BRG into the MFA 
evaluation framework. The evaluations provide comprehensive and detailed insights into 
the MFA’s macroeconomic impacts, its contribution to structural reforms and other issues 
relating to the design and implementation of the MFA operations. In particular, it can be 
concluded that most of the MFA evaluations provided a good assessment of what would 
have happened without the MFA operations. The counterfactual analysis typically focused 
on macroeconomic issues but was also extended in a number of cases to the assessment of 
the MFAs’ contribution to structural reforms. 

This  overall  positive  assessment  notwithstanding,  the  external  evaluator  has  also 
found  several  points  for  improvement  in  the  evaluation  of  the  individual  MFA 
programmes in the future: 

1.  The  comparative  dimension  of  the  ex  post  evaluations  was  a  relatively  weak 
aspect  of  the  assessments  and  ‘lessons  learned’  were  therefore  not  a  prominent 
feature. More specifically, only two evaluations had dedicated sections on this matter, 
whereas  around  two  thirds  of  the  MFA  ex  post  evaluation  reports  included 
‘recommendations’. This was particularly the case in pre-2015/pre-BRG evaluations.  

2.  The  assessment  of  coherence  can  be  further  improved  by  placing  more  emphasis 
on assessing all the EU-supported programmes as a whole in a particular beneficiary 
country. In addition, elements in MFA operations that contribute directly or indirectly 
to wider EU policy goals could be more clearly highlighted. 

3.  The  issue  of  low  MFA  visibility  also  extends  to  its  ex  post  evaluations.  The 
evaluations’  findings  were  usually  not  disseminated  systematically  after  the  reports 
were  finalised.  Some  stakeholders  such  as  central  banks,  ministries  of  finances, 
IMF/World Bank did at times have access to the information because they participated 
in validation workshops, but this was not the case for other stakeholders such as local 

29 ‘Adjustment and Growth in IMF-supported Programmes’, Draft Issues Paper for an Evaluation by the 
Independent Evaluation Office (IEO), September 2019. 

33 

 
 
 
                                                           
chambers  of  commerce  and  industry,  some  IFIs  (e.g.  the  EBRD),  representatives  of 
civil society, academics, and the business and financial sector.  

5.  WHAT ARE THE CONCLUSIONS AND LESSONS LEARNED? 

5.1 Conclusions 

The  overall  assessment  of  the  MFA  instrument  over  the  meta-evaluation  period  is 
very positive. It contributed to operations that stabilised the economy of beneficiaries. It 
was  very  cost-efficient  and  provided  an  appropriate  level  of  coherence  with  other  EU 
instruments in neighbourhood and pre-accession countries. Continuing demand confirms 
its  ongoing  relevance  and  stakeholders  widely  confirm  the  EU  value  added  of  past 
operations.  

Furthermore,  the  framework  for  MFA  evaluations  is  assessed  as  appropriate  and 
useful. Ex post evaluations were carried out to a high standard and were further enhanced 
with the introduction of the BRG into the MFA evaluation framework. More specifically, 
the  BRG  encouraged  a  more  in-depth  assessment  of  social  impacts  and  a  better 
appreciation  of  the  five  evaluation  criteria  of  effectiveness,  efficiency,  coherence,  EU 
value added  and relevance. The evaluations  have therefore provided comprehensive and 
detailed 
to  better 
governance,  more  economic  resilience  and/or  sustainable  growth  models  through 
structural  reforms;  and  other  issues  relating  to  the  design  and  implementation  of  the 
instrument.  Most  MFA  evaluations  have  therefore  provided  a  good  indication  of  what 
would have happened without the MFA operations.  

into  MFA’s  macroeconomic 

its  contribution 

insights 

impact; 

Several points for improvement have nonetheless been identified. 

1.  Concomitance  with  IMF  support:  concomitance  with  IMF  support  significantly 
improved  MFA’s  cost-efficiency  and  effectiveness,  mitigating  financial  risk  and 
allowing division of labour. At the same time, making the MFA strictly conditional on 
a  satisfactory  track  record  of  implementing  an  IMF-supported  programme  has 
sometimes caused implementation delays at the start of and during operations, and this 
has  prompted  some  stakeholders  to  call  for  more  flexibility.  Furthermore,  some  are 
concerned  that  the  standard  arrangements  with  regard  to  IMF  involvement  could 
compromise  the  EU’s  standing  as  an  autonomous  global  actor.  A  more  flexible 
approach  was  employed  in  this  respect  in  recent  exceptional  cases  (the  COVID-19 
MFA package and 2022 MFA to Ukraine). While clearly exceptional, this experience 
could,  once  evaluated,  usefully  feed  into  further  reflections  on  the  pros  and  cons  of 
potential alternatives to the current regime. 

2.  Decision-making procedure: the current standard procedures in the EU (the OLP) and 
the beneficiary country (parliamentary ratification) have been identified as a potential 
source  of  inefficiency  and  risk,  thus  reducing  the  instrument’s  effectiveness  in 
addressing  an  acute  or  impending  balance  of  payments  crisis  and  promoting  reform 
implementation  within  the  standard  time  window  of  IMF-supported  economic 
adjustment programmes. It was the intention of the Commission in 2011 to establish a 
framework  regulation  for  MFA  that  would  make  the  instrument  more  effective  by 

34 

 
 
 
streamlining  its  decision-making  process,  but  no  agreement  could  be  reached  on  a 
legal  set-up  that  was  acceptable  to  all  EU  institutions.  Recent  experiences  with  the 
COVID-19 MFA package and with MFA support to Ukraine in the context of Russia’s 
unjustified  invasion  of  the  country  have  shown  that  the  current  set-up  of  MFA 
decision-making can allow very swift adoption if all institutions agree to make full use 
of the available procedural flexibilities in particularly urgent cases.  

3.  Volume  of  financial  assistance:  although  critically  dependent  on  the  design  of  its 
policy  conditions,  the  impact  of  MFA  operations  (and  hence  their  effectiveness  and 
added  value)  is  (if  all  other  factors  are  the  same)  proportional  to  the  level  of  the 
financial support. The 2013 Joint Declaration no longer specifies specific ceilings, but 
it is not yet fully clear to what extent the current policies provide sufficient flexibility. 
This issue may therefore need to be revisited at a later stage. 

4.  Policy conditions: policy conditions that are appropriate for the chosen policy areas, 
their  degree  of  ambition  and  impact  on  long-term  sustainability,  as  well  as  their 
sequencing  over  time  and  focus  are  important  elements  of  a  successful  MFA 
operation.  Individual  ex  post  evaluations  have  generally  been  positive  regarding  the 
relevance and impact of MFA reform conditionality.  

5.  Flexibility: the inability to adapt the conditions listed in the MoU distinguishes MFA 
from  IMF support and the lack of adaptable disbursement modalities distinguishes it 
from  EU  budget  support.  A  more  flexible,  IMF-like  regime  appears  to  have  the 
advantage  of  being  able  to  effectively  anchor  and  accompany  important  policy 
reforms,  especially  in  a  fast-moving  economic  situation  which  could  influence 
decisions  on  the  most  suitable  policy  response  and  therefore  the  necessary  reform 
commitments. In the  context  of MFA, some flexibility is  provided through a limited 
number of waivers which exceptionally allow disbursements to take place despite the 
non-fulfilment  of  one  (or  a  few)  reform  conditions.  Furthermore,  in  the  2022 
exceptional  MFA  package  for  Ukraine,  allowances  for  the  uniquely  challenging  war 
situation  were  made  in  advance  when  designing  conditionality  in  the  MoU.  The 
increased  flexibility  enabling  the  use  of  MFA  conditionality  even  in  this  context 
proves  that,  while  recognising  the  exceptional  character  of  these  operations,  solid 
reform conditionality remains a cornerstone of successful MFA operations.  

6.  Visibility of MFA operations and their evaluations: MFA’s low visibility meant that 
the instrument did not realise its full potential to promote EU ‘economic diplomacy’ in 
beneficiary  countries.  More  specifically,  greater  visibility  would  make  it  possible  to 
reach  out  to  non-traditional  interlocutors  and  thus  create  opportunities  to  form 
‘coalitions  for  reform’  to  increase  a  sense  of  ownership  in  the  beneficiary  countries 
and thus support the MFA’s effectiveness in promoting important reforms. 

5.2 Lessons learned 

The above-mentioned conclusions provide a basis for further reflection, but there are 
also a few ‘lessons learned’ which the Commission could consider acting upon in the 
future. 

1.  Political  pre-conditions:  as  explained  in  Section  4.1,  the  external  study  finds  some 
challenges in the way political pre-conditions are assessed in practice that could have 

35 

 
 
a  bearing  on  MFA’s  effectiveness.  The  requirements  for  continued  MFA  support 
should  therefore  be  communicated  more  clearly  in  cases  where  actual  or  possible 
disruptions emerge in relation to the partner country’s respect of effective democratic 
mechanisms, including a multi-party parliamentary system, the rule of law and human 
rights. 

2.  Decision-making:  Like  the  external  meta-evaluation  of  2009/10,  the  current  one 
points  to  the  risks  brought  to  the  instrument’s  effectiveness  and  efficiency  from  the 
standard  decision-making  procedures  at  the  side  of  the  EU  and  in  the  beneficiary 
country. Although  the  EU  decision-making  was  greatly  expedited  in  the  most  recent 
COVID-19-related operations as well as for the  emergency and exceptional MFA to 
Ukraine, the use of urgency procedures, notably by the European Parliament, is fully 
within each institution’s prerogatives and hence highly uncertain as to its use in future 
cases. At this stage, however, there is no clear indication given that changing the legal 
set-up  would  significantly  speed  up  MFA  operations  beyond  what  can  be  achieved 
when making full use of the available procedural flexibilities. Further reflections may 
be  useful  on  potential  alternatives  to  the  current  decision-making  procedure.  These 
aspects  could  usefully  be  assessed  once  the  evaluations  of  the  COVID-19  MFA 
package and the exceptional Ukraine MFA package have been carried out.  

3.  MFA visibility and dissemination of MFA evaluations’ findings: the low visibility of 
MFA  operations  is  an  important  finding  of  the  evaluation  and  touches  on  MFA 
outcomes  in  several  dimensions  (effectiveness,  coherence,  and  EU  added  value). 
Possible  improvements  in  this  regard  should  therefore  be  explored,  including  with 
regard to the dissemination of ex post evaluation results. 

4.  ‘Lessons  learned’:  future  MFA  evaluations  could  place  more  emphasis  on  ‘lessons 
learned’.  This  is  in  line  with  the  November  2021  updates  to  the  Better  Regulation 
Guidelines and could be made a specific requirement in the terms of reference for final 
ex post evaluation reports. More emphasis on ‘lessons learned’ and methods of sharing 
these with key stakeholders would enhance the benefits of future evaluation activities. 

36 

 
 
 
 
ANNEX I:   PROCEDURAL INFORMATION 

In  2020,  the  Commission’s  DG  ECFIN  took  the  initiative  for  a  new  meta-evaluation30 
covering  15  MFA  evaluations  completed  in  the  last  ten  years.  The  exercise,  coinciding 
with the MFA instrument’s 30th anniversary, takes stock of previous evaluation results and 
aims to support organisational learning both in terms of the design and implementation of 
the  MFA  instrument  and  the  methodology  used  to  evaluate  individual  operations.  The 
meta-evaluation also aims to support transparency and accountability and to contribute to 
laying the  groundwork for any possible improvements  of the instrument’s design and/or 
workings  as  necessary.  The  Decide  planning  entry  for 
is 
PLAN/2020/6586. 

the  meta-evaluation 

In line with the requirements set out in the Commission Better Regulation Guidelines31, 
the  meta-evaluation  considered  the  criteria  of  relevance,  effectiveness,  efficiency,  EU 
value  added,  coherence  with  other  international  and  Union  instruments  for  financial 
assistance,  social  impact  and  impact  on  the  sustainability  of  public  debts.  The  exercise 
relies  primarily  on  an  external  independent  study  commissioned  by  the  Commission  in 
December 2020 and concluded in October 2021. In order to ensure validity, the analysis 
and  conclusions  of  the  external  study  are  based  on  the  evidence  obtained  using  several 
targeted 
evaluation  methods  (document  review,  semi-structured 
stakeholders,  open  public  consultation,  a  validation  workshop  and  a  roundtable  with 
relevant  Commission  staff).  A  number  of  limitations  were  experienced  during  the 
preparation and completion of the study. Annex 2 provides a detailed overview of these 
shortcomings, together with the mitigation strategies adopted.  

interviews  with 

The outcome of the external assessment was supplemented by input from (i) a high-level 
seminar  discussing  the  experiences  of  30  years  of  MFA  that  took  place  in  November 
2021 (see below for a report), and (ii) the Regulatory Scrutiny Board (RSB).  

The  lead  DG  to  carry  out  and  manage  the  meta-evaluation  has  been  DG  ECFIN.  DG 
ECFIN also chaired the inter-service group (ISG) that was set up to manage the external 
evaluation.  Apart  from  DG  ECFIN,  the  ISG  comprised  of  representatives  of  other 
Commission services (the Secretariat General, the Directorate-General for Neighbourhood 
and Enlargement Negotiations and the Directorate-General for International Partnerships).  
The ISG had the responsibility to: 

a.  Establish the evaluation roadmap for the external evaluation; 
b.  Establish the Terms of Reference (ToR); 
c.  Facilitate the evaluator’s access to the information needed; 
d.  Advise, monitor and comment on the work undertaken by the external contractor.  

The  call  for  tenders  was  launched  in  October  2020  and  two  applications  were  received. 
Following the verdict of an  evaluation  committee,  the specific  contract to  undertake the 
external  evaluation  was  awarded  to  a  consortium  led  by  OXFORD  RESEARCH  under 

30 A first external meta-evaluation was published in 2009. The exercise covered MFA operations evaluated 
between 2004 and 2008: Microsoft Word - MFA Meta Evaluation - FINAL Report.doc (europa.eu) 

31 SWD(2021) 305 final 

37 

 
 
 
                                                           
Framework Contract number ECFIN-003-2017 - ref. Ares(2017)1628394. The consortium 
included  VVA  Brussels  and  the  Centre  for  Strategy  &  Evaluation  Services.  The  total 
amount paid under this specific contract was EUR 135,200.00 covering all tasks executed.  

A kick-off meeting, where the ISG and the external contractor discussed the deliverables 
and  the  evaluation  methods,  took  place  in  December  2020.  The  evaluation  roadmap  ran 
between  21  January  2021  and  18  February  2021,  to  seek  wider  feedback.  This  was 
followed  by  the  launch  of  the  OPC  on  30  March,  for  a  period  of  twelve  weeks  (OPC 
Decide  planning  entry  PLAN/2020/6586).  Meetings  on  the  inception  and  interim  reports 
were held in, respectively, January 2021 and April 2021.  

In order to provide appropriate support and ensure the evaluator's access to the necessary 
information as comprehensively as possible, bi-weekly dialogues were organised between 
the evaluator and DG ECFIN from  May 2021 onwards. These dialogues continued until 
the  end  of  the  exercise.  To  address  identified  knowledge  gaps  on  the  MFA  instrument 
noted  in  the  interim  report,  ISG  members  also  set  up  a  dialogue  between  the  external 
contractor  and  DG  ECFIN  staff  in  charge  of  MFA.  The  purpose  of  this  roundtable 
discussion held  in June 2021 was to  inform  the evaluator on the specific features of the 
instrument  and  to  allow  her  to  gather  further  views  on  possible  limitations  and 
developments.  

The  draft  final  report  was  first  submitted  in  July  2021.  Subsequently,  the  report  was 
revised four times, before a final version was approved in October 2021.  

As with all ex-post evaluations related to programmes with a major impact on EU budget 
or which are of strategic importance for the Union, the meta-evaluation carried out by DG 
ECFIN is subject to scrutiny by the RSB. 

An  RSB  upstream  meeting  was  held  in  October  2021,  followed  by  the  official 
consultation on 19 February 2022. The board gave a positive opinion on the report. 

38 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Table A1.1 below summarises the main points of the RSB review and how they have been 
integrated into the meta-evaluation. 

Board’s recommendations  

Commission’s point of view. The report should reflect better the 
Commission’s point of view on the findings and conclusions of 
the underlying study by a team of external evaluators. The report 
should use additional evidence when available to deepen the 
findings and conclusions. 
Findings and conclusions.  The report should present more 
prominently the findings and conclusions related to the rationale, 
reliability and relevance of the methodologies employed in the 
fifteen ex-post evaluations investigated. The report should 
explain better why econometric analysis is discarded as a suitable 
methodological approach and why combining it with 
counterfactuals is considered not possible. 
Impacts on wider society. The report should provide more 
information on the impacts of MFA support on wider society. It 
should extract additional information from the social impact 
analyses undertaken in some of the ex-post evaluations 
investigated here. It should also strive to provide more 
information on the impacts of MFA on the respect of 
conditionalities regarding the rule of law and democratic values 
in the recipient countries 
Link with IMF programme. As MFA is only available to 
countries benefiting from a disbursing IMF programme, the 
report should explain better what is the added value of MFA 
support compared to the IMF. It should explain why the EU 
should be in partnership with the IMF and, vice-versa, why the 
IMF wants to act in partnership with the EU. It should be clearer 
on the EU’s influence in defining conditionality.  
Value added of MFA. The report should explain better what is 
the added-value of MFA support compared to other EU 
instruments such as NDICI budget and project support 
(Neighbourhood, Development and International Cooperation 
Instrument – Global Europe).  
Loans versus grants. The report should more clearly draw 
conclusions in areas that could lead to follow-up analysis or 
actions. For example, it should be more specific in its conclusion 
on the current methodology to determine the shares of loans and 
grants as well as the relationship between financing volume and 
expected impacts on targets. 
Views of stakeholders. The report should better explain how it 
tried to integrate the views of beneficiary countries and civil 
society in the evaluation.  

Integration of the recommendations 
into the meta-evaluation report 

Various sections of the report have 
been modified to implement this 
recommendation. 

The report has been modified in 
response to this recommendation, 
notably in section four. 

The report has been modified in 
response to this recommendation, 
notably in section number two and 
four. 

Various sections of the report have 
been amended to take into 
consideration this 
recommendation. 

Various sections of the report have 
been amended to take into 
consideration this 
recommendation. 

The report has been modified in 
response to this recommendation, 
notably in section number two and 
four. 

The report has been amended to 
take into consideration this 
recommendation, notably in 
section four. 

39 

 
 
 
 
 
 
 
 
  
 
ANNEX II. METHODOLOGY AND ANALYTICAL MODELS USED 

This  annex  presents  the  methodological  approach  to  the  meta-evaluation  study.  It 
describes the design of the methodology, the tools used for data and information gathering 
and the results obtained. It also provides insights on the limitations encountered during the 
study and the mitigation strategies adopted. 

Evaluation design 

The  methodology  of  the  evaluation  study  was  designed  to  respond  to  (i)  the  evaluation 
questions detailed in the Terms of Reference (ToR) for the meta-evaluation32, and (ii) the 
Better Regulation Guidelines (BRG) evaluation criteria33. It rested on three pillars:  

1.  Participatory and inclusive data collection and analysis through a (key) stakeholder 
consultation  programme  (semi-structured  interviews  and  public  consultation),  a 
validation  workshop  preceded  by  a  roundtable  with  the  Commission  staff  in 
charge and a high-level seminar; 

2.  A  mixture  of  qualitative  and  quantitative  input,  with  qualitative  input  obtained 
mainly  through  document  analysis  (chiefly  of  the  evaluation  reports  of  the  15 
aforementioned operations) and stakeholder input and quantitative data supporting 
the qualitative information to the extent possible; and  

3.  Triangulation,  i.e.  the  information  and  data  collected  from  a  range  of  different 
sources  using  a  range  of  methods  collectively  provides  answers  to  DG  ECFIN’s 
evaluation questions.  

Tools for information gathering and results obtained 

The  information  and  data  required  for  the  study  were  collected  using  the  following 
methodological tools: 

a.  Review  of  internal  documentation  concerning  all  Macro-Financial  Assistance 
(MFA) operations under study, including among others: the MoUs of the Macro-
Financial  Assistance  operations  under  analysis;  the  relevant  ex-post  evaluations 
and  Commission’s  staff  working  documents;  the  2009  MFA  meta-evaluation 
report;  budget  support  publications;  and  the  Commission’s  Better  Regulation 
Guidelines and toolbox.   

b.  Semi-structured  interviews  with  key  stakeholders  from  the  Union  Institutions, 
national authorities of the beneficiary countries, consultancy companies in charge 
of the previous evaluations, and International Financial Institutions (IFIs). A total 
of  50  detailed  interviews  were  carried  out  between  February  and  June  2021,  of 

32 Evaluation questions detailed in annex III 
33 SWD(2021) 305 final 

40 

 
 
 
 
 
                                                           
which  30  were  with  staff  from  the  Union  Institutions.  The  remaining  interviews 
ensured  a  geographical  coverage  of  all  concerned  MFA  countries.  To  implement 
the interview programme, the research team systematically contacted stakeholders 
from  the  list  which  was  agreed  with  DG  ECFIN  during  the  inception  phase.  For 
this  purpose,  email  invitations  were  sent  out,  giving  details  about  the  meta-
evaluation  and  interview,  and  providing  a  targeted  questionnaire.  To  ensure  that 
the  key  information  from  each  interview  was  well  recorded,  minutes  of  the 
conversations  were  taken  during  the  interview.  The  highest  response  rate  was 
found  with  the  European  Commission  staff  most  directly  involved  in  MFA 
operations  (DG  ECFIN,  DG  NEAR)  and  DG  INTPA,  together  with  the 
Commission Legal Service and DG BUDG. By contrast, staff and Members of the 
European  Parliament  showed  a  much  lower  response  rate,  with  a  moderate  final 
outreach. 

c.  Open  Public  Consultation  (OPC),  to  gather  the  views  of  civil  society  and  the 
general public and ensure transparency of the study, in line with the requirements 
of the Commission's BRGs. The OPC was launched on 30 March 2021 and closed 
on  22  June  2021,  collecting  a  total  of  20  responses.  To  further  increase  the 
response  rate,  the  external  study  team  disseminated  the  link  among  stakeholders 
who  were  identified  in  the  stakeholder  mapping  exercise  conducted  for  the 
interview programme, asking them to respond to the OPC and further disseminate 
the information among their professional circles. In addition, DG ECFIN reached 
out  to  the  EU  Delegations  in  the  beneficiary  countries,  as  well  as  other 
stakeholders, to inform them about the consultation. The OPC was also promoted 
on the European Commission's social media accounts (Twitter and LinkedIn). 

d.  Roundtable discussion with the Commission staff in charge of MFA operations, 
to discuss more technical elements concerning the instrument and the interventions 
under analysis. A total of 12 people took part in the roundtable, which focused on 
the 
themes:  MFA  conditionality  and  political  pre-conditions; 
macroeconomic  stability;  reforms  sustainability;  coherence  of MFA with parallel 
financial assistance instruments by the Union or by other international actors.   

following 

e.  Validation workshop, with key stakeholders and previous evaluators, to validate 
the preliminary findings of the meta-evaluation. A total of 22 participants from 14 
different  organisations  took  part  in  the  workshop,  which  took  place  on  22  June 
2021.  The  main  takeaway  points  of  the  workshop  contributed  to  refining  and 
validating  the  conclusions  of  the  evaluation,  thereby  contributing  to  the  final 
report. 

f.  High-level  seminar  to  present  the  preliminary  findings  of  the  study  to  a  wider 
audience,  including  beneficiary  countries,  and  gather  further  input  to  inform  the 
meta-evaluation. A total of around 200 participants attended the event held on 16 
November 2021, with representatives from EU institutions, partner countries, IFIs 
and  public  policy  consultancies.  The  event  was  kicked  off  by  Paolo  Gentiloni, 
Commissioner for the Economy at the European Commission. His keynote speech 
was  followed  by  two  panel  discussions  with  high-level  speakers,  comprising: 
representatives  of  Ministries  of  Finance  and  Central  Banks  of  MFA  countries; 
the  Commission;  Permanent 
Deputy  Directors-General  and  Directors  of 

41 

 
 
Representatives  and  Mission  Chiefs  of  the  IMF.    The  second  part  of  the  event 
included two case studies on MFA operations in Ukraine and Jordan, with related 
in-depth discussions and Q&As.  Input from the seminar was used as evidence to 
inform the analysis undertaken in the SWD.  

While the semi-structured interviews, the roundtable and the validation workshop enabled 
the  collection  of  information  of  a  highly  technical  nature,  the  OPC  and  the  seminar 
extended the hearing to include less specialised interest groups.  

The data collection tools (with the exception of the seminar) were used to inform the work 
of the external evaluator. DG ECFIN, together with the other members of the ISG, closely 
monitored the contractor's work throughout the study.  

In addition to providing all necessary documentation to the evaluator in a timely manner, 
DG ECFIN held bi-weekly meetings with the external evaluator, starting in May 2021, to 
verify  the  quality  of  the  information  and  analysis  used  to  inform  the  external  report. 
Furthermore, DG ECFIN initiated the roundtable with  the Commission  staff responsible 
for MFA, in order to give the evaluator a greater understanding of the instrument and its 
limitations and correct inaccuracies in knowledge.  

At  the  conclusion  of  the  external  study  the  ISG  completed  a  quality  assessment  of  the 
final report. According to the assessment, the study addressed all key issues as required in 
the  Terms  of  Reference,  although  some  of  the  underlying  analysis  lacked  depth.  The 
evaluator  provided  all  requested  deliverables  in  line  with  the  work  plan.  Some  of  the 
deliverables required several iterations before the quality was acceptable to the ISG. The 
assessment concludes that the report is broadly satisfactory and can be approved on that 
basis.  

Limitations encountered and mitigation strategies 

A number of limitations were experienced during the completion of the independent study 
and hence the preparation of the meta-evaluation.  

The following list summarises the main challenges as well as the specific measures taken 
to address them: 

1.  Limitations on data availability, mainly due to a low response rate in the OPC. 
The  consultation  ran  for  a  period  of  12  weeks,  collecting  a  total  of  only  20 
contributions. One of the reasons for the low response rate is possibly the highly 
specialised  nature  of  the  topic,  which  requires  a  high  level  of  expertise  and 
knowledge.  The  low  number  of  responses  affected  the  representativeness  of  the 
sample.  To  mitigate  this  limitation,  the  results  recorded  in  the  consultation  were 
supplemented by findings obtained through the use of other data collection tools. 
In  particular,  the  seminar  was  beneficial  in  filling  missing  gaps  after  the 
conclusion  of  the  external  assessment,  as  it  involved  a  wider  and  diversified 
audience including representatives of Ministries of Finance and Central Banks of 
MFA countries.  

2.  Difficulties  in  accessibility  to  stakeholders,  particularly  regarding  interviews 
undertaken during the external study. The response rate was lower than expected, 

42 

 
 
thus  limiting  the  reach  of  the  interviews.  The  study  team  addressed  the  issue  by 
expanding  the  list  of  stakeholders  according  to  priority  groups  assigned.  After 
reaching out to priority 1 stakeholders (EU institutions and local authorities), the 
team followed up with priority 2 (international organisations and past evaluators) 
and,  where  necessary,  priority  3  stakeholders  (potential  substitutes  for  each 
stakeholder group). This approach ensured that the final target of 50 stakeholders 
was reached. 

3.  Limited  diversification  of  stakeholders  consulted.  The  low  response  rates 
recorded  in  some  cases  led  to  a  partial  polarisation  of  the  answers  provided, 
creating  an  under-representation  of  some  categories  of  stakeholders  compared  to 
others.  In  order  to  rebalance  the  input  received  and  ensure  the  greatest  possible 
defensiveness  of  findings,  the  study  team  extended  the  original  list  of  people 
interviewed  in  a  targeted  manner.  The  November  seminar  also  provided  an 
excellent  opportunity  to  collect  input  from  external  stakeholders  and  further 
diversify the sources of information.  

4.  Limited  subject  knowledge  of  some  interview  respondents.  Many  of  the 
assessed  MFA  operations  were  carried  out  several  years  ago.  Not  all  of  the 
interviewed representatives of the EU delegations worked in the delegations at the 
time  of  the  MFA  implementation;  therefore,  their  knowledge  of  the  MFA  was 
somewhat limited. The same was also true for external evaluators, who had often 
left  their  position  in  the  consulting  companies  conducting  the  evaluations.  To 
mitigate this limitation, the study team attempted to establish contact with and to 
interview  officials  and  consultants  with  highest  possible  level  of  subject  matter 
expertise.  Furthermore,  the  team  mitigated  this  difficulty  by  diversifying  the 
content of the interviews and checking inputs collected against each other when on 
a similar topic or specific country.  

5.  Delays  in  some  parts  of  the  interview  programme  due  to  bureaucratic 
requirements of the respondent organisation. This limitation was mitigated by a 
work structure that allowed several parts of the study to run in parallel,  avoiding 
stalemates that would impact on compliance with deadlines. 

Whilst  such  limitations  have  undoubtedly  had  a  bearing  on  the  quality  of  the  exercise, 
they  do  not  impact  on  the  robustness  or  reliability  of  the  meta-evaluation  conclusions 
which serves as a useful basis for reflections on the instrument. 

43 

 
 
 
 
 
 
 
 
 
 
 
ANNEX  III.  EVALUATION  MATRIX  AND,  WHERE  RELEVANT,  DETAILS  ON  ANSWERS  TO 
THE EVALUATION QUESTIONS (BY CRITERION) 

This  annex  provides  a  detailed  overview  of  the  evaluation  matrix  used  for  the  meta-
evaluation  study.  It  was  developed  by  the  external  contractor,  as  a  framework  for  the 
independent assessment. The matrix assigns each evaluation question outlined in the ToR 
to  the  relevant  evaluation  criteria  and  serves  as  general  framework  for  the  applied 
methodology. 

Table A3.1: Evaluation questions matrix 

EVALUATION CRITERIA: RELEVANCE 

A 1. Are the current MFA principles and characteristics sufficiently clearly formulated? To what extent can they be 
clarified, complemented and/or formalised?  
What do we want to measure? 

1)  How clear are the current MFA principles and characteristics? 

2)  Are all of these still relevant? 

3)  To what extent could they be clarified, complemented or formalised? 

Success Criteria 

  MFA principles and characteristics are clear and relevant 

 

No clarifications are required 

Indicators 

1)  Quantitative 
  Number of ex-post 

evaluations mentioning 
principles’ clarity and/or 
unclarity 

  Number of principles 

mentioned 

2)  Qualitative 

  OPC and interview responses 

on clarity of principles 
  Description / summary of 

principles or characteristics 
requiring clarification, 
complementation or 
formalisation 

Sources 
Desk research / literature review 
  Ex-post evaluation reports 
  Declarations European Parliament and Council 
  Commission notes 

Stakeholder consultation 
Interviews with:  

 

European 

Commission staff 

 

 

EEAS Delegations 

IMF country desks 

Specific questions: 
Interview questions: 
  Are the current MFA principles and 

characteristics clearly understandable? 

  Are all of these still relevant? 
  To what extent could they be clarified, 

complemented or formalised? 

  Are there any other relevant principles and 
characteristics which could be added to the 
current list? 
OPC questions: 
  How clear are the current MFA principles and 

characteristics? [single choice] 

I do not know what the principles are 

- 
-  Very clear 
-  Somewhat clear 
-  Not very clear 
-  Not clear at all 

44 

 
 
 
 
 
 
 
 
 
EVALUATION CRITERIA: EFFECTIVENESS and RELEVANCE 

A 2. To what extent have the assessed MFA operations been effective in terms of the macroeconomic stabilisation 
of the recipient country in the short term and to returning the external financial situation to a sustainable path 
over the medium to longer-term? 
What do we want to measure? 
1)  Level of effectiveness of MFA operations to improve macro-economic situation of the country 
2)  Do the ex-post evaluations establish a positive causal link between MFA and the development of public finances? 
3)  When and how where the beneficiaries able to place sustainable loans on the market again? 

Success Criteria 

  MFA operations have been effective in improving internal and external imbalances situation of the recipient 

 
 

country 
Scientifically proven link between MFA and sustainable public finances through available evaluations 
Sustainable balance of trade and national budget and sustainable interest rate for government bonds after 
intervention 

Indicators 
1)  Quantitative 
  Number of ex-post evaluations 
mentioning positive correlation 
between MFA and 
macroeconomic stability 

  Differences recorded for macro-

economic indicators, such as: 
-  Government debt 
-  Current account 
-  Trade balance 
-  External debt 

2)  Qualitative 
 

Analysis and conclusions of 
medium- to long- term effect of 
the operations and measures 
taken  

Sources 
Desk research / literature review 
  Ex-post evaluation reports 
  Scientific ex-post evaluation 
 
  Supporting databases: Eurostat; OECD database 

IMF reports 

Stakeholder consultation 
Interviews with:  

Specific questions: 
Interview questions: 

 

 
 
 

European Commission 
staff 
EEAS Delegations 
IMF country desks 
National debt managing 
offices 

 

How effective have the different 

MFA programmes been in the 

short term (i.e. macroeconomic 

stabilisation of the recipient 

country)? 

 

How effective have the different 

MFA programmes been in medium 

to long term (i.e. returning the 

external financial situation to a 

sustainable path) 

 

Are there any differences between 

the different MFA programmes? 

Have these become more effective 

over time? What are the reasons 

rendering the MFA programmes 

more effective or less so? 

OPC questions 

45 

 
 
 
 
 
 
 
 
 
 
 
EVALUATION CRITERIA: EFFECTIVENESS, EFFICIENCY and COHERENCE 

A 3. To what extent has the conditionality of the instrument, as stated in the general principles of the instrument 
(depicted in the 2013 Joint Declaration of the European Parliament and the Council), impacted on the outcome, 
effectiveness and efficiency of the operations? 
What do we want to measure? 

1)  Has the conditionality of the MFA instrument impacted the outcome, effectiveness and efficiency of its operations? 

2) 

Is the list of conditionality principles still complete and relevant? 

3)  Can there be situations where exceptions to the conditionality are warranted? 

Success Criteria 

 

Conditionality had a positive impact on the outcome, effectiveness and efficiency of the MFA operations 

Indicators 
Quantitative 
  Number of ex-post 

evaluations mentioning 
positive and negative 
impacts of conditionality 

  Number of principles 

mentioned 

Qualitative 
  OPC and interview 
responses on 
appropriateness of 
conditionality principles 
  Explanation of how the 

different conditions had a 
positive impact on the 
outcome, effectiveness 
and efficiency of the MFA 
operations 

Sources 
Desk research / literature review 
  Ex-post evaluation reports 
  Declarations European Parliament and Council 
  Commission notes 

Stakeholder consultation 
Interviews with:  

 

European 

Commission staff 

 

National authorities 

in the beneficiary 

countries 

 

 

EEAS Delegations 

IMF country desks 

Specific questions: 
Interview questions: 
  To what extent has the conditionality of the MFA 

instrument, impacted on the outcome, effectiveness 
and efficiency of the operations? 

  Are all of the principles of conditionality still relevant? 
Are there any other relevant principles which could 
be added to the current list? 
Is this conditionality always appropriate? And is it 
always appropriate to enforce this conditionality? Are 
there any exceptions? 

 

EVALUATION CRITERIA:  EFFECTIVENESS 

A 4. Sustainability of structural reforms: To what extent did the structural reforms promoted by MFA endure once 
external conditionality faded? Or were they reversed? 
What do we want to measure? 
1)  Have the structural reforms been continued after the operation? 
2)  Has the country stayed on its sustainable path in the long term? 

Success Criteria 

 

The structural reforms have been continued after conditionality has faded 

Indicators 
Quantitative 

Sources 
Desk research / literature review 

 

Number of ex-post 

evaluations mentioning 

positive impacts after 

external conditionality 

faded 

 

 

 

 

Ex-post evaluation reports 

Annual reports on MFA operations and meeting of conditionality requirement 

Commission notes 

IMF reports 

 

Number of withdrawn 

Stakeholder consultation 
Interviews with:  

Specific questions: 
Interview questions: 

structural forms 

 

European 

 

To what extent did the structural reforms 

46 

 
 
 
 
 
 
 
 
 
EVALUATION CRITERIA:  EFFECTIVENESS 

A 4. Sustainability of structural reforms: To what extent did the structural reforms promoted by MFA endure once 
external conditionality faded? Or were they reversed? 

 

Long term development of 

sustainability finance 

indicators 

 
 
 

Commission staff 
EEAS Delegations 
IMF country desks 
National debt 
managing office 

Qualitative 

 

Explanation of political 

acceptance of structural 

reforms within the countries 

 

Outcomes, results, inputs on 

structural reforms’ 

sustainability and 

appropriateness in the 

country after operation has 

finished 

promoted by MFA endure once external 

conditionality faded? 

 

Are there differences between the various 

countries? How can these be explained? 

 

Are the reforms in some areas more 

sustainable than in others (see list of areas 

under description)?  

 

Are there any instances where these were 

reversed? 

 

Should the MFA consider including a 

conditionality clause on social outcomes within 

the MoU with beneficiary countries? 

EVALUATION CRITERIA: EFFECTIVENESS and EFFICIENCY 

A 5. How has the way in which the operations were designed (form of the assistance: loans and/or grants; 
availability period of maximum 2.5 years) and implemented conditioned their effectiveness and efficiency? 
What do we want to measure? 
1)  Do the ex-post evaluations support the idea that the effectivity or efficiency of the operations depended on the design 

of the interventions?  

2)  Do the sustainability indicators develop differently depending on the design of the operation? 

Success Criteria 

 

Positive causal relationship between operation design and effectiveness and efficiency of the operations 

Indicators 
Quantitative 

Sources 
Desk research / literature review 

  Number of ex-post evaluations 

that find positive impacts in the 

different design categories? 

  Development of sustainability 

indicators depending on 

operations design  

Qualitative 

  What economic incentives are 

set by different operation 

designs? 

 

Description of the effect of the 

format and design of the 

programme/operation on the 

outcomes and success 

 

 

 

 

Ex-post evaluation reports 

Commission notes 

IMF reports 

Supporting databases: Eurostat; OECD database 

Stakeholder consultation 
Interviews with:  

Specific questions: 
Interview questions: 

 

 
 
 

European Commission 
staff 
EEAS Delegations 
IMF country desks 
National debt managing 
offices 

 

How has the form of the MFA operations: (i.e. 

loans, grants or a mix of these) conditioned 

their effectiveness and efficiency? 

 

How has the availability period (i.e. maximum 

of 2.5 years) of the MFA operations 

conditioned their effectiveness and efficiency? 

 

Are there any other factors that contributed to 

conditioning the effectiveness and efficiency of 

the MFA operations? 

47 

 
 
 
 
 
 
 
 
 
 
 
 
 
EVALUATION CRITERIA: EFFICIENCY 

A 6. In turn, what has been the impact of the legislative and decision-making process in relation to the 
effectiveness and efficiency of MFA? What does the recent experience with the COVID19 MFA package say about 
this? 
What do we want to measure? 

1)  How effective/efficient are legislative & decision-making processes for MFA implementation? (Specifically: How have 

legislative and decision-making processes impacted the effectiveness and efficiency of MFA?) 

2)  What are the lessons learned from the experience with the COVID 19 MFA package? (How did the COVID 19 MFA package 

influence the effectiveness and efficiency of MFA?) 

Success Criteria 

 

Legislative and decision-making processes have a positive impact on effectiveness and efficiency of MFA 

COVID 19 MFA package is proven to support the effectiveness and efficiency of MFA 

 
Indicators 

1)  Quantitative 

  Number of ex-post evaluations 
mentioning positive/negative 
impacts of legislative & decision-
making processes 

  Timeframe (speed) of decision-

making processes / average days 
for decision making 

2)  Qualitative 

  Description of the decision-making 

processes under the past 
operations and for the COVID-19 
package 

  Explanation of how the decision-

 

making process for the COVID 19 
MFA package was different to the 
past operations (2010-2020) 
If different, the effect of this 
difference on the length and 
complexity of the overall decision-
making process  

  What alternative solutions to the 
current decision-making process 
were raised and considered in the 
past? What were their success and 
why? 

Sources 
Desk research / literature review 
  Ex-post evaluation reports 
  Declarations European Parliament and Council 
  Commission notes 
 
  Official EU sources/websites documenting the effect of the COVID 19 MFA 

IMF reports 

package 

  Annual Activity Reports ECFIN 

Stakeholder consultation 
Interviews with:  

 

European Commission 

staff 

 

 

 

EEAS Delegations 

IMF country desks 

Evaluators of MFA 

operations 

Specific questions: 
Interview questions: 
  What has been the impact of the 

legislative and decision-making process 
in relation to the effectiveness and 
efficiency of MFA operations? 

  What is the role of the institutional set-up 
of the MFA instrument (also compared to 
comparable instruments of other 
institutions) 

  What does the recent experience with 
the COVID19 MFA package say about 
this? 

  Are there any lessons learned from that 

process that could be incorporated in the 
general process for MFA operations? 

  How can the EU improve its internal 

decision-making mechanisms to speed-
up MFA deployment through the OLP?  

  Please explain the procedure and 

reasons for speeded decision in the EP 
and the Council on deployment of 
extraordinary operations (e.g. the 
COVID-19 crisis and Ukraine 2015 
crisis)? 

  Are there alternative solutions to the 

OLP? If yes, which? 

48 

 
 
 
 
 
 
 
EVALUATION CRITERIA: EU VALUE ADDED and COHERENCE  

A 7. EU value added and coherence: What lessons can be learned from previous MFA operations on how to 
maximise EU value added? To what extent is the impact dependent on the design and implementation of MFA 
operations? Are there synergies with other EU policies, or of those of other organisations and bodies? How can 
coherence with other EU external instruments be enhanced? 
What do we want to measure? 

1)  What is the EU added value and level of coherence? How can they be maximised? 

2)  To what extent does the EU added value depend on the design and MFA implementation? 

3)  Do synergies with other policies of EU and other organisations/bodies exist? How can coherence with other non-EU instruments 

be improved? 
Success Criteria 

 

 

There is a clear EU added value of the MFA instrument and its operations 

The MFA instrument ant its operations are coherent with other instruments/policies and programmes and create synergies 

with them 

Indicators 

1)  Quantitative 

  Number of ex-post evaluations 

mentioning positive 
synergies/coherence and EU added 
value 

2)  Qualitative 
  Explanation of lessons 

learned/results/conclusions 
regarding EU added value and 
coherence 

  Connection of level of EU added 

value and design/implementation of 
MFA 

  Results/conclusions on EU and 

non-EU instruments/policies that 
synergise with MFA instrument 

Sources 
Desk research / literature review 
  Ex-post evaluation reports 
  Declarations European Parliament and Council (?) 
  Commission notes 
 

IMF reports 

Stakeholder consultation 
Interviews with:  

 

European Commission 

staff 

 

 

EEAS Delegations 

IMF country desks 

Specific questions: 
Interview questions: 
  What could Member States do on their 
own in the area of MFA operations? 
What is the value of EU action on top of 
that? 

  What lessons can be learned from 

previous MFA operations on how to 
maximise EU value added?  

  To what extent is the EU added value 

dependent on the design and 
implementation of MFA operations?  

  Are there synergies with other EU 

policies that can be further exploited 
(internal coherence)? 

  Are there synergies with policies of 

other organisations and bodies that can 
be further exploited (external 
coherence)?  
Is the MFA instrument fully in line with 
all EU external policy?  

 

  How can coherence with other EU 
external instruments be further 
enhanced? 

49 

 
 
 
 
 
 
EVALUATION CRITERIA: RELEVANCE and COHERENCE 

A 8. Are the general principles of the instrument, as stated in the 2013 Joint Declaration of the European 
Parliament and the Council13, still suitable in the current geopolitical and inter-institutional context so as to enable 
a swift and effective response where required? 
In this context, to what extent should the following general principles of MFA be reviewed? 
i. Trigger and aim of assistance (Principle 1) 
ii. Eligible countries and territories (Principle 2) 
iii. Form and amount of the assistance (Principle 3-5) 
iv. Conditionality (Principle 6) 
v. Procedure, implementation and disbursement of the assistance (Principles 7-9) 
What do we want to measure? 

1)  Are the general MFA principles still up-to-date, effective and efficient in the current geopolitical and inter-institutional 

environment? 

2)  To what extent should the general MFA principles 1-9 be reviewed? 
Success Criteria 

  MFA principles are still suitable/up to date, enabling swift and effective response where required 

Indicators 

  Number of OPC responses 

1)  Quantitative 

  MFA principles require little to no revision 
Sources 
Desk research / literature review 
  Ex-post evaluation reports 
  Declarations European Parliament and Council 
  Commission notes 
 

recommending the revision of the 
general principles 

IMF reports 

2)  Qualitative 

  Conclusions/results how the 
general principles are still 
suitable today 

  Explanation of which and how 

the general principles should be 
reviewed and how 

Stakeholder consultation 
Interviews with:  

 

European Commission 

staff 

 

 

 

EEAS Delegations 

IMF country desks 

Evaluators of MFA 

operations 

Specific questions: 
Interview questions: 
  Are the general principles of the 

instrument, as stated in the 2013 Joint 
Declaration of the European Parliament 
and the Council, still suitable in the current 
geopolitical and inter-institutional context 
so as to enable a swift and effective 
response where required? 
In this context, to what extent should the 
general principles of MFA be reviewed? 

 

50 

 
 
 
 
 
 
 
 
 
 
EVALUATION CRITERIA: RELEVANCE and EFFICIENCY 

A 8.1. To what extent the processes governing the MoU can be made more flexible and efficient, i.e.: 
a) Where is conditionality defined ex-ante for each instalment: can some flexibility be included during the 
implementation of the assistance and how? 
b) Assess the practice of waivers. Should the Commission have the possibility to move conditions from one tranche 
to the next? 
c) Assess the possibility of partial disbursements in case of partial implementation? 
d) Assess the possibility to combine tranches with a view of keeping within the availability period? Or extension of 
the availability period? 
What do we want to measure? 

1)  How can the processes of the MoU be made more flexible and efficient? 

2)  When conditionality is defined before, could it be somewhat adjustable while the MFA is being implemented? How can this 

flexibility be integrated? 

3)  The potential of using waivers and giving the Commission the option of moving conditions from one tranche to the next and their 

potential effects 

4)  The possibility of partial disbursement in case of partial implementation and the likely impacts 

5)  The possibility of combining tranches within the availability period or the potential of extending the availability period 
Success Criteria 

 

Processes governing the MoU are flexible and efficient with regard to the current terms and flexibility of conditionality 

Indicators 

1)  Quantitative 
  Number of flexible 

arrangements in the past 
operations 

  Number of OPC responses 
perceiving higher degree of 
flexibility in terms of 
conditionality, waivers and 
tranches, (partial) 
disbursements, and 
availability periods as positive 

2)  Qualitative 

  Explanation of if and how the 
processes/conditionality can 
be made more flexible (e.g. 
through waivers and 
tranches, (partial) 
disbursements, and 
availability periods) 
Indications and explanations 
of instances where 
conditionality was not 
sufficiently flexible to ensure 
success of its implementation 
and what the effect was on 
the country’s operation 

 

Sources 
Desk research / literature review 
  Ex-post evaluation reports 
  MoUs 
  Declarations European Parliament and Council 
  Commission notes 
 

IMF reports 

Stakeholder consultation 
Interviews with:  

 

European 

 

 

 

Commission staff 

EEAS Delegations 

IMF country desks 

Evaluators of MFA 

operations 

Specific questions: 
Interview questions: 
  Do you see a need to make the processes 

 

governing the MoU for MFAs more flexible and 
efficient? 
In this light, where conditionality is defined ex-
ante for each instalment: can some flexibility be 
included during the implementation of the 
assistance and how? 
a)  Or, should the Commission have the 

possibility to move conditions from one 
tranche to the next? 

b)  Or, should the Commission assess the 

c) 

possibility of partial disbursements in case 
of partial implementation? 
Lastly, should the Commission assess the 
possibility to combine tranches with a view 
of keeping within the availability period? Or 
extension of the availability period? 
  Do you see any other ways to make the 

processes governing the MoU for MFAs more 
flexible and efficient? 

51 

 
 
 
 
 
 
 
 
 
 
B 1. What effect have had the different methodological guidelines ("Guidelines for the Ex-post Evaluation of MFA 
operations") in force (2010 versus 2015) on the evaluations? How appropriate are the latest guidelines given 
updates to the Better Regulation Guidelines and the requirement to perform a Social Impact analysis (SIA) and 
Debt Sustainability analysis (DSA)? 
What do we want to measure? 
1) The effects of the different methodological guidelines (2010 criteria vs. 2015 Better Regulation criteria) on the outcome 
of the ex-post evaluations  
2) The differences and similarities between evaluation criteria (2010 vs. 2015) 
3) The benefits and shortcomings of Better Regulation guidelines in evaluating MFA operations 
4) The added value of Social Impact analysis (SIA) and Debt Sustainability analysis (DSA) to ex-post evaluations after 
2015 
Success Criteria 

 

Better Regulation Guidelines have improved achieving the evaluation objectives 

SIA and DSA have improved the assessment/understanding of the MFA operation’s effectiveness and overall success 

 
Indicators 

Quantitative 
 

Number of ex-post evaluations 
using 2010 methodology  
Number of ex-post evaluations 
using 2015 methodology 
Number of ex-post evaluations with 
SIA  
Number of ex-post evaluations with 
DSA 

 

 

 

Qualitative 

 

 

 

 

List and compare evaluation 
criteria used before and after 2015  
Potential differences in success 
rate of achieving set evaluation 
objectives between guidelines from 
2010 versus 2015 
Shortcomings (e.g., lack of data) 
described in the evaluations and 
potential differences between the 
evaluation guidelines 
Appropriateness of DSA and SIA, 
potential shortcomings raised with 
regards to the analyses 

Sources 

Desk research / literature review 

 

 

Ex-post evaluations  

Guidelines for MFA Ex post evaluations before and after 2015  

Stakeholder consultation 
Interviews with:  

 

Companies writing ex-post 

evaluations 

 

European Commission, 

e.g. Better Regulation   

Specific questions: 
Interview questions: 
  What effect have had the different 

methodological guidelines 
("Guidelines for the Ex-post 
Evaluation of MFA operations") in 
force (2010 versus 2015) on the 
evaluations? 

  What has effectively changed with 
introduction of the MFA Ex-post 
Evaluation structure (2010) and 
Guidelines for the Ex-post 
Evaluation of MFA operations 
(2015)?  

  How appropriate are the latest 
guidelines given updates to the 
Better Regulation Guidelines and the 
requirement to perform a Social 
Impact Analysis (SIA) and Debt 
Sustainability analysis (DSA)? 

B 2. To what extent did the changeover from the Commission's evaluation standards to the Better Regulation 
Guidelines affect the depth of assessment of the evaluations? 
What do we want to measure? 

1)  Differences in degree of topical coverage (i.e. macro financial stability, structural reforms, debt sustainability etc.) between 

the evaluation criteria (before and after the 2015 Better Regulation 

Success Criteria 

Topical and operational coverage has improved with the introduction of the Better Regulation Guidelines  

 
Indicators 
Quantitative 

 

Number of topical aspects 
covered 

Qualitative 

 

 

List which operational 
topics (i.e. 
macroeconomic 
conditions, fiscal policy, 
structural reform, social 
impact and public debt 
sustainability or other 
data) were covered under 
which evaluation criteria 
Comparison of which data 
and topics were not 
covered before the 2015 
Better Regulation 

Sources 
Desk research / literature review 
  Ex-post evaluations  
  Guidelines for MFA Ex post evaluations before and after 2015 

Stakeholder consultation 
Interviews with:  

 

 

Companies writing Ex 
post evaluations 
(particularly before and 
after 2015, covering 
both evaluation 
methodologies) 
European Commission   

Specific questions: 
Interview questions: 
  Does changes in evaluation methods with the 
introduction of Better Regulation Guidelines 
(post 2015) give more comprehensive and in-
depth assessment, given the larger scope 
compared previous evaluation method? 

52 

 
 
 
 
 
 
B 2. To what extent did the changeover from the Commission's evaluation standards to the Better Regulation 
Guidelines affect the depth of assessment of the evaluations? 

Guidelines for ex-post 
evaluations 

B 3. How adequate and comprehensive are the methodologies applied for the SIA and the DSA? 

What do we want to measure? 
1) Degree of comprehensiveness and adequacy of the SIA and DSA  
2) Methodologies applied for the SIA and DSA  
Success Criteria 

The evaluations did not encounter any shortcomings in providing a comprehensive SIA and DSA 

 
Indicators 
Quantitative 

 

Number of shortcomings 
found in relation to DSA 
and SIA 

Qualitative  

 

 

Description of social 
impact of the MFA 
operation and answers to 
evaluation questions for 
the SIA / description of 
shortcomings experienced 
when conducting the SIA 
Description of public debt 
sustainability and answers 
to evaluation questions for 
the DSA / description of 
shortcomings experienced 
when conducting the DSA 

Sources 
Desk research / literature review 
  Ex post evaluations using SIA and DSA (post 2015 Ex post evaluations) 
  Memoranda of Understanding  
  Better Regulation (2015) 

Stakeholder consultation 
Interviews with:  

Specific questions: 

 

Companies writing Ex post 

evaluations using SIA and 

DSA (post 2015 Ex post 

evaluations) 

 

European Commission  

B 4. Did the SIA and the DSA bring additional benefits to the assessment of the five evaluation criteria? 

What do we want to measure? 

  What kind of inputs from the SIA and the DSA improve the quality of conclusions of ex-post evaluations?  

 

Could ex-post evaluations that did not implement the SIA and the DSA be considered as ‘less comprehensive’? 

Success Criteria 

 

The SIA and the DSA methodologies provide more quality and depth of the evaluations and the analysis of the criteria   

Indicators 
Qualitative  

 

Data from the SIA 
methodology helping the 
comprehensive 
assessment of the MFA 
operation  

 

Data from the DSA 

methodology helping the 

comprehensive assessment 

of the MFA operation  

Sources 
Desk research / literature review 
  Ex-post evaluations (after 2015) 
  Better Regulation Guidelines (2015) 

Stakeholder consultation 
Interviews with:  

 

 

Companies writing Ex post 
evaluations using SIA and 
DSA (post 2015 Ex post 
evaluations) 
European Commission  

Specific questions: 
Interview questions: 
  What benefits and shortcomings to evaluations 
brings the Social Impact analysis (SIA) and the 
Debt Sustainability Analysis (DSA) that was 
added in the Guidelines for the Ex-post Evaluation 
of MFA operations (2015)?  

  To what extend does Debt Sustainability Analysis 
(DSA) go beyond Macro-Economic Stabilisation of 
MFA ex-post evaluation structure (2010)? 

B 5. How could the usefulness of the evaluations (and the Commission’s Staff Working Document) be maximised? 

What do we want to measure? 

 

How the ‘lessons learned’ from the past evaluations were implemented in the design of new MFA 
operations?  

Success Criteria 

 
 

The evaluations’ conclusions have been implemented in the design of new operations 
The results of the evaluations have been used by beneficiary countries  

Indicators 

Sources 

53 

 
 
 
 
 
B 5. How could the usefulness of the evaluations (and the Commission’s Staff Working Document) be maximised? 

Desk research / literature review 
  Ex-post evaluations  
  Memoranda of Understanding  

Stakeholder consultation 
Interviews with:  

 
 

European Commission  
National authorities from 
countries that were 
involved in consecutive 
MFA country operations  

Qualitative 

 

List of key 

recommendations and 

conclusions of the 

evaluations 

 

Understanding the link 

between the design of MFA 

operations and previous ex-

post evaluations (i.e. in the 

same country or other earlier 

evaluations with 

recommendations on future 

MFA designs)  

Specific questions: 
Interview questions: 
  How useful and usable were the 

conclusions and recommendations of the 
evaluations? How could it be improved? 

B 6. How useful and usable were conclusions and recommendations (i.e. the way the evaluation design shaped the 
extent to which its outputs could be used)? 
What do we want to measure? 

1)  Were the evaluation conclusions non-biased and fully based on findings from the analysis? Did they highlight key insights, 

successes and shortcomings of the programme? 

2)  Were areas that needed improvement identified in coherence with the conclusions? Was the target audience of the 

recommendations, their scope and timeframe clearly defined? 

3)  Were suggested recommendations clear, realistic and impartial?  

4)  How many of the provided recommendations were addressed? To which extent?  

5)  How many of the provided recommendations were not addressed? What was the reason for that?  
Success Criteria 

 

The evaluation conclusions were non-biased and fully based on findings from the analysis. They highlight key insights, 

successes and shortcomings of the programme; 

 

 

 

 

Areas that needed improvement were identified in coherence with the conclusions;  

Target audience of the recommendations, their scope and timeframe were clearly defined; 

Recommendations were clear, realistic and impartial; 

(A big number of) recommendations were taken into account and addressed in due course; 

Conclusions and recommendations are considered useful and usable by stakeholders. 

 
Indicators 
1)  Quantitative 
  For each evaluation, number of 
recommendations that were 
subsequently addressed; 
  For each evaluation, number of 
recommendations that were 
rejected/ not taken into account  

2)  Qualitative 
  The extent to which the provided 

recommendations were 
addressed 

  For recommendations that were 
not addressed – reasons for that 
(were the provided 
recommendations not useful and 
usable enough, or some other 
factors interfered, not related to 
the evaluation design?)  

  The extent to which the provided 

recommendations were 
considered useful and usable by 

Sources 
Desk research / literature review 
  Ex-post evaluation reports 

Specific questions: 
Interview questions: 
 

 How useful and usable were the 
conclusions and recommendations of the 
evaluations? How could it be improved? 

Stakeholder consultation 
Interviews with:  

 

European 

Commission staff 

 

 

EEAS Delegations 

Evaluators of MFA 

operations 

54 

 
 
 
 
 
B 6. How useful and usable were conclusions and recommendations (i.e. the way the evaluation design shaped the 
extent to which its outputs could be used)? 

stakeholders 

B 7. Were the results adequately disseminated? How effectively has information about the evaluations, including 
the conclusions and executive summary been transmitted to stakeholders?  
What do we want to measure? 

1)  Were the evaluation results adequately disseminated? 

2)  Has information about the evaluations, including the conclusions and executive summary, been effectively transmitted to 

stakeholders? 

3)  Are evaluation conclusions and executive summary available in the official language of the country concerned?  
Success Criteria 

 

Evaluation results were disseminated among all relevant stakeholders and, where relevant, general public 

Evaluation conclusions and executive summary are available in the official language of the country concerned 

 
Indicators 
1)  Qualitative 
  Modes of dissemination of 

evaluation results 

  Stakeholder groups that 
were and were not 
covered by the 
dissemination activities 
  Availability of evaluation 

conclusions and executive 
summary in the official 
language of the country 
concerned 

  Availability of the 

evaluation results in open 
access 

Sources 
Desk research / literature review 
  Ex-post evaluation reports 
  Websites of relevant national authorities 
  Websites of EEAS Delegations to the concerned countries   

Stakeholder consultation 
Interviews with:  

 

 

 

European Commission staff 

EEAS Delegations 

Evaluators of MFA 

operations 

Specific questions: 
Interview questions: 
  Were evaluation results adequately 

disseminated?  

  How effectively has information about the 
evaluations, including the conclusions and 
executive summary been transmitted to 
stakeholders? 

  How could dissemination be improved? 

B 8. To what extent did the Commission follow-up on the results and conclusions? Are the latter pertinent as 
regards the future use of the MFA instrument? 
What do we want to measure? 

1)  What follow-up actions on the results and conclusions are undertaken by the Commission? 

2)  Are Commission’s follow-up actions pertinent as regards to the future use of the MFA instrument?  

3)  How could Commission’s follow-up actions be improved? 

Success Criteria 

  MFA decision documents refer to past conclusions and recommendations 

 

Commission regularly updates its "Internal charter for the evaluation function in DG ECFIN” 

Knowledge from past evaluations is used to design future evaluations 

 
Indicators 
1)  Quantitative 
 

For each evaluation, 
number of conclusions 
and recommendations 
that were subsequently 
addressed/ mentioned in 
official Commission 
documents 

2)  Qualitative 
 

The extent to which 
references to past 
evaluation results and 
conclusions are present 
in MFA decision 
documents and other 
official Commission 

Sources 
Desk research / literature review 
  Ex-post evaluation reports 
  MFA decision documents 
 
  Other official documents of the European Commission 
  Commission notes 

Internal charter for the evaluation function in DG ECFIN  

Stakeholder consultation 
Interviews with:  

 

European 

Specific questions: 
Interview questions: 
 

 Could you use knowledge from past evaluations to 
design future evaluations? If so, how? 

Commission staff 

  To what extent did the Commission follow-up on the 

 

Evaluators of MFA 

operations 

 

results and conclusions? 
Is the Commission regularly updating its "Internal 
charter for the evaluation function", as 
recommended in previous evaluations? 

55 

 
 
 
 
 
B 8. To what extent did the Commission follow-up on the results and conclusions? Are the latter pertinent as 
regards the future use of the MFA instrument? 

 

documents 
The extent to which 
knowledge from past 
evaluations is used to 
design future evaluations 

Source: Valdani, Vicari e Assoiati & Centre for Strategy and Evaluation Services, Oxford Analystics, 2021 

Answers to the evaluation questions 

Relevance 

On the basis of past operations, the evaluator draws a positive conclusion on the relevance 
of  policy  reforms  promoted  through  MFA.  As  such,  MFA  had  an  operational  and/or 
political reinforcing effect in all domains in which conditions were formulated (Moldova). 
In many cases, they hence acted as a positive contributing factor, rather than a sole causal 
factor of a reform (Ukraine). And in a similar vein, MFA promoted actions that might not 
have  been  implemented  otherwise  while  in  some  other  instances,  the  authorities  would 
probably  have  taken  action  also  without  the  MFA  conditions  (Moldova).  In  terms  of 
scope,  they  addressed  important  issues,  even  if  they  did  not  always  comprehensively 
cover all of the most pressing reform needs (Bosnia and Herzegovina). Reforms in Public 
Financial  Management  (PFM)  and  the  social  safety  net,  where  conditions  are  frequent, 
were particularly relevant (Ukraine). 

Against  this  backdrop,  the  evaluator  concludes  that  the  MFA  instrument  shows  good 
relevance  in  most  countries.  Structural  reform  conditions  enshrined  in  the  MoU  are 
relevant  and  most  of  them  have  been  implemented  as  agreed.  Only  in  a  few  cases  did 
implementation of the agreed reforms prove slow and ownership of the reform programme 
limited (e.g. Tunisia 14). 

Continuous  demand  for  MFA  operations  from  beneficiaries  demonstrates  the  persistent 
relevance of the instrument  (in the past  and present).  In the evaluation  period, MFA has 
been  requested  and  employed  15  times.  Moreover,  10  operations  were  requested  and 
authorised  at  the  inception  of  the  COVID-19  pandemic  in  2020.  The  Commission 
received further requests by end-2021, despite the increased availability of global liquidity 
given the 2021 general allocation of special drawing rights (SDRs) by the IMF. 

Effectiveness 

According  to  the  external  evaluation  study,  MFA  was  overall  effective  in  stabilising 
beneficiary’s (external) macroeconomic environment, especially in the short and medium-
term. Despite a challenging external environment at the inception of the respective MFA 
operations, market access could be maintained or regained in all cases. This includes one 
beneficiary (Ukraine) restructuring its debt in good faith negotiations. With the exception 
of one strategic default (by Ukraine on the Russian Federation), there were no defaults by 
beneficiaries.  Reserve  adequacy  and  current  accounts  mostly  improved  during  MFA 
operations, and inflation subsided.  

These findings are more nuanced for time horizons beyond the medium term, based on the 
evaluator’s  analysis.  Whilst  some  improvements  in  current  account  balances  are  visible 
over  that  time  window,  she  finds  that  gains  in  reserve  adequacy  could  usually  not  be 

56 

 
 
 
 
extended further and current accounts remained in deficit. Furthermore, participants at the 
validation workshop lent support to this interpretation of the data. 

The  instrument’s  geographical  focus,  design  and  form  played  a  supportive  role  in  the 
effectiveness of MFA in incentivising meaningful reform. 

On the other hand, the long procedure for authorising MFA operations adds risks to their 
effectiveness. In particular, the time lag up until the MoU (and thus the MFA) enters into 
force  eats  into  the  term  available  for  implementing  reforms  and  risks  some  unwanted 
economic  corrections  in  the  meantime  since  supportive  external  funding  is  not  yet 
available.  Furthermore,  the  evaluator  points  to  one  country  (i.e. Armenia  09)  in  urgent 
need of financial assistance that chose to seek frontloading from the IMF given the MFA’s 
lengthy procedures. 

Notwithstanding the relatively high  compliance rates, the MoU’s inflexibility appears to 
have  reduced  the  MFA’s  effectiveness  in  times  of  change.  In  contrast  to  Union  budget 
support, MFA instalments are not disbursed in part and proportion to fulfilled conditions 
but only at 100% or 0%. This reflects IMF policies, but in contrast to those policies, MFA 
practice  is  that  conditions  are  fixed  at  the  operation’s  inception  and  remain  unchanged 
later in the operation when the national parliament ratified the text.  

In  any  event,  ownership  of  the  agreed  reforms  is  of  critical  importance  for  the  MFA’s 
effectiveness in promoting important reforms. 

Efficiency 

The external  evaluation  confirms  that MFA is  a  relatively (cost-)efficient instrument  for 
the  Union.  MFA  loans  are  provisioned  at  9%  of  the  principal  amount  to  be  disbursed. 
Given  the  fact  that  no  beneficiary  ever  defaulted  on  an  MFA  loan,  these  9%  have  been 
routinely returned to the budget, bringing the budgetary impact of MFA loans over time 
down to zero. Grants, by contrast, have to be budgeted fully. 

The parallelism with IMF interventions provides some assurance concerning the country’s 
repayment capacity, also since the IMF does not usually provide funding to countries with 
an unsustainable debt (unless it is taking measures to address this issue). In addition, the 
synergies arising from relying on the IMF’s work provides for important cost-savings on 
the  Union  side.  Similarly,  the  geographical  focus  provides  for  synergies  in  terms  of 
expertise and human resources across and within different DGs. 

MFA  is  also  cost-efficient  for  the  beneficiary  countries,  since  its  loans  are  highly 
concessional. Due  to its  high  creditworthiness, the  EU can  borrow  money at much  lower 
interest  rates  than  the  beneficiary  countries  would  get  if  they  were  to  go  to  the  market 
themselves. As  the  Union  passes  on  these  interest  rates  in  its  loan  agreements,  the 
beneficiary countries receive an often substantial financial benefit.  

The  current  EU  decision-making  arrangements  entail  some  inefficiencies.  The  time 
needed  for  the  co-legislators  to  decide  on  the  Commission’s  proposal  generally  takes  5 
months. In addition, before the MFA enters into force, the MoU has to be formally agreed; 
that  requires  a  Commission  implementing  act  subject  to  Comitology.  It  takes  roughly  7 
months  to  conclude  the  negotiations  with  the  beneficiary  country  and  adopt  the  MoU, 
which  can  be  shorter  when  the  negotiations  happen  for  a  large  part  in  parallel  to  the 
legislative process. 

57 

 
 
Questions  can  also  be  raised  over  the  efficiency  of  the  decision-making  process  in 
beneficiary  countries,  where  MoUs  are  oftentimes  ratified.  Whilst  this  can  provide  for 
stronger ownership of the reforms to be implemented, it tends to increase the period up to 
the disbursement of the assistance and all but prohibits adaptations. 

Coherence 

MFA  usually 
instruments,  such  as 
ENI/ENPI/NDICI  and  IPA.  This  results  in  important  synergies  chiefly  with  budget 
support under these instruments. 

tandem  with  other  Union 

intervenes 

in 

The  stakeholder  consultation  shows  that  external  coherence  with  IFI  interventions  is 
generally  seen  as  high  and  supporting  the  workings  of  the  instrument.  Stakeholders 
mainly  referred  to  complementarity  and  lack  of  overlap  between  the  conditions  of  the 
different actors after accounting for their different interests, mandates and approaches.  

In  what  concerns  the  internal  coherence,  the  stakeholder  consultation  suggests  that  the 
MFA  has  been  developed  in  a  consistent  and  clear  manner,  so  that  the  implementation 
follows the same set of principles and rules  in all beneficiary countries. Coherence with 
the Union’s external policies allows economies of scope (within the Commission and the 
EEAS)  to  an  extent  that  in  particular  smaller  Member  States  would  have  difficulties  to 
mobilise  otherwise.  Such  value  added  hence  tends  to  increase  with  the  volume  of  the 
operation. 

EU value added 

From the Member States’ perspective, Union MFAs provide financial and political value 
added.  The  re-financing  of  MFA  loans  by  means  of  Union  borrowing  does  not  affect 
Member States’ finances or debt ratios whilst allowing support in regions that they have 
interest in.  

From  the  perspective  of  beneficiaries  an  added  value  of  MFA  is  recognised  by  all 
consulted stakeholders. According to them, a majority of the reforms implemented as part 
of MFA’s reform requirements would not have taken place otherwise, or not as quickly – 
e.g.  reforms  in  the  area  of  public  financial  management,  anti-corruption  and  judicial 
system. This is notwithstanding the fact that the scope and the overall impact of the MFA 
operations in the countries was relatively small in comparison to the overall EU presence 
there. 

As  some  key  stakeholders  suggest,  MFA  supports  the  Union’s  unique  role  in  fostering 
especially reforms  to  strengthen the rule of law.  Interviewed stakeholders both  from  the 
Commission  and  from  the  IFIs  suggested  that  not  all  conditions  imposed  by  the  MFA 
were  affecting  directly  macro-economic  stability  in  the  beneficiary  countries.  However, 
stakeholders  assessed  this  characteristic  of  the  MFA  conditions  as  an  advantage, 
especially since the MFA in the beneficiary countries works alongside the IMF and other 
Union  instruments.  Unlike  the  IMF,  which  is  committed  to  strong  neutrality,  MFA 
promotes  democratic  values  directly  as  well  as  other  values  that  are  of  interest  to  the 
Union and the beneficiary countries.  

58 

 
 
However,  interviewed  stakeholders  have  noted  that,  in  most  beneficiary  countries,  the 
visibility of the MFA was rather low. This lack of visibility reduced the added value of the 
MFA,  as  the  instrument  was  under-exploited  for  the  promotion  of  EU  economic 
diplomacy in beneficiary countries and their geographical surroundings. 

*** 

With  the  2015  “Guidelines  for  the  Ex-Post  Evaluation  of  Macro-Financial  Assistance 
Operations”,  the  MFA  ex-post  evaluations  started  following  the  Commission’s  Better 
Regulation  Guidelines  (BRG).  Whilst  previous  evaluations  did  examine  effectiveness, 
efficiency,  coherence,  relevance  and  EU  added  value,  the  BRG  provide  more  detailed 
guidance  on  the  general  approach  to  be  used.  More  specifically,  BRG  brought  them 
together,  gave  them  a  more  precise  and  standardised  definition,  and  placed  them  at  the 
heart  of  the  evaluation  framework  for  EU-supported  interventions  including  the  MFAs. 
Consequently,  the  post-BRG  ex-post  evaluation  reports  were  structured  more  closely 
around  the  five  criteria  after  2015  than  was  the  case  up  to  that  point,  according  to  the 
evaluator’s analysis. 

In  addition  to  the  five  evaluation  criteria,  the  BRG  encouraged  a  more  in-depth 
assessment  of  social  impacts  than  was  evident  before.  With  the  DSAs  the  difference 
between  the  pre-  and  post-BRG  level  of  analysis  was  less  pronounced  as  this  type  of 
analysis was already a key element in the earlier ex-post evaluations. 

The  BRG  also  placed  increased  emphasis  on  the  systemic  involvement  of  key 
stakeholders (both in terms of impact assessments and ex-post evaluations). 

ANNEX  IV.  OVERVIEW  OF  BENEFITS  AND  COSTS  AND,  WHERE  RELEVANT,  TABLE  ON 
SIMPLIFICATION AND BURDEN REDUCTION. 

The  typology  of  costs  in  the  Better  Regulation  toolbox  (defined  as  compliance, 
enforcement  and  other  indirect  costs)  is  impossible  to  apply  in  the  context  of  MFA 
operations, as such costs are not incurred here. As indicated in the Legislative Financial 
Statement accompanying Commission’s proposals for MFA, the budgetary impact at the 
level of the EU is limited to some administrative costs related to HR and outsourcing of 
operational assessments and ex-post evaluations. 

The  typology  of  benefits  (improved  welfare,  market  efficiency  and  wider  economic 
effects)  can  seem  to  be  more  relevant.  However,  as  shown  in  past  country  ex-post 
evaluations,  which  tried  to  quantify  the  macroeconomic  effects  of  the  MFA  assistance, 
either  by  counterfactual  modelling  or  by  other  more  qualitative  methods,  it  is  almost 
impossible to disentangle the effects of MFA from IMF assistance (both disbursed at the 
same time) and from other macroeconomic events. 

The  Guidelines  for  carrying  out  ex-post  evaluations  of  MFA  operations  of  June  2015, 
acknowledged the problem and stated that: 

[…]  EU  added  value  may  be  the  results  of  different  factors:  coordination  gains,  legal 
certainty,  greater  effectiveness  or  efficiency,  complementarities  etc.  In  all  cases, 

59 

 
 
 
concluding  on  the  continued  need  for  the  intervention  at  EU  level  may  be  difficult  as 
measuring EU added value is challenging. 
The  analysis  of  EU  added  value  is  often  limited  to  the  qualitative,  given  the  stated 
difficulties to identify a counterfactual. 

More specifically in  the  field  of ex-post evaluation  of MFA, challenges  can arise  which 
cannot be easily discarded and should be taken into consideration:  

  The  nature  of  the  instrument:  MFA  is  not  earmarked  in  any  way.  It  is  an 
instrument  complementing  an  IMF  programme.  It  cannot  be  linked  directly  to 
identifiable outputs as in the case of programme or project-linked aid;  

  Objectives of MFA operations can be implicit for political reasons and when they 
are explicitly presented in an MoU, they can sometimes be not specific in nature;  
  The  size  of  the  assistance:  MFA  grants/loans  are  provided  in  addition  to  the 
provision of an IMF loan and may represent a small proportion of the total funds 
accorded  to  the  recipient  country.  Thus  the  financial  components  of  both 
instruments cannot usually be easily or indeed meaningfully disassociated; 

The consequence of these challenges is that: 

  A classic assessment of  efficiency (i.e. cost-effectiveness) is  not  feasible, though 
an  examination  of  the  suitability  of  the  blend  of  grants  and  loans  making  up  the 
assistance  and  other  questions  about  its  design  and  implementation  and 
consequences for the achievement of objectives are pertinent issues.   

  Effects  on  macroeconomic  variables  over  time  cannot  be  uniquely  attributed  to 
MFA,  requiring  that  analyses  take  into  account  the  global  package  of  which  the 
assistance is a part. 

Past  country  ex-post  evaluations  have  taken  this  guidance  into  account  and  have  come 
with an evaluation methodology suited for the MFA instrument. 

ANNEX V. STAKEHOLDERS CONSULTATION - SYNOPSIS REPORT  

This  annex  presents  an  outline  of  the  consultation  strategy  adopted  for  the  meta-
evaluation.  It  is  largely  based  on  the  consultation  strategy  developed  as  part  of  the 
independent study undertaken by Oxford Reseach, in collaboration with Valdani, Vicari e 
Associati  and  the  Centre  for  Strategy  and  Evaluation  Services.  The  annex  details  the 
objectives  of  the  strategy  adopted,  the  stakeholders  and  tools  identified,  and  its  final 
dissemination. 

Objectives 

The consultation  strategy  followed the indications provided by the Commission’s Better 
Regulation Tool #54. 

The specific goals of the consultation activities were to:  

1.  Collect additional data on both MFA principles and characteristics and evaluation 

methodologies;  

60 

 
 
 
  
 
2.  Engage with stakeholders involved in the delivery and evaluation of MFAs; and  
3.  Validate the preliminary findings of the meta-evaluation.  

In order to ensure transparency and engagement, the process  followed the standards and 
methods  set  out  in  the  Better  Regulation  Guidelines  (BRG).  The  various  consultations 
complied with the Terms of Reference (ToR) and the following evaluation roadmap34. 

Identified stakeholders and tools 

The stakeholders targeted by the consultation activities were individuals or organisations 
who held a specific role in one or multiple MFAs and/or subsequent evaluations. The table 
below shows the stakeholder groups targeted through the consultation activities. 

Table A4.1 Identified stakeholders and tools 

Stakeholder group 

Beneficiary countries 

Civil society (individual citizens, unions, NGOs…) 

European Commission and other EU Institutions 

International  financial  institutions  (IMF  and  World 
Bank) 

Meta-evaluation contractor 

Previous evaluators 

Rationale for selection 

The  meta-evaluation  covers  past  MFA  operations 
(2010-2020) in 11 countries.  

For purposes of transparency and participation, it is a 
policy  of  the  European  Commission  to  openly  seek 
the views of citizens and stakeholders.  

DG  ECFIN  commissioned  the  meta-evaluation  and, 
together  with  EEAS,  DG  NEAR,  DG  INTPA,  the 
European  Parliament,  The  Council  of  the  European 
Union  and  other  EU  institutions,  conducted  past 
MFA operations.  

arrangements. 

in  combination  with  IMF 
MFAs  are  granted 
financial 
Therefore,  MFA 
implementation entails cooperation and coordination 
with  international  financial  institutions.  While  the 
IMF  runs 
it  often 
the  financial  arrangement, 
combines  efforts  with  ongoing  World  Bank 
programmes and expertise in each specific country.  

VVA  Economics  &  Policy,  the  Centre  for  Strategy 
(CSES),  and  Oxford 
&  Evaluation  Services 
Research carry out the Meta-evaluation study.  

The  previous  evaluations  of  MFA  operations  were 
conducted  by  external  companies  on  behalf  of  the 
European Commission.  

Source: Valdani, Vicari e Assoiati & Centre for Strategy and Evaluation Services, Oxford Analystics, 2021 

The  consultation  strategy  employed  five  different  consultation  tools,  following  the  ToR 
provisions  and  agreement  with  DG  ECFIN.  The  table  below  illustrates  the  type  of 

34 The meta-evaluation roadmap ran from 21 January 2021 to 18 February 2021. 

61 

 
 
 
 
 
 
 
 
                                                           
consultation tools used per each of the two parts of the external study, together with the 
seminar held in November. 

Table A4.2 Key methodological tools and relevance for meta-evaluation 

Part  A  –  MFA  principles  and 
characteristics 

Part  B  –  MFA  evaluation 
methodologies 

Semi-structured interviews 

X 

Open Public Consultation (OPC)  X 

Validation workshop 

Roundtable 

Seminar 

X 

X 

X 

X 

X 

Source: Valdani, Vicari e Assoiati & Centre for Strategy and Evaluation Services, Oxford Analystics, 2021; 
EC 

Putting  together  the  two  dimensions,  the  table  below  illustrates  the  type  of  consultation 
tool used per category of stakeholder. 

Table A4.3 Key methodological tools and relevance for meta-evaluation 

Consultation tools 

Semi-
structured 
interviews 

OPC 

Validation 
workshop 

Roundtable 

Seminar 

X 

X 

X 

Beneficiary 
countries 

Civil society 

European 
Commission 
and 
other EU institutions 

IFIs 

Meta-evaluation 
contractor 

Previous evaluators  X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Source: Valdani, Vicari e Assoiati & Centre for Strategy and Evaluation Services, Oxford Analystics, 2021; 
EC 

62 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Final outreach 

The total reach of the consultation activities, excluding the seminar, is shown below. 

Table A4.4 Total reach of consultation activities 

Stakeholder group 

Timeframe 

Number  of  individual  inputs 
obtained 

Semi-structured interviews 

22 February 2021 – 15 June 2021 

Open Public Consultation 

30 March 2021 – 22 June 2021 

Validation workshop 

22 June 2021 

Roundtable 

10 June 2021 

50 

20 

22 

12 

Total 

22 February – 16 November 2021 

104 

Source: Valdani, Vicari e Assoiati & Centre for Strategy and Evaluation Services, Oxford Analystics, 2021 

Delivery of the consultation  

This section presents the delivery of the completed consultations per type of activity.  

A.  Semi-structured interviews 

A total of 50 in-depth interviews were conducted with key stakeholders, all of which were 
conducted via teleconferences. The interviews were distributed as follows:  

  26  from  the  European  Commission  (DG  ECFIN,  DG  NEAR,  DG  INTPA,  DG 

BUDGET, Cabinet staff, Commission Legal Service (SJ);  

  6 from the European External Action Service (EEAS);  
  1 from the European Parliament;  
  3 from the Council of the European Union and EU Member States;  
  2 from authorities in beneficiary countries;  
  4 from consulting companies/past evaluators; and  
  8 from international financial institutions (IMF and World Bank). 

Taking  into  account  the  restraints  of  varying  responsiveness  among  stakeholders,  the 
research team aimed to collect information from the most relevant stakeholder groups in a 
balanced way. While the primary focus of the interview programme has been on the role 
of  the  EU  institutions  in  designing  and  implementing  the  MFA,  the  study  team 
interviewed  country  experts  (mainly  from  the  Commission  and  EU  Delegations)  in  all 
countries benefitting from MFA, hence ensuring a substantial geographical diversity. 

B.  Open public consultation 

63 

 
 
 
 
The aim of the open public consultation (OPC) was to collect views and opinions about 
the MFA operations from a wide range of stakeholders. Data collected through the OPC 
complemented  the  findings  obtained  through  semi-structured  interviews  and  document 
analysis.  
The  OPC  was  launched  on  30  March  2021  and  remained  open  until  22  June  2021, 
collecting  views  from  20  respondents,  of  which  12  came  from  officials  at  the  European 
institutions and EU delegations. Public  authorities in  beneficiary  countries  provided  one 
answer, while authorities and officials from EU Member States provided five replies. Two 
replies  came  from  individuals,  one  from  a  beneficiary  country  and  one  from  an  EU 
Member  State.  No  respondent  chose  the  options:  academic/research  institution,  business 
association  company/business  organisation,  consumer  organisation,  environmental 
organisation, non-EU citizen or trade union. 

C.  Workshop 

An  online  workshop  with  evaluation  experts  in  the  field  of  macro-financial  assistance 
(MFA) was organised on Tuesday June 22, 2021. At this point, the research process was 
almost finalised, with the desk research, interviews, and OPC all completed. This meant 
that  draft  conclusions,  for  both  Part  A  and  Part  B  of  the  evaluation,  were  available  for 
discussion and validation during the workshop. 

The  workshop  gathered  key  stakeholders  and  previous  evaluators  of  MFA  operations  in 
order to validate the preliminary findings of the study. The feedback gathered was used to 
fine  tune  the  final  report.  Overall,  the  participants  confirmed  the  findings  and  further 
developed some of them.  

More specifically, the purpose of the validation workshop was to:  

  Review and validate the overall draft findings of the meta-evaluation of the MFA;  
  Discuss the overall draft conclusions of the evaluation, and their implications for 

the content of future MFA operations and related programmes; and  

  Consider any lessons to be learned from the study for the Commission’s guidelines 

on programme evaluation, specifically the MFA operations.  

To  do  so,  the  research  team  reached  out  to  a  total  of  52  stakeholders  from  27  different 
organisations, with a final attendance of 22 participants from 14 organisations. 

D.  Roundtable 

On  June  10,  2021,  the  meta-evaluation  contractor  and  DG  ECFIN  co-organised  a 
roundtable with selected Commission staff working on MFA operations. The roundtable 
saw the participation of a total of 12 people.  

The  focus  of  the  discussion  was  on  identifying  the  core  limitations  encountered  in  the 
MFA  operations  and  the  potential  lessons  learned  for  future  interventions.  In  particular, 
the roundtable touched on four topics:  

  Conditionality,  including  political  pre-condition,  complementarity  with  IMF 

financial arrangement, and structural reform MoU policy conditions;  

64 

 
 
  
 
 
 
 
 
 
  Macroeconomic stability;  
  Reforms sustainability; and  
  Coherence. 

E.  Seminar 

On November 16, 2021, Directorate D organised an (online) event celebrating 30 years of 
the Macro-financial assistance (MFA) instrument. The event  aimed at taking note of the 
evolution of the instrument over the past 30 years, give visibility and provide useful input 
for its future. Politically, the event was intended to raise awareness of the instrument; the 
complementarity and value added compared to other forms of external policy instruments; 
how its set-up provides particular leverage, allowing the EU to advance its values; and the 
MFAs  importance  in  contributing  to  stability  and  prosperity  in  our  neighbourhood.  The 
event provided a forum for in-depth discussion on the MFA’s pros and cons by gathering 
key  stakeholders;  addressed  limitations  encountered  so  far  and  pointed  to  potential 
improvements. The event was comprised of two high-level panel discussions, case studies 
on the MFA operations for Ukraine and Jordan as well as discussions and Q & As. 

Almost 200 participants took part in the event: 
  EU  Institutions:  DGs  ECFIN  and  NEAR  as  well  as  the  European  External  Action 
Service were represented at senior-management level, the Council at technical level. 

  EU Member States were represented at technical level. 
  Partner countries in focus were represented at senior level. 
  Public  policy  consultancies  and  independent  international  financial  institutions  took 

also part in the event.  

General appraisal  

Overall,  the  MFA  has  been  rated  very  positively  across  all  intervening  stakeholders  for 
both its ability to support the restoration of macro-economic stability as well as its role in 
promoting  structural  reforms  in  recipient  countries.  Moreover,  it  was  praised  for  its 
complementing  role  to  other  EU  foreign  policy  tools,  constituting  a  powerful  lever. 
Following  senior  officials  from  the  IMF  and  COM  services,  to  recipient  countries,  the 
MFA instrument, in its close alignment and collaboration with IMF missions, provides a 
trigger  to  enact  and  a  framework  to  anchor  reform  policies.  While  being  a  very  case-
specific tool, general shortcomings were identified around the instruments inflexibility on 
conditionality,  its  visibility  and  its  role  as  a  crisis  instrument  that  under  the  current 
structural set up requires – in some cases lengthy – internal as well as external processes 
to be enacted.  

Role of MFA in EU external policies  

Participants  from  COM  services,  an  external  evaluator  and  IMF  officials,  considered 
MFA  to  have  an  important  role  in  the  EU’s  external  policy  tool  kit,  supporting  and 
complementing  other  instruments.  Central  to  the  tool’s  relevance  and,  according  to  a 
senior official from the COM services, one of the strongest features of the instrument is 
the  successful  close  collaboration  with  the  IMF,  beneficiary  countries  authorities  and 
other  EU  policies  on  the  bilateral  level,  e.g.  budget  support  efforts  and  technical 
assistance.  

65 

 
 
 
 
Moreover,  the  alignment  of  overall  policy  objectives  and  conditions  with  these 
instruments creates strong synergies and allows the EU to put weight on key priorities. In 
this  regard,  MFA  conditions  can  support  longer-term  cooperation  and  structural  reform 
efforts  by  enhancing  incentives  from  an  additional  angle,  also  given  the  sheer  and 
somewhat  unique  size  and  the  resulting  leverage,  as  senior  officials  from  COM/EEAS 
services  noted.  Furthermore,  the  close  collaboration  and  cross-conditionality  with  IMF 
interventions  carries  the  message  of  a  combined  undertaking  of  the  international 
community, further increasing the weight of conditions.  

Conditions 

Next to the financial assistance, the conditions attached to the disbursement of funds are 
an  important  pillar  of  MFA.  The  generally  large  pertinence  of  MFA  conditions  was 
highlighted  by  COM/EEAS  and  IMF  officials  and  stressed  by  high-level  officials  of 
beneficiary countries authorities. They emphasized that through their conditionality, MFA 
operations  add  incentives  for  national  decision-makers  to  also  undertake  unpopular 
reforms, creating a political “window of opportunity for home-grown reform efforts”.  

On the all-important issue of potential backsliding on reforms, a high-level official from a 
beneficiary country authority pointed out that such incidences could be considered proof 
of the concerned condition’s necessity. At the same time, beneficiary countries authorities 
also argued that politically sensitive reforms, e.g. aiming at changes to the constitution or 
the passage of laws, should be avoided, as they are very hard to fulfil in changing political 
conditions  and  outside  of  the  executive  powers.  Moreover,  they  highlighted  that  more 
focused conditions would lead to more success in meeting them – “rather two small steps 
than one failed big step”.  

Noting that, frequently, key reforms, particularly regarding the rule of law, are politically 
sensitive but of utmost importance for overall reform efforts, COM & IMF officials and 
an  external  evaluator  nonetheless  supported  more  flexibility  and  focus  of  conditions  for 
three reasons:  
1.  Beneficiary countries, by nature of the instrument, are usually in situations not only of 
financial, but also political instability that require the recalibration of conditions with 
changing environments.  

2.  As a crisis instrument, MFA is influential only during a limited time period and often 
met with scarce capacities in beneficiary countries, begging fewer and more nuanced 
conditions.  

3.   When  facing  political  deadlock,  applying  leverage  through  alternative  angles  by 
reorienting  conditions,  e.g.  transparency  measures  to  increase  public  attention  rather 
than changes to laws, can be more successful.  

Processes and procedures (internal and external)  

On  the  internal  processes,  the  discussions  revealed  that  the  current  institutional  set-up 
involving  different  EU  institutions  and  case-by-case  decisions  can  lead  to  lengthy 
negotiations  that  collide  with  the  instrument’s  ability  to  function  as  a  crisis  instrument. 
However,  as  COM/Council  officials  argued,  the  underlying  democratic  process  is  a 
crucial  measure  of  accountability  towards  EU  taxpayers.  Similarly,  the  alignment  of 
overall  policy  objectives,  cross-conditionality  with  internal  and  external  partners  is  a 
crucial  component  for  the  instrument’s  success  that  requires  time.  Another  inherent 

66 

 
 
difficulty to this process is the realization of compromises between the varying priorities 
among  internal  actors.  Meeting  these  expectations  risks  overburdening  local  capacities 
and  the  long-term  perspective  of  conditions  sometimes  deviates  from  the  short-term 
nature of the instrument.  

Nonetheless,  as the recent Covid-MFA has demonstrated, this process  can be completed 
sufficiently fast for the instrument to be rolled out in due time. Among participants, broad 
support was voiced for more flexibility in the internal process concerning the termination 
of operations when conditions  are not  fully met,  as  this  often constitutes the collapse of 
the  dialogue  with  beneficiary  countries  authorities  when  it  is  needed  the  most.  The 
external  processes,  particularly  the  negotiating  of  Memoranda  of  Understanding,  vary 
from case and size of the MFA operation. The continued collaboration and coordination 
with other EU organs and the IMF were highlighted as crucial to create synergies as well 
as to avoid redundancies and generally praised as successful in advancing common goals 

MFA’s impact within the beneficiary country  

Experience  shared  by  COM/EEAS  services  officials  and  high-level  participants  from 
beneficiary countries authorities confirm the MFA’s significant positive macro-economic 
effects  as  well  as  a  positive  reinforcing  role  in  promoting  structural  reforms.  Whilst 
contingent  on  the  size  and  scope  of  the  operation,  the  highly  concessional  nature  of  the 
instrument regarding interest rates, grace periods and maturities allows recipient countries 
to  restore  financial  and  generally  macro-economic  stability,  yielding  positive  effects  on 
the  overall  macro-economic  situation  and  generating  EU  added  value.  Being  most 
effective in times of acute crisis, more moderate effects can be observed during recoveries 
as one country official noted. An external evaluator further added that while the medium-
long  term  effects  are  also  significant,  these  are  hard  to  quantify.  The  absence  of  MFA 
operations  would  in  most  cases  lead  to  cuts  in  public  spending  with  severe  knock-on 
effects on the overall economy and force beneficiary countries to borrow at higher rates, 
contributing to a further worsening of the external sustainability of public finances.  

On the structural reform efforts inherent to MFA, senior IMF officials advance the same 
argument when considering cases of backsliding: Though performance on key structural 
reforms  can  be  slow  and  incomplete  in  some  cases,  even  fractional  successes  help  to 
forestall worse outcomes and help to provide a framework for international and domestic 
support.  MFA  conditions  have  in  many  cases  triggered  home-grown  reforms  and 
delivered positive social impacts, e.g. improvements to the rule of law or more equitable 
tax systems, as one IMF official further elaborated. Key aspects to success are the political 
ownership  in  beneficiary  countries,  government  capacity  as  well  as  the  overall  political 
stability. In this regard, senior level COM/EEAS officials have bemoaned the instrument’s 
low  visibility  in  beneficiary  countries  and  pointed  to  the  need  for  more  visible  public 
political  “naming  and  shaming”  on  the  international  stage  to  prevent  backsliding  and  to 
increase local public awareness. 

Results of the consultation 

This  section  discusses  the  results  of  the  completed  consultation  activities  by  each 
evaluation criterion.  

Effectiveness  

67 

 
 
 
The  MFA  instrument  has  proved  effective  in  stabilising  macroeconomic  fundamentals, 
especially  in  the  short  and  medium-term,  according  to  the  majority  of  interviewees. 
Nevertheless,  they  also  highlighted that  effectiveness  was  a result of the burden sharing 
among  international  partners.  On  its  own,  without  linkage  to  IMF  disbursements,  the 
MFA  would  have  hardly  resulted  in  the  same  effectiveness.  The  interview  programme 
suggested that the link to a disbursing IMF financial arrangement should be kept, although 
a few argued that the EU had the necessary capacity and skills to grant MFAs on its own. 
Most  of  the  respondents  to  the  OPC  also  favoured  keeping  the  link  with  the  IMF. 
Participants at the validation workshop underlined that the MFA was particularly effective 
for restoring stabilisation in the short term, rather than in longer term.  

The interview programme also showed that the effectiveness of the MFA is reduced due 
to inflexible conditionalities in the MoU. In contrast to the IMF, conditionalities enshrined 
in  the  MoU  are  fixed  ahead  of  the  programme  and  cannot  be  changed  during  the 
programme.  In  cases  where  the  lack  of  reform  implementation  has  delayed  a 
disbursement,  the  effectiveness  could  have  been  increased  had  conditionalities  been 
adapted.  Another  aspect  highlighted  by  interviewees  is  that  non-compliance  with  the 
political pre-condition has reduced the effectiveness of the MFA. This is mainly because 
the political  pre-condition  is  unique to  the MFA  instrument.  OPC respondents  generally 
believed that  reforms  of the judicial system  and  on anti-corruption were  very important. 
Moreover,  the  OPC  shows  that  the  political  pre-condition  remains  important,  and  that 
reform priorities should also focus on areas linked to the political pre-condition.  
During the validation workshop, a discussion on the added value of cooperation with the 
IMF arose. Participants were generally in favour of cooperation with the IMF.  

Efficiency  

The  consultation  confirmed  the  overall  efficiency  of  the  MFA  instrument,  although 
respondents  emphasised  that  the  degree  of  efficiency  is  difficult  to  assess.  Efficiency 
relates  to  the  costs  and  benefits  of  the  MFA  instrument,  which  are  perceived  to  clearly 
exceed  the  costs,  but  the  nature  of  the  benefits  makes  them  hard  to  quantify:  they  are 
linked to the mitigation of a balance-of-payments crisis that could have very far-reaching 
costs. Moreover, MFA operations promote structural reforms that aim at strengthening the 
economies and their growth potential, which only materialises in the long-term.  
The efficiency of the MFA is nevertheless hampered by the long decision-making process.  

The  costs  of  providing  MFA  are  assessed  to  be  relatively  low.  Neither  the  stakeholder 
consultation  nor  the  desk  research  indicated  any  late  payment  of  loans  from  beneficiary 
countries  to  the  EU.  Loans  are  provisioned  at  a  rate  of  9%  and  hence  have  a  limited 
budgetary impact.  On the other hand,  grants  have a full  impact  on the budget.  In recent 
years,  MFA  has  predominantly  been  granted  as  loans,  and  hence  had  limited  budgetary 
costs.  

The  validation  workshop  confirmed  the  efficiency  of  the  MFA.  Despite  the  lengthy 
decision-making process, participants at the workshop did not see late MFA approvals as 
a key factor affecting the efficiency of the MFA instrument.  

68 

 
 
 
 
 
 
 
 
 
In  some  cases,  interviewees  highlighted  that  the  efficiency  was  relatively  low  in 
beneficiary  countries  outside  of  the  standard  geographical  scope  of  the  MFA.  This  was 
explained  by  the  lack  of  country  knowledge  of  EU  institutions,  which  means  it  is 
necessary  to  rely  on  external  expertise.  This  was  also  confirmed  at  the  validation 
workshop,  where  some  participants  highlighted  that  the  relevance  of  MFA  is  higher  in 
countries  usually  not  eligible  for  MFA,  but  its  efficiency  is  lower  due  to  the  limited 
knowledge  of  structural  features  of  the  beneficiary  country  and  the  lack  of  long-term 
engagement.  

Relevance  

The  consultations  indicated  that  the  MFA  instrument  shows  good  relevance  in  most 
countries.  In some, the MFA operation was less relevant.  In a few countries,  the reform 
programme  did  not  enjoy  high  ownership  from  the  local  government,  reducing  its 
relevance. The lack of ownership was reflected in the slow implementation of the reform 
programme  or  in  the  implementation  of  the  reforms  happening  only  on  paper.  The 
interview programme showed that weak ownership and changes in the political landscape 
have  led  to  policymakers  reversing  some  of  the  reforms  or  adopting  other  reforms  that 
dilute the envisaged reforms.  

The  stakeholder  consultation  showed  that  structural  reforms  enshrined  in  the  MoU  are 
relevant. Most of the reforms has been implemented in line with the MoU. Most reforms 
were implemented in the area of public financial management, anti-corruption and judicial 
systems.  In  a  few  instances  (e.g.  Tunisia),  reform  implementation  proved  slow  and 
ownership of the reform programme remained weak. However, the reform programme in 
Tunisia  remained  relevant.  One  country  (i.e.  Armenia)  in  urgent  need  of  financial 
assistance chose to find alternative assistance. In this case the relevance of the MFA was 
hampered by the lengthy procedures.  
The OPC found that the relevance of the MFA instrument could be increased, for instance, 
by  expanding  its  geographical  scope.  This  view  was  presented  by  a  small  share  of 
respondents  to  the  public  consultation.  This  is  broadly  in  line  with  the  interview 
programme,  where  only  a  few  interviewees  supported  the  expansion  of  the  scope  and 
highlighted a trade-off between efficiency and relevance.  

Coherence  

Within  the  framework  of  this  evaluation,  two  types  of  external  coherence  of  the  MFA 
were  analysed:  (1)  coherence  with  other  EU  instruments  present  in  the  beneficiary 
countries;  and  (2)  coherence  with  programmes  of  IFIs.  The  stakeholder  consultation 
confirmed  that  both  external  and  internal  coherence  of  the  MFA  were  in  general  very 
good.  

With  regard  to  external  coherence  at  EU  level,  stakeholders  confirmed  that,  to  a  large 
extent,  the  objectives,  design  and  implementation  of  the  MFA  are  complementary  and 
consistent with other initiatives providing financial support to beneficiary countries. The 
MFA  is  complementary  to  the  budget  support  programmes  in  the  beneficiary  countries, 
and conditionalities linked to the two instruments are aligned to the extent possible. For 

69 

 
 
 
 
 
 
 
 
 
instance,  COVID-19  Resilience  contracts  were  aligned  with  the  MFA  conditionalities 
whenever relevant and possible.  

With  regards  to  external  coherence  with  programmes  implemented  by  IFIs,  the 
stakeholder consultation shows that it generally works well. Complementarity and lack of 
overlap between conditionalities imposed by different actors is specifically sought at the 
stage of the instrument design. However, the objectives of the presence of IFIs and the EU 
in  beneficiary  countries  are  different;  therefore  full  coherence  among  the  instruments 
would not be possible.  
In what concerns the internal coherence, stakeholder consultation suggests that the MFA 
has been developed in a consistent and clear manner, so that the implementation follows 
the same set of principles and rules in all beneficiary countries.  

EU added value  

All consulted stakeholders recognised an EU added value of the MFA. According to them, 
a  majority  of  the  reforms  implemented  following  the  MFA  conditionalities  would  not 
have taken place otherwise, or not as quickly – e.g. reforms in the area of public financial 
management, anti-corruption and judicial system. However, stakeholders have noted that 
in  most  cases,  the  scope  and  the  overall  impact  of  the  MFA  operations  in  the  countries 
was relatively small in comparison to the total EU presence there.  

Interviewed stakeholders have noted that, in  most  beneficiary  countries, the visibility of 
the MFA was rather low. Lack of visibility affected the added value of the MFA, as the 
instrument  was  under-exploited  for  the  promotion  of  EU  economic  diplomacy  in 
beneficiary countries and their geographical surroundings.  

70 

 
 
 
 
 
 
 
BIBLIOGRAPHY 

Boughton, James M. and Mourmouras, Alex. 2002. Is policy ownership an operational 
concept? International Monetary Fund. [Online] April 2002. 
https://www.imf.org/external/pubs/ft/wp/2002/wp0272.pdf. 

DG ECFIN. 2020. European Commission. Meta-evaluation of Macro-Financial 
Assistance (MFA). [Online] 9 September 2020. tbd. 

ECJ. 2015. Action for annulment — Macro-financial assistance to third countries — 
Decision of the Commission to withdraw a proposal for a framework regulation. 
C-409/13, s.l. : European Court of Justice, 14 April 2015. 

Ecorys, Economisti Associati, Case. 2010. Ex post evaluation of MFA operations in 
Moldova. European Commission - Completed evaluations. [Online] 12 February 2010. 
https://ec.europa.eu/dgs/economy_finance/evaluation/pdf/moldova_eval_en.pdf. 

Ecorys, Wiener Institut für Internationale Wirtschaftsvergleiche, Case. 2012. Ex-post 
Evaluation of Exceptional Financial Assistance Operation to Kosovo. European 
Commission - Completed Evaluations. [Online] 30 Septrember 2012. 
https://ec.europa.eu/dgs/economy_finance/evaluation/pdf/efa_kosovo_eval_final_report_o
ct_2012.pdf. 

European Commission. 2018. A Modern Budget for a Union that Protects, Empowers 
and Defends The Multiannual Financial Framework for 2021-2027. EUR-LEX. [Online] 2 
5 2018. https://eur-lex.europa.eu/legal-
content/EN/TXT/?uri=COM%3A2018%3A321%3AFIN. 

—. 2021. Background Analysis per beneficiary country. EUR-Lex. [Online] 8 July 2021. 
https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A52021SC0178. 

—. 2011. Commission Staff Working Document accompanying the Proposal for a 
Framework Regulation for Macro-Financial Assistance. EUR-LEX. [Online] 4 July 2011. 
https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52011SC0865. 

—. 2011. Proposal for a Regulation of the European Parliament and of the Council laying 
down general provisions for Macro-Financial Assistance to third countries. EUR-LEX. 
[Online] 4 July 2011. https://eur-lex.europa.eu/legal-
content/en/ALL/?uri=CELEX:52011PC0396. 

—. 2021. Report from the Commission to the European Parliament and the Council on 
financial instruments, budgetary guarantees, financial assistance and contingent liabilities 
- Situation at 31 December 2020. EUR-LEX. [Online] 5 November 2021. https://eur-
lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52021DC0676. 

European Parliament. 2012. Report on the proposal for a regulation of the European 
Parliament and of the Council laying down general provisions for Macro-Financial 
Assistance to third countries. European Parliament. [Online] 3 May 2012. 
https://www.europarl.europa.eu/doceo/document/A-7-2012-0157_EN.html. 

71 

 
 
GHK Consulting Limited & European Policy Evaluation Consortium. 2009. 
Completed Evaluations. European Commission - Completed Evaluations. [Online] 2 
October 2009. 
https://ec.europa.eu/dgs/economy_finance/evaluation/pdf/final_report_meta_en.pdf. 

ICF International. 2015. Ex-post Evaluation of the EU’s Macro Financial Assistance to 
Bosnia and Herzegovina (2009-2013) . European Commission - Completed Evaluations. 
[Online] January 2015. 
https://ec.europa.eu/dgs/economy_finance/evaluation/pdf/ex_post_evaluation_of_mfa_bos
nia_summary_en.pdf. 

ICF International, Cambridge Econometrics, Institute for Economic Research and 
Policy Consulting. 2017. Ex-post evaluation of Macro Financial Assistance Operations I 
& II to Ukraine. European Commission - Completed Evaluations. [Online] November 
2017. https://ec.europa.eu/info/sites/default/files/economy-
finance/evaluation_of_mfa_i_and_ii_to_ukraine_-_final_report_ecfin.pdf. 

ICF International, GHK, Cambridge Economics. 2013. Ex-post Evaluation of Macro 
Financial Assistance (MFA) operation to Armenia. European Commission - Completed 
evaluations. [Online] October 2013. 
https://ec.europa.eu/dgs/economy_finance/evaluation/pdf/mfa__armenia_evaluation_final
_report_en.pdf. 

IEO. 2007. Structural Conditionality in IMF-Supported Programs. Independent 
Evaluation Office of the International Monetary Fund. [Online] 2007. https://ieo.imf.org/-
/media/IEO/Files/evaluations/completed/01-03-2008-an-ieo-evaluation-of-structural-
conditionality-in-imf-supported-programs/01032008sc-main-report.ashx. 

IMF. 1944. Articles of Agreement of the International Monetary Fund. International 
Monetary Fund. [Online] 22 July 1944.  

—. 2005. Review of the 2002 Conditionality Guidelines. International Monetary Fund. 
[Online] 3 March 2005. https://www.imf.org/external/np/pp/eng/2005/030305.htm. 

Johnson, Omotunde E.G. 2005. Country Ownership of Reform Programmes. United 
Nations Conference on Trade and Development. [Online] January 2005. 
https://unctad.org/system/files/official-document/gdsmdpbg2420052_en.pdf. 

Klein, Michael and Hartford, Tim. 2005. Grants or Loans? Development Finance and 
Incentive Effects. Worldbank Open Knowledge Repository. [Online] March 2005. 
https://openknowledge.worldbank.org/handle/10986/11230. 

Marc, Francois. 2017. Rapport d'Information fait au nom de la commission des finances 
sur l’assistance macrofinancière de l’Union européenne aux pays tiers. Sénat. [Online] 28 
June 2017. https://www.senat.fr/rap/r16-590/r16-5901.pdf. 

OECD - DAC Working Party on Development Finance Statistics. 2018. Converged 
Statistical Reporting Directives for the Creditor Reporting. Organisation of Economic 
Cooperation and Development. [Online] 28 May 2018. 
https://one.oecd.org/document/DCD/DAC/STAT(2018)9/FINAL/en/pdf. 

Phillips, Steven T, et al. 2013. The External Balance Assessment (EBA) Methodology. 
IMF. [Online] 31 December 2013. 

72 

 
https://www.imf.org/en/Publications/WP/Issues/2016/12/31/The-External-Balance-
Assessment-EBA-Methodology-41200. 

Valdani, Vacari e Associati & Centre for Strategy and Evaluation Services, Oxford 
Analytics. 2021. European Commission. [Online] 8 October 2021.  

Von der Leyen, Ursula. 2019. A Union that strives for more - My agenda for Europe. 
European Commission. [Online] 2019. 
https://ec.europa.eu/info/sites/default/files/political-guidelines-next-
commission_en_0.pdf. 

73