Contents Glossary 1. Introduction Purpose Scope 2. Background to the intervention Description of the intervention and its objectives 3. Implementation / State of Play 3.1. Modality and implementation of the MFA programme 3.2. Economic developments during the lifecycle of MFA (Q4 2011- Q2 2016) 3.3. Economic developments after the implementation of MFA 3.4 Continued support from the EU after the implementation of MFA 4. Method Documentary review Macroeconomic data analysis (including Debt Sustainability Analysis) Interviews with key informants/ stakeholders Focus Group with development partners Qualitative counterfactual analysis Social Impact Analysis (SIA)

Limitations and Methodological Issues 5. Analysis and answers to the evaluation questions 5.1 Relevance 5.2 Effectiveness of MFA 5.3 Efficiency 5.4 EU Value Added 5.5 Coherence 5.6 Social Impact Analysis 5.7 Debt Sustainability Analysis 6. Conclusions Annex 1: Procedural information Annex 2: References Annex 3: Discussion of validity and reliability of the evaluation findings Annex 4: Comparison of MFA, IMF and World Bank conditionalities Annex 5: Evaluation Framework Annex 6: Synopsis Report Annex 7: Summary of findings from stakeholder consultation activities Glossary

ADB Asian Development Bank CPI Consumer price index CU Customs Union DCI Development Cooperation Instrument DFID Department for International Development (UK) DG DEVCO Directorate-General for International Cooperation and Development DG ECFIN Directorate-General for Economic and Financial Affairs DG ECHO Directorate General for European Civil Protection and Humanitarian Aid Operations DIPECHO Disaster-Preparedness ECHO DPO Development Policy Operations DSA Debt Sustainability Analysis EACU Eurasian Customs Union EAEU Eurasian Economic Union EBRD European Bank for Reconstruction and Development ECF Extended Credit Facility EDB Eurasian Development Bank EEAS European External Action Service EEU Eurasian Economic Union EIB European Investment Bank EIDHR European Initiative for Democracy and Human Rights

EIU Economist Intelligence Unit EU European Union EUR Euro EURASEC Eurasian Economic Community FATF Financial Action Task Force FDI Foreign direct investment FMIS Financial Management Information System GDP Gross Domestic Product GIZ Deutsche Gesellschaft für Internationale Zusammenarbeit GSP+ Generalised Scheme of Preferences GTAC Governance Technical Assistance Credit IBC International Business Council IcSP Instrument contributing to Stability and Peace IDA International Development Association IDB Islamic Development Bank IFCA Investment Facility for Central Asia IMF International Monetary Fund INTOSAI International Organization of Supreme Audit Institutions KGS Kyrgyz national currency (Som) KZT Kazakhstani tenge

LGBTI Lesbian, gay, bisexual, transgender, and intersex MDBF Medium Term Budget Framework MDTF Multi-Donor Trust Fund MFA Macro Financial Assistance MTDS Medium Term Development Strategy NBKR National Bank of the Kyrgyz Republic NSI Nuclear Safety Instrument NSSD National Strategy for Sustainable Development OA Operational Assessment OSCE Organization for Security and Cooperation in Europe PCA Partnership and Cooperation Agreement PEFA Public Expenditure and Financial Accountability PFM Public Finance Management PIP Public Investment Programme RCF Rapid Credit Facility RIP Regional Indicative Programme RUR Russian national currency (Ruble) SDPK Social Democratic Party of Kyrgyzstan SDR Special Drawing Rights SECO State Secretariat for Economic Affairs

SIA Social Impact Analysis SIDA Swedish International Development Cooperation Agency SRC Sector Reform Contracts TAF Technical Assistance Facility US United States USAID United States Agency for International Development USD United States Dollars VAT Value Added Tax WB World Bank WTO World Trade Organization

1.Introduction

Purpose

This Staff Working Document presents an assessment of the Macro-Financial Assistance (MFA) operation provided by the European Union (EU) to the Kyrgyz Republic over the period 2015-2016. It largely draws on an independent, ex-post evaluation conducted by an external contractor and extensive consultations with targeted stakeholders in the Kyrgyz Republic as well as Commission specialists on this field. The aim is to assess the results and efficiency of the completed MFA and to draw lessons for future decision-making and identify areas of improvement for similar on-going or future possible interventions. Scope



Macro-financial assistance , or MFA, is an EU financial instrument designed to address exceptional external financing needs of countries that are geographically, economically and politically close to the EU. It takes the form of medium/long-term loans or grants, or a combination of these, and is only available to countries benefiting from a disbursing International Monetary Fund (IMF) programme. The assistance is released in instalments strictly dependent on the successful implementation of reform measures aimed at putting the beneficiary country’s economy back on a long-term sustainable path. In this way, MFA complements regular EU cooperation assistance and contributes to the wider goal of preserving stability and promoting prosperity beyond the EU. In October 2013, the European Parliament and the Council approved a EUR 30 million MFA operation to the Kyrgyz Republic. The operation was disbursed in full, in two instalments, over the period June 2015 and April 2016. In accordance with the Financial Regulation (article 30 point 4), MFA operations in third countries are subject to an ex-post evaluation. In turn, the MFA Decision on Kyrgyzstan stipulates that the European Commission is required to "[...] submit to the European Parliament and to the Council an ex-post evaluation report, assessing the results and efficiency of the completed Union’s macro-financial assistance and the extent to which it has contributed to the aims of the assistance." The ex-post evaluation was undertaken by an external contractor and ran from March 2018 until April 2019. The ex-post evaluation assessed the implementation of the MFA programme to Kyrgyzstan that was initiated in 2011 and disbursed in full in 2015 and 2016. The external report has assisted in the preparation of the Staff Working Document (SWD). Because of the nature of the assistance, the SWD evaluation looks at both the immediate and medium-term impact of the MFA programme on the country's economy and state of play with structural reforms. The objectives of this ex-post evaluation are two-fold: ·to analyse the impact of the MFA on the economy of Kyrgyzstan, and in particular, on the sustainability of its external position; and ·to assess the added value of the European Union’s (EU) intervention. In general, the evaluation seeks to draw lessons with respect to the EU's financial assistance, i.e. owhether the ex-ante considerations, determining the design and terms of the operation were appropriate, taking due account of the economic, political and institutional context; and oand whether the outcome of the programme met the objectives. In turn, these main areas of analysis have been assessed conform the five Better Regulation evaluation criteria: relevance, effectiveness, efficiency, EU added-value, and coherence with other EU policies. In addition, the evaluation assesses the impact of the MFA on the social sector and the public debt sustainability in the Kyrgyz Republic. This is further specified in the Evaluation Roadmap.





2.Background to the intervention



Description of the intervention and its objectives

The Kyrgyz Republic was strongly hit by the global economic crisis in 2008-2009. Output growth declined from a buoyant 7.6% in 2008 to 2.9% and -0.5% in 2009 and 2010 respectively. This was largely due to external shocks, including a fall in remittances from migrant workers, lower demand for exports and shrinking foreign investments . In addition, mounting political instability further plunged Kyrgyzstan into an economic slowdown. In April 2010, the regime of President Bakiyev was overthrown under allegations of corruption and lack of democracy. In a vacuum of power, inter-ethnic violence erupted in June 2010 in the South of the country, leaving over 450 people dead and hundreds of thousands displaced. In spite of these tragic events however, the new, interim government secured an approval vote for democratic reforms in a constitutional referendum held on 27 June 2010. Subsequently, free parliamentary elections were held in October 2010 establishing the first parliamentary democracy in Central Asia. The 2010 revolutions became known as the Second Kyrgyz Revolution, and had adverse effects on economic activity, confidence within the private sector, the government's fiscal position, and other key economic indicators. Figures 1 to 4 overleaf illustrate. On the fiscal side, public debt to GDP ratio grew to 60% in 2010, while the fiscal deficit rose from 3.5% of GDP in 2009 to 5.9% of GDP in 2010, reflecting the budgetary cost of the crisis-related measures and the negative effect of weaker economic activity on tax revenues. On the external side, Kyrgyzstan’s trade balance had worsened due to a disruption in trade flows, primarily driven by border closures (notably with Kazakhstan), and an alarming security situation that markedly affected tourism and partly disrupted agricultural production. Indeed, the current account moved from a temporary surplus in 2009 to a deficit of 2.2% of GDP in 2010. On the monetary side, the central bank of the Kyrgyz Republic had tightened monetary policy substantially since the mid-2010, responding to an acceleration in inflation due to a sharp increase in the global prices of fuel and food. In response to these events and their economic implications, a donor conference “Emergency assistance to the Kyrgyz Republic: Reconciliation and Recovery” was organised in Bishkek in July 2010. At this conference, donors committed to providing USD 1.1 billion in emergency support by the end of 2011. The EU was among the major donors, pledging about EUR 118 million for support in a number of critical areas, through various instruments including: ·EUR 12 million of new assistance: EUR 5 million in humanitarian emergency assistance and EUR 7 million from the stability instrument ·EUR 106 from existing development assistance, including EUR 55 million of EU development assistance ready to be launched in 2010 with a focus on Special safety nets, Education and Agriculture; and EUR 51 million for the 2011-2013 period to advance reform in the areas of Social protection, Education and Rule of Law. Figure 1. Public debt, in % of GDP, 2000 - 2010 Figure 2.General government balance, in % of GDP, 2000 - 2010 Figure 3.Inflation, 2000 - 2010 Figure 4.Current account deficit, in % of GDP, 2000 - 2010 Source: IMF and WB data The IMF extended help to the Kyrgyz Republic in 2010 with a three-month Rapid Credit Facility in the amount of around USD 33.7 million, implemented in September-December 2010. As a follow-up, in June 2011, the IMF agreed with the Kyrgyz Republic authorities on a follow-up arrangement under the Extended Credit Facility for USD 106 million, in support of a comprehensive economic adjustment and reform programme for the period mid-2011 to mid-2014. Other major donors at the conference included the World Bank (pledged USD 200 million), and the Asian Development Bank (ADB) (pledged USD 100 million). Shortly after the donor conference, in October 2010, the President and the Minister of Finance of the Kyrgyz Republic formally requested the EU MFA to complement the financing from the IMF. In this context, the Commission assessed the macroeconomic situation and financing needs of the Kyrgyz Republic and concluded that the 2010 political events and the related social and reconstruction expenditure had left important external and fiscal financing needs for the period 2010-12. Furthermore, given the EU’s strong political support for the Kyrgyz Republic’s incipient parliamentary democracy, the Commission concluded that the political and economic pre-conditions for a MFA operation of a moderate amount are satisfied. The Kyrgyz Republic is out of the normal geographical scope of MFA, yet in exceptional and duly justified circumstances the possibility of approving operations outside that area is possible. In this case, the Commission concluded that this MFA operation was justified by “the strength of the political and economic reform momentum in the country and by its position in a region of strategic economic and political importance for the EU. It would provide a political signal of strong EU support to democratic reforms in Central Asia, consistent with the EU policy towards the region [...]” Against this background, and in order to supplement the resources made available by the IMF, the European Commission, on 20 December 2011, adopted a proposal to provide up to EUR 30 million of MFA to the Kyrgyz Republic, in the form of a grant of EUR 15 million and a medium-term loan of EUR 15 million. This was accompanied by a detailed assessment of the Kyrgyz Republic's needs (See footnote 5). Following the entry into force of the Lisbon Treaty, a procedural disagreement between the Council and European Parliament on the decision-making process of MFA operations delayed the adoption of the decision by the co-legislators to 22 October 2013. The objectives of the EU’s MFA was to: ·Contribute to covering the external financing needs of the Kyrgyz Republic and to alleviating budgetary financing needs; ·Support the fiscal consolidation effort and external stabilisation in the context of the IMF programme; ·Support structural reform efforts aimed at improving the overall macroeconomic management (e.g. targeting the transparency and efficiency of public finance management and the financial stability) and at improving conditions for sustainable growth. The intervention logic for the EU's MFA to Kyrgyzstan is detailed in Figure 5. Figure 5: MFA Intervention Logic

3.Implementation / State of Play

3.1. Modality and implementation of the MFA programme

To achieve the objectives detailed in Section 2, and as per the usual MFA procedure, disbursements under this operation were tied to the fulfilment of country-specific policy conditions, as laid down in the Memorandum of Understanding. The design of the conditionality for this programme was subject to negotiations with the Kyrgyz authorities, and additionally took into account discussions with the EU Delegation, the European External Action Service (EEAS) and other DGs, as well as the IMF and the World Bank. In addition, the Commission (DG ECFIN) commissioned an external Operational Assessment in order to ensure that the public finance management (PFM) system provided sufficient safeguards for the MFA programme. The operation covered seven specific conditions under four reform areas: (i) Public Finance Management (PFM), (ii) Financial sector, (iii) Trade policy and Investment, and (iv) Business environment. Table 1 below lists the exact structural reforms attached to the second disbursement. As for most of the MFA operations, policy measures were not attached to the first instalment, which was conditional only on a satisfactory track record of the IMF programme and the fulfilment of the so-called political pre-condition for MFA - the existence of effective democratic mechanisms in place and to respect the rule of law and human rights. Table 1: Overview of MFA to Kyrgyz Republic – Size, form, terms and attached reform conditions MFA characteristics Main Areas of Reform “ EUR 15 million (loan),

2 disbursement dates:



- EUR 5 million loan in October 2015 * Maturity: 14 years ** Interest rate: 1.375% - EUR 10 million loan in April 2016 * Maturity: 15 years ** Interest rate: 0.70% EUR 15 million (grant), 2 disbursement dates The structural reforms covered the following areas: Area 1: Public Finance Management (1) Medium-Term Budget Framework: approval of a draft resolution on organisational measures for the preparation of the Medium-Term Budget Framework 2015 - 2017 and approval of the draft republican budget of the Kyrgyz Republic for 2015 - 2017. (2) PEFA: reaching agreement on the Terms of Reference of the PEFA. (3) Reform of the accounting and reporting systems: endorsement of the following normative legal acts and methodological documents: (i) Accounting policy of the public administration sector; (ii) Regulation for preparation of financial reporting; and (iii) Regulation for the organization and maintenance of accounting and book-keeping. (4) External auditing: development and approval of an Action Plan for upgrading its audit standards to INTOSAI standards. Area 2: Financial sector (5) Submission of a new Banking Code to the Parliament Area 3: Trade policy (6) Official approval of the report on its trade policy to be presented in the Trade Policy Review Body of the WTO and development of a draft action plan responding to the recommendations of the WTO Secretariat. Area 4: Investment and business environment (7) Enactment of the law "On licensing and permitting system in the Kyrgyz Republic" This operation was characterised by unusual, long delays throughout the process. The Commission adopted the proposal in December 2011, yet the approval by the co-legislators was delayed until October 2013 as mentioned above. Subsequently, the signature of the MFA documents (i.e. the Memorandum of Understanding, the Loan Facility Agreement and the Grant Agreement) only took place in October 2014 due to legal issues involving the National Bank of the Kyrgyz Republic (NBKR). The MFA documents were subsequently ratified by the Kyrgyz Parliament in February 2015, yet by this time, the 2011 IMF programme had expired (in July 2014). A new three-year Extended Credit Facility (ECF) was agreed with the IMF in April 2015 and from then onwards, disbursements followed more swiftly. The operation was disbursed in full, in two instalments, over the period June 2015 - April 2016: The first instalment: a EUR 10 million grant disbursed in June 2015 and a EUR 5 million loan disbursed in October 2015; The second instalment: a EUR 5 million grant disbursed in February 2016 and a EUR 10 million loan disbursed in April 2016. Both loan tranches were characterised by very long maturity (14 years and 15 years, respectively) and low interest rate (1.375% and 0.70%, respectively). See Table 1. As MFA is an emergency instrument without legal base, its legislative process consists of an ad-hoc agreement on every single MFA operation between the European Parliament and the Council under the ordinary legislative procedure (TFEU Art. 212), which on average requires eight months, after which the MFA documents also necessitate sometimes-lengthy approval procedures in the beneficiary country. This operation, however, was characterised by an exceptionally long timetable between approval and disbursement, largely for reasons beyond the Commission’s control. The lifecycle of this MFA programme thus spanned an unusually long period from late 2011 to early 2016. Figure 6 below summarizes the timeline for this MFA operation, from the events that motivated the Commission proposal (See Section 2) to the last disbursements under MFA and IMF. Figure 6: Timeline for approval and disbursement of IMF and MFA support



Points of comparison The points of comparison, against which the MFA operation of 2011-2016 is assessed, refer to the situation in the Kyrgyz Republic (i) prior to the intervention on the one hand (Section 2), and (ii) during and after the implementation of the MFA operation on the other hand (Sections 3.2 – 3.3). The role of MFA in promoting macroeconomic stability, easing external financing constraints and alleviating Kyrgyzstan’s balance of payments and budgetary needs, as well as the effectiveness of the structural reforms attached to this MFA programme are analysed in Section 5.

3.2. Economic developments during the lifecycle of MFA (Q4 2011- Q2 2016)



Following the lingering effects of the global economic crisis of 2008-2009 and inter-ethnic violence of 2010, the Kyrgyz Republic GDP dropped by 0.5% in 2010. The next year was characterized by a rebound of 6.0% in economic activity, owing to a full resumption of trade and services flows, continued reconstruction works, and improved investor confidence. The external evaluation asserts that this renewed confidence in the Kyrgyz economy could have been due to, though not the sole result of, the IMF’s announcement of a three-year ECF arrangement (over the period June 2011 - July 2014) and the European Commission’s proposed MFA to Kyrgyzstan, signalling important structural reform actions in key areas. Nevertheless, 2011 was still characterized by a high fiscal deficit and a very high external debt (around half of its GDP), making the economy highly vulnerable to external shocks. While the peaceful transfer of power in 2011 and the swift formation of a new government in 2012 laid the foundation for growth, low gold production nevertheless caused GDP to contract in 2012. The external current account deficit widened substantially, reflecting lower export volumes and a slight fall in gold prices. The trade deficit is estimated to have increased significantly, as higher purchases of gasoline and diesel fuel caused imports to rise, while gold exports fell. Nonetheless, strong and positive growth was observed in other sectors of the economy, such that the non-gold GDP growth rate was estimated at a buoyant 6.3% for 2012. In 2013, economic growth soared to almost 11.0%, and the current account deficit narrowed further. This was driven by a rebound in gold production, lower prices for imported food and fuel, and a sustained improvement in the rest of the Kyrgyz economy. This strong growth was however short lived. In 2014, GDP growth slowed to 4.0%, reflecting a sharp fall in gold production and other major supply-side constraints. At the same time, the external environment (notably the neighbouring economies of the Russian Federation and Kazakhstan) weakened, while strong local currency depreciation pushed inflation up. At the end of 2014, external public debt surged to an estimated 51.0% of GDP from 43.7% a year earlier, whilst the current account deficit is estimated to have risen to around 15% of GDP from 14.1% in 2013. In 2015, GDP growth slowed to 3.9%. Continued recession in the Russian Federation and a slowdown in neighbouring Kazakhstan weighed heavily on the economy. External public debt soared to 63.0% of GDP from 51.0% a year earlier, mainly reﬂecting local currency depreciation. Kyrgyzstan’s vulnerable balance of payments situation was reflected in the structurally large and sustained current account deficit, which widened to 17% in 2015. The Kyrgyz economy further contracted in the first half of 2016, as a sharp decline in gold output (-35.6%) coincided with weak performance in the non-gold sector, in particular industry and service. The Commission disbursed the first instalment of the MFA programme to the Kyrgyz Republic (comprising of EUR 10 million in grants and EUR 5 million in loans) in June and October 2015, respectively. The second and final instalment (comprising EUR 5 million in grants and EUR 10 million in loans) was disbursed in February and April 2016, respectively. The disbursements coincided with a time of sluggish economic growth in Kyrgyzstan as the Russian economic crisis began unfolding (See Figure 7). Figure 7 below illustrates an overview of Kyrgyzstan’s economic situation, measured by GDP growth rate over the lifecycle of this MFA, including also specific events that help clarify the developments observed over these periods. Figure 7: Trends in quarterly GDP growth rates observed for Kyrgyzstan, 2011(Q4)-2016(Q2) Source: ICF External evaluation report Furthermore, Figures 8 to 13 illustrate the economic developments over 2011-2017, as described above. Figure 8. Decomposition of real GDP trend, 2011 - 2017, in % of GDP Source: NSC

Figure 9. Current account balance and its main components, in USD billion Source: NBKR Figure 10. General government revenue & expenditure (left axis), and deficit (rights axis), in % of GDP Source: IMF Figure 11. Public debt, in % of GDP Source: MoF Figure 12. Deposits and loans by banks and micro-financial organizations, in % Source: NBKR

Figure 13. Inflation rate (left axis) and official exchange rate (right axis) Sources: NSC, NBKR 3.3. Economic developments after the implementation of MFA



Economic growth in Kyrgyzstan over 2016 and 2017 was close to the long-term average (2000 - 2015) GDP growth value of about 4.5%. This was supported by some recovery in the neighbouring economies (especially Russia and Kazakhstan) and increase in the inflow of remittances. The debt and fiscal situations have somewhat improved, and inflation remains below 5% per annum. In 2018, the economy continued showing signs of recovery, benefiting from stable regional and external environment. GDP growth was estimated at 3.5 percent, whilst inflation averaged 1.5 percent for the year. Fiscal deficit stands moderately at 1.3 percent of GDP, gross official foreign exchange reserves at 4.0 months of imports and a stable banking sector point to the success of stabilization policies, carried out by the government and National Bank of the Kyrgyz Republic (NBKR). The latest assessment of the IMF (IMF, 2019) positively notes that, ‘’[…] While significant risks remain, debt and financial sector vulnerabilities have subsided. The authorities are committed to prudent macroeconomic policies and implementation of structural reforms, and see them as essential to promoting higher and more inclusive growth and to increase economic resiliency. Monetary policy remains appropriately focused on maintaining price stability.’’ Looking ahead, growth in Kyrgyzstan is expected to recover to around 4.0% in 2019, as per the latest forecasts of the IMF (World Economic Outlook Database, April 2019), the World Bank (Global Economic Prospects, June 2019) and the Asian Development Bank (Asian Development Bank Outlook, 2019). The medium-term growth prospects are shaped however by downside risks (i.e. slowdown in the region, especially in Kazakhstan and the Russian Federation, the country’s two main regional partners; lower remittance inflow; difficulties attracting foreign direct investments etc.). While debt sustainability has improved, the IMF assesses the country as facing moderate risk of debt distress because of continuing vulnerability involving currency stability and possible deterioration in the fiscal balance. For 2020-2024, the IMF predicts less buoyant activity, with GDP growth averaging around 3.0%. The latter value may be related to the gradual closure of Kumtor gold mine (amongst the largest contributors to the GDP of Kyrgyzstan), expected to start in 2022-2023. 3.4 Continued support from the EU after the implementation of MFA

Kyrgyzstan will continue to be an important partner for the EU. The EU operates in the Kyrgyz Republic on the basis of the Partnership and Cooperation Agreement which has been in force since 1999. In July 2019, the EU and Kyrgyzstan initialled a new Enhanced Partnership and Cooperation Agreement (EPCA). This is the second EPCA to be concluded with a Central Asian country (after Kazakhstan), creating the conditions for strengthened trade relations in a region of strategic interest for the EU, as outlined by the recent Joint Communication on the EU and Central Asia . Moreover, Kyrgyzstan is one of the countries where the EU implements its new EU strategy on Central Asia, adopted in June 2019. Meanwhile, for the timeframe 2014 - 2020, the EU will have allocated EUR 174 million of bilateral aid to the Kyrgyz Republic in grants. The focal sectors were rule of law, education and integrated rural development/social protection and public finance management. The most important implementation modality is budget support (with operations on education, social protection, and electoral reform). Kyrgyzstan is also the main beneficiary of the Investment Facility for Central Asia (IFCA). During the same period, it received a total of EUR 73.4 million in grant contributions, which catalysed investments of over EUR 300 million. 4.Method

This ex-post evaluation has been undertaken by an external contractor and was carried out from March 2018 to April 2019. The external evaluation, whose progress and quality was monitored by an Inter-service Steering Group (ISG), included the following steps: an inception report (which explained how the evaluation design would deliver the information required), field visits to Kyrgyz Republic, Belgium and the United States for discussions with key, targeted stakeholders, an interim report and a final report (providing responses to evaluation questions). The Evaluation was carried out in line with the principles commonly applied for the evaluation of EU initiatives, as laid down in the Better Regulation Guidelines. This included the publication of an evaluation roadmap between 22 December 2017 – 19 January 2018 and the incorporation of feedback, where relevant, in the external evaluation report. The methodology employed was comprehensive and responded to the very specific and unique nature of the MFA operation. For the evaluation conducted by the external contractors, the evidence and the data were collected through several complementary approaches, including (i) a theory based approach (See Section 2 from external evaluation report), (ii) participatory approaches to data collection and analysis; (iii) quantitative and qualitative research methods; and (iv) triangulation. Overall, the quality of the collected evidence by the external contractor (data, documentation, interviews and survey results) for this evaluation can be assessed as very good, within the limitations mentioned below.

The qualitative research was grounded in logic and economic theory, whilst the quantitative fieldwork was based on reliable statistical data, and purposeful sampling was used for the interviews and the focus group discussion. To collect a broad, multi-dimensional and triangulated picture of the economic, financial and structural issues surrounding the programme, a wide range of relevant stakeholders and civil society organisations was also involved. Based on this and the triangulation of evidence, this evaluation can be considered reliable and valid. The data collection methods and analytical techniques used to address the evaluation criteria’s are explained below. Documentary review The evaluation is based on an in-depth review of documentation assembled from a variety of sources, including the European Commission, IMF, WB, Kyrgyz authorities and others. A full list of references is available in Annex 2. Macroeconomic data analysis (including Debt Sustainability Analysis) The evaluation examined how the key macroeconomic indicators have evolved over time and specifically, the direction and magnitude of the changes observed over the period of interest (before, during and after the MFA implementation) as well as any deviations from the initial projections made by the IMF and the underlying reasons for these. Similarly, the Debt Sustainability Analysis (DSA) involved an analysis of key debt-burden indicators and macroeconomic variables, which influence the path of a country’s debt and its capacity to manage this burden sustainably in the medium term to long term. This was carried out using the DSA toolkit developed by the IMF and the WB that is the benchmark tool for analysing debt sustainability of countries. Interviews with key informants/ stakeholders Interviews with key stakeholders constituted an essential part of the fact-finding work. A total of 41 interviews were held in several rounds over the course of 2018. They covered a wide variety of issues related to the design and implementation of the MFA programmes, their impact, the overall financial support package to Kyrgyz, progress with macroeconomic stabilisation and implementation of structural reforms, and others. Focus Group with development partners Towards the end of the evaluation, a one-day focus group discussion was organised with locally-based development actors to complement as well as cross-check information collected from other sources such as desk research and key informant interviews. Qualitative counterfactual analysis The external contractor adopted a qualitative approach in this evaluation to counterfactual analysis. Specifically, the external evaluators applied counterfactual reasoning within a qualitative framework (using a theory-based approach) to draw inferences regarding the role and contribution of the MFA in promoting macroeconomic stabilisation and cushioning the social impact of the crisis. Evidence and insights collected from desk research, interviews, focus group and expert opinions were used to deduce what might have happened in the absence of the MFA (and IMF). The external report also paid particular attention to ensuring that the counterfactual scenario(s) and any inferences drawn were grounded in logic and economic theory. This was an appropriate approach given the difficulty of constructing reliable counterfactual scenarios using econometric techniques or macro-economic models in a crisis context. Social Impact Analysis (SIA) The evaluation examined the role and contribution of the MFA operation in cushioning the social impacts of the crisis. This was done by analysing trends in a number of social indicators prior to, during and after the MFA operation, and then applying counterfactual analysis to determine outcomes. The detailed approach to the SIA is presented in Annex 13 of the external evaluation report. Limitations and Methodological Issues

While the overall reliability and validity of the evaluation is strong, a number of methodological limitations and challenges effected the evaluation: ·The access and the quality of the macroeconomic data in the Kyrgyz context has been satisfactory, yet there were exceptions whereby the contractors could not trace particular events. The external contractor specifies that “[…] the data related to the financial assistance provided by Russia is somehow obscure and the study required an assumption regarding the specific grant disbursement in the late 2015 that was announced by the Kyrgyz officials, but without any evident traces in the data published by the Ministry of Finance.” ·When organising and conducting interviews, the contractors were faced with ‘memory loss’ as in certain cases stakeholders were unable to recall in detail aspects related to the programmes or/and the relevant context due to the time that had elapsed. Moreover, there was a consistent and relatively pronounced reluctance among interviewees to express their views on matters that could have been seen by them as potentially sensitive. Additionally, with respect to the one-day focus group discussion organised with locally-based development actors, most participants had very limited awareness and knowledge of the MFA.

·Evaluating the outcomes of MFA programmes is particularly challenging, since financial support normally has a lagged impact on the real economy. Furthermore, it was difficult to isolate the impact of MFA from other interventions (IMF, WB, other EU interventions, support from other donors etc.) and other exogenous factors, particularly during the abnormally longer timeframe of the operation. ·Similarly, with respect to the Debt Sustainability Analysis, the un-earmarked nature of MFA implies that it was not possible to trace down how MFA funding was exactly used (i.e. maintain public expenditure; meet the foreign debt repayment obligations etc.). The ISG as well as other staff from DG DEVCO, EEAS and the EU delegation in Kyrgyzstan actively participated in dealing with these challenges, including by providing missing data to the evaluators and multiple rounds of feedback. In turn, the identified risks and limitations do not put into question the overall reliability of the evaluation analysis as they were mitigated by the fact that information was obtained from a wide range of sources, using different evaluation techniques, alternative scenarios and multiple rounds of feedback. Therefore, the conclusions reached in the evaluation can be considered as valid. 5.Analysis and answers to the evaluation questions

5.1 Relevance

Question 1: To what extent was the MFA operation design (including adequateness of financing envelope, focus of conditionality) appropriate in relation to the outputs to be produced and objectives to be achieved? By 2010, following the lingering effects of the global economic crisis of 2008-2009 and inter-ethnic violence of 2010, Kyrgyzstan was facing pressing external and fiscal financing needs. The size of Kyrgyzstan’s financing gap was estimated at USD 271 million and USD 149 million for 2011 and 2012 respectively. However, as a result of procedural delays (see Section 3), the MFA disbursements took place in 2015 (EUR 15 million) and 2016 (EUR 15 million), where the size of the country’s financing gap was slightly improved. As indicated in Table 2 below, actual MFA disbursements corresponded to around 10% and 22 % of the financing gaps in 2015 and 2016. The amounts accounted for 14% and 12% of the total assistance package (budget support financing exclusively) provided by the multilateral and bilateral donors in 2015 and 2016, and corresponded to around 0.3% of the GDP each year. In this context, the external evaluation report confirm that the size of the MFA operation in Kyrgyzstan was of a satisfactory amount, particularly since it was only intended to supplement the assistance offered by the IMF and other international and bilateral donors. Table 2: Kyrgyzstan’s financial requirements (2011-2016) Financing requirements 2011 2012 2015 2016 Estimated financing gap 271.0 149.0 171.5 76.8 Donor Financing IMF ECF disbursement 29.8 29.7 28.9 28.9 WB 30.0 n/a 40.9 34.5 EURASEC anti-crisis fund 106.7 n/a n/a n/a ADB 50.6 n/a n/a n/a Other grants n/a n/a 70.0 0.0 EU (Sectoral Budget Support Programmes) 16.4 17.1 n/a n/a EU (MFA) – in USD million 20.5 20.5 16.498* 16.776* Originally planned Based on actual disbursements EU MFA – as a share of the financing gap 8% 14% 10% 22% Source: ICF Evaluation & Internal European Commission calculations With respect to timing, some stakeholders argued that the MFA was somehow less vital during the disbursement period. Nevertheless, the external evaluation confirms that despite the delays in disbursement, the MFA was still relevant in 2015-2016 as they coincided with important debt repayment obligations and with the time Kyrgyzstan faced difficulties as the Russian economic crisis began unfolding. The MFA comprised an equal mix of grants and loans, reflecting the compromise reached between the Council of the European Union (who strongly opposed grant-only financing to Kyrgyzstan) and the European Parliament. The interviewed stakeholders and the external contractors judged this appropriate, particularly in the context of the overall package of EU support (which comprised a significant grant component). Furthermore, Kyrgyz government officials have regarded the un-earmarked nature of MFA disbursements and the highly concessional terms of the MFA loans as important and attractive characteristics of this MFA operation. The MFA conditionality reform package aimed to address structural weaknesses affecting Kyrgyzstan. In turn, the seven structural conditions attached to the MFA covered four reform areas: (1) public finance management (PFM); (2) banking; (3) trade; and (4) business and investment. The specific conditions that were part of the MFA programme to Kyrgyzstan are detailed in Annex 10 of the external evaluation report. The evaluation confirmed that all seven MFA conditions included in the MoU were considered relevant back in 2013 at the time of the MoU negotiations. The Kyrgyz authorities were effective in the implementation of the conditionality, meeting six out of seven conditions prior to the signature of the MoU in October 2014. In turn, a number of interviewed stakeholders from the EU delegation questioned (1) the appropriateness of the conditions by the time the MFA was formally approved and disbursed, and (2) the extent to which the EU could have re-opened the negotiations and revised the specific MFA conditions. Nevertheless, swift implementation of MFA reforms is not uncommon, as authorities speed up the reform process to ensure that MFA will be disbursed, and convey the right commitment signals to the donor community. In addition, any re-opening of discussions related to this programme could have resulted in further delays with the disbursements, thus seriously undermining the relevance of the entire operation and triggered further tensions between the EU and the Kyrgyz government. This argumentation is shared by the external contractor in their detailed assessment of the relevance of this MFA in Section 7 and Section 8.2.2 of their final report. Finally, MFA reforms were generally in line with the priorities set by the Kyrgyz government as well as other donors / IFIs (as part of their support programmes). These were also consistent with wider objectives associated with EU MFA operations, notably to promote structural measures/reforms capable of driving and maintaining macroeconomic stability. The evaluation concludes that overall, at a thematic level, all areas of conditionality were relevant and well chosen. 5.2 Effectiveness of MFA





Question 2: To what extent have the objectives of the MFA operation been achieved? The objectives of MFA to Kyrgyzstan are, as set out, inter alia, in the Memoranda of Understanding (MoU) to “support the restoration of a sustainable external financial situation for the Kyrgyz Republic, to alleviate its balance of payments and budgetary needs and thereby support its economic and social development.” Beyond this, the objectives are also to support structural reforms. The external evaluators relied on two strands of analysis to answering this question: Part 1: The role of MFA in promoting macroeconomic stability, easing external financing constraints and alleviating Kyrgyzstan’s balance of payments and budgetary needs The exact impact of the EU financial support on economic outcomes is inherently difficult to quantify, in particular due to external factors and the fact that MFA was only part of the international support package. Nevertheless, the evaluation employs a set of quantitative and qualitative analysis tools to get to plausible alternative scenarios. With the disbursements corresponding to around 0.3% of GDP in both 2015 and 2016, the EU MFA was not negligible and contributed to the stabilisation of the Kyrgyz economy. At the same time, the absence of the MFA (‘Scenario 1’) would not have caused significant, adverse economic impacts. The impact of this MFA on the sustainability of the Kyrgyz debt was considered positive, yet not too significant. Results from the Debt Sustainability Analysis (DSA) (Section 5.7) show that having the MFA operation will lower the present value of the debt-to-GDP ratio by 1.6 percentage points on average over 2015 - 2025, compared to not having the MFA. On the other hand, results from the Social Impact Analysis (SIA) (Section 5.6) show that the impact of this MFA on the social situation was considered less prominent, mostly because none of the MoU conditionality specifically refers to social policies and/or outcomes. Yet, evidence gathered from interviews with Kyrgyz’s civil society points to the MFA operation providing the EU with some leverage in its discussions on the controversial ‘anti-LGBTI ‘and ‘foreign agents’ bills with the Kyrgyz authorities. The Kyrgyz government ultimately did not pursued these laws for adoption in the parliament. Furthermore, the evaluation did not find evidence suggesting that the sole absence of the MFA financing could have had any material impact on business and investor confidence. This could be explained in part by the low awareness/visibility surrounding this MFA operation (due to the experienced delays and the smaller magnitude of the MFA compared to assistance from other donors). In this context, a presumed negative impact on foreign exchange rates, inflation and households’ disposable incomes, in the absence of the EU MFA, is also unlikely. With respect to financing, the Kyrgyz government could potentially have resorted to other options had MFA not been implemented, thus containing the expected negative impact on aggregate demand. Figure 14 overleaf presents these options together with the likelihood of each happening. Overall, there was a high level of consensus among all stakeholders regarding these hypothetical scenarios, with the most likely being that the government would have withdrawn some of its reserve deposits at the central bank, undertaken cuts in public expenditure, and/or sought increased assistance from Russia. Figure 14. Alternative scenarios for obtaining finance had MFA not been available (but with IMF continuing) Source: ICF External Evaluation Report With respect to structural reforms, the MFA programme was effective in bringing about additional stimulus for reform in priority areas, and encouraging the implementation of specific, short-term, measures that may have otherwise been overlooked by national authorities, primarily owing to a lack of financing. At the same time, the reforms were not found to have had a significant impact on economic outcomes, predominantly due to the limited scope and since some were intended to pave the way for more extensive reforms (driven by the EU and/or other international donors) over the medium to longer term. The hypothetical absence of both the EU’s MFA and IMF’s ECF (‘Scenario 2’) would have had more severe implications. This is not surprising, given that by its nature, MFA is meant to compliment IMF programmes, and that, it usually comes in smaller magnitudes compared to the IMF loans.

The evaluation finds that the absence of the EU’s MFA and the IMF’s ECF would have most certainly affected the financing provided by other donors, particularly the World Bank who requires a stable macroeconomic framework to deploy its budget support operations (e.g. Development Policy Operation(s)). In nominal terms, the absence of the assistance provided by the EU, the IMF and the WB in 2015 (i.e. USD 29 million from the IMF, USD 16.5 million from the WB and USD 16.5 million from the EU via the MFA), would have corresponded to a financing gap of USD 62 million, or ~1 % of GDP in 2015 only. In turn, this would have resulted in consequences that are more significant. According to the IMF, the depth of the spending cuts would have been greater, most likely affecting social spending. A potentially deeper loss of business and investor confidence would have triggered inter alia, an even greater depreciation of the national currency and a further hampering of FDI. In addition, a ‘no MFA and IMF’ alternative would have also resulted in foregone benefits from the IMF reform package. Part 2: Effectiveness of structural reforms The Kyrgyz authorities fulfilled all the economic and structural policy conditions related to the disbursement of the MFA, without the need for waivers. Implementation was swift, as six out of seven conditions were fulfilled prior to the approval and signature of the MoU in October 2014. Some of the interviewed stakeholders questioned the appropriateness and effectiveness of the MFA conditions and whether the Commission should have re-negotiated more meaningful reforms with the Kyrgyz. The evaluation finds fully justified that the Commission did not re-open negotiations to avoid further delays that would have only amplified tensions that prevailed at the time between the Kyrgyz government and the EU. Furthermore, interviewed stakeholders from the Commission explained why the effectiveness of the reforms attached to this MFA need to be assessed in the light of the exceptional and highly political nature of this operation. Indeed this MFA was considered an additional political gesture of the EU in endorsing the country’s efforts towards becoming a fully-fledged parliamentary democracy (rather than purely an anti-crisis support mechanism aiming to stabilise the economy). The MFA reform package conferred added leverage to the EU in its discussions of the anti-LGBTI and foreign agents bills with the Kyrgyz authorities that allowed for the withdrawal of these laws back in 2015/2016. 5.3 Efficiency



Question 3: Was the disbursement of the financial assistance appropriate in the context of the prevailing economic and financial conditions in the beneficiary country? Question 4: In what way has the design of the MFA assistance conditioned the performance of the operation in respect to its cost and its objectives? The timing of the disbursements of this MFA operation (over June 2015 and April 2016) were still relevant and appropriate given the economic difficulties that prevailed in Kyrgyzstan at the time (See Section 2). Throughout the process, the frequent dialogues between the EU (via DG ECFIN and the EU Delegation) and Kyrgyz authorities were critical in supporting the understanding and commitment to MFA conditionality and management of disbursements. In general, stakeholders consulted during the evaluation acknowledged a smooth running of the negotiation process, with respondents from the EU delegation emphasising the importance of technical assistance support accompanying the implementation of certain reforms (for both this MFA operation and other EU’s budget support programmes). The Commission also closely coordinated its activity with other international creditors, in particular the IMF and the WB, thereby ensuring that actions/reforms under the EU MFA complemented those promoted by other donors on the ground. The main synergies were achieved in the public financial management, banking and trade policy areas. On a national level, synergy was also existent between the MFA conditionality and national policies. Interviewed stakeholders acknowledge that in some instances MFA opened a window of opportunity to some government departments in promoting/accelerating their own reform plans. Overall, the Kyrgyz’s reform ownership was appropriate and there was no need for waivers. On the EU’s side, the monitoring process was frequent and adequate and the Commission complied with all checks ensuring that Kyrgyzstan had satisfactory fulfilled the reform measures supported by the MFA. In terms of visibility, evidence gathered from local stakeholders and various development partners indicates poor visibility of the EU MFA in Kyrgyzstan. This is partly explained by the delays experienced during the negotiations and disbursements, which, equally, did not receive much attention at national level. In this context, with the aim to improve the visibility and public understanding of MFA programmes more generally, the Commission started in mid-2014 to publish on the web site of DG ECFIN the Memorandum of Understanding that lays down the reform measure related to the operations. 5.4 EU Value Added

Question 5: What was the rationale for an intervention at EU level and to what extent did the MFA operation add value compared to other interventions by other international donors? Did the operation actually lead to the expected impacts and added value of international cooperation, and what can be learnt for future operations? The response to the economic and socio-political crisis hitting Kyrgyzstan around 2010 (See Section 2) required active co-operation of all international partners, both in terms of financial commitments and agreement on the type of conditionalities proposed. The EU was a constructive party to these discussions, and overall, the added value from the EU’s MFA to Kyrgyzstan was judged positive by the external evaluators. The size of the MFA operation (EUR 30 million or 0.3% of GDP in 2015 and 2016), though relatively smaller when compared to the IMF programme, made a significant contribution in terms of financial value added. An important attribute of the EU MFA versus alternative sources of financing was its highly concessional terms, i.e. relatively low interest rates, long maturity and long grace period. As pointed out in the evaluation, this generated fiscal space for the Kyrgyz government at a time when the country was faced with various economic difficulties as well as preparing for parliamentary elections. The external evaluation finds that the EU added value was most apparent in strengthening the promotion of structural reforms in several priority areas. The MFA conditions generally supplemented reforms driven by existing EU support programmes and by key international donors, such as the IMF and the WB. This reinforced the rationale for adjustment measures and provided local authorities with an impetus to reform swiftly in targeted sectors. More broadly, however, interviewed stakeholders from the IMF, WB, DG ECFIN and the EU Delegation reported low and varying level of reform ownership among Kyrgyz authorities. In turn, various projects led by the IMF, such as the Financial Management Information System (FMIS) and the Banking Code, were not materialising. Beyond economic reforms, the EU MFA served wider political objectives in the Kyrgyz context, enhancing bilateral political relations. Moreover, the actual disbursements (coincidentally) came at a politically important moment ahead of parliamentary elections and thus served as a signal of EU support to democratic reforms in Central Asia, consistent with the EU policy towards the region. Furthermore, though the EU MFA did not specifically target social reform, interviews conducted with EU and Kyrgyz stakeholders suggest that the MFA played an important role in driving the Kyrgyz government to prevent the enactment of two controversial laws back in 2015 and 2016, notably the ‘anti-LGBTI’ and ‘foreign agents’ laws.

Overall, however, the EU’s value added was hampered by the delays experienced during the implementation of this MFA operation. Interviewed stakeholders felt that these delays had adversely impacted the extent of awareness and visibility surrounding the operation. Moreover, this was a completely new operation for Kyrgyzstan and the conditions associated with it were viewed as “too technical” and difficult to convey to “laymen.” 5.5 Coherence

Question 6: Were the measures of the MFA operation in line with key principles, objectives and measures taken in other EU external actions towards Kyrgyz Republic? The European Union and the Kyrgyz Republic have been partners since the country's independence in 1991, increasing their cooperation and areas of interaction year-on-year. The EU’s support is channelled through various instruments, including the Development Cooperation Instrument, the Regional Indicative Programme (RIP) for Central Asia, the Investment Facility for Central Asia (IFCA), and various other thematic programmes, humanitarian and budget support operations, including MFA. As a sign of renewed commitment, in May 2019, the Commission and the High Representative of the Union for Foreign Affairs and Security Policy adopted a new Joint Communication that sets out a fresh vision for a stronger partnership with the five countries of Central Asia, including the Kyrgyz Republic. The Commission and the High Representative/Vice-President presented the new EU Strategy on Central Asia at the 15th EU-Central Asia Ministerial meeting, which took place on 7 July in Bishkek, Kyrgyz Republic. More specifically on MFA, over the period 2012 - 2016, between 30 and 50% of EU assistance to the Kyrgyz Republic was delivered in the form of budget support or MFA grants. Figure 15 below illustrates. Figure 15 - EU external aid to Kyrgyzstan, by type, over 2010 - 2016 (in EUR million) Source: Annual Report on the implementation of the European Union’s instruments for financing external actions and DEVCO disbursement data. Please note that there is little information available on the types of EU aid provided to Kyrgyzstan prior to 2012. In turn, evidence gathered by the external contractor suggests that the MFA was coherent with key principles, objectives and measures taken as part of other EU external actions towards Kyrgyzstan. Furthermore, MFA conditionalities were coherent with those in other EU and international donor support programmes, notably in the PFM and banking areas. Annex 4 provides a comparison of reform areas driven by the EU through MFA, and the IMF and the WB through their respective support programmes in Kyrgyzstan. Overall therefore, the MFA operation was both, internally as well as externally coherent. 5.6 Social Impact Analysis



MFA is a balance of payments support instrument aiming primarily at covering external financing needs and at restoring the sustainability of the external position. It is not directly targeting specific social outcomes like other EU programmes do, such that the EU’s MFA to Kyrgyzstan had no explicit conditions on social outcomes. This presents a challenge when assessing the exact social impact of the operation. In light of this challenge, the evaluation employs a comprehensive counterfactual analysis to compare what happened to the Kyrgyz Republic’s social situation during and after the MFA operation against two alternative scenarios: of no MFA (Alternative 1) and that of no MFA nor support from the IMF (Alternative 2). Evidence from the counterfactual analysis suggests that the social consequences stemming from the absence of EU’s MFA (Alternative 1) would have been relatively limited, both in absolute terms and relatively when compared to those under Alternative 2. The shortfall of the EUR 30 million would have likely put more pressure on government procurement, yet significant impacts on the labour market, the cost of living and provision of key public services (e.g. health, education, pensions) are deemed unlikely. However, the hypothetical absence of both the EU’s MFA and IMF’s ECF (Alternative 2) suggest that the social consequences would have been more severe. As explained in previous sections, this is not surprising since by its nature, MFA is meant to compliment IMF programmes, and that, it usually comes in smaller magnitudes compared to IMF loans. More broadly, the MFA supported social development through its assistance in preserving macro-economic stability in Kyrgyzstan. Indeed, a more aggravated balance of payments crisis could lead to the depletion of foreign reserves, an increase in uncertainty, and a depreciation in local currency, which would in turn have negative implications on the country’s standard of living, growth potential, unemployment and social benefits. 5.7 Debt Sustainability Analysis The EU MFA’s contribution to Kyrgyzstan’s public debt sustainability was clearly positive, particularly when combined with the IMF assistance, which it complemented. This was confirmed by the results of the evaluation report which used the IMF's Debt Sustainability Analysis (DSA) framework to quantify the effect of the IMF and MFA on public debt sustainability for 2015-2017 (medium term) and beyond (longer term). To this end, the evaluation analyses the possible paths of debt-burden indicators under different scenarios, comparing what happened (the baseline) with scenarios of no MFA financing (Alternative 1), and no MFA and IMF financing (Alternative 2). The direct impact on Kyrgyzstan’s debt dynamics comes from the concessional component of this MFA, which entails a grant element and loans with favorable terms. Specifically, the two loan tranches of this MFA bared long maturity (14 and 15 years) and very low interest rates (1.375% and 0.750%), this leading to improvement of both the maturity and interest profile of the public debt in Kyrgyzstan. The favorable terms also suggest that the Kyrgyz Republic may have benefited from savings on the cost of borrowing, and so limited the accumulation of debt to some extent. More broadly, the impact of EU’s MFA and the IMF assistance could be seen through their positive impact in supporting economic recovery, which is key to bringing debt metrics to a sustainable path. Indeed, over 2015 to 2017, a period when MFA funds and most of the IMF’s second tranche were disbursed, Kyrgyzstan’s economic performance was relatively satisfactory. Real GDP grew at just over 4% annually on average, inflation stood at 3.2% per annum, while public debt-to-GDP ratio averaged 59.7%.

This performance is, at least in part, due to financial assistance received from the EU and IMF during this period. The interventions reduced the need for further fiscal adjustments by the authorities (tax hikes, cuts in wages and social payments), and funded an increase in (or maintaining of) government spending levels in a way that may have contributed towards stabilising the economy. In addition, the analysis estimates that over 2015-2025, the debt-to-GDP ratio is considerably higher (+ 11 percentage points) in the no MFA and IMF scenario ratio, compared to the baseline case of having the MFA and IMF operations in place. Figure 16 illustrates. Figure 16: Present value of the debt-to-GDP ratio (%) (Alternative 2: no MFA and IMF) Sources: Cambridge Econometrics calculations; IMF datasets, DSA toolkit and Country Reports on the Kyrgyz Republic. Overall, therefore, the analysis reveals that the absence of MFA and IMF assistance would likely have negatively affected confidence in the Kyrgyz economy and the impact of reforms that were implemented as part of IMF conditionalities. Overall, the public debt outlook in the Kyrgyz Republic following the MFA operation has been, and is expected to remain, stable in part because of support that the country received from international partners such as the EU and the reinforcing role those donors played in driving through necessary reforms associated with financial aid from donors. 6.Conclusions



Hit by the lingering effects of the global economic crisis of 2008 and the interethnic violence of 2010, Kyrgyzstan’s economy plunged in 2010, resulting in significant balance of payments and budgetary financing needs. In response, the European Commission, among other major international donors, announced the mobilisation of EUR 118 million of assistance through various instruments, as well as a EUR 30 million package of MFA. The MFA was disbursed in full, in two instalments, over the period June 2015 and April 2016. The EU’s MFA programme to Kyrgyzstan was exceptional in nature, as the Kyrgyz Republic is out of the normal geographical scope of MFA. This MFA operation was however justified by the strength of the political and economic reform momentum in the country at the time and by its position in a region of strategic economic and political importance for the EU. Indeed, this MFA operation bore great political significance at times of severe political and economic crisis. The evaluation finds that the design of the MFA operation, both in terms of financing envelope and focus of reforms, was relevant to Kyrgyzstan's economic challenges and coherent with EU priorities. MFA disbursements accounted for around 0.3% of Kyrgyzstan’s GDP in both 2015 and 2016, and though delayed, were still relevant given the economic difficulties that prevailed in Kyrgyzstan at the time. The EU’s added value was most apparent in, firstly, reinforcing the promotion of structural reforms in several priority areas, including public finance management, financial sector, trade policy and investment, and business environment. Indeed, evidence suggests that the MFA encouraged the implementation of specific, short-term, measures that may have otherwise been overlooked / further delayed by national authorities. Additionally, MFA reforms were closely aligned to and coherent with those in other EU support programmes (including budget support programmes) and those set by other important donors, including the IMF and the WB. This reinforced the rationale for adjustment measures and provided local authorities with an impetus to reform swiftly in targeted sectors. More broadly, the evaluation finds that MFA was effective in contributing to the stabilisation of the Kyrgyz economy, particularly when combined with the IMF’s support programme, which it complemented. Due to its grant component and loans bearing long maturity (14 and 15 years) and very low interest rates (1.375% and 0.750%), MFA contributed to improving Kyrgyzstan’s public debt sustainability. The favorable terms also suggest that the Kyrgyz Republic may have benefited from savings on the cost of borrowing, and so limited the accumulation of debt to some extent. More broadly, the impact of EU’s MFA and the IMF assistance could be seen through their positive impact in supporting economic recovery, which is key to bringing debt metrics to a sustainable path. Beyond contributing to economic stabilisation, the EU MFA served important political objectives in the Kyrgyzstan. The actual disbursements (coincidentally) came at a politically important moment ahead of parliamentary elections and thus served as a signal of EU support to democratic reforms in Central Asia, consistent with the EU policy towards the region. Furthermore, though the EU MFA did not specifically target social reform, interviews conducted with EU and Kyrgyz stakeholders confirm that the MFA provided the EU with (added) leverage to prevent the enactment of contested legislations in 2015 and 2016, notably the ‘anti-LGBTI’ and ‘foreign agents’. Indeed, the perceived high risk of losing the MFA financial envelope constituted a critical determining factor in the Kyrgyz government’s decision to not pursue the two laws for adoption in parliament. Finally, the evaluation confirms that the MFA operation was implemented efficiently and in close coordination with the Kyrgyz’s authorities, the IMF and the World Bank. This enabled the EU to draw on the expertise of partner institutions. The Kyrgyz authorities were swift and effective in the implementation of the conditionality, fulfilling all conditions, without the need for waivers. Overall therefore, the evaluation finds evidence that the EU MFA to Kyrgyzstan was relevant, coherent, effective and efficiently implemented. Nevertheless, the operation was characterised by an exceptionally long timetable throughout the process, which may have hampered the EU added value and the visibility surrounding the operation. Indeed, the review of the visibility of the operation and the EU communication activity indicates scope for improvement. Since then, with the aim to improve the visibility and public understanding of MFA programmes, the Commission started in mid-2014 to publish on the web site of DG ECFIN the Memorandum of Understanding that lays down the reform measure related to the operations. Nevertheless, the external evaluation confirms that despite the delays in disbursement, the MFA was still relevant in 2015-2016 as they coincided with important debt repayment obligations and with the time Kyrgyzstan faced difficulties as the Russian economic crisis began unfolding. A number of the conclusions and limitations identified will require further consideration and may benefit from cross-comparison with other recently completed MFA operations. A meta-evaluation is planned for 2020 and it will explore these points in more detail. The meta-evaluation will provide a synthesis of the results of the evaluation studies of MFA operations carried out in the last ten years and will consider the reliability and relevance of the methodology, process, outputs, and outcomes of these evaluations. In addition the exercise will assess the timeline of MFA operations and potential linkages to added-value and operational visibility.





Annex 1: Procedural information 1. Organisation and Timing This evaluation assesses the Macro-financial assistance (MFA) programme for Kyrgyz Republic of EUR 30 million, which was launched in October 2013 and completed in April 2016. The assessment is in line with the Financial Regulation (article 30 point 4) and the relevant MFA Decision that call for a submission of an ex post evaluation report to the European Parliament and the Council. The objective of the evaluation is to draw lessons with respect to the EU’s financial assistance, in particular the design and implementation of the programmes and the way they contributed to achieving macroeconomic stabilisation and fostering structural reforms. Apart from identifying areas of improvement for similar on-going or future possible interventions, the evaluation also aims at ensuring better transparency and accountability of the Commission's activities. The evaluation looks at various aspects of this particular EU intervention such as relevance and efficiency, effectiveness, coherence and EU value added. In addition, the evaluation explores the social impact of the MFA operations and their effect on Kyrgyzstan’s public debt sustainability. The work of the external consultant was complemented by internal analysis. The indicative roadmap for the ex post evaluation of MFA for Kyrgyz Republic was published in December 2017. In the context of the framework contract for the provision of evaluation services related to MFA programmes, the Commission awarded on 30 January 2018 the specific contract to Consortium ICF (ICF Consulting Services Ltd. and Cambridge Econometrics Ltd.) The lead DG to carry out and manage this evaluation has been the Directorate General for Economic and Financial Affairs (DG ECFIN). DG ECFIN chaired the Inter-service Steering Group (ISG) that was set up to manage the evaluation. Apart from DG ECFIN, the ISG comprised of representatives of other Commission services (namely the Secretariat General and the Directorate-General for International Cooperation and Development – DG DEVCO) as well as the European External Action Service (EEAS) and the EU Delegation in Kyrgyzstan. A kick-off meeting at which the ISG and the external consultant discussed the foreseen deliverables and the evaluation techniques in depth, took place in March 2018. It was followed by meetings on the inception and interim reports in May 2018 and in July 2018, respectively. Overall, the ISG met three times. In addition, ISG members were continuously informed and consulted, through e-mails and phone calls, during the various stages of the evaluation. 2. Consultation of stakeholders The consultation strategy included a range of appropriate tools to reach key, relevant stakeholders in order to collect pertinent evidence on the implementation of the MFA in Kyrgyzstan. In turn, this did not include an open public consultation, but was rather targeted to specialists - people who have either been closely involved in the development and/or the implementation of the MFA-I operation or persons with expert knowledge in the areas related to the objectives of the MFA operation. In addition, an indicative evaluation roadmap was published in December 2017 to seek wider feedback. Finally, the results of this evaluation will, in due course, feed into a wider meta-evaluation of EU MFA operations, which will include an open public consultation. 3. Evidence, sources and quality The evidence used for the ex-post evaluation of MFA for Kyrgyz Republic included a variety of documents, official data sources, media outlets as well as interviews and surveys. Approximately 60 documents were reviewed and analysed. These could be grouped in four main types - (i) documents directly related to the design and implementation of the MFA programmes, (ii) EU policy-related documents, (iii) documents published by the Kyrgyz authorities, and (iv) documents prepared by various international financial institutions and economic researchers. In addition, a variety of data sources was used and analysed such as Kyrgyz authorities (including Ministry of Finance and Central Bank of Kyrgyzstan), the World Bank, the World Economic Outlook of the IMF, and the EuroAsian Development Bank among others. Overall, the quality of the collected evidence (data, documentation, interviews and survey results) for this evaluation can be assessed as very good, demonstrating a satisfactory degree of confidence regarding the various findings of this evaluation. The quantitative fieldwork was based on reliable statistical data, while purposeful sampling was used for the interviews, the Delphi survey and the focus group discussion. To collect a broad, multi-dimensional and triangulated picture of the economic, financial and structural issues surrounding the programme, a wide range of civil society organisations was also involved. 4. Limitation and Challenges Aside from the limitations and challenges detailed in Section 4 of this SWD, the main challenge surrounding this exercise appeared to be the timeline of the evaluation as initially envisaged. Given the number, complexity and depth of the data treatment, the organisation of several missions, and the availability of the persons to be interviewed, the time line of the evaluation proved somewhat tight. Therefore, to address these pressures an extension of the evaluation time line was deemed necessary.

Annex 2: References Bibliography OECD (2010), Kyrgyz Republic, Aid Effectiveness 2011: Progress in Implementing the Paris Declaration Ministry of Finance of the Kyrgyz Republic (2010), Reporting document, information letter European Commission DG ECFIN (2011), Ex-Ante evaluation statement on EU macro-financial assistance to the Kyrgyz Republic, Staff Working Paper Council of the European Union (2011), Possible Macro-Financial Assistance to the Kyrgyz Republic Further Information, Information note for the Financial Counsellors Working Party European Commission DG ECFIN (2011), Proposal for a decision of the European Parliament and of the Council providing macro-financial assistance to the Kyrgyz Republic. COM(2011) 925 final European Commission (2012), Operational Assessment of the financial circuits and procedures in the Kyrgyz Republic, Report (July 2012) European Commission (2012), Second instalment review, Policy Support Programme, Social Protection and Public Finance Management, Aide Memoire – Review Mission (May 2012) European Commission (2013), Decision of the European Parliament and the Council of 22 October 2013 providing macro-financial assistance to the Kyrgyz Republic, DECISION No 1025/2013/EU European Commission DG ECFIN (2014), Macro-Financial Assistance to the Kyrgyz Republic, MS Committee meeting of 26 June 2014, meeting minutes (June 2014) European Commission DG ECFIN (2014), Memorandum of understanding between the European Union and the Kyrgyz Republic on the Macro Financial Assistance for the Kyrgyz Republic, Grant and Loan from the EU of up to EUR 30 million, Memorandum of Understanding (July 2014) European Parliament (2015), Joint motion for a resolution on Kyrgyzstan homosexual propaganda bill (January 2015) European Commission (2015), European Parliament urges Kyrgyzstan to drop anti-LGBTI bill, press release (January 2015)

European Commission DG ECFIN (2015), Macro-Financial Assistance the Kyrgyz Republic: Disbursement of the first tranche, Information note to the European Parliament and to the Council (May 2015) European Commission DG ECFIN (2015), Commission Decision on the borrowing of the EU for the macro-financial assistance loan to the Kyrgyz Republic, C(2015) 3582 final European Commission DG ECFIN (2015), Commission Decision on the release by the EU of Macro Financial assistance to the Kyrgyz Republic, C(2015) 3581 final European Commission DG ECFIN (2016), Macro-Financial Assistance the Kyrgyz Republic: Disbursement of the second tranche, Information note to the European Parliament and to the Council (May 2015) ECORYS (2017), Ex-post evaluation of Macro-Financial Assistance to Jordan, Report (October 2017) EC Mission reports European Commission (2011), Discussion on possible macro-financial assistance for Kyrgyzstan. Staff Report on mission to the Kyrgyz Republic (23-27 May 2011, Bishkek) European Commission (2012), Mission report to the Kyrgyz Republic in the framework of the MFA Operational Assessment, Staff Report (11-15 June 2012) European Commission (2013), IMF conference on Caucasus and Central Asia 19-21 May 2013, and discussions on the conditionality for the macro-financial assistance operation with the Kyrgyz Republic. Staff Report on mission to the Kyrgyz Republic (21-24 May 2013, Bishkek). European Commission (2013), Discussions on the legal documents associated with the macro-financial assistance operation with the Kyrgyz Republic. Staff Report on mission to the Kyrgyz Republic (8-11 October 2013, Bishkek). European Commission (2013), Report on mission to the Kyrgyz Republic to finalize an agreement on EUR 30 MM macro-financial assistance operation. Staff Report on mission to the Kyrgyz Republic (8-11 October 2013, Bishkek). European Commission (2016), Report on the review mission to Bishkek, Kyrgyzstan, 17-19 November 2015 related to the disbursement of the 2nd tranche of the MFA. Staff Report on mission to the Kyrgyz Republic (17-19 November 2016, Bishkek) IMF mission reviews International Monetary Fund (2011), Public Finance Management Reforms, Report (August 2011) International Monetary Fund (2011), Kyrgyz Republic: First Review Under the Three-Year Arrangement Under the Extended Credit Facility and Request for Modification of Performance Criteria, Staff Report (December 2011) International Monetary Fund (2014), Kyrgyz Republic: Sixth Review under Three Year Agreement under ECF. International Monetary Fund (2015), Staff Report for the 2015 Article IV consultation, Staff Report (November 2015) International Monetary Fund (2016), Kyrgyz Republic: Second Review Under the Three-Year Arrangement Under the Extended Credit Facility and Request for Modification of Performance Criteria, Staff Report (June 2016) International Monetary Fund (2017), Kyrgyz Republic: Third Review Under the Three-Year Arrangement Under the Extended Credit Facility and Request for Modification of Performance Criteria, Staff Report (June 2017) International Monetary Fund (2018), Kyrgyz Republic: Fourth and Fifth Review Under the Three-Year Arrangement Under the Extended Credit Facility and Request for Modification of Performance Criteria, Staff Report (February 2018)

MFA annual reports and SWD European Commission (2012), Report from the Commission to the council and the European Parliament on the implementation of macro-financial assistance to third countries in 2011, Staff Working Document (June 2012) European Commission (2012), Background analysis per beneficiary, accompanying the document: Report from the Commission to the council and the European Parliament on the implementation of macro-financial assistance to third countries in 2011, Staff Working Document (June 2012) European Commission (2013), Report from the Commission to the council and the European Parliament on the implementation of macro-financial assistance to third countries in 2012, Staff Working Document (June 2013) European Commission (2013), Background analysis per beneficiary, accompanying the document: Report from the Commission to the council and the European Parliament on the implementation of macro-financial assistance to third countries in 2012, Staff Working Document (June 2013) European Commission (2014), Report from the Commission to the council and the European Parliament on the implementation of macro-financial assistance to third countries in 2013, Staff Working Document (June 2014) European Commission (2014), Background analysis per beneficiary, accompanying the document: Report from the Commission to the council and the European Parliament on the implementation of macro-financial assistance to third countries in 2013, Staff Working Document (June 2014) European Commission (2015), Report from the Commission to the council and the European Parliament on the implementation of macro-financial assistance to third countries in 2014, Staff Working Document (June 2015) European Commission (2015), Background analysis per beneficiary, accompanying the document: Report from the Commission to the council and the European Parliament on the implementation of macro-financial assistance to third countries in 2014, Staff Working Document (June 2015) European Commission (2016), Report from the Commission to the council and the European Parliament on the implementation of macro-financial assistance to third countries in 2015, Staff Working Document (June 2016) European Commission (2016), Background analysis per beneficiary, accompanying the document: Report from the Commission to the council and the European Parliament on the implementation of macro-financial assistance to third countries in 2015, Staff Working Document (June 2016) European Commission (2017), Report from the Commission to the council and the European Parliament on the implementation of macro-financial assistance to third countries in 2016, Staff Working Document (June 2017) European Commission (2017), Background analysis per beneficiary, accompanying the document: Report from the Commission to the council and the European Parliament on the implementation of macro-financial assistance to third countries in 2016, Staff Working Document (June 2017) Key data sources / statistics Centre for International Development of Harvard University, Atlas of Economic Complexity. Available at: http://atlas.cid.harvard.edu/ IMF, (2018). World Economic Outlook April/July. Available at: https://www.imf.org/en/Publications/WEO/Issues/2018/07/02/world-economic-outlook-update-july-2018 World Bank (2018). Available at: https://data.worldbank.org/ Various statistics from the Ministry of Finance and Economy Various statistics from the CIA World Factbook Various statistics from the National Statistical Committee of the Kyrgyz Republic

Various statistics from the National Bank of the Kyrgyz Republic Various IMF datasets, and Country Reports on the Kyrgyz Republic



Annex 3: Discussion of validity and reliability of the evaluation findings In the Commission’s and the external evaluator’s judgement, the overall reliability and validity of the evaluation is strong. There are multiple lines of evidence and inquiry contributing to answering each evaluation question, mitigating the limitations associated with individual research activities. Moreover, a series of measures were undertaken by the study team to ensure the reliability and validity of the overall findings of this evaluation as indicated in the table below. Overview of the main elements underpinning reliability and validity of findings Criteria for judging quality Measures undertaken for improving quality Judgement Credibility (internal validity) - the extent to which the findings are plausible, believable and trustworthy; and thus can be defended when challenged. Triangulation – combining and cross-checking theory and evidence generated by multiple perspectives, theories, methods, and data sources Respondent validation- sharing interview write-ups with interviewees to ensure accuracy and completeness Hypothesis exploration: multiple hypotheses were tested to identify the best, most probable explanation Strong Transferability (external validity) – the degree to which findings can be transferred to other contexts by the readers. This means that the results are generalizable and can be applied to other similar settings, populations, situations and so forth Detailed description of the context of the MFA operation and methodology to assist the reader in being able to generalise the findings and apply them appropriately Medium Reliability (dependability) - the consistency with which the results could be repeated and result in similar finding Triangulation- evidence compiled from different sources was corroborated and cross-validated Thorough record keeping, ensuring a clear decision trail and transparency in data interpretation Information synthesis: going beyond simply collecting, listing and describing distinct data elements in the interpretive process Strong Confirmability (objectivity) - the degree to which the results could be confirmed or corroborated by others Stance analysis: taking account key informants’ and stakeholders’ backgrounds to assess how their perspective might have biased the information they provided Acknowledging biases in sampling and critical reflection of methods, ensuring sufficient depth and relevance of data collection and analysis Explanation critique - the interpretive chain of reasoning and inferences drawn have been subject to ‘peer’ review and critical challenge by stakeholders Strong Annex 4: Comparison of MFA, IMF and World Bank conditionalities Areas of focus MFA IMF 2011 IMF 2015 WB DPO I WB DPO II PFM - Medium-Term Budget Framework V PFM - PEFA V PFM - Accounting, reporting / Treasury Management Information System / Financial Management Information System V V V PFM - External auditing V PFM - Internal auditing V PFM - Budget consolidation V PFM - Budget law V PFM - Debt management V PFM - Public Investment Management V PFM - Public Procurement V V V PFM - Structure of the MoF V PFM – Budget discipline / transparency V V Banking sector – Banking Code / Law V V V Banking sector - AML V V Banking sector - Crisis Management and Bank Resolution V V Banking sector - De-dollarisation of the financial sector V Banking sector - Level playing field across banks V Banking sector - Macroprudential regulation V Banking sector - Supervision V V V Banking sector - Small and medium enterprise access to finance and leasing Banking sector – Reinforcement of the NBKR’s supervisory function V Banking sector – Strengthening of the Deposit Protection System V Banking sector - Access to finance V Trade policy - application of WTO commitments V Investment and business environment – startup and operations V Investment and business environment – Reduction of red tape V Investment and business environment – Deposit protection arrangements V Investment and business environment – Business inspections V V Investment and business environment – Increased accountability of executive directors V Energy sector V V V Fiscal policy - Expenditure management V Public administration reform V Social protection V VAT / Tax policy / Tax administration reform/ Customs valuation V V Governance / anti-corruption V V Judicial reform V V



Annex 5: Evaluation Framework Judgement criteria Quantitative analysis Qualitative analysis Sources of information Evaluation criteria: ‘Relevance’ Evaluation Question 1: To what extent was the MFA operation design (including adequateness of financing envelope, focus of conditionality) appropriate in relation to the outputs to be produced and objectives to be achieved? The size of the financial assistance was adequate in relation to Kyrgyzstan’s financing needs Form of support was appropriate given Kyrgyzstan’s debt position and income status MFA conditionalities were consistent with and relevant to Kyrgyzstan’s needs and EU’s and other donors’ programmes and realistic given the short-term nature of the instrument The MFA package was generally regarded as relevant to Kyrgyzstan’s needs by stakeholders, local economists, media etc. Financing needs in 2012-13 and how these were met Analysis of financing needs in 2015- 2016 (as done by IMF) and the role of MFA in meeting these needs Comparison between projected and actual financing needs – reasons for deviations and relevance and appropriateness of MFA in light of any changes Analysis of how Kyrgyzstan meets criteria defining eligibility to grants and of EU budgetary constraints Debt sustainability analysis Degree of consensus among key stakeholders/ key informants regarding the relevance and importance of the MFA (in absolute and relative terms) Stakeholders and local economists’ assessment of the use of a mix of loan and grant and focus of the conditionality Examination of whether the focus of MFA conditionality was relevant and appropriate in Kyrgyz context bearing also in mind the characteristics of the MFA instrument Analysis of synergies with the IMF ECF programme / other EU programmes Documentary analysis: -Ex-ante evaluation of MFA to Kyrgyzstan -Memorandum of Understanding (MoU) and Loan and Grant Agreements -Reports and supporting documentation submitted by the Kyrgyz authorities to the European Commission on the fulfilment of the structural reform criteria; -Commission’s assessment of compliance with conditionality requirements (i.e. after mission reviews) -IMF reviews -Other reports including on the progress of PFM reforms accompanied with performance indicators/metrics (e.g. 2015 PEFA report for Kyrgyzstan) Semi-structured interviews: -EC officials: DG ECFIN, DG DEVCO -EEAS -Key representatives of Member States -IMF/ WB officials; -Kyrgyz authorities including also Ministry of Finance and Central Bank of Kyrgyzstan -EU Delegation in Kyrgyzstan -Civil society representatives Focus group with other development partners Evaluation criteria: ‘Effectiveness’ Evaluation Question 2: To what extent have the objectives of the MFA operation been achieved? Macroeconomic impact -There has been an improvement in Kyrgyzstan’s macroeconomic situation including, external financial sustainability -The role and contribution of MFA can be identified -MFA had a positive social impact -The evidence suggests that Kyrgyzstan would have been worst off in absence of the MFA Structural reforms -There is evidence of reform e.g. INTOSAI standards applied at Chamber of Accounts -There is evidence to suggest that MFA accelerated, reinforced or promoted reform in certain areas e.g. the financial sector -There is general consensus among stakeholders that the MFA contributed positively to macroeconomic stabilisation and Kyrgyzstan’s reform effort Analysis of trends in key indicators (National accounts, Balance of payments statistics, Government finance statistics, Monetary statistics, External sustainability before, during and after MFA Analysis of data on lending conditions available for Kyrgyzstan (focusing on financing available from bilateral / multilateral donors and to some extent from national capital markets) Debt sustainability analysis Social impact analysis – evolution of key indicators relating to unemployment, poverty etc. Counterfactual analysis Stakeholders and local economists’ views on the specific contribution of MFA to short-term macroeconomic stabilisation of Kyrgyzstan Stakeholders and local economists’ assessment of the contribution of MFA including structural reforms; Assessment of alternatives available to fill the financing gap if MFA (and IMF) resources had not been provided Stakeholders’ perceptions on other impacts Document and data review -Macroeconomic data sourced from IMF/WB and national sources; -MFA documentation -IMF reviews and country reports; -Credit rating agency reports (i.e. Moody’s) -Academic and grey literature on the Kyrgyz economy; -Data on public borrowing (scale/ maturity/ costs) and prevailing market conditions at the time of MFA/IMF programmes Semi-structured interviews -EC officials: DG ECFIN -IMF/ WB officials -Kyrgyz authorities, in particular, Ministry of Finance -EU Delegation in Ukraine Inputs from local experts -Focus Group with key bilateral/ multilateral donors Evaluation criteria: ‘Efficiency’ Evaluation Question 3: Was the disbursement of the financial assistance appropriate in the context of the prevailing economic and financial conditions in the beneficiary country? Evaluation Question 4: In what way has the design of the MFA assistance conditioned the performance of the operation in respect to its cost and its objectives? MFA disbursements were timely given Kyrgyzstan’s financing needs Timing of disbursements in relation to key macroeconomic developments and Kyrgyzstan’s financing needs Analysis of the timing of disbursements of both MFA and IMF and factors affecting disbursements Time taken between Kyrgyz authorities request for MFA assistance and approval/ disbursement of MFA Document and data review: -MFA documentation -IMF/ WB documentation Semi-structured interviews -EC officials: DG ECFIN, DG DEVCO -EEAS -IMF/ WB officials -Kyrgyz authorities -EU Delegation There were favourable entry conditions for the MFA operation e.g. political commitment; public buy-in, capacity to implement reform The design of the MFA operation was flexible and it adjusted to changes in context and/or feedback mechanisms There was effective dialogue between the European Commission and Kyrgyz authorities There was effective monitoring of the MFA operation Donors were well coordinated Not applicable The extent of liaison between the European Commission and Kyrgyz authorities; and between the European Commission and IMF/ other donors The communication channels used and media treatment received Whether there was effective monitoring of the MFA operation Stakeholders’ feedback on what could have been done differently with the benefit of hindsight Analysis of the choice of conditionality – see also relevance Analysis of synergies with the IMF ECF programme / other EU programmes – see also relevance Identification of good practice / lesson learned from the design and implementation of MFA operation in Kyrgyzstan Document and data review: -Macroeconomic data sourced from IMF and national sources -MFA documentation Semi-structured interviews: -EC officials: DG ECFIN, DG DEVCO -EEAS -IMF/ WB officials -Kyrgyz authorities -EU Delegation Focus group Evaluation criteria: ‘EU added-value’ Evaluation Question 5: What was the rationale for an intervention at EU level and to what extent did the MFA operation add value compared to other interventions by other international donors? Financial added value over intervention at MS level Evidence that MFA reinforced the Government’s commitment to socio-economic reform There is demonstrable evidence of signalling and confidence building effect of MFA operation – building investor and private sector confidence EU had a discernible influence on the design and application of conditionalities There is clear financial added value of EU support – national authorities would have struggled to meet their financing needs in absence of the EU MFA Trends in confidence indicators and proxy indicators of confidence such as interest rates for T-bills and longer-term bonds denominated in local currency and key indicators of the foreign exchange market Qualitative assessment of links between wider fluctuations in confidence indicators and EU assistance Mapping of conditionalities (see also relevance) Stakeholders’ views on the role and influence of EU in the design and application of support package Document and data review: -Macroeconomic data sourced from IMF and national sources; -MFA documentation; -IMF reviews and country reports; -Academic and grey literature; -Credit Rating Agencies reports -Financial markets data Semi-structured interviews: -EC officials: DG ECFIN, DG DEVCO; -IMF/ WB officials; -Other key bilateral/ multilateral donors; -Kyrgz authorities; -EU Delegation in Kyrgyzstan; -Civil society organisations Focus group -Insights from study experts Evaluation criteria: ‘Coherence’ Evaluation Question 6: To what extent was the MFA operation in line with key principles, objectives and measures taken in other EU external actions towards the Kyrgyz Republic? The MFA was fully in line with EU objectives and reinforced EU action deployed via other instruments Not applicable Stakeholders assessment of the coherence of the MFA with other EU external actions Qualitative assessment of the adequacy of the conditionality, potential synergies/ overlapping with other EU instruments Document and data review: -MFA documentation including ex-ante evaluation of MFA to Kyrgyzstan; -Identification of relevant programmes/ actions and review of their documentation; Semi-structured interviews: -EC officials: DG ECFIN, DG DEVCO; -EEAS -EU Delegation in Kyrgyzstan; -Kyrgyz authorities -Other interviews including civil society organisations. Focus group Annex 6: Synopsis Report Introduction Stakeholder consultation is a key element in the ex-post evaluation of MFA to the Kyrgyz Republic and Commission minimum standards have been met, as a comprehensive range of key stakeholders have been reached as part of the consultation.

The implementation of the stakeholder consultation strategy followed the publication of the ex-post evaluation roadmap of MFA to the Kyrgyz Republic in December 2017. The strategy was developed with the overall objective to capture as much information as possible with regard to the MFA programme, in addition to information collected via desk research and data analysis. The consultation focused on extracting recollections from the time when the operations were designed and implemented – See Table A6. The consultation also focused on collecting views on the period after the MFA was ended to assess its impact on achieving its key objectives and drawing lessons for ongoing and future similar EU interventions. Table A6: General Timeline of MFA operation

This consultation strategy described below sets out the objectives of the consultation, maps key stakeholders, presents the consultation methods and tools which are used and demonstrates how the stakeholder consultation fits in the evaluation framework. The diagram below presents a timeframe for the implementation of this consultation during 2018 and 2019. The following section elaborates. Objective setting The objective of stakeholder consultation was to collect as much valuable and relevant information as possible from the key stakeholders involved to construct an ex-post assessment of the design, implementation and impact of the MFA operation. Stakeholders were consulted on the following key aspects: ·Relevance of the MFA operation: the relevance of the objectives, the financial envelop and the conditionality, both at the time of designing the MFA operation and currently with the benefit of hindsight; ·Effectiveness of the MFA operation: the direct results of the MFA operation, i.e. the results of the MFA operation on macroeconomic level and the results in the area of the structural reforms; ·Efficiency of the MFA operation: the design and process of the MFA in terms of value-for-money; ·EU-added value of the MFA operation: the added value of the operation when considering other possible scenarios and alternatives; ·Coherence of the MFA operation: alignment with other support initiatives implemented at the time of the MFA; ·Social impact: the more indirect impact of the MFA operation in the context of social development in Kyrgyzstan, in particular alleviation of the impact of the refugee crisis; ·Debt sustainability: the longer-term result of the MFA I in terms of implications for Kyrgyzstan’s fiscal sustainability. Stakeholder mapping Consultation was targeted to specialists – either people who have either been closely involved in the development and/or the implementation of the MFA-I operation or persons with expert knowledge in the areas related to the objectives of the MFA-I operation (i.e. macroeconomic and fiscal policy, structural reforms in the areas of PFM, social and labour policy, investment climate and trade, and energy). Therefore, the following groups of stakeholders have been central in this consultation strategy: (1) Kyrgyzstan authorities; (2) International Financial Institutions,