# Methodological Limitations of the Literature in the Study of Economic Sanctions, the Ukrainian Crisis Case

Morad Bali

► To cite this version:

Morad Bali. Methodological Limitations of the Literature in the Study of Economic Sanctions, the Ukrainian Crisis Case. DATA SCIENCE International Scientific and Practice Conference Contribution papers, Ministry of Science and Higher Education Russian Federation; Saint-Petersburg State University of Economics; Territorial Body of the Federal State Statistics Service on St. Petersburg and the Leningrad Region (Petrostat); Russian Public Organization "Russian Association of Statisticians"; Sociological Institute of the Russian Academy of Sciences a Branch of The Federal Center of Theoretical and Applied Sociology of the Russian Academy of Sciences, Feb 2020, Saint-Petersburg, France. pp.55-54. [hal-02472943](https://hal.science/hal-02472943)

HAL Id: hal-02472943

https://hal.science/hal-02472943

Submitted on 13 Feb 2020

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

Public Domain
# Наука о данных

## Материалы международной научно-практической конференции

Санкт-Петербург
5 - 7 февраля 2020 г.

<figure><img src="image_1.png" /></figure>

<figure><img src="image_2.png" /></figure>

Российская
Ассоциация
Статистиков

<figure><img src="image_3.png" /></figure>

Санкт-Петербург
2020
# Methodological Limitations of the Literature in the Study of Economic Sanctions, the Ukrainian Crisis Case

**Brief Literature Assessment.** There are not many papers focusing on the economic conflict between the European Union and Russia. Moreover, only a few own a relevant quantitative analysis -well structured and using a robust econometric model. As if this is not enough, the attention to detail is often set aside for the benefit of biased and results-oriented analyses -examples are legion. In an European Parliament Briefing, Russell (2016) admits that sanctions happened simultaneously with the collapse of oil prices. It is also admitted that "Russia economic performance has always correlated closely with crude oil prices" (Russell (2016), p. 7. Yet, without providing any model or methodological path, the author managed to conclude that sanctions had a significant impact on Russia; it seems that readers shall guess by themselves how to isolate sanctions effects from oil price collapse. This example is not the only case of its kind. The same issue exists in Nicoll and Delaney (2015)<fnref n="1" />, Shirov et al. (2015) (even if this paper is a great statistical study), Arnold (2016), et cetera. There is, unfortunately, a large number of papers which don't integrate oil price effects in their analysis. This fact is real, even if the importance of oil price has been put forward in Dreger et al. (2016) where authors demonstrate that the Russian rouble depreciation is mostly due to Brent prices.

**Methodological Biases in Economic Analysis.** Do economic coercive measures have an effect on economies targeted? A simple way to answer this question is to check how GDPs of involved economies react to sanctions. Authors tried to do so, but result differs depending on sources, on studied period, or on selected methods (Table A). Thus, it seems that while coercive measures would have cost to the Russian economy from 0.2% to 9% of its GDP per year; the impact on the European Union appears to be much lower. Nevertheless, this conclusion arising from the literature possesses a weak point. One must remember that we cannot compare apples and pears. For example, in Kholodilin and Netšunajev (2019) and Kholodilin and Netsunajev (2016) authors compare sanctions effects on the sum of all Euro Area GDPs to the lone Russian GDP. The problem of doing so is that

<footnote n="1">¹ Editor and Assistant Editor names.</footnote>
if it is true that some European economies suffered from Russian embargo -mostly countries of the Baltic Sea and other small economies, see Oja (2015), Havlik (2014), or Haraldsson and Skúlason (2016), some others are less exposed. They are less sensitive to Russian counter measures since Russia represent a small share of their trade (export to Russia), and because this small share itself is a small share of their GDP (Malta, Spain, Croatia, Portugal, United Kingdom, and so on). Ineluctably, it means that some economies might counterbalance others, which has the ability to create statistical and econometric limitations.

<table><caption>Table A
Effects of sanctions on GDPs</caption><thead><tr><th>Authors</th><th>Sanctions from</th><th>GDP of</th><th>Effect on the GDP</th></tr></thead><tbody><tr><td>Finance Minister Anton Siluanov (RU)<sup>2</sup></td><td>All</td><td>Russia</td><td>2%</td></tr><tr><td>Deputy Minister Economic Development Alexei Likhachev (RU)<sup>3</sup></td><td>EU &amp; Russian embargo</td><td>Russia</td><td>2%</td></tr><tr><td>International Monetary Fund<sup>4</sup></td><td>All</td><td>Russia</td><td>1-1.5% in medium term &amp; 9% cumulative impact</td></tr><tr><td rowspan="2">Unpublished European Commission Study<sup>5</sup></td><td>EU</td><td>Russia</td><td>0.6% in 2014 &amp; 1.1% in 2015</td></tr><tr><td>EU</td><td>EU</td><td>0.2% in 2014 &amp; 0.3% in 2015</td></tr><tr><td>Davis (2016)</td><td>All</td><td>Russia</td><td>3.8% (growth deceleration)</td></tr><tr><td>Aalto and Forsberg (2016)</td><td>All</td><td>Russia</td><td>1-1.5% per year</td></tr><tr><td>Ashford (2016)</td><td>All</td><td>Russia</td><td>3%</td></tr><tr><td>Kholodilin and Netsunajev (2016)</td><td>All</td><td>Russia</td><td>2% points GDP quarter-on-quarter growth</td></tr><tr><td>Kholodilin and Netsunajev (2019)</td><td>All</td><td>Russia &amp; EU</td><td>Small effects on GDPs</td></tr><tr><td>Oja (2015)</td><td>Russia</td><td>Baltic countries</td><td>0.2% - 0.5%</td></tr><tr><td>Havlik (2014)</td><td>All</td><td>Russia</td><td>1% in 2014-2016; 20 b€ in 2014, 30 b€ in 2015, 50 b€ in 2016</td></tr><tr><td>Harrell and Rosenberg (2016)</td><td>All</td><td>Russia</td><td>1% to 2%</td></tr></tbody></table>

**Methodological Biases in Trade Analysis.** The same kind of bias exists in papers studying effects of economic coercive measures on EU-Russia trade, explaining why results also differ a lot depending on papers (Table B). Indeed, if it is interesting to take a look at changes in trade, it must be done in a very rigorous way. A typical mistake is to observe general changes in trade and avoiding to study goods which are directly targeted by sanctions, Szczepański (2015) for example. It is essential to dig into exports using Standard International Trade Classification (SITC). Mostly because doing so allows to isolate products under sanctions from others. It is then possible to follow their path and to have a better understanding of their fate. For instance, did they really stop being exported to Russia? Third-party countries played a vital role in this economic war. Several circumvention

<footnote n="2">² Russell (2016)</footnote>

<footnote n="3">³ Ibid., 1.</footnote>

<footnote n="4">⁴ https://www.imf.org/en/News/Articles/2015/09/28/04/53/socar080315b</footnote>

<footnote n="5">⁵ https://www.wsj.com/articles/eu-projects-impact-of-sanctions-on-russian-economy-1414583901</footnote>
strategies have been set up in order to avoid coercive measures (shrimp or Camembert from "Belarus", for example). It seems complicated (not to say impossible) to study all these nuances without product-by-product analysis.

<table><caption>**Table B**
Sanctions' effects on trade</caption><thead><tr><td>Authors</td><td>Effects</td></tr></thead><tbody><tr><td>Christen (2015)</td><td>From 34 to 92b€ of trade loss ; 30b€ exports decline over the last quarter of 2014; Decline in exports and tourism expenditure from 44 to 92b€, jobs lost from 0.9 to 2.2 million</td></tr><tr><td>Dragoi and Balgar (2016)</td><td>75b€ of EU-Russia trade decline ; 30b€ trade loss for the EU</td></tr><tr><td>Nureev and Petrakov (2016)</td><td>Imports decline: meat and poultry (-30% 2014), pork (twice less), cheese (-20%), flour grains and leguminous crops (-40%) Russia's import share from EU -48.8% (Jan-Oct 2014) and -45.3% (Jan-Oct 2015); share from APEC +26.6% and 27.9% respectively ; share from CIS unchanged (0.9% points decrease for Ukraine while others increased (particularly Belarus)</td></tr><tr><td>Szczepański (2015)</td><td>In 2014, -12.1% EU exports to Russia ; -13.5% imports from Russia ; 41b€ of total trade value decline EU agri-food exports to Russia fell by 43% (no only goods concerned by the embargo)</td></tr><tr><td>Leering and Nassiri (2014)</td><td>130 000 jobs potentially lost in the EU</td></tr><tr><td>Gros and Mustilli (2016)</td><td>Sanctions impact on trade flow is minimal</td></tr><tr><td>Haraldsson and Skúlason (2016)</td><td>Small economies are more exposed to trade loss.</td></tr><tr><td>Davis (2016)</td><td>Russian counter-sanctions have negligible effects and</td></tr><tr><td>Ashford (2016)</td><td>Russian embargo created shortages and increased food prices in Russia; Germany has 400 000 jobs at stake.</td></tr></tbody></table>

**Conclusion.** The first identified issue is related to the absence of oil prices in econometric models, when studying effects of sanctions on the Russian economy. It is an essential *control variable* and must be introduced in any model pretending to study sanctions effects. Secondly, when comparing effects of coercive measures between countries, it is essential to compare what can be compared. Comparing the GDP of one country to the sum of several other countries makes no sense and induces biased results. Thirdly, when focusing on changes in trade, one must be certain to isolate goods depending on whether they are targeted by coercive economic measures or not. The main interest of doing so is to avoid *counter-balancing phenomenon* between those targeted by sanctions and others. Finally, it seems rational and logical to use a reliable Sanction Index<fnref n="6" /> in order to simulate sanctions in econometric models, as for any other economic variable.

**References**

Aalto, P., Forsberg, T. (2016). "The structuration of Russia's geo-economy under economic sanctions". Asia Eur. J. 14, 221–237.

Arnold, A. (2016). "The true costs of financial sanctions". *Survival*. 58, 77–100.

Ashford, E. (2016). "Not-so-smart sanctions: The failure of Western restrictions against Russia". *Foreign Aff.* 95, 114.

<footnote n="6">⁶ The author is working on such index, and it is available for publication upon request.</footnote>
**Christen, E.** (2015). "Disrupted Trade Relations Between the EU and Russia: The Potential Economic Consequences for the EU and Switzerland".

**Davis, C.M.** (2016). "The Ukraine conflict, economic–military power balances and economic sanctions". *Post-Communist Econ.* 28, 167–198.

**Dragoi, A.-E., Balgar, A.-C.** (2016). "Economic Sanctions against Russia. A critical evaluation". *Knowl. Horiz. Econ.* 8, 63.

**Dreger, C., Kholodilin, K.A., Ulbricht, D., Fidrmuc, J.** (2016). "Between the hammer and the anvil: The impact of economic sanctions and oil prices on Russia’s ruble". *J. Comp. Econ.* 44, 295–308.

**Gros, D., Mustilli, F.** (2016). "The Effects of Sanctions and Counter-Sanctions on EU-Russian Trade Flows. CEPS Commentary, 5 July 2016".

**Haraldsson, G., Skúlason, M.Á.** (2016). "The Economic Impact of the Russian Counter-Sanctions on Trade Between Iceland and the Russian Federation. Nónó EHF". *Tilgjengelig Paa Httpswww.Stjornarradid Ismediaforsaetisraduneyti-MediamediaSkyrslurTheEconomicImpactoftheRussianSanctionsonTradebetween Icel. Pdf*.

**Harrell, P.E., Rosenberg, E.** (2016). "THE NEXT GENERATION OF SANCTIONS".

**Havlik, P.** (2014). "Economic consequences of the Ukraine conflict". Policy Notes and Reports.

**Kholodilin, K.A., Netšunajev, A.** (2019). "Crimea and punishment: the impact of sanctions on Russian economy and economies of the euro area". *Balt. J. Econ.* 19, 39–51.

**Kholodilin, K.A., Netsunajev, A.** (2016). "Crimea and punishment: The impact of sanctions on Russian and European economies".

**Leering, R., Nassiri, M.** (2014). "Russian sanctions: 130,000 European jobs at stake".

**Nicoll, A., Delaney, J.** (2015). "Negative outlook for Russian economy as sanctions bite, 21 (1)".

**Nureev, R., Petrakov, P.** (2016). "Economic sanctions against Russia: expectations and reality". *Bus. Manag. Rev.* 7, 165.

**Oja, K.** (2015). "No milk for the bear: the impact on the Baltic states of Russia’s counter-sanctions". *Balt. J. Econ.* 15, 38–49.

**Russell, M.** (2016). "Sanctions over Ukraine: Impact on Russia". *Members’ Res. Serv. Eur. Parliam. Res. Serv.* 1–12.

**Shirov, A.A., Yantovskiy, A.A., Potapenko, V.V.** (2015). "Estimating potential effect of sanctions on economic development in Russia and EU".

**Szczepański, M.** (2015). "Economic impact on the EU of sanctions over Ukraine conflict". European Parliamentary Research Service.