metadata: authors: 0: author: Doemeland,Doerte
1: author: Estevão,Marcello
2: author: Jooste,Charl
3: author: Sampi Bravo,James Robert Ezequiel
4: author: Tsiropoulos,Vasileios
count: World
docna: 0: docna: Debt Vulnerability Analysis : A Multi-Angle
docty: Policy Research Working Paper
subtopic: Financial Sector Policy,Economic Adjustment and Lending,Macro-Fiscal Policy,Public Sector Economics,Public Finance Decentralization and
            Poverty Reduction,Inequality,Industrial Economics,Economic Theory & Research,Economic Growth,International Trade and Trade Rules
teratopic: Public Sector Development,Poverty Reduction,Industry,Macroeconomics and Economic Growth,International Economics and Trade
abstracts: cdata!: Countries with high debt exposure are
            vulnerable to economic and financial shocks that could lead
            to sovereign defaults. This paper develops a methodology to
            identify countries that are at risk of debt default based on
            four elements of debt vulnerability. These elements capture
            the different ways in which risks associated with high debt
            are assessed, namely: (i) the fundamental, (ii) the
            subjective, (iii) the judgmental, and (iv) the theoretical.
            The fundamental element considers the liquidity, solvency,
            and institutional risk elements of debt vulnerability. The
            subjective element captures the investors’ perceptions of
            debt default, while the judgmental element is based on the
            debt thresholds as defined by Debt Sustainability
            Frameworks. Finally, the theoretical element is normative
            and captures what ought to be. The methodology constructs an
            index for each of these four elements and uses them as
            predictors in a model of public debt default. The
            methodology flags countries that are at risk of default by
            means of machine learning techniques and delivers outputs
            that point to underlying causes of vulnerability. The
            methodology complements existing monitoring tools for
            assessing debt sustainability.
repnme: repnme: Debt Vulnerability Analysis : A Multi-Angle Approach
display_title: Debt Vulnerability Analysis : A
            Multi-Angle Approach : Debt Vulnerability Analysis : A Multi-Angle
listing_relative_url: /research/2022/02/33723532/debt-vulnerability-analysis-multi-angle-approach-debt-vulnerability-analysis-multi-angle
topicv3: Economic Growth m1326241 471,Poverty m1326687 695,Trade m1326924 698,Fiscal Policy m1327144 445,Financial Sector Development m1327681 1920
publishtoextweb_dt: 2022-02-07T00:00:00Z
dois: 10.1596/1813-9450-9929
historic_topic: Public Sector Development,Poverty Reduction,Industry,Macroeconomics and Economic Growth,International Economics and Trade
origu: Off of Sr VP Dev Econ/Chief Econ (DECVP)
no_of_pages: 42
author_keys: 0: author_key: 605307
1: author_key: 1368476
2: author_key: 1496952
3: author_key: 1678658
model_extractions: Policy Research Working Paper 9929

# Debt Vulnerability Analysis

A Multi-Angle Approach _Doerte Doemeland Marcello Estevão Charl Jooste James Sampi Vasileios Tsiropoulos_ Macroeconomics, Trade and Investment Global Practice February 2022
Policy Research Working Paper 9929

# **Abstract**

Countries with high debt exposure are vulnerable to economic and financial shocks that could lead to sovereign defaults. This paper develops a methodology to identify countries that are at risk of debt default based on four elements of debt vulnerability. These elements capture the different ways in which risks associated with high debt are assessed, namely: (i) the _fundamental_ , (ii) the _subjective_ , (iii) the _judgmental_ , and (iv) the _theoretical_ . The fundamental element considers the liquidity, solvency, and institutional risk elements of debt vulnerability. The subjective element captures the investors’ perceptions of debt default, while the judgmental element is based on the debt thresholds as defined by Debt Sustainability Frameworks. Finally, the theoretical element is normative and captures what ought to be. The methodology constructs an index for each of these four elements and uses them as predictors in a model of public debt default. The methodology flags countries that are at risk of default by means of machine learning techniques and delivers outputs that point to underlying causes of vulnerability. The methodology complements existing monitoring tools for assessing debt sustainability.

This paper is a product of the Macroeconomics, Trade and Investment Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at ddoemeland@worldbank.org; mestevao@worldbank.org; cjooste@worldbank.org; jsampibravo@worldbank.org; and vtsiropoulos@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# Debt Vulnerability Analysis: A Multi-Angle Approach<sup>*</sup>

Doerte Doemeland Marcello Estev ̃ao Charl Jooste James Sampi Vasileios Tsiropoulos _Keywords_ : Debt, Default, Factor Model, Prediction.

_JEL classification_ : H12, H63, C32

> *World Bank Group. Ideas in the paper are those of the authors and not that of the World Bank. We thank Timothy Kehoe and Juan Conesa for their help and for sharing their code to simulate our theoretical framework. The ideas in this paper benefited from remarks by Paloma Casero, Andrew Burns, Ivailo Isvorski, Lea Hakim, Jorge Araujo, Aart Kraay, Pablo Saavedra, Gallina Vincelette, Franziska Ohnsorge, Norbert Fiess, Stefano Curto, Naoko Kojo, Gregor Schwerhoff, and Alexander Haider. Author contacts: ddoemeland@worldbank.org; mestevao@worldbank.org; cjooste@worldbank.org; jsampibravo@worldbank.org; vtsiropoulos@worldbank.org
# **1 Introduction**

A balloon is elastic and can expand when filled with air. Good latex balloons can contain even more air. One thing is certain however, inject more air than the capacity of that balloon and it will burst. The noise of the pop is loud and the consequences sudden.

A country’s debt is very much like a balloon. There exists a threshold beyond which a country cannot accumulate more debt lest it defaults. The problem for the economist is that each country is a different balloon, and it is up to country officials and multilateral institutions to understand a country’s capacity to accumulate debt in a sustainable manner. Many countries already have inflated balloons or sizable debts. Even though they do not necessarily need to default, it is important to understand their ability to repay debts in the face of unknown events.

Some of these debt sensitivities and vulnerabilities depend on economic fundamentals, political changes, and transparent debt accounting practices. Assuming that debt is transparently communicated and the political environment known, then policymakers can use, as best as possible, available fiscal and economic data to evaluate debt sustainability.

Using the analogy of the balloon, debt vulnerability analysis is further complicated by the environment, which is state and time dependent. First, one needs to understand the structure of debt (i.e. the balloon itself), which includes expected repayments, ownership, debt size, sensitivity to domestic economic environment, and maturity structure. Next, one needs to consider the operator of the balloon (e.g. institutions, policy makers, and politicians). Political statements and the credibility of institutions may change the fiscal policy landscape and hence influence debt sustainability. Finally, one has to consider the external environment in which the balloon floats (e.g. investor perceptions and exchange rates). Unforeseen exogenous shocks do occur and, depending on the debt structure, may have a material impact on debt sustainability. Careful debt vulnerability analysis considers all three aspects.

Reflecting on the structure of debt over the last couple of years, it is evident that public debt in IBRD and IDA countries has been rising significantly. Debt increases have been particularly acute for IBRD countries (Figure 1a) as a result of large issuance of international bonds by middle-income countries (Figure 1b).<sup>1</sup> External sovereign debt carrying variable interest rates has risen markedly in recent years. A few countries have significant and growing non-resident participation in debt. Shifts in debt composition towards foreign reliance leave countries more exposed to various shocks, such as a sharp increase in global interest rates or capital outflows followed by currency depreciation (Panizza et al. (2009)). The increase in debt also limits fiscal space, making it harder for country authorities to conduct counter-cyclical fiscal policy and requiring future resources to be diverted towards repayments.

Changes in debt composition are also followed by a reduction in the average time to maturity and higher average interest rates (Figure 2a). This leaves countries more vulnerable to liquidity shocks and debt rollover risks, although debt increases and composition changes do not necessarily imply an inevitable default. Several emerging market economies (EMEs) have increased their resilience to shocks thanks mainly to prudent policies, the 1LICs = Low Income Countries; LMICs = Lower Middle Income Countries; UMICs = Upper Middle Income Countries; EMDEs = Emerging Market Developing Economies.

1
<!-- Start of picture text -->
- |<br>Public Debt in IDA and IBRD<br>(percent of GDP, median)<br>— IDA eligible —IBRD —all ——Non-cmdty exporter --=Percentile 25 and 75th<br>80 .<br>70 ee<br>60 wee<br>ae<br>50 weeneee”<br>40 ae .<br>30 pete<br>oo<br>20<br>2010 2012 2014 2016 2018 2020<br><!-- End of picture text -->

<!-- Start of picture text --> Average maturity on official external debt<br>(years)<br>2010 2015 = 2020<br>40<br>35,<br>30<br>25<br>20<br>15<br>10<br>5<br>o<br>EAP ECA Lac MNA SSA SAS<br><!-- End of picture text --> <!-- Start of picture text --> Composition of PPG external debt<br>(percent of total PPG external debt)<br>mBilateral m Multilateral m= Private mTM Bonds<br>100%<br>80%<br>60%<br>40%<br>20%<br>0% 2010 2020 2010 2020 2010 2020<br>uc umic umic<br><!-- End of picture text --> <!-- Start of picture text --> Average interest rate on official external debt<br>(rate)<br>2010 = 2015 = 2020<br>5<br>4<br>3<br>2<br>1<br>0<br>EAP ECA bac MNA SSA SAS:<br><!-- End of picture text -->
lessons have not been learned in spite of a growing policy dialogue on ways to manage debt and run fiscal policy in a sustainable manner (see Figure 3).

**Figure 3:** Number and size of defaults

<!-- Start of picture text --> 100<br>100<br>75<br>75 No Default<br>50<br>Default<br>25<br>50<br>0<br>1990 2000 2010 1990 2000 2010<br>Count<br>Value of Default (%GDP)<br><!-- End of picture text -->

**_Source: Own calculations using 2020 version of Beers and Mavalwalla (2017)_**

Consequently, the work program for addressing public debt vulnerabilities in emerging and low-income countries remains more relevant than ever. The objective is to minimize the economic and fiscal costs associated with shocks and to increase debt data transparency and the efficiency and sustainability of fiscal and debt policies. Four pillars are needed to achieve this: 1) improving debt analysis and early warnings systems; 2) enhancing debt transparency; 3) strengthening debt management capacity; and 4) reviewing debt policies. Developing an early-warning framework to monitor public debt vulnerabilities and fiscal risks for market-access countries, would go a long way to bolster this agenda.<sup>2</sup> That includes: i) developing a methodological framework for assessing debt vulnerabilities; ii) developing a tool to support the analysis; and iii) establishing a consultation process and internal mechanisms to discuss action plans WB teams could propose for countries with elevated debt vulnerabilities.

This paper presents the methodological outline of a framework for monitoring debt vulnerabilities in market-access economies. It seeks to complement existing World Bank debt vulnerability assessments, including the joint Bank-Fund Debt Sustainability Framework.

The paper produces four indicators of debt vulnerability, which are used for predicting debt default. The four indicators capture overlapping-but-different angles of vulnerabil2MACs or market access countries are defined as countries that are not eligible for the Poverty Reduction and Growth Trust (PRGT). Emerging market countries (EMs) are a subset of MACs. MACs thus cover all advanced economies and the majority of emerging markets and may include LICs that have significant access to market financing. The classifications of advanced economies and emerging markets are defined by the IMF’s World Economic Outlook.

3
ity. The indicators by themselves can be used to evaluate sources of risk and, hence, for designing strategies to minimize potential default. Empirical tests show that these indicators improve model predictions of default relative to simple rules of thumb regarding debt thresholds and standard multivariate regressions of vulnerability. The methodology can thus be used for describing sources of vulnerability and for assessing whether a country is becoming more vulnerable.

The rest of the paper is structured as follows: The rationale and a summary of related exercises are presented in Section 2. The overview of the framework with detailed exposition of the different indices are covered in Section 3. Section 4 presents the results of the default prediction. Section 5 discusses the shortcomings of the exercise and how some gaps are filled. Section 6 concludes the paper.

# **2 Rationale and related exercises**

For a creditor like the World Bank, the timely monitoring of debt vulnerabilities is important for a variety of reasons:

1. **Provide better policy advice to countries** . The recognition and assessment of existing vulnerabilities is important for providing sound policy advise to countries and help prevent costly mistakes in the event of large shocks. Delayed government reactions to shocks often entail very costly policy adjustments (Laeven and Valencia (2008)), requiring steep corrective fiscal measures, often with adverse political consequences and a negative impact on growth and poverty reduction. Heightened vulnerabilities may also increase relative price volatility and overall economic and financial instability (Calvo et al. (2006)). Lack of preparedness often leads to delayed or inadequate policy responses (Solomon (2009)).

2. **Help establish crisis prevention mechanisms** . Crisis events can quickly erase gains in growth and poverty reduction. Debt vulnerability assessments can help inform World Bank responses to prevent possible crisis in countries.

3. **Help countries build resilience** . World Bank operations and analytical work focus on building resilience to financial, climate, and fiscal shocks to protect and deepen reductions in poverty and inequality. The importance of a proactive Word Bank response to anticipated crises is emphasized by the Independent Evaluation Group (IEG) of the World Bank (Group (2017)).

There already exists a large body of work regarding the prediction of defaults or earlywarning systems. To mention a few, Manasse et al. (2003) create an early warning system of default using several liquidity and solvency indicators in a logit and tree-based model. Their results suggest that a high external debt-to-GDP ratio makes a country more likely to enter a crisis (one of their measures of solvency), but is not a good predictor of remaining in a crisis. The World Bank produced a book in 2011 citing various sources of default (Primo Braga and Vincelette (2011)). High external debt and high credit spreads significantly indicate a rising probability of default. Interestingly, using a Bayesian Model Averaging methodology to uncover the importance of different explanatory variables, only 4
external debt enters the model significantly. There is also an increase in the use of machine learning techniques to determine which variables are useful for predicting crisis. Savona and Vezzoli (2015) show that regression-tree based methods outperform competing methodologies such as logit. In their example, liquidity and past default behavior predict oncoming crisis. Finally, there are also attempts to use theoretical models to predict crisis. Gumus et al. (2017) show how a theoretical model can be used to derive endogenous time varying probabilities of default. They show that these model-based-probabilities outperform some other empirical methods such as a logit model to predict sovereign crises in Argentina.

The methodology proposed in this paper describes several analytical advances. Our proposed framework would complement existing monitoring tools by establishing:

1. An established methodological framework and a set of analytical results that assesses country-specific public debt vulnerabilities. The Bank produces and monitors country risk from various angles. The purpose of this work is to supplement these analyses with a deeper focus on sovereign debt vulnerabilities. These include a proposed metric of public debt risks, default probabilities related to debt sustainability and to deteriorating perceptions, and country-specific safe debt thresholds, which are described in detail below.

2. An easy-to-use system that delivers key results to country teams and will serve as a quick one-stop shop to assess public debt risks.

# **3 An overview of the framework**

Debt-vulnerable countries have a high probability of default either conditional or unconditional (e.g. standard debt dynamics when for example a large primary deficit is accompanied by real interest rates that exceed real GDP growth yielding higher debt levels in subsequent periods) on shocks. A country, by design, is not vulnerable to default if it can weather volatile political periods, can sustain a continuous and large exchange rate depreciation, changes to external creditor market conditions, manage fiscal policy in a sustainable manner, and face a slowdown in domestic and trade partner economic growth.

This paper posits that four measured aspects of an economy may capture and predict the likelihood of default.

The first considers fundamentals. Measures that capture liquidity, solvency, and institutional risks are combined into a single vulnerability variable called the **Fundamental Index** . There exists potentially hundreds of economic indicators that could affect a country’s ability to service its debt. Combining these measures, using country-specific weights, adds the advantage of weighing the most important indicators for debt vulnerability while keeping all other variables in quantitative analysis. The Fundamental Index is a proxy for the core of what makes debt vulnerable. Increases in any of the indicators point to an increase in exposure and hence make a country more vulnerable to shocks.

The second aspect captures investor’s subjective perceptions regarding risks. Credit default swaps, or interest rate differentials between a country’s average bond interest rate and that of a risk-free rate (e.g., of bonds issued by the German or the U.S. government) are used to derive an implied **Probability of Default Index** . There are periods when market 5
perceptions deviate from fundamentals, sometimes abruptly and significantly. An implied default probability measure can capture the market’s views as they change, which would otherwise not have been picked up in low-frequency variables connected to fundamentals. As will be shown below, the index captures a key ingredient in a DSA by essentially inverting a ”fan chart” to produce a measure of how far a country is from its maximal effort to implement discretionary fiscal policies in an attempt to reduce debt.

The third aspect is expert evaluation of risks. Using standard DSA debt indicators, we evaluate for each country the number of variables that exceed thresholds of risk as defined by the DSAs into a **Count Index** . DSAs have been a staple input for evaluating debt risks. Economists at the IMF and the WBG use judgment and history as a guiding principle to assess whether a country is in debt distress. The Count Index reflects some of this judgment.

The fourth and final aspect captures what ought to be. Using a theoretical framework that embeds endogenous default, we calculate time varying debt default thresholds for each country to construct our **Theory Index** . The theoretical model is used to find debt ratios that are indicative of risk. These theoretical thresholds are endogenous and hence different than DSA guidelines on debt. They are time varying and embed the possibility of a self-fulfilling debt crisis - i.e., when market perceptions deteriorate significantly from fundamentals.

These four indices are constructed for each country and then used as explanatory variables for debt default. Figure 4 illustrates the objective. The red circle represents the unit disk (i.e. 1 = default), while the blue area represents the space spanned by the four angles of debt.

**Figure 4:** Angles of debt vulnerability

<!-- Start of picture text --> Vulnerabilityindex<br>Implieddefaultindex<br>Capturesfundamentals<br>Capturesperceptions<br>Theoryindex Capturesjudgement<br>Captureswhatoughttobe<br>Countindex<br>Unitcircle= Default<br>Parabola= Fitteddefaultprobability<br><!-- End of picture text --> Since a key aim is predicting default, the analysis attempts to incorporate as much information as possible (reduce omitted variables), while not trying to overfit relationships. To this end, the indices are used as explanatory variables in estimating observed or historical default episodes. The strength of each of the indices in terms of prediction is assessed empirically. Given that this part of the work is one related to a prediction (i.e., to default or not), several advances in econometrics and machine learning are employed. The meth6
ods employed for prediction include a classical logit regression, and two methods from the machine learning literature: random forests and the least absolute shrinkage and selection operator (LASSO) (Tibshirani (1996)).

The following subsections describe each of the analytical angles for analyzing the risks of debt default.

# **3.1 The Fundamental Index**

As much data as possible are collected for each country and then reduced to a single indicator of vulnerability. A resilience index is also constructed to measure the attempts of government to offset default risks. We define resilience as a country’s ability to mitigate adverse shocks. Some measures of resilience entail having healthy buffers such as large reserves for extraordinary financing if called upon, strong FDI flows instead of portfolio flows (hence reducing the likelihood of sudden stops), floating exchange rates (although this can be a risk too if a country accumulated large foreign currency denominated public debt), the existence of large and accessible sovereign wealth funds, long-maturity debt with moderate-to-low upcoming redemptions, low short-term interest rates, credible monetary policy, financial sector stability, stable exchange rates, extent of political and economic stability, and a sustainable current account balance.

The selection of variables to be used for the **fundamental** and **resilience** indices is based on the literature, which has identified that public debt vulnerabilities typically arise due to three types of fundamental risks:

1. Solvency risks are associated with the size of net present value liabilities exceeding the net present value of assets - or a country is insolvent when the net present value of future primary balances is smaller than outstanding debt (Bouabdallah et al. (2017), Roubini (2001)). Thus, if obligations become so large that no future primary balances can finance the debt, a government might default.

2. Liquidity risk arises if the government does not have new financing or cash at hand to service upcoming redemptions (Bouabdallah et al. (2017), Roubini (2001)). This is often a short-term indicator of risk.

3. Commitment risk is related to the government’s fiscal stance and the prudent management of fiscal policy.

The literature shows that one of the first warnings that a country might default is due to the maturity structure of debt - where short-term debt cannot be easily rolled over (Cole and Kehoe (1996), Rodrik and Velasco (1999)). Sanchez et al. (2018) show that before a default episode, the maturity of debt shortens and interest rate spreads increase as a consequence of lower economic growth. Bocola and Dovis (2016) show that interest rates are a significant source of default risk - they calibrate a model for the Italian economy and find that 12 percent of the spreads during 2008-2012 were due to rollover risks. Similarly, Arellano and Ramanarayanan (2012) show that higher interest rate spreads lead to a shortening of debt maturities and, thus, rollover risks.

7
A second warning is the size of foreign-currency (FX) denominated debt (Tomz and Wright (2013)), which exposes countries to currency risk. If countries only issued in domestic currency then a country can implicitly default by inflating debt away; an option not available for FX-denominated debt.

Political institutions matter for repaying debts. As an example, Biglaiser and Staats (2012) show that countries with strong courts are less likely to default. Another factor that could lead to default is the mere issuance of new debt to devalue existing debt - since new issuance increases the likelihood of default. This debt dilution can account for up to 78 percent of default risk and eliminating dilution decreases optimal duration by roughly two years (Hatchondo et al. (2015)).

”Twin-crisis” events occur when the financial and public sector default. Balteanu and Erce (2017) show that balance sheet interconnections (e.g. materializing contingent claims and guarantees), credit dynamics (non-performing loans increase due to higher interest rates that reflect both public and private sector risks) and economic growth (e.g. the link between asset prices, investments, and economic growth) are drivers of banking and public sector crises. This is due to falling capital ratios requiring public sector intervention in the case of a financial crisis. Balteanu and Erce (2017) also argue procyclical fiscal policy along with a lack in competition might make banks vulnerable to shocks. Obstfeld (2011) argues that foreign debt (both public and private), especially short-term liabilities, worsen the credit worthiness of the public sector.

The selected variables for the public debt fundamental index, which are associated with liquidity or solvency risk include: (i) public debt stock; (ii) financing needs (both public and external), (ii) exchange rates, (iii) interest rates, (iv) currency composition of debt, (v) debt service, and (vi) current account balances. Standard variables for liquidity risk are: (i) redemption schedules and (ii) amount of short term debt. Commitment risks include: (i) reserves proxying as buffers to unexpected shocks and (ii) differences between observed primary balances and primary balances required to stabilize debt. Links to the external and financial market include external financing needs, short-term external debt, non-performing loans, and interest rate spreads between loans and deposits. Data are sourced from the World Bank’s WDI indicators, the Macro Poverty Outlook and International Debt Statistics, others are sourced from Bloomberg, the Bank of Canada and the IMF’s WEO. Ideally, one would want to cast a wide net and capture all of these data but they are not all available for every market-access country in this study. Thus, the set of variables will vary across countries in our sample.

A summary of data and their sources used in this study include:

- **Indicators of public debt** : Debt maturity profiles (World Development Indicators (WDI), Bloomberg) and repayment schedules (Bloomberg) to assess liquidity risks; debt service costs (MPO, WDI, WEO) and currency composition (WDI, Bloomberg) to assess exchange rate risks; fiscal space (Macro Poverty Outlook (MPO), WDI, World Economic Outlook (WEO)), gross public financing needs, and required primary balances(MPO, WDI, WEO) to assess scope and ability of policy adjustments; and actual defaults (Bank of Canada, credit ratings (Bloomberg).

- **Indicators of resilience** : Foreign exchange reserves (WDI), exchange rate regime (Ilzetzki et al. 2017), monetary policy regimes, credible institutions (Country Pol-

8
icy and Institutional Assessment (CPIA) from the WDI), and sovereign wealth funds (country economists).

- **Macroeconomic indicators** : Core measures to calculate ratios, real variables and general indicators of economic health include real and nominal GDP, exports, public debt, the consumption price deflator, real and nominal exchange rates, average interest rates, credit spreads on public debt, public expenditures and revenues, the current and capital accounts of the balance of payments, credit ratings and default episodes (MPO, WDI, WEO, Bank of Canada, Bloomberg), and non-performing loans (WDI).

The methodology aggregates the above mentioned variables into a single index. The weights are estimated via a standard dimension reduction technique (e.g. dynamic factor models or principle components) using data starting from 2002. The advantage of this technique is that weights are neutral and completely dependent on the data. This methodology controls for country characteristics, where individual series might have a different weight across countries in each index (see Appendix A for more details on the methodology).

The proposed index measures whether country debt vulnerabilities are increasing or abating: (i) an increase in the vulnerability index highlights risk build-up, (ii) an increase in the vulnerability index relative to its trend highlights risk build-up with respect to its history, or (iii) an increase in the country index relative to regional trends to identify large outlier countries within a region. An increase in the resilience index implies that countries are building up buffers against possible unexpected shocks.

Principal components analysis (PCA) is used for extracting the vulnerability index from the annual data module. PCA is appealing because of its computational advantages and asymptotic properties in large data sets (see Bai (2003)).

After estimation, a fundamentals vulnerability index is generated. Figure 5 illustrates build-up of public and private debt vulnerabilities within each country using the proposed index. It is important to note that an increase in each of the variables used in the index point to an increase in vulnerability, such that the index has the same interpretation and such that the weights are positive (i.e., an increase in the index implies an increase in one of the variables - e.g., larger foreign denominated debt issuance will contribute to an increase in the index).

The black dots in Figure 5 represent the debt vulnerability indices by country while the red lines are the country trends of these indices (i.e. by construction the average is roughly zero). The figure summarizes vulnerability since 2002, while the making of each country’s index takes the full sample size (which differs by country) into account.<sup>3</sup> A positive value indicates that a country is now more vulnerable than, on average, in the past. As an example, Argentina’s index has been rising steadily since 2012 and is above its trend. Using this measure, ex post, one would have inferred rising vulnerabilities faced by Argentina and using the decomposition chart in the Appendix, that most of the risk was emanating from upcoming amortization. Other countries in the sample (e.g., Brazil and Belarus) also have rising vulnerabilities, however, have not defaulted (in Brazil’s case, probably because foreign debt is a very small share of its large public sector debt).

> 3See Appendix E for the same chart but for the resilience indicator.

9
The results suggest that these countries should be closely scrutinized and action plans to mitigate a debt disaster should be set.

**Figure 5:** Country indices

<!-- Start of picture text --> ALB DZA AGO ARG ARM AZE BGD BLR BLZ<br>−5 05<br>BOL BIH BWA BRA BGR CMR CHN COL COG<br>−5 05<br>CRI DJI DMA DOM ECU EGY SLV GAB GEO<br>−5 05<br>GHA GTM GUY HND IND IDN IRN JAM JOR<br>−5 05<br>KAZ KEN LBN LSO MRT MUS MEX MDA MNE<br>−5 05<br>MAR MMR MYS NIC NGA PAN PAK PRY PER<br>−5 05<br>PHL ROU RUS RWA SEN SRB SLB ZAF LCA<br>−5 05<br>SDN SWZ TJK TZA THA TUN TUR UKR UZB<br>−5 05<br>2002 2011 2002 2011 2002 2011 2002 2011 2002 2011 2002 2011<br>VUT VNM ZMB<br>−5 05<br>2002 2011 2002 2011 2002 2011<br>Exposure to public debt risk<br><!-- End of picture text --> The results can also be summarized for a cross-section (i.e., a given year). The vulnerability and the resilience indices jointly determine the ”Red-Square” measure (see Figure 6). The horizontal line depicts the vulnerability indices (centered on zero) for public and external debt, while the vertical line represents the resilience index.

A country that has rising vulnerabilities and a reduction in resilience falls into the red square. Countries with a reduction in vulnerabilities and an increase in resilience find themselves in the green square and are possibly less vulnerable to default risk. The two pink squares still indicate risk, but less than the red-square measure. The red square highlights countries such as Argentina, Lebanon, and Pakistan with heightened public debt exposure. For a country like Turkey, the predominant debt risk is related to the private sector and with falling reserves related to attempts to defend the currency.

# **Sources of vulnerability**

The methodology, with its estimated weights, allows decomposing the index into its components (see Appendix A for the relative weights). These weights summarize the importance of each of the variables in the construction of the indices in terms of total explained variation. The sum of the weights add up to 1. All the indicators, as mentioned above, are constructed such that an increase in any one of them should lead to an increase in vulnerability. This is done to avoid counteracting effects. As an example, for Argentina, 1 percent of the variation in the Fundamental Index is due to larger short-term debt, while 10
**Figure 6:** The red-square

<!-- Start of picture text --> PRY ECU<br>4 PAN GTM<br>BGD IDN ALB<br>KEN<br>THA CMR<br>MEX DOM<br>RUS<br>VNM PHL BGR MUS CRI MDAMNEBIHCOL ZAFTUN HND<br>2 DMA UKR IND JOR LCA BLZ BLR BRA TJK GEO SEN<br>GUY ARG AZE AGO<br>GHA NGA KAZ GAB<br>COG SWZ UZB MYS JAM<br>MRT MAR LSO DJI ARM TZA<br>0 BWA BOL ROU CHN SLV PAKNIC LBNSLB ZMB<br>VUT PER EGY<br>SRB<br>SDN<br>MMR<br>RWA<br>−2<br>TUR<br>IRN<br>−4 DZA<br>−4 0 4<br>Exposure to debt risk<br>Resilience<br><!-- End of picture text --> 16 percent accounts for upcoming redemptions. The rest is split between the remaining variables.

# **3.2 The Default Index**

## **Proxying government ability**

The fiscal solvency constraint states that if debt is larger than the sum of the discounted future primary balances, then a country defaults. This gives rise to a debt stabilizing primary balance, or the deficit required to keep debt equal to the previous period. One can use this measure and compare it to a country’s historical efforts (measured as past primary balances). If the required primary deficit is larger than the historical deficits, then the likelihood that a country will default increases. In most DSAs this information is captured through a ”fan chart” that summarizes the possible paths of debt as an indication of sustainability. If debt is rising (or if its colored fans are expanding upwards) then there is a chance that debt will not be kept fixed at a certain level - in this case the risk of default increases (and policy changes may not be sufficient to cover upcoming interest repayments). The fan chart is constructed by drawing random values from a joint distribution (usually real interest rates and GDP growth from a macroeconomic model) and constructing the subsequent debt paths.

One can invert the formulas and proxy it by calculating the probability distribution function of primary balances and contrast it to the required primary balance to stabilize debt. This is not a perfect proxy, but yields an estimate of a country’s default probability associated with ability (i.e., its past actions speak to its resolve to keep debt stable). A large 11
required primary deficit to finance debt implies a larger probability of default.

The primary balance required to stabilize debt is defined by a stable or decreasing level of debt given projections for GDP growth, interest rates, and primary balances. This can be expressed as: where _r_ is the average implied real government interest rate on debt, _g_ is average real GDP growth, _d_ is the gross debt to GDP ratio and _pb_<sup>_∗_</sup> _t_<sup>istheprimarybalancerequiredto</sup> stabilize debt.

Assuming that historical primary balances _pbt_ give us an indication of the government’s effort to conduct sustainable fiscal policy, the probability of default associated with ability is defined as: If _pbt < pb_<sup>_∗_</sup> _t_<sup>_∀t_then a government cannot meet the debt stabilizing condition.We assume</sup> that this probability can be expressed as a cumulative normal distribution (Φ). The use of a normal distribution is based on convenience and simplifies the analysis (Figure 7 removes deficits larger than 20% and surpluses larger than 20% for all countries in the sample - historical primarily balances are approximately normal). Future work may improve this part of the work by including distributions that have fatter tails. The probability of default can then be expressed as:

**Figure 7:** Distribution of of primary balances

<!-- Start of picture text --> Primary balance (%GDP)<br>.14<br>.12<br>Histogram<br>.10 Normal<br>.08<br>.06<br>.04<br>.02<br>.00<br>-24 -20 -16 -12 -8 -4 0 4 8 12 16 20 24<br>Density<br><!-- End of picture text --> That is, we compute rolling averages of the required primary balance _pb_<sup>_∗_</sup> using the interest rate–growth differential each period.

Figure 8 illustrates this idea. Assume that a country requires a primary balance of 6 percent of GDP to stabilize debt in the following year. According to the historical outcomes 12
<!-- Start of picture text --> 0.3 Required PB to keep<br>debt stable |!<br>0.2 l<br>l<br>l<br>0.2 I<br>l<br>0.1 l<br>l<br>0.1 I<br>l<br>Historical PBs I<br>0.0<br>-8.0 -6.0 -4.0 -2.0 0.0 2.0 4.0 6.0 8.0<br><!-- End of picture text -->
of political instability or debt related issues. O’Kane (2011) proposes to account for additional volatility by incorporating a Gaussian term, where the default rate is defined as follows: where _σCDS,t_ is the CDS conditional volatility (e.g. estimated from conditional heteroskedasticity models, ARCH or GARCH). Finally, the probability of default associated with deteriorating perceptions (SDP) is calculated as follows: Figure 9 illustrates the perceived probability of default for Argentina, Lebanon, Turkey, and the Rep ́ublica Bolivariana de Venezuela until 2019 as an example using the methodology above. Argentina shows higher risk during the 2009-2014 period, when EMBI spreads peaked at 900 bps, mainly because of the public debt restructuring negotiations. Similarly, Turkey’s probability sharply declined during the 2012-2013 period, after Fitch upgraded Turkey’s credit rating to investment grade following an 18-year gap. Finally, the Rep ́ublica Bolivariana Venezuela shows a constant risk-trend since 2014, after the Maduro election. Nonetheless, a resurrection of risk is pointed out during 2018/2019 for all of them.

**Figure 9:** Perceived probability of default

<!-- Start of picture text --> ARGENTINA LEBANON<br>1.0 .7<br>0.8 .6<br>.5<br>0.6<br>.4<br>0.4 .3<br>0.2 .2<br>09 10 11 12 13 14 15 16 17 18 09 10 11 12 13 14 15 16 17 18<br>TURKEY VENEZUELA<br>.45 1.0<br>.40<br>0.9<br>.35<br>.30 0.8<br>.25 0.7<br>.20<br>0.6<br>.15<br>.10 0.5<br>09 10 11 12 13 14 15 16 17 18 09 10 11 12 13 14 15 16 17 18<br><!-- End of picture text --> Figure 10 below summarizes the two types of default calculations. Rational expectations suggest that investor perceptions should be aligned with fundamentals and that there should be no arbitrage opportunities, implying that the two probability measures (one aligned with fundamentals and ability and one aligned with perceptions) should co-move. In most cases the two default probabilities move in the same direction. However, in some cases perceived default exceeds the probability associated with ability. We view this case as a heightened risk of default (or the probability of a panic), which may lead to self-fulfilling debt crisis. These estimates are used later in the estimation of debt thresholds.

14
**Figure 10:** Default probabilities

<!-- Start of picture text --> AGO ALB ARG ARM AZE BGD BGR BIH BLR<br>0.9<br>0.6<br>0.3<br>BLZ BOL BRA BWA CHN CMR COG COL CRI<br>0.9<br>0.6<br>0.3<br>DJI DMA DOM DZA ECU EGY GAB GEO GHA<br>0.9<br>0.6<br>0.3<br>GTM GUY HND IDN IND IRN JAM JOR KAZ<br>0.9<br>0.6<br>0.3<br>KEN LBN LCA LSO MAR MDA MEX MMR MNE<br>0.9<br>0.6<br>0.3<br>MRT MUS MYS NGA NIC PAK PAN PER PHL<br>0.9<br>0.6<br>0.3<br>PRY ROU RUS RWA SDN SEN SLB SLV SRB<br>0.9<br>0.6<br>0.3<br>SWZ THA TJK TUN TUR TZA UKR UZB VNM<br>0.9<br>0.6<br>0.3<br>2008 2012 2016 2008 2012 2016 2008 2012 2016 2008 2012 2016 2008 2012 2016 2008 2012 2016<br>VUT ZAF ZMB<br>0.9<br>0.6<br>0.3<br>2008 2012 2016 2008 2012 2016 2008 2012 2016<br>Ability Perceptions<br>Probability of default<br><!-- End of picture text -->

# **3.3 The Count Index**

An important element of the analysis is a display of the raw data and organizing the variables by indication of vulnerability or lack thereof. This is important for several reasons, which include (i) transparency about the data used, (ii) serving as an additional filter to spot risks that might not be picked up by the indices, and (iii) serving as a benchmark against other models of vulnerability. Some of these indicators have already been used in standard DSA tables. We rely on thresholds (determined jointly by Bank-Fund staff) to highlight risk in the various indicators (Appendix D highlights the table of indicators).

In total there are 12 indicators for which thresholds are provided: Current account balance (CAB), which has a threshold of 5 percent to GDP; external debt as a share of GDP (threshold equals 60 percent); external private non-guaranteed debt as a share of GDP (threshold equals 20 percent); external gross financing needs as a share of GDP (threshold equals 15 percent); short-term external debt as a share of total external debt (threshold equals 5 percent); gross public debt as a share of GDP (threshold equals 60 percent); external public debt as a share of GDP (threshold equals 30 percent); budget deficit as a share of GDP (threshold equals 4 percent); the primary deficit as a share of GDP (threshold 15
equals 3 percent); short-term debt as a share of GDP (threshold equals 4 percent); public debt held by non-residents as a share of gross debt (threshold equals 45 percent); and EMBI global diversified yield spread (threshold equals 600 basis points). The debt sustainability framework (DSF) uses indicative thresholds, linked by country classification, to analyze the risk of external debt distress. Thresholds are (statistically determined) bounds above which the risk of debt distress is considered elevated. Note that these thresholds were derived to maximize their joint correlation with a very specific definition of a debt distress event conditional on a very specific set of control variables. In this methodology we use the data on actual defaults, which may not correlated perfectly with the debt distress indicators used for deriving these thresholds.<sup>8</sup> Figure 11 compares the unweighted average debt to GDP ratios for different shares of indices exceeding their thresholds over time. In the majority of cases very high debt is associated with a higher share of indicators breaching their thresholds.

**Figure 11:** Share of indicators exceeding thresholds

<!-- Start of picture text --> Debt and judgment<br>2011 2012 2013 2014 2015 2016 2017 2019 2020<br>150<br>100<br>50<br>0<br>0.4 0.8 0.4 0.8 0.4 0.8 0.4 0.8 0.4 0.8 0.4 0.8 0.4 0.8 0.4 0.8 0.0 0.4<br>Share of indices<br>Debt (%GDP)<br><!-- End of picture text --> Note that DSA’s are subject to revisions. This makes sense as new types of information and analytical tools help to make analysis more concise. As an example, the new MAC-DSA framework includes several new features and is being rolled out towards the end of 2021.<sup>9</sup>

# **3.4 The Theory Index**

In this section we derive an angle of vulnerability by constructing an index related to a theoretical model of default. This theoretical model derives the economic conditions for when it is optimal for a country to default, and when it is strategic to avoid defaults. The derived quantities of the model produces time varying threshold debt to GDP ratios, which are codified into an index used in the DVA. The Theory Index relies on the models developed by Cole and Kehoe (1996, 2000) and Conesa and Kehoe (2017) which we extend to study the debt-thresholds of multiple economies. The underlying model is a dynamic stochastic general equilibrium model that characterizes the values of government debt and

> 8For more detailed discussion of why the DSF employs thresholds, and on the probit methodology, please see IMF (2017), and Kraay and Nehru (2006). 9SeeSee

> 9SeeSee `https://www.imf.org/en/Publications/Policy-Papers/Issues/2021/02/03/ Review-of-The-Debt-Sustainability-Framework-For-Market-Access-Countries-50060`

16
the debt’s maturity structure consistent with a sustainable debt dynamics in the case of financial crises brought on by a loss of confidence in the government. The model also characterizes the optimal policy response of governments to the threat of such a crisis. The results of this section yield time varying debt theoretical thresholds that we can compare against actual debt outcomes. Three thresholds are estimated - one for a safe level of debt (where the likelihood of default is extremely low), one for risky levels of debt (where a country is vulnerable to exogenous shocks and can default), and another for default levels of debt to GDP (this is where a country will default). A country that exceeds the latter for a given year is a assigned a value of 3 for that index. A country that is in the safe area is assigned a value of 0. Thus, increasing values represent a deterioration in the debt sustainable outlook - and hence an increasing chance of default.

We start by defining the theoretical model used for calibrating risky debt thresholds.

Following Conesa and Kehoe (2017) we simplify the Cole and Kehoe (1996, 2000) model by eliminating the representative household’s consumption-investment choice, while extending Cole and Kehoe (1996, 2000) by introducing debt maturity. The state of the economy in every period is: _s_ = ( _B, z−_ 1 _, ζ_ ), where _B_ is the level of government debt, whether default has occurred in the past _z−_ 1 = 0 or not _z−_ 1 = 1, and _ζ_ is the value of the sunspot variable. The country’s GDP is: As in Conesa and Kehoe (2017) and Cole and Kehoe (1996, 2000) the drop in productivity by the factor Z is the country’s default penalty. Once _z_ = 0 , it stays equal to 0 forever. Here the default penalty occurs in the same period as the crisis. We keep the structure of the income shocks deliberately simple, so that incentives to smooth consumption even in periods of vulnerability to lenders’ panics are tractable. Government’s tax revenue is _θy_ ( _z_ ), where we assume that the tax rate _θ_ is fixed. Given that there is no consumptioninvestment choice as in Conesa and Kehoe (2017), the consumption of the representative household is The benevolent government offers _B_<sup>_′_</sup> in new bonds for sale and chooses whether or not to repay the fraction _δ_ of the existing stock of debt that becomes due, _δB_ . As in Conesa and Kehoe (2017) we follow Chatterjee and Eyigungor (2012) and Hatchondo and Martinez (2009) which use a ”memory-less” debt maturity form, so it is not necessary to keep track of the entire distribution of maturities of debt. Hence, the government’s budget constraint is: where _q_ ( _B_<sup>_′_</sup> _, s_ ) is the price that international bankers pay for _B_<sup>_′_</sup> , _g_ is government expenditure, and _z_ is a binary variable that denotes the government decision to default or repay. Notice that the problem reduces to the one-period debt case when _δ_ = 1, or to infinitely lived debt as _δ_ tends to 0.

In every period, _ζ_ is drawn from the uniform distribution on a support [0,1]. If _ζ >_ 1 _−π_ , international bankers expect a crisis to occur and do not lend to the government if such a crisis would be self-fulfilling. The empirical part of our methodology allows us to set the 17
probability of a self-fulfilling crisis at a specific value _π_ , 0 _≤ π ≤_ 1. The timing within each period is like that in Cole and Kehoe:

1. The shock _ζ_ realizes, the aggregate state is _s_ = ( _B, z−_ 1 _, ζ_ ), and the government chooses how much debt _B_<sup>_′_</sup> to sell.

2. Each of a continuum of measure one of international bankers chooses how much debt _b_<sup>_′_</sup> to purchase. In equilibrium, _b_<sup>_′_</sup> = _B_<sup>_′_</sup> .

3. The government makes its default decision _z_ , which determines _y, c,_ and _g_ .

Given this timing, we can reduce the government’s problem choosing _c_ , _g_ , _B_<sup>_′_</sup> , _z_ to solve International bankers are risk neutral with discount factor _β_ so that the bond prices _q_ ( _B_<sup>_′_</sup> _, s_ ) are determined by the probability of default in the next period. There is a continuum of measure one of bankers. For further details please refer to Conesa and Kehoe (2017). The solution for the international bankers’ utility maximization problem implies that if internationalwhere<sup> ̄</sup> _b_ is thebankerslower dothresholdnot lend.and _B_ ̄ifis _B_ the<sup>_′_</sup> _≤_ upper<sup> ̄</sup> _b_ the governmentthreshold anddoesif ̄ _b_ not _< B_ default<sup>_′_</sup> _≤ B_ ̄,eventhe government does not default if international bankers lend. If _B_<sup>_′_</sup> _> B_<sup> ̄</sup> the government defaults even if international bankers lend.

To solve for the equilibrium, we choose the same functional form for the utility function as in Conesa and Kehoe (2017), For the equilibrium definition and further details please refer to Conesa and Kehoe (2017) and Cole and Kehoe (1996, 2000).

The framework identifies thresholds that distinguish three zones of debt that rely on calibrations for each country:

- In the safe zone, the government would not find it optimal to default even if there is a panic in financial markets and it could not roll over its stock of debt. The debt ratio for the countries in the safe zone is relatively small. Hence, the optimal strategy is to be solvent and sacrifice consumption to repay the stock of debt, rather than experiencing an output loss because of an insolvency.

18
- In the crisis zone, the economy is vulnerable to a self-fulfilling crisis and the government’s optimal policy is to default only if it could not roll over its stock of debt. Countries that fall in this zone are susceptible to uncertain shocks. These countries might find themselves in a crisis without even knowing that they are at risk when an uncertain shock were to materialize.

- In the default zone, the interest payments are so large that the government would choose to default even if investors were willing to refinance the stock of debt. Countries in the default zone can be considered vulnerable since they are estimated to face an elevated risk of default in the event of a crisis.

To produce outputs, the model requires a set of calibrated parameters. The model requires an estimate of the probability of a panic (proxied by the subjective default rates identified above<sup>10</sup> ); the maturity composition of public debt;<sup>11</sup> the structural levels of revenues and expenditures; output loss upon a default; and the average yield to maturity.

As an example Cole and Kehoe (1996) illustrates the usefulness of these models in the case of Mexico. Mexico was not facing imminent default right before the 1994-1995 crisis. Yet, public debt was high and the share of short-term debt was high. Mexico had debt levels that were in the crisis zone due to **short maturity bonds** . In 1994, there was political turmoil followed by the assassination of the ruling party’s presidential candidate Luis Donaldo Colosia. Subsequently, Mexico defaulted. Mexico could have been better prepared to address a potential shock by increasing its maturity structure and issuing bonds in local currency. Also, the government might have pursued fiscal consolidation more forcefully if there had been more awareness about the risk of default (Cole and Kehoe, 1996). The Theory index would then have picked up if Mexico was getting close to the default zone and would have coded it as a value of 3.

Figure 12 below illustrates an output for a hypothetical country given a set of calibrated parameters. In this chart, the y-axis expresses the future debt-to-GDP ratio, while the x-axis the current debt-to-GDP ratio. The red line is a 45-degree line. Any value above the red line suggests that government’s optimal policy (black line) is to increase the debt-to-GDP ratio. On the contrary, any value below the red line implies an optimal policy of reducing future debt-to-GDP ratios. In this scenario, the lower threshold is about 70 percent of GDP and it divides the safe zone from the crisis zone. The upper threshold is about 90 percent of GDP and it divides the crisis zone from the default zone. Above this level, debt drops to a value of zero given that the country’s optimal decision is to default even if investors were willing to refinance the stock of debt. Hence, investors will not lend to this country and the future debt level drops to a value of zero.

Finally, once we compute the time varying probability of a panic, we can express the thresholds also as a varying function of time. In Figure 13, we illustrate this for South Africa and Argentina in the sample using a simple 1 period maturity. The upper and risky thresholds change over the sample period. Argentina breaches the threshold by 2018, while South Africa is reaching the upper threshold as it accumulates more debt. Interestingly, the

> 10Due to computational restrictions, we have created a mapping between the theoretical model’s probability of panic and the subjective probability. The probability of panic is 0.4 times the subjective probability.

> 11We obtained maturity rates on non-concessional borrowing for 42 countries from 5 regions using Bloomberg. To increase the sample size to 159 countries, we used the average rate of the 42 countries for each of the 5 regions.

19
**Figure 12:** Growing indebtedness

<!-- Start of picture text --> 100<br>75<br>50<br>25<br>0<br>Current Debt / GDP (%)<br>Future Debt / GDP (%)<br>0 25 50 75 100<br><!-- End of picture text --> upper threshold is declining for Argentina, which is in line with a higher probability of a panic. For South Africa, the threshold has remained fairly constant.

**Figure 13:** Illustrating time varying debt thresholds

<!-- Start of picture text --> ZAF ARG<br>80.0 120.00<br>70.0<br>100.00<br>60.0<br>80.00<br>50.0<br>40.0 60.00<br>30.0<br>40.00<br>20.0<br>20.00<br>10.0 Debt Upper Lower Debt Upper Lower<br>0.0 0.00<br>% GDP % GDP<br>2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020<br><!-- End of picture text -->

# **4 Results and tests**

Apart from using the DVA as a means of uncovering underlying vulnerabilities, one can also test how well it does in terms of predicting defaults. This is an essential part of an early warning system. Does it warn that a country is becoming more exposed to shocks (and hence more vulnerable) and does it indicate whether a country’s default probability is rising? The empirical results are only useful if they provide reasonable signals about actual 20
sovereign defaults.

We compare the DVA against several alternative specifications. These alternative specifications include simple explanatory variables that typically give rise to defaults, to more complex multivariate systems (see as an example Kraay and Nehru (2006)). For the simpler models we use the debt to GDP ratio as a simple indicator of default - the assumption is that higher debt makes a country more vulnerable and hence should be associated with a higher default prediction. We also use a simple interest rate spread (each country’s interest rate relative to the U.S. average interest rate on debt) as an indicator of default. The assumption here is that a higher spread signals liquidity risk and should be associated with a higher default prediction. The last model for comparison includes several of the variables in Kraay and Nehru (2006) that analyse periods of debt distress. The variable list includes short-term external debt, debt service of external public debt relative to exports, foreign currency composition of debt, governance (as a proxy for the non-available CPIA variable) and we also include external financing needs, which equals short term debt plus amortization of debt minus the current account balance.

We use several methodologies to predict defaults. We start by using a simple Logit specification as a first case. There are other methods that can improve on Logit regressions. The data-science literature has made several advances to test whether additional variables in regressions aid in predicting outcomes, or whether they are creating noise or are correlated with other explanatory variables. We apply two of the these methodologies. Thus, for the second regression methodology we use random forests. Random forests build on the Classification and Regression Tree (CART) methodology for prediction. The basic idea is to partition results into smaller groups based on the predictors. Random forests reduce the correlation between trees; the objective is to reduce the variance of each tree by introducing randomness into the tree growing process (or selection of input variables) via bootstrapping (see Breiman (2001)).<sup>12</sup> The third empirical methodology applies a Lasso, equation (15), by means of zeroing out coefficients that do not increase prediction. For the bi-variate equations, Lasso makes very little sense, unless the intercept is better at prediction than the explanatory variables. In the case of the DVA and the longer set of variables for Kraay and Nehru (2006) (which we refer to Classical in the results section), Lasso regressions may help us to eliminate variables that have poor prediction properties. In one sense, instead of using PCA for the generating the Fundamental index, we could have just used a Lasso regression to rid the regression of poor predictors. This would have certainly eliminated several variables. The only reason for doing this is to generate better predictions, but this comes at the cost of losing a descriptive part of the DVA - i.e., to assess where the vulnerabilities might arise. Hence we retain the causal implication of the Fundamental index using PCA.

How do we test our models? We follow the data-science literature by dividing the data into a training and a test data sample. The data set is split using a random break. Our training data consists of 80% of the entire data pool, while our test data is the remaining 20%. We train each of the models using cross-validation methods. This means that we randomly select five data sets from our training data and estimate the regressions on each of those. The reason for doing this is to find the coefficients in the regressions that minimize a loss function (e.g., RMSE’s, ROC, or type 1 or type 2 errors). Once our regressions are

> 12500 trees were selected to minimize the out of bag sample error.

21
”trained” we then analyze how well they predict the test data set.<sup>13</sup> Our dependent variable, historical defaults, is sourced from The Bank of Canada (Beers and Mavalwalla (2017)). A default is defined as a late interest payment, an outright payment default, or when creditors incur losses owing to an extension of maturities or a reduction in rates. Using historical defaults helps us avoid having to define a vulnerability indicator and then trying to predict that. We are interested in predicting when a default occurs in time , i.e., when the 0 flips to a 1 in the sample. We test two cases for the timing of defaults, one where default occurs in period _T_ and data are expressed in period _T −_ 1, and the other where both default and data are contemporaneous. The results do not change much when we vary the timing. Admittedly the variables that lead to default might be different than the variables that keep a country in default. We are not separating out these effects and instead code the default data as equal to 1 when in default and 0 otherwise, and not only 1’s when the default occurs. This is partially related to the default data, where default states tend to be very persistent in the sample of countries.

Furthermore, all our data are stacked. We thus have _t_ annual observations from 2008 until 2020 for _k_ = 65 market access countries. We control for within country specific fixed effects via demeaning.

# **4.1 Econometric specification**

The standard logistic regression (with binary response equal to default (1) and no default (0)) can be expressed as: where _x_ is the input vector (i.e., the four indices from the DVA, or the three alternative models).

The parameters are estimated via maximizing the log-likelihood: An additional constraint is added for the Lasso. Lasso regressions are useful for model selection by shrinking coefficients to zero. As an example, it is possible that the subjective probability estimate tracks the fundamental index over time. Lasso will penalize this and shrink one of the coefficients to zero. The general goal of Lasso is to minimize the sum of squared errors by including a penalty function: where _j_ is the _j_ th variable in the list of _K_ , _t_ is the time index, _y_ is default, and _x_ are the four components of debt in a matrix. The tuning parameter _λ_ controls the amount of

> 13We also run these tests over an in-sample period and a pseudo out of sample period where 2019 is used as the out of sample year.

22
shrinkage. When _λ_ = 0 then no shrinkage occurs and the minimized function boils down to a standard least squares estimate. As _λ_ increases more coefficients are set to zero. Note that there is a bias-variance trade-off in parameter estimation: with high _λ_ the bias increases, while a low _λ_ increases the variance. The four models yield predicted probabilities of default.

In terms of statistical significance of the variables used, only the theoretical index enters significantly in the Logit estimate. The Lasso shrinks the count and fundamental index parameters to zero and indicate that the default probability calculation and theoretical index explain default. The random forest, however, suggest that the fundamental and default probability indices are relatively more important in explaining default. Using an ensemble of methods and pooling estimates may produce a more accurate reflection of default likelihoods. Figure 14 summarizes the ”Gini impurity”, a measure of when to split a tree. The relative values are considered here (not the scale) - the fundamental index is roughly thrice as important as the count index in this example.

**Figure 14:** Variable importance in random forest estimation

<!-- Start of picture text --> Theory Index<br>Probability Index<br>Fundamental Index<br>Count Index<br>0 30 60 90<br>Decrease In Gini<br><!-- End of picture text -->

# **4.2 Prediction**

Once the regressions are trained we can turn to prediction. A standard output is producing confusion matrices. In Appendix F we include confusion matrices for all regression - variable combinations for both the training and test data sets. The column headers are labeled ”no crisis” and ”crisis”. These correspond to actual defaults and no defaults. The row headers are labeled ”no signal” and ”signal” and correspond to the regression’s predictions. Given that each regression will only produce estimated default values (range between 0 23
and 1), we need to determine cut-off values for the classification (i.e., to generate a value of 1, which equals default and a value of 0 otherwise).

To determine the cut-off values from the continuous predictions, we follow Gabriele et al. (2017) (it expands Kaminsky (1999)). This test computes a noise-to-signal ratio to obtain critical cut-offs for when an indicator points to a default. As an example, the objective here would be to find a critical value of the estimated default probabilities that minimizes the noise-to-signal ratio, which is our objective function.

The approach of Gabriele et al. (2017) attempts to minimize the noise-to-signal ratio by varying the cut-off points of an indicator (in this case the estimated probability of default from either the logit, Lasso or random forest regressions). A simple illustration is presented in Table 1. The noise-to-signal ratio is calculated as: _NTSR_ = [ _B/_ ( _B_ + _D_ )] _/_ [1 _−_ ( _C/_ ( _A_ + _C_ ))], where the capitalized letters correspond to the blocks in the table. This test thus attempts to minimize Type 2 errors. An alternative approach defined in Gabriele et al. (2017) (which is used in this paper) is to take the weighted average of both errors, _θ CC_ + _A_<sup>+</sup> (1 _− θ_ ) _B_ + _<u>BD</u>_<sup>,definedasthepolicymaker’slossfunction(</sup><sup>_PMLF_)where</sup><sup>_θ_measuresthe</sup> preference between the two types of errors and is determined endogenously by maximizing _min_ ( _θ_ ; (1 _− θ_ )) _− PMLF_ . The parameter _θ_ is the value that minimizes average loss.

**Table 1:** Noise-to-signal ratio

||Default|No Default|
|---|---|---|
|Signal|A|B(Type II)|
|No Signal|C(Type I)|D|

The cut-offs of the Logit, Lasso and random forest regressions are summarized in Table 3. For the different cut-offs, the missed crisis and the false alarms are calculated (Appendix F). These cut-off values are then used to determine the accuracy of the test case. As an example, the accuracy of the DVA random forrest regression is equal to 72% - i.e., it correctly classified up to 72% of actual defaults. The distinction between the regressions for the different variables are not that large.

**Table 2:** Cut-offs

||Logit|Random Forest|Lasso|
|---|---|---|---|
|DVA|0.58|0.55|0.51|
|Debt|0.58|0.62|0.50|
|Spread|0.62|0.64|0.50|
|Classic|0.52|0.58|0.50|

Making a Type 2 error is not as serious in this case as making a Type 1 error. If the model predicts a crisis and no default ensues then the economic consequences are likely smaller compared to the case where a default occurs and no signal was given. Furthermore, the costs of a type 2 error (i.e., indicating a crisis when none occurs) can be a reflection of the consolidation of governments to avoid a crisis - and hence there is an endogenous anticipation response where the averted crisis due to good fiscal policy would could have resulted in a crisis without the policy intervention. This implies that type 2 errors are not really knowable when a policy action followed a signal.

24
<!-- Start of picture text --> toon<br>ooSoS | ome<br>Lo<br>$ ‘AUC: 65.5%<br>9°BE AUC:. 50.8%<br>o<br>rN AUC:AUC: 61.7958.1%<br>50 100<br>False Positive Percentage<br><!-- End of picture text -->

<!-- Start of picture text --> S toon<br>raooSoSLo |mom<br>a<br>£ ‘AUC: 74.0%<br>fo}Be ; AUC:. 55.5%9<br>mS3 o AUC:AUC: 61.4%55.3%<br>0 50 100<br>False Positive Percentage<br><!-- End of picture text -->

<!-- Start of picture text --> S toon<br>ooSoraSLo |mom<br>a<br>£ ‘AUC: 68.9%<br>fo}Be : AUC:sep 56.1%40<br>mS3 o AUC:AUC: 61.4%58.1%<br>0 50 100<br>False Positive Percentage<br><!-- End of picture text -->
# **5 Discussion**

The approach taken in this methodology combines several elements in the debt-default literature to not only predict possible default, but to also describe sources of vulnerabilities.

This is where the vulnerability index plays an important role. While looking at several single indicators is useful to understand potential vulnerabilities, a country-specific weighted index summarizes a large set of information. Dimension reduction techniques are well known for picking up signals in a large data group.

A key element of this analysis is a description of what is meant by default probability. There are several definitions of default probability that can be applied. However, here we use only two measures. The first relates to the ability of the government to maintain debt on a stable path. The second relates to investor perceptions regarding default. Monitoring both measures reveal interesting details regarding periods of weakened perceptions over and above the government’s own ability. Once the two default estimates diverge then it points to a panic.

Our methodology attempts to incorporate expert judgment in describing defaults. DSAs provide guidance on key thresholds of risk for several indicator variables. We use these thresholds along with historical data to codify judgment. We note that this judgment is subject to revisions as more information and better analytical tools help to bridge several gaps.

A novel addition to default prediction is to incorporate more structural and theoretical models. We utilize a standard general equilibrium framework to compute debt thresholds. This is different compared to empirical relationships between the size of debt and default to define thresholds. In this example, debt thresholds are determined by the maturity structure, the probability of a panic, and the size of the government. These measures provide insights on whether countries will do well to reduce debts to reduce risks. Two thresholds provide three regions of risk. There is a debt to GDP ratio where the risk of a default is very low. At the same time, there is a debt to GDP ratio where default is very likely or imminent. Finally, there is a region between the no default and default thresholds where countries are vulnerable to exogenous shocks. These thresholds are useful from a policy dialogue perspective to identify sources of risks and policies to reduce those risks.

All of these measures are used jointly to predict defaults. We benefit from advances in the econometric literature and test several specifications. In general, the predictions from the models do a reasonable job in predicting both in and out-of-sample defaults. Interestingly, the count index has the weakest explanatory power in predicting defaults.

There are obvious caveats to this methodology. As with any model, the outputs depend on the quality of the inputs. In the exercise described above, three significant sources of risk are not entirely captured due to lack of data: the realization of both implicit or explicit guarantees, the uncovering of hidden debts, and the risks associated with systemic shocks that spill over to countries. In cases where these data are available, the methodology can be appended to revise the results.

It should also be emphasized that the outputs of the debt vulnerability methodology are intended to supplement and support country risk assessments and not replace them. They indicate rising or abating vulnerabilities and analyze where the vulnerability build-up occurs. Consequently, the outputs in this paper tie in with expert opinion using DSAs and 26
other assessments. As such, the DVA can be seen as a screening and filtering system to identify countries that are potentially at risk.

# **6 Conclusion**

This paper describes a methodology for analyzing public debt vulnerability. Four distinct elements make up this framework: (1) A Fundamental Index identifies vulnerable countries by mapping default risk to changes in fundamentals; (2) a Default Probability Index measures the degree of weakened perceptions regarding risk; (3) a Count Index maps expert opinion to debt vulnerability; and (4) a Theory Index estimates debt thresholds that vary by the probability of a panic for each market access country in the study.

The Fundamental Index consists of a large set of variables related to debt vulnerabilities and summarizes risks by looking at available low-frequency data. The Fundamental Index shows rising/abating vulnerabilities by country.

Two default probabilities are calculated to create the Default Probability Index. One uses required primary balances and actual primary balances to produce a debt sustainability default probability related to a government’s ability to finance its debts. The other one uses interest rate spreads to produce measures of defaulting expectations. The two default probabilities track each other for most of the sample, but diverge just before default. The difference between the two probability estimates is a proxy for a probability of a panic - if markets perceive a higher risk than what the fundamentals suggest, then it can lead to an increase in roll-over risk.

Thresholds from the IMF-WBG DSAs are used in the construction of the Count Index. The index measures the proportion of variables of a country that exceed a threshold. Countries that have many variables exceeding their thresholds are more likely to default.

The Theory Index of the framework consists of identifying debt thresholds that are indicative of risky, safe, and default regions. These debt-to-GDP thresholds are time varying as opposed to static thresholds typical of DSAs. They measure an angle of default related to what it ought to be. The theoretical model controls for debt maturity, spreads, and a panic probability to model endogenous default.

The four indices are used as predictors of sovereign default. Three statistical methodologies are applied. A classical logit regression, a LASSO logit regression, and random forests. The latter two methodologies apply machine learning techniques to maximize the prediction aspect of the framework.

The results suggest that the four indices do better than rules of thumb and standard estimates in identifying defaults, with a LASSO type regression yielding the highest level of precision.

This framework is distinct from traditional early-warning systems. In addition to flagging vulnerable countries, outputs help identify sources of vulnerability and hence provide a policy angle to reduce risks. The theoretical model, as an example, illustrates that debtto-GDP thresholds are unique to countries as opposed to applying a single threshold to all countries that fall within an income group. In addition, the Fundamental Index decomposes the variability of vulnerability into a set of variables.

The framework can be improved with access to more data. Some data are not publicly available for all countries - such as implicit and explicit government guarantees, which 27
are serious source of risk for countries. As argued in MTI-WBG (2021) debt data transparency is a fundamental agenda for improving our (debtors, creditors, researchers, and international organizations) ability for evaluating risks of debt crises.

28