metadata: authors: 0: author: Herrara, Santiago
1: author: Garcia, Conrado
count: Peru,Mexico,Venezuela, Republica Bolivariana de,Ecuador,Chile,Colombia,Brazil,Argentina
docna: 0: docna: User's guide to an early warning system
            for macroeconomic vulnerability in Latin American countries
docty: Policy Research Working Paper
subtopic: Economic Conditions and Volatility,Poverty Impact Evaluation,Poverty Monitoring & Analysis,International Trade and Trade Rules,Inflation,Macroeconomic Management,Poverty Assessment,Poverty Diagnostics,Poverty Lines,Small Area Estimation Poverty Mapping
teratopic: Education
abstracts: cdata!: The authors develop an early warning
            system for macroeconomic vulnerability for several Latin
            American countries, drawing on the work of Kaminsky,
            Lizondo, and Reinhart (1997) and Kaminsky (1988). They build
            a composite leading indicator that signals macroeconomic
            vulnerability, showing that, historically, crises tend to
            happen in certain "vulnerable " situations.
            Interested mainly in providing an operational tool, the
            authors use a different approach to the problem than
            Kaminsky did. First, they use fewer variables to generate
            the signals. Then, after the variables are aggregated, a
            signal is issued, depending on the behavior of the composite
            index. (Kaminsky's procedure was to generate signals
            with each variable and then aggregate them.) Their results
            are satisfactory both statistically and operationally.
            Statistically, Type I and II errors are smaller than those
            reported in previous papers. Operationally, this system of
            leading indicators is less costly to maintain, given fewer
            variables-which are widely available and reported with
            timeliness. The authors tested the models'
            out-of-sample predictive ability on crises that occurred
            after the first stage of their project was finished:
            Colombia (September 1998), Brazil (January 1999), and
            Ecuador (February 1999). In all cases the models correctly
            anticipated the speculative attacks. Moreover, Mexico's
            models, estimated with information available two years
            before the 1994 crisis, show that these signaling devices
            would have been useful for signaling the macroeconomic
            vulnerability before December 1994.
geo_reg_key: 119228,517191
repnme: repnme: User's guide to an early warning
            system for macroeconomic vulnerability in Latin American countries
display_title: User's guide to an early
            warning system for macroeconomic vulnerability in Latin
            American countries
listing_relative_url: /research/1999/11/439628/users-guide-early-warning-system-macroeconomic-vulnerability-latin-american-countries
topicv3: Macroeconomic Vulnerability and Debt
            m1326057 1104,Economic and Social Mobility m1326259 1152,Inflation m1326709 0870F4727ACED1CF8525815C00471347,Macro-Vulnerability m1326711 1148,Trade m1326924 698
publishtoextweb_dt: 2010-07-01T00:00:00Z
historic_topic: Education
origu: Economic Policy Sector (LCSPE)
no_of_pages: 22
model_extractions: I POLICY RESEARCH WORKING PAPER 2233

# Summary findings

Herrera and Garcia develop an early warning system for macroeconomic vulnerability for several Latin American countries, drawing on the work of Kaminsky, Lizondo, and Reinhart (1997) and Kaminsky (1988).

They build a composite leading indicator that signals macroeconomic vulnerability, showing that, historically, crises tend to happen in certain "vulnerable" situations.

Interested mainly in providing an operational tool, Herrera and Garcia use a different approach to the problem than Kaminsky did. First, they use fewer variables to generate the signals. Then, after the variables are aggregated, a signal is issued, depending on the behavior of the composite index. (Kaminsky's procedure was to generate signals with each variable and then aggregate them.) Their results are satisfactory both statistically and operationally.

Statistically, Type I and Type II errors are smaller than those reported in previous papers.

Operationally, this system of leading indicators is less costly to maintain, given fewer variables - which are widely available and reported with timeliness.

Herrera and Garcia tested the models' out-of-sample predictive ability on crises that occurred after the first stage of their project was finished: Colombia (September 1998), Brazil (January 1999), and Ecuador (February 1999). In all cases the models correctly anticipated the speculative attacks.

Moreover, Mexico's models, estimated with information available two years before the 1994 crisis, show that these signaling devices would have been useful for signaling the macroeconomic vulnerability before December 1994.

- - This paper a product of the Economic Policy Sector Unit, Latin America and the Caribbean Region is part of a larger effort in the region to build tools that policymakers can use to prevent crises. Copies of the paper are available free from the World Bank, 1818 H Street NW, Washington, DC 20433. Please contact Conrado Garcia, room 18-146, telephone 202458-7969, fax 202-522-2119, email address cgarciacorado@worldbank.org. Policy Research Working Papers are also posted on the Web at www.worldbank.org/research/workingpapers. Santiago Herrera may be contacted at sherrera Cworldbank.org. November 1999. (17 pages)

_The Policy Research Working Paper Sees disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polisbed. The papers carry thve names of the autbors and sbould be cited accordingly. The findings, interpretation$, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the view of the World Bank, its Executive Directors, or the countries they represent._ Produced by the Policy Research Dissemination Center
Santiago Herrera Conrado GarcIa

# User's Guide to an Early Warning System for Macroeconomic Vulnerability in Latin American Countries'

_It turns out that an eerie type of chaos can lurk just behind a facade of order-and yet, deep inside the chaos lurks an even eerier type of order._ Douglas Hofstadter

# I. Introduction

The object of this paper is to develop an early warning system (EWS) of a country's macroeconomic fragility. The idea is to have an instrument that helps policymakers identify and anticipate situations in which crises are more likely to happen, in the vein of the leading indicators literature. Previous work has already been done in this area, mainly by Kaminsky, Lizondo and Reinhart (1997), and Kaminsky (1998). There are three main differences between our paper and previous ones upon which it is based: a) The main interest is to have an operational tool . The ultimate objective is to build the simplest possible EWS to be updated monthly at the lowest feasible cost. Data i Paper presented in the XVII Latin American Meeting of the Econometric Society, in Cancun, August, 1999.

We thank Sara Calvo, Norman Loayza, Guillermo Perry, and Carmen Reinhart for insightful comments.
2 availability on a timely basis was a crucial determinant of the set of variables employed. Our work in this stage was based on papers by Eichengreen, Rose and Wyploz<sup>2</sup> , Kaminsky, Lizondo and Reinhart<sup>3</sup> (KLR), Kaminsky<sup>4</sup> and the _IMF_<sup>_5_</sup> _._ b) The aggregation method of the individual leading indicators into a composite inceex; and the way this index is used as a signaling device. Kaminsky (1998) construcls a composite index by aggregating signals of the different indicators. We take a diffeient approach in aggregating the variables and then generating the signals depending on the behavior of the composite index. The reason for adopting this strategy is that we beli -eve that for a crisis to take place the set of leading indicators must jointly drift in the sa.mrte direction over some period of time.

c) The exclusive focus on LAC countries, estimating the models and showing details on a country by country basis that other papers do not present due to the large number of countries in their pane] s.

With this in mind, the paper has four sections besides this introduction: the first one identifies crises periods, or periods of unusual market volatility, based on monithly information for the period from 1980 to June,1998. The second section presents the leading indicators and the aggregation procedures that will allow identifying regularii:ics in pre-crisis periods We build four different signal-generating mechanisms, three of which have previously been used and a fourth one that is novel in this paper. The tLird chapter evaluates the alternative filtering mechanisms for each country. showing tlhat, even though different criteria lead to different model selection outcomes, the new methtod proposed in this paper leads to the minimum Type I error. The recent crisis in BraziL, Ecuador and Colombia are used to test the out of sample predictive ability of the mdcl with good results. The fourth and last section summarizes the results, stressing the usefulness of these types of leading indicators methodology but it also points out its rri ain limitations.

# II. Periods of Unusual Market Volatility

This section seeks to determine the "crises" periods, in order to study the behavior of :)ur leading indicators prior to their occurrence. In line with most of the literature on l his topic, we define an Index of Speculative Pressure (ISP) as follows: _ISP =_ A% exchange rate + A% interest rates - A% intemational reserves All the variables (expressed in monthly percentage changes) were standardized to h~.vmean zero and unit vaiiance. We avoided the issue of weighting differently each of 1he

> 2 Eichengreen, B., A. Rose **and** C. Wyploz (1996) "Contagious currency crises: first tests". Scandinaviar Journal of Economics, 98, 4, 463-484.

> 3 Kaminsky, G., S. Lizondo and C. Reinhart (1997)"Leading indicators of currency crises" Policy Reseacbe Working Paper 1852, The World Bank

> 4Kaminsky, G, (1998) "Currency and banking crises: A composite leading indicator" Board of Governors of the Federal Reserve System Working Paper.

5IMF (1998) World Econornic Outlook, Chapter IV, "Financial crises: characteristics and indicators of vulnerability'.
3 variables, but Eichengreen's sensitivity analysis to different weighting patterns indicate there should be no major changes expected.

A crisis is defined as a period in which ISP t > _u_ + 1.5a (where u is the sample mean and cr the standard deviation of the ISP). Table 1 summarizes the dates when the ISP surpasses the threshold. For countries experiencing hyperinflation (Argentina, Brazil and Peru), different thresholds were calculated both for these episodes and for more stable periods<sup>6</sup> . A total of 64 crises result and will be used in the analysis.

|Perio|Table 1<br>ds of Excessive Market Volatility in Several LAC Countries<br>1980:01-1998:04|
|---|---|
|Argentina|83:12; 89:04-05; 89:12; 90:02; 91:12; 92:11; 95:03|
|Brazil|82:09; 87:01-02; 89:06; 89:11-12:90:01-02; 90:11;91:09; 94:04-06; 95:03;<br>97:10|
|Chile|82:06-11; 83:03; 84:10-11; 85:07; 89:04-06; 92:04; 98:01|
|Colombia|84:01-04; 85:01-05; 86:09; 90:10: 92:08; 95:08; 97:09|
|Ecuador|81:12; 82:05; 83:03; 84:01; 85:12; 86:08; 88:07-08; 89:07; 90:01; 91:01; 92:05;<br>93:06; 95:02|
|Mexico|82:02-06; 82:12; 85:07; 87:12; 90:03; 94:04; 94:12; 95:01; 95:03; 95:10|
|Peru|81:01; 83:07; 87:10-12; 88:03-09; 90:03;92:04-10; 93:01|
|Venezuela|84:02; 86:12; 89:02-03; 90:07; 94:05; 95:12; 96:04; 98:01|

# III. Leading Indicators of Crisis and Signal-Generating Mechanisms

There is a wide set of options regarding which variables to use and how to use them. Concerning the variables to use, the IMF (WEO, ch. IV) narrowed the list to 3: M2/Reserves, real domestic credit growth, and the real exchange rate. We will also include, alternatively, the inflation rate due to the consistency of this variable in papers by A. Demirguc-Kunt and E. Detragiache<sup>7</sup> on determinants of banking crises.

Following the IMF procedure, we construct an index of macroeconomic vulnerability (IMV) with these variables, standardized to have mean zero and unit variance, circumventing the issue of weighting the individual indicators differently.

IMV= REER + DCG + M2/R + ,z REER Real effective exchange rate DCG Domestic credit Growth in real terms M2/R = M2/International reserves ;T = Inflation

> 6 For Argentina 2 sub-samples were used: 1980:01-91:06 and 91:07-98:03; for Brazil, two sub-samples 80:01-94:06 and 94:07-98:03. And for Peru, 3 sub-samples were used 1980:01-1988:07, 88:08-91:08, 91:09-98:04.

7 Asli Demirguc-Kunt and Enrica Detragiache (1997) "Banking Crises around the World: Are there Common Threads?"
4 This aggregation method differs from Kaminsky's since the signals are extracted from the behavior of our composite index, while in her case each individual variable generates signals that are then aggregated into the composite index. Our aggregation procedure assumes that the leading variables drift more or less in the same direction, or have a common element in their behavior prior to the crisis. If this is not the case, it will not be a good indicator. For example, if the real exchange rate appreciates (increases) but there is a contraction in domestic credit growth, our IMV may not change and hence no signal will be issued.

Once the composite index of macroeconomic vulnerability (IMV) is built, its sinmple evolution is not informative enough about a potential risk situation. Thresholds have., to be used in assessing when the IMV has reached an "anomalous" level.

We applied four transformations or filters to the IMV to generate signals: detrending the variable with respect to its long-run level (defined as the Hodrick-Prescott trend), the variable in levels (no iFilter), detrending the variable with respect to a short-run (6 mos.) moving average and thLe residuals of an ARIMA model fitted to the IMV. The first three (or variants) have been used in other papers while the ARIMA residuals approach is novel, and produced the lowest Type I error (probability of not anticipating the crisis). Initially we focus on the IMV as described above, to make our sample period as comparable as possible to that of other papers. In a second stage stock market prices in real terms are included as an additional leading variable, but the sample period is shortened to 1986-199,8 due to availability of consistent information from the IFC on this variable for most of the sample countries.

# A. Deviations from trend model (DT model)

This model uses deviations from a long run trend to generate signals. To determine wriat the long run trend of a series is, the IMF used a 3-year moving average, but we prefer-ed the Hodrick-Prescott filter since the first procedure induces autocorrelation to he detrended series. This is an undesirable feature since once the mechanism sends a sigi a;, it will tend to produce signals in successive periods. On the other hand, the HodrilkPrescott filter induces spurious cycle behavior when applied to non-stationary data so _we_ have to be aware of this phenomenon induced by the detrending procedure.

The deviations from the trend for each of the variables were standardized and aggregaled to build the Index of M/Iacro Vulnerability (IMV). The index was computed alternatively, including inflation or not, and based on simple Granger causality tests between thl, se indexes and the crises with lags up to 24months, the best index was chosen. Inflation was an informative variable only in the cases of Argentina and Mexico. This IMV signal i a crisis when it surpasses a threshold determined by the mean plus 1.5 standard deviatiors.

8 Cogley, T. and J. Nason (1995) "Effects of the Hodrick-Prescott filter on trend and difference stationary time series: Implications for business cycle research", Journal of Economic Dynamics and Control, 19, pp.253-2<sup>7</sup> 8.
_5_ A characteristic of all the computed IMVs is that their volatilities change through time. Most of the indexes were particularly volatile from the mid eighties until the early nineties, so the standard deviations that were used were computed from the conditional variance of the series estimated by a Generalized Autoregressive Conditional Heteroskedastic (GARCH) model<sup>9</sup> . The feature of these types of models is that the variance of the IMV is taken to be an ARMA process that is estimated simultaneously with the mean of the series. The GARCH (p,q) model that was used for all the countries wasi°: IMVt= ao+aiIMVti+et et= vt _h_ v is white noise with I-=1 _q_ 2 p and _h, = a+ aie, + h,-i_ With the conditional standard deviations, the threshold was computed and the signaling device is complete.

B. The levels model (simple model) An implicit assumption in the previous model is that temporary departures of variables from their trend provide information regarding future crises but the trend itself is not an informative variable about the macroeconomic vulnerability of a country. Because this might imply discarding useful information we computed the IMV index with the variables in levels. In this case, inflation was an informative variable in all cases, except for Chile and Colombia The thresholds for the IMV were constructed with the conditional standard deviations of GARCHI models for each country's IMV<sup>12</sup> , and the signaling mechanism is ready. The signal is flashed if the IMV exceeds the mean plus 1.5 standard deviations.

> 9 Developed by Engle, R. (1982) "Autoregressive conditional heteroskedasticity with estimates of the variance of United Kingdom inflation", Econometrica, 50, 987-1007 and Bollerslev, T. "Generalized autoregressive conditional heteroskedasticity", Journal of Econometrics, 31, 307-327.

> '° The order of p and q varied with each country as follows: Argentina 6,1; Brazil 3,1; Chile 6,1; Colombia 3,0; Ecuador 6,0; Mexico 3,1; Peru 1,1; Venezuela 3,0.

" The IMV was computed as the sum of standardized variables (without detrending) including inflation or not, and simple Granger causality tests were used to decide which was the best index for each country. 12 The procedure is identical to that described in the DT model. In most cases the whole sample period was split into sub-samples implying different models. The sub-samples and the orders of p and q are: Argentina 1980-1988 (1,1); 1989-1991 (2,1); 1992-1998 (0,2). Brazil 1980-1985 (1,1), 1986-1991 (5,0), 1992-1998 (1,1); Chile 1980-1986 (2,1) 1987-1998 (1,1). Colombia 1980-1998 (1,0). Ecuador 1980-1985 (1,1), 1986-1994 (1,1), 1995-1998 (1,1). Mexico 1980-1983 (2,1), 1984-1995:02 (1,0), 1995:03-1998 (1,2). Peru 1980-87 (1,0) 1988-91 (1,0), 1992-98 (1,0). Venezuela 1980-82 (1,1) 1983-94 (1,1), 1995-98 (1,1)
6

# C. The chartist model (moving average model)

The previous approach may be subject to criticism on the grounds that, since there are several break-points in the sample that are known to the researcher ex-post, it is diffic.ult from an operational viLewpoin **t** o know at every point in time if there has been a change in the mean or variance models. To avoid this problem, a simple approach coimcmnly used by financial markets practitioners is to compare the variable (IMV) with its mo),irng average. Based on Granger causality tests, the 6-month moving average was selected. A signal is flashed when the IMV exceeds the 6-month moving average.

# D. The ARIMA residuals model

Finally, one could hypothesize that a crisis is more likely to happen when the set of leading indicators are behaving "strangely". The "normal" or regular behavio: is described by an ARIMA model for the IMV, so the residuals summarize the deviations from the "normal" behavior. In theory, the residuals must be a white noise process and have mean zero. But some residuals will randomly be positive. We're interested in mrore permanent positive deviations, so we constructed a moving average of the residuals and a signal was generated when this statistic exceeds zero. Conceptually, this filter is similar to applying any detrending methodology, except that it considers a richer set of information besides the trend in the variable's history.

# IV. **Evaluation Criteria and Results**

With the four models described above (deviations from trend, simple, chartist and ARIMA residuals) different signals were generated. With their empirical clistributions, as well as those of the crises, we evaluated the models using a 24-month window prior to each crisis<sup>13</sup> . Contemporaneous signals were not counted because they are not ':ad signals and they are not leading variables either. Similarly if a crisis occurs withi l4 months of another one, they are counted as one episode.

# A. Evaluation criteria

Our evaluation of each model is based on four 4 statistics; the sizes of Type I and Typt II errors, the noise to signal ratio, and the probability that a crisis occurs given that a sigiial was produced. A short description of each one and details on the computation follow: We borrow a very useful table from Kaminsky-Lizondo and Reinhart (KLR) (1997) to visualize the different criteria:

**13** 24 months is the most common size for the window. The results don't change dramatically if an 18month window is used, but a noticeable improvement is achieved with the wider one.
7 Table 2 Possible Scenarios of Signals and Crisis

||Crisis|No crisis|
|---|---|---|
|Signal issued|A|B|
|No signal issued|C|D|

# 1. Types I and II errors.

- If H 0 = Crisis occurs H a = No crisis occurs

Size of Type I error = P [reject Ho/ Ho is true]= Probability of not anticipating a crisis Size of Type I error = C/(A+C) Given that the null hypothesis is true (crisis occurs), the perfect signaling device would send 24 signals (with a 24-month window). The size of Type I error was computed as 1- the number of signals prior to each crisis (as a ratio of 24).

Size of Type II error = P[not rejecting Ho/Ho is false]= Probability of sending a false signal.

Size of Type II error = B/(B+D) For every non-crisis period, the signals in the 24 month period prior to the each crisis are counted and expressed as a ratio of the sum of the no signal-no crises (good signals) and the signal-no crisis (false signal), given that no crisis occurred.

Since both types of errors are undesirable, a criterion that we will use is to select the model that minimizes the sum of both. An alternative rule will be to choose the model that minimizes Type I error, given that it can be argued that it is more costly not anticipating the extreme risk situations.

# 2. The Noise/Signal ratio (NSR)

The Noise/Signal ratio (NSR) measures the false signals (size of Type II error) as a ratio of the good signals issued (1- size of Type I error) The selection rule is to pick the model that minimizes the NSR for each country. This was the criterion followed by Kaminsky and KLR in their ranking of the different variables.

# 3. Probability of a crisis given that a signal was issued P(C/S)

Given that models generate different signals, an alternative criterion is to select the model that maximizes the probability of a crisis occurring given that a signal was issued.
8

# B. Results

The four models (deviations from trend, the simple, chartist and ARIMA residuals) were used for each of the 8 countries. For each model the Type I and II errors, the Noise to Signal Ratio (NSR), and the Probability of having a crisis given that a signal is issued (P(C/S)) were computed, and the results are summarized in Table 3. Unconditional probabilities of crises occurrence are also presented for each country, to allow gauriing the effectiveness of the models (TABLE 3 A). Two sample periods are presented because in a latter part of the paper we'll work with a shorter sarnple period due to information availabilily of certain variables.

|**Argentina**|**0.86**|0.14|0.99|0,61|0,84|0.29|1.80|044|0.77|036|1.57|054|0.55|0.37|0<br>.83|0.tS|
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|**Brazil**|0.75|0.13|0.51|**0.84**|0as9|0.16|0.50|0.2|9.42|014|0.25|0,93|031|0.2T|0.39|80**.**|
|**C4lfe**|0,72|0.15|0.52|:0.5|0;60|D.l1|0Q27|0.88|0.58|033|080|067|038|**040**|0<br>,65|**0.74**|
|**Cotomib**|**ia**0.60|f13|0.33|0. .9|0|3590.12|<br>0.29|0.98|0.63|01 7|0.46|0.56|0.41|0.16|Ct2i|0.62|
|**Ecuador**|0.41|0.49|0.82|1 0.13|0.76|D0,|0,36|0.91|0,19|0.49|0.60|0.89|0.52|0.24|Ct49|0O.0|
|**Mexico**|**0T0**|0.09|0.31|0_S5_|0.52|_0.08_|0t17|0.91|0.51|0.43|_0.69_|_00i_|0.44|0.32|C57|0.79|
|**Peru**|0.83|0.36|2.13|0.46|0.75|0.12|0.47|0.79|0.63|0D41|1.12|0.57|0.40|0.50|C.83|0.62|
|**Venezue**|**la0.11**|0.1|0,t1|0.,32|-0.74|0.05|0.19|0.9|0.58|0.50|1.1|0.70|A0.49|.62|1.20|0Q,7|

Differenced Arima residuals model.

Table 3A Unconditional Crisis Probabilities in LAC Countries in Different Sample Periods

||1980-1998|:1986-1998|
|---|---|---|
|Argentina|.037|.049|
|Brazil|.065|.092|
|Chile|.060|.028|
|Colombia|.047|.035|
|Ecuador|.065|.063|
|Mexico|.056|.049|
|Peru|.060|.077|
|Venezuela|.042|.056|
9 In general we observe (Table 3) high Type I errors and low Type II errors. However, compared to Kaminsky's and KLR's results (for individual variables)'<sup>4</sup> , the first are lower and the second higher. The NSR is lower here, and the P(C/S) is very high, especially when compared to the unconditional probability.

Table 4 presents the selection of the best model for each of the countries according to the 4 criteria: minimizing the sum of the Type I and II errors, minimizing Type I error, minimizing the NSR and maximizing P(C/S). In most of the cases, the simple model performs the best, except in Argentina where the deviations from trend model outperforms the rest, and Brazil where the chartist model is selected. In Ecuador, the chartist model is preferred because of the low Type I error.

Table 4 Model Selection According to Different Evaluation Citeria (IMV period 1980-1998)

||**Mini.Type I**<br>+Type**II errors**|**Min. TypeI **<br>**error**|**Min..**<br>**Noise/S`igal**|**Mai.P(C/S)**|
|---|---|---|---|---|
|Argentina|Residuals|Residuals|Residuals|**Residuals**|
|Brazil|Residuals, Chartist|Residuals|Chartist|Chartist|
|Chile|Simaple|Residuals|Simple|Simple|
|Colombia|Residuals|Residuals|Residuals, Simple|<br>Residuals|
|Ecuador|Residuals|Chartist|Simple|Residuals,<br>Simple|
|Mexico|Simple|Residuals|Simple|Simple|
|Peru|Residuals, Simple|Residuals|Simple|Simple|
|Venezuela|Simple|Residuals|Simple|Simple|

One can think that the Type I error is too high, especially in the case of Argentina and Ecuador, where it exceeds .5. These models can be calibrated to reduce this error with a combination of two procedures: first, by reducing the number of crisis limiting the analysis to the extreme cases where it was difficult not having any signals sent; and second, by altering the signal generating mechanism in order to have more signals issued. The first option implies changing the definition of crisis by setting a higher threshold for the Index of Speculative Pressure (ISP): instead of working with the mean plus 1.5 standard deviations we'll work with 3 standard deviations. And in order to have more signals sent we can lower the threshold for the IMV: instead of working with the mean plus 1.5 standard deviations we'll work with one standard deviation.

We followed this approach with the largest economies Argentina, Brazil and Mexico and obtained significant improvement with respect to the models shown in Table 4 in the case

> 14 KLR and Kaminsky report type I and II errors, N/S and P(C/S) for individual variables. Kaminsky does not report these statistics for the composite indicator, but shows other forecasting evaluation statistics.
10 of Argentina, marginal improvement in the case of Mexico and none whatsoever in the case of Brazil. Table _5_ summarizes the relevant statistics for the best models selected.

|'1~~|~~~~h1|||||
|---|---|---|---|---|---|
|Argentina'|0.03,7|0.55|37|0.83|0.65|
|<br>Brazil2|<br>0S065$|v 00<br>t0<br>R4!|<br>it0<br>$014|0s0000<br>>t<br>;<br>_0250_<br>D|l92<br> <br>4093|
|<br>Chile<sup>3</sup>|.<br>0.060|<br>.<br> <br>0.60|<br> <br>.<br>0.11|<br> <br> <br>_._<br> <br>0.27|<br> <br>.<br>0.88|
|_C3li,bia'_|004|0.|0W16|0.28|02|
|<br>Ecuador'|_0.065_|0.52|0.24|0.49|0.90|
|Mexico<sup>3</sup>|00043||_0.1_<br>0|_.3_<br>0.877||
|PeruX|0.060|0.75|0.12|0.47|0.79|

Regarding the anticipation of crises, the selected models do fairly well signaling in advance the extreme risk situations. However there are big differences across countries in the distribution of the signals within the 24-month period prior to the crisis (Table 6). In Brazil, Chile and Peru the signals are evenly distributed, with half the signals taking place in the 12 months prior to the crisis. The least anticipation is registered in Ecuador, Mexico and Venezuela, where close to 50% of the signals took place within 6 months of the crisis. The most delayed response happens in Argentina, where in the year prior to the crisis only 40% of the signals are issued.

||=.~~~~|ro<br>**toC**<br>**a**|<br>**=rii**|||
|---|---|---|---|---|---|
|**Arenin**|**1.-**<br>0X:i-d|.<br>**2.1**<br>**.**<br>**'.**|**..**||**100**|
|Brazil|18|36|54|90|100|
|Chile0|<br>t;0<br>2Xt0|_35_|56<br>-<br>X<br>G-<br>|<br>N0Nt78|t1oo00,|
|<br>Colombia|<br> <br> <br>19|<br>37|<br> <br> <br>61|<br>85|100|
|Ecuado'|\ugrrEtf0jwaSA<br>29'|El;0<br>'<ijj51|95|99|10000000040|
|<br>Mexico|<br>25|<br> <br>49|71|90|100|
|eezul|22|31<br>4|50<br>775||100|
|<br>Venezuela|<br>31|<br> <br>46|<br> <br>74|95|100|
11

- C. Introducing the Stock Market Price information 1986:01-1998:04

In the previous section we saw how the best models were, in general, the ARIMA residuals model and the simple one. In this section we'll add the stock market price index in real terms to the list of leading indicators. Our modified IMV is IMVEQ(including equity prices) defined as: IMVEQ= REER + DCG + M2/R + ;z-+ EQ Where EQ = stock market price in local currency deflated by the CPI.

The cost of introducing this variable is loosing valuable information from the early eighties, since our source of consistent information for most countries (IFC) reports it since 198615.

We conclude that stock market prices are an informative variable of vulnerability based on two tests. First, simple Granger causality tests of the IMV with stock market prices (labeled IMVEQ) and without them and the Index of Speculative Pressure (ISP), point at the value of the information content of stock prices. And second, the comparison between the four alternative models (DT, Simple, Chartist and Residuals) with and without stock market prices (resulting in a total of 8 models) shows that including stock market prices is a superior strategy than the alternative (Appendix 1 shows the four statistics for each model). Table 7 summarizes the model selection information; recall the unconditional probabilities of crisis during this sample period (Table 3A) to see the gains in using these models. The signal distribution in the 24-month window prior to the crisis (Table 8) shows that the timing signal-crisis does not change significantly when stock prices are included and the sample period is shortened.

Table 7- Model Selection According to D _Eifferrt,_ Evaluatii Criteria (IMVE~~Q peid **ilo ti** -:8 -----

||**Miii. Typ**<br>**I**<br>**+Type II etrrors**|**e**<br>**Miii. Type I**<br>**rror**|Miii~........<br>-<br>**.**<br>**N-:se-Sigal**|.......<br>**Ma..**<br>..<br>**IS**|
|---|---|---|---|---|
|Argentina|Simple|Residuals|DT|DT|
|<br>Brazil|<br>Chartist|Chartist|Chartist|Chartist|
|Chile|Simple|Simple|Simple|Simple|
|Colombia|<br>Residuals|<br>Residuals|<br>DT|DT|
|Ecuador|Residuals|Residuals|Residuals|Simple|
|Mexico|Residuals, Simple|Residuals|Simple|Simple|
|Peru|<br>Simple|Residuals|<br>Simple|simple|
|Venezuela|<br>Simple|Residuals|<br>Simple|Simple|

_15_ For Peru the information exists since 1990, and for Ecuador since 1995. Hence Ecuador's IMV does not include this information.
12

|Argentina<br>N|<br>16<br>ta72|34|||100|
|---|---|---|---|---|---|
|Brazil|21|43|57|91|100|
|Chile|14|28|51|74|100|
|Colombia|16|36|67|81|100|
|Ecuar &|20|37|83|98|100|
|Mexico|15|36|57|94|100|
|Venezuela|29|50|85|100|100|

# D. Plenty Out of Sample Testing: (Brazil 99 Ecuador 99 Colombia 98 Mexico 94)

Three crises (Brazil, Ecuador and Colombia) have taken place in our group of countries since we finished the first draft of our paper (July,1998). In this section we show how :he models predicted the vulnerabilities in each of them. Additionally, we'll see how ilhis methodology would have predicted the Tequila crisis. The methodology is based o:l estimating an ARIMA model for the IMVEQ, using a five year rolling sample, ; nd beginning two years prior to each crisis. A signal will be generated in each of the 24 months of the window prior to the crisis.

In the case of the Brazilian crisis (Jan/99), we estimated an ARIMA model for Ihhe IMVEQ beginning in January 97, with monthly information since Jan 92. For Febru:Lrv 97, the same procedure is repeated with information since February 92, and so on. 1 he signal is flashed when the six-month moving average exceeds zero. Graph I summariz es the evolution of the leading indicator derived from the Arima-residuals model, show:ing that 10 signals were issued in the 24-month window prior to the crisis. Graph 2 is -he leading indicator derived from the chartist model, showing that 13 signals were issuLe(d prior to the crisis.

Exactly the same procedure was followed to study the crisis that took place in Ecuador in February,1999. Graphs 3 and 4 summarize the signals sent by the Arima residual moJ:LeL and the chartist one (chosen simply because of its computational ease): the first cne signaled 23 times and the second one 20 times in the 24-month pre-crisis period. For lihe Colombian crisis of September/98 the same approach is taken and the results , rc summarized in Graphs 5 and 6. The residuals model flashed 14 signals while the chart sil one did it 10 times.
13 Finally, the exercise is replicated to see how the models would have<sup>anticipated the</sup> Mexican crisis of December 94. Graphs 7 and<sup>8 show that theses tools would have been</sup> useful in signaling the vulnerability experienced by the Mexican<sup>economy. In the 24-</sup> month window prior to the crisis, the residuals model issued<sup>20 signals, while the chartist</sup> one flashed them 14 times.

Graph 2 <!-- Start of picture text --> Graph 1 Graph 2<br>Sig nals of Macro Vulnerability Preceding Brazil's Crisis Sig nal of Macro Vulnerability Preceding Brazil's Crisis<br>Arina residuals nodel Chartist Model<br>0.2- 2<br>0.1 1<br>00 ~~~~~~~~~~~~~~~~~~~~~0<br>0.0 -~~~~~~~~~~~~~~-<br>- 1 - X -<br>-2-<br>-0.3 -4-<br>97:01 97:07 97:01 9807<br>(a sig nal is flashed when the indicator is positive) (a sig nal is flashed if the dashed line exteeds the solid one)<br>Graph 3 Graph 4<br>Signals of Macro Vulnerability Preceding Ecuador's Crisis Signals of Macro Vulnerability Preceding Ecuador's Crisis<br>Arimna residuals model 0.0 Chartist model<br>0.30<br>0.25 -0.<br>0.20<br>0.15 -1.0<br>0.10<br>0.05 - -1. 5<br>0.00 -<br>-0. 907 9801 987-9:1-2.0- 7' 8: ... 8:O...690<br>97:07 98:01 98:07 99:01 -2 0 97:07 98:01 98:07 99:01<br>(a signal is flashed if it is positive) (a signal is flashed when the dashed line exceeds the solid one<br><!-- End of picture text --> <!-- Start of picture text --> (a signal is flashed when the dashed line exceeds the solid one<br><!-- End of picture text -->
14 <!-- Start of picture text --> Graph 5<br>Sig nals of Macro Vulnerability Preceding Colombian Crisis<br>Arima residuals model<br>0.24<br>0.0<br>-0.2 _<br>-0.4<br>97:01 97:07 98:01 98:07<br>(a signal is flashed when the indicator is positive)<br><!-- End of picture text --> Graph 6 Signals of **lVbcro** Vulnerability Preceding Colombian Ciisi s Chartist model <!-- Start of picture text --> 0.0<br>-0.5<br>-1.0<br>-1.5 -<br>97:01 97:07 98:01 913:07<br>(a signal is flashed when the dashed line exceE!Js the solid one)<br><!-- End of picture text --> <!-- Start of picture text --> Graph 7<br>Sig nals of Macro Vulnerability Preceding Me)acan Crisis<br>Arima residuals rnodel<br>0.2 -2<br>0.0<br>-0.2-<br>-0.4-<br>-0.6 I9:130<br>93:01 93:07 94:01 94:07<br><!-- End of picture text --> <!-- Start of picture text --> Graph 8<br>Signals of Nbcro Vulnerability Preceding IlJxican Cnis;is<br>Chartisi: model<br>0<br>40 40<br>93:01 93:07 94:0 94:07<br><!-- End of picture text --> (a signal is flashedwhen the indicator is positive) (a signal is flashed when the dashed line exceeds **tie** solid one)

# V. Conclusions and Agenda for Future Developments of this Project

The Early Warning System models presented for the group of Latin American countrie s do a good job in anticip.ating vulnerability to crises. The limited set of variables clear' y indicate periods when crises are more likely to happen. However, Type I and II errors aie still high, though smaller than previous papers have found This can be because tie models need improvenment or simply because crisis are events that are inherently unpredictable. At this point, with the tools we used it's impossible to tell. We' [e inclined to believe it's more the first reason. Therefore, a mechanical application of tl e signals issued by these models can lead to a false sense of security or to unwarranted nervousness at some poinits. The country analyst's criteria is in no moment substituted by these tools.

Several modifications are in line within this project. Among the most important is thie inclusion of external interest rates. We begun this work and this variable seems more
15 important for some countries (Mexico) than for others. Including the price of certain commodities, like oil for Venezuela and Ecuador should also improve results. Similarly, incorporating information on the state of the real sector is crucial. We have begun this work with excellent results for most countries (Appendix Tables 2 and 3). However, increasing the dimension of the vector of leading indicators is done at a cost of additional complexity. To deal with the dimensionality problem, a common unobserved component model could be estimated. There are two options on this front: a. If this component is continuous and enters linearly in the postulated relationships, then it can be estimated via a Kalman filtering technique as proposed by Stock and Watson (1991)16. On the other hand if this component has a non linear behavior and is discontinuous, switching from one regime to another, then Hamilton's (1989)17 estimation technique can be used, as has already been done for the EMS countries'<sup>8</sup> .

> **16** Stock, J. and M. Watson (1991) "A probability model of the coincident economic indicators", in Leading Economic Indicators, ed. K. Lahiri and G. Moore. Cambridge University Press. "Hamilton, J. (1989) "A new approach to the economic analysis of non-stationary time series and the business cycle", Econometrica 57 pp.357-84.

> 18 Maria Soledad Martinez "A regime-switching approach to the study of speculative attacks: a focus on EMS crises". UC Berkeley, Nov/97.
16

|Ar9ent|na**0.7**|**8**U.07|0.34|0.8|80.50|0.24|0.48|0.66|0.51|0.54|<br>1,09|0.65|0.49|0.30|<br>0.59|<br>0 77|
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|**Brailt**|0,.2|0.42|<br>2.31|<br>0.58|0.5|70.24|<br>0.56|<br>0.90|0.4|7 0.24|0.;|0.9|30.44|0.38|0.68|0,90|
|Chile|0.7|5 0.23|<br>0.90|<br>0.57|0.4|00.13|0.22|0,84|0.5|6-0.41|<br>0.92|0,5|60.47|0.51|<br>0.96|<br>0.55|
|**Colob**|0.7|50.00|<br>00|1.0|00.52|0.33|0.70|||0.21|0.0<br>,|0.4|0.3|50.10|<br>0.16|<br>091|
|Ecuad|or"||||0.7|90.11|<br>0.49|0.87|||||0.41|0.26|0.45|<br>0,83|
|tMexic|o 0.6|70.04|0.11|0.|06|02|0.05|<br>0.9|0g.5|1 G.A|<br>0.85|0.62|0|0.30|0.50|0.76|
|**iPeru**|t0.9|0- 0.27|<br>2.78|0.29|0.42|0.13.|<br>0.22|0.60|0.77|.0.52|<br>2.29|0025|0.52|0.54|<br>1.13|0.31|
|**Verez**|**uela**<br>0.5|<br> <br>20|00.205|70.|0..0.|04|;|0 .95' 0|.5<br>033|:0.|01|0a.8|0:4|2 02|10.2<br>0.4|3<br>0.39|

2 Differenced Arima residual model in the case ofBrazil, Die and Peru. In the case ofEcuador the comparison is made between the Arima Residual Model and the simple model, both of them without equity prices.

__ __

|**Argen**|**tina**<br>0.7|<br>8 0.07|0.34|0.88|0.7|0 0.14|<br>0.47|<br>0.86|0.5|9 0.53|1.27|0.67 0.5|2 0.30 0.63|t.31|
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|Brazil|0.8|2 0.42|2.31|0.58|0.7|2 0.33|<br>_1.1_|0.84|0.3|5 0.76|U1.1|0,840.5|6 042<br>1.02|(.84|
|**Chile**.|0.75.|0.23.|090|0.57|0.51|0.02|<br>0.04|<br>0.97|.0.65|<br>0|1.70|0.430.64|0.50 1.38|**0.48**|
|**Colmb**|**ia 0.75**|**0.00**|<br>0.00|1.00|0.54<br>- -|<br>0.0|<br>0.10|_O_.95|0O0|0.47|<br>1.17|0.75 .4|0.120.23|(.C9|
|Ecuad|or":||||0.8|9 0.11|<br>0.95|<br>0.43|0.5|2 0.61|1.27|0.44 .0.5|7 0.39 0.93|056|
|**Mexic**|**o**0.6|79004|0.11|<br>0.9|3 0 .63|0.05|<br>0,14|<br>0.91|0.5|6 041|M 0.94|0.66 0.4|4 0.41 2.73|F0'0|
|Peru|0.9|0 0.27|2.78|<br>0.2|9 0.2|9 0.13|<br>0.18|0.69|0.8|0 0.52|2.56|0.18 0. 19|.0.43<br>3.53|_040_|
|**Venez**|**uela**<br>0.8|<br>5 0.20|0.57|0.86|0O|R.52<br>0.09<br>.|<br>0.20|<br>0.91|053|.0.-|0.80|0.82 0.3|2 0.10 3.14|0 95|

± Venezuela industrial production index was avaiable from 1989. Ecuador IP was available only from 1990.
17

|**Summary,**<br>**,f**|<br>**Best Modtls**|**Append**<br>**Statbtiesfra**|**ix: Table 3**<br>**ch**<br>**Cotry**|**(including IP 1986- 18**|
|---|---|---|---|---|
||Unconditninal.<br>Probability|Type I elror|TypeX1errr|Noise/Sipna ratio<br>P(CtS|
|Argentina'|.049|0.70|0.14|0.47<br>0.86|
|<br>Brazil|9**2**|_0.47_|0.24|_0.45_<br>0.93|
|Chile'|.028|_0.51_|0.02|0.04<br>0.97|
|Colombia'|_.035_|0.54|0.05|0.10<br>0.95|
|Ecuador<sup>3</sup>|.063|0.41|0.26|0.45<br>0.83|
|Mexico'|.049|0.63|_0.05_|0.14<br>0.91|
|Perul|.077|0.29|0.13|**0.18**<br>0.69|
|Venezuela<sup>3</sup>|.056|0.32|0.10|0.14<br>0.97|

Notes: I/ Simple Modef 2/ Chartist Model 3/ Residuals Model. In the case of Ecuador the Residuals model gave as good results as the Simple model.
**Policy Research Working Paper Series**

|**Title**|**Author**|**Date**|**Contact**<br>**for paper**|
|---|---|---|---|
|WPS2214 Trade Policy and Market Access<br>Issues for Developing Countries:<br>Implications for the Millennium<br>Round|Constantine Michalopou|los October 1999|L. Tabada<br>36896|
|WPS2215 Implementation<br>of Uruguay Round<br>Commitments: The Development<br>Challenge|J. Michael Finger<br>Philip Schuler|October 1999|L. Tabada<br>36896|
|WPS2216 Corruption and Trade Tariffs, or<br>a Case for Uniform Tariffs|Roberta Gatti|October 1999|R. Gatti<br>38735|
|WPS2217 Border, Border, Wide and Far,<br>How We Wonder What You Are|David C. Parsley<br>Shang-Jin Wei|November 1999|H. Sladovich<br>37698|
|WPS2218 Who Avoids and Who Escapes<br>Poverty during the Transition:<br>Evidence from Polish Panel Data,<br>1993-96|Wlodzimierz Okrasa|November 1999|S. Fallon<br>38009|
|WPS2219 The Effect of the United States'<br>Granting Most Favored Nation<br>Status to Vietnam|Emiko Fukase<br>Will Martin|November 1999|L. Tabada<br>36896|
|WPS2220 A Quantitative Evaluation of<br>Vietnam's Accession to the ASEAN<br>Free Trade Area|Emiko Fukase<br>Will Martin|November 1999|L.Tabada<br>36896|
|WPS2221 The Dynamics of Poverty and the<br>Effectiveness of Poland's Safety<br>Net (1 993-96)|Wlodzimierz Okrasa|November 1999|S. Fallon<br>38009|
|WPS2222 Labor Market Integration in the<br>Presence of Social Capital|Maurice Schiff|November 1999|L. Tabada<br>36896|
|WPS2223 Integrated Financial Supervision:<br>Lessons from Northern European<br>Experience|Michael Taylor<br>Alex Fleming|November 1999|S. Torres<br>39012|
|WPS2224 Growth Forecasts Using Time Series<br>and Growth Models|Aart Kraay<br>George Monokroussos|November 1999|R. Bonfield<br>31248|
|WPS2225 How Did Highly Indebted Poor<br>Countries Become Highly Indebted?<br>Reviewing Two Decades of Debt Relie|William Easterly<br>f|November 1999|K. Labrie<br>31001|
|WPS2226 Money, Politics, and a Future for the<br>International Financial System|Michael Klein|November 1999|M. Salehi<br>37157|
**Policy Research Working Paper Series**

|**Title**|**Author**|**Date**|**Contact**<br>**for paper**|
|---|---|---|---|
|WPS2227 The Sri L.ankan Unemployment<br>Problem Revisited|Martin Rama|November 1999|S. Fallon<br>38009|
|WPS2228 Fiscal Contingency Planning for<br>Banking Crises|Patrick Honohan|November 1999|A. Yaptenco<br>38526|
|WPS2229 Do School Facilities Matter? The Case<br>of the Peruvian Social Fund<br>(FONCODES)|Christina Paxson<br>Norbert Schady|November 1999|N. Schady<br>88247|
|WPS2230 Bankruptcy Organization through<br>Markets: Auction-Based Creditor<br>Ordering by Reducing Debts<br>(ACCORD)|David Hauszh<br>S. Ramachandran|November 1999|L. Tsang<br>80516|
|WPS2231 What's Behind Mercosur's Common<br>External Tariff|Marcelo Olarreaga<br>Isidro Soloaga<br>L. Alan Winlers|November 1999|L. Tabada<br>35555|
|WPS2232 Market Access Advances and<br>Retreats: The Uruguay Round and<br>Beyond|J. Michael Finger<br>Ludger Schiknecht|November 1999|L. Tabada<br>35555|

> _If_ j F _I'_