metadata: authors: 0: author: Calice,Pietro
count: Middle East and North Africa
docna: 0: docna: Predicting bank insolvency in the Middle
            East and North Africa
docty: Policy Research Working Paper
subtopic: Legal Institutions of the Market Economy,Hazard Risk Management,Financial Crisis Management & Restructuring,Social Risk Management,Bankruptcy and Resolution of Financial Distress,Economic Insecurity,Disaster Management
teratopic: Finance and Financial Sector Development,Macroeconomics and Economic Growth
abstracts: cdata!: This paper uses a panel of annual
            observations for 198 banks in 19 Middle East and North
            Africa countries over 2001-12 to develop an early warning
            system for forecasting bank insolvency based on a
            multivariate logistic regression framework. The results show
            that the traditional CAMEL  indicators are significant
            predictors of bank insolvency in the region. The predictive
            power of the model, both in-sample and out-of-sample, is
            reasonably good, as measured by the receiver operating
            characteristic curve. The findings of the paper suggest that
            banking supervision in the Middle East and North Africa
            could be strengthened by introducing a fundamentals-based,
            off-site monitoring system to assess the soundness of
            financial institutions.
repnme: repnme: Predicting bank insolvency in the Middle
            East and North Africa
display_title: Predicting bank insolvency in the
            Middle East and North Africa
listing_relative_url: /research/2014/07/19875103/predicting-bank-insolvency-middle-east-north-africa
topicv3: Bank Resolution and Crisis Management
            m1326972 319,Early Warning Systems m1327419 570,Bankruptcy and Liquidation m1327467 1277,Financial Institutions m1505015 3797,Credit Risk m1505083 3885
publishtoextweb_dt: 2014-07-01T00:00:00Z
historic_topic: Finance and Financial Sector Development,Macroeconomics and Economic Growth
no_of_pages: 19
author_keys: 0: author_key: 1297294
model_extractions: WPS6969 Policy Research Working Paper 6969 Predicting Bank Insolvency in the Middle East and North Africa _Pietro Calice_ Finance and Markets Global Practice July 2014
Policy Research Working Paper 6969

# **Abstract**

This paper uses a panel of annual observations for 198 banks in 19 Middle East and North Africa countries over 2001–12 to develop an early warning system for forecasting bank insolvency based on a multivariate logistic regression framework. The results show that the traditional CAMEL indicators are significant predictors of bank insolvency in the region. The predictive power of the model, both in-sample and out-of-sample, is reasonably good, as measured by the receiver operating characteristic curve. The findings of the paper suggest that banking supervision in the Middle East and North Africa could be strengthened by introducing a fundamentals-based, off-site monitoring system to assess the soundness of financial institutions.

This paper is a product of theFinance and Markets Global Practice.. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The author may be contacted at pcalice@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **Predicting Bank Insolvency in the Middle East and North Africa**

Pietro Calice<sup>†</sup> JEL Classification: G21, N25, N27 Keywords: Early Warning System, Financial stability, Middle East and North Africa

> † Pietro Calice is a Senior Financial Sector Specialist within the Finance & Markets Global Practice of the World Bank Group. The author would like to thank Simon Bell, Subika Farazi, Erik Feyen, Roberto Rocha and Gabriel Sensenbrenner and for their comments.
# **Predicting Bank Insolvency in the Middle East and North Africa**

## **1. Introduction**

The financial crises experienced in recent decades prompted efforts to develop early warning systems (EWSs) which could help identify the factors underlying the occurrence of a crisis (Bell and Pain, 2000; Gaytán and Johnson, 2002; Berg et al., 2004; Demirgüç-Kunt and Detragiache, 2005). The recent global financial crisis has intensified these efforts, with academics and policy makers increasingly focusing attention on developing indicators and methodologies which can assist in the timely identification of signs of distress (Ghosh et al., 2009; Barrell et al., 2010; Babecký et al.; 2013). This paper attempts to identify the factors that affect the risk of bank insolvency in the Middle East and North Africa (MENA) as a way to provide a fundamentalsbased surveillance tool which can serve as an early warning device.

The impact of the global financial crisis on banking systems and banks in MENA has highlighted the importance of differentiating across countries and among financial institutions. While the region avoided systemic banking distress, the crisis had a stronger impact on countries in the Gulf Cooperation Council (GCC), where financial systems were more globally integrated and banks more overextended (Rocha et al., 2011). Most importantly, the global financial crisis revealed significant differences in the resilience of individual banks. This is largely the result of the quality of management, corporate governance arrangements and banks’ business models, sometimes mitigated by preemptive policy actions taken by country authorities. In many cases, knowledge about the specific features underpinning individual financial institutions’ health has been crucial for identifying problems and informing policy action. This suggests that an EWS model built on bank-specific fundamentals can provide a useful monitoring tool, helping supervisors detect vulnerable financial institutions and take preemptive steps to aid ailing financial institutions.

This paper builds on and expands the extant literature on bank default. The latter mostly focuses on testing a wide variety of accounting-based variables on the probability of bank default in discriminant analyses and logistic regression frameworks. The assumption is that insolvent banks present _ex ante_ weaker fundamentals than solvent banks. The literature on individual bank default draws heavily on the CAMEL rating system, a widely used microprudential supervisory tool to monitor the performance and soundness of financial institutions based on five categories: Capital adequacy, Asset quality, Management capacity, Earnings power, and Liquidity position.

Most of these studies focus on the US, where the data are most readily available (e.g. Thompson, 1991; Whalen, 1991; Cole and Gunther, 1995, 1998; Estrella et al., 2000; Wheelock and Wilson, 2000; and most recently Torna, 2010; Jin et al., 2011; Cole and White, 2012). Other studies take a cross-country perspective and analyze bank fragility in the context of the European Union (e.g., Poghosyan and Čihák, 2011; Betz et al., 2013), Latin America (e.g., Gonzalez-Hermosillo et al., 1997; 1999; Arena, 2008) or Eastern Europe (e.g., Maechler et al., 2007; Männasoo and Mayes, 2009). While there emerges a great deal of heterogeneity in terms of consistency of the predictive power of CAMEL variables, mainly due to the different definitions of bank distress employed and to the difficulty of comparing accounting rules across countries, bank-level indicators are in general found to be significant determinants of bank fragility. To the best of our 2
knowledge, none of the relevant existing studies focus on the MENA region. This paper attempts to fill this gap.

To derive an EWS for monitoring bank insolvency risk in the MENA region, this paper estimates a pooled logistic regression on a sample of 198 banks in 19 countries during the period 2001-12. We account for bank default by employing a definition of _technical_ insolvency, given the absence of actual bank defaults over the sample period, and use standard proxies for the CAMEL indicators as explanatory variables. Not surprisingly, we find that traditional CAMEL indicators are significant predictors of bank insolvency in MENA, in line with previous research in different geographical contexts. Large banks with more capital, better asset quality and higher managerial quality are less likely to experience a _technical_ insolvency during the subsequent two-year period. Our results are robust to the inclusion of additional potentially relevant control variables and to different estimation methodologies. Equally important, results show that our EWS performs reasonably well in terms of both the number of crises correctly called and the number of false alarms produced, as measured by the receiver operating characteristic curve (ROC), which accounts for the unknown shape of the policy maker’s utility function.

The structure of the paper is as follows. Section 2 describes the econometric methodology and the data used for the study. Section 3 presents the results on the determinants of bank insolvency in MENA and discusses their robustness. Section 4 discusses the predictive power of the model. Section 5 concludes the paper and offers some policy recommendations.

# **2. Methodology and data**

# _Methodology_

As is customary in the literature on bank default, we estimate a pooled binary logit model to evaluate the impact of various financial indicators on the probability of bank insolvency. In a logit model, the probability of bank insolvency is assumed to be a function of a vector of potential explanatory variables. Let Pt,i denote a dummy variable that takes value of one if at time _t_ bank _i_ is insolvent and zero otherwise. Let β be the vector of parameters to be estimated, and F�β′xt,i� the cumulative probability distribution function, assumed to be logistic. Then the log-likelihood function of the model that must be maximized is: It must be noticed that while the signs of the coefficients can be easily interpreted as representing an increasing or a decreasing effect on insolvency probability, their values are not as immediate to interpret. As Eq. (1) shows, the coefficients on xt,i reflect the impact of a change in the correspondent explanatory variable on lnPt,i/(1 −Pt,i), not on Pt,i, with the magnitude of the impact depending on the slope of the cumulative distribution function evaluated at β′xt,i. Therefore, the magnitude of the change depends on the initial value of the variables and their coefficients. Finally, as we are interested in developing a model which can help predict the occurrence of bank insolvency, all our explanatory variables are lagged by two years to capture 3
past effects of these variables on the probability of bank insolvency.<sup>1</sup> This approach also helps deal with potential endogeneity of regressors.

# _Data_

The paper uses annual data for 19 MENA countries—Algeria, Bahrain, Djibouti, Egypt, Iran, Iraq, Jordan, Kuwait, Lebanon, Libya, Morocco, Oman, Qatar, Saudi Arabia, Syria, Tunisia, UAE, West Bank and Gaza and Yemen—over the period 2000-12.

The main source of data is the Bankscope database, from which we extract financial information on 320 banks classified as commercial banks, savings banks, cooperative banks, real estate and mortgage banks, and Islamic banks. Due to a large number of missing entries, the initial sample was reduced to 198 banks in the econometric analysis. To minimize the loss of information, we focused on Bankscope’s consolidation code “ _Institutions_ ”, which includes both consolidated and unconsolidated data. We were nonetheless left with an unbalanced sample, reflecting different country data coverage and varied degree of financial development of the countries in the sample. Bank-level information is complemented by a number of control variables drawn from various sources. Appendix 1 presents a detailed list of variables and related sources.

The identification of bank insolvency in MENA is a very challenging exercise. First, systemic banking problems in the region are a rare event. According to the widely used Laeven and Valencia (2012) dataset, there have been only 10 crisis episodes in our sample economies over the period 1970-2012, and none in our sample period. Second, actual bank failures are also rare in MENA. The Bankscope database reports only two cases of receivership in our sample of banks over 2000-2012 and no episode of bankruptcy or liquidation. We therefore resort to a _technical_ definition of bank insolvency or _balance-sheet insolvency_ . In line with the existing literature (e.g., Hannan and Hanweck, 1988; Boyd et al, 1993), we define bank insolvency as a state where ( _car_ + _roa_ ) ≤ 0, where _car_ is the bank’s capital-asset ratio and _roa_ its return on assets. Using this strategy, we identify 14 insolvency events for 10 banks in 6 countries (see Table 1).

We use two-year lagged bank-level financial indicators to generate determinants of bank insolvency in the MENA region. In particular, our explanatory variables are selected based on the exiting literature, supervisory practices and data availability. Indicators from individual banks’ income statements and balance sheets are first identified to account for the categories included in the CAMEL rating system. We then check for additional potential determinants such as those related to bank ownership and business model, the institutional and regulatory framework, banking market structure, and the macroeconomic environment. A full list of variables with indication of the relative source is reported in Appendix 1.

The first covariate we use is the ratio of equity to total loans as a measure of bank capitalization. We use this indicator as opposed to the more standard equity to total assets ratio to avoid potential endogeneity problems, given that our dependent variable is constructed using the latter

> 1 Two-year lags are common in off-site surveillance models employed by supervisors. For example, since 1994 the Federal Reserve uses a EWS known as SEER or System to Estimate Examination Ratings to monitor the condition and performance of state member banks. The model estimates the probability that a bank will fail or become critically undercapitalized within the next two years.

4
variable. Nevertheless, the measure of the extent of leverage using loans instead of assets provides a sensible indication of the bank’s buffer stock which can serve as a cushion to absorb losses since banking crises typically involve shock to the loan portfolio and MENA banks’ main risk is lending risk. Regulatory measures of capital adequacy such as the Tier I capital ratio or the total capital ratio are not available on a consistent basis for the whole sample. Higher levels of capital act as a buffer against financial losses and are expected to reduce the risk of bank insolvency. We therefore expect a negative sign for this variable.

As regards the second CAMEL dimension, asset quality, we use two covariates: an _ex-ante_ measure of credit risk given by the ratio of total loans to total assets and a measure of _ex-post_ asset risk represented by the ratio of loan loss provisions to net interest income. An asset mix characterized by a high proportion of loans relative to other earning assets can potentially indicate a riskier asset portfolio hence we expect a positive sign for this variable. On the other hand, we do not have any prior as to the sign on the coefficient for our second measure of credit risk. High provisioning may reflect high nonperforming loans (NPLs) and therefore may be positively associated with higher insolvency risk. But high provisioning may also reflect prudent behavior if a bank decides to boost precautionary reserves rather than distribute profits. A more standard indicator of asset quality, the NPL ratio, is not used as it is not available for a large majority of the banks included in our sample.

Bank management quality, the third CAMEL dimension, is proxied by the cost to income ratio. Higher values for this indicator are expected to signal relatively poor managerial quality and therefore we expect a positive sign for the related coefficient. To measure bank earnings, our fourth CAMEL dimension, we use the return on average equity (ROAE). We expect a higher ROAE to be negatively associated with bank insolvency hence we anticipate a negative sign for this covariate.

The final dimension of the CAMEL ratings system, liquidity, is represented by the ratio of liquid assets to deposits and short-term funding. Low liquid assets compared to short-term sources of funding signal a stretched liquidity position which can ultimately jeopardize the bank’s solvency. Hence we expect a negative sign for our liquidity indicator.

In addition to standard CAMEL indicators, we also consider bank size, proxied by the log of total assets, as a potential determinant of bank insolvency. This accounts for the fact that larger banks may be better able to diversify their loan portfolio, both sectorally and geographically, thus reducing asset risk and ultimately the probability of insolvency. For various robustness checks, we add a number of additional independent variables to control for differences in bank ownership and business models, the institutional and regulatory environment, banking market structure and macroeconomic environment.

Table 2 provides a basic analysis of the main determinants of bank insolvency in MENA. It presents average values for our covariates in the year preceding the insolvency event and the mean test for differences in bank fundamentals between insolvent and solvent banks. With respect to capitalization, insolvent banks present a lower ratio of equity to loans, denoting the inability of leveraged banks to absorb negative shocks. Regarding credit risk, insolvent banks are characterized by a higher ratio of loans to assets and a higher ratio of loan loss provisions to net interest income than solvent institutions, reflecting the fact that not only high lending but also 5
bad lending characterizes insolvent banks. Results for the provisioning ratio, however, do not show statistical differences, probably reflecting accounting problems related to lax standards for loan classification and loan loss provisioning. Regarding managerial quality, insolvent banks show a higher cost to income ratio, lending support to the view that efficiency matters for solvency. However, managerial quality is not significantly different between solvent and insolvent banks. In addition, insolvent banks are characterized by low profitability, which makes them less able to increase their capital base and improve their viability. Finally, insolvent banks show lower liquidity ratios than solvent banks in the year before the onset of a crisis event, and are significantly smaller than solvent institutions. We now turn to a more formal analysis.

# **3. Regression results**

# _Main findings_

The results of our baseline regression analysis are shown in column (1) of Table 3. In line with economic theory, bank insolvency in MENA is negatively associated with capitalization levels, suggesting that banks which have a low leverage are less likely to experience insolvency in the subsequent two-year period. Similarly, the probability of incurring insolvency is inversely related to asset quality. Assuming that both a higher ratio of loans to total assets and higher provisions relative to net interest income imply a riskier profile, the positive sign for these two variables indicate that the probability of bank insolvency is influenced by the composition of the balance sheet and a deterioration of the loan portfolio, respectively.<sup>2</sup> We also find evidence that managerial quality as proxied by the cost-to-income ratio is a significant factor for bank insolvency in MENA. A low cost base and the ability to control expenses relative to revenues indicate a better likelihood of preventing bank insolvency. Size too comes out as a significant predictor of bank insolvency. The coefficient of our proxy for bank size, the logarithm of total assets, has a negative sign as expected, suggesting that actual or perceived size-related diversification benefits matter for viability. This is also consistent with the “too-big-too-fail” hypothesis.

On the other hand, bottom-line profitability as measured by the ROAE does not come out significant in our baseline estimation. Similarly, our proxy for liquidity risk, the ratio of liquid assets to deposits and short-term funding, is not significantly associated with the probability of bank insolvency. This is not surprising, as our objective is to identify the sources of insolvency over a two-year window whereas liquidity problems can turn into solvency problems very quickly. Moreover, bank liquidity varies substantially over time, while our indicator provides the liquidity position of the bank on the last day of financial reporting.<sup>3</sup>

# _Sensitivity analysis_

We run several robustness tests to assess the validity of our results. We first check whether results are robust to the ownership structure of the banks in our sample. This is not a trivial issue

> 2 We experimented with an alternative _ex post_ measure of asset quality, the ratio of loan loss provision to total loans, and found similar results. 3 We ran our baseline regression with an alternative liquidity risk variable, the loan to deposit ratio, and there were no qualitative differences.

6
in the MENA region, which is characterized by high presence of government-owned institutions, which account for 19 percent of banks in our sample. Government ownership can increase insolvency risk given weak banking skills, weak governance structures, unstable business models and overall misaligned incentives. Cross-country evidence has consistently pointed to a higher share of government ownership resulting in higher banking fragility (La Porta et al., 2002; Barth et al., 2004). This result has been confirmed for MENA banks by Farazi et al., (2011), who find that state-owned banks exhibit a higher share of NPLs than private banks. To control for bank ownership, we add to our baseline estimation a dummy variable which takes value of one if the bank is owned by the government and zero otherwise. The results, which are reported in column (2) of Table 3, corroborate our findings for our baseline specification, suggesting that our main results are not driven by government ownership.<sup>4</sup> In the same vein, we check whether our baseline results are driven by public ownership. Half of the banks classified as insolvent in our sample are publicly listed on stock exchanges. On the one hand, incentive problems arising from the separation of ownership and control become more severe when ownership is more dispersed, therefore increasing risk-taking behavior and ultimately insolvency risk (Jensen and Meckling, 1976). On the other hand, a bank whose shares are publicly traded is supposed to be positively affected by market discipline mechanisms, hence a negative relationship with insolvency risk (Bliss and Flannery, 2002). To test whether public ownership has an independent effect on bank insolvency risk in our sample, we augment our baseline specification with a dummy variable that equals one if the bank is listed in a stock market and zero otherwise. Our baseline results remain unaffected (Table 3, column (3)).

We then control for differences in the macroeconomic environment among individual MENA economies. We are primarily interested in estimating a EWS focusing on risks in individual banks. However, the macroeconomic environment plays a determinant role in the overall performance and soundness of the banking sector (Demirgüç-Kunt and Detragiache, 1998; Kaminsky and Reinhart, 1999; Borio and Drehmann, 2009). Country-specific macroeconomic indicators identify imbalances in the economy and control for conjunctural variation in the business cycle which may impact the banking system as a whole. It is not clear whether to expect macroeconomic variables to have an impact on the probability of bank insolvency. On the one hand, our dependent variable measures a state of _technical_ insolvency as opposed to _actual_ default. This may well be solely the outcome of bank-specific factors (bad management, for example, other things being equal) rather than country-specific macroeconomic factors. On the other hand, many banks in our sample have operations in other MENA countries and therefore the impact of country-specific macroeconomic variables may be limited. To control for the macroeconomic environment, we add three macroeconomic variables to our baseline specification: real GDP growth, inflation and nominal exchange rate depreciation. The results, presented in column (4) of Table 3, show that these variables do not have a significant impact on the probability of bank insolvency.

Next we check for the structural characteristics of the banking system, particularly financial deepening and market concentration. According to most studies (e.g. Demirgüç-Kunt and

> 4 Our results do not necessarily contradict those of Farazi et al., (2011) as we are interested in estimating leading indicators of bank insolvency in MENA whereas the latter analyze the (contemporaneous) determinants of bank performance.

7
Detragiache, 1998), financial deepening is associated with increased financial vulnerability. On the other hand, the expected impact of concentrated banking markets on insolvency risk is ambiguous. Market power may contribute to boost bank profits and strengthen capitalization thus reducing the risk of insolvency (Marcus, 1984; Chan et al., 1986; Keeley, 1990) yet more concentrated banking systems may lead to higher interest rates in the economy and make more problematic for borrowers to pay back their debt thus increasing the risk of bank instability (Boyd and De Nicoló, 2005). Introducing the M2 to GDP ratio as a proxy for financial deepening and the market share held by the three largest banks as a measure of concentration does not change the qualitative findings of our baseline specification (Table 3, column (5)).<sup>5</sup> Another check consists of controlling for the institutional environment. Countries with higher supervisory capacity are likely to show less risk of insolvency because problems are addressed at an early stage. The quality of bank supervision, in terms of compliance with international supervisory practices, can be approximated by the GDP per capita (Čihák and Tieman, 2008). The estimation results do not change (Table 3, column (6)). We also account for the quality of governance in the country by using the Worldwide Governance Indicators. These include six governance measures (voice and accountability, political stability, government effectiveness, regulatory quality, rule of law, and control for corruption) that we average into a single index per country. Again, our results for our baseline specification are not affected (Table 3, column (7)).<sup>6</sup> A final check consists of controlling for the impact of repeated incidence of bank insolvency. Banks that are technically insolvent often require extraordinary support from the government and/or shareholders even if they still meet minimum prudential requirements. They often struggle to improve their soundness and gain back the confidence of depositors and customers. In our model, this results in repeated observations for three of our ten insolvent banks. When the repeated observations are excluded (column (8), Table 3), the results of our baseline specification still hold, suggesting that our findings are not driven by the repetitive insolvency events.<sup>7</sup> All in all, the sensitivity tests performed show that our baseline model performs reasonably well. The main advantage of this model is that it uses widely available accounting-based CAMEL indicators which are helpful in predicting _technical_ insolvency in MENA. The statistical significance of these variables does not change much when other potentially relevant control variables are included or when a different estimation methodology is used.

# **4. In-sample and out-of-sample performance of the model**

It is customary to assess the predictive performance of EWSs by looking at the percentage of observations and insolvencies correctly called and the percentage of false alarms. This requires the specification of a cut-off or threshold probability above which the predicted probability can be interpreted as sending a signal of a pending insolvency.

> 5 Another popular measure of financial deepening, credit to the private sector as a share of GDP, is not appropriate for the MENA region, characterized by a large fraction of credit to the public sector, both government and stateowned enterprises. Hence we resort to M2 to GDP ratio.

> 6 We also experimented with the six governance indicators individually but found no significant impact. Results are available upon request.

> 7 In other words, we estimate the logit model where repeated observations are dropped.

8
The key issue to be solved is what the ‘‘optimal’’ threshold level is. This is not trivial from a policymaker’s perspective. The difficulties arise because there is a trade-off between the “truepositive rate” (TPR) or “sensitivity”, i.e. the probability of signaling an insolvency event when the insolvency actually occurs, and the “false-positive rate” (FPR) or “1 - specificity”, i.e. the probability of signaling an insolvency event when there is no insolvency. The perfect signal would be one such that TPR equals 100 percent and FPR equals 0 percent. There is, however, a trade-off between the two which depends on the value chosen for the threshold. Setting the value of the threshold above which an insolvency event will be signaled at too low a level would trigger a high TPR (correctly classified insolvencies) but also a high FPR (too many false alarms). Conversely, setting too high a value would trigger a low TPR (large number of not called insolvencies) and a low FPR (low number of false alarms).

Setting the right value of the threshold depends on the shape of the policy maker’s utility function. If, for example, letting a bank go burst is very costly compared to the expected cost of implementing a policy measure, for example a recapitalization, the policymaker would be relatively averse to low TPR. If, on the other hand, the expected cost of implementing an intervention is instead very high relative to the expected benefit, then the policymaker would be relatively averse to high FPR.

Unfortunately, the utility function of the policy maker is not known. Moreover, it is likely that it varies over time and across countries. A way to account for the importance of the policy maker’s preferences without being forced to impose a specific functional form common to all countries is the use of the ROC curve.<sup>8</sup> The latter is the mapping between TPR and FPR for all possible thresholds. The ROC curve has, therefore, the nice property of providing a measure of the quality of the signal over the full range of possible utility functions. In particular, the area under the ROC curve (AUC) represents the relative forecasting success for true-positives reduced by the proportion of false-positives generated for any given forecasting threshold. An AUC of 50 percent indicates prediction no better than chance, while an AUC of 100 percent indicates perfect prediction.

To assess the predictive power of our EWS, we calculate both the AUC statistic for the fitted model (in-sample) and the AUC measure out-of-sample. For the latter, we conduct the validation test on the sub-period 2008-2012, which includes three insolvency episodes: two in Egypt and Kuwait in 2008, and one in Tunisia in 2010.<sup>9</sup> For this purpose, we estimated the baseline model over the sub-period ending in 2007 (training dataset) and calculated the AUC for the period 2008-2012.

Results show that our EWS estimated for the whole data set captures around 93.79 percent of the AUC. Similar results are obtained for the fitted model applied to the training dataset (AUC is 93.51 percent), whereas when applied to the validation data set (out-of-sample), the AUC is 92.06 percent. Overall, then, we conclude that our EWS for forecasting bank insolvency in MENA displays reasonably good predictive power both in-sample and out-of-sample.

> 8 See Hsieh and Turnbull (1996) for a general discussion of the ROC and Drehmann and Juselius (2013) for an application to banking crises.

> 9 Insolvency observations are, however, five as there are two cases of repeated insolvency.

9
# **5. Concluding remarks**

This paper developed an EWS for predicting individual bank insolvency in the MENA region, using bank-level indicators of vulnerability. Our findings indicate that small banks with low capitalization, low asset quality and high cost to income ratios are more likely to be insolvent over the next two years. Our results are robust to the inclusion of additional control variables and to different estimation methodologies. The predictive power of the EWS as measured by both insample and out-of-sample performance in terms of AUC is reasonably good.

It should be noted that given the definition of insolvency employed in the analysis, our EWS is generally able to predict the occurrence of an insolvency event only when the latter originates from widespread losses on the asset side of a bank’s balance sheet, which leads to a progressive deterioration of the bank’s fundamentals. Insolvency events, however, can also originate from the liability side of a bank’s balance sheet, such as a traditional bank run or a wholesale bank run, as the recent crisis showed on a large scale. In these cases, it would be useful to investigate the determinants of bank fragility in MENA within a structural credit risk model using marketbased measures of bank vulnerability such as credit default swaps or default risk measures implied by stock prices. In the same vein, it would be interesting to take a systemic risk perspective in the analysis of the sources of bank vulnerability in MENA to capture the risk of contagion. This implies capturing co-dependence of individual bank risk. We leave these topics to further research.

Our findings have important policy implications at a time when implementing effective riskbased supervision remains a challenge in the MENA region. In particular, our results highlight the usefulness of a simple off-site monitoring system based on traditional financial metrics associated with the CAMEL rating system. However, the findings of this paper should not be construed as detracting from the critical dependence of a successful banking supervision program on the on-site examination process. A systematic off-site risk analysis based on a EWS is useful to the extent that it can allow prioritization of on-site inspections and prompt supervisory interventions in case of problematic institutions.

In addition, our results point to a set of indicators which regulators may find it useful to disclose on a consistent basis in order to build a more effective market discipline as a component of the regulatory framework. Banks in MENA could introduce general types of public disclosure, including their capital adequacy, risk exposures (credit, market and operational risk), risk assessment and management processes to allow market participants to evaluate banks’ ability to absorb losses and remain solvent.

10