Central Bank Stress Tests: Mad, Bad, and Dangerous

In my youth it was said that what was too silly to be said may be sung. In modern economics it is put into mathematics. --Ronald Coase One of the most important aspects of the remarkable transformation of central banking following the onset of the 2008 global financial crisis is the growth of regulatory stress tests for the larger banks. The relevant regulator--typically the central bank--uses these to determine the ability of the banks to withstand stress, and uses the results of the tests to assess the overall financial health of the banking system. A key purpose of the stress tests is to reassure the public that the banking system is sound. When putting banks to such a test, the relevant authority starts by imagining some stress scenario(s) to which banks might be exposed--these are effectively just guesses pulled from thin air-and uses a bunch of models based on a bunch of further guesses to determine how the scenario(s) will affect the banks' capital adequacy (i.e., their ratio of capital to assets) over the course of the stress period. It then passes or fails individual banks according to whether their capital ratio has remained above some minimum by the end of that period. To take a typical example, in its latest (2014) stress tests, the European Central Bank (ECB) assumed a single scenario, took the capital ratio to be the ratio of Common Equity Tier 1 (CET1) capital to risk-weighted assets (BWA), and selected a minimum required ratio of 5.5 percent measured in terms of the CET1/RWA ratio. Any bank that maintained a CET1/RWA ratio of at least 5.5 percent by the end of the stress period was then deemed to have passed, and any bank whose capital ratio fell below this minimum was deemed to have failed. These regulatory stress tests are the ultimate in the appliance of financial "rocket science" to the banking system, and many of the models themselves are derived from the physical science models used so successfully in real rocket science. However, by their very nature, all these models--the financial models and the stress tests themselves--are impenetrable black boxes to any outsider, and we are asked to take their reliability on trust. The analogy with rocket science, though appealing and even comforting, then breaks down in two critical respects: * Real rocket science is grounded in the science of physics, and the laws of physics are well established. By contrast, so-called financial rocket science is merely a set of beliefs and practices based on sets of convenient assumptions that ape some of the assumptions made in physics, but are wide of the mark as descriptions of how financial markets really work. * We know that the methodology underpinning real rocket science actually works because it is scientifically tested, but we have no such assurance with its financial and central bank equivalents. Indeed, going further, we can say, with confidence, that we know that the methodologies underpinning both financial models and regulatory stress tests do not work: the stress tests provide an extremely unreliable radar system. My purpose in this article is to spell out this latter claim--or, more precisely, to assess the methodology of regulatory stress testing both by reference to first principles and by reference to its track record. The results are shocking. Financial Risk Models Are Worse than Useless The first point to appreciate is that central bank stress tests are based on models of financial risk--models that predict potential losses and their associated probabilities--and these models are not so much useless as worse than useless. More precisely, the stress tests are dependent on risk models because they make use of risk-weighted asset measures that are dependent on the risk models. These models are useless at predicting financial losses and worse than useless as risk management tools because of their game-ability and the false risk comfort that they provide. …