then extended to include the present crisis. 3 these unconditional vars for uk banks increase, on average, by almost 60 % once the sample is extended ; and for some banks these risk measures more than double. for the system as a whole, one way of illustrating the consequences of underestimating risk is to translate it into “ fair value ” insurance premia. for example, consider a financial firm offering insurance against moves in future equity prices by writing put options in mid - 2007. pricing of this insurance is assumed to be based on the distribution of equity returns during for example, guttentag and herring ( 1986a ) and herring ( 1999 ). the banks themselves have been anonymised to protect the innocent. the golden decade. if the “ true ” distribution of returns were its long - run average, by how much would this insurance have been under - priced in 2007? chart 10 provides some answers for a selection of strike prices for the option. the degree of under - pricing of risk is large and is larger for options designed to protect against tail risks ( lower strike prices ). for at - the - money options on uk equities, the insurance premium would have been under - priced by around 45 % ; for options well out - of - the - money – say, 50 % below equity prices at the time – the mis - pricing would have been nearer 90 %. this is risk underpricing on a dramatic scale. these examples are no more than illustrative. but they help illustrate that the quantitative consequences of disaster myopia were material ahead of crisis and may have contributed importantly to the price of risk being set too low. and that, in turn, helped sow the seeds of the credit boom. network externalities any asset portfolio is, in essence, a financial network. so the balance sheet of a large financial institution is a network, with nodes defined by the assets and links defined by the correlations among those assets. the financial system is similarly a network, with nodes defined by the financial institutions and links defined by the financial interconnections between these institutions. evaluating risk within these networks is a complex science ; indeed, it is the science of complexity. 4 when assessing nodal risk, it is not enough to know your counterparty ; you need to know your counterparty ’ s counterparty too. in other words, there are network externalities. 5 in financial networks, these externalities are often referred to as contag

andrew g haldane : why banks failed the stress test speech by mr andrew g haldane, executive director, financial stability, bank of england, at the marcus - evans conference on stress - testing, london, 9 - 10 february 2009. i would like to thank geoff coppins, harry goodacre, nigel jenkinson, salina ladha, iman van leyveld, jack mckeown, joe noss, adrian penalver, nick vause and matthew willison for contributions and comments. * * * by any historical standard, the financial crisis of the past 18 months has been extraordinary. some have suggested it is the worst since the early 1970s ; others, the worst since the great depression ; others still, the worst in human history. time will tell. risk managers are of course known for their pessimistic streak. back in august 2007, the chief financial officer of goldman sachs, david viniar, commented to the financial times : “ we are seeing things that were 25 - standard deviation moves, several days in a row ” to provide some context, assuming a normal distribution, a 7. 26 - sigma daily loss would be expected to occur once every 13. 7 billion or so years. that is roughly the estimated age of the universe. a 25 - sigma event would be expected to occur once every 6 x 10124 lives of the universe. that is quite a lot of human histories. when i tried to calculate the probability of a 25 - sigma event occurring on several successive days, the lights visibly dimmed over london and, in a scene reminiscent of that little britain sketch, the computer said “ no ”. suffice to say, time is very unlikely to tell whether mr viniar ’ s empirical observation proves correct. fortunately, there is a simpler explanation – the model was wrong. of course, all models are wrong. the only model that is not wrong is reality and reality is not, by definition, a model. but risk management models have during this crisis proved themselves wrong in a more fundamental sense. they failed keynes ’ test – that it is better to be roughly right than precisely wrong. with hindsight, these models were both very precise and very wrong. for that reason, 2008 might well be remembered as the year stress - testing failed. failed those institutions who invested in it in the hope it would transform their management of risk. failed the authorities who had relied – perhaps over - relied – on the signal it provided