The navigation team at NASA’s Jet Propulsion Laboratory mistook this data as being in the required metric units of Newton-seconds. This led to errors in the spacecraft’s trajectory calculations. One of the root causes was NASA’s adoption of a ‘faster, better cheaper’ philosophy that had not given sufficient prominence to risk management. It had also not carried out an independent, end-to-end validation. Finally, developers had had an insufficient sense of ownership of the whole project as opposed to their bit of it. The second source of model risk is misuse of models. This can arise when management take decisions based on the outputs of a model without sufficiently understanding the uncertainty around those outputs. Complicated, dynamic problems are more difficult to model than simple, stable problems. Model outputs are therefore more uncertain for problems that are more complex and where the underlying process in the real world is changing. Calculating the capital requirements of an insurer is surely a prime example. The IFoA Model Risk Working Party has an interesting typology of corporate cultures according to their concern for model uncertainty and the legitimacy of modelling. Although they explain that all are valid viewpoints, I would feel more comfortable if insurers fell into the category of conscientious modellers, with modelling having high legitimacy but management also having a high concern for model uncertainty. (see Chart 3)  The Solvency II use test requires models to play an important role in risk management, decision-making and capital allocation. Management should therefore have confidence in their models.

If recent claims experience is suggesting increasing risk when models are not, that should raise questions for management. The risk of misuse of models might be particularly acute when an insurer buys in a model from a third party. Its people have not developed the model and become familiar with its limitations. Management might be tempted to put too much trust in the reputation of the supplier and assume that if many other insurers are using the model, ‘it must be fine’. The Senior Managers and Certification Regime requires insurers to allocate certain prescribed responsibilities to individual senior managers. Responsibilities for managing capital, funding and liquidity and for the Own Risk and Solvency Assessment, for example, entail a good understanding of an insurer’s internal capital model. In practice however, it is unrealistic to expect that everyone using a model will fully comprehend how it works. The crux of good model risk management is to ensure that decision makers know enough. My former boss, Andrew Bailey summarised this when discussing our expectations of boards in 2015, “the challenge is to reduce complexity to simplicity, so that Board members feel that they understand:  where is the model expected to work well;  in what circumstances is it likely to break down;  is the overall model output credible;  what “moves the dial” in terms of key assumptions or judgements; and  are those assumptions and judgements reasonable?”8 LIBOR Transition  Before I conclude, I would like to spend a moment on a different subject, LIBOR transition.