, however, the errors of both groups in predicting stock prices were roughly the same. in a subsequent set - up, both groups were asked to pick the best - performing stock from two options. here, both the professionals and the laymen proved to be overconfident with regard to their ability to pick the best - performing stock. eventually, the professionals picked the correct stock in only 40 % of cases. had they randomly picked one of the two options that were offered they would have been right about 50 % of the time. 4 cheng, i ; raina, s ; xiong, w ( 2014 ), wall street and the housing bubble. in american economic review, vol 104, no 9, pp 2797 – 2829. torngren, g ; montgomery, h ( 2004 ), worse than chance? performance and confidence among professionals and laypeople in the stock market. in the journal of behavioral finance, vol 5, no 3, pp 148 – 153. bis central bankers ’ speeches studies such as these led nassim taleb to provocatively state that : “ certain professionals, while believing they are experts, are in fact not. based on their empirical record, they do not know more about their subject matter than the general population, but they are much better at narrating – or worse at smoking you with complicated mathematical models. ” this is certainly an exaggeration, but it seems justified to conclude that our ability to see in the dark is somewhat limited. the quote points to another issue that is relevant in this regard. can statistical models mitigate people ’ s biases when it comes to handling risk? to some degree, they certainly can raise our awareness of potential risks but they are by no means perfect. with regard to the financial crisis, some banks claimed to have experienced a “ 25 sigma ” event – a point in the probability distribution that is 25 standard deviations away from the mean. statistically speaking, such an event should have only occurred once since the universe came into existence. statistical models are often criticised for systemically neglecting such “ high - severity - lowprobability ” risks. 5 one reason for this is that many models use the past as the basis for the future when assessing the probability of certain events. this weakness could be referred to as the chicken fallacy. a chicken that is fed by a farmer every morning may expect this pattern to continue in the future. however, as the philosopher bertrand russell concludes : “ the

##ing all new applications at banks together – that is the underlying principle of risk - oriented supervision. the responses to the consultation reaffirm the approach we are taking. there is a broad consensus that no explicit definition should be prescribed. it would in any case be virtually impossible to capture the diversity of procedures and their continuous development in a rigid definition. the focus should not be on generalised requirements, but on specific use cases. and the results of the consultation also make it clear that ml does not need its own new set of rules. many supervisory requirements, such as those covering model development, operation and validation or data management, come about because they address complex statistical models. i believe that the current technology - neutral and risk - oriented regulatory approach should also stake out a general framework for ml. our supervisory approach also focuses on ensuring that data are of the right quality. that was a point that elicited a unanimous response in the consultation, although good and meaningful data have always been a key factor in the success of models, whether ml or not. procedures which process huge amounts of data do not alter this principle. another key criterion is explainability. ml is often referred to as a “ black box ”, an opaque system in which data are fed in at one end and results are produced at the other without us being able to understand how one relates to the other. in this sense, dinosaurs, too, are a black box that we cannot fully comprehend, never having seen them. did they have fur or feathers? how large was their brain? and what colour were they even? with ml, too, we often can ’ t tell straight away which relationships serve as a basis for decision - making. however, if decisions made by ml systems have far - reaching effects on us as individuals or on a bank ’ s risk situation, the decision - making process is significant. consider, for example, ai - based assessments of creditworthiness. we want to understand decisions, we want to shed light into the dark. the more complex an ml model ’ s design, the more difficult it is to describe the relationship between input and output verbally or using mathematical formulae. it is then often difficult for modellers, users, validators and also for us supervisors to verify the results in detail. and every stakeholder also has a different understanding of and need for explainability. checking that a model ’ s behaviour overall is explainable and plausible is therefore more important than for every detail to be ve