David Spiegelhalter Chair of the Winton Centre for Risk & Evidence Communication, University of Cambridge President, Royal Statistical Society (2017-2018) ISM Consortium for Human Resource Development of Statistical Experts in Japan.  February 2022 Robert 5. Cowell : A. Philip Dawid en L. Launizen - David J. Spiege hatter Exact Computational Methods for Bayesian Networks ② Springer Bavesian Approaches Bayesian Analysis Christopher Jackson Nicky Best Andrew Thom David Spiegelna I used to do statistical methodology.... until I was philanthropically funded in 2007.. | B | C Weather Shop Earth Travel More Search B B C ● Sign in FOUR Climate Change by Numbers FOUR Tails You Win: The Science of Chang Next on Why statistics should make you suspicious Amanpour Renowned statistician and author of "The Art of Statistics" Sir David Spiegelhalter breaks down some common numerical misconceptions. Source: CNN Weather IPlayor Radio More... Search Winton Centre for Risk and Evidence Communication A PELICAN BOOK The Art of Statistics
Learning from Data
David Spiegelhalter
INTRODUCTION The numbers have no way of speaking for themselves. We speak for them. We imbue them with meaning. Nate Silver, The Signal and the Noise¹
• Describing data with summary statistics o dull • Probability theory for drawing random observation from a population distribution ○ difficult and mathematical • Probability theory for distributions of summary statistics ○ mathematical and incomprehensible • Formulae for statistical tests 。 mathematical, unmotivated, just a bag of tools • (If lucky) Examples of using statistical models in real life.
• Motivate by problem solving • Start with visualisation and exploring data • Focus on what can be reasonably learned from data, biases in data, concluding causation, etc • Models and algorithms • Assessing uncertainty through re-sampling data (‘bootstrap’) • Probability theory as neat way of turning random variation into uncertainty about what is true • Hypothesis testing and its potential problems • Bayesian methods All these rather abstract, challenging, ideas are there to help answer real questions • The 'data cycle' • eg PPDAC (promoted in New Zealand)
Data detectives use PPDAC Looking at data What was the pattern of Harold Shipman's murders? ‘I have nothing to hide’ Dr Harold Shipman, general practitioner, on his arrest in September 1998 Shipman Inquiry July 2002: 215 definite victims, 45 probable year Looking at data What was the pattern of Harold Shipman's murders? • Problem: can more detail tell us more about what Shipman did? • Plan: compare actual times at which his patients died with the times of deaths recorded by other local GPs • Data: a huge exercise requiring examination of death certificates • Analysis: simple plotting.....
hour
People die at all hours
hour
People die at all hours - but not Shipman’ s victims Inference and bias How many sexual partners have people in Britain had in their lifetime? • Problem: cannot know this as a fact • Plan: survey in which people are carefully asked about the sexual activity (Natsal) • Data: reports of numbers of partners • Analysis: plotting and summary statistics
Reported number of lifetime opposite-sex partners Inference and bias How many sexual partners have people in Britain really had in their lifetime?
• Conclusions: can we generalise this to the whole population?????
•  1 to 2.  How reliable are the reports? •  Poor memory, social acceptability bias etc •  2 to 3. How representative is the sample of those eligible for the study? •  Random sampling of families (soup), 66% response •  3 to 4.  How close does the study population match the target population? •  No people in institutions, etc Causation (or correlation) The power of the press release.... Socioeconomic position and the risk of brain tumour: a Swedish national population-based cohort study Amal R Khanolkar, 1.2 Rickard Ljung, 2 Mats Talbäck, 2 Hannah L Brooke, 2 Sofia Carlsson, ² Tiit Mathiesen, ³ Maria Feychting² • abstract: • We observed consistent associations between higher socio-economic position and higher risk of glioma • press release • High levels of education linked to heightened brain tumour risk • Daily Mirror... News-   Politics Football Sport -    Celebs -    TV & Film Weird News ul Most read   ★ Videos IPHONE SE | APPLE | GOOGLE | FACEBOOK | WHATSAPP | VIDEO GAMES TRENDING Money Travel Fashion Technology Why going to university increases risk of getting a brain tumour 23:30, 20 JUN 2016/ BY ANDREW GREGORY Highly educated people are more likely to suffer from brain tumours than those who do not progress as far in their education Are Your Savings Enough to Retire lf you have a £250,00 porttolio, download th Regression, prediction and algorithms Who was the luckiest person on the Titanic?
NT /ひご(201 SYON (1975年)
• Challenge: can we build an algorithm that will accurately predict who survives the Titanic? • Based on factors in data-base, produce either a yes/no judgement, or a probability of survival • Split the data-base of 1309 passengers at random into a training set (70%) on which to build algorithms, and a test set (30%) to assess how good it is. • Currently over 59,000 entries in a similar online Kaggle competition survival Family size Percentage survival Percentage survival
Fare Age Title
• ‘Accuracy’ is a very crude way of judging an algorithmic prediction • Better to use the probabilities provided • If probability p is given to an event X (0,1), then the Brier score is (X– p)²
• Potentially a very misleading graphic! • When comparing, need to acknowledge that tested on same cases • Calculate differences and their standard error • How confident can we be that simple CART is best algorithm?
• Bootstrap sample from test set (ie sample of same size, drawn with replacement) • Rank algorithms by performance on the bootstrap sample • Repeat ‘000s of times • (ranks actual algorithm – if want to rank methods, need to bootstrap training data too, and reconstruct algorithm each time) Distribution of true rank of each algorithm Probability of ‘best’: 63% simpleCART 23% ANN 8% randomforest
• Karl Dahl, a 45-year-old Norwegian/Australian joiner travelling on his own in third class, paid the same fare as Francis Somerton • Had the lowest average Brier score among survivors – a very surprising survivor • He apparently dived into the freezing water and clambered into Lifeboat 15, in spite of some on the lifeboat trying to push him back. • Hannah Somerton was left just £5, less than Francis spent on his ticket. Hypothesis testing Could Harold Shipman have been caught earlier? • Using mortality rates from local GPs, calculate how many deaths he would have been expected to observe each year, under the null hypothesis that his mortality rates were normal. • Subtract expected from observed number to get excess mortality (NB: Shipman Inquiry total of definite or probable victims: 189 female > 65,  55 male over 65)
Could Harold Shipman have been caught earlier? • But when to ‘blow the whistle’? • This are two possible types of error - • Type I error: falsely accuse an innocent person • Type II error: miss someone with true increased risk • This is an example of a hypothesis test used throughout science.  Again, two possible types of error • Type I error: falsely claim an 'effect' when nothing is there (ie the null hypothesis is true) • Type II error: miss a true effect • Generally, we want to • control the probability of a Type I error at a low value (α) • make experiments large enough to make Type II errors rare (β) Shipman:  “Sequential probability ratio test” (SPRT) older females would have set off ‘alarm’ in 1985, after only 40 deaths Probability and Bayes True state of null hypothesis Result of significance test • 1000 experiments • Assume that in 10%, there is really an effect • Size (α) = 5% • Power (1-β) = 80%
Result of significance True state of null hypothesis • If reject the null, only 80/125 = 64% chance that null is False
• Initial odds that null hypothesis False = 10 / 90 • After 'significant' results, final odds that null hypothesis False = 80/ 45 Pr(significant result | null hypothesis False) • Likelihood ratio = Pr(significant result | null hypothesis True)
● • Bayes theorem: the initial odds for the hypothesis       x      the likelihood ratio = the final odds for a hypothesis.
Probability and Bayes What is the probability that the skeleton in a Leicester car park was really Richard III?
•  On Saturday 25 August 2012, archeologists started digging in a car park in Leicester – the site of Grey Friars friary •  In a few hours they found their first skeleton •  This was later claimed to be Richard III ARTICLE Received 5 Aug 2014 | Accepted 21 Oct 2014 | Published 2 Dec 2014 Identification of the remains of King Richard III Turi E. King 12 , Gloria Gonzalez Fortes 3.4.4 , Patricia Balaresque 5.4 , Mark G. Thomas 6 , David Balding 6 , Pierpaolo Maisano Delser', Rita Neumann', Walther Parson 7.8 , Michael Knapp 9 , Susan Walsh'o'n, Laure Tonasso 5 , John Holt 12 , Manfred Kayser 11 , Jo Appleby 2 , Peter Forster 314 , David Ekserdjian 15 , Michael Hofreiter 3.4 & Kevin Schürer 16 probability of evidence, if skeleton were Richard III probability of evidence, if someone else
Suggested ‘verbal equivalents’ for bands of likelihood ratios
Standards for the formulation of evaluative forensic science expert opinion
Association of Forensic Science Providers

• Motivate by problem solving • Start with visualisation and exploring data • Focus on concepts: what can be reasonably learned from data, biases, causation, etc • Models and algorithms • Probability can come much later • Conditional probability / Bayes theorem taught through ‘expected frequency trees/