FCSM 2020

14 Sep. 2020

Samantha C. Tyner, Ph.D.

 sctyner

 sctyner

 sctyner.me

Slides available at bit.ly/viz-fcsm

Image credit: Allison Horst

Visual Complements to -values p
John Arbuthnot. Source: Wikipedia

John Arbuthnot, "An argument for Divine Providence, taken from the constant

regularity observed in the births of both sexes" [1]

Used probability ("p-value") to "prove" divine intervention in the human birth

ratio (males are more common than females)

Probability calculation that today would be the -value in a sign test.

(binom.test() in R)

Case Study: Early use of "p-values"

p

2 / 18

Image by Allison Horst

Resulting -value1 is incredibly small. How to understand it's smallness?

Analyzing the Arbuthnot Data

p

[1] To arrive at the same -value as Arbuthnot, do this test in R: binom.test(x = 82, n = 82, p = .5, alternative =

"greater")

p

4 / 18
Image by Allison Horst 3 visual complements for 3 types of

analyses:

Simulation

Frequentist model

Bayesian parameter estimation

Notation:

: year {1629, 1630, ..., 1710}

: number of recorded births in

year 

: number of male births in year 

: 

Resulting -value1 is incredibly small. How to understand it's smallness?

Analyzing the Arbuthnot Data

p

i

ni

i

yi i

pi yi/ni

[1] To arrive at the same -value as Arbuthnot, do this test in R: binom.test(x = 82, n = 82, p = .5, alternative =

"greater")

p

5 / 18
Image by Allison Horst

For some large :

1. For each year , draw samples: for from 1 to B.

2. For each year , compute the proportion of male births in each simulation: 

3. For each year , compute the proportion of simulations in which the proportion of

males is greater than or equal to the observed proportion of males in Arbuthnot's

data: .

4. Compute the average: .

5. Assuming years are independent, compute the joint probability of observing

values as or more extreme than Arbuthnot's data: .

Simulation Study

B

i B y∗

ib ∼ Binom(ni, 0.5) b

i

p

∗

ib = y

∗

ib/ni

i

P

∗

i = ∑Bb=1 I(p

∗

ib ≥ pI )

1

B

P

∗ = ∑

1710

i=1629 P

∗

i

1

82

p^ = ∏

1710

i=1629 P

∗

i

6 / 18
Average probability, 

: 0.012

Joint probability, :

Results P ∗ p^

< 10−100

7 / 18
Visualization

8 / 18
Image by Allison Horst

Generalized Linear Mixed-Effects Model:

Use lme4::glmer() in R to fit the model to Arbuthnot's data.

Frequentist model

yi ∼ Binom(ni, pi)

logit(pi) = μ + ai

ai ∼ N(0, σ

2

)

9 / 18
Table: Parameter estimates

term estimate std.error statistic p.value

mu 0.0657 0.003 21.9947 0

sigma 0.0190 NA NA NA

Table: 95% Confidence Intervals

2.5 % 97.5 %

sigma 0.0132 0.0255

mu 0.0598 0.0716

Fitted Model:

Estimate of population proportion:

Table: Fit metrics

AIC BIC logLik deviance df.resid

946.31 951.12 -471.15 942.31 80

Results

yi ∼ Binom(ni, pi)

logit(pi) = 0.06565 + ai

ai ∼ N(0, 3.6149 × 10−4)

logit−1(μ^) = logit−1(0.06565) = 0.51641

10 / 18
Visualization

11 / 18
Image by Allison Horst

Bayesian Hierarchical Model:

Fit to Arbuthnot's data with the runjags package in R.

Bayesian approach

yi ∼ Binom(ni, pi)

pi ∼ Beta(α, β)

μ = η = α + β

μ ∼ Beta(1, 1)

log(η) ∼ Logistic(log(n ̄), 1)

α

α+β

12 / 18
Results

Table: Summary of posterior draws of , 

param Lower95 Median Upper95 Mean SD

alpha 2805.40 5808.95 10673.32 6233.47 2246.01

beta 2628.34 5442.88 10003.01 5837.60 2104.54

α β

13 / 18
Visualization

14 / 18
Visualization

15 / 18
Image by Allison Horst

Dig deep into your model results

Think about how you can you show probabilities

What other ways are there to model your

data?

What other parameter values could result in

your data?

Simulation is your friend

Take advantage of existing tools (in R or

elsewhere)

ggplot2 extensions: ggridges, ggdist

Further reading: Model Visualization, Statistics on

Street Corners [2; 3]

Takeaways

16 / 18
References

[1] J. Arbuthnot. "An argument for Divine Providence, taken from the constant regularity observed in the births of

both sexes". In: Philosophical Transactions of the Royal Society of London 27.4 (1710), pp. 186-190. DOI:

https://doi.org/10.1098/rstl.1710.0011.

[2] H. Wickham, D. Cook, and H. Hofmann. "Visualizing statistical models: Removing the blindfold". In: Statistical

Analysis and Data Mining: The ASA Data Science Journal 8.4 (Jul. 2015), pp. 203-225. DOI:

https://doi.org/10.1002/sam.11271.

[3] A. Loy, H. Hofmann, and D. Cook. "Model Choice and Diagnostics for Linear Mixed-Effects Models Using

Statistics on Street Corners". In: Journal of Computational and Graphical Statistics 26.3 (2017), pp. 478-492. DOI:

10.1080/10618600.2017.1330207.

R Packages Used

tidyverse (ggplot2, dplyr, tidyr), lme4, runjags, tidybayes, ggdist, ggridges

17 / 18
Thank you!

Contact:

 sctyner90@gmail.com

 sctyner

 sctyner

 sctyner.me

Slides available at bit.ly/viz-fcsm