Chapter 8. Random Error
and the Role of Statistics
December 15, 2022
Epidemiology (10)Minato Nakazawa<minato-nakazawa@people.kobe-u.ac.jp>

The roles of statistics in epidemiologic study●Measuring variability to assess the role of chance (There are dual meanings in the wordchance)–Outcome of a random process → Outcome could not be predicted under any circumstances

The coin-toss is usually considered as such random process, the result is completely
unpredictable– Outcomes that cannot be predicted easily but are not necessarily random phenomena●If we have sufficient information about the initial conditions and the forces applied to thecoin, the result of flipping coin can be predicted (Since usually we don’t have suchinformation, it’s considered as randomizing event). Some individuals can predict flipping
coins after practices enough to predict the result●If we learn more about the source of error in a body of data, we can reduce errors that
may appear random at first (“Physicists tell us that we will never be able to explain allcomponents of error” means Heisenberg’s uncertainty principle [see,https://www.youtube.com/watch?v=TQKELOE9eY4 ]).
●Estimating effects after correcting for biases such as confounding●●Among those two roles, this chapter address the former (Random error is variability. In cohort
studies, some of the variability in the outcome is random error, much of variation may reflect
hidden biases, unmeasured, even undiscovered). The latter is addressed in Chapter 10 and12.

Estimation●The analysis of data should report the magnitude of thatepidemiologic quantity and portray the degree of precision
with which it is measured–(eg.) Case-control study may be undertaken to estimate
the IRR between the use of cellular phone and theoccurrence of brain cancer
The report should present a clear estimate of IRR suchas IRR=2.5 – such estimate shown as single value =“point estimate”●
To show precision, a “confidence interval” (a range ofvalues around the point estimate) is used. → Wideconfidence interval = low precision, narrow confidence
interval = high precision●

POINT ESTIMATES, CONFIDENCE
INTERVALS, AND P VALUES
Point estimate (being a single value) cannot express the statistical variation (random error)
Large study → estimation process is comparatively precise → little random error●
Small study → less precision, more random error●Confidence interval: The amount of random error in the estimate– A given confidence interval is tied to an arbitrarily set level of confidence (eg., 95%, 90%, …)– Statistical definition: The level of confidence is set to 95% → If the data collection and analysis could be replicatedmany times and the study were free of bias, the confidence interval would include within it the correct value of themeasure 95% of the time.
– The statistical definition presumes (1) the difference among replication is only caused by chance, (2) variability inthe data is adequately described by a statistical model without any bias (or bias is completely controlled) →unrealistic even in randomized controlled trials– Practical meaning: A rough estimate of the statistical variability (actually random error) in a set of data
●The confidence interval is calculated from the same equations that are used to generate another commonly reported
statistical measure, the P value (the statistic used for statistical hypothesis testing).
●●●

The P value is calculated in relation to a specific hypothesis (usually null hypothesis = there is no relation between
exposure and disease)–

For RR measure, null hypothesis is RR=1.0–

The P value represents probability (under the condition that the null hypothesis is true, the study is free of bias,
and the observations are independent each other) that the data obtained in the study would demonstrate anassociation as far from the null hypothesis or farther than what was actually obtained. → The required condition isseldom met, so that the P value is not a meaningful probability. Instead, a measure of relative consistency
between the null hypothesis and the data in hand.

–More specifically, if a P value were as small as 0.01, it would mean that the data were not very consistent with thenull hypothesis.

STATISTICAL HYPOTHESISTESTING VERSUS ESTIMATION●●Often a P value is used to determine the presence or absence of statisticalsignificance. Statistical significance is sometimes considered as meaningful
term, but only means whether the P value is less than some arbitrary value(almost always 0.05).
Statistical hypothesis testing is used to describe the process of deciding
whether to reject or not to reject a specific hypothesis (usually nullhypothesis).
–If an analysis gives a result of statistically significant, the null hypothesis isrejected as false.

–

If a result is not statistically significant, it means that the null hypothesis
cannot be rejected. It doesn’t mean that the null hypothesis is correct.

–The declaration of “statistically significant” or “not significant” is lessinformative than giving a P value.

–

Separation of the information on strength of relation and precision isimportant. Thus both point estimates and precision information (P value orconfidence interval) are needed.

P-value (confidence interval) functions●●●

P-value function (= confidence interval function)
enlarges on the concept of the P value.

Figure 8-1

Instead of testing only the null hypothesis, we
can also calculate a P value for a range ofother hypothesis.
–Consider RR ranging from 0 to infinity,equaling to 1.0 if the null hypothesis iscorrect.

–We can calculate infinite number of P values
that test every possible value of RR.

–If we do so and plot the result, it becomes pvalue function.

(eg. Figure 8-1) The ordinary P value is 0.08.
Point estimate of OR is 3.2 (In main text, it’swritten a RR, but actually OR). Though P value
is larger than 0.05, p-value function in Figure 81 makes it clear that there is a strong
association in the data. Since the P values aresame for RR=1 and RR=10.5, there is no
reason to prefer the interpretation of RR=1 overthe interpretation of RR=10.5.

Table 8-1. Case-control data for congenital heartdisease and chlordiazepoxide use in earlypregnancy
Chlordiazepoxide useYesNo

Total

Cases

4386

390

Controls

41250

1254

Total

81636

1644

OR = (4/386)/(4/1250) = 1250/386 = 3.2

P-value (confidence interval) functions(cont’d) Data in
●

●

●

●

Table 8-2

Table 8-2 is the data with large samplesize.
Point estimate of OR is 1.1
A P value for null hypothesis is 0.04(less than 0.05), so “significant”.
The narrowness of the p-valuefunction for Table 8-2 reflects thelarger sample size, which only means
better precision.
In Table 8-1, the association is “notsignificant” but the study is properlyinterpreted as raising concern about
the effect of the exposure. In Table 82, the study provides reassurance
about the absence of a strong effect,
but the significance test gives a resultthat is “significant”.

Figure 8-2

Null hypothesis●Data in Table8-1

Table 8-2. Hypothetical case-control data
ExposureYesNoTotalCases

1090

14910

16000

Controls

1000

15000

16000

Total

2090

29910

32000

OR = (1090/14910)/(1000/15000) = 1.1

P-value (confidence interval) functions(cont’d)●●

The most straightforward way
to get both messages(strength of association andthe effect of precision) is fromthe upper and lower
confidence limits: The two
numbers that form theboundaries to a confidenceinterval.
The p-value function is closely
related to the set of all
confidence intervals for agiven estimate (see, Figure 83, showing p-value function as
nested confidence intervals).

●●Instead of showing full p-value function, asingle confidence interval is sufficient todetermine the entire p-value function.
Confidence intervals are too often notinterpreted with the image of thecorresponding p-value function in mind.
Sometimes only used to determine statisticalsignificance by whether the confidence
interval includes null hypothesis or not. It’s
regrettable.

Figure 8-3

Example: Is flutamide effective intreating prostate cancer?
Figure 8-4 shows the result of metaanalysis (the calculation of pooled
odds ratio, ORMH is explained in
Chapter 10, p.187)–––

The result of 10 studies showed“not significant” protective effect of
flutamide on prostate cancer(ORMH=0.89, p>0.05)
By adding Table 8-3 data (the study
itself is also “not significant”,OR=0.87, p=0.14), the result of 11
studies showed “significant”
protective effect (ORMH=0.89,p<0.05), simply due to theimprovement of precision.
However, p-value functions arealmost same.

●●

https://www.ncbi.nlm.nih.gov/pubmed/7630245
https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(05)74403-2/fulltext

Table 8-3. Summary of survival data from thestudy of flutamide and prostate cancer
Flutamide

Placebo

Died

468

480

Survived

229

205

Total

697

685

OR = 0.87, 95%CI = 0.70-1.10← BeneficialeffectNo effect●

10studies11
studiesAdverseeffect →Figure 8-4

Example: Is St. John’s wort effective inrelieving major depression?
●

●

●

●

●

Extract of St John’s wort have long beenused as a folk remedy for depression
Table 8-4 is the result of randomized
trial only among those whosedepression was relatively less severeRR=2.0 means that the use of St. John’swort increased the remission insymptoms two times more than placebogroup.
The authors misinterpreted the result by
the lack of statistical significance(p=0.14 for null hypothesis)
Figure 8-5 shows that the data
regarding remissions among the lessseverely affected patients hardly support
the theory that St. John’s wort isineffective.

●

https://jamanetwork.com/journals/jama/f
ullarticle/193754
Table 8-4. Remission among patients with lesssevere depression
St. John’s wort

PlaceboRemission

125No remission

4745

Total

5950

RR = (12/59)/(5/50) = 2.0, 90%CI: 0.90-4.6

Simple approaches to calculating
confidence intervals●●●

Confidence intervals are usuallycalculated on the presumptionthat the estimate comes fromthe statistical distribution calleda normal distribution, the usual
bell-shaped curve.
Estimates based on the normal
distribution are always
reasonable with enough data. If
data are sparse, specialized
formula for small numbers(exact methods) are needed.
A given normal distribution isdescribed with regard to its
mean and its spread (SD)●●For rate difference (or risk difference)–

The formula [8-1] gives 90% confidence interval for the rate
difference.

–

RDL is lower confidence limit–

RDU is upper confidence limit–

1.645 corresponds to 90% confidence. 1.96 corresponds to 95%
confidence–

In R, qnorm(1-(1-0.90)/2) gives 1.645. The normal distribution
from qnorm((1-0.90)/2) to qnorm(1-(1-0.90)/2) includes 90% ofdata.

For rate ratio (or risk ratio or odds ratio)–

The distribution of ratio measures is asymmetricallyskewed toward large values. To conteract theskewness, it is customary to set the confidence
limits on the log scale (after a logarithmictransformation, take exponential again)–[8-2] gives 90% confidence interval for the risk ratio

Statistical significance testing versus
estimation (box)
●
●

●

●

Significance testing is qualitative, not quantitative
Ideally, a confidence interval should be viewed as a tool toconjure up an image of the full P-value function, a smooth
curve with no boundary on the estimate
In most instances, there is no need for any test of statisticalsignificance to be calculated, reported, or relied on, and weare much better off without them.
(For today’s caluculation, R code is available as
http://minato.sip21c.org/epispecial/codes-for-Chapter8.R )