How to design sample size?

●Minato Nakazawa, Ph.D.
●
Dept. Public Health●●1References■ Textbook in English

 "Chapter 14. Sample size issues." in Machin D, Campbell
MJ, Walters SJ (2007) Medical Statistics, 4th ed., Wiley,pp. 261-275.
 "Chapter 4. Comparing groups with p values: Reporting
hyothesis tests." in Lang TA, Secic M (2006) How to reportstatistics in medicine: Annotated guidelines for authors,editors, and reviewers. 2nd ed., American College ofPhysicians., pp.45-60.
 "Chapter 1. Research design" in Peacock JL, Peacock PJ(2011) Oxford handbook of medical statistics. Oxford Univ.
Press, pp.1-73 (especially 56-73).

■ Textbook in Japanese 永田靖 (2003) サンプルサイズの決め方 . 朝倉書店
 新谷歩 (2011) 今日から使える医療統計学講座【 Lesson 3 】
サンプルサイズとパワー計算 . 週刊医学界新聞， 2937 号
http://www.igaku-shoin.co.jp/paperDetail.do?id=PA02937_06

2

A hypothetical case
■ Due to the availability of patients within graduate schoolterms, only 10 patients were investigated to test thespecific null-hypothesis X that factor Y is not related withfactor Z, and the null-hypothesis could not be rejected ata significance level of 5%.
■ Based on the consideration that the result of "notsignificant" is also to be published to avoid publicationbias, a manuscript entitled as "No relationship in thepatients ..." was written and submitted to a journal.
■ The referee judged as "No significance may be caused by
the less sample size." (In other words, the statisticalpower was not enough.) This is the fatal fault of the study
design. Usually the manuscript is rejected.
■ The student asked to a statistician of helpful advices, but
the statistician can do nothing to help the one.
■ It's a kind of common tragedy for many students.
3

What can the student do?
■ The study framework was the hypothesis testing.
■ Statistical power may increase with the sample size, sothat it could be possible to determine the sample size toachieve enough statistical power before the study.
■ After the study, the student can do nothing but writing
excuse: e.g. due to less availablity of the patients, shortof funding, tradition in that specific study field.
Sometimes it will pass the peer review because the result
can contribute to future meta-analysis.

4Statement in medical statistics textbooks■ A study that is too small may be unethical, since it is not
powerful enough to demonstrate a worthwhile correlation
or difference.
■ Similarly, a study that is too large may also be unethicalsince one may be giving people a treatment that could
already have been proven to be inferior.
■ Many journals now have checklists that include aquestion on whether the process of determining sample
size is included in the method section (and to bereassured that it was carried out before the study and notin retrospect).
■ The statistical guidelines for the British Medical Journal in
Altman et al. (2000) state that: `Authors should include
information on … the number of subjects studied and whythat number of subjects was used.'5

Why not sample size calculations?
■ A cynic once said that sample size calculations are aguess masquerading as mathematics. To perform such a
calculation we often need information on factors such as
the standard deviation of the outcome which may not be
available. Moreover the calculations are quite sensitive
to some of these assumptions.
■ Any study, whatever the size, contributes information,and therefore could be worthwhile and several small
studies, pooled together in a meta-analysis are moregeneralisable than one big study.
■ Often, the size of studies is determined by practicalities,
such as the number of available patients, resources, time
and the level of finance available.
■ Studies, including clinical trials, often have several
outcomes, such as benefit and adverse events, each ofwhich will require a different sample size.
6Not necessary cases■ Qualitative studies / Case report■ Small survey / pilot study In descriptive study, usually previous information about themeasures is unavailable, so that the sample size
calculation is impossible.
 Rules of thumb: at least 12 individuals in each group➔ List the main cross tabulations that will be needed to ensurethat total numbers will give adequate numbers in theindividual tables cells.

7

In the exploratory studies ...
■ Two kinds of study Testing the null-hypothesis always requires the sample
size calculation before the study (already explained).
 Exploring the hidden hypothesis or describing estimateswith 95% confidence intervals may not always require thesample size calculation, but power analysis (to evaluatesampling adequacy) after the study is possible.

■ In the exploratory or descriptive studies

 Prevalence estimates from small samples will be imprecise
and may be misleading. For example, when we wish to get
the prevalence of a condition for which studies in othersettings have reported a prevalence of 10%. A small
sample of, say, 20 people, would be insufficient to produce
a reliable estimate since only 2 would be expected to havethe condition and ±1 would change the estimate by 5%.
 Sample size calculation determine the number of subjects
needed to give a sufficiently narrow confidence intervals. 8

Examples in exploratory studies■ Values are obtained from previous studies in advance.
 When we would like to estimate a mean, the following 3
values are needed.
➔ The standard deviation (SD) of the measure being
estimated➔ The desired width of the confidence interval (d)➔ The confidence level (usually 90, 95, or 99 %; 1-alpha) Necessary number of samples (n) is obtained by:
n = qnorm(1-alpha/2)^2 * 4 * SD^2 / d^2
 (e.g.) Suppose we wish to estimate mean systolic bloodpressure in a patient group with a 10mmHg-wide (or5mmHg-wide) 95% confidence interval. Previous worksuggested using a standard deviation of 11.4.
n = 1.96^2 * 4 * 11.4^2 / 10^2 = 19.97... = 20
n = 1.96^2 * 4 * 11.4^2 / 5^2 = 79.88... = 80
 Doubling the precision needs quadrupling the sample size.

■ Estimating proportions will be given in the next slide.

9

Examples in exploratory studies (cntd.)■ Required information from previous studies and study
purpose to estimate proportion Expected population proportion (p) Desired width of confidence interval (d) Confidence level (1-alpha)■ Approximate equation to estimate the number of subjects
needed [ qnorm(1-alpha/2) means z 1-α/2 ; ^2 = squared ]:n = qnorm(1-alpha/2)^2*4*p*(1-p) / d^2■ (e.g.) Suppose we wish to estimate the prevalence ofasthma in an adult population with the width of the 95%
confidence interval 0.10, an accuaray of ±0.05. An
estimate of the population prevalence of asthma is 10%.
 p = 0.10, d = 0.10, alpha = 0.05
 n = qnorm(0.975)^2*4*0.1*0.9 / 0.1^2 = 1.96^2*36 = 138

10

Principles in hypothesis-testings■ What kind of information is needed?

 Method of statistical test (including null-hypothesis)
 Type I error (alpha error: probability to reject the true nullhypothesis, in other words, false positive)
 Type II error (beta error: probability to fail to reject the false
null-hypothesis / false negative) = 1 – statistical power Expected values from previous studies
 Minimum differences of clinical importance■ Equations are quite different by statistical tests (and bytextbooks, because all of those are of approximation)
 Compare means by t-test:
n= 2*(zα-z1-β)2*SD2/d2 + zα2/4 Compare proportions by χ2 test:
n= (zα/2+z1-β)2*{p1(1-p1)+p2(1-p2)}/(p1-p2)2■ Usually special softwares (nQuery, PASS, PS) or general
statistics softwares (SAS, SPSS, STATA, EZR, R, etc.)
11will be applied.

An example■ Suppose we wish to compare the mean increased degreeof elbow flexion between stimulated and control patients.
 4 degree difference has clinical importance.
 Let alpha error 0.05 and statistical power 90%.
 SD of increase of elbow flexion is assumed as 5 degree.

■ Calculation by the previous equation

 n= 2*(zα-z1-β)2*SD2/d2 + zα2/4= 2*(-1.64-1.28)^2*5^2/4^2+(-1.64)^2/4= 27.3174 ~ 27

■ Based on this, 26 treated patients and 25 control patientswere measured. They showed increase in elbow flexsion
by 16±4.5 and 6.5±3.4, respectively. The meandifference was 9.5 (95%CI was 7.23 to 11.73), t-test
resulted in t=8.43, df=49, p<0.001.
■ Typical description of this design and statistical results
should be written as follows:12

Typical description of the situation■ We designed the study to have 90% power to detect a 4degree difference between the groups in the increased
range of elbow flexion. Alpha was set at 0.05. Patients
receiving electrical stimulation (n=26) increased theirrange of elbow flexion by a mean of 16 degrees with astandard deviation of 4.5, whereas patients in the control
group (n=25) increased their range of flexion by a meanof only 6.5 degrees with a standard deviation of 3.4. This
9.5-degree difference between means was statisticallysignificant (95%CI = 7.23 to 11.73 degrees; two-tailed
Student's t test, t=8.43; df=49; p<0.001).
(Lang and Secic, 2006, pp.47)■ Here the expected standard deviation 5 nor applied
equation is not clearly written (both seems implicit).
13Usage of the PS■ PS: Power and Sample
Size Calculator■ http://biostat.mc.vanderbilt.edu/twiki/bin/view/Main/PowerSa
mpleSize■ Free Software■ Survival (logrank test), ttest, Regression1,Regression2,Dichotomous (chisqtest), Mantel-Haenszel
are included.
■ An example ofdescription is given astext.
■ Ratio of two groups canbe specified as m14

Usage of EZR on Rcmdr■ EZR on Rcmdr is developed by Jichi Medical School■ http://www.jichi.ac.jp/saitama-sct/SaitamaHP.files/statmed.html■ Free software; latest version is 0.96 on 31 July 2011.
■ Installation is very easy. Just click the downloaded
executable installer. Messages are given in Japanese.
■ Usage is also easy, but manual is not enough.

1.00.8

Power0.6

0.4

0.2

0.0102030
N14015

R console■ > power.t.test(delta=4, sd=5, sig.level=0.05, power=0.9)
Two-sample t test power calculationn = 33.82555
delta = 4
sd = 5sig.level = 0.05
power = 0.9
alternative = two.sided
NOTE: n is number in *each* group■ The result is 34 (similar to PS)
16