Statistical Science
2001, Vol. 16, No. 2, 101–133

Interval Estimation fora Binomial Proportion
Lawrence D. Brown, T. Tony Cai and Anirban DasGupta

Abstract. We revisit the problem of interval estimation of a binomial
proportion. The erratic behavior of the coverage probability of the standard Wald conﬁdence interval has previously been remarked on in theliterature (Blyth and Still, Agresti and Coull, Santner and others). We
begin by showing that the chaotic coverage properties of the Wald interval are far more persistent than is appreciated. Furthermore, common
textbook prescriptions regarding its safety are misleading and defective
in several respects and cannot be trusted.
This leads us to consideration of alternative intervals. A number ofnatural alternatives are presented, each with its motivation and context. Each interval is examined for its coverage probability and its length.
Based on this analysis, we recommend the Wilson interval or the equaltailed Jeffreys prior interval for small n and the interval suggested in
Agresti and Coull for larger n. We also provide an additional frequentist
justiﬁcation for use of the Jeffreys interval.
Key words and phrases: Bayes, binomial distribution, conﬁdence
intervals, coverage probability, Edgeworth expansion, expected length,
Jeffreys prior, normal approximation, posterior.
1. INTRODUCTIONof the t test, linear regression, and ANOVA, its
popularity in everyday practical statistics is virtually unmatched. The standard interval is known as
the Wald interval as it comes from the Wald large
sample test for the binomial case.
So at ﬁrst glance, one may think that the problem
is too simple and has a clear and present solution.
In fact, the problem is a difﬁcult one, with unanticipated complexities. It is widely recognized that theactual coverage probability of the standard interval is poor for p near 0 or 1. Even at the level ofintroductory statistics texts, the standard interval
is often presented with the caveat that it should be
used only when n · minp 1 − p is at least 5 (or 10).
Examination of the popular texts reveals that thequaliﬁcations with which the standard interval ispresented are varied, but they all reﬂect the concern
about poor coverage when p is near the boundaries.
In a series of interesting recent articles, it has
also been pointed out that the coverage properties of the standard interval can be erraticallypoor even if p is not near the boundaries; see, forinstance, Vollset (1993), Santner (1998), Agresti and
Coull (1998), and Newcombe (1998). Slightly older
literature includes Ghosh (1979), Cressie (1980)
and Blyth and Still (1983). Agresti and Coull (1998)

This article revisits one of the most basic andmethodologically important problems in statistical practice, namely, interval estimation of theprobability of success in a binomial distribution. There is a textbook conﬁdence interval forthis problem that has acquired nearly universal
acceptance in practice. The interval, of course, isp̂ ± zα/2 n−1/2 p̂1 − p̂1/2 , where p̂ = X/n is
the sample proportion of successes, and zα/2 is the1001 − α/2th percentile of the standard normal
distribution. The interval is easy to present andmotivate and easy to compute. With the exceptions
Lawrence D. Brown is Professor of Statistics, The
Wharton School, University of Pennsylvania, 3000
Steinberg Hall-Dietrich Hall, 3620 Locust Walk,
Philadelphia, Pennsylvania 19104-6302. T. Tony Cai
is Assistant Professor of Statistics, The Wharton
School, University of Pennsylvania, 3000 Steinberg
Hall-Dietrich Hall, 3620 Locust Walk, Philadelphia,
Pennsylvania 19104-6302. Anirban DasGupta is
Professor, Department of Statistics, Purdue University, 1399 Mathematical Science Bldg., West
Lafayette, Indiana 47907-1399
101

102

L. D. BROWN, T. T. CAI AND A. DASGUPTA

particularly consider the nominal 95% case andshow the erratic and poor behavior of the standard interval’s coverage probability for small neven when p is not near the boundaries. See their
Figure 4 for the cases n = 5 and 10.
We will show in this article that the eccentricbehavior of the standard interval’s coverage probability is far deeper than has been explained or isappreciated by statisticians at large. We will showthat the popular prescriptions the standard interval comes with are defective in several respects andare not to be trusted. In addition, we will motivate, present and analyze several alternatives to thestandard interval for a general conﬁdence level. We
will ultimately make recommendations about choosing a speciﬁc interval for practical use, separately
for different intervals of values of n. It will be seenthat for small n (40 or less), our recommendationdiffers from the recommendation Agresti and Coull(1998) made for the nominal 95% case. To facilitate greater appreciation of the seriousness of the
problem, we have kept the technical content of thisarticle at a minimal level. The companion article,
Brown, Cai and DasGupta (1999), presents the associated theoretical calculations on Edgeworth expansions of the various intervals’ coverage probabilities and asymptotic expansions for their expected
lengths.
In Section 2, we ﬁrst present a series of examples on the degree of severity of the chaotic behavior of the standard interval’s coverage probability.
The chaotic behavior does not go away even whenn is quite large and p is not near the boundaries.
For instance, when n is 100, the actual coverage
probability of the nominal 95% standard interval
is 0.952 if p is 0.106, but only 0.911 if p is 0.107.
The behavior of the coverage probability can be evenmore erratic as a function of n. If the true p is 0.5,
the actual coverage of the nominal 95% interval is0.953 at the rather small sample size n = 17, but
falls to 0.919 at the much larger sample size n = 40.
This eccentric behavior can get downrightextreme in certain practically important problems. For instance, consider defective proportions inindustrial quality control problems. There it would
be quite common to have a true p that is small. If
the true p is 0.005, then the coverage probabilityof the nominal 95% interval increases monotonically in n all the way up to n = 591 to the level
0.945, only to drop down to 0.792 if n is 592. This
unlucky spell continues for a while, and then the
coverage bounces back to 0.948 when n is 953, but
dramatically falls to 0.852 when n is 954. Subsequent unlucky spells start off at n = 1279, 1583 andon and on. It should be widely known that the coverage of the standard interval can be signiﬁcantly

lower at quite large sample sizes, and this happens
in an unpredictable and rather random way.
Continuing, also in Section 2 we list a set of common prescriptions that standard texts present while
discussing the standard interval. We show what
the deﬁciencies are in some of these prescriptions.
Proposition 1 and the subsequent Table 3 illustrate
the defects of these common prescriptions.
In Sections 3 and 4, we present our alternative intervals. For the purpose of a sharper focus
we present these alternative intervals in two categories. First we present in Section 3 a selected
set of three intervals that clearly stand out inour subsequent analysis; we present them as our“recommended intervals.” Separately, we present
several other intervals in Section 4 that arise asclear candidates for consideration as a part of acomprehensive examination, but do not stand out
in the actual analysis.
The short list of recommended intervals contains
the score interval, an interval recently suggested
in Agresti and Coull (1998), and the equal tailed
interval resulting from the natural noninformative Jeffreys prior for a binomial proportion. The
score interval for the binomial case seems tohave been introduced in Wilson (1927); so we callit the Wilson interval. Agresti and Coull (1998)
suggested, for the special nominal 95% case, theinterval p̃ ± z0 025 ñ−1/2 p̃1 − p̃1/2 , where ñ = n + 4
and p̃ = X + 2/n + 4; this is an adjusted Waldinterval that formally adds two successes andtwo failures to the observed counts and then uses
the standard method. Our second interval is theappropriate version of this interval for a general
conﬁdence level; we call it the Agresti–Coull interval. By a slight abuse of terminology, we call ourthird interval, namely the equal-tailed interval
corresponding to the Jeffreys prior, the Jeffreys
interval.
In Section 3, we also present our ﬁndings on theperformances of our “recommended” intervals. As
always, two key considerations are their coverage
properties and parsimony as measured by expected
length. Simplicity of presentation is also sometimes
an issue, for example, in the context of classroom
presentation at an elementary level. On consideration of these factors, we came to the conclusion thatfor small n (40 or less), we recommend that either
the Wilson or the Jeffreys prior interval should
be used. They are very similar, and either may be
used depending on taste. The Wilson interval has aclosed-form formula. The Jeffreys interval does not.
One can expect that there would be resistance tousing the Jeffreys interval solely due to this reason. We therefore provide a table simply listing theINTERVAL ESTIMATION FOR BINOMIAL PROPORTION

limits of the Jeffreys interval for n up to 30 andin addition also give closed form and very accurate
approximations to the limits. These approximations
do not need any additional software.
For larger n n > 40, the Wilson, the Jeffreys
and the Agresti–Coull interval are all very similar, and so for such n, due to its simplest form,
we come to the conclusion that the Agresti–Coullinterval should be recommended. Even for smaller
sample sizes, the Agresti–Coull interval is strongly
preferable to the standard one and so might be thechoice where simplicity is a paramount objective.
The additional intervals we considered are twoslight modiﬁcations of the Wilson and the Jeffreys
intervals, the Clopper–Pearson “exact” interval,
the arcsine interval, the logit interval, the actual
Jeffreys HPD interval and the likelihood ratio
interval. The modiﬁed versions of the Wilson and
the Jeffreys intervals correct disturbing downward
spikes in the coverages of the original intervals very
close to the two boundaries. The other alternativeintervals have earned some prominence in the literature for one reason or another. We had to apply acertain amount of discretion in choosing these additional intervals as part of our investigation. Since
we wish to direct the main part of our conversation
to the three “recommended” intervals, only a briefsummary of the performances of these additional
intervals is presented along with the introductionof each interval. As part of these quick summaries,
we indicate why we decided against including themamong the recommended intervals.
We strongly recommend that introductory texts
in statistics present one or more of these recommended alternative intervals, in preference to thestandard one. The slight sacriﬁce in simplicity
would be more than worthwhile. The conclusions
we make are given additional theoretical support
by the results in Brown, Cai and DasGupta (1999).
Analogous results for other one parameter discrete
families are presented in Brown, Cai and DasGupta(2000).
2. THE STANDARD INTERVAL
When constructing a conﬁdence interval we usually wish the actual coverage probability to be close
to the nominal conﬁdence level. Because of the discrete nature of the binomial distribution we cannot
always achieve the exact nominal conﬁdence levelunless a randomized procedure is used. Thus our
objective is to construct nonrandomized conﬁdence
intervals for p such that the coverage probability
Pp p ∈ CI ≈ 1 − α where α is some prespeciﬁed
value between 0 and 1. We will use the notation

103

Cp n = Pp p ∈ CI 0 < p < 1, for the coverage
probability.
A standard conﬁdence interval for p based on normal approximation has gained universal recommendation in the introductory statistics textbooks andin statistical practice. The interval is known to guarantee that for any ﬁxed p ∈ 0 1 Cp n → 1 − αas n → ∞.
Let φz and z be the standard normal densityand distribution functions, respectively. Throughout
the paper we denote κ ≡ zα/2 = −1 1 − α/2 p̂ =X/n and q̂ = 1 − p̂. The standard normal approximation conﬁdence interval CIs is given by(1)

CIs = p̂ ± κ n−1/2 p̂q̂1/2

This interval is obtained by inverting the acceptance region of the well known Wald large-sample
normal test for a general problem:(2)θ̂ − θ/seθ̂ ≤ κwhere θ is a generic parameter, θ̂ is the maximum
likelihood estimate of θ and seθ̂ is the estimatedstandard error of θ̂. In the binomial case, we have
θ = p θ̂ = X/n and seθ̂ = p̂q̂1/2 n−1/2
The standard interval is easy to calculate andis heuristically appealing. In introductory statistics texts and courses, the conﬁdence interval CIs
is usually presented along with some heuristic justiﬁcation based on the central limit theorem. Most
students and users no doubt believe that the larger
the number n, the better the normal approximation,and thus the closer the actual coverage would be to
the nominal level 1 − α. Further, they would believethat the coverage probabilities of this method areclose to the nominal value, except possibly when n
is “small” or p is “near” 0 or 1. We will show how
completely both of these beliefs are false. Let us
take a close look at how the standard interval CIs
really performs.
2.1 Lucky n, Lucky p
An interesting phenomenon for the standard
interval is that the actual coverage probabilityof the conﬁdence interval contains nonnegligible
oscillation as both p and n vary. There exist some“lucky” pairs p n such that the actual coverage
probability Cp n is very close to or larger than
the nominal level. On the other hand, there also
exist “unlucky” pairs p n such that the corresponding Cp n is much smaller than the nominal
level. The phenomenon of oscillation is both in n,
for ﬁxed p, and in p, for ﬁxed n. Furthermore, drastic changes in coverage occur in nearby p for ﬁxed
n and in nearby n for ﬁxed p. Let us look at ﬁve
simple but instructive examples.

104

L. D. BROWN, T. T. CAI AND A. DASGUPTA

Fig. 1.

Standard interval; oscillation phenomenon for ﬁxed p = 0 2 and variable n = 25 to 100

The probabilities reported in the following plots
and tables, as well as those appearing later inthis paper, are the result of direct probability
calculations produced in S-PLUS. In all cases
their numerical accuracy considerably exceeds thenumber of signiﬁcant ﬁgures reported and/or theaccuracy visually obtainable from the plots. (Plotsfor variable p are the probabilities for a ﬁne gridof values of p, e.g., 2000 equally spaced values of p
for the plots in Figure 5.)
Example 1. Figure 1 plots the coverage probability of the nominal 95% standard interval for
p = 0 2. The number of trials n varies from 25 to100. It is clear from the plot that the oscillation issigniﬁcant and the coverage probability does notsteadily get closer to the nominal conﬁdence levelas n increases. For instance, C0 2 30 = 0 946 andC0 2 98 = 0 928. So, as hard as it is to believe,
the coverage probability is signiﬁcantly closer to0.95 when n = 30 than when n = 98. We see thatthe true coverage probability behaves contrary toconventional wisdom in a very signiﬁcant way.
Example 2. Now consider the case of p = 0 5.
Since p = 0 5, conventional wisdom might suggest
to an unsuspecting user that all will be well if n isabout 20. We evaluate the exact coverage probability of the 95% standard interval for 10 ≤ n ≤ 50.
In Table 1, we list the values of “lucky” n [deﬁned
as Cp n ≥ 0 95] and the values of “unlucky” n[deﬁned for speciﬁcity as Cp n ≤ 0 92]. The conclusions presented in Table 2 are surprising. We

note that when n = 17 the coverage probability
is 0.951, but the coverage probability equals 0.904
when n = 18. Indeed, the unlucky values of n arise
suddenly. Although p is 0.5, the coverage is still
only 0.919 at n = 40. This illustrates the inconsistency, unpredictability and poor performance of thestandard interval.
Example 3. Now let us move p really close to
the boundary, say p = 0 005. We mention in the
introduction that such p are relevant in certain
practical applications. Since p is so small, now one
may fully expect that the coverage probability ofthe standard interval is poor. Figure 2 and Table
2.2 show that there are still surprises and indeed
we now begin to see a whole new kind of erratic
behavior. The oscillation of the coverage probability does not show until rather large n. Indeed, thecoverage probability makes a slow ascent all theway until n = 591, and then dramatically drops to
0.792 when n = 592. Figure 2 shows that thereafter
the oscillation manifests in full force, in contrast
to Examples 1 and 2, where the oscillation started
early on. Subsequent “unlucky” values of n again
arise in the same unpredictable way, as one can see
from Table 2.2.
2.2 Inadequate Coverage
The results in Examples 1 to 3 already show thatthe standard interval can have coverage noticeably
smaller than its nominal value even for values of nand of np1 − p that are not small. This subsec-

Table 1
Standard interval; lucky n and unlucky n for 10 ≤ n ≤ 50 and p = 0 5
Lucky nC0 5 n

17
0.951

20
0.959

25
0.957

30

.957

35
0.959

37
0.953

42
0.956

44
0.951

49
0.956

Unlucky nC0 5 n

10
0.891

12
0.854

13
0.908

15
0.882

18
0.904

23
0.907

28
0.913

33
0.920

40
0.919

INTERVAL ESTIMATION FOR BINOMIAL PROPORTION
Table 2Standard interval; late arrival of unlucky n for small p
Unlucky nC0 005 n

592
0.792

954
0.852

1279
0.875

1583
0.889

1876
0.898

tion contains two more examples that display further instances of the inadequacy of the standard
interval.
Example 4. Figure 3 plots the coverage probability of the nominal 95% standard interval with ﬁxed
n = 100 and variable p. It can be seen from Figure 3 that in spite of the “large” sample size, signiﬁcant change in coverage probability occurs in nearby
p. The magnitude of oscillation increases signiﬁcantly as p moves toward 0 or 1. Except for valuesof p quite near p = 0 5, the general trend of thisplot is noticeably below the nominal coverage valueof 0 95.
Example 5. Figure 4 shows the coverage probability of the nominal 99% standard interval with n =20 and variable p from 0 to 1. Besides the oscillation phenomenon similar to Figure 3, a striking fact
in this case is that the coverage never reaches thenominal level. The coverage probability is always
smaller than 0.99, and in fact on the average thecoverage is only 0.883. Our evaluations show thatfor all n ≤ 45, the coverage of the 99% standard
interval is strictly smaller than the nominal level
for all 0 < p < 1.
It is evident from the preceding presentationthat the actual coverage probability of the standard
interval can differ signiﬁcantly from the nominal
conﬁdence level for moderate and even large sample sizes. We will later demonstrate that there areother conﬁdence intervals that perform much better

Fig. 2.

105

in this regard. See Figure 5 for such a comparison.
The error in coverage comes from two sources: discreteness and skewness in the underlying binomial
distribution. For a two-sided interval, the rounding
error due to discreteness is dominant, and the error
due to skewness is somewhat secondary, but still
important for even moderately large n. (See Brown,
Cai and DasGupta, 1999, for more details.) Notethat the situation is different for one-sided intervals. There, the error caused by the skewness canbe larger than the rounding error. See Hall (1982)
for a detailed discussion on one-sided conﬁdence
intervals.
The oscillation in the coverage probability iscaused by the discreteness of the binomial distribution, more precisely, the lattice structure of
the binomial distribution. The noticeable oscillations are unavoidable for any nonrandomized
procedure, although some of the competing procedures in Section 3 can be seen to have somewhat
smaller oscillations than the standard procedure.
See the text of Casella and Berger (1990) for introductory discussion of the oscillation in such acontext.
The erratic and unsatisfactory coverage properties of the standard interval have often beenremarked on, but curiously still do not seem to
be widely appreciated among statisticians. See, forexample, Ghosh (1979), Blyth and Still (1983) andAgresti and Coull (1998). Blyth and Still (1983) also
show that the continuity-corrected version still has
the same disadvantages.
2.3 Textbook Qualiﬁcations
The normal approximation used to justify thestandard conﬁdence interval for p can be signiﬁcantly in error. The error is most evident when thetrue p is close to 0 or 1. See Lehmann (1999). In
fact, it is easy to show that, for any ﬁxed n, theStandard interval; oscillation in coverage for small p

106

L. D. BROWN, T. T. CAI AND A. DASGUPTA

Fig. 3.

Standard interval; oscillation phenomenon for ﬁxed n = 100 and variable p

conﬁdence coefﬁcient Cp n → 0 as p → 0 or 1.
Therefore, most major problems arise as regards
coverage probability when p is near the boundaries.
Poor coverage probabilities for p near 0 or 1 arewidely remarked on, and generally, in the popular texts, a brief sentence is added qualifying when
to use the standard conﬁdence interval for p. It
is interesting to see what these qualiﬁcations are.
A sample of 11 popular texts gives the following
qualiﬁcations:
The conﬁdence interval may be used if:
1. np n1 − p are ≥ 5 (or 10);
2. np1 − p ≥ 5 (or 10);
3. np̂ n1
 − p̂ are ≥ 5 (or 10);
4. p̂ ± 3 p̂1 − p̂/n does not contain 0 or 1;
5. n quite large;
6. n ≥ 50 unless p is very small.
It seems clear that the authors are attempting to
say that the standard interval may be used if thecentral limit approximation is accurate. These prescriptions are defective in several respects. In theestimation problem, (1) and (2) are not veriﬁable.
Even when these conditions are satisﬁed, we see,for instance, from Table 1 in the previous section,that there is no guarantee that the true coverage
probability is close to the nominal conﬁdence level.

Fig. 4.

For example, when n = 40 and p = 0 5, one has
np = n1 − p = 20 and np1 − p = 10, so clearly
either of the conditions (1) and (2) is satisﬁed. However, from Table 1, the true coverage probability inthis case equals 0.919 which is certainly unsatisfactory for a conﬁdence interval at nominal level 0.95.
The qualiﬁcation (5) is useless and (6) is patently
misleading; (3) and (4) are certainly veriﬁable, butthey are also useless because in the context of frequentist coverage probabilities, a data-based prescription does not have a meaning. The point is thatthe standard interval clearly has serious problems
and the inﬂuential texts caution the readers about
that. However, the caution does not appear to serve
its purpose, for a variety of reasons.
Here is a result that shows that sometimes thequaliﬁcations are not correct even in the limit as
n → ∞.
Proposition 1. Let γ > 0. For the standard conﬁdence interval,
(3)

lim

inf

n→∞ p np n1−p≥γCp n≤ Paγ < Poissonγ ≤ bγ 

Coverage of the nominal 99% standard interval for ﬁxed n = 20 and variable p.

INTERVAL ESTIMATION FOR BINOMIAL PROPORTION

Fig. 5.

Coverage probability for n = 50.

Table 3Standard interval; bound (3) on limiting minimum coveragewhen np n1 − p ≥ γ

lim

inf

n→∞ p np n1−p≥γCp n

107

5

7

10

0.875

0.913

0.926

where aγ and bγ are the integer parts ofκ2 + 2γ ± κ κ2 + 4γ/2
where the − sign goes with aγ and the + sign with bγ .
The proposition follows from the fact that thesequence of Binn γ/n distributions converges
weakly to the Poisson(γ) distribution and so thelimit of the inﬁmum is at most the Poisson probability in the proposition by an easy calculation.
Let us use Proposition 1 to investigate the validityof qualiﬁcations (1) and (2) in the list above. The
nominal conﬁdence level in Table 3 below is 0.95.
Table 4
Values of λx for the modiﬁed lower bound for the Wilson interval
1−

x=1

x=2

x=3

0.90
0.95
0.99

0.105
0.051
0.010

0.532
0.355
0.149

1.102
0.818
0.436

It is clear that qualiﬁcation (1) does not work at
all and (2) is marginal. There are similar problems
with qualiﬁcations (3) and (4).
3. RECOMMENDED ALTERNATIVE INTERVALS
From the evidence gathered in Section 2, it seems
clear that the standard interval is just too risky.
This brings us to the consideration of alternativeintervals. We now analyze several such alternatives,
each with its motivation. A few other intervals arealso mentioned for their theoretical importance.
Among these intervals we feel three stand out intheir comparative performance. These are labeledseparately as the “recommended intervals”.
3.1 Recommended Intervals
3.1.1 The Wilson interval. An alternative to thestandard interval is the conﬁdence interval basedon inverting the test in equation (2) that uses thenull standard error pq1/2 n−1/2 instead of the estimated standard error p̂q̂1/2 n−1/2 . This conﬁdence
interval has the form
X + κ2 /2
κn1/2(4) CIW =±p̂q̂ + κ2 /4n1/2
n + κ2n + κ2This interval was apparently introduced by Wilson(1927) and we will call this interval the Wilson
interval.
The Wilson interval has theoretical appeal. Theinterval is the inversion of the CLT approximation

108

L. D. BROWN, T. T. CAI AND A. DASGUPTA

to the family of equal tail tests of H0  p = p0 .
Hence, one accepts H0 based on the CLT approximation if and only if p0 is in this interval. As
Wilson showed, the argument involves the solutionof a quadratic equation; or see Tamhane and Dunlop(2000, Exercise 9.39).

the Jeffreys prior is Beta1/2 1/2 which has thedensity function

3.1.2 The Agresti–Coull interval. The standard
interval CIs is simple and easy to remember. For
the purposes of classroom presentation and use intexts, it may be nice to have
 an alternative that has
the familiar form p̂ ± z p̂1 − p̂/n, with a betterand new choice of p̂ rather than p̂ = X/n. This canbe accomplished by using the center of the Wilson
 = X + κ2 /2 andregion in place of p̂. Denote X
2ñ = n + κ . Let p̃ = X/ñ and q̃ = 1 − p̃. Deﬁne theconﬁdence interval CIAC for p by

(6)

(5)

CIAC = p̃ ± κp̃q̃1/2 ñ−1/2

Both the Agresti–Coull and the Wilson interval are
centered on the same value, p̃. It is easy to checkthat the Agresti–Coull intervals are never shorter
than the Wilson intervals. For the case when α =0 05, if we use the value 2 instead of 1.96 for κ,
this interval is the “add 2 successes and 2 failures”
interval in Agresti and Coull (1998). For this reason, we call it the Agresti–Coull interval. To thebest of our knowledge, Samuels and Witmer (1999)
is the ﬁrst introductory statistics textbook that recommends the use of this interval. See Figure 5 forthe coverage of this interval. See also Figure 6 forits average coverage probability.
3.1.3 Jeffreys interval. Beta distributions are thestandard conjugate priors for binomial distributions
and it is quite common to use beta priors for inference on p (see Berger, 1985).
Suppose X ∼ Binn p and suppose p has a priordistribution Betaa1  a2 ; then the posterior distribution of p is BetaX + a1  n − X + a2 . Thus a1001 − α% equal-tailed Bayesian interval is givenbyBα/2 X + a1  n − X + a2 B1 − α/2 X + a1  n − X + a2 
where Bα m1  m2  denotes the α quantile of a
Betam1  m2  distribution.
The well-known Jeffreys prior and the uniform
prior are each a beta distribution. The noninformative Jeffreys prior is of particular interest to us.
Historically, Bayes procedures under noninformative priors have a track record of good frequentist
properties; see Wasserman (1991). In this problemfp = π −1 p−1/2 1 − p−1/2
The 1001 − α% equal-tailed Jeffreys prior interval
is deﬁned as
CIJ = LJ x UJ x

where LJ 0 = 0 UJ n = 1 and otherwise(7) LJ x = Bα/2 X + 1/2 n − X + 1/2
(8) UJ x = B1 − α/2 X + 1/2 n − X + 1/2The interval is formed by taking the central 1 − αposterior probability interval. This leaves α/2 posterior probability in each omitted tail. The exception
is for x = 0n where the lower (upper) limits are
modiﬁed to avoid the undesirable result that thecoverage probability Cp n → 0 as p → 0 or 1.
The actual endpoints of the interval need to be
numerically computed. This is very easy to do using
softwares such as Minitab, S-PLUS or Mathematica.
In Table 5 we have provided the limits for the caseof the Jeffreys prior for 7 ≤ n ≤ 30.
The endpoints of the Jeffreys prior interval are
the α/2 and 1−α/2 quantiles of the Betax+1/2 n−
x + 1/2 distribution. The psychological resistanceamong some to using the interval is because of theinability to compute the endpoints at ease without
software.
We provide two avenues to resolving this problem.
One is Table 5 at the end of the paper. The second
is a computable approximation to the limits of theJeffreys prior interval, one that is computable withjust a normal table. This approximation is obtained
after some algebra from the general approximation
to a Beta quantile given in page 945 in Abramowitz
and Stegun (1970).
The lower limit of the 1001 − α% Jeffreys priorinterval is approximately(9)

x + 1/2
n + 1 + n − x + 1/2e2ω − 1

where
ω=


κ 4p̂q̂/n + κ2 − 3/6n2 
4p̂q̂+

1/2 − p̂p̂q̂κ2 + 2 − 1/n6np̂q̂2

The upper limit may be approximated by the same
expression with κ replaced by −κ in ω. The simple
approximation given above is remarkably accurate.
Berry (1996, page 222) suggests using a simpler normal approximation, but this will not be sufﬁciently
accurate unless np̂1 − p̂ is rather large.

109

INTERVAL ESTIMATION FOR BINOMIAL PROPORTION
Table 595% Limits of the Jeffreys prior interval
x
0
1
2
3
4
5
6
x
0
1
2
3
4
5
6
7
8
9
x
0
1
2
3
4
5
6
7
8
9
10
11
12
x
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

n=7
0
0.016
0.065
0.139
0.234

0.292
0.501
0.648
0.766
0.861

n = 13
0
0.008
0.033
0.070
0.114
0.165
0.221
0.283

0.173
0.307
0.409
0.497
0.577
0.650
0.717
0.779

n = 19
0
0.006
0.022
0.047
0.076
0.108
0.144
0.182
0.223
0.266
0.312

0.122
0.221
0.297
0.364
0.426
0.484
0.539
0.591
0.641
0.688
0.734

n = 25
0
0.004
0.017
0.035
0.056
0.081
0.107
0.135
0.164
0.195
0.228
0.261
0.295
0.331

0.095
0.172
0.233
0.287
0.337
0.384
0.429
0.473
0.515
0.555
0.594
0.632
0.669
0.705

n=8
0
0.014
0.056
0.119
0.199

0.262
0.454
0.592
0.705
0.801

n = 14
0
0.008
0.031
0.064
0.105
0.152
0.203
0.259

0.162
0.288
0.385
0.469
0.545
0.616
0.681
0.741

n = 20
0
0.005
0.021
0.044
0.072
0.102
0.136
0.172
0.211
0.251
0.293

0.117
0.211
0.284
0.349
0.408
0.464
0.517
0.568
0.616
0.662
0.707

n = 26
0
0.004
0.016
0.034
0.054
0.077
0.102
0.129
0.158
0.187
0.218
0.250
0.282
0.316

0.091
0.166
0.225
0.277
0.325
0.371
0.415
0.457
0.498
0.537
0.576
0.613
0.649
0.684

n=9
0
0.012
0.049
0.104
0.173
0.254

0.238
0.414
0.544
0.652
0.746
0.827

n = 15
0
0.007
0.029
0.060
0.097
0.140
0.188
0.239
0.294

0.152
0.272
0.363
0.444
0.517
0.584
0.647
0.706
0.761

n = 21
0
0.005
0.020
0.042
0.068
0.097
0.129
0.163
0.199
0.237
0.277
0.319

0.112
0.202
0.272
0.334
0.392
0.446
0.497
0.546
0.593
0.638
0.681
0.723

n = 27
0
0.004
0.016
0.032
0.052
0.074
0.098
0.124
0.151
0.180
0.209
0.239
0.271
0.303
0.336

0.088
0.160
0.217
0.268
0.315
0.359
0.402
0.443
0.482
0.521
0.558
0.594
0.630
0.664
0.697

n = 10
0
0.011
0.044
0.093
0.153
0.224

0.217
0.381
0.503
0.606
0.696
0.776

n = 16
0
0.007
0.027
0.056
0.091
0.131
0.174
0.222
0.272

0.143
0.257
0.344
0.421
0.491
0.556
0.617
0.674
0.728

n = 22
0
0.005
0.019
0.040
0.065
0.092
0.123
0.155
0.189
0.225
0.263
0.302

0.107
0.193
0.261
0.321
0.376
0.429
0.478
0.526
0.571
0.615
0.657
0.698

n = 28
0
0.004
0.015
0.031
0.050
0.072
0.095
0.119
0.145
0.172
0.201
0.230
0.260
0.291
0.322

0.085
0.155
0.210
0.259
0.305
0.348
0.389
0.429
0.468
0.505
0.542
0.577
0.611
0.645
0.678

n = 11
0
0.010
0.040
0.084
0.137
0.200
0.270

0.200
0.353
0.467
0.565
0.652
0.730
0.800

n = 17
0
0.006
0.025
0.052
0.085
0.122
0.163
0.207
0.254
0.303

0.136
0.244
0.327
0.400
0.467
0.530
0.589
0.644
0.697
0.746

n = 23
0
0.005
0.018
0.038
0.062
0.088
0.117
0.148
0.180
0.214
0.250
0.287
0.325

0.102
0.186
0.251
0.309
0.362
0.413
0.461
0.507
0.551
0.594
0.635
0.675
0.713

n = 29
0
0.004
0.015
0.030
0.048
0.069
0.091
0.115
0.140
0.166
0.193
0.221
0.250
0.279
0.310
0.341

0.082
0.150
0.203
0.251
0.295
0.337
0.378
0.416
0.454
0.490
0.526
0.560
0.594
0.627
0.659
0.690

n = 12
0
0.009
0.036
0.076
0.124
0.180
0.243

0.185
0.328
0.436
0.529
0.612
0.688
0.757

n = 18
0
0.006
0.024
0.049
0.080
0.115
0.153
0.194
0.237
0.284

0.129
0.232
0.311
0.381
0.446
0.506
0.563
0.617
0.668
0.716

n = 24
0
0.004
0.018
0.036
0.059
0.084
0.112
0.141
0.172
0.204
0.238
0.273
0.310

0.098
0.179
0.241
0.297
0.349
0.398
0.444
0.489
0.532
0.574
0.614
0.653
0.690

n = 30
0
0.004
0.014
0.029
0.047
0.067
0.088
0.111
0.135
0.160
0.186
0.213
0.240
0.269
0.298
0.328

0.080
0.145
0.197
0.243
0.286
0.327
0.367
0.404
0.441
0.476
0.511
0.545
0.578
0.610
0.641
0.672

110

L. D. BROWN, T. T. CAI AND A. DASGUPTA

Fig. 6. Comparison of the average coverage probabilities. From top to bottom: the Agresti–Coull interval CIAC  the Wilson interval CIW 
the Jeffreys prior interval CIJ and the standard interval CIs . The nominal conﬁdence level is 0 95

In Figure 5 we plot the coverage probability of thestandard interval, the Wilson interval, the Agresti–
Coull interval and the Jeffreys interval for n = 50
and α = 0 05.
3.2 Coverage Probability
In this and the next subsections, we compare theperformance of the standard interval and the threerecommended intervals in terms of their coverage
probability and length.
Coverage of the Wilson interval ﬂuctuates acceptably near 1 − α, except for p very near 0 or 1. It
might be helpful to consult Figure 5 again. It canbe shown that, when 1 − α = 0 95,γ
lim inf C
 n = 0 92
n→∞ γ≥1nγ
lim inf C
 n = 0 936
n→∞ γ≥5
n
and




γ
lim inf C
 n = 0 938
n→∞ γ≥10n

for the Wilson interval. In comparison, these threevalues for the standard interval are 0.860, 0.870,
and 0.905, respectively, obviously considerably
smaller.
The modiﬁcation CIM−W presented in Section
4.1.1 removes the ﬁrst few deep downward spikesof the coverage function for CIW . The resulting coverage function is overall somewhat conservative forp very near 0 or 1. Both CIW and CIM−W have thesame coverage functions away from 0 or 1.
The Agresti–Coull interval has good minimum
coverage probability. The coverage probability of
the interval is quite conservative for p very close
to 0 or 1. In comparison to the Wilson interval it
is more conservative, especially for small n. This
is not surprising because, as we have noted, CIAC
always contains CIW as a proper subinterval.

The coverage of the Jeffreys interval is qualitatively similar to that of CIW over most of theparameter space 0 1. In addition, as we will see
in Section 4.3, CIJ has an appealing connection to
the mid-P corrected version of the Clopper–Pearson“exact” intervals. These are very similar to CIJ ,over most of the range, and have similar appealing
properties. CIJ is a serious and credible candidate
for practical use. The coverage has an unfortunate
fairly deep spike near p = 0 and, symmetrically,another near p = 1. However, the simple modiﬁcation of CIJ presented in Section 4.1.2 removes thesetwo deep downward spikes. The modiﬁed Jeffreys
interval CIM−J performs well.
Let us also evaluate the intervals in terms of theiraverage coverage probability, the average being overp. Figure 6 demonstrates the striking difference inthe average coverage probability among four intervals: the Agresti–Coull interval, the Wilson interval
the Jeffreys prior interval and the standard interval. The standard interval performs poorly. Theinterval CIAC is slightly conservative in terms of
average coverage probability. Both the Wilson interval and the Jeffreys prior interval have excellent
performance in terms of the average coverage probability; that of the Jeffreys prior interval is, ifanything, slightly superior. The average coverageof the Jeffreys interval is really very close to thenominal level even for quite small n. This is quite
impressive.
 1Figure 7 displays the mean absolute errors,
0 Cp n − 1 − α dp, for n = 10 to 25, andn = 26 to 40. It is clear from the plots that among
the four intervals, CIW  CIAC and CIJ are comparable, but the mean absolute errors of CIs are
signiﬁcantly larger.
3.3 Expected Length
Besides coverage, length is also very importantin evaluation of a conﬁdence interval. We compareINTERVAL ESTIMATION FOR BINOMIAL PROPORTION

111

Fig. 7. The mean absolute errors of the coverage of the standard solid the Agresti–Coull dashed the Jeffreys + and the Wilson
dotted intervals for n = 10 to 25 and n = 26 to 40

both the expected length and the average expected
length of the intervals. By deﬁnition,
Expected length= En p lengthCI n=Ux n − Lx npx 1 − pn−x 
x
x=0
n


where U and L are the upper and lower limits of the conﬁdence interval CI, respectively.
The 1 average expected length is just the integral
0 En p length(CI) dp.
We plot in Figure 8 the expected lengths of thefour intervals for n = 25 and α = 0 05. In this case,
CIW is the shortest when 0 210 ≤ p ≤ 0 790, CIJ isthe shortest when 0 133 ≤ p ≤ 0 210 or 0 790 ≤ p ≤0 867, and CIs is the shortest when p ≤ 0 133 or p ≥0 867. It is no surprise that the standard interval is
the shortest when p is near the boundaries. CIs is
not really in contention as a credible choice for suchvalues of p because of its poor coverage properties
in that region. Similar qualitative phenomena holdfor other values of n.
Figure 9 shows the average expected lengths of
the four intervals for n = 10 to 25 and n = 26 to

40. Interestingly, the comparison is clear and consistent as n changes. Always, the standard interval
and the Wilson interval CIW have almost identical
average expected length; the Jeffreys interval CIJ is
comparable to the Wilson interval, and in fact CIJ
is slightly more parsimonious. But the difference is
not of practical relevance. However, especially whenn is small, the average expected length of CIAC isnoticeably larger than that of CIJ and CIW . In fact,
for n till about 20, the average expected length of
CIAC is larger than that of CIJ by 0.04 to 0.02, andthis difference can be of deﬁnite practical relevance.
The difference starts to wear off when n is larger
than 30 or so.
4. OTHER ALTERNATIVE INTERVALS
Several other intervals deserve consideration,either due to their historical value or their theoretical properties. In the interest of space, we had toexercise some personal judgment in deciding which
additional intervals should be presented.
4.1 Boundary modiﬁcation
The coverage probabilities of the Wilson interval
and the Jeffreys interval ﬂuctuate acceptably nearFig. 8. The expected lengths of the standard solid the Wilson dotted the Agresti–Coull dashed and the Jeffreys + intervals forn = 25 and α = 0 05.

112

L. D. BROWN, T. T. CAI AND A. DASGUPTA

Fig. 9. The average expected lengths of the standard solid the Wilson dotted the Agresti–Coull dashed and the Jeffreys +intervals for n = 10 to 25 and n = 26 to 40.

1 − α for p not very close to 0 or 1. Simple modiﬁcations can be made to remove a few deep downward
spikes of their coverage near the boundaries; see
Figure 5.
4.1.1 Modiﬁed Wilson interval. The lower boundof the Wilson interval is formed by inverting a CLT
approximation. The coverage has downward spikes
when p is very near 0 or 1. These spikes exist for alln and α. For example, it can be shown that, when
1 − α = 0 95 and p = 0 1765/n,
lim Pp p ∈ CIW  = 0 838

n→∞and when 1 − α = 0 99 and p = 0 1174/n
limn→∞ Pp p ∈ CIW  = 0 889 The particular
numerical values 0 1174 0 1765 are relevant only
to the extent that divided by n, they approximate
the location of these deep downward spikes.
The spikes can be removed by using a one-sided
Poisson approximation for x close to 0 or n. Suppose
we modify the lower bound for x = 1 x∗ . For aﬁxed 1 ≤ x ≤ x∗ , the lower bound of CIW should be

replaced by a lower bound of λx /n where λx solves(10) e−λ λ0 /0! + λ1 /1! + · · · + λx−1 /x − 1! = 1 − αA symmetric prescription needs to be followed tomodify the upper bound for x very near n. The valueof x∗ should be small. Values which work reasonably
well for 1 − α = 0 95 arex∗ = 2 for n < 50 and x∗ = 3 for 51 ≤ n ≤ 100+.
Using the relationship between the Poisson andχ2 distributions,PY ≤ x = Pχ221+x ≤ 2λwhere Y ∼ Poissonλ, one can also formally
express λx in (10) in terms of the χ2 quantiles:
λx = 1/2χ22x α  where χ22x α denotes the 100αth
percentile of the χ2 distribution with 2x degrees of
freedom. Table 4 gives the values of λx for selectedvalues of x and α.
For example, consider the case 1 − α = 0 95 andx = 2. The lower bound of CIW is ≈ 0 548/n +4. The modiﬁed Wilson interval replaces this by alower bound of λ/n where λ = 1/2 χ24 0 05 . Thus,

Fig. 10. Coverage probability for n = 50 and p ∈ 0 0 15. The plots are symmetric about p = 0 5 and the coverage of the modiﬁed intervals
solid line is the same as that of the corresponding interval without modiﬁcation dashed line for p ∈ 0 15 0 85.

113

INTERVAL ESTIMATION FOR BINOMIAL PROPORTION

Fig. 11. Coverage probability of other alternative intervals for n = 50.

from a χ2 table, for x = 2 the new lower bound is
0 355/n.
We denote this modiﬁed Wilson interval by
CIM−W . See Figure 10 for its coverage.
4.1.2 Modiﬁed Jeffreys interval. Evidently, CIJ
has an appealing Bayesian interpretation, and,its coverage properties are appealing again except
for a very narrow downward coverage spike fairly
near 0 and 1 (see Figure 5). The unfortunate downward spikes in the coverage function result because
UJ 0 is too small and symmetrically LJ n is toolarge. To remedy this, one may revise these two
speciﬁc limits as
UM−J 0 = pland LM−J n = 1 − pl 

where pl satisﬁes 1 − pl n = α/2 or equivalently
pl = 1 − α/21/n .
We also made a slight, ad hoc alteration of LJ 1
and set
LM−J 1 = 0

and UM−J n − 1 = 1

In all other cases, LM−J = LJ and UM−J = UJ .
We denote the modiﬁed Jeffreys interval by CIM−J .
This modiﬁcation removes the two steep downward spikes and the performance of the interval isimproved. See Figure 10.

4.2 Other intervals
4.2.1 The Clopper–Pearson interval. The Clopper–
Pearson interval is the inversion of the equal-tailbinomial test rather than its normal approximation. Some authors refer to this as the “exact”
procedure because of its derivation from the binomial distribution. If X = x is observed, thenthe Clopper–Pearson (1934) interval is deﬁned by
CICP = LCP x UCP x, where LCP x and UCP x
are, respectively, the solutions in p to the equations
Pp X ≥ x = α/2

andPp X ≤ x = α/2

It is easy to show that the lower endpoint is the α/2quantile of a beta distribution Betax n − x + 1,and the upper endpoint is the 1 − α/2 quantile of abeta distribution Betax + 1 n − x. The Clopper–
Pearson interval guarantees that the actual coverage probability is always equal to or above thenominal conﬁdence level. However, for any ﬁxed p,
the actual coverage probability can be much larger
than 1−α unless n is quite large, and thus the conﬁdence interval is rather inaccurate in this sense. See
Figure 11. The Clopper–Pearson interval is wastefully conservative and is not a good choice for practical use, unless strict adherence to the prescriptionCp n ≥ 1−α is demanded. Even then, better exact
methods are available; see, for instance, Blyth andStill (1983) and Casella (1986).

114

L. D. BROWN, T. T. CAI AND A. DASGUPTA

4.2.2 The arcsine interval. Another interval isbased on a widely used variance stabilizing transformation for the binomial distribution [see, e.g.,
Bickel and Doksum, 1977: Tp̂ = arcsinp̂1/2 This variance stabilization is based on the delta
method and is, of course, only an asymptotic one.
Anscombe (1948) showed that replacing p̂ byp̌ = X + 3/8/n + 3/4 gives better variancestabilization; furthermore
2n1/2 arcsinp̌1/2  − arcsinp1/2  → N0 1
as n → ∞.

The interval (13) has been suggested, for example,in Stone (1995, page 667). Figure 11 plots the coverage of the logit interval for n = 50. This interval
performs quite well in terms of coverage for p awayfrom 0 or 1. But the interval is unnecessarily long;in fact its expected length is larger than that of the
Clopper–Pearson exact interval.
Remark. Anscombe (1956) suggested that λ̂ =X+1/2
 is a better estimate of λ; see also Coxlog n−X+1/2
and Snell (1989) and Santner and Duffy (1989). The
variance of Anscombe’s λ̂ may be estimated byThis leads to an approximate 1001−α% conﬁdence
interval for p,(11)

CIArc = sin2 arcsinp̌1/2  − 12 κn−1/2 sin2 arcsinp̌1/2  + 12 κn−1/2 

See Figure 11 for the coverage probability of thisinterval for n = 50. This interval performs reasonably well for p not too close to 0 or 1. The coverage
has steep downward spikes near the two edges; infact it is easy to see that the coverage drops to zero
when p is sufﬁciently close to the boundary (seeFigure 11). The mean absolute error of the coverageof CIArc is signiﬁcantly larger than those of CIW ,CIAC and CIJ . We note that our evaluations showthat the performance of the arcsine interval withthe standard p̂ in place of p̌ in (11) is much worse
than that of CIArc .
4.2.3 The logit interval. The logit interval is
obtained by inverting a Wald type interval for theplog odds λ = log 1−p
; (see Stone, 1995). The MLEof λ (for 0 < X < n) is




p̂
X
λ̂ = log= log
1 − p̂
n−X
which is the so-called empirical logit transform. The
variance of λ̂, by an application of the delta theorem,
can be estimated byn=VXn − XThis leads to an approximate 1001−α% conﬁdence
interval for λ,1/2  λ̂ + κV
1/2 
(12) CIλ = λl  λu  = λ̂ − κV
The logit interval for p is obtained by inverting theinterval (12),
(13)

eλ l
eλu
CILogit =
1 + e λl 1 + e λu=Vn + 1n + 2nX + 1n − X + 1A new logit interval can be constructed using the Our evaluations show thatnew estimates λ̂ and V.
the new logit interval is overall shorter than CILogitin (13). But the coverage of the new interval is notsatisfactory.
4.2.4 The Bayesian HPD interval. An exact
Bayesian solution would involve using the HPDintervals instead of our equal-tails proposal. However, HPD intervals are much harder to computeand do not do as well in terms of coverage probability. See Figure 11 and compare to the Jeffreys’
equal-tailed interval in Figure 5.
4.2.5 The likelihood ratio interval. Along with
the Wald and the Rao score intervals, the likelihood ratio method is one of the most used methods
for construction of conﬁdence intervals. It is constructed by inversion of the likelihood ratio test
which accepts the null hypothesis H0  p = p0 if−2 log2n  ≤ κ2 , where 2n is the likelihood ratio
2n =

n−X
Lp0 
pX
0 1 − p0 =Xsupp Lp
X/n 1 − X/nn−X

L being the likelihood function. See Rao (1973).
Brown, Cai and DasGupta (1999) show by analytical calculations that this interval has nice properties. However, it is slightly harder to compute. For
the purpose of the present article which we view as
primarily directed toward practice, we do not further analyze the likelihood ratio interval.
4.3 Connections between Jeffreys Intervals
and Mid-P Intervals
The equal-tailed Jeffreys prior interval has someinteresting connections to the Clopper–Pearson
interval. As we mentioned earlier, the Clopper–

INTERVAL ESTIMATION FOR BINOMIAL PROPORTION

Pearson interval CICP can be written as
CICP = Bα/2 X n − X + 1B1 − α/2 X + 1 n − X
It therefore follows immediately that CIJ is always
contained in CICP . Thus CIJ corrects the conservativeness of CICP .
It turns out that the Jeffreys prior interval,
although Bayesianly constructed, has a clear andconvincing frequentist motivation. It is thus no surprise that it does well from a frequentist perspective. As we now explain, the Jeffreys prior interval
CIJ can be regarded as a continuity correctedversion of the Clopper–Pearson interval CICP .
The interval CICP inverts the inequality Pp X ≤Lp ≤ α/2 to obtain the lower limit and similarly
for the upper limit. Thus, for ﬁxed x, the upper limitof the interval for p, UCP x, satisﬁes(14)

PUCP x X ≤ x ≤ α/2and symmetrically for the lower limit.
This interval is very conservative; undesirably sofor most practical purposes. A familiar proposal toeliminate this over-conservativeness is to instead
invert(15) Pp X ≤ Lp−1+1/2Pp X = Lp = α/2This amounts to solving(16)

1/2PUCP x X ≤ x − 1
+ PUCP x X ≤ x = α/2which is the same as(17)

Umid-P X = 1/2B1 − α/2 x n − x + 1
+ 1/2B1 − α/2 x + 1 n − xand symmetrically for the lower endpoint. These
are the “Mid-P Clopper-Pearson” intervals. They are
known to have good coverage and length performance. Umid-P given in (17) is a weighted averageof two incomplete Beta functions. The incomplete
Beta function of interest, B1 − α/2 x n − x + 1, iscontinuous and monotone in x if we formally treat
x as a continuous argument. Hence the average ofthe two functions deﬁning Umid-P is approximately
the same as the value at the halfway point, x + 1/2.
Thus
Umid-P X ≈ B1−α/2x+1/2n−x+1/2 = UJ xexactly the upper limit for the equal-tailed Jeffreys
interval. Similarly, the corresponding approximate
lower endpoint is the Jeffreys’ lower limit.
Another frequentist way to interpret the Jeffreys
prior interval is to say that UJ x is the upper

115

limit for the Clopper–Pearson rule with x − 1/2 successes and LJ x is the lower limit for the Clopper–
Pearson rule with x + 1/2 successes. Strawderman
and Wells (1998) contains a valuable discussion of
mid-P intervals and suggests some variations basedon asymptotic expansions.
5. CONCLUDING REMARKSInterval estimation of a binomial proportion is avery basic problem in practical statistics. The standard Wald interval is in nearly universal use. We
ﬁrst show that the performance of this standard
interval is persistently chaotic and unacceptably
poor. Indeed its coverage properties defy conventional wisdom. The performance is so erratic andthe qualiﬁcations given in the inﬂuential texts
are so defective that the standard interval should
not be used. We provide a fairly comprehensive
evaluation of many natural alternative intervals.
Based on this analysis, we recommend the Wilsonor the equal-tailed Jeffreys prior interval for smallnn ≤ 40). These two intervals are comparable inboth absolute error and length for n ≤ 40, and webelieve that either could be used, depending on
taste.
For larger n, the Wilson, the Jeffreys and theAgresti–Coull intervals are all comparable, and theAgresti–Coull interval is the simplest to present.
It is generally true in statistical practice that only
those methods that are easy to describe, remember
and compute are widely used. Keeping this in mind,
we recommend the Agresti–Coull interval for practical use when n ≥ 40. Even for small sample sizes,
the easy-to-present Agresti–Coull interval is muchpreferable to the standard one.
We would be satisﬁed if this article contributes
to a greater appreciation of the severe ﬂaws of the
popular standard interval and an agreement that it
deserves not to be used at all. We also hope thatthe recommendations for alternative intervals will
provide some closure as to what may be used in
preference to the standard method.
Finally, we note that the speciﬁc choices of thevalues of n, p and α in the examples and ﬁgures
are artifacts. The theoretical results in Brown, Cai
and DasGupta (1999) show that qualitatively similar phenomena as regarding coverage and length
hold for general n and p and common values of
the coverage. (Those results there are asymptotic
as n → ∞, but they are also sufﬁciently accurate
for realistically moderate n.)

116

L. D. BROWN, T. T. CAI AND A. DASGUPTA

APPENDIX
Table A.1
95% Limits of the modiﬁed Jeffreys prior interval
x
0
1
2
3
4
5
6
x
0
1
2
3
4
5
6
7
8
9
x
0
1
2
3
4
5
6
7
8
9
10
11
12
x
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

n=7
0
0
0.065
0.139
0.234

n=8
0.410
0.501
0.648
0.766
0.861

n = 13
0
0
0.033
0.070
0.114
0.165
0.221
0.283

0.247
0.307
0.409
0.497
0.577
0.650
0.717
0.779

n = 19
0
0
0.022
0.047
0.076
0.108
0.144
0.182
0.223
0.266
0.312

0.176
0.221
0.297
0.364
0.426
0.484
0.539
0.591
0.641
0.688
0.734

n = 25
0
0
0.017
0.035
0.056
0.081
0.107
0.135
0.164
0.195
0.228
0.261
0.295
0.331

0.137
0.172
0.233
0.287
0.337
0.384
0.429
0.473
0.515
0.555
0.594
0.632
0.669
0.705

0
0
0.056
0.119
0.199

n=9
0.369
0.454
0.592
0.705
0.801

n = 14
0
0
0.031
0.064
0.105
0.152
0.203
0.259

0.232
0.288
0.385
0.469
0.545
0.616
0.681
0.741

n = 20
0
0
0.021
0.044
0.072
0.102
0.136
0.172
0.211
0.251
0.293

0.168
0.211
0.284
0.349
0.408
0.464
0.517
0.568
0.616
0.662
0.707

n = 26
0
0
0.016
0.034
0.054
0.077
0.102
0.129
0.158
0.187
0.218
0.250
0.282
0.316

0.132
0.166
0.225
0.277
0.325
0.371
0.415
0.457
0.498
0.537
0.576
0.613
0.649
0.684

0
0
0.049
0.104
0.173
0.254

n = 10
0.336
0.414
0.544
0.652
0.746
0.827

n = 15
0
0
0.029
0.060
0.097
0.140
0.188
0.239
0.294

0.218
0.272
0.363
0.444
0.517
0.584
0.647
0.706
0.761

n = 21
0
0
0.020
0.042
0.068
0.097
0.129
0.163
0.199
0.237
0.277
0.319

0.161
0.202
0.272
0.334
0.392
0.446
0.497
0.546
0.593
0.638
0.681
0.723

n = 27
0
0
0.016
0.032
0.052
0.074
0.098
0.124
0.151
0.180
0.209
0.239
0.271
0.303
0.336

0.128
0.160
0.217
0.268
0.315
0.359
0.402
0.443
0.482
0.521
0.558
0.594
0.630
0.664
0.697

0
0
0.044
0.093
0.153
0.224

0.308
0.381
0.503
0.606
0.696
0.776

n = 16
0
0
0.027
0.056
0.091
0.131
0.174
0.222
0.272

0.206
0.257
0.344
0.421
0.491
0.556
0.617
0.674
0.728

n = 22
0
0
0.019
0.040
0.065
0.092
0.123
0.155
0.189
0.225
0.263
0.302

0.154
0.193
0.261
0.321
0.376
0.429
0.478
0.526
0.571
0.615
0.657
0.698

n = 28
0
0
0.015
0.031
0.050
0.072
0.095
0.119
0.145
0.172
0.201
0.230
0.260
0.291
0.322

0.123
0.155
0.210
0.259
0.305
0.348
0.389
0.429
0.468
0.505
0.542
0.577
0.611
0.645
0.678

n = 11
0
0
0.040
0.084
0.137
0.200
0.270

0.285
0.353
0.467
0.565
0.652
0.730
0.800

n = 17
0
0
0.025
0.052
0.085
0.122
0.163
0.207
0.254
0.303

0.195
0.244
0.327
0.400
0.467
0.530
0.589
0.644
0.697
0.746

n = 23
0
0
0.018
0.038
0.062
0.088
0.117
0.148
0.180
0.214
0.250
0.287
0.325

0.148
0.186
0.251
0.309
0.362
0.413
0.461
0.507
0.551
0.594
0.635
0.675
0.713

n = 29
0
0
0.015
0.030
0.048
0.069
0.091
0.115
0.140
0.166
0.193
0.221
0.250
0.279
0.310
0.341

0.119
0.150
0.203
0.251
0.295
0.337
0.378
0.416
0.454
0.490
0.526
0.560
0.594
0.627
0.659
0.690

n = 12
0
0
0.036
0.076
0.124
0.180
0.243

0.265
0.328
0.436
0.529
0.612
0.688
0.757

n = 18
0
0
0.024
0.049
0.080
0.115
0.153
0.194
0.237
0.284

0.185
0.232
0.311
0.381
0.446
0.506
0.563
0.617
0.668
0.716

n = 24
0
0
0.018
0.036
0.059
0.084
0.112
0.141
0.172
0.204
0.238
0.273
0.310

0.142
0.179
0.241
0.297
0.349
0.398
0.444
0.489
0.532
0.574
0.614
0.653
0.690

n = 30
0
0
0.014
0.029
0.047
0.067
0.088
0.111
0.135
0.160
0.186
0.213
0.240
0.269
0.298
0.328

0.116
0.145
0.197
0.243
0.286
0.327
0.367
0.404
0.441
0.476
0.511
0.545
0.578
0.610
0.641
0.672

INTERVAL ESTIMATION FOR BINOMIAL PROPORTIONACKNOWLEDGMENTS
We thank Xuefeng Li for performing some helpful
computations and Jim Berger, David Moore, Steve
Samuels, Bill Studden and Ron Thisted for useful conversations. We also thank the Editors andtwo anonymous referees for their thorough and constructive comments. Supported by grants from the
National Science Foundation and the National Security Agency.
REFERENCES
Abramowitz, M. and Stegun, I. A. (1970). Handbook of Mathematical Functions. Dover, New York.
Agresti, A. and Coull, B. A. (1998). Approximate is better than
“exact” for interval estimation of binomial proportions. Amer.
Statist. 52 119–126.
Anscombe, F. J. (1948). The transformation of Poisson, binomial
and negative binomial data. Biometrika 35 246–254.
Anscombe, F. J. (1956). On estimating binomial response relations. Biometrika 43 461–464.
Berger, J. O. (1985). Statistical Decision Theory and Bayesian
Analysis, 2nd ed. Springer, New York.
Berry, D. A. (1996). Statistics: A Bayesian Perspective.
Wadsworth, Belmont, CA.
Bickel, P. and Doksum, K. (1977). Mathematical Statistics.
Prentice-Hall, Englewood Cliffs, NJ.
Blyth, C. R. and Still, H. A. (1983). Binomial conﬁdence intervals. J. Amer. Statist. Assoc. 78 108–116.
Brown, L. D., Cai, T. and DasGupta, A. (1999). Conﬁdence intervals for a binomial proportion and asymptotic expansions.
Ann. Statist to appear.
Brown, L. D., Cai, T. and DasGupta, A. (2000). Interval estimation in discrete exponential family. Technical report, Dept.
Statistics. Univ. Pennsylvania.
Casella, G. (1986). Reﬁning binomial conﬁdence intervals
Canad. J. Statist. 14 113–129.
Casella, G. and Berger, R. L. (1990). Statistical Inference.
Wadsworth & Brooks/Cole, Belmont, CA.
Clopper, C. J. and Pearson, E. S. (1934). The use of conﬁdence
or ﬁducial limits illustrated in the case of the binomial.
Biometrika 26 404–413.

117

Cox, D. R. and Snell, E. J. (1989). Analysis of Binary Data, 2nd
ed. Chapman and Hall, London.
Cressie, N. (1980). A ﬁnely tuned continuity correction. Ann.
Inst. Statist. Math. 30 435–442.
Ghosh, B. K. (1979). A comparison of some approximate conﬁdence intervals for the binomial parameter J. Amer. Statist.
Assoc. 74 894–900.
Hall, P. (1982). Improving the normal approximation whenconstructing one-sided conﬁdence intervals for binomial or
Poisson parameters. Biometrika 69 647–652.
Lehmann, E. L. (1999). Elements of Large-Sample Theory.
Springer, New York.
Newcombe, R. G. (1998). Two-sided conﬁdence intervals for the
single proportion; comparison of several methods. Statistics
in Medicine 17 857–872.
Rao, C. R. (1973). Linear Statistical Inference and Its Applications. Wiley, New York.
Samuels, M. L. and Witmer, J. W. (1999). Statistics for
the Life Sciences, 2nd ed. Prentice Hall, Englewood
Cliffs, NJ.
Santner, T. J. (1998). A note on teaching binomial conﬁdence
intervals. Teaching Statistics 20 20–23.
Santner, T. J. and Duffy, D. E. (1989). The Statistical Analysis
of Discrete Data. Springer, Berlin.
Stone, C. J. (1995). A Course in Probability and Statistics.
Duxbury, Belmont, CA.
Strawderman, R. L. and Wells, M. T. (1998). Approximately
exact inference for the common odds ratio in several 2 × 2
tables (with discussion). J. Amer. Statist. Assoc. 93 1294–
1320.
Tamhane, A. C. and Dunlop, D. D. (2000). Statistics and Data
Analysis from Elementary to Intermediate. Prentice Hall,Englewood Cliffs, NJ.
Vollset, S. E. (1993). Conﬁdence intervals for a binomial proportion. Statistics in Medicine 12 809–824.
Wasserman, L. (1991). An inferential interpretation of default
priors. Technical report, Carnegie-Mellon Univ.
Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. J. Amer. Statist. Assoc. 22
209–212.

Comment
Alan Agresti and Brent A. Coull
In this very interesting article, Professors Brown,
Cai and DasGupta (BCD) have shown that discreteAlan Agresti is Distinguished Professor, Department of Statistics, University of Florida, Gainesville,
Florida 32611-8545 (e-mail: aa@stat.uﬂ.edu). Brent
A. Coull is Assistant Professor, Department of Biostatistics, Harvard School of Public Health, Boston,
Massachusetts 02115 e-mail: bcoull@hsph.harvard.edu.

ness can cause havoc for much larger sample sizes
that one would expect. The popular (Wald) conﬁdence interval for a binomial parameter p has been
known for some time to behave poorly, but readers
will surely be surprised that this can happen forsuch large n values.
Interval estimation of a binomial parameter is
deceptively simple, as there are not even any nuisance parameters. The gold standard would seemto be a method such as the Clopper–Pearson, basedon inverting an “exact” test using the binomial dis-

118

L. D. BROWN, T. T. CAI AND A. DASGUPTA

Fig. 1. A Comparison of mean expected lengths for the nominal 95% Jeffreys J Wilson W Modiﬁed Jeffreys M-J Modiﬁed Wilson
M-W and Agresti–Coull AC intervals for n = 5 6 7 8 9.

tribution rather than an approximate test using
the normal. Because of discreteness, however, thismethod is too conservative. A more practical, nearlygold standard for this and other discrete problems
seems to be based on inverting a two-sided test
using the exact distribution but with the mid-P
value. Similarly, with large-sample methods it isbetter not to use a continuity correction, as otherwise it approximates exact inference based on an
ordinary P-value, resulting in conservative behavior. Interestingly, BCD note that the Jeffreys interval (CIJ ) approximates the mid-P value correctionof the Clopper–Pearson interval. See Gart (1966)
for related remarks about the use of 12 additions
to numbers of successes and failures before using
frequentist methods.
1. METHODS FOR ELEMENTARYSTATISTICS COURSES
It’s unfortunate that the Wald interval for p
is so seriously deﬁcient, because in addition to
being the simplest interval it is the obvious one
to teach in elementary statistics courses. By contrast, the Wilson interval (CIW ) performs surprisingly well even for small n. Since it is too complex for many such courses, however, our motivation for the “Agresti–Coull interval” (CIAC ) was toprovide a simple approximation for CIW . Formula(4) in BCD shows that the midpoint p̃ for CIW isa weighted average of p̂ and 1/2 that equals the
sample proportion after adding z2α/2 pseudo observations, half of each type; the square of the coefﬁcient of zα/2 is the same weighted average of thevariance of a sample proportion when p = p̂ andwhen p = 1/2, using ñ = n + z2α/2 in place of n. The
CIAC uses the CIW midpoint, but its squared coefﬁcient of zα/2 is the variance p̃q̃/ñ at the weighted

average p̃ rather than the weighted average of thevariances. The resulting interval p̃ ± zα/2 p̃q̃/ñ1/2
is wider than CIW (by Jensen’s inequality), in particular being conservative for p near 0 and 1 where
CIW can suffer poor coverage probabilities.
Regarding textbook qualiﬁcations on sample sizefor using the Wald interval, skewness considerations and the Edgeworth expansion suggest thatguidelines for n should depend on p through 1 −2p2 /p1−p. See, for instance, Boos and HughesOliver (2000). But this does not account for theeffects of discreteness, and as BCD point out, guidelines in terms of p are not veriﬁable. For elementary course teaching there is no obvious alternative(such as t methods) for smaller n, so we think it issensible to teach a single method that behaves reasonably well for all n, as do the Wilson, Jeffreys andAgresti–Coull intervals.
2. IMPROVED PERFORMANCE WITH
BOUNDARY MODIFICATIONSBCD showed that one can improve the behaviorof the Wilson and Jeffreys intervals for p near 0
and 1 by modifying the endpoints for CIW whenx = 1 2 n − 2 n − 1 (and x = 3 and n − 3 forn > 50) and for CIJ when x = 0 1 n − 1 n. Onceone permits the modiﬁcation of methods near the
sample space boundary, other methods may perform decently besides the three recommended inthis article.
For instance, Newcombe (1998) showed that when0 < x < n the Wilson interval CIW and the Wald
logit interval have the same midpoint on the logit
scale. In fact, Newcombe has shown (personal communication, 1999) that the logit interval necessarily

INTERVAL ESTIMATION FOR BINOMIAL PROPORTION

119

Fig. 2. A comparison of expected lengths for the nominal 95% Jeffreys J Wilson W Modiﬁed Jeffreys M-J Modiﬁed Wilson M-W
and Agresti–Coull AC intervals for n = 5.

contains CIW . The logit interval is the uninformative one [0 1] when x = 0 or x = n, but substituting the Clopper–Pearson limits in those cases yields
coverage probability functions that resemble those
for CIW and CIAC , although considerably more conservative for small n. Rubin and Schenker (1987)
recommended the logit interval after 12 additions tonumbers of successes and failures, motivating it as anormal approximation to the posterior distributionof the logit parameter after using the Jeffreys prior.
However, this modiﬁcation has coverage probabilities that are unacceptably small for p near 0 and 1(See Vollset, 1993). Presumably some other boundary modiﬁcation will result in a happy medium. In
a letter to the editor about Agresti and Coull (1998),
Rindskopf (2000) argued in favor of the logit interval partly because of its connection with logit modeling. We have not used this method for teachingin elementary courses, since logit intervals do notextend to intervals for the difference of proportionsand (like CIW and CIJ ) they are rather complex forthat level.
For practical use and for teaching in moreadvanced courses, some statisticians may prefer the
likelihood ratio interval, since conceptually it is simple and the method also applies in a general modelbuilding framework. An advantage compared to theWald approach is its invariance to the choice ofscale, resulting, for instance, both from the original scale and the logit. BCD do not say much about
this interval, since it is harder to compute. However,
it is easy to obtain with standard statistical software (e.g., in SAS, using the LRCI option in PROCGENMOD for a model containing only an intercept
term and assuming a binomial response with logit
or identity link function). Graphs in Vollset (1993)

suggest that the boundary-modiﬁed likelihood ratio
interval also behaves reasonably well, although conservative for p near 0 and 1.
For elementary course teaching, a disadvantage
of all such intervals using boundary modiﬁcations
is that making exceptions from a general, simple
recipe distracts students from the simple conceptof taking the estimate plus and minus a normalscore multiple of a standard error. (Of course, thisconcept is not sufﬁcient for serious statistical work,
but some over simpliﬁcation and compromise is necessary at that level.) Even with CIAC , instructors
may ﬁnd it preferable to give a recipe with thesame number of added pseudo observations for all
α, instead of z2α/2 . Reasonably good performanceseems to result, especially for small α, from thevalue 4 ≈ z20 025 used in the 95% CIAC interval (i.e.,the “add two successes and two failures” interval).
Agresti and Caffo (2000) discussed this and showedthat adding four pseudo observations also dramatically improves the Wald two-sample interval forcomparing proportions, although again at the cost ofrather severe conservativeness when both parameters are near 0 or near 1.
3. ALTERNATIVE WIDTH COMPARISON
In comparing the expected lengths of thethree recommended intervals, BCD note that thecomparison is clear and consistent as n changes,
with the average expected length being noticeably
larger for CIAC than CIJ and CIW . Thus, in theirconcluding remarks, they recommend CIJ and CIW
for small n. However, since BCD recommend modifying CIJ and CIW to eliminate severe downward
spikes of coverage probabilities, we believe that a

120

L. D. BROWN, T. T. CAI AND A. DASGUPTA

more fair comparison of expected lengths uses themodiﬁed versions CIM−J and CIM−W . We checked
this but must admit that ﬁgures analogous to
the BCD Figures 8 and 9 show that CIM−J and
CIM−W maintain their expected length advantage
over CIAC , although it is reduced somewhat.
However, when n decreases below 10, the resultschange, with CIM−J having greater expected widththan CIAC and CIM−W . Our Figure 1 extends the
BCD Figure 9 to values of n < 10, showing how thecomparison differs between the ordinary intervals
and the modiﬁed ones. Our Figure 2 has the formatof the BCD Figure 8, but for n = 5 instead of 25.
Admittedly, n = 5 is a rather extreme case, one forwhich the Jeffreys interval is modiﬁed unless x = 2
or 3 and the Wilson interval is modiﬁed unless x = 0
or 5, and for it CIAC has coverage probabilities thatcan dip below 0.90. Thus, overall, the BCD recommendations about choice of method seem reasonable
to us. Our own preference is to use the Wilson interval for statistical practice and CIAC for teaching inelementary statistics courses.
4. EXTENSIONSOther than near-boundary modiﬁcations, anothertype of ﬁne-tuning that may help is to invert a test
permitting unequal tail probabilities. This occurs
naturally in exact inference that inverts a single two-tailed test, which can perform better thaninverting two separate one-tailed tests (e.g., Sterne,
1954; Blyth and Still, 1983).

Finally, we are curious about the implications of
the BCD results in a more general setting. How
much does their message about the effects of discreteness and basing interval estimation on the
Jeffreys prior or the score test rather than the Wald
test extend to parameters in other discrete distributions and to two-sample comparisons? We haveseen that interval estimation of the Poisson parameter beneﬁts from inverting the score test rather
than the Wald test on the count scale (Agresti and
Coull, 1998).
One would not think there could be anything
new to say about the Wald conﬁdence interval
for a proportion, an inferential method that must
be one of the most frequently used since Laplace(1812, page 283). Likewise, the conﬁdence interval for a proportion based on the Jeffreys prior
has received attention in various forms for sometime. For instance, R. A. Fisher (1956, pages 63–70) showed the similarity of a Bayesian analysis
with Jeffreys prior to his ﬁducial approach, in a discussion that was generally critical of the conﬁdence
interval method but grudgingly admitted of limits
obtained by a test inversion such as the Clopper–
Pearson method, “though they fall short in logicalcontent of the limits found by the ﬁducial argument,
and with which they have often been confused, they
do fulﬁl some of the desiderata of statistical inferences.” Congratulations to the authors for brilliantly
casting new light on the performance of these oldand established methods.

Comment
George Casella
1. INTRODUCTION
Professors Brown, Cai and DasGupta (BCD) areto be congratulated for their clear and imaginative
look at a seemingly timeless problem. The chaotic
behavior of coverage probabilities of discrete conﬁdence sets has always been an annoyance, resulting in intervals whose coverage probability can be

George Casella is Arun Varma Commemorative
Term Professor and Chair, Department of Statistics, University of Florida, Gainesville, Florida
32611-8545 e-mail: casella@stat.uﬂ.edu.

vastly different from their nominal conﬁdence level.
What we now see is that for the Wald interval, an
approximate interval, the chaotic behavior is relentless, as this interval will not maintain 1 − α coverage for any value of n. Although ﬁxes relying on
ad hoc rules abound, they do not solve this fundamental defect of the Wald interval and, surprisingly,
the usual safety net of asymptotics is also shown
not to exist. So, as the song goes, “Bye-bye, so long,
farewell” to the Wald interval.
Now that the Wald interval is out, what is in?
There are probably two answers here, dependingon whether one is in the classroom or the consulting room.

INTERVAL ESTIMATION FOR BINOMIAL PROPORTION

Fig. 1.

121

Coverage probabilities of the Blyth-Still interval upper and Agresti-Coull interval lower for n = 100 and 1 − α = 0 95.

2. WHEN YOU SAY 95%
In the classroom it is (still) valuable to have aformula for a conﬁdence intervals, and I typicallypresent the Wilson/score interval, starting fromthe test statistic formulation. Although this doesn’t
have the pleasing p̂ ± something, most students
can understand the logic of test inversion. Moreover, the fact that the interval does not have asymmetric form is a valuable lesson in itself; thestatistical world is not always symmetric.
However, one thing still bothers me about thisinterval. It is clearly not a 1 − α interval; that is,
it does not maintain its nominal coverage probability. This is a defect, and one that should not
be compromised. I am uncomfortable in presenting a conﬁdence interval that does not maintain its

stated conﬁdence; when you say 95% you shouldmean 95%!
But the ﬁx here is rather simple: apply the “continuity correction” to the score interval (a techniquethat seems to be out of favor for reasons I do notunderstand). The continuity correction is easy tojustify in the classroom using pictures of the normal density overlaid on the binomial mass function, and the resulting interval will now maintainits nominal level. (This last statement is not basedon analytic proof, but on numerical studies.) Anyone
reading Blyth (1986) cannot help being convincedthat this is an excellent approximation, coming at
only a slightly increased effort.
One other point that Blyth makes, which BCD do
not mention, is that it is easy to get exact conﬁdence limits at the endpoints. That is, for X = 0 the122

L. D. BROWN, T. T. CAI AND A. DASGUPTA

lower bound is 0 and for X = 1 the lower bound is
1 − 1 − α1/n [the solution to PX = 0 = 1 − α].
3. USE YOUR TOOLSThe essential message that I take away from thework of BCD is that an approximate/formula-based
approach to constructing a binomial conﬁdence
interval is bound to have essential ﬂaws. However,this is a situation where brute force computing will
do the trick. The construction of a 1 − α binomial
conﬁdence interval is a discrete optimization problem that is easily programmed. So why not use thetools that we have available? If the problem will
yield to brute force computation, then we should
use that solution.
Blyth and Still (1983) showed how to compute
exact intervals through numerical inversion oftests, and Casella (1986) showed how to compute
exact intervals by reﬁning conservative intervals.

So for any value of n and α, we can compute anexact, shortest 1 − α conﬁdence interval that will
not display any of the pathological behavior illustrated by BCD. As an example, Figure 1 shows the
Agresti–Coull interval along with the Blyth–Still
interval for n = 100 and 1 − α = 0 95. While
the Agresti–Coull interval fails to maintain 0 95
coverage in the middle p region, the Blyth–Still
interval always maintains 0 95 coverage. What ismore surprising, however, is that the Blyth–Still
interval displays much less variation in its coverage probability, especially near the endpoints.
Thus, the simplistic numerical algorithm produces
an excellent interval, one that both maintains its
guaranteed coverage and reduces oscillation in the
coverage probabilities.
ACKNOWLEDGMENT
Supported by NSF Grant DMS-99-71586.

Comment
Chris Corcoran and Cyrus Mehta
We thank the authors for a very accessible
and thorough discussion of this practical problem. With the availability of modern computational tools, we have an unprecedented opportunity to carefully evaluate standard statistical procedures in this manner. The results of such work
are invaluable to teachers and practitioners ofstatistics everywhere. We particularly appreciate
the attention paid by the authors to the generally oversimpliﬁed and inadequate recommendations made by statistical texts regarding when touse normal approximations in analyzing binary
data. As their work has plainly shown, even in
the simple case of a single binomial proportion,
the discreteness of the data makes the use of

Chris Corcoran is Assistant Professor, Department of Mathematics and Statistics, Utah
State University, 3900 old Main Hill, Logon,
Utah,84322-3900
e-mail:corcoran@math.
usu.edu. Cyrus Mehta is Professor, Departmentof Biostatistics, Harvard School of Public Health,655 Huntington Avenue Boston, Massachusetts
02115 and is with Cytel Software Corporation, 675
Massachusetts Avenue, Cambridge, Massachusetts
02319.

some asymptotic procedures tenuous, even when theunderlying probability lies away from the boundaryor when the sample size is relatively large.
The authors have evaluated various conﬁdence
intervals with respect to their coverage properties
and average lengths. Implicit in their evaluationis the premise that overcoverage is just as bad as
undercoverage. We disagree with the authors on thisfundamental issue. If, because of the discreteness ofthe test statistic, the desired conﬁdence level cannot
be attained, one would ordinarily prefer overcoverage to undercoverage. Wouldn’t you prefer to hirea fortune teller whose track record exceeds expectations to one whose track record is unable to live
up to its claim of accuracy? With the exception of
the Clopper–Pearson interval, none of the intervals
discussed by the authors lives up to its claim of
95% accuracy throughout the range of p. Yet theauthors dismiss this interval on the grounds that
it is “wastefully conservative.” Perhaps so, but they
do not address the issue of how the wastefulness is
manifested.
What penalty do we incur for furnishing conﬁdence intervals that are more truthful than was
required of them? Presumably we pay for the conservatism by an increase in the length of the conﬁdence
interval. We thought it would be a useful exercise

INTERVAL ESTIMATION FOR BINOMIAL PROPORTION

Fig. 1. Actual coverage probabilities for BSC and LR intervals
as a function of pn = 50 Compare to author’s Figures 5 10
and 11.

to actually investigate the magnitude of this penalty
for two conﬁdence interval procedures that are guaranteed to provide the desired coverage but are notas conservative as Clopper–Pearson. Figure 1 displays the true coverage probabilities for the nominal
95% Blyth–Still–Casella (see Blyth and Still, 1983;
Casella, 1984) conﬁdence interval (BSC interval)
and the 95% conﬁdence interval obtained by inverting the exact likelihood ratio test (LR interval; theinversion follows that shown by Aitken, Anderson,
Francis and Hinde, 1989, pages 112–118).
There is no value of p for which the coverage of the
BSC and LR intervals falls below 95%. Their coverage probabilities are, however, much closer to 95%
than would be obtained by the Clopper–Pearson procedure, as is evident from the authors’ Figure 11.
Thus one could say that these two intervals are uniformly better than the Clopper–Pearson interval.
We next investigate the penalty to be paid for theguaranteed coverage in terms of increased length of
the BSC and LR intervals relative to the Wilson,
Agresti–Coull, or Jeffreys intervals recommended
by the authors. This is shown by Figure 2.

123

In fact the BSC and LR intervals are actuallyshorter than Agresti–Coull for p < 0 2 or p > 0 8,
and shorter than the Wilson interval for p < 0 1
and p > 0 9. The only interval that is uniformly
shorter than BSC and LR is the Jeffreys interval.
Most of the time the difference in lengths is negligible, and in the worst case (at p = 0 5) the Jeffreys
interval is only shorter by 0.025 units. Of the three
asymptotic methods recommended by the authors,
the Jeffreys interval yields the lowest average probability of coverage, with signiﬁcantly greater potential relative undercoverage in the 0 05 0 20 and0 80 0 95 regions of the parameter space. Considering this, one must question the rationale for preferring Jeffreys to either BSC or LR.
The authors argue for simplicity and ease of computation. This argument is valid for the teaching ofstatistics, where the instructor must balance simplicity with accuracy. As the authors point out, it iscustomary to teach the standard interval in introductory courses because the formula is straightforward and the central limit theorem provides agood heuristic for motivating the normal approximation. However, the evidence shows that the standard method is woefully inadequate. Teaching statistical novices about a Clopper–Pearson type interval is conceptually difﬁcult, particularly because
exact intervals are impossible to compute by hand.
As the Agresti–Coull interval preserves the conﬁdence level most successfully among the three recommended alternative intervals, we believe that
this feature when coupled with its straightforward
computation (particularly when α = 0 05) makes
this approach ideal for the classroom.
Simplicity and ease of computation have no roleto play in statistical practice. With the adventof powerful microcomputers, researchers no longer
resort to hand calculations when analyzing data.
While the need for simplicity applies to the classroom, in applications we primarily desire reliable,
accurate solutions, as there is no signiﬁcant difference in the computational overhead required by
the authors’ recommended intervals when comparedto the BSC and LR methods. From this perspective, the BSC and LR intervals have a substantial
advantage relative to the various asymptotic intervals presented by the authors. They guarantee coverage at a relatively low cost in increased length.
In fact, the BSC interval is already implemented in
StatXact (1998) and is therefore readily accessible topractitioners.

124

L.D. BROWN, T.T. CAI AND A. DASGUPTA

Fig. 2. Expected lengths of BSC and LR intervals as a function of p compared respectively to Wilson Agresti–Coull and Jeffreys
intervals n = 25. Compare to authors’ Figure 8

Comment
Malay GhoshThis is indeed a very valuable article which bringsout very clearly some of the inherent difﬁculties
associated with conﬁdence intervals for parameters of interest in discrete distributions. Professors
Brown, Cai and Dasgupta (henceforth BCD) areto be complimented for their comprehensive andthought-provoking discussion about the “chaotic”
behavior of the Wald interval for the binomial proportion and an appraisal of some of the alternatives
that have been proposed.
My remarks will primarily be conﬁned to thediscussion of Bayesian methods introduced in thispaper. BCD have demonstrated very clearly that the
Malay Ghosh is Distinguished Professor, Department of Statistics, University of Florida, Gainesville,
Florida 32611-8545 e-mail: ghoshm@stat.uﬂ.edu.

modiﬁed Jeffreys equal-tailed interval works wellin this problem and recommend it as a possible contender to the Wilson interval for n ≤ 40.
There is a deep-rooted optimality associated withJeffreys prior as the unique ﬁrst-order probability
matching prior for a real-valued parameter of interest with no nuisance parameter. Roughly speaking, a probability matching prior for a real-valued
parameter is one for which the coverage probabilityof a one-sided Baysian credible interval is asymptotically equal to its frequentist counterpart. Before
giving a formal deﬁnition of such priors, we provide an intuitive explanation of why Jeffreys prioris a matching prior. To this end, we begin withthe fact that if X1 
 Xn are iid Nθ 1, thenn = n Xi /n is the MLE of θ. With the uniX
i=1
form prior πθ ∝ c (a constant), the posterior of θ

125

INTERVAL ESTIMATION FOR BINOMIAL PROPORTIONn  1/n. Accordingly, writing zα for the upperis NX
100α% point of the N0 1 distribution,
n 
n + zα n−1/2 XPθ ≤ X
n + zα n−1/2 θ= 1 − α = Pθ ≤ X
and this is an example of perfect matching. Now
if θ̂n is the MLE of θ, under suitable regularity conditions, θ̂n θ is asymptotically (as n → ∞)Nθ I−1 θ, where Iθ is the Fisher Information θ 1/2I t,
number. With the transformation gθ =by the delta method, gθ̂n  is asymptoticallyNgθ 1. Now, intuitively one expects the uniform
prior πθ ∝ c as the asymptotic matching prior forgθ. Transforming back to the original parameter,
Jeffreys prior is a probability matching prior for θ.
Of course, this requires an invariance of probability
matching priors, a fact which is rigorously established in Datta and Ghosh (1996). Thus a uniform
prior for arcsinθ1/2 , where θ is the binomial proportion, leads to Jeffreys Beta (1/2, 1/2) prior for θ.
When θ is the Poisson parameter, the uniform priorfor θ1/2 leads to Jeffreys’ prior θ−1/2 for θ.
 Xn be iid
In a more formal set-up, let X1 
conditional on some real-valued θ. Let θπ1−α X1 
Xn  denote a posterior 1−αth quantile for θ under
the prior π. Then π is said to be a ﬁrst-order probability matching prior if(1)

 Xn θPθ ≤ θπ1−α X1 = 1 − α + on−1/2 This deﬁnition is due to Welch and Peers (1963)
who showed by solving a differential equation that
Jeffreys prior is the unique ﬁrst-order probability
matching prior in this case. Strictly speaking, Welchand Peers proved this result only for continuous
distributions. Ghosh (1994) pointed out a suitable
modiﬁcation of criterion (1) which would lead to thesame conclusion for discrete distributions. Also, forsmall and moderate samples, due to discreteness,
one needs some modiﬁcations of Jeffreys interval asdone so successfully by BCD.
This idea of probability matching can be extended
even in the presence of nuisance parameters.
 θp T , where θ1 is the parSuppose that θ = θ1 ameter of interest, while θ2  θp T is the nuisance parameter. Writing Iθ = Ijk  as the
Fisher information matrix, if θ1 is orthogonal toθ2  θp T in the sense of Cox and Reid (1987),
 p, extendingthat is, I1k = 0 for all k = 2
1/2
the previous intuitive argument, πθ ∝ I11 θis a probability matching prior. Indeed, this priorbelongs to the general class of ﬁrst-order probability matching priors
1/2πθ ∝ I11 θhθ2  θp 

as derived in Tibshirani (1989). Here h· is an arbitrary function differentiable in its arguments.
In general, matching priors have a long successstory in providing frequentist conﬁdence intervals,
especially in complex problems, for example, theBehrens–Fisher or the common mean estimation
problems where frequentist methods run into difﬁculty. Though asymptotic, the matching property
seems to hold for small and moderate sample sizes
as well for many important statistical problems.
One such example is Garvan and Ghosh (1997)
where such priors were found for general dispersion models as given in Jorgensen (1997). It may
be worthwhile developing these priors in the presence of nuisance parameters for other discrete cases
as well, for example when the parameter of interestis the difference of two binomial proportions, or the
log-odds ratio in a 2 × 2 contingency table.
Having argued so strongly in favor of matchingpriors, I wonder, though, whether there is any special need for such priors in this particular problem ofbinomial proportions. It appears that any Beta (a a)prior will do well in this case. As noted in this paper,by shrinking the MLE X/n toward the prior mean
1/2, one achieves a better centering for the construction of conﬁdence intervals. The two diametrically
opposite priors Beta (2, 2) (symmetric concave withmaximum at 1/2 which provides the Agresti–Coullinterval) and Jeffreys prior Beta (1/2 1/2) (symmetric convex with minimum at 1/2) seem to be equally
good for recentering. Indeed, I wonder whether any
Beta α β prior which shrinks the MLE toward
the prior mean α/α + β becomes appropriate forrecentering.
The problem of construction of conﬁdence intervals for binomial proportions occurs in ﬁrst courses
in statistics as well as in day-to-day consulting.
While I am strongly in favor of replacing Wald intervals by the new ones for the latter, I am not quite
sure how easy it will be to motivate these new intervals for the former. The notion of shrinking can be
explained adequately only to a few strong students
in introductory statistics courses. One possible solution for the classroom may be to bring in the notionof continuity correction and somewhat heuristcallyask students to work with X+ 12  n−X+ 12  insteadof X n − X. In this way, one centers aroundX + 12 /n + 1 a la Jeffreys prior.

126

L. D. BROWN, T. T. CAI AND A. DASGUPTA

Comment
Thomas J. Santner
I thank the authors for their detailed look ata well-studied problem. For the Wald binomial pinterval, there has not been an appreciation of
the long persistence (in n) of p locations having
substantially deﬁcient achieved coverage comparedwith the nominal coverage. Figure 1 is indeed apicture that says a thousand words. Similarly, theasymptotic lower limit in Theorem 1 for the minimum coverage of the Wald interval is an extremely
useful analytic tool to explain this phenomenon,
although other authors have given ﬁxed p approximations of the coverage probability of the Waldinterval (e.g., Theorem 1 of Ghosh, 1979).
My ﬁrst set of comments concern the speciﬁc binomial problem that the authors address and then theimplications of their work for other important discrete data conﬁdence interval problems.
The results in Ghosh (1979) complement the calculations of Brown, Cai and DasGupta (BCD) by
pointing out that the Wald interval is “too long” inaddition to being centered at the “wrong” value (theMLE as opposed to a Bayesian point estimate suchis used by the Agresti–Coull interval). His Table 3
lists the probability that the Wald interval is longer
than the Wilson interval for a central set of p values (from 0.20 to 0.80) and a range of sample sizes
n from 20 to 200. Perhaps surprisingly, in view of
its inferior coverage characteristics, the Wald interval tends to be longer than the Wilson intervalwith very high probability. Hence the Wald interval
is both too long and centered at the wrong place.
This is a dramatic effect of the skewness that BCD
mention.
When discussing any system of intervals, one
is concerned with the consistency of the answers
given by the interval across multiple uses by asingle researcher or by groups of users. Formally,
this is the reason why various symmetry propertiesare required of conﬁdence intervals. For example,
in the present case, requiring that the p interval
LX UX satisfy the symmetry property(1)

Lx Ux = 1 − Ln − x 1 − Un − x

for x ∈ 0
 n shows that investigators whoreverse their deﬁnitions of success and failure will
Thomas J. Santner is Profesor, Ohio State University, 404 Cockins Hall, 1958 Neil Avenue, Columbus,
Ohio 43210 e-mail: tjs@stat.ohio-state.edu.

be consistent in their assessment of the likely values
for p. Symmetry (1) is the minimal requirement of abinomial conﬁdence interval. The Wilson and equaltailed Jeffrey intervals advocated by BCD satisfy
the symmetry property (1) and have coverage thatis centered (when coverage is plotted versus true p)about the nominal value. They are also straightforward to motivate, even for elementary students, andsimple to compute for the outcome of interest.
However, regarding p conﬁdence intervals as theinversion of a family of acceptance regions corresponding to size α tests of H0  p = p0 versus
HA  p = p0 for 0 < p0 < 1 has some substantial advantages. Indeed, Brown et al. mention
this inversion technique when they remark on thedesirable properties of intervals formed by inverting likelihood ratio test acceptance regions of H0
versus HA . In the binomial case, the acceptanceregion of any reasonable test of H0  p = p0 is of
the form Lp0  Up0 . These acceptance regions
invert to intervals if and only if Lp0 and Up0 are
nondecreasing in p0 (otherwise the inverted p conﬁdence set can be a union of intervals). Of course,
there are many families of size α tests that meetthis nondecreasing criterion for inversion, including the very conservative test used by Clopper andPearson (1934). For the binomial problem, Blyth andStill (1983) constructed a set of conﬁdence intervalsby selecting among size α acceptance regions thosethat possessed additional symmetry properties andwere “small” (leading to short conﬁdence intervals).
For example, they desired that the interval should“move to the right” as x increases when n is ﬁxed
and should “move the left” as n increases when x
is ﬁxed. They also asked that their system of intervals increase monotonically in the coverage probability for ﬁxed x and n in the sense that the higher
nominal coverage interval contain the lower nominal coverage interval.
In addition to being less intuitive to unsophisticated statistical consumers, systems of conﬁdence
intervals formed by inversion of acceptance regions
also have two other handicaps that have hindered
their rise in popularity. First, they typically requirethat the conﬁdence interval (essentially) be constructed for all possible outcomes, rather thanmerely the response of interest. Second, their rather
brute force character means that a specialized computer program must be written to produce theacceptance sets and their inversion (the intervals).

INTERVAL ESTIMATION FOR BINOMIAL PROPORTION

Fig. 1.

127

Coverage of nominal 95% symmetric Duffy–Santner p intervals for n = 20 bottom panel and n = 50 top panelHowever, the beneﬁts of having reasonably shortand suitably symmetric conﬁdence intervals are sufﬁcient that such intervals have been constructed forseveral frequently occurring problems of biostatistics. For example, Jennison and Turnbull (1983) andDuffy and Santner (1987) present acceptance set–
inversion conﬁdence intervals (both with available
FORTRAN programs to implement their methods)
for a binomial p based on data from a multistage
clinical trial; Coe and Tamhane (1989) describe amore sophisticated set of repeated conﬁdence intervals for p1 − p2 also based on multistage clinical
trial data (and give a SAS macro to produce theintervals). Yamagami and Santner (1990) present
an acceptance set–inversion conﬁdence interval and
FORTRAN program for p1 − p2 in the two-sample
binomial problem. There are other examples.
To contrast with the intervals whose coverages
are displayed in BCD’s Figure 5 for n = 20 andn = 50, I formed the multistage intervals of Duffyand Santner that strictly attain the nominal conﬁdence level for all p. The computation was done
naively in the sense that the multistage FORTRAN
program by Duffy that implements this method
was applied using one stage with stopping bound-aries arbitrarily set at a b = 0 1 in the notation of Duffy and Santner, and a small adjustment
was made to insure symmetry property (1). (Thenonsymmetrical multiple stage stopping boundaries
that produce the data considered in Duffy and Santner do not impose symmetry.) The coverages of thesesystems are shown in Figure 1. To give an idea ofcomputing time, the n = 50 intervals required lessthan two seconds to compute on my 400 Mhz PC.
To further facilitate comparison with the intervals
whose coverage is displayed in Figure 5 of BCD,
I computed the Duffy and Santner intervals for aslightly lower level of coverage, 93.5%, so that the
average coverage was about the desired 95% nominal level; the coverage of this system is displayedin Figure 2 on the same vertical scale and compares favorably. It is possible to call the FORTRAN
program that makes these intervals within SPLUS
which makes for convenient data analysis.
I wish to mention that are a number of othersmall sample interval estimation problems of continuing interest to biostatisticians that may wellhave very reasonable small sample solutions basedon analogs of the methods that BCD recommend.

128

L. D. BROWN, T. T. CAI AND A. DASGUPTA

Fig. 2.

Coverage of nominal 93 5% symmetric Duffy–Santner p intervals for n = 50.

Most of these would be extremely difﬁcult to handle by the more brute force method of inverting
acceptance sets. The ﬁrst of these is the problem
of computing simultaneous conﬁdence intervals forp0 − pi  1 ≤ i ≤ T that arises in comparing a control binomial distribution with T treatment ones.
The second concerns forming simultaneous conﬁdence intervals for pi − pj , the cell probabilities
of a multinomial distribution. In particular, theequal-tailed Jeffrey prior approach recommended by
the author has strong appeal for both of these problems.
Finally, I note that the Wilson intervals seemto have received some recommendation as themethod of choice in other elementary texts. In his
introductory texts, Larson (1974) introduces the
Wilson interval as the method of choice although
he makes the vague, and indeed false, statement, as
BCD show, that the user can use the Wald interval if“n is large enough.” One reviewer of Santner (1998),
an article that showed the coverage virtues of the
Wilson interval compared with Wald-like intervals
advocated by another author in the magazine Teaching Statistics (written for high school teachers) commented that the Wilson method was the “standard”
method taught in the U.K.

Rejoinder
Lawrence D. Brown, T. Tony Cai and Anirban DasGupta
We deeply appreciate the many thoughtful andconstructive remarks and suggestions made by the
discussants of this paper. The discussion suggeststhat we were able to make a convincing case thatthe often-used Wald interval is far more problem-

atic than previously believed. We are happy to seea consensus that the Wald interval deserves tobe discarded, as we have recommended. It is not
surprising to us to see disagreement over the speciﬁc alternative(s) to be recommended in place ofINTERVAL ESTIMATION FOR BINOMIAL PROPORTION

this interval. We hope the continuing debate will
add to a greater understanding of the problem, andwe welcome the chance to contribute to this debate.
A. It seems that the primary source of disagreement is based on differences in interpretationof the coverage goals for conﬁdence intervals.
We will begin by presenting our point of viewon this fundamental issue.
We will then turn to a number of other issues,
as summarized in the following list:
B. Simplicity is important.
C. Expected length is also important.
D. Santner’s proposal.
E. Should a continuity correction be used?
F. The Wald interval also performs poorly inother problems.
G. The two-sample binomial problem.
H. Probability-matching procedures.
I. Results from asymptotic theory.
A. Professors Casella, Corcoran and Mehta comeout in favor of making coverage errors always fall
only on the conservative side. This is a traditional
point of view. However, we have taken a different
perspective in our treatment. It seems more consistent with contemporary statistical practice to expectthat a γ% conﬁdence interval should cover the true
value approximately γ% of the time. The approximation should be built on sound, relevant statistical calculations, and it should be as accurate as thesituation allows.
We note in this regard that most statistical models are only felt to be approximately valid as representations of the true situation. Hence the resulting coverage properties from those models are atbest only approximately accurate. Furthermore, abroad range of modern procedures is supported
only by asymptotic or Monte-Carlo calculations, andso again coverage can at best only be approximately the nominal value. As statisticians we dothe best within these constraints to produce procedures whose coverage comes close to the nominal
value. In these contexts when we claim γ% coverage we clearly intend to convey that the coverage isclose to γ%, rather than to guarantee it is at leastγ%.
We grant that the binomial model has a somewhat special character relative to this general discussion. There are practical contexts where one canfeel conﬁdent this model holds with very high precision. Furthermore, asymptotics are not required inorder to construct practical procedures or evaluate
their properties, although asymptotic calculations
can be useful in both regards. But the discreteness of the problem introduces a related barrier

129

to the construction of satisfactory procedures. This
forces one to again decide whether γ% should mean
“approximately γ%,” as it does in most other contemporary applications, or “at least γ%” as canbe obtained with the Blyth–Still procedure or the
Cloppe–Pearson procedure. An obvious price of thelatter approach is in its decreased precision, as measured by the increased expected length of the intervals.
B. All the discussants agree that elementary
motivation and simplicity of computation are important attributes in the classroom context. We ofcourse agree. If these considerations are paramountthen the Agresti–Coull procedure is ideal. If theneed for simplicity can be relaxed even a little, thenwe prefer the Wilson procedure: it is only slightly
harder to compute, its coverage is clearly closer tothe nominal value across a wider range of values ofp, and it can be easier to motivate since its derivation is totally consistent with Neyman–Pearson theory. Other procedures such as Jeffreys or the mid-P
Clopper–Pearson interval become plausible competitors whenever computer software can be substituted
for the possibility of hand derivation and computation.
Corcoran and Mehta take a rather extreme position when they write, “Simplicity and ease of computation have no role to play in statistical practice[italics ours].” We agree that the ability to perform
computations by hand should be of little, if any, relevance in practice. But conceptual simplicity, parsimony and consistency with general theory remain
important secondary conditions to choose amongprocedures with acceptable coverage and precision.
These considerations will reappear in our discussion of Santner’s Blyth–Still proposal. They also
leave us feeling somewhat ambivalent about theboundary-modiﬁed procedures we have presented in
our Section 4.1. Agresti and Coull correctly imply
that other boundary corrections could have been
tried and that our choice is thus somewhat ad hoc.
(The correction to Wilson can perhaps be defended
on the principle of substituting a Poisson approximation for a Gaussian one where the former isclearly more accurate; but we see no such fundamental motivation for our correction to the Jeffreys
interval.)
C. Several discussants commented on the precision of various proposals in terms of expected
length of the resulting intervals. We strongly concur that precision is the important balancing criterion vis-á-vis coverage. We wish only to note that
there exist other measures of precision than interval expected length. In particular, one may investigate the probability of covering wrong values. In a

130

L. D. BROWN, T. T. CAI AND A. DASGUPTA

charming identity worth noting, Pratt (1961) shows
the connection of this approach to that of expected
length. Calculations on coverage of wrong values of
p in the binomial case will be presented in DasGupta (2001). This article also discusses a numberof additional issues and presents further analytical
calculations, including a Pearson tilting similar tothe chi-square tilts advised in Hall (1983).
Corcoran and Mehta’s Figure 2 compares average
length of three of our proposals with Blyth–Still andwith their likelihood ratio procedure. We note ﬁrst
that their LB procedure is not the same as ours.
Theirs is based on numerically computed exact percentiles of the ﬁxed sample likelihood ratio statistic.
We suspect this is roughly equivalent to adjustmentof the chi-squared percentile by a Bartlett correction. Ours is based on the traditional asymptotic
chi-squared formula for the distribution of the likelihood ratio statistic. Consequently, their procedure
has conservative coverage, whereas ours has coverage ﬂuctuating around the nominal value. They
assert that the difference in expected length is “negligible.” How much difference qualiﬁes as negligible
is an arguable, subjective evaluation. But we notethat in their plot their intervals can be on average about 8% or 10% longer than Jeffreys or Wilson
intervals, respectively. This seems to us a nonnegligible difference. Actually, we suspect their preference for their LR and BSC intervals rests primarilyon their overriding preference for conservativity incoverage whereas, as we have discussed above, ourintervals are designed to attain approximately thedesired nominal value.
D. Santner proposes an interesting variant of theoriginal Blyth–Still proposal. As we understand it,

he suggests producing nominal γ% intervals by constructing the γ ∗ % Blyth–Still intervals, with γ ∗ %chosen so that the average coverage of the resulting intervals is approximately the nominal value,
γ%. The coverage plot for this procedure compares
well with that for Wilson or Jeffreys in our Figure 5.
Perhaps the expected interval length for this procedure also compares well, although Santner does not
say so. However, we still do not favor his proposal.
It is conceptually more complicated and requires a
specially designed computer program, particularly if
one wishes to compute γ ∗ % with any degree of accuracy. It thus fails with respect to the criterion of scientiﬁc parsimony in relation to other proposals thatappear to have at least competitive performance
characteristics.
E. Casella suggests the possibility of performing a continuity correction on the score statistic
prior to constructing a conﬁdence interval. We do
not agree with this proposal from any perspective. These “continuity-corrected Wilson” intervals
have extremely conservative coverage properties,
though they may not in principle be guaranteed tobe everywhere conservative. But even if one’s goal,
unlike ours, is to produce conservative intervals,
these intervals will be very inefﬁcient at their normal level relative to Blyth–Still or even Clopper–
Pearson. In Figure 1 below, we plot the coverageof the Wilson interval with and without a continuity correction for n = 25 and α = 0 05, andthe corresponding expected lengths. It is seems
clear that the loss in precision more than neutralizes the improvements in coverage and that thenominal coverage of 95% is misleading from any
perspective.

Fig. 1. Comparison of the coverage probabilities and expected lengths of the Wilson dotted and continuity-corrected Wilson solidintervals for n = 25 and α = 0 05.

INTERVAL ESTIMATION FOR BINOMIAL PROPORTION

131

Fig. 2. Comparison of the systematic coverage biases. The y-axis is nSn p From top to bottom: the systematic coverage biases of theAgresti–Coull Wilson Jeffreys likelihood ratio and Wald intervals with n = 50 and α = 0 05.

F. Agresti and Coull ask if the dismal performance of the Wald interval manifests itself inother problems, including nordiscrete cases. Indeed
it does. In other lattice cases such as the Poisson
and negative binomial, both the considerable negative coverage bias and inefﬁciency in length persist. These features also show up in some continuous exponential family cases. See Brown, Cai and
DasGupta (2000b) for details.
In the three important discrete cases, the binomial, Poisson and negative binomial, there is in factsome conformity in regard to which methods work
well in general. Both the likelihood ratio interval(using the asymptotic chi-squared limits) and theequal-tailed Jeffreys interval perform admirably inall of these problems with regard to coverage andexpected length. Perhaps there is an underlying theoretical reason for the parallel behavior of these
two intervals constructed from very different foundational principles, and this seems worth further
study.
G. Some discussants very logically inquire about
the situation in the two-sample binomial situation.
Curiously, in a way, the Wald interval in the twosample case for the difference of proportions is lessproblematic than in the one-sample case. It cannevertheless be somewhat improved. Agresti and
Caffo (2000) present a proposal for this problem,
and Brown and Li (2001) discuss some others.
H. The discussion by Ghosh raises several interesting issues. The deﬁnition of “ﬁrst-order probability matching” extends in the obvious way to any
set of upper conﬁdence limits; not just those corresponding to Bayesian intervals. There is also an
obvious extension to lower conﬁdence limits. This

probability matching is a one-sided criterion. Thusa family of two-sided intervals Ln  Un  will be ﬁrstorder probability matching if
Prp p ≤ Ln  = α/2 + on−1/2  = Prp p ≥ Un 
As Ghosh notes, this deﬁnition cannot usefully
be literally applied to the binomial problem here,
because the asymptotic expansions always have a
discrete oscillation term that is On−1/2 . However,
one can correct the deﬁnition.
One way to do so involves writing asymptotic
expressions for the probabilities of interest that canbe divided into a “smooth” part, S, and an “oscillating” part, Osc, that averages to On−3/2  withrespect to any smooth density supported within (0,1). Readers could consult BCD (2000a) for moredetails about such expansions. Thus, in much generality one could write(1)

Prp p ≤ Ln 
= α/2 + SLn p + OscLn p + On−1 

where SLn p = On−1/2 , and OscLn p has theproperty informally described above. We would thensay that the procedure is ﬁrst-order probability
matching if SLn p = on−1/2 , with an analogous
expression for the upper limit, Un .
In this sense the equal-tail Jeffreys procedure
is probability matching. We believe that the midP Clopper–Pearson intervals also have this asymptotic property. But several of the other proposals,
including the Wald, the Wilson and the likelihood
ratio intervals are not ﬁrst-order probability matching. See Cai (2001) for exact and asymptotic calculations on one-sided conﬁdence intervals and hypothesis testing in the discrete distributions.

132

L. D. BROWN, T. T. CAI AND A. DASGUPTA

The failure of this one-sided, ﬁrst-order property,
however, has no obvious bearing on the coverage
properties of the two-sided procedures consideredin the paper. That is because, for any of our procedures,(2)

SLn p + SUn p = 0 + On−1 even when the individual terms on the left are onlyOn−1/2 . All the procedures thus make compensating one-sided errors, to On−1 , even when they are
not accurate to this degree as one-sided procedures.
This situation raises the question as to whether
it is desirable to add as a secondary criterion for
two-sided procedures that they also provide accurate one-sided statements, at least to the probability matching On−1/2 . While Ghosh argues strongly
for the probability matching property, his argument
does not seem to take into account the cancellation
inherent in (2). We have heard some others argue infavor of such a requirement and some argue against
it. We do not wish to take a strong position on
this issue now. Perhaps it depends somewhat on thepractical context—if in that context the conﬁdencebounds may be interpreted and used in a one-sided
fashion as well as the two-sided one, then perhaps
probability matching is called for.
I. Ghosh’s comments are a reminder that asymptotic theory is useful for this problem, even though
exact calculations here are entirely feasible and convenient. But, as Ghosh notes, asymptotic expressions can be startingly accurate for moderatesample sizes. Asymptotics can thus provide valid
insights that are not easily drawn from a series ofexact calculations. For example, the two-sided intervals also obey an expression analogous to (1),(3) Prp Ln ≤ p ≤ Un 
= 1 − α + Sn p + Oscn p + On−3/2 
The term Sn p is On−1  and provides a usefulexpression for the smooth center of the oscillatory
coverage plot. (See Theorem 6 of BCD (2000a) fora precise justiﬁcation.) The following plot for n =50 compares Sn p for ﬁve conﬁdence procedures.
It shows how the Wilson, Jeffreys and chisquared likelihood ratio procedures all have coverage that well approximates the nominal value, with
Wilson being slightly more conservative than theother two.
As we see it our article articulated three primarygoals: to demonstrate unambiguously that the Waldinterval performs extremely poorly; to point out thatnone of the common prescriptions on when the interval is satisfactory are correct and to put forward
some recommendations on what is to be used in its
place. On the basis of the discussion we feel gratiﬁedthat we have satisfactorily met the ﬁrst two of thesegoals. As Professor Casella notes, the debate aboutalternatives in this timeless problem will linger on,
as it should. We thank the discussants again for alucid and engaging discussion of a number of relevant issues. We are grateful for the opportunity
to have learned so much from these distinguishedcolleagues.
ADDITIONAL REFERENCES
Agresti, A. and Caffo, B. (2000). Simple and effective conﬁdence intervals for proportions and differences of proportions
result from adding two successes and two failures. Amer.
Statist. 54. To appear.
Aitkin, M., Anderson, D., Francis, B. and Hinde, J. (1989).
Statistical Modelling in GLIM. Oxford Univ. Press.
Boos, D. D. and Hughes-Oliver, J. M. (2000). How large does nhave to be for Z and t intervals? Amer. Statist. 54 121–128.
Brown, L. D., Cai, T. and DasGupta, A. (2000a). Conﬁdence
intervals for a binomial proportion and asymptotic expansions. Ann. Statist. To appear.
Brown, L. D., Cai, T. and DasGupta, A. (2000b). Interval estimation in exponential families. Technical report, Dept. Statistics, Univ. Pennsylvania.
Brown, L. D. and Li, X. (2001). Conﬁdence intervals forthe difference of two binomial proportions. Unpublishedmanuscript.
Cai, T. (2001). One-sided conﬁdence intervals and hypothesis
testing in discrete distributions. Preprint.
Coe, P. R. and Tamhane, A. C. (1993). Exact repeated conﬁdence
intervals for Bernoulli parameters in a group sequential clinical trial. Controlled Clinical Trials 14 19–29.
Cox, D. R. and Reid, N. (1987). Orthogonal parameters andapproximate conditional inference (with discussion). J. Roy.
Statist. Soc. Ser. B 49 113–147.
DasGupta, A. (2001). Some further results in the binomial interval estimation problem. Preprint.
Datta, G. S. and Ghosh, M. (1996). On the invariance of noninformative priors. Ann. Statist. 24 141–159.
Duffy, D. and Santner, T. J. (1987). Conﬁdence intervals fora binomial parameter based on multistage tests. Biometrics
43 81–94.
Fisher, R. A. (1956). Statistical Methods for Scientiﬁc Inference.
Oliver and Boyd, Edinburgh.
Gart, J. J. (1966). Alternative analyses of contingency tables. J.
Roy. Statist. Soc. Ser. B 28 164–179.
Garvan, C. W. and Ghosh, M. (1997). Noninformative priors fordispersion models. Biometrika 84 976–982.
Ghosh, J. K. (1994). Higher Order Asymptotics. IMS, Hayward,
CA.
Hall, P. (1983). Chi-squared approximations to the distributionof a sum of independent random variables. Ann. Statist. 111028–1036.
Jennison, C. and Turnbull, B. W. (1983). Conﬁdence intervals
for a binomial parameter following a multistage test withapplication to MIL-STD 105D and medical trials. Technometrics, 25 49–58.
Jorgensen, B. (1997). The Theory of Dispersion Models. CRC
Chapman and Hall, London.
Laplace, P. S. (1812). Théorie Analytique des Probabilités.
Courcier, Paris.
Larson, H. J. (1974). Introduction to Probability Theory and Statistical Inference, 2nd ed. Wiley, New York.

INTERVAL ESTIMATION FOR BINOMIAL PROPORTION
Pratt, J. W. (1961). Length of conﬁdence intervals. J. Amer.
Statist. Assoc. 56 549–567.
Rindskopf, D. (2000). Letter to the editor. Amer. Statist. 54 88.
Rubin, D. B. and Schenker, N. (1987). Logit-based interval estimation for binomial data using the Jeffreys prior. Sociological Methodology 17 131–144.
Sterne, T. E. (1954). Some remarks on conﬁdence or ﬁducial
limits. Biometrika 41 275–278.

133

Tibshirani, R. (1989). Noninformative priors for one parameterof many. Biometrika 76 604–608.
Welch, B. L. and Peers, H. W. (1963). On formula for conﬁdence points based on intergrals of weighted likelihoods. J.
Roy. Statist. Ser. B 25 318–329.
Yamagami, S. and Santner, T. J. (1993). Invariant small sample
conﬁdence intervals for the difference of two success probabilities. Comm. Statist. Simul. Comput. 22 33–59.