Median of the p Value Under the Alternative Hypothesis
Bhaskar BHATTACHARYA and Desale HABTZGHI

Due to absence of an universally acceptable magnitude of theType I error in various  elds, p values are used as a wellrecognized tool in decision making in all areas of statisticalpractice. The distribution of p values under the null hypothesis is uniform. However, under the alternative hypothesis the
distribution of the p values is skewed. The expected p value(EPV) has been proposed by authors to be used as a measure of
the performance of the test. This article proposes the median of
the p values (MPV) which is more appropriate for this purpose.
We work out many examples to calculate the MPV’s directly
and also compare the MPV with the EPV. We consider testing
equality of distributionsagainst stochastic ordering in the multinomial case and compare the EPV’s and MPV’s by simulation.
A second simulation study for general continuous data is also
considered for two samples with different test statistics for thesame hypotheses. In both cases MPV performs better than EPV.
KEY WORDS: Comparing test statistics; Expected p value;Median p value; Power; Stochastic order.

1.

INTRODUCTION

The theory of hypothesis testing depends heavily on the prespeci ed value of the signi cance level. To avoid the nonuniqueness of the decision of testing the same hypotheses using thesame test statistic but different signi cance levels, it is a popular choice to report the p value. The p value is the smallest
level of signi cance at which an experimenter would reject thenull hypothesis on the basis of the observed outcome. The user
can compare his/her own signi cance level with the p value andmake his/her own decision. The p values are particularly usefulin cases when the null hypothesis is well de ned but the alternative is not (e.g., composite) so that Type II error considerations
are unclear. In this context to quote Fisher, “The actual value of p
obtainable from the table by interpolation indicates the strengthof evidence against the null hypothesis.”
We will consider tests of the form “Reject H0 when T ¶ c,”where T is a real-valued test statistic computed from data when
testing the null hypothesis H0 against the alternative H1 . The
value c is determined from the prespeci ed size restrictions suchthat PH0 (T ¶ c) = ¬ . Of course, when T is a discrete randomvariable, one needs to adopt randomization so that all sizes are
possible. If t is the observed value of T and the distribution of T
under H0 is given by F0 (¢), then the p value is given by 1 ¡ F0 (t)which is the probability of  nding the test statistic as extreme as,
Bhaskar Bhattacharya is Associate Professor, Department of Mathematics,
Southern Illinois University, Carbondale, IL 62901 (E-mail: Bhaskar@math.
siu.edu). Desale Habtzghi is a Graduate Student, Department of Statistics, University of Georgia, Athens, GA 30602.

202

The American Statistician, August 2002, Vol. 56, No. 3

or more extreme than, the value actually observed. Thus, if the pvalue is less than the preferred signi cance level, then one rejects
H0 . Over the years several authors have attempted to properly
explain p values. Gibbons and Pratt (1975) provided interpretation and methodology of the p values. Recently, Schervish(1996) treated the p values as signi cance probabilities. Discussions on p values can be found in Blyth and Staudte (1997) andin Dollinger, Kulinskaya, and Staudte (1996). However, thesearticles do not treat the p values as random.
The p values are based on the test statistics used and hence
random. The stochastic nature of the p values was investigatedby Dempster and Schatzoff (1965) and Schatzoff (1966) who
introduced “expected signi cance value.” Recently, Sackrowitz
and Samuel-Cahn (1999) investigated this concept further andrenamed it as the expected p value (EPV). Under the null hypothesis, the p values have a uniform distribution over (0, 1) for
any sample size. Thus, under H 0 , EPV is 1/2 always, and there
is no way to distinguish p values derived from large studies andthose from small-scale studies. Also it would be impossible under H0 to differentiate between studies well powered to detect
a posited alternative hypothesis and the underpowered to detect
the same posited alternative value.
In contrast, the distribution of the p values under the alternative hypothesis is a function of the sample size and the true
parameter value in the alternative hypothesis. As the p values
measure evidence against the null hypothesis, it is of interest toinvestigate the behavior of the p values under the alternative atvarious sample sizes. We reject H0 when p value is small whichis expected when H1 is true. As noted by Hung, O’Neill, Bauer,
and Kohne (1997), the distribution of the p values under thealternative is highly skewed. The skewness increases with thesample size and the true parameter value under the alternativere ecting the ability to detect the alternativeby increasing power
under these situations. Hence it is more appropriate to consider
the median of the p value (MPV) instead of the EPV under thealternative as a measure of the center of its distribution which is
the main focus of this article. Applications of the distribution of
the p values under the alternative in the area of meta-analysis ofseveral studies is considered by Hung et al. (1997). Studying thep value under the alternative is also bene cial over the power ofa test and is explained in the next paragraph.
When several test procedures are available for the same testing situation one compares them by means of power. However,power calculations depend on the chosen signi cance levels,and in discrete cases involves randomization. These steps can
be avoided by considering MPV’s. Also as the power functions
depend on the chosen signi cance levels, it is dif cult to comparethem when different power functions use different signi cancelevels. On the other hand, MPV’s depend only on the alternativeand not on the signi cance level. The smaller the value of MPV,
the stronger the test. The value of an MPV can tell us whichalternative an attained p value best represents for a given sam®
c 2002 American Statistical Association DOI: 10.1198/000313002146

ple size. Also, it helps to know the behavior of the MPV’s forvarying sample sizes.
In Section 2, we obtain a general expression for the MPV’s.
We also derive a computationally favorable form to be usedlater in the article. In Section 3, we consider several examples tocompute the MPV’s directly. In Section 4, we perform a simulation study to compute the MPV’s in the case of testing against
stochastic ordering for multinomial distributions. We also perform a second simulation study for general distributions when
testing the same hypotheses with two samples using three test
statistics of t test, Mann–Whitney–Wilcoxon,and Kolmogorov–
Smirnov. In Section 5, we make concluding remarks.

Since F³ (t) µ F0 (t); 8t, it is seen that F0 (F³ ¡1 (:5)) ¶ :5 which
implies that ¬ ¤ µ :5 and equality holds when H0 is true. The
smaller the MPV the better it is to detect the alternative. Given
the stochastic nature of the p value under the alternative, it isalso true that the MPV is smaller than the EPV and hence clearly
preferable over the EPV. The MPV being smaller than the EPV
produces a smaller indifference region in the sense that higher
power is generated closer to the null hypothesis region using
MPV than with using EPV.
It is also possible to express the MPV in another way. Let T ¹
F³ (¢) and, independently, T ¤ ¹ F0 (¢). If the observed value of
T is t, then the p value is simply g(t) = P (T ¤ ¶ tjT = t). The
MPV is

2. p VALUE AS A RANDOM VARIABLE AND ITSMEDIAN

med g(t) = MPV(³ ) = P (T ¤ ¶ med T ):For the test statistic T with distribution F0 (¢) under H0 , let
F³ (¢) be its distributionunder H1 . Also, let F0¡1 (¢) be the inverse
function of F0 (¢), so that, F0 (F0¡1 (® )) = ® , for any 0 < ® < 1.
Since the p value is the probability of observing a more extreme
value than the observed test statistic value, as a random variable
it can be expressed asIf H0 is true, then T and T ¤ are identicallydistributed,and hence
the above probability is .5 for any continuous distribution. For
discrete distributions, although the above expressions still hold,
MPV will be slightly higher as P (T ¤ = med T ) > 0. For an
UMP test, the MPV will be uniformly minimal for all ³ values
in the alternative as compared to the MPV’s of any other test of
the same H0 versus H1 . For computational purposes the above
form of the MPV in (3) is very useful.

X =1¡

F0 (T ):

3.

As F0 (T ) ¹ U (0; 1) under H0 , so is X. The power of the test
is related to the p value as­ = P³ (X µ ¬ )= 1 ¡ F³ (F0¡1 (1 ¡

(1)

¬ )):

Note the above is also the distribution function of the p value
under the alternative. As T is stochastically larger under thealternative than under the null hypothesis, it follows that the pvalue under the alternative is stochastically smaller than under
the null (Lehmann 1986). This explains why the distribution of
the p value is skewed to the right under the alternative. Hence toestimate the center of the distribution of the p value, the medianis a better choice than the mean. The median is any value of¬ = ¬ ¤ which satis es
P³ (X < ¬¤) µ :5andP³ (X > ¬¤For continuous distributions, the median is the value of ¬ = ¬ ¤which satis es
P³ (X µ ¬ ¤ ) = :5:

Example 1. Let X1 ; X2 ; : : : ; Xn be a random sample of size nfrom a N (· ; ¼ 2 ) distribution, and we like to test H0 : · µ · 0
versus H 1 : · > · 0 where ¼ is known. Using the test statistic
T = X and a particular value · 1 in H1 it follows from (2) thatµp¶n(· 0 ¡ · 1 )MPV = ©
;
(4)¼
where © is the cdf of the standard normal distribution. It is wellknown that for a size ¬ test to achieve power ­ at a speci edalternative value · 1 the sample size n satis es(z1¡¬ + z­ )2 ¼ 2;(· 0 ¡ · 1 )2

F0 (F³ ¡1 (:5)):(2)

Table 1. MPV’s in the Testing for the Normal Mean as a Function of the Signi cance Level ¬ and Power ­
¬ =­

0.40

0.50

0.60

0.70

0.80

0.90

0.95

0.01
0.05
0.10
0.15

0.01910.08200.15190.21680.01000.05000.10000.15000.00490.02880.06240.09860.00220.01500.03550.05930.00080.00650.01690.03020.00020.00170.00520.01020.00000.00050.00170.0036(5)

where z® is the ® th quantile of the standard normal distribution.
Using (4) and (5), it follows that
MPV = ©(¡ z1¡¬ ¡It follows from (1) by simple manipulations that¬ ¤ =1¡

EXAMPLESThis section considers several examples to calculate the
MPV’s directly.

n=) µ :5:

(3)

z­ ):

(6)

In Table 1, we have provided values of the MPV’s in (6) using
some commonly used values of ¬ and ­ . Each MPV value issmaller than the corresponding EPV value of Table 1 of Sackrowitz and Samuel-Cahn (1999). We have also graphed the EPV
and the MPV in Figure 1 for various values of · 1 > 0 when
· 0 = 0 at sample sizes 10 and 50. The MPV’s decrease fromthe .5 value at much faster rate than the EPV’s although forboth the rate increases with the sample size. In Figure 2, wehave graphed the EPV and the MPV for various sample sizes at· 1 = :3 and at · 1 = :5. It is observed that the MPV’s decreaseat a faster rate than the EPV’s at smaller sample sizes and whencloser to the null hypothesis.
The American Statistician, August 2002, Vol. 56, No. 3

203

to test H0 : · 1 = · 2 versus H1 : · 1 > · 2 where ¼ is known.
Using the test statistic T = X ¡ Y and a particular value · 1 ¡ · 2
in H1 it follows thatµr¶mn · 2 ¡ · 1
MPV = ©
:
(7)m+n¼
When m = n, formula (6) is still valid and hence Table 1 is also
correct in this case. For unknown ¼ , a consistent estimator of ¼
may be used in (7) for moderately large m; n, and the formulain (7) becomes approximately correct in this case.
Example 3. Suppose X1 ; X2 ; : : : ; Xn is a random sample ofsize n from a N (· ; ¼ 2 ) distribution, and we like to test H0 :
¼ µ ¼ 0 versus H1 : ¼ > ¼ 0 when · is unknown. Using thetest statistic (n ¡ 1)S 2 =¼ 2 where S 2 is the sample variance, itfollows from (4) that at the alternative point ¼ 1 ,µ 2¶¼ 1 ¡1
MPV = 1 ¡ G
G (:5) ;(8)¼ 02
Figure 1. EPV’s and MPV’s at various · 1 > 0 when testing for thenormal mean with · 0 = 0 for sample sizes n = 10 and n = 50.

If the value of ¼ is unknown, the sample standard deviation S(or any other consistent estimator of ¼ ) may be used to replace
it for moderately large n. The formula in (4) is approximately
correct in this case, consequently, Table 1 is approximately correct for the one-sample t test situation. After observing an actual
p value for an approximately normally distributed statistic, theexpression in (4) can be used to determine the · 1 for which thegiven value would be an MPV.
Example 2. Let X1 ; X2 ; : : : ; Xn be a random sample of sizen from N(· 1 ; ¼ 2 ), and independently, let Y1 ; Y2 ; : : : ; Ym be
another random sample of size m from N (· 2 ; ¼ 2 ), and we like

where G is the cdf of a chi-square distributionwith n¡ 1 degrees
of freedom. Note we need not use the F -distribution as needed
for the computation of the EPV in this problem (Sackrowitz andSamuel-Cahn 1999).
The following two examples are concerned with testing thescale and location parameters of the exponential distribution.
Example 4. Suppose T is exponentially distributed with parameter ³ (from pdf f (t) = ³ e¡t³ for t > 0) denoted by exp(³ )and we like to test H0 : ³ ¶ ³ 0 versus H 1 : ³ < ³ 0 . It is seenthat med T = (1=³ )ln 2. For a particular value of ³ 1 < ³ 0 , if
T ¤ ¹ exp(³ 0 ) and T ¹ exp(³ 1 ), the MPV is given by
P (T ¤ ¶ med T ) = 2¡³ 0 =³ 1 :For a size ¬ test with power ­ , since ln ¬ = ln ­ = ³ 0 =³ 1 , it follows that for ¬ = :1 and ­ = :9, the MPV is 2:645 £ 10¡7 . The
EPV is .0438 in this case (Sackrowitz and Samuel-Cahn 1999).
If a random sample
1 ; X 2 ; : : : ; Xn is available, the test may
PXn
be based on T = i= 1 Xi . Here T has a gamma distribution
with shape parameter n and scale parameter ³ , we will denoteits cdf by Gn;³ (¢). Then it follows that the MPV at alternative
point ³ 1 is given by
P (T ¤ ¶ G¡1
n;³ 1 (:5)) = 1 ¡Gn;³ 0 (G¡1n;³ 1 (:5)):

Example 5. Suppose X1 ; X2 ; : : : ; Xn is a random sampleof size n from an exponential distribution with parameters· ; ³ (with pdf f (x) = (1=³ )e¡(x¡· )=³ for x ¶ · ) denoted by exp(· ; ³ ) and we like to test H 0 : · µ · 0 versus H1 : · > · 0 where ³ is known. The test statistic is
T = min(X1 ; X2 ; : : : ; Xn ) whose distribution is exp(· ; ³ =n).
It is seen that med T = · + (³ =n)ln 2. For a particular alternative· 1 > · 0 , the MPV is given by :5e¡n(· 1 ¡· 0 )=³ .
4.
Figure 2. EPV’s and MPV’s at various sample sizes when testing forthe normal mean with · 0 = 0 and · 1 =.3 and · 1 = .5.
204

GeneralSIMULATION STUDIES

We perform two simulation studies to calculate and compare
the EPV’s and MPV’s. For the discrete case we consider thebinomial distribution, with m trials and probability of successTable 2. EPV’s and MPV’s for Testing H0 : p = q Against H1 : p is Stochastically Larger than q for Different Combinations of m; n; p
m=3

m=6

EPV

MPV

EPV

MPV

p

n = 50n = 100

n = 50n = 100

n = 50n = 100

n = 50n = 100

0.50
0.51
0.52
0.53
0.54
0.55
0.56
0.57
0.58
0.59
0.60
0.61
0.62
0.63
0.64

0.509
0.454
0.404
0.350
0.300
0.252
0.208
0.169
0.135
0.102
0.075
0.054
0.037
0.026
0.017

0.508
0.432
0.364
0.291
0.235
0.174
0.129
0.087
0.054
0.036
0.021
0.011
0.006
0.003
0.001

0.510
0.439
0.363
0.292
0.230
0.165
0.117
0.078
0.050
0.031
0.017
0.008
0.003
0.002
0.001

0.506
0.407
0.309
0.222
0.143
0.081
0.045
0.020
0.007
0.003
0.001
0.000
0.000
0.000
0.000

0.509
0.445
0.380
0.319
0.258
0.202
0.149
0.102
0.069
0.044
0.026
0.013
0.007
0.003
0.001

0.511
0.421
0.335
0.250
0.170
0.110
0.065
0.037
0.019
0.008
0.002
0.001
0.000
0.000
0.000

0.505
0.421
0.332
0.248
0.168
0.103
0.055
0.027
0.012
0.006
0.002
0.001
0.000
0.000
0.000

0.510
0.401
0.279
0.164
0.079
0.033
0.011
0.003
0.001
0.000
0.000
0.000
0.000
0.000
0.000

p, given by
pj =likelihood ratio test statistic is given by
µ

m
j

¶

pj (1 ¡p)m¡j ;

j = 0; : : : ; m(9)which is symmetric when p = :5. When p > :5, the binomial
distribution is skewed to the left, that is, it becomes stochastically larger than the p = :5 case. Let q = (q0 ; : : : ; qm )and p = (p0 ; : : : ; pm ) be the vectors of binomial probabilities obtained from (9) with p = :5 and p > :5, respectively. We consider testing H0 : p = q (i.e., piP= qi ; 8i)magainst H1 : p is stochastically larger than q (i.e., i= j pi ¶Pm
PmPm
i= j qi ; 8j = 1; : : : ; m; andi= 0 pi =i= 0 qi = 1). The

T = 2nmXp^i ln (pi =qi )i= 0^ E^p (q=^where pi is the ith coordinateof p = ppjA); E^p (q=^pjA)
is the isotonic regression of q=^
p (all multiplications and divisions of vectors are done coordinatewise) onto the nonincreasingcone A = fx = (x0 ; x1 ; : : : ; xm ) : x0 ¶ x1 ¶ ¢ ¢ ¢ ¶ xm g
^ . It is well established (Robertson, Wright, andwith weights p
Dykstra 1988) that under H0 , asymptotically, the statistic T hasa chi-bar squared distribution.

Table 3. MPV’s for the One-Sided, Two-Sample t , Mann–Whitney–Wilcoxon (MWW), and Kolmogorov–Smirnov (KS) Tests
for Various Sample Sizes, Shift Parameters and Underlying Distributionsn = 10n = 20

t

MWW

KS

t

0.49430.21150.10840.04460.5196
0.24970.1271
0.05670.67450.40720.20390.08430.49640.12750.03990.00850.50220.20300.09050.0312.5125
0.12350.05440.01680.68580.21570.08510.0281

0.48630.19550.09470.03410.52020.22340.11290.03760.49590.21540.11250.04770.50160.21190.09840.0370n = 50

MWW

KS

t

MWW

KS

0.50320.13680.04580.0086
Exponential

0.64850.16620.08860.01890.5066
0.03890.00270.0001

0.49270.04130.00240.00000.61810.08880.01000.00070.49240.12210.03360.00380.50780.05050.01040.0012Chi-square(10)0.6397
0.09130.00600.00040.48640.03090.00130.00000.50940.00410.00000.00000.60310.00170.00000.00000.68150.41720.20010.07640.51510.13000.03760.00630.48780.11290.02990.0049
Uniform(0,1)

0.64910.16820.08440.01530.48910.03490.00230.00000.50500.03180.00170.00000.59900.05330.00540.00100.5196
0.24970.1271
0.06470.67450.40720.20390.20390.50130.50320.64850.13050.14260.28970.04310.05060.08860.01040.01220.0416
Double exponential

0.5086
0.03650.00260.00000.49270.04420.00360.00010.61810.08880.01870.00220.51190.17930.08410.03270.68300.21380.08420.02860.50080.12600.03570.00540.49470.03290.00190.00000.50130.01560.00030.00000.60740.01830.00040.0000Normal(0,1)
0 =0.0000¢

1 =0.3600¢

2 =0.5692¢¢

3 =0.8050¢

0 =0.0000
1 =0.3600¢

2 =0.5692¢¢

3 =0.8050¢

0 =0.0000
1 =1.6100¢

2 =2.5456¢¢

3 =3.6000¢

0 =0.0000
1 =0.1039¢

2 =0.1643¢¢

3 =0.2324¢

0 =0.0000
1 =0.5091¢

2 =0.8050¢¢

3 =1.1384

0.5122
0.09280.01970.00330.65070.16880.04260.0068The American Statistician, August 2002, Vol. 56, No. 3

205

We create a random sample T1 ; : : : ; Tn distributed like T , andindependently, another random sample T1¤ ; : : : ; Tn¤ distributed
like T ¤ . An unbiased estimator of EPV is given by
n

AE =

1X
I(Ti¤ ¶ Ti );n i= 1and an unbiased estimator of MPV is given by
n

1X
AM =
I(Ti¤ ¶ med Ti ):ni= 1

The variance of AE is EPV(1 ¡ EPV)=n and that of AM is
MPV(1 ¡ MPV)=n
We consider m = 3; 6 with sample sizes n = 50, 100 and
replications 10,000. The results are given in Table 2. When p =:5, both of the EPV and MPV start slightly higher than .5 as
expected for discrete distributions. For p > :5, the MPV’s are
consistently smaller than the EPV’s, both being very close tozero when p > :65. The effects are more pronounced for larger
n. Note the exact value of the EPV or the MPV is dif cult to
calculate in this case.
We consider a second simulation study to compare the performance of several tests using the MPV’s for a general continuous case. We use the same set up as Sackrowitz and SamuelCahn (1999) but calculate the MPV’s instead. Thus we consider
the two-sample problem of testing H0 : F = G versus H1 :F is stochastically larger than G using two independentrandom
samples from F and G respectively. The test statistics considered are the two-sample t test, the Mann–Whitney–Wilcoxon(MWW) test and the Kolmogorov–Smirnov (KS) test. The comparison is made for shift alternatives so that G(x) = F (x + ¢),where F is chosen as various distributions. We consider F as
normal (0,1), exponential, chi-square with degrees of freedom
10, uniform (0,1) and double exponential. We chose
samplepsizes 10,p20, 50 and ¢0 = 0;¢=
2:546¼
=
50;
¢2 =
1
p
2:546¼ = 20; ¢3 = 2:546¼ = 10, where ¼ is the actual standard deviation of the underlying distribution F . This choice
is made so that when the underlying distribution is normal, atest based on the normal two-sample statistic and n observations would have size ¬ = :10 and power ­ = :7 (as opposedto ­ = :9 of Sackrowitz
and Samuel-Cahn 1999, table 4) forp¢ = (2:546¼ = n) and thus from Table 1 we have MPV =.0355. Note that we have used the same ¢i values for all n. We
consider 10,000 replications.
The MPV values in Table 3 have similar magnitudeas the EPV
values of Sackrowitz and Samuel-Cahn (1999, table 4) even at

206

General­ = :7. They also have similar pattern as the corresponding EPV
values. So their conclusions are also valid in our case. However,
our KS values of MPV perform worse than the corresponding
EPV values of Sackrowitz and Samuel-Cahn (1999) in all cases
considered.
5.

CONCLUSION

The distribution of the p values under the alternative is a
skewed distributionto the right, and hence the median of this distribution is advocated as a more appropriate tool than its meanfor determination of the strength of a test for a particular alternative. The alternatives closer to H 0 are detected easily with
MPV than with EPV. The MPV is easily computed in most casesand does not depend on the speci ed signi cance level of a test.
Thus, it may be used as a single number which can help to choose
among different test statistics when testing the same hypotheses.
For approximately normally distributed statistics, Table 1 can be
consulted to relate the MPV value to the usual signi cance leveland power combinations.
[Received January 2001. Revised August 2001.]

REFERENCES
Blyth, C. R., and Staudte, R. G. (1997), “Hypothesis Estimation and Acceptability Pro les for 2£2 Contingency Tables,” Journal of the American Statistical
Association, 92, 694–699.
Dempster, A. P., and Schatzoff, M. (1965), “Expected Signi cance Level asa Sensibility Index for Test Statistics,” Journal of the American Statistical
Association, 60, 420–436.
Dollinger, M. B., Kulinskaya, E., Staudte, R. G. (1996), “When is a p Value a
Good Measure of Evidence,” in Robust Statistics, Data Analysis and Computer Intensive Methods, ed. H. Reider, New York: Springer-Verlag, pp. 119–
134.
Gibbons, J. D., and Pratt, J. W. (1975), “P-values: Interpretation and Methodology,” The American Statistician, 29, 20–25.
Hung, H. M. J., O’Neill, R. T., Bauer, P., and Kohne, K. (1997), “The Behavior of
the p Value When the Alternative Hypothesis is True,” Biometrics, 53, 11–22.
Lehmann, E. L. (1986), Testing Statistical Hypotheses, (2nd ed.), New York:
Wiley.
Robertson, T., Wright. F. T., and Dykstra, R. L. (1988), Order Restricted Statistical Inference, New York: Wiley.
Sackrowitz, H., and Samuel-Cahn, E. (1999), “P Values as Random VariablesExpected p Values,” The American Statistician, 53, 326–331.
Schatzoff, M. (1966), “Sensitivity Comparisons Among Tests of the General
Linear Hypotheses,” Journal of the American Statistical Association, 61, 415–
435.
Schervish, M. J. (1996), “P Values: What They Are and What They Are Not,”
The American Statistician, 50, 203–206.