MA
ap^oved
P
For ReleaseU)N£iiaASSlR6
JR-00789R002200390001-5
we developed
(May, Radin
et
al,
1985)
requires binary
statistics.
In
addition,
to
avoid the
possibility that reported p values or z scores were rounded up,
whenever possible we recorded the
reported number
of
trials
(bits
generated
in an experiment)
and
hits (number
of times the
designated bit was obtained) in these experiments.
Of these 332 binary experiments, 188 were reported in'joumals and conference proceedings, of
which 58 were reported significant at p < .05, 2-tailed,
(against 9.4 expected by chance).
The
probability of observing 58 significant studies out of 188 is less than 10~57
. We refer to this body of
data
as
the
“survey.”
The
remaining
144
experiments
were
obtained
from
the
Princeton
Engineering
Anomalies
Research
Laboratory
(Dunne,
Jahn
and
Nelson,
1982).
Of
these
experiments, 13 were significant 2-tailed, resulting in p < .04 (corrected for continuity). We refer
to these experiments as the “Princeton” data.
Experiments with incomplete descriptions
Of the 188 survey studies, 30 were simulated by
Monte Carlo techniques because the experiment was reported as nonsignificant but neither the z
score nor the number of
trials and
hits were provided.
To perform the
simulation, we had
a
pseudorandom generator
(cf. May, Humphrey and Hubbard, 1980) choose a
z score
at random
from a normal distribution [N(0,1)], but bounded between lO"25
to 1.64 and -10"25
to -1.64.3
In five additional studies, the results were reported as significant and p or z values were provided,
but the number of trials or hits were not given.
For these five studies, since the z score was known
or could be calculated from a p value, the trials or hits (whichever was missing) were calculated.
Table
1 shows a breakdown of the number of experiments reported
in each of the seven
reference sources we used.
It is clear that the reports provided in the Research in Parapsychology
series are not as detailed as one might have wished, but it is not surprising since the contents of this
reference are only abstracts of the full papers presented at the annual Parapsychological Association
conventions.
Table
1.
Experiment breakdown by source of reference.
Reference
Experiments
Experiments
with
full detail
with partial detail
Journal of the American Society for Psychical Research
European Journal of Parapsychology
Journal of the Society for Psychical Research
Proceeding* of the Parapsychological Association*
Journal of Parapsychology
Research in Parapsychology
Princeton Engineering Anomaliei Research Lab
144
• tor the years 1971 and 1984
3. We did not generate z scores of zero because this data was ultimately used in an evaluation of our model
(May, Radin et al, 1985), in which log(z)
is taken.
B-6
UNCLASSIFIED
Approved For Release 2000/08/08
: CIA-RDP96-00789R002200390001-5
I
IM
I A C C I C I C
Meta-Analysis Part
Approved For Release MW/06wft%1«-RwM-00789R002200390001-5
In summary,
of the
332
experiments we
considered
(188 from the
survey and
144 from
Princeton), 71 were reported significant at p < .05, 2-tailed, for an overall binomial probability of
p <
5.4
x lO" 43
.
ADDRESSING CRITICISMS OF THE DATA
Taken as prima facie evidence, one might think that
this body of published data provides
indisputable evidence that an anomaly exists.
But there are numerous reasons why the data may be
suspect.
The main criticisms (Akers, 1984; Hansel, 1980; Hyman,
1985;
Kurtz, 1981) include
1. Results are due to chance
2. Basic statistical assumptions are violated
3. Only significant studies are published
4. Experiments are not replicable
5. RNGs are nonrandom
6. Poorer studies are included with better studies
Let us consider each of these six steps as successive filters for the reliability of the data.
If each
criticism can be satisfactorially refuted or countered, then a persuasive case for an anomalous effect
can be made.
1. Results are due to chance
In any one experiment we cannot establish the
reality of a phenomenon,
regardless of the
significance
level,
unless
strong
theoretical
predictions have preceeded the
experiments.
For
example, the recent experiments suggesting that Bell’s inequality is violated (e.g. Aspect, Dalibard,
and Roger, 1982; Aspect, Grangier and Roger, 1982) have been widely accepted within the physics
community on the basis of only a few empirical studies despite its profound implications on our view
of the nature of reality
(cf. d’Espagnat, 1979; Mermin, 1985; Rohrlich,
1983).
Parapsychology,
however, has had the disadvantage of not having a firm theoretical base on which to stand.
Thus
the nature
of the claim
(any claimed
psi
effect) understandably requires extremely persuasive
evidence.
One wonders how statistically strong an effect must be to bring about a consensual agreement
within the scientific community that a psi effect on RNGs is real. Would p < 10"43 be sufficient?
If this figure were revised to take into account all of the criticisms noted above, and the end result
were say, 10" s
, would that be sufficient?
Clearly an overall p =
.1 would not satisfy anyone, so
there
is a decision curve related to this question.
This curve
is probably different according to
individual prejudices and predilictions, but the resolution of this question is beyond the scope of the
present paper.
Note that
if an anomaly did exist,
it would not necessarily imply that psi was the
mediating factor.
Such an anomaly may, for example, reveal some heretofore unknown statistical
peculiarities about random numbers.
B-7
UNCLASSIFIED
Approved For Release 2000/08/08
: CIA-RDP96-00789R002200390001-5