* Approved For Release U(WGl<)A:S5c|-6lfi€^-00789R002200390001 -5
then be evaluated
statistically, where under the
null hypothesis an average
of 50
hits would be
expected by chance.
If the average number of hits over thousands of repetitions were say, 52, this
deviation
from
chance would be
interpreted
as
evidence
of
a
psi
effect
(provided
that the
probability of observing this deviation was less than
1 in 20).
PROCEDURE
Because we were ultimately interested in testing among several different models of mechanisms
possibly
operating in these RNG experiments,
in
Part
of this meta-analysis
(this paper) we
surveyed the parapsychological literature with two goals in mind:
First, we wanted to see whether
the aggregated
result of the RNG experiments showed evidence
for an anomalous
effect.
And
second, we needed the details of these experiments for use in evaluating a model of the underlying
mechanism.
[Our modeling
effort
is discussed in May, Radin, Hubbard, Humphrey and Utts
(1985).]
Source of references
We searched through the five major English language parapsychological journals 1
over the years
1969 to 1984. We also included the (refereed) Proceedings of Presented Papers for the Annual
Parapsychological
Association
Conventions
(1971
and
1984),
and
a
report published by the
Princeton Engineering Anomalies Research Laboratory
(Nelson,
Dunne and Jahn,
1984).
The
literature search was started in the year 1969 because that was the year Helmut Schmidt (1969)
published the seminal RNG study that has since spawned many replications.
Defining “an experiment
One of the difficulties faced in reviewing the articles for this meta-analysis was to decide what
constituted an experiment.
In most papers, authors analyze their data repeatedly in various ways,
sometimes as a priori analyses, sometimes as post hoc afterthoughts.
Even in cases of planned
analyses, there are many ways of interpreting which of several conditions is the “real” experiment.
How we decide what
is an experiment
is important to the meta-analysis for two main reasons:
First, the meta-analytic
statistical power depends on the number of experiments we
find; and
second, the z scores are different depending on how we break down the reported results.
To illustrate the difficulty of deciding what an experiment
is, consider this example.
Say an
author uses three
different groups
of
percipients each
(e.g.
meditators,
truck
drivers and
athletes) and subjects each group to two different conditions
(e.g. mental imagery vs. muscular
tension)
in a study on psi-conducive states.
The results can be broken into one big, combined
experiment,
six experiments
(3 groups x 2
conditions), two experiments
(2
conditions), three
J.
These are the
Journal of Parapsychology, European Journal of Parapsychology, Research In Parapsychology,
Journal of the Society for Psychical Research, Journal of the American Society for Psychical Research.
B-4
UNCLASSIFIED
Approved For Release 2000/08/08
: CIA-RDP96-00789R002200390001-5
Approved For Release
ASWWE&oo789Roo22
a
o"o39i001-5
experiments
(3 groups), 30 experiments
(subject by subject analysis), and so on.
How do we
decide what to use?
We resolved this issue for this first-pass analysis in the following way:
For cases where there
were multiple hypotheses under test and multiple analyses of the data, we chose as the experimental
unit the largest possible accumulation of data compatible with a single "direction of effort” assigned
to the subjects. A clearly defined direction of effort meant that the experimental protocol required
either more
l's or more 0’s from the RNG to successfully complete the assigned task, regardless of
whether or not the subjects actually knew their task in detail.
Say, for example, a hypothesis predicted that group A would score higher than group B, and
it
was stated that “higher” meant more
bits.
Then we would take this study as two experiments:
Group A’s and group B’s scores.
In this particular case, since group A was predicted to score
higher than group B,
if in fact the difference between z(A) and z(B) were significant, then both z
scores would be taken as positive, regardless of the reported z s.
Thus if z(A) = 1.5, z(B) — —1.0,
then the z-score difference between them would be significant one-tailed with zditt= 1.77.
If the
number of trials run in each case were 10000, then the number of hits assigned per experiment
would be
hits (A) = 5075 and hits(B) = 5050, which are both positive
deviations;
similarly the z
scores would be recorded as z(A) =
1.5, z(B) =
1.0.
If z(A) = 1.2 and z(B) = —1.0, the z scores
would be recorded as originally reported since
zditt is not significant.
The same would be true
if
z(A)
= -2.0 and
z(B)
=
2.0.
(Fortunately,
such problems
of
interpretation were
not often
encountered in the survey.)
As another example,
if groups A, B, and C
all tried to influence an RNG in a particular way,
and no predictions were made as to interactions, then their overall result would be combined as one
experiment.
In this way, we attempted to emphasize in the meta-analysis the underlying question
of whether or not observers could influence or otherwise affect the
statistical output of an RNG
according to the stated intention of the experimenter .
Results of literature review
We found 73 pertinent references in the journals and reports.2
These references included 381
experiments contributed by 38
different principal
investigators,
representing about
different
laboratories around the world. We say "about 10 laboratories” because over the years labs have
come and gone, researchers have moved among different labs, and
in many cases, one or two
individuals at an academic or private research institution are considered a “laboratory.”
Breakdown of experiments
Of the 381 experiments found, 332
(in 56 remaining references)
were described as using binary generators based on either radioactive decay or electronic noise.
For this meta-analysis, we considered only studies using binary RNGs (or any study in which the hit
rate was defined
or could be
interpreted
as 50%)
for three
reasons:
First,
since 87% of the
experiments
(332/381) employed binary generators, we felt that this sample was representative of
the entire RNG database; second, for the sake of simplicity; and, third because the test of a model
2. These references are listed under the heading “Meta-Analysis” in the references at the end of this paper.
B-5
UNCLASSIFIED
Approved For Release 2000/08/08
: CIA-RDP96-00789R002200390001 -5