Approved For Release 2000/08/08
: CIA-RDP96-00789R003200100001-5
Piycholofical Bulletin
Copyright 1994 bvihe American Psychological Awociation, Inc,
1994. Vni.
I M, No.
I. 19-24
‘
0033-2909/94/S 3.00
Anomaly or Artifact? Comments on Bern and Honorton
Ray Hyman
Bern and Honorton imply that the
1 autoganzfeld experiments demonstrate the existence of psi
a communications anomaly. They claim that the autoganzfeld results are consistent with previous
parapsychological findings and constitute evidence for a replicable psi effect. Although the auto-
ganzfeld experiments arc methodologically superior to previous parapsychological experiments, the
tests of their randomization procedures were inadequate. The autoganzfeld experiments consistently
produced positive hit rates, whose combined effect was highly significant. However, these experi-
ments produced important inconsistencies with
the previous ganzfeld experiments. They
also
showed a unique pattern in the data that may reflect a systematic artifact. Because of these unique
features, we have to wait for independent replications of these experiments before we can conclude
that a replicable anomaly or psi has been demonstrated.
Bern and Honorton (1994) imply that if psychologists were
familiar with the most recent parapsychological research, they
would be more willing to accept the possibility that a commu-
nications anomaly existed. In particular, Bern and Honorton
focus on the experiments that are based on the ganzfeld proce-
dure. They “believe that the replication rates and effect sizes
achieved by one particular experimental method, the ganzfeld
procedure, are now sufficient to warrant bringing this body of
data to the attention of the wider psychological community"
(Bern & Honorton, 1994, p. 4). They review the debate between
Honorton and me over the original ganzfeld experiments. Hy-
man (1985) found that these studies suffered from
statistical,
methodological,
and
documentation
problems.
Honorton
(1985) responded that these flaws were not sufficient to account
for the observed hit rates. Bern and Honorton (1994) review
this controversy and cite reviewers who apparently agree with
Honorton’s position. The implication
is that despite the defi-
ciencies in the ganzfeld experiments, the results support the ex-
istence of psi—a communications anomaly.
To Honorton’s credit, he initiated a new scries of experiments
that would be free from the flaws of the earlier ganzfeld database
(Honorton et al„ 1990). These
1 new experiments, called the
autoganzfeld studies yielded consistently positive hit rates and a
highly significant overall effect. Because these new experiments
showed positive results and allegedly were consistent with the
earlier ganzfeld database and other psi research. Bern and Hon-
orton implied that parapsychology had found its previously elu-
sive repeatable experiment.
Since the beginnings of psychical research in the mid-nine-
teenth century, its investigators have believed that they have sci-
entific evidence sufficiently strong to place before the general
scientific community. Each generation has tried to get the atten-
tion of the scientific community with findings that they claim
to be irrefutable. The particular evidence put forth has changed
from generation to generation. What a previous generation of
Correspondence concerning this article should be addressed to Ray
Hyman, Department of Psychology, University of Oregon, Eugene, Or-
egon 97403.
parapsychologists considered to be a solid case for psi was aban-
doned by later generations in favor of a more current candidate.
This shifting database for parapsychology's best case may be
why parapsychology still has not achieved the recognition it de-
sires from the general scientific community.
Now Bern and Honorton (1994) believe that they have a
strong case to put before the psychological community. They
admit that the autoganzfeld findings still require independent
confirmation. To their credit, they specify the conditions and
the required sample size needed to provide adequate power. The
informed critic of parapsychology might ask what makes the
current situation different from the past claims for psi? Why
should we now believe that Honorton and his colleagues have
finally found a way to consistently produce evidence for psi?
We must wait for future attempts at replication before we
have an answer to the question. Bern and Honorton appear con-
fident that this time is different. Their review of the ganzfeld and
autoganzfeld databases encourages them to believe that consis-
tent psi results are within reach. In this commentary,
I provide
reasons for believing that the autoganzfeld results contain in-
consistencies and some unique patterns that raise doubts about
their replicability.
Agreements and Differences
Although my com mentary focuses on my disagreements with
Bern and Honorton’s (1994) presentation,
I would like to briefly
specify some points of agreement. The autoganzfeld studies do
comply with most of the “stringent standards” (p. 353) spelled
out in the joint communique by Hyman and Honorton (
1 986).
I commend Honorton and his colleagues (1990) for creating a
protocol that eliminates most of the flaws that plagued the orig-
inal ganzfeld experiments. The
1 autoganzfeld studies consis-
tently yield positive effects that, taken together, are highly sig-
nificant.
I concur with Bern and Honorton’s admission that
1 Although
I take a pessimistic position regarding future replications.
I think
it
is good that Bern and the parapsychologists are optimistic.
Such optimism should encourage investigators to attempt replications.
These replications will eventually decide the issue.
Approved For Release 2000/08/08
: CIA-RDP96-00789R003200100001-5
Approved For Release 2000/08/08
: CIA-RDP96-00789R003200100001-5
RAY HYMAN
“the autoganzfeld studies by themselves cannot satisfy the re-
quirement that replications be conducted by a ‘broader range of
investigators’ ” (p.
13).
I also support their suggestion that sev-
eral parapsychologists pool
their resources and plan a
large-
scale ganzfeld replication in which each laboratory contributes
a set of trials to the total pool.
So what is there to disagree about?
I disagree with Bern and
Honorion about how strongly the autoganzfeld studies support
the hope for a replicable psi experiment. Where they see consis-
tency between the autoganzfeld studies and previous parapsy-
chological findings,
I see inconsistency. Although
1 agree that
the autoganzfeld studies meet most of the stringent standards
that Honorton and
I spelled out,
1 disagree that they meet all of
those standards. Our disagreements are a matter of degree. The
value of discussing our disagreements
is to help clarify what
should constitute adequate evidence
for the existence of an
anomaly. Theexistence or nonexistence of psi will not be settled
by debate. The existence issue will be settled by independent
attempts at replication—at least four ofwhich are currently un-
derway (McCrone, 1993).
In explaining my disagreements,
1 point to weaknesses in the
autoganzfeld experiments.
I want to emphasize that as a single
contribution to the ganzfeld database, these are commendable
experiments of high quality. But no single experiment or set of
studies can be perfect in all respects. When such a series is given
the
responsibility of carrying a burden beyond
its original
purposes, then various deficiencies will inevitably become ap-
parent. This is the case,
I believe, with the autoganzfeld studies.
Internal Consistency Within the Autoganzfeld Studies
Bern and Honorton describe the autoganzfeld studies as
separate experiments conducted by eight different experiment-
ers. The hit rates are positive and consistent across the studies
and the experimenters. Although this is encouraging, the con-
sistency tells us little about potential replicability. Neither the
studies nor the experimenters are independent. The studies vary
in whether they use naive or experienced subjects. However, the
target set, the selection and judging procedures, the laboratory,
the setting, and the procedures are identical across studies and
experimenters. No experimenter
is associated with a
single
study, nor does an experimenter have independent input into
the design and procedure as happens in an independent replica-
tion. Indeed, the term experimenter in this context simply refers
to a person who plays an already scripted role. Any unique fea-
tures of the autoganzfeld procedure—including possible arti-
facts—would be the same for all
1 studies and the eight differ-
ent
experimenters.
Consequently
the
autoganzfeld
studies
should be looked on as
l large experiment rather than
1 sepa-
rate contributions.
Consistency With the Original Ganzfeld Database
Bern and Honorton claim that “[the autoganzfeld] results are
statistically significant and consistent with those in the earlier
database” (p.
1 3). They cite only two reasons to support this
claim. The overall effect size or hit rate
is approximately the
same in the two databases. This apparent agreement in overall
effect size
is meaningless. The overall effect size in the auto-
ganzfeld studies
is a composite of two significantly different
effect sizes—-that for the static targets and that for the dynamic
targets. The overall effect size in the ganzfeld data base
is an
arbitrary composite of heterogeneous effect sizes, contributed
in unequal numbers, from different laboratories. The fact that
the two composites yield approximately the same effect size is
accidental. Both numbers could
easily have been
larger or
smaller, depending on the mix of the arbitrary sources from
which they were composed.
The dynamic targets yielded a significantly higher hit rate
than did the static targets in the autoganzfeld studies. Bern and
Honorton argued that this was consistent with the finding that
the multiple-image targets (View Master stereoscopic slide reels)
in the ganzfeld database yielded significantly higher hit rates
than did the single-image targets.
1 do not believe that multiple
static images on a View Master reel can be equated to the dy-
namic moving image on a videoclip. However, I will not argue
this point.
Clearly the dynamic targets outperform the static targets in
the autoganzfeld studies. Even if this is consistent with the ap-
parent superiority of the View Master targets over the single-
image targets, Bern and Honorton ( 1 994) overlook a serious dis-
crepancy. Single-image targets constituted 76% of the 835 ses-
sions in the ganzfeld experiments. Their average hit rate was
.346. Given this effect size and the 166 trials using static targets,
the power or probability of replicating this effect in the auto-
ganzfeld experiments was .82. This failure to find a significant
effect with the static targets was even more notable given that
these experiments were conducted in “the warm social ambi-
ence’* (p. 14) of Honorton’s laboratory.
Bern and Honorton acknowledge that the autoganzfeld stud-
ies
failed
to
replicate
the predicted
sender-receiver pairing
effect. In the original ganzfeld database, the trials on which the
receiver chose a friend as a sender produced a hit rate of .44
compared with a hit rate of only .26 for those trials on which
the experimenter assigned a sender.
1 would emphasize that
given this size effect with the 198 trials with friends as senders
and 128 with someone else as senders, the power of getting a
significant replication of the effect is over .92. Again, given the
‘psi conducive’ atmosphere of Honorton’s laboratory, this fail-
ure to get significance is a noteworthy inconsistency.
On two key comparisons with the original ganzfeld database,
the autoganzfeld
fails to replicate even with adequate power.
The positive hit rate and overall significance ofthe autoganzfeld
studies are due to an essentially new type of target, presented in
a new way. Even if we agree that there is a kinship between the
View Master reels of the ganzfeld experiments and the dynamic
targets of the autoganzfeld, we cannot ignore the differences be-
tween multiple images of a travel scene presented statically with
a slide projector and excerpts from motion pictures presented
with their accompanying audio on videocassettes. The prob-
lems of selecting, presenting, and controlling such targets pres-
ent new challenges. During the judging procedure in the origi-
nal ganzfeld experiments, the target and the decoys were dis-
played
simultaneously.
The
judging
procedure
for
the
autoganzfeld involves presenting the target and its decoys one
at a time. Because the positive hit rate and significance are due
to an essentially new type of target presented in a new way, the
need for independent replication is especially urgent.
Approved For Release 2000/08/08
: CIA-RDP96-00789R003200100001-5