Approved For^mmmmSMfmmm000^ 30002^
not all have a chance baseline of .25. The next most obvi-
ous candidate, the difference in each study between the
hit rate observed and the hit rate expected under the null
hypothesis, is also intuitively descriptive but is not appro-
priate for statistical analysis because not all differences
between proportions that are equal are equally detectable
(e.g., the power to detect the difference between .55 and
.25 is different from the power to detect the difference be-
tween .50 and .20).
In order to provide a scale of equal detectability, Cohen
(1988) devised the effect size index h, which performs an
arcsine transformation on the proportions before calculat-
ing their difference. Cohen’s h is quite general and can as-
sess the difference between any two proportions drawn
from independent samples or between a single proportion
and any specified hypothetical value. For the 28 studies
examined in the meta-analyses, h is .28, with a 95% confi-
dence interval from .11 to .45.
But because values of h do not provide an intuitively
descriptive scale, Rosenthal and Rubin (1989; Rosenthal,
1991) have recently suggested a new index,
it, which ap-
plies
specifically to one-sample, multiple-choice data of
the kind obtained in ganzfeld experiments. In particular,
it expresses
all
hit rates as the proportion of hits that
would have
l>een obtained
if there had been only two
equally likely alternatives—essentially a coin flip. Thus,
it
ranges from 0 to
1, with
.5 expected under the null hy-
pothesis. The formula is:
P(k- 1)
~P(k-2) +
where P is the raw proportion of hits and k is the number
of alternative choices
available. Because
it has such a
straightforward intuitive interpretation, we will use it (or
its conversion back to an equivalent four-alternative hit
rate) throughout this article whenever it is applicable.
For the 28 studies examined in the meta-analyses, the
mean value of it is
.62, with a 95% confidence interval
from .55 to .69. This corresponds to a four-alternative hit
rate of 35%, with a 95% confidence interval from 28% to
43%.
Cohen (1988, 1992) has also categorized effect sizes into
small, medium, and large, where medium denotes an ef-
fect size that should be apparent to the naked eye of a
careful observer. For a statistic like
it, which indexes the
deviation of a proportion from .5, Cohen considers .65 to
be a medium effect size: A statistically unaided observer
should be able to detect the bias of a coin that comes up
heads on 65% of the trials. Thus, at .62, the psi ganzfeld
effect size falls just short of Cohen’s naked-eyeball crite-
rion. From the phenomenology
of the ganzfeld
experi-
menter, the corresponding hit rate of 35% implies that he
or she will see a subject obtain a hit approximately every
third session rather than every fourth.
It is also instructive to compare the psi ganzfeld effect
with the resul ts of a recent medical study that sought to
determine
whether
aspirin can
prevent
heart
attacks
(Steering Committee of the Physicians’ Health Study Re-
search Group, 1988). The study was discontinued after six
years because it was already clear that the aspirin treat-
ment was effective (%2 = 25.01, p < .00001) and it was con-
sidered unethical
to keep the control group on placebo
medication. The study was widely publicized as a major
medical breakthrough. But despite its undisputed reality
and practical importance, the size of the aspirin effect is
quite small: Taking aspirin reduces the probability of suf-
fering a heart attack by only 0.008. The corresponding ef-
fect size
(h) is .068—about 1/3 to 1/4 the size of the psi
ganzfeld effect (Atkinson et al., 1993, p. 236; Utts, 1991b).
In sum, we believe that the psi ganzfeld effect is large
enough
to be of both theoretical interest and potential
practical importance.
Experimental Correlates of the Psi Ganzfeld Effect
We saw above that the technique of correlating vari-
ables with effect sizes across studies can help to assess
whether methodological flaws might have produced arti-
factual
positive
outcomes. The same technique can be
used more
affirmatively
to
explore
whether an
effect
varies
systematically with
conceptually
relevant
varia-
tions in experimental procedure. The discovery of such
correlates can help to establish an effect as genuine, sug-
gest ways of increasing replication rates and effect sizes,
and enhance the chances of moving beyond the simple
demonstration of an effect to its explanation. This strat-
egy is only heuristic, however. Any correlates discovered
must be considered
quite
tentative, both because they
emerge from post hoc exploration and because they neces-
sarily involve comparisons across heterogeneous studies
that differ simultaneously on many interrelated variables,
known and unknown. Two such correlates emerged from
the meta-analyses of the psi ganzfeld effect.
Single versus multiple-image targets.
Although most of
the 28 studies in the meta-analysis used single pictures as
targets, 9 of the studies (conducted by three different in-
vestigators) used
View Master
stereoscopic
slide
reels
which presented multiple images focused on a
central
theme. Studies using the View Master reels produced sig-
nificantly higher hit rates than did studies using the sin-
gle-image targets (50% vs. 34%), t (26) = 2.22, p = .035,
two-tailed.
Sender / receiver pairing. In 17 of the 28 studies, partici-
pants were free to bring in friends to serve as senders. In
studies,
only
laboratory-assigned
senders
were em-
ployed. (Three
studies used no sender.) Unfortunately,
there is no record of how many participants in the former
studies actually brought in friends. Nevertheless, those 17
studies (by six different investigators) had significantly
higher hit rates than did the studies that used only labo-
ratory-assigned senders (44% vs. 26%),
t (23) = 2.39, p =
.025, two-tailed.
The Joint Communique
Following their published exchange in
1985, Hyman
and Honorton agreed to contribute a joint communique to
the subsequent discussion which was published in 1986.
First they set forth
their areas of agreement and
dis-
agreement:
We agree that there
is an overall significant effect in this
data base that cannot reasonably be explained by selective
reporting or multiple analysis. We continue to differ over
the degree to which the effect constitutes evidence for psi,
but we agree that the final verdict awaits the outcome of fu-
ture experiments conducted by a broader range of investiga-
tors and according to more stringent standards. (Hyman &
Honorton, 1986, abstract, p. 351)
Approved For Release 2000/08/10
: CIA-RDP96-00792R0001 001 30002-1
Approved For
001 30002-1
They then
spelled out in
detail
the “more
stringent
standards” they
believed should govern
future
experi-
ments. These include strict security precautions against
sensory leakage, testing and documentation of randomiza-
tion methods
for
selecting targets and sequencing the
judging pool, statistical correction for multiple analyses,
advance specification of the status of the experiment (e.g.,
pilot study, confirmatory experiment), and full documen-
tation in the published report of the experimental proce-
dures and the status of statistical tests (e.g., pre-planned
or post hoc).
The NRC Report
In 1988, the National Research Council (NRC) of the
National Academy of Sciences released a widely publi-
cized report commissioned by the U. S. Army which as-
sessed several controversial technologies
for enhancing
human performance, including accelerated learning, neu-
rolinguistic programming, mental practice, biofeedback,
and parapsychology (Druckman & Swets, 1988; summa-
rized in Swets & Bjork,
1990). The
report’s conclusion
concerning
parapsychology
was
quite
negative:
“The
Committee finds no scientific justification from research
conducted over a period of 130 years for the existence of
parapsychological phenomena” (p. 22).
An extended refutation strongly protesting the Commit-
tee’s
treatment
of parapsychology has been
published
elsewhere (Palmer et
al., 1989). The pertinent point here
is simply that the NRC’s evaluation of the ganzfeld stud-
ies does not reflect an additional, independent examina-
tion of the ganzfeld database but is based on the same
meta-analysis by Hyman that we have discussed in this
article.
Hyman chaired the NRC’s Subcommittee on Parapsy-
chology; and, although he had concurred with Honorton
two years earlier in their joint communique that “there is
an overall significant effect in this data base that cannot
reasonably be explained by selective reporting or multiple
analysis”
(p.
351) and that
“significant outcomes have
been produced by a number of different investigators” (p.
352),
neither
of these
points
is
acknowledged
in
the
Committee’s report.
The NRC also solicited a background paper from Harris
and Rosenthal (1988a), who provided the Committee with
a comparative methodological analysis of the five contro-
versial areas
listed above. Harris and Rosenthal noted
that of these areas, “only the Ganzfeld ESP studies [the
only psi studies they evaluated] regularly meet the basic
requirements of sound experimental design” (p. 53), and
they concluded that “it would be implausible to entertain
the
null
given the combined p
from these 28
studies.
Given the various problems or flaws pointed out by Hy-
man and Honorton. ..we might estimate the obtained ac-
curacy rate to be about
1/3. ..when the accuracy rate ex-
pected under the null is 1/4” (p. 51)3
3In a troubling development, the chair of the NRC Committee
phoned Rosenthal tind asked him to delete the parapsychology
section
of the
papier
(R.
Rosenthal,
private
communication,
September 15, 1992). Although Rosenthal refused to do so, that
section of the Harris-Rosenthal paper is nowhere cited in the
NRC report.
;;
The Autoganzfeld Studies
In '1983, Honorton and his colleagues initiated a new
series of ganzfeld studies designed to avoid the method-
ological problems he and others had identified in earlier
. studies (Honorton, 1979; Kennedy, 1979). These studies
complied with all the detailed guidelines that he and Hy-
man were to publish later in their joint communique. The
program continued until September of 1989, when a loss
of funding forced the laboratory to close.
The major innovations of the new studies were the com-
puter
control
of the
experimented
protocol—hence
the
name “autoganzfeld”—and the introduction of videotaped
film clips as target stimuli.
Method4
The basic design of the autoganzfeld studies was the
same as that described earlier: A receiver and sender were
sequestered in separate,
acoustically-isolated chambers.
Following a 14-minute period of progressive relaxation,
the
receiver underwent
ganzfeld
stimulation while
de-
scribing his or her thoughts and images aloud for 30 min-
utes. Meanwhile, the sender concentrated on a randomly
selected target. At the end of the ganzfeld period, the re-
ceiver was shown
four
stimuli
and,
without knowing
which of the four had been the target, rated each stimulus
for
its
similarity
to
his
or
her mentation
during
the
ganzfeld.
The targets consisted of 80 still pictures (Static Targets)
and 80 short video segments complete with squndtracks
(Dynamic
Targets),
all
recorded on
videocassette. The
static targets included art prints, photographs, and mag-
azine advertisements; the dynamic targets included ex-
cerpts of approximately one minute duration from motion
pictures, TV shows, and cartoons. The 160 targets were
arranged in judging sets of four static or four dynamic
targets each, constructed to minimize similarities among
targets within a set.
Target selection and presentation.
The VCR containing
the taped targets was interfaced to the controlling com-
puter, which
selected the
target and controlled
its
re-
peated presentation to the sender during the ganzfeld pe-
riod, thus eliminating the need for a second experimenter
to accompany the sender. Following the ganzfeld period,
the computer randomly sequenced the four-clip judging
pool and presented it to the receiver on a TV monitor for
judging. The receiver used a computer game paddle
to
make his or her ratings on a 40-point scale which ap-
peared on the TV monitor after each clip was shown. The
receiver was permitted to see each clip and to change the
ratings repeatedly until he or she was satisfied. The com-
puter then wrote these and other data from the session
into a
file on a floppy
disk. At that point, the sender
moved to the receiver’s chamber and revealed the identity
of the target to both the receiver and the experimenter.
Note that the experimenter did not even know the identity
of the four-clip judging pool until it was displayed to the
receiver for judging.
Because Honorton and his colleagues have complied with the
Hyman-Honorton specification that experimental reports be suf-
ficiently complete to permit others to reconstruct the investiga-
tor’s procedures, readers who wish to know more detail than we
provide here are likely to find whatever they need in the archival
publication of these studies
in the Journal of Parapsychology
(Honorton et al., 1990X
Approved For Release 2000/08/10
: CIA-RDP96-00792R0001 001 30002-1
Approved For Rele«seN£flO(MD^0oi(MAiRDPflfi^F'S&R0001 001 30002-1
Randomization. The random selection of the target and
sequencing of the judging pool were controlled by a noise -
based random number generator interfaced to the com-
puter. Extensive testing confirmed that the generator was
providing a uniform distribution of values throughout the
full target range (1—160), a uniform distribution of targets
from among the four alternatives within each judging set,
and
a uniform
distribution
of judging sequences from
among the 24 permutations of 4 stimuli.
Additional control features.
Both
the
receiver’s and
sender’s rooms were sound-isolated, electrically-shielded
chambers with single-door access that could be continu-
ously monitored by the experimenter. There was two-way
intercom communication between the experimenter and
the
receiver but only one-way communication into
the
sender’s room; thus neither the experimenter nor the re-
ceiver could monitor events inside the sender’s room. The
archival record for each session includes an audio tape
containing the receiver’s mentation during the ganzfeld
period and all verbal exchanges between the experimenter
and the receiver throughout the experiment.
The automated ganzfeld protocol has been examined by
several
dozen
parapsychologists
arid
behavioral
re-
searchers from other fields, including well-known critics
of parapsychology. Many have participated as subjects or
observers. All have expressed satisfaction with the han-
dling of security issues and controls.
Parapsychologists have often been urged to employ ma-
gicians as consultants
to ensure that the experimental
protocols are not vulnerable either to inadvertent sensory
leakage or to deliberate cheating. Two “mentalists,” magi-
cians who specialize in the simulation of psi, have exam-
ined the autoganzfeld system and protocol. Ford Kross, a
professional mentalist and officer of the mentalist’s pro-
fessional organization, the Psychic Entertainers Associa-
tion, provided the following written statement “In my pro-
fessional capacity as a mentalist,
I have reviewed Psy-
chophysical
Research
Laboratories’ automated ganzfeld
system and found it to provide excellent security against
deception
by
subjects”
(private
communication,
May,
1989).
The first author of this article has also performed as a
mentalist for many years and is a member of the Psychic
Entertainers Association. As noted in the author footnote,
this article has its origins in a 1983 visit he made to Hon-
orton’s laboratory, where he was asked to critically exam-
ine the research protocol from the perspective of a mental-
ist, a research
psychologist, and a subject. Needless to
say, this article would not exist if he did not concur with
Ford Kross’s assessment of the security procedures.
Experimental Studies
Altogether 100 men and 141 women participated as re-
ceivers in 355 sessions during the research program. The
participants ranged in age from 17 to 74 years (mean =
37.3, SD =
11.8), with a mean formal education of 15.6
years (SD = 2.0). Eight separate experimenters, including
Honorton, conducted the studies.
The
experimental program included
three
pilot and
eight formal studies. Five of the formal studies employed
novice (first-time) participants who served as the receiver
in one session each. The remaining three formal studies
employed experienced participants.
Pilot Studies. Sample sizes were not preset in the three
pilot studies. Study 1 comprised 22 sessions and was con-
ducted during the initial development and testing of the
autoganzfeld system. Study 2 comprised 9 sessions testing
a procedure in which the experimenter, rather than the
receiver, served as the judge at the end of the session.
Study 3 comprised 36 sessions and served as practice for
participants who had completed the allotted number of
sessions in the ongoing formal studies but who wanted
additional ganzfeld experience. This study also included
several demonstration sessions when TV film crews were
present.
Novice Studies. Studies 101-104 were each designed to
test 50 participants who had had no prior ganzfeld experi-
ence; each participant served as the receiver in a single
ganzfeld session. Study 104 included 16 of 20 students re-
cruited from the Juilliard School of Music in New York
City in order to test an artistically gifted sample. Study
105 was initiated to accommodate the overflow of partici-
pants who had been recruited for Study 104, including the
four remaining Juilliard students.
Sample size for this
study was set to 25, but only 6 sessions had been com-
pleted when the laboratory closed. For purposes of exposi-
tion, we have divided the 56 sessions from Studies 104
and 105 into two parts: Study 104/105 (a) comprises the
36 non-Juilliard participants; Study 104/105 (b) comprises
the 20 Juilliard students
Study 201.
This study was designed to retest the most
promising
participants from
the
previous
studies. The
number of trials was set to 20, but only 7 sessions with 3
participants had been
completed when
the
laboratory
closed.
Study 301.
This study was designed to compare static
and dynamic targets. Sample size was set to 50 sessions.
Twenty-five experienced participants each served as the
receiver in two sessions. Unknown to the participants, the
computer
control program was modified
to ensure that
they would each have one session with a static target and
one session with a dynamic target.
Study 302.
This study was designed to examine a dy-
namic target set which contained one target that had of-
ten been correctly identified in the previous studies and
another target that had never been correctly identified.
The study involved experienced participants who had had
no prior experience with this particular target set and
who were unaware that only one target set was being
sampled. Each served as the receiver in a single session.
The design called for the study to continue until 15 ses-
sions were completed with each of the targets, but only
twenty-five sessions had been completed when the labora-
tory closed.
The 11 studies just described comprise all sessions con-
ducted during the 6-1/2 years of the program. There is no
“file drawer” of unreported sessions.
Results
Overall hit rate.
As in the earlier meta-analysis,
re-
ceivers’ ratings were analyzed by tallying the proportion
of hits achieved and calculating the exact binomial prob-
ability for the observed number of hits compared with
chance expectation of
.25. As noted above, 241
partici-
pants contributed 355 sessions. For reasons discussed be-
low, Study 302 is analyzed separately, reducing the num-
ber of sessions in the primary analysis to 330.
As Table
1 shows, there were 106 hits in the 330 ses-
sions, a hit rate of 32%
(
z
= 2.85, p =
.002, one-tailed),
with a 95% confidence interval from 30% to 35%. This cor-
Approved For Release 2000/08/10
: CIA-RDP96-00792R0001 001 30002-1
Approved For
001 30002-1
Table
Outcome by Study
N
N
N
%
Effect size
Study
Study type
subjects
trials
hits
hits
K
z
Pilot
.62
.99
Pilot
.60
.25
Pilot
.54
.22
101
Novice
.47
-.30
102
Novice
.63
1.60
103
Novice
.55
.67
104/105 (a)
Novice
.60
.97
104/105 (b)
Juilliard Sample
.75
.
2.20
201
Experienced
.69
.69
301
Experienced
.56
.67
302
Experienced
'
44a
.70 a
2.02 a
Overall
(Studies 1-301)
241
330
106
.59
2.85
Note. Z scores are based on the exact binomial probability with p = .25 and q=.75.
Corrected for maximum possible response bias. Hit rate actually observed was 64%.
responds to an effect size
of .59, with a 95% confidence
interval from .53 to .64.
Table 1 also shows that when Studies 104 and 105 are
combined and re-divided into the non-Juilliard and Juil-
liard samples, 9 of the
10 studies yield
positive
effect
sizes, with a mean effect size
(zr) of .61, t (9) = 4.35, p =
.0009, one-tailed. This effect size is equivalent to a four-al-
ternative hit rate of 34%. Alternatively, if we retain Stud-
ies 104 and 105 as separate studies, 9 of the 10 studies
again yield positive effect sizes, with a mean effect size (x)
of .62,
t (9) = 3.67, p = .003, one-tailed. This effect size is
equivalent to a four-alternative hit rate of 35% and
is
identical to that found across the 28 studies of the earlier
meta-analysis.5
Considered together, sessions with novice participants
(Studies 101-105) yielded a
statistically
significant
hit
rate of 32.5% (p = .009), which is not significantly differ-
ent from the 31.6% hit rate achieved by experienced par-
ticipants in Studies 201 and 301. And finally, each of the 8
experimenters also achieved a positive effect size, with a
mean n of .60, t (7) = 3.44, p = .005, one-tailed.
5As noted above, the laboratory was forced to close before three
of the formal studies could be completed. If we assume that the
remaining trials in Studies 105 and 201 would have yielded only
chance results, this would reduce the overall z for the first 10
autoganzfeld studies from 2.85 to 2.73 (p = .003). Thus, inclusion
of the two incomplete studies does not pose an optional stopping
problem. The
thii-d incomplete study, Study 302,
is discussed
below.
The Juilliard sample. There are several reports in the
literature of a relationship between creativity or artistic
ability and psi performance (Schmeidler, 1988). In order
to explore this possibility in the ganzfeld setting, 10 male
and 10 female undergraduates were recruited from the
Juilliard School of Music in New York City. Of these, 8
were music students, 10 were drama students, and 2 were
dance students. Each served as the receiver in a single
session in Studies 104 or 105. As shown in Table
1, these
students achieved a hit rate of 50% (p = .014), one of the
five highest hit rates ever reported for a single sample in
a ganzfeld study. The musicians were particularly suc-
cessful: Six of the eight (75%) successfully identified their
targets (p = .004). Further details about this sample and
their ganzfeld performance are reported in
Schlitz and
Honorton (1992).
Study size and effect size. There is a significant negative
correlation across the 10 studies listed in Table 1 between
the number of sessions in a study and its effect size in): r
= -.64, t (8) = 2.36, p < .05, two-tailed. This is reminiscent
of Hyman’s discovery that the smaller studies in the orig-
inal ganzfeld database were disproportionately likely to
report statistically significant results. He interpreted this
finding as evidence
for a bias against the reporting of
small studies that
fail
to achieve significant results. A
similar
interpretation
cannot be
applied
to
the
auto-
ganzfeld
studies, however, because there
are no unre-
ported sessions.
One reviewer of this article suggested that the negative
correlation might reflect a decline effect in which earlier
sessions of a study are more successful than
later ses-
Approved For Release 2000/08/10
: CIA-RDP96-00792R0001 001 30002-1