% academic colleagues as evidence for their beliefs about psi agner & Monnet, 1979). ronically, however, psychologists are probably not more fa- \jar than others with recent experimental research on psi. ke most psychological research, parapsychological research is orted primarily in specialized journals; unlike most psycho- ical research, however, contemporary parapsychological re- rch is not usually reviewed or summarized in psychology’s xtbooks, handbooks, or mainstream journals. For example, ly | of 64 introductory psychology textbooks recently sur- yed even mentions the experimental procedure reviewed in is article, a procedure that has been in widespread use since e early 1970s (Roig, Icochea, & Cuzzucoli, 1991). Other sec- ndary sources for nonspecialists are frequently inaccurate in heir descriptions of parapsychological research. (For discus- ions of this problem, see Child, 1985, and Palmer, Honorton, Utts, 1989.) This situation may be changing. Discussions of modern psi esearch have recently appeared in a widely used introductory extbook (Atkinson, Atkinson, Smith, & Bem, 1990, 1993), two mainstream psychology journals (Child, 1985; Rao & Palmer, 1987), and a scholarly but accessible book for nonspecialists (Broughton, 1991). The purpose of the present article is to sup- plement these broader treatments with a more detailed, meta- analytic presentation of evidence issuing from a single experi- mental method: the ganzfeld procedure. We believe that the replication rates and effect sizes achieved with this procedure are now sufficient to warrant bringing this body of data to the | attention of the wider psychological community. The Ganzfeld Procedure | By the 1960s, a number of parapsychologists had become dis- satished with the familiar ESP testing methods pioneered by J. B. Rhine at Duke University in the 1930s. In particular, they believed that the repetitive forced-choice procedure in which a subject repeatedly attempts to select the ‘correct “target” sym- bol from a set of fixed alternatives failed to capture the circum- stances that characterize reported instances of psi in everyday life. Historically, psi has often been associated with meditation, hypnosis, dreaming, and other naturally occurring or deliber- ately induced altered states of consciousness. For example, the view that psi phenomena can occur during meditation is ex- pressed in most classical texts on meditative techniques; the be- lief that hypnosis is a psi-conducive state dates all the way back to the days of early mesmerism (Dingwall, 1968); and cross- cultural surveys indicate that most reported “real-life” psi ex- periences are mediated through dreams (Green, 1960; Prasad & Stevenson, 1968; L. E. Rhine, 1962; Sannwald, 1959). There are now reports of experimental evidence consistent with these anecdotal observations. For example, several labora- tory investigators have reported that meditation facilitates psi performance (Honorton, 1977). A meta-analysis of 25 experi- ments on hypnosis and psi conducted between 1945 and 1981 in 10 different laboratories suggests that hypnotic induction may also facilitate psi performance (Schechter, 1984). And dream-mediated psi was reported in a series of experiments conducted at Maimonides Medical Center in New York and Approved For Release 2000/08/08 : CIA-RDP96-00789R003200110001-4 ANOMALOUS INFORMATION TRANSFER 5 published between 1966 and 1972 (Child, 1985; Ullman, Krippner, & Vaughan, 1973). In the Maimonides dream studies, two subjects—a “receiver” and a “‘sender”—spent the night in a sleep laboratory. The re- ceiver’s brain waves and eye movements were monitored as he or she slept in an isolated room. When the receiver entered a period of REM sleep, the experimenter pressed a buzzer that signaled the sender—under the supervision of a second experi- menter—to begin a sending period. The sender would then con- centrate on a randomly chosen picture (the “target”) with the goal of influencing the content of the receiver’s dream. Toward the end of the REM period, the receiver was awak- ened and asked to describe any dream just experienced. This procedure was repeated throughout the night with the same target. A transcription of the receiver’s dream reports was given to outside judges who blindly rated the similarity of the night’s dreams to several pictures, including the target. In some studies, similarity ratings were also obtained from the receivers them- selves. Across several variations of the procedure, dreams were judged to be significantly more similar to the target pictures than to the control pictures in the judging sets (failures to repli- cate the Maimonides results were also reviewed by Child, 1985). These several lines of evidence suggested a working model of psi in which psi-mediated information is conceptualized as a weak signal that is normally masked by internal somatic and external sensory “noise.” By reducing ordinary sensory input, these diverse psi-conducive states are presumed to raise the sig- nal-to-noise ratio, thereby enhancing a person’s ability to detect the psi-mediated information (Honorton, 1969, 1977). To test the hypothesis that a reduction of sensory input itself facilitates psi performance, investigators turned to the ganzfeld procedure - (Braud, Wood, & Braud, 1975; Honorton & Harper, 1974; Par- ker, 1975), a procedure originally introduced into experimental psychology during the 1930s to test propositions derived from gestalt theory (Avant, 1965; Metzger, 1930). Like the dream studies, the psi ganzfeld procedure has most often been used to test for telepathic communication between a sender and a receiver. The receiver is placed in a reclining chair in an acoustically isolated room. Translucent ping-pong ball halves are taped over the eyes and headphones are placed over the ears; a red floodlight directed toward the eyes produces an undifferentiated visual field, and white noise played through the headphones produces an analogous auditory field. It is this ho- mogeneous perceptual environment that is called the Ganzfeld (“total field”). To reduce internal somatic “noise,” the receiver typically also undergoes a series of progressive relaxation exer- cises at the beginning of the ganzfeld period. The sender is sequestered in a separate acoustically isolated * room, and a visual stimulus (art print, photograph, or brief vid- eotaped sequence) is randomly selected from a large pool of such stimuli to serve as the target for the session. While the sender concentrates on the target, the receiver provides a con- tinuous verbal report of his or her ongoing imagery and menta- tion, usually for about 30 minutes. At the completion of the ganzfeld period, the receiver is presented with several stimuli (usually four) and, without knowing which stimulus was the target, is asked to raté the degree to which each matches the imagery and mentation experienced during the ganzfeld period. If the receiver assigns the highest rating to the target stimulus, it Approved For Release 2000/08/08 : CIA-RDP96-00789R003200110001-4
. Approved For Release 2000/08/08 : CIA-RDP96-00789R003200110001-4 6 ~ DARYL J. BEM AND CHARLES HONORTON is scored as a “‘hit.’ Thus, if the experiment uses judging sets containing four stimuli (the target and three decoys or control stimuli), the hit rate expected by chance is .25. The ratings can also be analyzed in other ways; for example, they can be con- verted to ranks or standardized scores within each set and ana- lyzed parametrically across sessions. And, as with the dream studies, the similarity ratings can also be made by outside judges using transcripts of the receiver’s mentation report. Meta-Analyses of the Ganzfeld Database In 1985 and 1986, the Journal of Parapsychology devoted two entire issues to a critical examination of the ganzfeld database. The 1985 issue comprised two contributions: (a) a meta-analy- sis and critique by Ray Hyman (1985), a cognitive psychologist and skeptical critic of parapsychological research, and (b) a competing meta-analysis and rejoinder by Charles Honorton (1985), a parapsychologist and major contributor to the ganz- feld database. The 1986 issue contained four commentaries on the Hyman—Honorton exchange, a joint communiqué by Hy- man and Honorton, and six additional commentaries on the joint communiqué itself. We summarize the major issues and conclusions here. Replication Rates Rates by study. Hyman’s meta-analysis covered 42 psi ganz- feld studies reported in 34 separate reports written or published from 1974 through 1981. One of the first problems he discov- ered in the database was multiple analysis. As noted earlier, it is possible to calculate several indexes of psi performance in a ganzfeld experiment and, furthermore, to subject those indexes to several kinds of statistical treatment. Many investigators re- ported multiple indexes or applied multiple statistical tests without adjusting the criterion significance level for the number of tests conducted. Worse, some may have “shopped” among the alternatives until finding one that yielded a significantly suc- cessful outcome. Honorton agreed that this was a problem. Accordingly, Honorton applied a uniform test on a common index across all studies from which the pertinent datum could be extracted, regardless of how the investigators had analyzed the data in the original reports. He selected the proportion of hits as the common index because it could be calculated for the largest subset of studies: 28 of the 42 studies. The hit rate is also a conservative index because it discards most of the rating information; a second place ranking—a “‘near miss”’—receives no more credit than a last place ranking. Honorton then calcu- lated the exact binomial probability and its associated z score for each study. Of the 28 studies, 23 (82%) had positive z scores (p = 4.6 X 107‘, exact binomial test with p = g = .5). Twelve of the studies (43%) had z scores that were independently significant at the 5% level (p = 3.5 X 107°, binomial test with 28 studies, p = .05, and g = .95), and 7 of the studies (25%) were independently significant at the 1% level (p = 9.8 X 107°). The composite Stouffer z score across the 28 studies was 6.60 (p = 2.1 x 107"). A more conservative estimate of significance can be obtained by including 10 additional studies that also used the relevant judging procedure but did not report hit rates. If these studies are assigned a mean z score of zero, the Stouffer z across all 38 studies becomes 5.67 (p = 7.3 X 107°). Thus, whether one considers only the studies for which the relevant information is available or includes a null estimate for the additional studies for which the information is not available, the aggregate results cannot reasonably be attributed to chance. And, by design, the cumulative outcome reported here cannot be attributed to the inflation of significance levels through multiple analysis. Rates by laboratory. One objection to estimates such as those just described is that studies from a common laboratory are not independent of one another (Parker, 1978). Thus, it is possible for one or two investigators to be disproportionately responsible for a high replication rate, whereas other, indepen- dent investigators are unable to obtain the effect. The ganzfeld database is vulnerable to this possibility. The 28 studies providing hit rate information were conducted by investigators in 10 different laboratories. One laboratory con- tributed 9 of the studies, Honorton’s own laboratory contrib- uted 5, 2 other laboratories contributed 3 each, 2 contributed 2 each, and the remaining 4 laboratories each contributed 1. Thus, half of the studies were conducted by only 2 laboratories, 1 of them Honorton’s own. Accordingly, Honorton calculated a separate Stouffer z score for each laboratory. Significantly positive outcomes were re- ported by 6 of the 10 laboratories, and the combined z score across laboratories was 6.16 (p = 3.6 X 107'°). Even if all of the studies conducted by the 2 most prolific laboratories are discarded from the analysis, the Stouffer z across the 8 other laboratories remains significant (z = 3.67, p = 1.2 X 1074), Four . of these studies are significant at the 1% level (p = 9.2 x 1076, binomial test with 14 studies, p = .01, and g = .99), and each was contributed by a different laboratory. Thus, even though the total number of laboratories in this database is small, most of them have reported significant studies, and the significance of the overall effect does not depend on just one or two of them. Selective Reporting In recent years, behavioral scientists have become increas- ingly aware of the “file-drawer” problem: the likelihood that successful studies are more likely to be published than unsuc- cessful studies, which are more likely to be consigned to the file drawers of their disappointed investigators (Bozarth & Roberts, 1972; Sterling, 1959). Parapsychologists were among the first to become sensitive to the problem, and, in 1975, the Parapsycho- logical Association Council adopted a policy opposing the selec- tive reporting of positive outcomes. As a consequence, negative findings have been routinely reported at the association’s meet- ings and in its affiliated publications for almost two decades. As has already been shown, more than half of the ganzfeld studies included in the meta-analysis yielded outcomes whose signifi- cance falls short of the conventional .05 level. A variant of the selective reporting problem arises from what ' Stouffer’s z is computed by dividing the sum of the z scores for the individual studies by the square root of the number of studies (Rosen- thal, 1978). ; Approved For Release 2000/08/08 : CIA-RDP96-00789R003200110001-4
yman (1985) has termed the “retrospective study.” An inves- sator conducts a small set of exploratory trials. If they yield Il results, they remain exploratory and never become part of B.- official record; if they yield positive results, they are defined . a study after the fact and are submitted for publication. In pport of this possibility, Hyman noted that there are more gnificant studies in the database with fewer than 20 trials than Lc would expect under the assumption that, all other things being equal, statistical power should increase with the square Byot of the sample size. Although Honorton questioned the as- Eimption that “all other things” are in fact equal across the hudies and disagreed with Hyman’s particular statistical analy- s, he agreed that there is an apparent clustering of significant hudics with fewer than 20 trials. (Of the complete ganzfeld da- Ease of 42 studies, 8 involved fewer than 20 trials, and 6 of Rose studies reported statistically significant results.) Because it is impossible, by definition, to know how many nknown studies—exploratory or otherwise—are languishing fy file drawers, the major tool for estimating the seriousness of elective reporting problems has become some variant of Ro- nthal’s file-drawer statistic, an estimate of how many unre- boricd studies with z scores of zero would be required to exactly bancel out the significance of the known database (Rosenthal, 579), For the 28 direct-hit ganzfeld studies alone, this estimate is 423 fugitive studies, a ratio of unreported-to-reported studies bf approximately 15:1. When it is recalled that a single ganzfeld ssion takes over an hour to conduct, it is not surprising that— Hespite his concern with the retrospective study problem—Hy- an concurred with Honorton and other participants in the mublished debate that selective reporting cannot plausibly ac- ount for the overall statistical significance of the psi ganzfeld atabase (Hyman & Honorton, 1986). a tethodological Flaws If the most frequent criticism of parapsychology is that it has ot produced a replicable psi effect, the second most frequent criticism is that many, if not most, psi experiments have inade- quate controls and procedural safeguards. A frequent charge is hat positive results emerge primarily from initial, poorly con- i olled studies and then vanish as better controls and safeguards re introduced. Fortunately, meta-analysis provides a vehicle for empirically evaluating the extent to which methodological flaws may have contributed to artifactual positive outcomes across a set of stud- kes. First, ratings are assigned to each study that index the degree to which particular methodological flaws are or are not present; these ratings are then correlated with the studies’ outcomes. Large positive correlations constitute evidence that the ob- j Served effect may be artifactual. { In psi research, the most fatal flaws are those that might per- muta subject to obtain the target information in normal sensory fashion, either inadvertently or through deliberate cheating. This is called the problem of sensory leakage. Another poten- ually serious flaw is inadequate randomization of target selec- Lon. _ Sensory leakage. Because the ganzfeld is itself a perceptual isNation procedure, it goes a long way toward eliminating po- tential sensory leakage during the ganzfeld portion of the ses- Approved For Release 2000/08/08 : CIA-RDP96-00789R003200110001-4 ANOMALOUS INFORMATION TRANSFER 7 sion. There are, however, potential channels of sensory leakage after the ganzfeld period. For example, ifthe experimenter who interacts with the receiver knows the identity of the target, he or she could bias the receiver’s similarity ratings in favor of correct identification. Only one study in the database contained this flaw, a study in which subjects actually performed slightly below chance expectation. Second, if the stimulus set given to the re- ceiver for judging contains the actual physical target handled by the sender during the sending period, there might be cues (e.g., fingerprints, smudges, or temperature differences) that could differentiate the target from the decoys. Moreover, the process of transferring the stimulus materials to the receiver’s room itself opens up other potential channels of sensory leakage. Although contemporary ganzfeld studies have eliminated both of these possibilities by using duplicate stimulus sets, some of the earlier studies did not. ; Independent analyses by Hyman and Honorton agreed that there was no correlation between inadequacies of security against sensory leakage and study outcome. Honorton further reported that if studies that failed to use duplicate stimulus sets were discarded from the analysis, the remaining studies are still highly significant (Stouffer z = 4.35, p = 6.8 X 107°). Randomization. In many psi experiments, the issue of target randomization is critical because systematic patterns in inadequately randomized target sequences might be detected by subjects during a session or might match subjects’ preexisting response biases. In a ganzfeld study, however, randomization is a much less critical issue because only one target is selected dur- ing the session and most subjects serve in only one session. The primary concern is simply that all the stimuli within each judg- ing set be sampled uniformly over the course of the study. Sim- ilar considerations govern the second randomization, which takes place after the ganzfeld period and determines the se- quence in which the target and decoys are presented to the re- ceiver (or external judge) for judging. Nevertheless, Hyman and Honorton disagreed over the find- ings here. Hyman claimed there was a correlation between flaws of randomization and study outcome; Honorton claimed there was not. The sources of this disagreement were in conflicting definitions of flaw categories, in the coding and assignment of flaw ratings to individual studies, and in the subsequent statisti- cal treatment of those ratings. Unfortunately, there have been no ratings of flaws by inde- pendent raters who were unaware of the studies’ outcomes (Morris, 1991). Nevertheless, none of the contributors to the subsequent debate concurred with Hyman’s conclusion, whereas four nonparapsychologists—two statisticians and two psychologists—explicitly concurred with Honorton’s conclu- sion (Harris & Rosenthal, 1988b; Saunders, 1985; Utts, 199 1a). For example, Harris and Rosenthal (one of the pioneers in the use of meta-analysis in psychology) used Hyman’s own flaw rat- ings and failed to find any significant relationships between flaws and study outcomes in each of two separate analyses: 2 A 1980 survey of parapsychologists uncovered only 19 completed but unreported ganzfeld studies. Seven of these had achieved signifi- cantly positive results, a proportion (.37) very similar to the proportion of independently significant studies in the meta-analysis (.43) (Black- more, 1980). Approved For Release 2000/08/08 : CIA-RDP96-00789R003200110001-4
Approved For Release 2000/08/08 : CIA-RDP96-00789R003200110001-4 8 DARYL J. BEM AND CHARLES HONORTON “Our analysis of the effects of flaws on study outcome lends no support to the hypothesis that Ganzfeld research results are a significant function of the set of flaw variables” (1988b, p. 3; for a more recent exchange regarding Hyman’s analysis, see Hy- man, 1991; Utts, 1991a, 1991b). Effect Size Some critics of parapsychology have argued that even if cur- rent laboratory-produced psi effects turn out to be replicable and nonartifactual, they are too small to be of theoretical inter- est or practical importance. We do not believe this to be the case for the psi ganzfeld effect. In psi ganzfeld studies, the hit rate itself provides a straight- forward descriptive measure of effect size, but this measure can- not be compared directly across studies because they do not all use a fourstimulus judging set and, hence, do not all have a chance baseline of .25. The next most obvious candidate, the difference in each study between the hit rate observed and the hit rate expected under the null hypothesis, is also intuitively descriptive but is not appropriate for statistical analysis because not all differences between proportions that are equal are equally detectable (e.g., the power to detect the difference be- tween .55 and .25 is different from the power to detect the difference between .50 and .20). To provide a scale of equal detectability, Cohen (1988) de- vised the effect size index 4, which involves an arcsine transfor- mation on the proportions before calculation of their difference, Cohen’s #1 is quite general and can assess the difference between any two proportions drawn from independent samples or be- tween a single proportion and any specified hypothetical value. For the 28 studies examined in the meta-analyses, A was .28, with a 95% confidence interval from .11 to .45. But because values of 4 do not provide an intuitively descrip- tive scale, Rosenthal and Rubin (1989; Rosenthal, 1991) have recently suggested a new index, 2, which applies specifically to one-sample, multiple-choice data of the kind obtained in ganz- feld experiments. In particular, x expresses all hit rates as the proportion of hits that would have been obtained if there had been only two equally likely alternatives—essentially a coin flip. Thus, m ranges from 0 to 1, with .5 expected under the null hypothesis. The formula is eee tol 0 Pk—2)+1’ where P is the raw proportion of hits and k is the number of alternative choices available. Because a has such a straightfor- ward intuitive interpretation, we use it (or its conversion back to an equivalent four-alternative hit rate) throughout this article whenever it is applicable. For the 28 studies examined in the meta-analyses, the mean value of x was .62, with a 95% confidence interval from .55 to .69. This corresponds to a four-alternative hit rate of 35%, with a 95% confidence interval from 28% to 43%. Cohen (1988, 1992) has also categorized effect sizes into small, medium, and large, with medium denoting an effect size that should be apparent to the naked eye of a careful observer. For a statistic such as +, which indexes the deviation of a pro- portion from .5, Cohen considers .65 to be a medium effect size: A statistically unaided observer should be able to detect the bias of a coin that comes up heads on 65% of the trials. Thus, at .62, the psi ganzfeld effect size falls just short of Cohen’s naked-eye criterion. From the phenomenology of the ganzfeld experi- _menter, the corresponding hit rate of 35% implies that he or she will see a subject obtain a hit approximately every third session rather than every fourth. It is also instructive to compare the psi ganzfeld effect with the results of a recent medical study that sought to determine whether aspirin can prevent heart attacks (Steering Committee of the Physicians’ Health Study Research Group, 1988). The study was discontinued after 6 years because it was already clear that the aspirin treatment was effective (p < .00001) and it was considered unethical to keep the control group on placebo med- ication. The study was widely publicized as a major medical breakthrough. But despite its undisputed reality and practical importance, the size of the aspirin effect is quite small: Taking aspirin reduces the probability of suffering a heart attack by only .008. The corresponding effect size (4) is .068, about one third to one fourth the size of the psi ganzfeld effect (Atkinson etal., 1993, p. 236; Utts, 199 1b). In sum, we believe that the psi ganzfeld effect is large enough to be of both theoretical interest and potential practical impor- tance, Experimental Correlates of the Psi Ganzfeld Effect We showed earlier that the technique of correlating variables with effect sizes across studies can help to assess whether meth- odological flaws might have produced artifactual positive out- comes. The same technique can be used more affirmatively to explore whether an effect varies systematically with conceptu- ally relevant variations in experimental procedure. The discov- ery of such correlates can help to establish an effect as genuine; suggest ways of increasing replication rates and effect sizes, and enhance the chances of moving beyond the simple demonstra- tion of an effect to its explanation. This strategy is only heuris- tic, however. Any correlates discovered must be considered quite tentative, both because they emerge from post hoc explo- ration and because they necessarily involve comparisons across heterogencous studies that differ simultaneously on many inter- related variables, known and unknown. Two such correlates emerged from the meta-analyses of the psi ganzfeld effect. Single-versus multiple-image targets, Although most of the 28 studies in the meta-analysis used single pictures as targets, 9 (conducted by three different investigators) used View Master stereoscopic slide reels that presented multiple images focused on a central theme. Studies using the View Master reels pro- duced significantly higher hit rates than did studies using the single-image targets ($0°% vs. 34%), (26) = 2,22, p = .035, two- tailed. Sender-receiver pairing In 17 of the 28 studies, partici- pants were free to bring in friends to serve as senders, In 8 stud- ies, only laboratory-assigned senders were used. (Three studies used no sender.) Unfortunately, there is no record of how many participants in the former studies actually brought in friends. Nevertheless, those 17 studies (conducted by six different inves- tigators) had significantly higher bit rates than did the studies Approved For Release 2000/08/08 : CIA-RDP96-00789R003200110001-4