CIA-RDP96-00789R002200610001-0 Approved For Release 2000/08/08 ournal of Parapsychology, Vol. 53, December 1989 psychology “FUTURE TELLING”: A META-ANALYSIS OF FORCED-CHOICE PRECOGNITION EXPERIMENTS, 1935-1987 By CHARLES LLONORTON AND DIANE CC.’ FERRARI ABSTRACT: We report a meta-analysis of forced-choice precognition experiments published in the English-language parapsychological literature between 1935 and 1987. These studies involve attempts by subjects to predict the identity of target stimuli selected randomly over intervals ranging from several hundred milli- seconds to one year folowing the subjects’ responses. We retrieved 309 studies reported by 62 investigators. Nearly two million individual trials were contributed by more than 50,000 subjects. Study outcomes are assessed by overall level of sta- tistical significance and effect size. There is a small, but reliable overall effect (z = 11.41, p = 6.3 x 10°”). Thirty percent of the studies (by 40 investigators) are significant at the 5% significance level. Assessment of vulnerability to selective re- porting indicates that a ratio of 46 unreported studics averaging null results would be required for each reported study in order to reduce the overall result to nonsig- nificance. No systematic relationship was found between study outcomes and eight indices of research quality. Effect size has remained essentially constant over the survey period, whereas research quality has improved substantially. Four moder- ating variables appear to covary significantly with study outcome: Studies using subjects selected on the basis of prior testing performance show significantly larger effects than studies using unselected subjects. Subjects tested individually by an experimenter show significantly larger effects than those tested in groups. Studies in which subjects are given trial-by-trial or run-score feedback have significantly larger effects than those with delayed or no subject feedback. Studies with bricf intervals between subjects’ responses and target generation show significantly stronger effects than studies involving longer intervals. The combined impact of these moderating variables appears to be very strong. Independently significant outcomes are observed in seven of the eight studies using selected subjects, who were tested individually and received trial-by-trial feedback. Precognition refers to the noninferential prediction of future events. Anecdotal claims of “future telling” have occurred through- out human history in virtually every culture and period. Today such This work was funded by SRI International and the John E. Fetzer Foundation. We wish to thank our PRL colleague George P. Hansen, who is primarily responsible for retrieving the studies used in the meta-analysis. We are grateful to Edwin C. May, Jessica Utts, and to five anonymous review SRI for valuable comments on an earlier draft of this report. Valuable comments were also made by Ephraim Schechter and by three anonymous referees. The division of authorship responsibility is as fol- lows: Honorton is responsible for the design of the meta-analysis, definition of study coding criteria, the actual analyses, and the report itself, Ferrari coded the individual research reports in consultation with Honorton and/or Hansen. CIA-RDP96-00789R002200610001-0 Approved For Release 2000/08/08
- 282 The Journal of Parapsychology claims are generally believed to be based on factors such as dehasion, i@ptionality, and superstitious thinking. ‘Vhe concept of precogni- tidn runs counter to accepted nouons of causality and appears to cGhflict with current scientific theory. Nevertheless, over the past ta@f-century a substantial number of experiments have been re- ered claiming empirical support for the hypothesis of precogni- (@n. Subjects in forced-choice experiments, according to many re- eects, have correctly predicted to a statistically significant degree the i@tntity (or order) of target stimuli randomly selected at a later iQre. ve performed a meta-analysis of forced-choice precognition ex- Criments published in the English-language research literature be- Feen 1935 and 1987. Four major questions were addressed (rough this meta-analysis: (1) Is there overall evidence for accurate Peet identification (above-chance hitting) in experimental precog- @Lion studies? (2) What is the magnitude of the overall precognition ect? (3) Is the observed effect related to variations in methodo- Bical quality that could allow a more conventional explanation? (4) nes precognition performance vary systematically with potential Moderating variables, such as differences ino sabject populidons, stimulus conditions, experimental setting, knowledge of results, and ime interval between subject response and target generation? 8 DELINEATING THE DOMAIN 00/08/ Roveval of Studies e ® Parapsychological research is still academically taboo, and it is Walikely that there have been many dissertations and theses in this Mea that have escaped publication. Our retrieval of studies for this eta-analysis is therefore based on the published literature. The Mucdies include all forced-choice precognition experiments appear- rin the peer-reviewed English-language parapsychology journals: rnal of Parapsychology, Journal (and Proceedings) of the Society for Cone Research, Journal of the American Society for Psychical Research, ropean Journal of Parapsychology (including the Research Letter of Re Utrecht University Parapsychology Laboratory), and abstracts of ¢@er-reviewed papers presented at Parapsychological Association meetings published in Research in Parapsychology. a Criteria for Inclusion Our review is restricted to fixed-length studies in which signifi- cance levels and effect sizes based on direct hitting can be calcu- A Meta-Analysis of Forced-Choice Precognition Experiments 283 lated. Studies using outcome variables other than direct hitting, such as run-score variance and displacement effects, are included only if the report provides relevant information on direct hits (i.e., numf?r of trials, hits, and probability of a hit). Finally, we exclude stugags conducted by two investigators, $. G. Soal and Walter J. Levy, whe work has been unreliable. = Many published reports contain more than one experiment r experimental unit. In experiments, involving multiple conditiges, significance levels and effect sizes are calculated for each condition. Oo Outcome Measures o co Significance level. Significance levels (z scores) were calculated fpr each study from the reported number of trials, hits, and probabaty of success using the normal approximation to the binomial diggi- bution with continuity correction. Positive z scores indicate abqye- chance scoring, and negative z scores reflect below-chance scori Effect size. Because most parapsychological experiments, palitc- ularly those in the older literature, have used the trial rather thon the subject as the sampling unit, we use a Urial-based estimatoWbt effect size. ‘The effect size (ES) for each study is the z score divided by the square root of the number of trials in the study.’ 0/08/0 General Characteristics of the Domain oO We located 309 studies in 113 separate publications. These sel- ies were contributed by 62 different senior authors and were p@yb- lished over a 53-year period, between 1935 and 1987. Considerfig the half-century time-span over which the precognition experim@&ts were conducted, it is not surprising that the studies are very diveMe. The database comprises nearly two million individual trials ghd more than 50,000 subjects. Study sample sizes range “from: 2%Oto 297,060 trials (median = 1,194). The number of subjects rates from | to 29,706 (median = 16). The studies use a variety of m@h- odologies, ranging from guessing ESP cards and other card symigpis to automated random number generator experiments. The donfgin encompasses diverse subject populations: the most frequently uged . < ' Elsewhere (Honorion, 1985), we have used the effect size index Cohen’s h (Cohen, 1977), and one referee has asked that we explain why we are now using V2 + uN. The answer is that h and #N'™ yield virtually identical results, and z/N™ is. computationally simpler. For the present sample of 309 precognition studies, the mean difference between the two indices is .00047, and the standard deviation of the difference is .026: 4308) = 0.312, p = .756, two-tailed. ‘Vhe correlation between the two indices is .97.