Approved For Release 2000/08/08 : CIA-RDP96-00789R002200410001-2 Meta-analysis of Forced-Choice Precognition Experiments 8 successful parapsychological outcomes are achieved by only a few investigators. The Filedrawer Problem A well-known reporting bias exists throughout the behavioral sciences favoring publication of “significant” studies (e.g., Sterling, 1959). The extreme view of this “filedrawer problem,” as Robert Rosenthal describes it, “is that the journals are filled with the 5% of the studies that show type I errors, while the filedrawers back at the lab are filled with the 95% of the Studies that show nonsignificance. . .” (Rosenthal, 1984, p. 108). Recognizing the importance of this problem, the Parapsychological Association in 1975 adopted an official policy against selective reporting of positive results. Examination of the parapsychological literature shows that nonsignificant results are frequently published and in the precognition database, 60% to 70% of the studies have reported nonsignificant results. Nevertheless, 75% of the precognition studies were published before 1975, and we must ask to what extent selective publication bias could account for the cumulative effects we observe.” The central section of Table 1 uses Rosenthal’s (1984) filedrawer statistic to estimate the number of unreported studies with z-scores averaging zero that would be necessary to reduce the known database to nonsignificance. The filedrawer estimate suggests that over 50 unreported studies must exist for each reported study to reduce the cumulative hitting (directional) outcomes to a nonsignificant level. For the predicted outcomes, the filedrawer ratio is more than 200:1. Another approach to the filedrawer problem is described by Robyn Dawes (Dawes, Landman and Williams, 1984; personal communication to Honorton, July 14, 1988). Dawes calculates the expected mean z and variance for various significance levels on the assumption that reported significant outcomes reflect nothing more than type I error. He then tests 2Analyses indicate no significant differences in the magnitude of reported study outcomes before and after 1975, The mean directional z-score for studies prior to 1975 is 0.719 (sd = 2.6) and for studies reported thereafter the mean is 0.605 (sd = 2.81) (¢ = 0.325, 307 df, p = .746, 2-tailed). For predicted z-scores, the comparable values are 1.43 (sd = 2.29) and 1.22 (sd = 2.60); t = 0.675, 307 df, p = .675, 2-tailed). ee Psychophysical Research Laboratories Approved For Release 2000/08/08 : CIA-RDP96-00789R002200410001-2
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200410001-2 Meta-analysis of Forced-Cholce Precognition Experiments 9 the difference between the observed and expected values. Applying this method to the precognition domain, it is extremely unlikely that the reported significance levels are just type I error. For the 5% significance level, for example, the mean observed and expected directional z-scores are 3.59 and 2.06, respectively. The observed mean is significantly larger than the expected value (z = 4.10, p = .000021). For the 0.5% significance level, the observed and expected means are 4.97 and 2.87 (z = 7.0,p = 2.7x 10°), Based on these analyses, we conclude that the cumulative significance of the precognition studies cannot plausibly be attributed to selective reporting. Psychophysical Research Laboratories Approved For Release 2000/08/08 : CIA-RDP96-00789R002200410001-2