Approved For Release 2bddN Side ASS koHeOb0789R002200390001-5 CONCLUSIONS AND SUGGESTIONS FOR EXTENSIONS We are proposing a target-pool independent method (figure of merit analysis) for scoring free-response material. The FMA provides a number of advantages over previous methods. @ Figures of merit can be used in correlation studies. @ FMA provides a novel technique for free-response controls. © Target pool independent exact p-values can be computed for each free-response session. @ Since the FM is computed by simple counting, the computer coding burden is sharply reduced. Because of the lack of descriptor bit independence (and thus a need for logically consistent rules) the effective number of descriptor bits is reduced. We are presently investigating a way to utilize a hierarchical descriptor list: that is, each level of the hierarchy consists of a variable number of independent descriptors. Finally, the ideal descriptor list would include arbitrary weighting factors for the level of hierarchy as well as for the individual descriptors within the level. REFERENCES Dunne, B.J., Jahn, R.G., and Nelson, R.D., “Precognitive Remote Perception,” Engineering Anomalies Research Laboratory, School of Engineering/Applied Science, Princeton University, Princeton NJ, Technical Note PEAR 83003 (August 1983) Hubbard, G.S., May, E.C., and Puthoff, H.E., “Possible Production of Photons During a Remote Viewing Task: Preliminary Results,” Proceedings of the 28th Convention of the Parapsychological Association, (Radin, D.I, ed.) Tufts University, Medford, MA, (August 1985) Jahn, R.G., Dunne, B.J., and Jahn, E.G., “Analytical Judging Procedure for Remote Perception Experiments,” The Journal of Parapsychology, Vol. 44, No. 3, pp. 207-231 (September 1980). Utts, J.M., and May, E.C., “An Exact Method for Combining P-values,” Proceedings of the 28th Convention of the Parapsychological Association, (Radin, D.I, ed.) Tufts University, Medford, MA, (August 1985) A-10 UNCLASSIFIED Approved For Release 2000/08/08 : CIA-RDP96-00789R002200390001-5
Approved For Release 2OPAPE'ESA S'S FAB 00789R002200390001-5 APPENDIX B PSI EXPERIMENTS WITH RANDOM NUMBER GENERATORS; META-ANALYSIS PART 1 (This Appendix Is Unclassified) Approved For Release MING LASS LELED, .70R002200390001-5
Approved For Release 206d @dsAG4 REHS &00789R002200390001-5 Psi Experiments with Random Number Generators: Meta—Analysis Part 1 Dean I. Radin Edwin C. May Martha J. Thomson SRI International Menlo Park, California ABSTRACT: A meta-analysis of 332 psi experiments involving binary random number generators is described. The combined binomial probability for data reported in 56 references published from 1969-1984 is p & 10-43, A “filedrawer” analysis reveals _ that over 4500 additional, nonsignificant, unpublished or unretrieved studies would be required to bring the overall result down to a nonsignificant level. Using a novel approach, we estimate the actual size of the “filedrawer” to be 95 studies. Adding the equivalent of 95 nonsignificant studies to the existing data results in p © 10-'8, while a meta-analysis of 98 reported control studies results in p © .78. An analysis of variance indicates that experimenters’ mean z scores are significantly different from each other. We discuss an approach and propose criteria for performing a quality-weighted analysis on the existing data. We conclude that the prima facie evidence supports the notion that observers’ intentions can affect the statistical properties of truly random number generators. INTRODUCTION This is Part 1 of a two part meta-analysis of psi experiments involving truly random number generators (RNG) published from 1969-1984. This part describes the results of a “first~pass” analysis, in which the published data was taken at face value. Part 2 will report on a quality-weighted analysis in which the results of each experiment (in terms of z score) will be evaluated on each of a dozen criteria to produce an adjusted z score reflecting that experiment’s overall quality. . Background: On the scent of a trail When Albert Einstein was asked about his way of thinking, he reportedly replied, “All I have is the stubbornness of a mule; no, that’s not quite all, I also have a nose” (Bower, 1985, p.330). What he meant was that he was not only extraordinarily obstinate in tracking down solutions to problems, he was also able to sniff out when he was on the right track. The centennial anniversary B-1 UNCLASSIFIED Approved For Release 2000/08/08 : CIA-RDP96-00789R002200390001-5
Approved Eo? Rélease 20QDNBOB AcHSH BEd) 0789R002200390001-5 of the American Society for Psychical Research, celebrated this year (1985), clearly demonstrates that parapsychologists have displayed Einstein’s stubbornness over the years. One question we might ask after 100 years, however, is whether the parapsychological nose has been sniffing along a clearly defined trail, and if so, is the trail likely to grow more fragrant or more noxious as we progress? There is evidence that the nose has not been shirking its duty. This can be seen in the Single most predictable feature found in the parapsychological literature, that is, the perennial call for a replicable experiment. The ideal experiment is supposed to produce a significant result regardless of the phase of the moon, the price of pork bellies, and the experimenter’s shoe size. This Quest for replicable experiments is by no means unique to parapsychology, however. Social and behavioral scientists in general have been acutely aware of the slow progress in the “softer” sciences as compared to the natural sciences such as physics, chemistry, and biology. In experimental psychology, for example, Epstein (1980) has stated, Psychological research is rapidly approaching a crisis as the result of extremely inefficient procedures for establishing replicable generalizations. The traditional solution of attempting to obtain a high degree of control in the laboratory is often ineffective because much human behavior is so sensitive to incidental sources of stimulation that adequate control cannot be achieved.... Not only are experimental findings often difficult to replicate when there are the slightest alterations in conditions, but even attempts at exact replication frequently fail. (p. 790) Many observers of parapsychology (both within and outside the field) claim that the repeatable parapsychological experiment does not exist. For example, Beloff (1977) has written, “There is still no repeatable [psi] experiment on the basis of which any competent investigator can verify a given phenomenon for himself” (p.759). Critics of the field have pointed to the lack of replicability as perhaps the single most serious problem in parapsychology (e.g. Kurtz, 1981, p.12). In response, proponents often point to significant psi studies involving ESP card-guessing (Honorton, 1975), ganzfeld stimulation (Honorton, 1978), remote perception (Dunne, Jahn, and Nelson, 1983), and RNGs (May, Hubbard and Humphrey, 1980) to indicate that there are some significant replications. The problem is that from different perspectives the proponents and critics are both right. There are indeed many psi experiments that have been repeated, but whether they are considered robust, successful replications is the crux of the debate. One of the primary reasons for this debate, in our opinion, is because the traditional approach of assessing the results of a set of related studies is by descriptive literature review. Within parapsychology there are many excellent examples of such reviews (e.g. Carpenter, 1977; Palmer, 1982; Rush, 1982; Schmeidler, 1984; Stanford, 1977; Stanford, 1984). Unfortunately, what one has typically learned after studying such a review is a hodge-podge of variables, conditions, and p-values. Rarely is one left with a quantitative statement of the degree of significance obtained in the studies as a whole. Addressing this issue empirically, Cooper and Rosenthal (1980) demonstrated that when knowledgeable individuals are instructed to make judgments about the overall significance of a set of studies based on their readine of a comprehensive, descriptive literature review, it is possible for B-2 UNCLASSIFIED Approved For Release 2000/08/08 : CIA-RDP96-00789R002200390001-5