Approved For Release JOMNGhA SS IFAELD-00789Rob TOGA OTS * them to draw conclusions that are completely the opposite of the results obtained when the same studies are summarized by more explicit, quantitative methods. Given the difficulties in assessing evidence from existing psi studies, is the replication trail likely to be heading — to reinvoke our metaphor —- towards a flowering meadow or something decidedly less pleasant? In general, we believe that the prospects are aromatic. In the last few years, quantitative techniques of combining and comparing research results in systematic ways have been developed — called meta-analysis (Rosenthal, 1984) —- that show great promise in demonstrating that some areas of social science have been progressing much better than previously thought. In parapsychology, initial meta-analyses applied to ganzfeld research (Honorton, 1985), hypnotic induction (Schechter, 1984), RNG studies (May, Hubbard and Humphrey, 1980; Nelson, Dunne _ and Jahn, 1984; Tart, 1983), and remote viewing (Dunne, Jahn, and Nelson, 1983),have shown that the overall evidence for these psi phenomena is actually quite strong. Because meta-analysis involves the agerepation of results of numerous studies, several criticisms of this technique have been raised (Rosenthal, 1984, p.124-132). Perhaps the three categories of criticism most pertinant to review of parapsychological data are the following: First, authors may tend to report only the studies with significant results and leave the nonsignificant studies unpublished (called the filedrawer problem). Second, the meta-analysis combines poorer quality studies with better studies. And third, meta-analysis may be comparing “apples and oranges” by combining different experiments studying different variables. The first two problems may inflate the estimate of an overall effect; the third criticism may make the overall summary difficult or impossible to interpret. In the present meta-analysis, however, we actually are interested in whether these psi experiments have borne fruit, not whether they have borne specific flavors of apples or oranges. In other words, we are not concerned with whether hypnotic induction, say, has an effect on RNG outputs, but whether there is evidence for any psi effect on RNG outputs. Thus, in this investigation we have concentrated on the filedrawer issue (in this report) and the quality of studies (to be described in Part 2 of this study). OVERVIEW OF A TYPICAL RNG EXPERIMENT The typical psi experiment with RNGs involves three main components: An observer (e.g. a human, goldfish, cat or dog), a truly random number generator based on radioactive decay or electronic noise, and an experimental task linking the observer with the device, such as a video game, a set of instructions, a need to keep a heat lamp on or avoid a shock, and so on. The aim of these experiments is to show that the instructions (when humans are involved) or the induced need (when animals or plants are involved) are associated in some way — but not necessarily causally -- to the statistical output of the RNG. For example, say an RNG was designed to produce 100 random bits at the press of a button. An individual in this experiment might see a digital display of the number of 1’s (called hits) produced immediately after he or she pressed a button. The instructions in the experiment would typically be . to get as many hits as possible for each button press. The results of many presses, or trials, would UNCLASSIFIED Approved For Release 2000/08/08 : CIA-RDP96-00789R002200400001-3