Approved For Release 2000/08/10 : CIA-RDP96-00792R000100130003-0 Psi Phenomena 6 Methodological inadequacies plague all the sciences, but the history of parapsychology is embarrassingly full of promising results that collapsed when the procedures were critically examined (Akers, 1984). One common charge against parapsychology is that preliminary, poorly controlled studies often obtain positive results, but that as soon as better controls and safeguards are introduced, the results weaken or disappear. Meta-analysis provides an empirical way of evaluating this charge, too. Once a flaw is discovered in a completed experiment, there is no persuasive way of arguing that the flaw did not contribute illegitimately to a positive outcome; the only remedy is to redo the experiment correctly. In a database of several studies, however, it is possible to test whether the more poorly controlled studies in the database did in fact obtain more positive results than the well controlled studies. If there is a correlation between a procedural flaw and positive results across the studies, then there is a problem. In the case of the ganzfeld database, both critic Hyman and parapsychologist Honorton agree that flaws of inadequate security and possible sensory leakage do not correlate with positive results. Hyman claimed to find a correlation between flaws of randomization and positive results, but both Honorton’s analysis and two additional analyses by nonparapsychologists dispute his conclusion (Harris & Rosenthal, 1988; Saunders, 1985). Moreover, a series of 10 new studies designed to control for flaws identified in the original database yielded results quite consistent with the original set of 28 studies (Harris & Rosenthal, 1988). File-Drawer Problem Suppose that each of 20 investigators independently decides to conduct a ganzfeld study. Even if there were no ganzfeld effect, there is a reasonable probability that at least one of these investigators would obtain a significant result by pure chance. That lucky investigator would then publish a report of the experiment, but the other 19 investigators—who obtained “null” results—are likely to become discouraged, put their data into a file drawer, and move onto something more promising. As a result, the scientific community would learn about the one successful study but have no knowledge of the 19 null studies buried in the file drawers. The database of known studies would thus be seriously Approved For Release 2000/08/10 : CIA-RDP96-00792R000100130003-0
Aparoved For Release 2000/08/10 : CIA-RDP96-00792R000100130003-0 mena 7 biased toward positive studies, and any meta-analysis of that database would arrive at similarly biased conclusions. This is called the file-drawer problem. The problem is particularly tricky because it is impossible, by definition, to know how _ many unknown studies are languishing in file drawers somewhere. Nevertheless, parapsychologists offer two lines of defense against the charge that the file-drawer problem seriously compromises their database. First they point out that the Parapsychological Association may be the only professional scientific society to have an official policy against selective reporting of positive findings and that negative findings are frequently reported in the Association’s five affiliated journals. Moreover, the community of parapsychologists is relatively small, and most investigators know about ongoing work in the active laboratories around the world. When conducting meta- analyses, they attempt to ferret out all unpublished studies through their personal networks. But the major defense is statistical, and again meta-analysis provides an empirical approach to the problem. By knowing the overall statistical significance of the known database, it is possible to compute the number of studies with null results that would have to exist in file drawers to cancel out that significance. In the case of the ganzfeld database, it is estimated that there would have to be over 400 unreported ganzfeld studies with null results—the equivalent of 12,000 sessions—to cancel out the statistical significance of the 28 studies analyzed in the debate (Honorton, 1985). Not surprisingly, there is consensus that the overall significance of the ganzfeld studies cannot reasonably be explained by the file-drawer effect (Hyman & Honorton, 1986). Rather than continuing their debate, Hyman and Honorton issued a joint communiqué in which they set forth their areas of agreement and disagreement and made a series of suggestions for the conduct of future ganzfeld studies (Hyman & Honorton, 1986). Their debate and the subsequent discussion provide a valuable model for evaluating all disputed domains of scientific inquiry. Approved For Release 2000/08/10 : CIA-RDP96-00792R000100130003-0
Approved For Release 2000/08/10 : CIA-RDP96-00792R000100130003-0 Psi Phenomena 8 Anecdotal Evidence In the public mind, the evidence for psi consists primarily of personal experiences and anecdotes. Virtually all such evidence is scientifically unacceptable because it suffers fatally from the same problems that threaten experimental evidence—nonreplicability, inadequate controls, and the file-drawer problem. The replication problem is insurmountable because most such evidence consists of one- time occurrences that cannot be repeated. A woman announces a premonition that she will win the lottery that day—and she does. You dream about an unlikely event that actually occurs a few days later. A “psychic” correctly predicts the assassination of a public figure. And because such incidents occur under unexpected and ambiguously specified conditions, the problem of inadequate controls is also decisive. There is usually no way of ruling out such alternative interpretations as coincidence (chance), faulty memories, and deliberate deception. The file-drawer problem is also pervasive. The lottery winner who announced ahead of time that she would win is prominently featured in the news. But the thousands of others with similar premonitions who did not win are never heard from; they remain in the file drawers. It is true that the probability of this woman’s winning the lottery was very low. But the critical criterion in evaluating this case is not the probability that she would win but the probability that any one of the thousands who thought they would win would do so. That probability is much higher. Moreover, this woman has her own personal file drawer which contains all those past instances in which she had similar premonitions and then lost. The same reasoning applies to precognitive dreams. We tend to forget our dreams unless and until an event happens to remind us of them. We thus have no way of evaluating how often we might have dreamt of similar unlikely events that did not occur. We fill our database with positive instances and unknowingly exclude the negative instances. The fullest file drawers probably belong to the so-called “psychics” who make annual predictions in the tabloid newspapers. Everybody remembers their occasional “direct hits,” but ‘Approved For Release 2000/08/10 : CIA-RDP96-00792R000100130003-0
Approved For Release 2000/08/10 : CIA-RDP96-00792R000100130003-0 Psi Phenomena iy 9 nobody remembers the predictions that fail. In fact, these psychics are almost always wrong (Frazier, 1987; Tyler, 1977). Skepticism About Psi If some of the experimental evidence for psi is as impressive as it seems, why hasn’t it become part of established science? Why the continuing skepticism? Extraordinary Claims Most scientists believe that extraordinary claims require extraordinary proof. A study reporting that students who work harder get higher grades will be believed even if the study was seriously flawed because the data are consistent with our understanding of how the world works. But the claim that two people in a ganzfeld study communicate telepathically is more extraordinary; it violates our a priori beliefs about reality. We thus rightly demand a higher measure of proof from parapsychologists because their claims, if true,.would require us to radically revise our model of the world—something we should not undertake lightly. In this way, science is justifiably conservative. Many open-minded nonparapsychologists are genuinely impressed by the ganzfeld studies, for example, but they reasonably can and do ask to see more evidence before committing themselves to the reality of psi. Extraordinariness is a matter of degree. Telepathy seems less extraordinary to most of us than precognition because we are already familiar with the invisible transmission of information through space. We may not all understand how television pictures get to our living rooms, but we know that they do so. Why should telepathy seem that much more mysterious? Precognition on the other hand seems more extraordinary because we have no familiar phenomena in which information flows backward in time. Extraordinariness also depends upon our current model of reality. As our understanding of the world changes, a phenomenon that seemed extraordinary at an earlier time may no longer seem so—even if the quality of the evidence has not changed. Any child who has visited a museum of natural history has seen fragments of a meteorite. But before the _ Approved For Release 2000/08/10 : CIA-RDP96-00792R000100130003-0
PANEAvedneOr Release 2000/08/10 : CIA-RDP96-00792R000100130003-01 asymmetries in the human brain (see Chapter 2) have spawned a host of pop-psychology books and media reports containing unsubstantiated claims about left-brained and right-brained persons. Irresponsible reports about states of consciousness—including hypnosis and psi— appear daily in the media. It is thus pertinent to note that when the college professors in the survey cited above were asked to name the sources for their beliefs about ESP, they most frequently cited reports in newspapers and magazines. And finally, research in cognitive and social psychology has sensitized psychologists to the biases and shortcomings in our abilities to draw valid inferences from our everyday experiences (see Chapter 19). This makes them particularly skeptical of anecdotal reports of psi where, as we saw above, our judgments are subject to many kinds of errors. For these several reasons, then, much of the skepticism of psychologists toward psi is well-founded. But some of it is not. As we noted earlier, some scientists declare psi to be an impossibility and reject the legitimacy of parapsychological inquiry—a priori judgments we believe to be out of place in science. Only 4% of the college professors in the survey declared psi to be an impossibility—but 34% of the psychologists did so. Two hundred years ago, these same skeptics would have been equally certain that God does not hurl stones at us from heaven. Rosenthal, R. (1986). Meta-analytic procedures and the nature of replication: the ganzfeld debate. Journal of Parapsychology, 50, 315-337. Saunders, D. R. (1985). On Hyman’s factor analyses. Journal of Parapsychology, 49, 86-88. Spence, K. W. (1964). Anxiety (Drive) level and performance in eyelid conditioning. Psychological Bulletin, 61, 129-139. Approved For Release 2000/08/10 : CIA-RDP96-00792R000100130003-0