Demonstration Research and Meta-Analysis in Parapsychology

This is the second of several reports to be made available by the Parapsychological Association (PA) for people outside of the association who are interested in its activities. The previous report (Parapsychological Association, 1989) described four research approaches in parapsychology: survey research, field research, demonstration research, and process research. This report will focus on demonstration research, which involves the comparison of observed events with what would ordinarily be expected in a single carefully defined situation. In this type of research, statistically unlikely outcomes in parapsychological experiments indicate that events have occurred that do not appear to be explicable by known mechanisms and are unlikely to be coincidence, but the research does not give information about how or why the events occurred. Demonstration research is research aimed at obtaining incontrovertible evidence for the existence of a phenomenon. It may mean conducting a crucial experiment that has appropriate controls against all possible sources of error. In such complex subjects as parapsychology, it is hardly feasible to prespecify all sources of possible error and hence to control against them; it is always possible to invent--after the fact--a scenario of possible error. In areas where the hypothesis of experimenter fraud is entertained, there can be no crucial experiment inasmuch as the fraud hypothesis, in the final analysis, is itself unfalsifiable. Therefore, parapsychologists generally tend to favor replication as a means of demonstrating psi instead of conducting the so-called crucial experiment. The Issue of Repeatability The literature on demonstration research in parapsychology is embedded in the assumption that the field needs to develop a repeatable experiment in order to establish itself as a science. PA members have diverse views on this topic; some agree with critics of the field that strict repeatability deserves a high priority in the field. Most, however, argue that strict replication is not essential; instead, they require the kind of statistical replicability that is common in most of the social and behavioral sciences (see Shapin & Coly, 1985). There are also those who take the extreme position that nonreplicability is inherent in the very nature of psi and how it functions in nature (e.g., Eisenbud, 1983). It is generally agreed that the attention given to this issue should not eclipse other important evaluation criteria such as the control of sensory channels and the elimination of alternative (i.e., non-psi) explanations in appraising data. Parapsychology, perhaps because many of its leaders in the past three decades were trained as psychologists, has evidenced a strong tendency to model itself after experimental psychology. In this paradigm, the experimental approach is the method of choice and repeatability of experimental results is considered to be a primary aim. Critics of parapsychology tend to concur on the value of this approach, but they have not made it clear just what would constitute acceptable evidence of a repeatable experiment. In the early days of psi research, some of these critics insisted that a single "fraud-proof" experiment would convince them of the reality of ESP. However, as soon as well-designed experiments showing statistical significance emerged, critics realized that a single experiment could be significant simply by chance. Critics then emphasized the need for repeatability, and one critic suggested that a significant experiment be replicated at least two times to demonstrate that the result was not due to chance (Hansel, 1980, p. 298). Several experimental approaches, usually those comparing two attitudinal types (e.g., "believers" in psi and "non-believers," introverts and extraverts) have yielded results that achieved a striking level of replication (Utts, 1991). Parapsychologists, again following psychology, were able to detect another problem with this line of reasoning. …