Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8 15 It is well established that some individuals achieve greater effects than others in psi experiments. Figure 1 shows the distribution of results across 33 operators (subjects) in a series of REG runs.29 Given this correlation of individual operator identities and characteristics with some of the alleged psi phenomena, it is also important to understand how individual operator effects are related to collective effects. The point was made that, in seeking larger data bases and greater (collective) significance levels, researchers should not lose sight of individual effects by combining many operators. Scientifically, the individual and group effects pose two distinct sets of questions. Effect size is another crucial issue in parapsychology. From the standpoint of practical significance, the effect appears to be quite small. In the largest REG data base of over 750,000 trials (150 million bits), compiled at the Princeton Engineering Anomalies Research Laboratory over the past decade, the overall means were shifted in the intended direction by only 0.035%. Figure 2 plots graphically the total cumulative deviations from baseline of all 33 operators in the program (accounting for 250,000 trials in each direction) .21 It can be seen that, while the deviation is well beyond the .05 significance level, it is not an enormous effect -- although it is said to be beyond the tolerance level of many contemporary microelectronics processors, communications discriminators, and dedicated computational arrays. Nevertheless, parapsychologists point out that psi effects, if indeed that is what they are, could be amplified by information theory techniques such as "majority vote counting." It is said that the effect size in the best ganzfeld psi research is approximately equal to that found in the recent studies of aspirin in heart attack prevention. And small effect sizes in physical phenomena (e.g., the bending of light by gravity) have been very important in the evolution of human knowledge in the past. Indeed, the evolution of life itself depends on genetic mutations which occur at very low rates. In other words, significance (in the "real world" sense) and effect size are two separate considerations. It is not the magnitude of an event, but its centrality and relevance to established beliefs or processes, that determines its importance. Effect size means different things in the context of different types of experiments. The average effect sizes found in psi ganzfeld experiments (about -17 in one series of experiments described at the workshop) are several times larger than those found in the REG work. The anomalies being displayed (if indeed they are true anomalies) may well be of fundamentally different types. The primary value of using effect sizes appears to be in compensating for differences in sample size. The key to satisfying proponents and critics of parapsychology may, in the end, be replicability. If a "recipe" for replication of successful parapsychology experiments could be developed, it would render moot many of the issues of statistical analysis. However, failure to replicate does not disprove the existence of psi; and the criteria for replicability can be difficult to 20 Dunne, B.J,, et al. Individual operator contributions in large data base anomalies experiments (Technical Note PEAR 88002). Research Report, Princeton Engineering Anomalies Research, July 1988. 21 Jahn, R.G. & B.J. Dunne, op. cit., p. 110. Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8 | (a) HIGH INTENTION + tay 10 41 62 +eettr + + 9953.53 42 42 + 5g 16 96 68 (ec) LOW INTENTION ) 5 10 %$ 20 2 x” xs WUMBER OF OPERATORS FIGURE 1: 200-SampLe REG, ASCENDING. OPERATOR MEANS Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8 UMOMAMHAH Hw BAe SH eat (28) SHIWAS TIW “(EE) SHOLWYSdO TW “SLINSIY OFN AATLVIAWAD #2 JUNOT STVIdL dO UYaadNNNn o000S2 000002 oooost OQOOOOT 0000S OOOOT (SLT@) AR PHM HHO se OPEZEMAM HH Pw Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8 establish. What is needed is consensus among the various camps on the ground rules for replicability. For example, individuals must rigorously apply the same criteria to themselves that they do to others, and apply to others the same criteria that they themselves use elsewhere. One approach that might move parapsychology out of the quagmire of statistical significance levels could be to control for desired conditions -- i.e., for large effect size -- stripping away in advance the "noise" injected by those subjects with low capability. It may be that replicability will hinge on getting large effect sizes. But even replicability in operators with large effect sizes is often problematical across time and changing experimental conditions. In the end, some parapsychologists believe, it may be that replicability will follow from the development of theory in parapsychology. In this view, the priority should be to pursue process-oriented research that can elucidate the mechanisms controlling these anomalies. It will then be possible to design experiments and conditions to produce the desired effects, rather than relying on experiments designed to produce statistical significance and satisfy a critical audience. (Critics, if they are given the last word, retort that parapsychologists should not bother presenting their case to the scientific establishment until they have succeeded in developing an adequate theory.) Iv. THE MEANING OF EXPERIMENTAL RESULTS ASSESSMENTS OF MEANING AND MEANINGFULNESS Leading off the "Findings and Conclusions" on parapsychology in the recent NRC report was a provocative assertion: "The Committee finds no scientific justification from research conducted over a period of 130 years for the existence of parapsychological phenomena." Although the treatment of parapsychology in the body of the report was generally more moderate in tone, this finding was taken by the media and many readers to be the "bottom line" of the report with respect to parapsychology. It produced considerable distress and resentment among parapsychologists and others, who felt that the field had been dealt an unwarranted blow.23 24 However, the statement did clearly delineate the view of many of those who monitor the field. Some critics are willing to concede that parapsychology research has demonstrated definite anomalies, but they believe that these reflect statistical or experimental artifacts, not psi. In their view, the 22 National Research Council, op. cit., p. 22. 23 See, for example: Palmer, J.A., C. Honorton, & J. Utts, op. cit. 24 This statement was based on a previously published assertion by Ray Hyman, chairman of the NRC subcommittee which authored the report (Proc. IEEE, June 1986, 74(6): 823-849). At the workshop, Hyman said that the precise wording of the NRC's statement was not his. Discussion ensued over the semantic differences between Hyman's statement and the NRC report statement. Hyman said that he would have phrased the statement differently. He later referred to any differences between the wording he would have used and the wording in the report as a matter of style, not substance (personal communication with Dr. Alan Shaw, 28 December 1988). Approved For Release 2000/08/08 : CIA-RDP96-00789R002200450001-8