Comments on the Extraversion-ESP Meta-Analysis by Honorton, Ferrari, and Bem

Among the most important contributions made by the late Charles Honorton were meta-analyses involving psi effects that had been studied in a sufficiently large numbers of experiments to merit such treatment (Honorton, 1985; Honorton & Ferrari, 1989; Honorton, Ferrari, & Bern, 1998). All of them had been published in the Journal of Parapsychology except the one involving ESP and the psychological trait of extraversion, a deficiency we are pleased to remedy in this issue. The principal conclusion from this latter meta-analysis was that there exists a positive relationship between extraversion and ESP scores, for both forced-choice and free-response measures of ESP, that is statistically significant (p = 4 x [10.sup.-6]) although small in magnitude (r = +.09). More refined analyses led Honorton and his co-authors to conclude that for forced-choice ESP tests, the result was due to an artifact, namely, the effect of "subjects' knowledge of their ESP performance upon their responses to the extraversion measure" (Honorton et al., 1998, p. 255). The purpose of this short paper is to challenge this latter conclusion, or, to put it conversely, defend the legitimacy of the extraversion-ESP relationship for forced-choice experiments. The general problem that underlies our disagreement with Honorton et al. has to do with confounded predictors. When we compare the performance of two groups of subjects in a single experiment that is properly conducted, the two groups differ (whether in terms of their individual traits or the treatments administered to them) on only one factor or independent variable. Therefore, significantly different scores on the dependent variable can be attributed only to this one independent variable or predictor. However, such is not the case when the results of two groups of experiments are compared in a meta-analysis. Because such databases usually include studies designed by many different investigators who generally do not seek to address in a controlled fashion the relationship of interest to the meta-analyst, such comparability on extraneous variables is not to be expected. In other words, the two groups of studies may differ in many other ways besides the one the meta-analyst seeks to assess, and one of these may be the real cause of differences in scores on the dependent variable. In the extraversion meta-analysis by Honorton et al., one independent variable of interest to the authors was whether subjects completed the extraversion questionnaire before or after they learned their ESP scores. They found that the positive extraversion-ESP correlation was only significant for those subjects who completed the extraversion scale after they took the ESP test, and thus after they knew their ESP scores. The authors offered two explanations for the results of these compromised studies. First, subjects' perception of their success or failure in the ESP task might have influenced how extraverted or introverted they perceived themselves to be. Second, the experimenters' knowledge of the ESP scores might have led to experimenter expectancy effects. With regard to these explanations, it should be noted that the authors provided no rationale for their plausibility, nor did they provide any evidence that extraversion scales are susceptible to such biases. We were first moved to look into the authors' claim of an artifactual relationship because their explanations did not appear plausible to us. This is not for parapsychological reasons, but because of the nature of the measures of extraversion that the studies employed. Personality inventories have been constructed in such a way as to meet the criterion that the scores they produce are very, nonreactive to a person's ongoing experiences - changes of mood or state of mind, etc. This psychometric aim has generally been achieved (e.g., Cattell, 1957; Gough, 1957; Kelly, 1955). Another way of putting this is to say that these instruments have high test-retest reliability. …