Approved For Release 2000/08/08 : CIA-RDP96-00789R002200410001-2 Meta-analysis of Forced-Cholce Precognition Experiments 21 The difference across subject populations largely results from the superiority of studies with selected subjects: Studies using subjects selected on the basis of prior performance in experiments or pilot tests show larger effects than studies using unselected subjects. As shown in Table 6, 60% percent of the studies with selected subjects are significant at the 5% level. The mean z-score for these studies is 1.41 (sd = 1.36). The magnitude of effect size is significantly higher for selected subjects studies than for studies with unselected subjects. The t-test of the difference in mean effect size is equivalent to a point-biserial correlation of .186. TABLE 6: Selected versus Unselected Subjects Subjects N studies Stouffer Z Mean ESdir SD %SIG.0S Selected 25 7.05 0.055 0.072 60.0% Unselected 223 4.70 0.012 0.068 21.0% (246 = 2.97, p = 0015 Does this difference result from less stringent controls in studies with selected subjects? The answer appears to be “No.” The average quality of studies with selected subjects is higher than studies using unselected subjects (27 = 2.05, p = .051, 2-tailed). This result appears to reflect a general tendency toward increased rigor and more detailed reporting in studies with selected subjects. individual versus Group Testing Subjects were tested in groups, individually, or through the mail. Studies in which subjects were tested individually by an experimenter have a significantly larger mean effect size than studies involving group testing (Table 7). The t-test of the difference is equivalent to a point-biserial correlation of .234, favoring individual testing. Of the studies with subjects tested individually, 30.6% are significant at the 5% level. Psychophysical Research Laboratories Approved For Release 2000/08/08 : CIA-RDP96-00789R002200410001-2
Approved For Release 2000/08/08 : CIA-RDP96-00789R002200410001-2 Meta-analysis of Forced-Cholce Precognition Experiments 22 eee The methodological quality of studies with subjects tested individually is significantly higher than that of studies involving group testing (t143 = 3.5, p = .001, 2-tailed). This result is consistent with the conjecture that group experiments are frequently conducted as “targets of opportunity,” and may often be carried out hastily in an afternoon without the preparation and planning that goes into a study with individual subjects that may be conducted over a period of weeks or months. TABLE 7: Individual versus Group Testing —— Test Setting Nstudies Stouffer Z Mean ESgir SD %SIG.05 a Individual 98 7.24 0.029 0.074 30.6% Group 104 1.49 0.005 0.064 18.3% 1200 = 2.40, p = .0085 Thirty-five studies were conducted through the mail. In these studies, subjects completed the task at their leisure and mailed their responses to the investigator. These correspondence studies yield outcomes similar to those involving individual testing. The combined z-score is 3.01, with a mean effect size of 0.021 (sd = .079). Ten correspondence studies (28.6%) are significant at the 5% level. Eleven studies are unclassifiable with regard to experimental setting. Feedback A significant positive relationship exists between the degree of feedback subjects receive about their performance and precognitive effect size (Table 8). Subject feedback information is available for 95 studies. These studies fall into four feedback categories: No feedback, delayed feedback (usually notification by mail), run-score feedback, and trial-by-trial feedback. We gave these categories numerical values between 0 and 3. Precognition effect size correlates .258 with feedback level (103 df, p = .004). Of the 48 studies Psychophysical Research Laboratories Approved For Release 2000/08/08 : CIA-RDP96-00789R002200410001-2
ro” : = -00789R002200410001-2 Approved pl ReleRat, PROD UNRDE 9G 0078 recognition Experiments 23 involving trial-by-trial feedback, 21 (43.8%) are significant at the 5% level. None of the studies without subject feedback are significant. TABLE 8: Feedback Received by Subjects Feedback Level Nstudies Stoufferz Mean ESzir SDes % SIG. 05 $$ OOS No Feedback 15 0.00 -0.002 0.027 0.0% Delayed 21 2.27 0.009 0.035 23.8% Run-score 21 4.80 0.024 0.047 33.3% Trial-by-trial 48 7.59 0.048 0.094 43.8% Sr it OS SS si SSSR While trial-by-trial feedback is associated with the largest effect sizes and significance levels, there is no evidence that subjects’ performance improved over time. Feedback level correlates positively though not significantly with research quality (7103 = .134, p = .145), Inadequate randomization is the most plausible source of potential artifacts in studies with trial-by-trial feedback. We therefore performed a separate analysis on the 48 studies in this group, blocking on the randomization and control quality measures. Studies with optimal randomization do not differ significantly in either mean significance level or mean effect size from those with suboptimal randomization. For significance levels, t is 0.74 with 46 df (p = .465, 2-tailed). For ES,t is 0.89 with 14 df(p = .525, 2-tailed). Similarly, studies reporting randomness control data do not differ significantly in either significance level or effect size from those not including randomness controls. For significance levels, t is 0.25 with 46 df (p = .803, 2-tailed). For ES, t is 1.19 with 46 df (p = .241, 2-tailed), Psychophysical Research Laboratories Approved For Release 2000/08/08 : CIA-RDP96-00789R002200410001-2