Ap a-Ana proved For Release INI CIA: SSTRAPES-00789R002200390001-5 Table 3: Results of one-way analysis of variance Grand mean N MEAN Z sD SE 297 0.5979 1.5823 0.0918 Person N MEAN Z SD SE Andre 4 2065 1.9091 0.9546 Bierman 2 2.7570 1.3863 0.9803 Braud 4 1.8797 0.9373 0.4687 Broughten 4 -0,2347 0.3048 0.1524 Debes 8 0.1260 1.8205 0,6437 Dunne 144 0.3386 1.1842 0.0987 Edge 10 0.1166 2.0067 0.6346 Giesler 12 0.7778 0.8011 0.2313 Heseltine 19 -0.0885 1.7124 0.3928 Hill 1 2.9498 Honorton 14 0.4071 1.1328 0.3028 Houtkooper 4 0.4906 1,4944 0.7472 Jungerman 1 2.3322 Kelly 2 2.3799 0.3015 0,2132 Matas 2 0.3625 2.9522 2.0875 May 1 —2.3841 Millar 2 -0.6187 1.6406 1.1601 Morris 5 0.8206 1.0562 0.4723 Morrison 3 0.7746 0.4926 0.2844 Palmer 1 1.7500 Pantas 4 0.7625 2.4453 1.2226 Radin 4 2.1712 0.8822 0.4411 Randall 6 -0.0120 1.1753 0.4798 Schechter 2 —0.7496 4.2411 2.9989 Schmeidler 1 -1.2728 Schmidt 30. 2.4144 2.0341 0.3714 Shafer 2 -1.0178 1.1158 0.7890 Winnett 5 -~0.0396 0.5795 0.2592 SOURCE SS df MS F p person 197.9629 27 7.3320 3.631 2.78x10 8 error 543.1467 269 2.0191 B-18 UNCLASSIFIED Approved For Release 2000/08/08 : CIA-RDP96-00789R002200390001-5
Approved For Release NECA SSE Oooo Roz 20088000 Fes 1 To see whether the mean z score might be related to the number of experiments each investigator ran, we performed a correlation between N and MEAN Z (in Table 3). Results were as follows: Correlation r-squared t(21) . p -0.0185 0.0003 -0.0941 0.9257 In summary, taking the data taken at face value (i.e. not weighted by quality analysis), we can make two statements: First, considering all available data, there do appear to be significant differences among mean z scores obtained by different experimenters. Second, there is a nonsignificant correlation between the number of experiments run by principal investigators and their mean z scores. So to return to the question at the beginning of this section: Do different experimenters obtain about the same results? The answer is no -~ experimenters in this survey showed mean z scores ranging from -2.38 to 2.95. As to the question of whether only one or two individuals may be responsible for the overall significance, the answer is also no; 25% of the experimenters in our survey reported mean z scores beyond 2 and -2. 5. RNGs were nonrandom This criticism may be addressed by examining the results of control studies reported in the literature. The results shown in Table 4 were compiled from 14 of the 44 detailed references referred to in Table 3, and were contributed by the following twelve authors: Dunne (Princeton), 57 control studies; Schmidt, 23; Broughten, 8; Braud, 2; and one each for Bierman, Hill, May, Millar, Morris, Schechter, Honorton, and Palmer. The other references did not report control results in detail and could not be used. Table 4: Combined control studies Number of _ Data control studies z's Zz overall z p (2-tail) Survey 41 -0.012 -0.0003 -0.002 0.999 1.036 Princeton 57 2.829 0.0496 0.375 0.708 Combined 98 2.817 0.0287 0.285 0.776 A variance test of the observed standard deviation (sd = .905) against the expected variance of 1.0 for 98 samples results in a chi-square = 80.2645 (97 df), z = -1.22, and p < .222 (2-tailed). Thus, for the references where control runs were described in sufficient detail to determine the B-19 Approved For Release sob AGE Ad SITIED . 29R002200390001-5