Approved For Release 2000/4 eS]-Ch Ar bh ddehd02200470001-6 (U) Finally, to answer question 4, approximate 95% confidence intervals may be computed for the average effect size within a category using i- 10 UNCLASSIFIED Approved For Release 2000/08/08 : CIA-RDP96-00789R002200470001-6
Approved For Release 2000/08/08 : §if-QU¥R8-4o789R002200470001-6 V RESULTS AND DISCUSSION (U) (U) The results of the meta-analysis are presented here, where possible, in quantitative analytic terms, and various interpretations are discussed in detail. In addition, items that cannot be analyzed are discussed from a qualitative perspective. (U) The analysis proceeds in a top-down fashion in accordance with the hierarchy shown in Figure 1. A. (U) Overall Results (U) The overall analysis was conducted from three different perspectives: (1) All of the data, regardless of the purported skill of the subjects, (2) A subset of the data contributed by an experienced group of viewers, G/ (i.e., long-term, generally accepted expert viewers—002, 009, 131, 372, 414, and 504) (3) All of the data except for the group GJ (i.e., All-G1). Table 2 shows the number of trials n, total z score, p value, and effect size d for informational and putative causal processes and for the combination of the two. Table 2 (U) STATISTICAL RESULTS FOR MAJOR CLASSES Psychoenergetics 25,449 3.69 (-21) 9,825 3.46 (-12) 15,624 3.46 (-11) Informational 24,450 5.83 (-20) 9,702 1.14 (-11) 14,748 1.96 (-10) Causal 6.39 (-03) 1.99 (-02) 2.95 (-02) SECRET * (U) Powers-of-ten are shown in parentheses, 11 SECRET Approved For Release 2000/08/08 : CIA-RDP96-00789R002200470001-6
‘acne A _ Approved For Release 2000/08/08 : GERD RO ENP789R002200470001-6 (U) The number of trials shown in Table 2 differs slightly from those shown in Figure 1. ™ A few trials in each category were analyzed from a post-hoc point of view and therefore have not been included in the formal analysis. - (S/NF) The heterogeneity of effect size within each group for all classes is very large (i.e., the chi-squares for within-groups were large). This is to be expected for such a global analysis - and is frequently seen in meta-analyses of psychological data.5 The sources of the within-group variation include the psychoenergetic skill level of the subjects and fundamental differences om between psychoenergetic tasks. (S/NF) The data, regardless of subjects or process, show strongly significant evidence for _ psychoenergetic functioning (p < 3.69 x 10-21). Both the informational and putative causal processes show significant evidence of psychoenergetic functioning, as well. -_ (S/NF) Since p values are strongly dependent upon the number of trials, the modern trend in meta-analysis is to consider the trial-independent measure of effect size. From this a point of view, the magnitude of the psychoenergetic functioning appears roughly constant for all the data shown in Table 2, and, according to Cohen’s criteria for the interpretation of effect size,* corresponds to small effects.© The method of calculating overall effect size, however, = involves a weighted average (see Table 1) and thus may not provide an accurate picture of the size of the psychoenergetic functioning within a given category. To obtain more insight into the - nature of the functioning, we must examine the data within each category. vn) B. (U) Results for Categories Within the Informational Process (S/NF) Table 3 shows the number of trials, total z score, p value, and effect size for - categories within the informational process. The data show strongly significant evidence for psychoenergetic functioning for all categories regardless of subjects. The effect size, however, ~ begins to demonstrate category differences. (S/NF) The forced-choice effect size (d = 0.052) is equivalent to the overall effect size at shown in Table 2 (d = 0.059). Since the forced-choice category accounts for 77% of the total number of trials, the effect-size averaging technique biases the overall result. For example, the effect size (d = 0,209) for the RV-—Lab category is significantly larger than for the Forced—Choice ” case (X2 = 22.70, v= 1; p < 6.63 x 10-6). The RV-Lab effect sizes meet Cohen’s criterion for a medium-sized behavioral effect. oni ~ * (U) Values of 0.1, 0.3, and 0.5 correspond to small, medium, and large effects, respectively. 12 - SECRET Approved For Release 2000/08/08 : CIA-RDP96-00789R002200470001-6