Approved For Release 2000/08/08 : S'E-EERFE-o789R002200470001-6 (U) For the purpose of this analysis, all putative causal-process experiments are considered under the general heading of remote action, (U) Figure 1 shows a schematic representation of these categories and the total number of individual trials that were conducted within each category. Psychoenergetics 26,074 : Informational 25,045 Forced Choice Search 4,153 SECRET FIGURE 1 (U) CATEGORIES AND NUMBER OF TRIALS (S/NF) The total number of psychoenergetic trials (26,074) was collected in 154 different experiments involving 227 different subjects.* All the data were entered into a computer database management system (DBMS). B. (U) Database Management System 1. (U) Database Requirements (U) One of the main purposes of performing a meta-analysis is to be able to look at data gathered from multiple studies conducted under a wide variety of circumstances. In order to collect and store the data in a meaningful way, one must know what kind of data manipulations will be performed. To evaluate the effect of certain parameters on psychoenergetic functioning, we needed to focus our attention on the conditions of a wide array of potentially important variables. As a result, the database design is primarily determined by the data and provides for the selection of information, by experiment, given parameter specifications. * (U) The number of subjects does not include the preliminary mass screening partici- pants. The formal screening participants were, however, included in the analysis. SECRET Approved For Release 2000/08/08 : CIA-RDP96-00789R002200470001-6
Approved For Release 2000/84 ClafaiS Sd fob Ex) 02200470001-6 2. (U) Database Design (U) The database schema that was used consists of four basic tables (people, documents, experiments, and units), and two basic relationships (author and parameter). See Figure 2 for an illustration of this schema. The units-table contains information about the lowest level of statistical analysis in a given experiment. For example, if 6 viewers participated in 20 trials each, the database would contain 6 unit entries—one for the overall result for each viewer. (U) Although our database management system is a relational database, our requirements were inherently hierarchical. That is, each of the documents contains several experiments, and each of our experiments contain several trials. In order to minimize the redundancy within the database, we attempted to include all pertinent information as high in the hierarchy as possible. That is, if a parameter or condition applied to an entire experiment, we would record that data at the experiment level. If, on the other hand, the parameter varied across units within a given experiment, we made provision to record those data as a function of unit instead. (U) The analyses of most of our experiments contain both individual and group statistics. In order to prevent any trial from being “counted” multiple times, we required that all experiments be broken up into the “units” which represent the basic grouping of trials upon which a hypothesis was being tested. Thus, any given trial appears only once in the database yet we can reconstitute the group statistics at a later time. (U) This approach offers two advantages. First, any arbitrary parameter which does not have an explicit slot in the database can be stored, thus providing flexibility. Second, we can distinguish between “independent variables” and “incidental variables... The former are variables which are intentionally manipulated by the experimenter, and the latter are actually parameters which the experimenter either could not control or treated as insignificant. (U) Some of the documents detail multiple analyses for a given experiment in order to compare and evaluate standard and new analytic techniques. For this effort, however, we required that only one analysis be recorded for each experiment, since our primary focus was to evaluate the parameters that effect psychoenergetic functioning and not to compare different evaluation techniques. In determining which analysis to enter into the database, we always chose a blind method over a post hoc method. Ifa choice still remained, we then always chose the technique that had been developed first. 6 UNCLASSIFIED Approved For Release 2000/08/08 : CIA-RDP96-00789R002200470001-6
Approved For Release 2000/43) @8-Gih Av 8 -PP Hei 2200470001-6 Principal Investigator Subject ID Monitor ID P One-to-Many arameters Relationship Entity Relationship UNCLASSIFIED FIGURE 2. (U) DATABASE SCHEMA DESIGN FOR META-ANALYSIS (U) The Appendix contains examples of the DBMS input sheets that were used to encode psychoenergetic data for the database, and the instructions that were given to analysts. They are included in the Appendix for completeness; there is no further discussion about them in this report. C. (U) Statistical Methods 1. (U) Effect Size Calculations (U) Effect sizes were calculated for each experiment or condition using the formula given by Rosenthal: 4 d=—, vn where n is the number of trials and z is the usual normalized output score. If no z score was given for an experiment, but a p value was, the z that would have given that p value was computed and used in the formula. The exception to this procedure was for experiments based on a sum-of-rank statistic. For those, a more appropriate effect size formula was used and is given by UNCLASSIFIED Approved For Release 2000/08/08 : CIA-RDP96-00789R002200470001-6
Approved For Release 20009 Pq ClaAaS Sldeh bb) 02200470001-6 (U) Se (R+ 1) R?-1i 12 where S is the average rank and R is the number of choices for each rank. 2. (U) Comparisons Across Classes (U) Experiments can be categorized in accordance with a number of specific variables (e.g., type of feedback, type of target, distance between the viewer and the target). Effect sizes can be examined within a given category and compared across categories. For each categorization, the following questions are of interest: (1) Question 1: Is there any evidence of psychoenergetic functioning within each of the individual categories? (2) Question 2: Is the level of psychoenergetic functioning constant across all experiments within a category? (3) Question 3: Is the level of psychoenergetic functioning constant across categories? (4) Question 4: If there are differences across categories, what is the relative size of the effect in each category? (U) Table 1 shows the notation that is used in the formalism that answers these questions. (U) To answer question 1, compare the average z score in each category with the standard normal tables. (U) To answer question 2, compute k mM; Ow= > > niy(diz-di.)?. isije=l If effect sizes are homogeneous within categories, the distribution of Qw will be approximately X2 with v = (Zmi - k) degrees of freedom. The hypothesis of homogeneity is rejected if Qw is large compared to the chi-square table entry with v degrees of freedom. To test for homogeneity within a single category, i, compute mi Owi= > nij(dij-di.)?. jel 8 UNCLASSIFIED Approved For Release 2000/08/08 : CIA-RDP96-00789R002200470001-6