Target ARP Be De pemcaciee 02/05/17 CIA-RDP96-00789R00320073 000K Gust 1994 Target Preparation Prior to beginning the study, an experiment coordinator randomly generated a unique, counterbal- anced set of 20 static and 20 dynamic targets for each of the five receivers.” Within each target type, a counter-balanced set of sender/no sender conditions was also generated, and all target selections were done with replacement. A copy of each target (i.e., either a color photograph or a short clip on video tape) was placed in an envelope and a trial number, 1-40, was written on the outside. Those envelopes containing targets from the no-sender condition were sealed and those for the sender condition re- mained unsealed. Each set of 40 targets were packaged separately and shipped to the PI. Trial Schedule Two of the five receivers resided in California, and the others resided in Kansas, New York and Virginia. The experiment was conducted over a five-month period. Individual schedules were developed so as to cause as little inconvenience to the receiver’s daily routine as possible. Not more than one trial per day or three trials per week were conducted. Session Sequence For each trial and for each receiver: (1) The PI selected the appropriately numbered envelope from the box of targets for the receiver. (2) In the sender condition, he looked at the selected target for 15 minutes and attempted to “trans- mit” it to the intended receiver during a prearranged trial period. (3) In the no-sender condition for the static targets, he placed the sealed envelope on his uncluttered desk for the 15 minute trial period. (4) In the no-sender condition for the dynamic targets, he played the video repeatedly for 15 minutes without sound, and with the TV monitor located in an unoccupied room. (5) At the conclusion of the 15-minute trial period, and after the receipt of the receiver’s response by FAX, he sent a copy of the target material (i.e., either a photograph or video tape) to the receiver by mail. During each trial: (1) At the prearranged time, the receiver withdrew to a quiet room in his or her home and sat at a desk. (2) For a period lasting up to 15 minutes, the receiver wrote and drew his or her impressions of the intended target material. (3) At the end of the trial, she/he sent a copy of the response to the PI by FAX machine. (4) By return mail, she/he obtained a copy of the target as feedback for the trial. The target copy and original response were subsequently sent to the experiment coordinator in Menlo Park, California. We did not provide specific instructions beyond logistical information to the receivers, because they were all experienced at this type of task. They were, however, knowledgeable about the general charac- teristics of the two target pools. When the experiment coordinator received the receiver’s response, all identifying information (i.e. name, date, and time of trial) was removed. Periodically during the course of the experiment, the ex- * Allrandomizations were donewith a standard computer algorithm, which is based ona shift-register algorithm by Kendell and has been shown to meet the general criteria for “randomness” (Lewis, 1975). Approved For Release 2002/05/17 : CIA-RDP96-00789R003200220001-2 4
Target ARB'SUHoF BoBSleBRE 290205"17 : CIA-RDP96-00789R00220087 99012 wor 1094 periment coordinator provided an analyst, who was blind to the target choice, with a set of responses and associated target packs for analysis. Analysis We conducted two different analyses in this study: (1) Our standard 1-of-5 rank-order technique to construct effect sizes and p-values. The targets that were used as feedback for the receivers were duplicates of the ones used in the analysis. (2) An analysis of variance (ANOVA) to address the 2 x2 questions. It is general policy in the Cognitive Sciences Laboratory not to combine the data of receivers. In this study, all data combinations are post hoc; however, all other analysis are a priori. Rank-Order For each trial, there was a single response and its associated target pack (i.e., either static or dynamic). An experienced analyst, who was blind to the condition and target for the trial, was asked to rank-order five targets (i.e., the intended target and four decoys) within the given pack.” This was a forced ranking, so regardless of the quality of match between the response and targets, he/she had to assign a first place match, a second place match, and so on for each of the five targets. The output from this part of the analysis was a rank-order number (i.e., one to five, one corresponding to a first place match) for the correct target. For each receiver, target type, and condition, there were 10 such rank-order numbers which constituted a block of data. A rank-order effect size was computed for a block as: R, ~ Ri; wee” 12 fy = where R,; is the average rank for target type i and sender condition j, and Ry is the expected average rank, which for this study is equal to three for all cases. N is the number of possible ranks and is equal to five throughout this study. The effect size reduces to: ~ TR é, = —_->,—. tj 2 Analysis of Variance A two-way analysis of variance (ANOVA) was computed for each receiver. The two primary variables were target type and sender condition (i.e, ANOVA main effects), Each of these variables possessed the two states shown in Table 1 above. * Since during the 1992 experiment, the analyst remained in the Menlo Park Cognitive Sciences Laboratory and all other partici- pants were never present during the study, he/she was blind to all conditions. Approved For Release 2002/05/17 : CIA-RDP96-00789R003200220001-2 5
Target AARPSQN8G ER pBAURAIEREZOO2/05/17 : CIA-RDP96-00789RO0320G22AORI:Zuct 1994 Hypotheses The overall null hypothesis was that ¢;; will not be significantly different from zero. Using an F-test, we hypothesized that the quality of AC does not depend upon a sender regardless of target type. Similarly, we used an F-test to test the hypothesis that the quality of AC does not depend upon target type, regardless of the sender condition. The ANOVA also tests for potential interactions between the target and sender conditions. For exam- ple, it might be that a sender is required for dynamic targets and not for static ones. We did not hypothe- size with regard to interactions. Effect Size Results Five receivers completed 40 trials each. Table 2 shows the effect sizes computed for the 10 trials in each cell. The underlined effect sizes indicate 1-tailed significant results. Receiver 009 showed significant evidence for AC in the static target, no-sender condition (p < 0.02); receiver 372 in the static target, sender condition; and receiver 518 in the static target, no-sender condition (p < 0.05). Combined, the Static, no-sender condition was significant (p < 0.02) Table 2. Effect Sizes Static Static Dynamic | Dynamic oe [on [tea [ar [ose ae [am [asa | =0ase | 0a [si [ 008s [asa | =009s [028 | [oa [one [naar | =n02s [om * Totals are post hoc. ANOVA Results Table 3 shows the results of an ANOVA on these data. Since there were 10 trials within each cell, the degrees of freedom are the same for all receivers and, therefore, are only shown in the column headings. Two receivers show significant main effects. Receiver 372 showed a tendency to favor static over dy- namic targets (i.e. p < 0.03), and receiver 518 showed a tendency to favor no sender conditions (i.e.,p < 0.04). Notice the underlined values in Table 3—for these receivers the ANOVA hypothesis that the data were drawn from the same distribution is rejected, and there were no significant interactions between target type and sender condition. Combining results, post hoc, across receivers, the ANOVA showed no significant main effect for the sender condition. The main effect for target type, while not significant, was strongly in favor of the stat- Approved For Release 2002/05/17 : CIA-RDP96-00789R003200220001-2
Target ARP SURFS pH aaneids 02/05/17 : CIA-RDP96-00789R003200930NPA 7 ust 1994 ic targets (F(1,196) = 2.91, p < 0.09)." We found no significant interactions for the combined data (F(1, 196) = 0.02, p < 0.89). Since there were no significant interactions, we combined the data for static targets regardless of the sender condition (i.e., 100 trials), The sum-of-ranks was 265 (i.e., exact sum-of-rank probability of p< 0.007, effect size = 0.248). The total sum-of-ranks for the dynamic targets was 300 (i.e., p < 0.50, effect size = 0.000). From these data, we concluded that static targets may be better than dynamic targets. Table 3. ANOVA Results pon [sat [a rar [oor [03s Discussion and Hypothesis Formulation Static targets apparently being better than dynamic ones (¢ = 1.75, df = 198, p <0.08, 2-t) is surprising not only because it fails to support the ganzfeld result, but also because it actually suggests the opposite. There are a number of possible contributing factors for this outcome. They include statistical artifacts, idiosyncrasies of our receivers compared to the ganzfeld participants, and procedural differences. Another possibility may be that rank-order statistics were used, as they were in the ganzfeld. We find absolute measures of AC are better than relative measures in process-oriented research, and since the target-type inference was based on relative measures, perhaps this accounts for some of the result. Please see an expanded discussion of this point in the 1993 experiment below. We propose, however, a different explanation: the fundamental differences between the target pools in this experiment are, in themselves, a source of noise and confound the interpretation. To understand this noise source, we must first assume that AC data are weak and difficult to recognize. Target pools which contain a large number of differentiable cognitive elements, in conjunction with re- ceivers who believe that this is the case, are a source of noise. Receivers are encouraged to report any imagined impressions, since those impressions might be part of the target. Since AC is assumed to be weak, most of what is generated is more from the receiver’s imagination than from the signal. This noise is generated from an active imagination coupled with an agreement not to edit the internal experience. A full description of these points can be found in May, Spottiswoode, and James (1994a). * Wecomputed a trial effect size of 0.121 given the F for the target type analysis. Since the published effect size of 0.159 is gener- ally considered as a robust effect for the ganzfeld (Bem and Honorton, 1994), we are justified in claiming that the static targets are “strongly” favored over the dynamic ones. Approved For Release 2002/05/17 : CIA-RDP96-00789R003200220001-2 7
Target SAAT SURdeF Bf REN@BRGIZYO2/05/17 : CIA-RDP96-00789R003200%2000 Argust 1994 The receivers in our experiments have learned the natural limitations of our usual National Geographic target pool by experience and by instruction. They have become skilled at internal editing and do not report impressions that they know are absent from the overall target pool, thus there is less incorrect material in their responses. We conclude, therefore, that in this experiment, receivers were unable to produce significant evidence of AC with dynamic targets. They produced, what is for us, significant reduced functioning with static targets, We speculated that this drop of functioning in both target conditions arose because the proto- col would not allow the receivers to edit their internal experience. They were told that the dynamic targets could be virtually anything, and since they were blind to the static-vs-dynamic target condition, they were unable to edit their imaginations, even for the static targets. Based on this speculation, we developed the following hypotheses for our replication study in 1993: (1) An overall significant increase of AC will be observed for dynamic targets if the dynamic pool is designed with a similar set of topics that match the static pool from the 1992 study. (2) An overall increase of AC will be observed for static targets because the receivers will be able to edit their internal experience. The 1993 Experiment In this experiment, we included a static vs dynamic target condition to replicate the findings from the ganzfeld, but dropped the sender condition, since it appeared not to influence the results of our 1992 investigation. All trials were conducted with a monitor but without a sender. Target Pools We redesigned both the static and dynamic targets with the constraint that they all must conform to the topic, size, and affectivity homogeneity of the original static targets. We identified a large number of videos that could be edited to produce 50 National Geographic-like segments. A single frame from with- in each video clip, which was characteristic of the entire clip, acted as its static target pool equivalent. Thus, we improved the target pools from our 1992 experiment in two ways: (1) The new dynamic pool possessed a reduced number of differentiable cognitive elements compared with the dynamic pool we used in 1992, (2) The content of the dynamic and static pools were nearly identical, by design. During the experiment, the targets were chosen randomly and were counter-balanced with regard to static and dynamic target types, within receivers. All static frames were digitized (Le., 640 x 480 pixels) for 24 bits of color information, compressed by JPEG, and stored on-line for feedback and display purposes. The dynamic targets were digitized at near real-time rate and stored on three magneto-optical read/write diskettes. The “video” clips could then be displayed on our full-color, Sun Microsystems computer monitor in real-time. Approved For Release 2002/05/17 : CIA-RDP96-00789R003200220001-2 8
Target ARP ERR BE RAIBARARPO2/05/17 : CIA-RDP96-00789R003200720004: Gust 1994 Receiver, Monitor, and Sender Selection For the new experiment, we chose four of the five experienced receivers who had participated in our 1992 study. All trials were conducted without a sender and were monitored by the PI, who was blind to target type and content for each trial. Protocol Three receivers contributed 10 trials in each of the two target conditions, and a fourth (ie., receiver 372) contributed 15 trials in each condition. Trial Schedule The experiment was conducted over a seven-month period, and all trials were conducted at our labora- tory in Menlo Park. One of the four receivers (i.e., Receiver 009) lives locally, but the others traveled to our facility for one-week visits. All viewers participated in no more than one trial per day. Session Sequence Before the session began, and after the receiver and monitor were sequestered in our AC laboratory, an assistant, who was otherwise not involved in the experiment, randomly generated a target (with replace- ment) in accordance with the target selection criteria (i-e., counter-balanced for type within receivers and randomly within type). During the session: @ The monitor provided the following tasking statement to the receiver: “There is a scene that needs a description. Access to that scene is through the word target.” © For a period lasting no longer than 15 minutes, the receiver wrote and drew his or her impressions of the intended target material, with the monitor asking for clarification on specific response elements. © When the monitor and receiver agreed that the data was complete, the monitor halted the session, copied the response material, and secured the original. © The monitor provided computer-based feedback of the intended target material and emphasized the points of agreement between the response and target. We again emphasize that for each trial the monitor and receiver were blind to the target selection. In this study, there was a single, experienced monitor, and he/she was trained only to seek clarification of receivers’ response rather than suggesting responses. All four receivers participated in a total of 20 trials with this design. At no time during these trials was the target material displayed during the AC session. Instead, the intended target, which existed on a computer disk, was designated only by name, and a laboratory assistant had exclusive knowledge of that name. Only during the feedback phase was the intended target displayed to the monitor and receiver. We asked receiver 372 to participate in an additional 10 trials that were randomly counter-balanced between static and dynamic targets. We used an automated version of the above procedure and, during the session, the target material was silently displayed on a computer monitor in another room. The session protocol was identical to the one above except for the automatic target generation and display. Approved For Release 2002/05/17 : CIA-RDP96-00789R003200220001-2 9
Target Ape GaribF Db RANeaReIgP 02/05/17 : CIA-RDP96-00789R003 20082 OGM ag ust 1994 For these 10 trials, the monitor initiated an automatic computer program after receiver 372 had entered the AC laboratory. This program randomly counter-balanced the target type and selected a single tar- get for the session. Regardless of the type, the program required that a specific optical disk, unlabeled with regard to content, be mounted and the dynamic version of the selected target was then copied to an internal hard disk. All static equivalent targets were already resident on the internal hard disk. Once the transfer was complete, the monitor was instructed to initiate the trial. For the next 60 seconds, the computer screen remained blank, thus allowing the monitor sufficient time to enter the adjacent AC laboratory and remain blind to the target choice. At the end of the 60 seconds, the computer program began to continuously display the target regardless of type. The computer program kept track of all the specific details that were used later during the analysis phase. Analysis We conducted two different a priori analyses in this experiment: (1) Our standard 1-of-5 rank-order technique to construct effect sizes and p-values. (2) A blind rating from a predetermined rating scale. Our rank-order procedure was similar to the one we used in our 1992 experiment. The sole difference was how and when the decoy targets were chosen. In our earlier investigation, the decoys were prede- termined using fuzzy set analysis and fine tuning. Thus, they existed prior to the start of the experiment. In this study, the decoys were chosen by computer at the time of analysis, and did not exist during the actual trials. Prior to the start of this experiment, we divided our 50 targets into 10 sets of five targets each. Differing from our earlier approaches, the targets within each pack were as similar as possible. We were able to identify five broadly different topic categories (e.g., cities near water, ruins, etc.), and we created two different packs of five targets for each specific category. We made all target pack decisions based on our experience and subjective assessment. Decoys were chosen by the computer at analysis time. First, the computer selected the topic set of five packets from which the actual target was chosen. Then, the computer randomly selected one target from each of the remaining four target packs for the decoys. The analyst was the same individual as in the 1992 study. Differing from that study, however, all the trials were conducted in the Cognitive Sciences Laboratory. The PI/monitor and the receivers were present only during the sessions, and the analysis was performed at the conclusion of the data acquisi- tion without the PI or the receivers present. Otherwise, care was taken to prevent contact between the analyst and the experiment participants during the sessions. Blind Rating Scale Rank-order analysis does not usually indicate the absolute quality of the AC. For example, a response that is a near-perfect description of the target receives a rank of one. Buta response which is barely matchable to the target may also receive a rank of one. Table 4 shows the rating scale that we used to perform a blind assessment of the quality of the AC responses, regardless of their rank. Even though ranks correlated with ratings, we feel that rating scales like this potentially reduce an additional source of variance in correlational or comparative studies. Approved For Release 2002/05/17 : CIA-RDP96-00789R003200220001-2 10
Target Ane SarnieF ob mealaanieig? 02/05/17 : CIA-RDP96-00789R003200930Q0A Rust 1994 To apply this subjective scale to an AC trial, an analyst begins with a score of seven and determines if the description for that score is correct. If not, then the analyst tries a score of six and so on. In this way the scale is traversed from seven to zero until the score-description seems reasonable for the trial. Table 4. 0-7 Point Assessment Scale Score Description 7 Excellent correspondence, including good analytical detail, with essentially no incorrect information Good correspondence with good analytical information and relatively little incorrect information. Good correspondence with unambiguous unique matchable elements, but incorrect information. ws some Good correspondence with several matchable elements intermixed with incorrect information. Mixture of correct and incorrect elements, but enough of the former to indicate eiver has made contact with the site. > ta rec i) Some correct elements, but not sufficient to suggest results beyond chance expectation. Little correspondence. [0 [- No correspondence Figures 1 through 3 (pages 13 through 15), illustrate the application of this scale and show that the quali- ty of an AC response is not indicated by a first-place ranking. All three examples were given a rank of one in a blind analysis from our 1992 study. The response to the waterfall target in Figure 1 included a number of pages of material about a city and other man-made elements. In all of our analyses, we strict- ly adhered to the concept that any material a receiver deletes from the response prior to feedback is not counted in the analysis. As the receiver deleted the descriptions of man-made elements during the trial, the response in Figure 1 is considered as complete. This target-response pair received a score of seven. Figures 2 and 3 show examples of scores of four and one, respectively. In both cases, these responses were not edited by the receiver. Hypotheses The overall null hypothesis was that the effect sizes will not be significantly different from zero. A t-test were used to test the hypothesis that the quality of AC, as measured by rank-order, does not depend upon target type. Data Analysis and Results The analysis for this study was partially automated. All the trial information was stored in a computer file and could be read only by the analysis program to guard against inadvertent display. An analyst initiated the program and selected which receiver to analyze. Since the program kept track of the re- sults, it instructed the analyst which response to examine for the current trial. If the target for that trial was dynamic, the program instructed the analyst to insert enough disks, which were unlabeled with re- Approved For Release 2002/05/17 : CIA-RDP96-00789R003200220001-2 "
Target Ana SuinieF Db RMaaneias02/05/17 : CIA-RDP96-00789R003200920MA rust 1994 gard to content, so that the target and four decoys could be copied to the computer hard disk. If the trial target was static, this step was unnecessary, as the static targets were already present on the hard disk. A randomized order of the decoys and the target were presented in tabular form. A mouse click on the target name would launch either the dynamic or static display of the selected target. By this method, an analyst could review the entire target pack and rank-order them as usual. The ranks were entered into an appropriate place on the computer form. The ratings were done at the same time and entered into the form. Only after the completion of the analysis for this single trial was the data was locked into a file. The analyst could then, and only then, learn the correct answer. The results for individual receivers were maintained in separate files. Three receivers participated in 10 trials for each target type anda fourth, 372, participated in 15 trials per target type. Tables 5 shows the average rank, the effect size and its associated p-value for the static target condition. We see that the combined data is significant and three of the four receivers produced independently significant results. Table S. Results for Static Targets [reste [ents [os _ [pe i a Approved For Release 2002/05/17 : CIA-RDP96-00789R003200220001-2 12
Target An@iseinthF pb alaaneief02/05/17 : CIA-RDP96-00789R003200920Q04 Rust 1994 TARGET 2) Troubled by city fesiing. Could be that the uprights are natural rather than man— made. in which case the city interpretation is incorrect and | am feeling MESA. |'l! check verticaia. 3) DELETE Lights, structure, structures, building, and city. We gots a watarfall, dude. ‘ Figure 1. Target and response with a rating of 7. Approved For Release 2002/05/17 : CIA-RDP96-00789R003200220001-2 13