reed Target and Geprerdappendaneias 2002106M%GGIA-RDP96-00789RO032002400 FG" 1994 one in a blind analysis from our 1992 study. The response to the waterfall target in Figure 1 included a number of pages of material about a city and other man-made elements. In all of our analyses, we strict- ly adhered to the concept that any material a receiver deletes from the the response prior to feedback is not counted in the analysis. As the receiver deleted the descriptions of man-made elements during the trial, the response in Figure 1 is considered as complete. This target-response pair received a score of seven. Figures 2 and 3 show examples of scores of four and one, respectively. In both cases, these re- sponses were not edited by the receiver. Hypotheses The overall null hypothesis was that the effect sizes will not be significantly different from zero. We used an X? to test the hypothesis that the quality of AC, as measured by rank-order, does not depend upon target type. Data Analysis and Results The analysis for this study was partially automated. All the trial information was stored in a computer file and could be read only by the analysis program to guard against inadvertent display. An analyst initiated the program and selected which receiver to analyze. Since the program kept track of the re- sults, it instructed the analyst which response to examine for the current trial. If the target for that trial was dynamic, the program instructed the analyst to insert enough disks, which were unlabeled with re- gard tocontent, so that the target and four decoys could be copied to the computer hard disk. If the trial target was static, this step was unnecessary, as the static targets were already present on the hard disk. A randomized order of the decoys and the target were presented in tabular form. A mouse click on the target name would launch either the dynamic or static display of the selected target. By this method, an analyst could review the entire target pack and rank-order them as usual. The ranks were entered into an appropriate place on the computer form. The ratings were done at the same time and entered into the form. Only after the completion of the analysis for this single trial was the data was locked into a file. The analyst could then select feedback as to the correct answer. The results for individual receivers were maintained in separate files. Three receivers participated in 10 trials for each target type and a fourth, 372, participated in 15 trials per target type. Tables 5 shows the average rank, the effect size and its associated p-value for the static target condition. We see that the combined data is significant and three of the four receivers produced independently significant results. Table 5. Results for Static Targets Paearier [Rant [S| pra
Target and Appdev Ob fendtelelase 2d02idart7 | CIA-RDP96-00789R003200220 08dr@h 1994 3) DELETE Lights, structure, structures, een, and city. We gots a waterfall, Figure 1. Target and response with a rating of 7. Approved For Release 2002/05/17 : CIA-RDP96-00789R003200240001-0
Target and SOABEP Dubsnadtelwase 2002/05/47. C|A-RDP96-00789R0032002000arb 1994 Fe ote LR SG aoe Wan hs a Su TE PESe Tt white rectangular box like an upsida—down sheet cake long hollow tube, like crashing surf on a beach — “Hawaii Pipeline”
BEGIN—10:20 AM putty balls — almost cotton—like. Cottony putty splotches. Movement — whizzing through these cottony puffs fast. Damp- ness. A long walkway & metal girders. Target and SAU PusBdriteReieese 2002/05/12 -GHA-RDP96-00789R00320 G24 0006101994 ! keep wanting to say — specifically fleid landing strip. Fiat land. Sig airplanes would land here like naval carriers. Has a broken white line down the center of strip & you see it straight on — like you wouid be _ Goming in for a landing. — ak-
Target and SéweeeDepémdidnelesse 2002/05/17 : CIA-RDP96-00789R00320020 dherrb 1994 CPYRGHT Rank-order We «-bserved a strong increase of AC for the static targets in the 1993 trials as compared to that of the 1992 trials (X? = 3.158, df = 1, p <0.075). Three of the four receivers had improved results in the 1993 trials as compared to those of 1992. Thus, the second hypothesis (i.e., an increase in AC for static tar- gets) was supported. Table 6 shows the same data for the dynamic targets. Table 6. Results for Dynamic Targets [Rear [meni [S| ee i [sm0 | _oaoo [0x00 | [se [ 20 [ote [oan Using the rank-order statistics above, we saw no difference between static and dynamic targets in this study. The first hypothesis was confirmed: we observed a significant increase of AC with dynamic tar- gets in 1993 from that of 1992 (X2 = 9.942, df = 1, p <1.6x 107%). We then examined the question of static vs dynamic targets with regard to our blind rating system. Fig- ure 4 shows the relative density for the static vs dynamic targets for the three significant receivers only. The mean and standard deviation for the static and dynamic targets was 3.3/-/.73 and 3.91+2.06, re- spectively (t = 1.32, df = 68, p <0.096). Including all receivers the means and standard deviations were 3.224187 and 3.51+2.06, respectively (¢ = 0.690, df = 88, p < 0.246). 9° a Relative Density fe) 0 2 4 Blind Rating Score 0 —> 7 Figure 4. Static (dotted) vs Dynamic Ratings for Three Significant Receivers Approved For Release 2002/05/17 : CIA-RDP96-00789R003200240001-0 15
Target and SGRePERpERERelesse 2002/05/17 : CIA-RDP96-00789R00320020 dba 1994 CPYRGHT It is difficult to interpret this analysis. If we claim that dynamic targets should be more readily sensed’ D AC, then we are entitled to examine only the significant receivers. While not overtly so, the trend sup- ports that assertion. Ratings, however, can be biased because of content. We could argue that this dif- ference is simply due to that fact that there is more content in the dynamic targets than in the static ones. There are two arguments against this assertion, however. In this experiment, the content of the dynam- ic targets was carefully chosen to match that of the static targets. In addition, our rating scale is sensitive to both incorrect and correct information. It seems unlikely, therefore, that the increase in scoring can be accounted for by content bias. We see little evidence for a target type dependency when we include all receivers, or when we examine the overall difference, using the rank-order data (X? = 0, df = 1,p <0.5). General Discussion and Conclusions — In our first experiment, we found that AC statistics with static targets were better than with dynamic ones. We hypothesized that this difference resulted from a combination of the target pool design and the receivers’ expectations. Following this idea, May, Spottiswoode, and James (1994a) define target pool bandwidth as the number of differentiable cognitive elements in a target pool. They suggest that a target pool, such as our original static pool, which contains enough elements to prevent guessing, while at the same time allowing for some internal editing of receivers’ imagination, is optimal with regard to the reduction of noise. In the first experiment, the dynamic target pool did not fit this ideal. When we constructed a better dynamic pool for the second experiment, we observed commensurate increases in the effect sizes. May, Spottiswoode, and James suggest that their target pool bandwidth concept is test- able, and it is our hope that these tests will be conducted in the near future. In the second experiment, even after correcting possible defects in our target pool design, we were un- able to observe a significant target type dependency. On the other hand, the direction for a replication is clear. May, Spottiswoode, and James (1994b) suggest that they have identified an intrinsic target property that correlates with the quality of AC (i.c., gradient of Shannon’s entropy). If this is true, then there might be a fundamental argument that implies that dynamic targets should be better than static targets, all else being equal. If a dynamic and static target pool were constructed on the basis of the largest possible gradients of Shannon’s entropy, then we would expect a significant improvement of the AC effect size and result that strongly favors the dynamic targets. Finally, we comment upon the sender condition. Our results show, asin forced-choice AC, that a sender is not a requirement. It is reasonable to expect that if the sender condition is not blind, then some de- pendencies might be observed. Dr. Robert Morris and the research group of the Psychology depart- ment at the University of Edinburgh are currently conducting a study to answer the necessary and/or sufficient requirement of a sender. Approved For Release 2002/05/17 : CIA-RDP96-00789R003200240001-0 16