Approved For Release 2000/08/10 : CIA- RDP96- 00789R003800440001- UNCLASSIFIED (U) This analysis produces the average figure of merit for a series and an indicator, the slope, of learning. C. (U) Mean Chance Expectation (U) The calculation for the mean chance expectation (MCE) must be sensitive to a number of possible artifacts or confounding factors: Viewer variations (e.g., viewers’ different response biases). General knowledge of the target pool (e.g., targets are known to be National Geographic magazine photographs). ® Specific knowledge of the target pool resulting from _trail-by-trial feedback. @ Methodological considerations (e.g., viewers are asked to respond with more data at the end of the series comparéd to what was asked of them at the beginning of the series). All of these factors will affect the expected average figure of merit and any session-to-session : wv systematic variation that may be present. (U) A method for determining the figure of merit MCE, which requires the fewest number of assumptions about the structure of the data or the response biases, involves the cross—matching of all the responses to the same target set used in the series in question. A cross-match is defined as.a comparison between a response and a target other than the one used in the session. If a figure of merit distribution is calculated for a large number of cross-matches, a number of the confounding factors listed above will be addressed. To determine the session-to-session dependencies of the MCE, however, the session order must be preserved. By preserving the order of the responses and by calculating [n] sets of cross—matches at a time, MCE figure of merit, slope, and intercept distributions can be calculated. . (U) As before, let [n] be the number of sessions in a series for a single viewer. Also, let the order of the responses, Rx be preserved. Define [N] as the number of cross-match cycles through the ordered set of [n] responses. The MCE calculation proceeds as follows: 1. Randomly choose a target order, i = i,n , such that k # i -where k = 1, n is the preserved response order. 2. Calculate the figure of merit for the kth response/target cross~matches as 7 Approved For Release 2008 NEC LARSSI BEE 003800440001-2
- (U) My = 4g X Tyr K Ais where a ki and r «i ore the accuracy and reliabil cross-matched against target [i], respectively. 3. Do step 2 above for all [n] sessions. 4. Calculate a slope and intercept for the resulting figures of merit by the linear least-squares analysis described above. 5. Repeat steps 1 through 4 above for [N] cycles to produce MCE figure of merit, slope, and intercept distributions. (U) It is important to note that MCE distributions are generated for each viewer and are not summed across viewers. Therefore, individual viewer differences in response “biases” are accounted for by definition. (U) This procedure also accounts for general knowledge of the target pool by the viewer, because information learned by this method in a given session will not necessarily be associated with the intended target for that session. The net effect of this type of artifact will be to “bias” the MCE figure of merit distribution toward larger values. , (U) Because the order of viewings is preserved, any knowledge of the target pool that is learned by the viewer as a result of trial-by-trial feedback is accounted for in two | ways: 1. \ nformation resulting from increasing knowledge of the target pool will “bias” the MCE figure of merit distribution toward larger values. 2. Information resulting from increasing knowledge of the target pool as a function of session number will “bias” the MCE slope distribution toward larger values. (U) Similarly, any artifact caused by methodological considerations as a function’ of session, will also “bias” the MCE figure of merit and slope distributions. d. (U) Probability Assessment and Analysis (U) There are a number hypotheses that can be tested using the various MCE distributions described above: e An individual remote viewing is statistically beyond MCE. 8 _.. Approved For Release 2000/4 NCL ALSSLELED3800440001-2
| Approved For Release 2000/08/10 : CIA-RDP96-00789R003800440001-2 " ; & UNCLASSIFIED (U) @ The series generated by a single viewer shows statistical evidence for remote viewing. There is evidence above MCE for remote viewing “learning.” The mean of the observed figure of merit distribution is significantly larger than the mean of the MCE distribution. @ The observed figure of merit distribution is significantly different than the MCE distribution. (U) Using the MCE figure of merit distribution, a straightforward calculation of areas will determine whether a particular figure of merit from a single session is significant. Figure 1 shows an example of an MCE figure of merit distribution described above. Mkx is the figure of merit resulting from session [k]. The probability of obtaining a figure of merit of Mxx or larger caused by artifact is the patterned area divided by the total area under the curve shown in Figure 1. This technique can be used to assess the chance likelihood for all sessions in a series by a single viewer. w My Figure of Merit FIGURE 1 (U) AN MCE FIGURE OF MERIT DISTRIBUTION (U) To determine whether there is statistical evidence of remote viewing within a given series for a single viewer, the p-values for the individual sessions must be ‘combined. The primary method used for combining p-values was developed by Fisher.6 A X2 with two degrees of freedom is computed for each p-value and summed. The resulting X? is evaluated with 2n degrees of freedom where [n] is the number of p-values that were combined. If [k] is the session number, the appropriate total X? is given by 9 Approved For Release 2000)QBNDCYI4AR SS 4 Fup Ba) 03800440001-2
Approved For Release 2000/08/10 : CIA- RDP96- 00789R003800440001-2 UNCLASSIFIED (U) n , x= > -2.0 In py, df= 2n k=1 ; : where the p are the p-values for each of the [k] sessions. A second technique involves k : ; testing the significancé of the average p-value across all sessions. ‘A standard z-score is calculated by Z = Vi2n (0.5-p), where p is the average p-value and [n] is number of sessions” (U) These two measures are sensitive to different aspects of the remote viewing series. For approximately 20 or more sessions, the two techniques will yield similar probability estimates if there is slight, but consistent evidence of remote viewing. On the other hand, if there are a few very good results (i.e., individual p-values < 0.001), then the X? technique more accurately reflects the series as a whole. (U) As an example of consistency, suppose 20 sessions having individual p-values of 0.35 each are analyzed. Then the z-score for the average p-value is 2.32, corresponding to a combined p-value of 0.01. The X? technique yields a total X? of 42.0 with df = 40, corresponding to combined p-value of 0.40. To illustrate the X? technique’s sensitivity to “good” remote viewings, consider the following p-values for 5 individual sessions: 0.45, 0.72, 0.55, 0.004, and 0.00005. The average p-value technique yields a combined p-value of 0.11, while the xe technique yields a combined p-value of approximately 0.0005. (U) To determine whether there is evidence of “learning” and whether the means of the actual and MCE figure of merit distributions are significantly different, an ANOVA technique is used.” By transforming the data about the average value of the session number, the slope and intercept hypothesis testing may be done separately. The F-ratios (from the ANOVA) for the two tests are given below. 10 Approved For Release 200d NC LASSIEIED coss00s40001-2
Approved For Release 2000/08/10 : CIA-RDP96-00789R003800440001-2 UNCLASSIFIED (U) n (b - by? x GD ( iw - F(slope) «= ——2———__________—_ df = 1, df2 = (n-2) A... and n(a- a’)? A F(intercept) = dfi = 1; df2 = (n-2), where [a] and [b] are the intercept and slope from the remote viewing figure of merit data, and [a’] and [b’] are the intercept slope from the MCE figure of merit data. A is given by n n n nn. n DM, +na?4 b? DK 4 2ab Dk-2a DM,- 2b DKM, ket kel k=l k=l k=l y n-2 (U) Because the F-ratio for the slope for the figure of merit data is a "statistical test between the observed slope and that computed from the MCE, it constitutes an estimate of the probability that remote viewing “learning” occurred over and above any contribution that might have occurred because of some artifact. The F-ratio for the intercept constitutes an estimate ‘of the probability that the mean of the figure of merit distribution is different from the mean of the MCE. We use a standard X? measure, a more sensitive test, to determine if the observed figure of merit distribution is statistically identical to that of the MCE distribution. B. (U) Fuzzy Set Theory--An Enhancement Technique for Descriptor List Technology 1. (U) Overview (U) The figure of merit analysis is predicated on descriptor list technology, which represents a significant improvement over earlier “conceptual analysis” techniques, both in terms of “objectifying” the analysis of free response data and in increasing the speed and efficiency with which evaluation can be accomplished. It has become increasingly evident, 11 Approved For Release 20d CLA SSIFLED003800440001-2