Approved For Release 2000/08/10 : CIA-RDP96- 00789R003800440001-2 ‘UNCLASSIFIED (U) however, that current lists are inadequate in providing discriminators that are “fine” enough both to describe a complex target accurately and to exploit fully the more subtle or abstract information content of the RV response. To decrease the granularity of the RV evaluation system, therefore, it was determined that the technology would have to evolve in the direction of allowing the analyst a gradation of judgment about target and response features, rather than the current “all-or-nothing” binary determinations. A preliminary survey of various disciplines and their evaluation methods (spanning such diverse fields as artificial intelligence, linguistics, and environmental psychology) has revealed a branch of mathematics, known as “fuzzy set theory,” which provides a mathematical framework for modeling situations that are inherently imprecise. The principal architect of fuzzy sets, L. A. Zadeh, has stated: “One of the aims of the theory of fuzzy sets is the development of a methodology for the formulation and solution of problems which are too complex or ill-defined tobe susceptible to analysis by conventional techniques.”® (U) Because the task of RV analysis requires human judgments about imprecise situations--namely, the categorization of natural sites and the interpretation of abstract representations of those sites-—it would appear, according to the above definition, that fuzzy set theory is a promising line of inquiry. In the next section, some of the basic concepts of fuzzy set theory will be examined, with the aim of understanding how this technology might be applied to the specific problem of RV evaluation. 2. (U) A Tutorial (U) In traditional set theory, an element is either a member of a set or it isn’t——e.g., the number 2 is a member of the set of even numbers; the number 3 is not. Fuzzy set theory is a variant of traditional set theory, in that it introduces the concept of degree of membership: herein lies the essence of its applicability to the modeling of imprecise systems. For example, if we take the concept of age (known as a linguistic variable in fuzzy set parlance), we might ascribe to it certain subcategories (i.e., fuzzy sets) such as very young, young, middle-aged, old, etc. Looking at very young, only, as a fuzzy set example, we must 12 _ Approved For Release 2000/3 NGL ASS FEED) 3800440001-2
Approved For Release 2000/08/10 : CIA- RDP96-00789R003800440001-2 UNCLASSIFIED - (U) define what we mean by this concept vis-a-vis the linguistic variable age.* If we examine the chronological ages from 1 to 30, we might subjectively assert that we consider the ages 1 through 4 to represent rather robustly a spectrum of the concept very young, whereas the age of 30 probably does not accurately represent very young at all. As depicted in Figure 2, fuzzy set theory allows us to assign a numerical value between 0 and 1 that represents our best subjective estimate as to how much each of the ages 1 though 30 embodies the concept very young. Very young 0.1 0.01 7.0 DWee0eteeeeee 30 FIGURE 2 (U) THE FUZZY SET “VERY YOUNG” (U) Clearly, a.different set of numerical values would be assigned to the ages 1 through 30 for the fuzzy sets young, middle-aged, and old—-e.g., the age of 6 might receive a value of 0.5 for very young, but a value of 1.0 for young, depending on context, consensus, and the particular application of the system. In this way we are able to provide manipulatable numerical values for imprecise natural language expressions; in addition, we are no longer forced into making inaccurate binary decisions such as, “Is the age of 7 very young-—yes or no?” * (U) It is important to note that the design of the fuzzy application occurs in accordance with the subjectivity of.the system designer. Fortunately, the fuzzy set technology is rich enough that it allows for a virtually unrestricted range of expression. Technically speaking, young is the fuzzy set and very is a modifier, but it is beyond the scope of this paper to present terminology in depth. 13 Approved For Release sooo NCLASSIEIER03200440001-2
Approved For Release 2000/08/10 : CIA- RDP96- 00789R003800440001 “2 UNCLASSIFIED 3. (WU) Initial Application to RV Evaluation (U) In FY 1986, work began onan initial application of fuzzy set technology to RV evaluation, which simply entails an extension of the current descriptor list capabilities. In coding the targets, an analyst employs numerical values between. 0 and 1, inclusively, to rate each of the 20 descriptors according to the importance of its visual representation in the target. For example, in rating a National Geographic magazine picture of the Khyber Pass, an analyst might ascribe a value of 0.80 for a “mountain” descriptor, a value of 0.20 for a “desert” descriptor, and values for other | appropriate descriptors in accordance with their perceived importance to the target as a whole.* (U) The rating of responses is considerably more subjective than the rating of © targets. The analyst is required to apply a “confidence rating" *—-i.e., again, a value between 0 and 1, inclusively--as to what degree an abstract ideogram is representative ofa given descriptor. For example, if a novice subject draws a conical-shaped object and labels it, - “fuzzy cone...wider at the bottom...” the analyst may decide that there is some justification for interpreting this ideogram as a volcano covered with vegetation. Clearly, however, the confidence factor for making this highly subjective determination is quite low; the net result might be, therefore, that the “volcano” descriptor might receive a rating of 0.15, while . “foliage” might receive 0.05. (U) We anticipate that the primary effect of implementing this rudimentary application of fuzzy set technology will be to “fine tune” the figure of merit scores, such that they are more representative of the “true” information content of an RV response. The current figure of merit application penalizes certain responses and inflates others (especially given the “noisy” aspect of novice data), based on the correctness or incorrectness of the analyst’s “all-or-nothing” determination with regard to any given descriptor... To summarize, fuzzy set technology is attractive in two important respects: (1) it affords the. analyst a wider range of expression, thereby enabling him/her to provide a more realistic portrayal of the information contained in both targets and responses, and (2). it is compatible with the figure of merit mathematical formalism. 4. J) Potential Future Applications (U) It is anticipated that the initial application of fuzzy set technology to RV responses and targets will greatly enhance the accuracy with which their information content is *“ (U) Coding of both targets and responses might be more “objectively” arrived at via the consensus of a group of experienced analysts. Approved For Release 20003 Chak SSA Fol EB) 03800440001-2
Approved For Release 2000/08/10 : CIA- RDP96- 00789R003800440001 2 UNCLASSIFIED . (U) depicted. A problem remains, however, with the inherently large granularity of the current descriptor list, which is independent of the potential “fineness” of its application allowed by fuzzy set theory——although the analyst will be allowed a gradation of response in interpreting an abstract ideogram (i.e., the confidence factors ranging from 0 to 1), he/she will still be constrained to interpreting that ideogram according to 20 concrete descriptors. These descriptors are significantly limited in their ability both to portray rich environments and to distill the most usable information from abstract RV mentations. (U) It is projected that future descriptor lists will afford the analyst greater latitude in interpreting the more abstract aspects of RV responses, by providing basis vector descriptors. Such descriptors would represent, in essence, the lowest practicable common denominator of abstraction from which more concrete descriptors might be. generated using fuzzy set operations (such as intersection and union). An example of a basis vector descriptor might be the concept of vertical, which is an abstraction that is represented to varying degrees in such concrete descriptors as building, cliff, mountain, waterfall, etc. of (U) Ultimately, we envision that evaluation would proceed along the lines of analyzing both the RV responses and targets in terms of fuzzy—weighted basis vector: descriptors. A comparison of basis vector descriptors between responses and targets could then be effected, which would culminate in a figure of merit analysis reflecting the subject’s | ability to debrief the more abstract components of the psi signal. By using fuzzy set operations, concrete target and response descriptors could subsequently be generated on a “best fit” basis from the basis vector descriptors, and a figure of merit evaluation could be performed at this higher‘order level also. The primary benefits of this type of procedure would be in providing objectification of abstract response data, and in affording’more automated interpretation of these data in concrete terms. Furthermore, it would also be possible to track, in a systematic and quantifiable manner (on both a “subject—by-subject” and “across subject” basis), the kinds of abstract signals that subjects are receiving reliably; presumably, this capability might then be used to illuminate important lines of future investigation within RV fundamentals. 15 .. Approved For Release 200 NIC A S841 FaEDr003800 440001-2°.
Approved For Release 2000/08/10 : CIA- RDP96- 00789R003800440001 -2 UNCLASSIFIED III’ RESULTS (U) two sources: (1) identification of inter-analyst reliability factors, based on analysis of figure of merit statistics, and (2) insights into descriptor list formulations and target pool composition, based on post hoc analysis and observations. Each of these areas is explored in turn below. (U) The results of the FY 1986 evaluation effort have been obtained primarily from ; A. (WU) Inter-Analyst Reliability Factors | (U) A method was developed in FY 1986 for rating the abilities of potential remote viewing analysts. The most direct method of accomplishing this was simply to ask a candidate to analyze a known series of remote viewings; the results could then be compared to those produced by a proven analyst, 374. ; . {U) Three individuals, 432, 579, and 642, were asked separately to score first thé targets and then the responses used in a remote viewing series from a novice viewer. They used a twenty-bit descriptor list (see Table 1) under a “blind” protocol. The procedure described in Section II.A. was used to calculate figures of merit, session p-values, and overall p-values for each analyst. 4 (U) Novice remote viewing data, which have been collected under our stimulus/response protocol, contains two distinguishing characteristics: : @ The data tend to be sparse and abstract. e The data tend to be-noisy (i.e., large amounts of incorrect information). I (U) If the descriptor list contains mostly concrete items rather than abstract concepts (e.g., “Is there a waterfall?” versus “Are there vertical features?”), then an analyst who is unwilling or unable to interpret abstract and/or sparse data will miss whatever remote viewing information may be present. In the extreme case, a literal analyst may not answer any questions on the descriptor list affirmatively. If it is assumed that there was some remote viewing information present in the abstract response, then it is clear that the literal analyst will miss it. As the responses become less abstract and possibly more accurate, the difference between an interpretive and literal analyst becomes less important. 16 ‘lt Approved For Release 2000 IN C LAS SdobEBd003800440001-2 | . |
i Hi 4 Approved For Release 2000/08/10 : CIA-RDP96- 00789R003800440001 -2 UNCLASSIFIED © (U) Based upon these concepts, three hypotheses were formulated that could be tested with the three candidate analysts listed above: @ For an analyst to be sensitive to novice data, he/she must be willing to interpret abstract data. ® An interpretive analyst cannot demonstrate a remote viewing effect where there is none. e For literal analysts, the difference between their p-values and those of an interpretive analyst will correlate significantly, on a session—by-session . basis, with their own session p-values. (U) The first hypothesis is true by inspection. Because the only remote viewing output that is analyzed is the one coded into the descriptor list, it follows that an analyst must interpret abstract data, or there are no data for analysis. (U) Given that the analyst must. be interpretive, we must consider whether an artifact could be induced by being interpretive. This is not the case. Because the analyst is “blind” to the correct target for a given session, there is no reason to expect that the interpretation of the abstract response would be selective in such a way ‘as to match the intended target better than any other target in the series. Because the probability assessment of a single session © involves the MCE cross-matched figure of merit distribution, any “enhanced” effects are: canceled by the differential comparison. (U) As a result, it was predicted that the difference between the means of the actual figure of merit and the MCE (AM) would reflect the remote viewing information content, and that the difference would decrease as the analyst tends to be more literal. Table 2 shows the results from four analysts, in assessing the same 45 novice remote viewing sessions. It should be noted that p(AM) is the probability (derived from ANOVA) of observing AM under the MCE hypothesis. Table 2 also shows the probability correlation, [r], described above, its degrees of freedom, [df], its associated probability, [p(r)], the slope, [Slope(r)], of the regression line, and the overall p-value achieved by each analyst for the series of 45 remote viewings. 17 Approved For Release 2049 SHG LA SS PFIE B003800440001-2 =
Tam fool oan] - |-| - | UNCLASSIFIED (U) All three analyst candidates produced highly significant positive correlations between their p-values and the difference between their p-values and 374’s p-value. This indicates that the literal and interpretive analysts will tend to provide more similar p-value estimates as the quality of the data improves. ’ (U) Another indication of the difference between a literal and interpretive analyst is _ AM. Even the most interpretive analyst did not find significance in this data set-——i.e., p(AM) = 0.44. Figure 3 demonstrates the effect on p-value estimates of the same data for a literal and an interpreéfive analyst from a AM perspective. ‘An analyst with a large and positive AM will observe a larger number of significant sessions (i.e., that portion of the curve labeled “Matches” above Mkk—-the critical value of the figure of merit) than an analyst with a small, or negative, value of AM. Thus, an optimal way of selecting analysts is to choose those with larger values for AM and smaller values for Slope(r). 18 Approved For Release sooo IN CLASSIFIER os200440001-2
Approved For Release 2000/08/10 : Ct A-RDF f-00780R00280044000"- 2 Interpretive “Analyst . Cross-matches _ Matches Figure of Merit Literal Analyst |. AM Cross—matches , Matches Figure of Merit FIGURE 3 (U) COMPARISON BETWEEN TYPES OF ANALYSTS B. (U) Response Definition: Descriptor List Formulation 1. (U) Novice Response Descriptor List (U) A post hoc examination of the FY 1986 novice RV transcripts has resulted in a summary list of responses that were considered by the analysts to be the most troublesome to interpret within the highly specific framework of a twenty—bit descriptor list (see Table 1). In the RV training paradigm currently used by novices, interpretation by the viewer is largely discouraged. As a result, concrete words, such as city, lake, tree, or boat, are often labeled as analytical overlay (AOL) and must be discarded from the analysis by definition. Abstract 19 - Approved For Release 20 CLAS SLELED .02800440001-2
(U) descriptors, therefore, would enable analysts to quantify the contents: without having to make the considerable interpretive leap. from th descriptors for incorporation into future lists.* (U) It has yet to be determined how abstract and concrete descriptors will be, | : t structured within a given list--e.g., their interdependence could be either hierarchical in a re gn a RCE nature or configured along the lines of semantic networks. Whatever the mathematical - formalism, it is anticipated that the addition of abstract descriptors will alleviate tnuch of the burden of novice response interpretation for analysts. 2. (U) Advanced Response Descriptor List v (U) An FY 1986 experiment consisting of 12 outbound sessions was performed in which an advanced remote viewer (No. 342) was permitted, in an unsupervised fashion, to create his own descriptor list in advance of the experiment. The viewer was told only that the experiment was to be of the outbound beacon variety using San Francisco Bay Area targets and that his list should consist of approximately 20 to 30 descriptors. He was given the novice RV descriptor list as a template (see Table 1). The hypothesis under test was that the viewer, himself, would be most knowledgeable about his internal perceptions and would ! therefore be most qualified to objectify these perceptions in the form of his own . i “personalized” descriptors. : 4, Sn aE ENNIS * (U) Table 3 is not meant to be an exhaustive list of potential abstract descriptors. It is merely meant to be illustrative by highlighting some of the more commonly encountered novice RV responses. a 20 Approved For Release 2000/8 BUGL ALS: LE LES sc00440001-2
a Approved For Release 2009 eT ROC FIED 20. Table 3 (U) CANDIDATE ABSTRACT DESCRIPTORS FOR NOVICE RESPONSES Typical Novice Responses Suggested Abstract Descriptor Category of Response Actual Responses Patterns-- curved, © circular, rounded Curved, circular, circle, oval, ellipse, round, wavy, rolling, contours, contoured, sloping. Are patterns of round, curved, or circular lines significant at the site? Patterns—- straight, angled Straight, angled, parallel, horizontal lines, verticality, vertical lines, vertical objects, diagonal Are patterns of straight, angled, or parallel lines significant at the site? Are a mixture ‘of curved and straight patterns significant at the site? Patterns—— combined curved and straight No discernible patterns Irregular, shapeless, uneven, — rough, bumpiness, rugged terrain, clusters, irregular blobs, irregular shapes There are no significant patterns at the site. Distinct boundaries between light and dark are significant at the site. Distinct boundaries Areas of light and dark, light and dark contrast or Contrasting areas of light and dark are significant at the site. Unspecified (generic) water Water, wavy, waves, rippling, water movement, water blue Is water a significant part of the scene UNCLASSIFIED (U) Table 4 provides a comparison of target and response codings (assigned by an analyst on a blind basis) for the target matched with its correct response (see Figure 4).* 4 This particular example was chosen because our subjective appraisal tells us that the quality o fthe response does not seem to be reflected in its overall p-value (0.3289). The analyst’s 4 coding of the response and target were evaluated on a post hoc basis and were found to be * (U) The abstract descriptors proposed by the viewer appear to hold promise for codifying some of the information contained in novice responses (see Table 3). ai 7 Approved For Release 2o000Md NCLASSIFIED, .sossooor.2