96-00789R003800450001-1 P ED Approved For Release UNCL rN SSiFl (U) however, that current lists are inadequate in providing discriminators that are “fine” enough both to describe a complex target accurately and to exploit fully the more subtle or abstract information content of the RV response. To decrease the granularity of the RV evaluation system, therefore, it was determined that the technology would have to evolve in the direction of allowing the analyst a gradation of judgment about target and response features, rather than the current “all-or—-nothing” binary determinations. A preliminary survey of various disciplines and their evaluation methods (spanning such diverse fields as artificial intelligence, linguistics, and environmental psychology) has revealed a branch of mathematics, known as “fuzzy set theory,” which provides a mathematical framework for modeling situations that are inherently imprecise. The principal architect of fuzzy sets, L. A. Zadeh, has stated: “One of the aims of the theory of fuzzy sets is the development of a methodology for the formulation and solution of problems which are too complex or ill-defined to be susceptible to analysis by conventional techniques.”® (U) Because the task of RV analysis requires human judgments about imprecise situations—-namely, the categorization of natural sites and the interpretation of abstract representations of those sites—-it would appear, according to the above definition, that fuzzy set theory is a promising line of inquiry. In the next section, some of the basic concepts of fuzzy set theory will be examined, with the aim of understanding how this technology might be applied to the specific problem of RV evaluation. 2. (U) A Tutorial (U) In traditional set theory, an element is either a member of a set or it isn’t-—e.g., the number 2 is a member of the set of even numbers; the number 3 is not. Fuzzy set theory is a variant of traditional set theory, in that it introduces the concept of degree of membership: herein lies the essence of its applicability to the modeling of imprecise systems. For example, if we take the concept of age (known as a linguistic variable in fuzzy set parlance), we might ascribe to it certain subcategories (i.e., fuzzy sets) such as very young, young, middle-aged, old, etc. Looking at very young, only, as a fuzzy set example, we must 12 Approved For Release JEDLGLA SSIEAES2.00789R003800450001-1
Approved For Release UNCLASSIFIED ee UNCL (U) define what we mean by this concept vis-a-vis the linguistic variable age.* If we examine the chronological ages from 1 to 30, we might subjectively assert that we consider the ages 1 through 4 to represent rather robustly a spectrum of the concept very young, whereas the age of 30 probably does not accurately represent very young at all. As depicted in Figure 2, fuzzy set theory allows us to assign a numerical value between 0 and 1 that represents our best subjective estimate as to how much each of the ages 1 though 30 embodies the concept very young. Very young 0.01 0.0 1Oe3ceceteeavee 30 FIGURE 2 (U) THE FUZZY SET “VERY YOUNG” (U) Clearly, a different set of numerical values would be assigned to the ages 1 through 30 for the fuzzy sets young, middle-aged, and old--e.g., the age of 6 might receive a value of 0.5 for very young, but a value of 1.0 for young, depending on context, consensus, and the particular application of the system. In this way we are able to provide manipulatable numerical values for imprecise natural language expressions; in addition, we are no longer forced into making inaccurate binary decisions such as, “Is the age of 7 very young—~yes or no?” oe * (U) It is important to note that the design of the fuzzy application occurs in accordance with the subjectivity of the system designer. Fortunately, the fuzzy set technology is rich enough that it allows for a virtually unrestricted range of expression. Technically speaking, young is the fuzzy set and very isa modifier, but it is beyond the scope of this paper to present terminology in depth. 13 Approved For Release cold NGLARSIEIED, -soR003800450001-1
Approved For Release 2000/08/10 : CIA-RDP96-00789R003800450001-1 UNCLASSIFIED 3, (U) Initial Application to RV Evaluation (U) In FY 1986, work began on an initial application of fuzzy set technology to RV evaluation, which simply entails an extension of the current descriptor list capabilities. In coding the targets, an analyst employs numerical values between 0 and 1, inclusively, to rate each of the 20 descriptors according to the importance of its visual representation in the target. For example, in rating a National Geographic magazine picture of the Khyber Pass, an analyst might ascribe a value of 0.80 for a “mountain” descriptor, a value of 0.20 for a “desert” descriptor, and values for other appropriate descriptors in accordance with their perceived importance to the target as a whole. * (U) The rating of responses is considerably more subjective than the rating of targets. The analyst is required to apply a “confidence rating”-—i.e., again, a value between 0 and 1, inclusively-—as to what degree an abstract ideogram is representative of a given descriptor. For example, if a novice subject draws a conical-shaped object and labels it, “fuzzy cone...wider at the bottom...” the analyst may decide that there is some justification for interpreting this ideogram as a volcano covered with vegetation. Clearly, however, the confidence factor for making this highly subjective determination is quite low; the net result might be, therefore, that the “volcano” descriptor might receive a rating of 0.15, while “foliage” might receive 0.05. (U) We anticipate that the primary effect of implementing this rudimentary application of fuzzy set technology will be to “fine tune” the figure of merit scores, such that they are more representative of the “true” information content of an RV response. The current figure of merit application penalizes certain responses and inflates others (especially given the “noisy” aspect of novice data), based on the correctness or incorrectness of the analyst’s “all-or-nothing” determination with regard to any given descriptor. To summarize, fuzzy set technology is attractive in two important respects: (1) it affords the analyst a wider range of expression, thereby enabling him/her to provide a more realistic portrayal of the information contained in both targets and responses, and (2) it is compatible with the figure of merit mathematical formalism. 4. (U) Potential Future Applications (U) It is anticipated that the initial application of fuzzy set technology to RV responses and targets will greatly enhance the accuracy with which their information content is * (U) Coding of both targets and responses might be more “objectively” arrived at via the consensus of a group of experienced analysts. 14 Approved For Release ALMELA §:8 HED 00789R003800450001-1
(U) depicted. A problem remains, however, with the inherently large granularity of the current descriptor list, which is independent of the potential “fineness” of its application allowed by fuzzy set theory~-although the analyst will be allowed a gradation of response in interpreting an abstract ideogram (i.e., the confidence factors ranging from 0 to 1), he/she will still be constrained to interpreting that ideogram according to 20 concrete descriptors. These descriptors are significantly limited in their ability both to portray rich environments and to distill the most usable information from abstract RV mentations. (U) It is projected that future descriptor lists will afford the analyst greater latitude in interpreting the more abstract aspects of RV responses, by providing basis vector descriptors. Such descriptors would represent, in essence, the lowest practicable common denominator of abstraction from which more concrete descriptors might be generated using fuzzy set operations (such as intersection and union). An example of a basis vector descriptor might be the concept of vertical, which is an abstraction that is represented to varying degrees in such concrete descriptors as building, cliff, mountain, waterfall, etc. (U) Ultimately, we envision that evaluation would proceed along the lines of analyzing both the RV responses and targets in terms of fuzzy—weighted basis vector descriptors. A comparison of basis vector descriptors between responses and targets could then be effected, which would culminate in a figure of merit analysis reflecting the subject’s ability to debrief the more abstract components of the psi signal. By using fuzzy set operations, concrete target and response descriptors could subsequently be generated on a “best fit” basis from the basis vector descriptors, and a figure of merit evaluation could be performed at this higher-order level also. The primary benefits of this type of procedure would be in providing objectification of abstract response data, and in affording more automated interpretation of these data in concrete terms. Furthermore, it would also be possible to track, in a systematic and quantifiable manner (on both a “subject—by-subject” and “across subject” basis), the kinds of abstract signals that subjects are receiving reliably; presumably, this capability might then be used to illuminate important lines of future investigation within RV fundamentals. 15 Approved For Release YN CL ASAT EIED o07s9R003800480001-1
Approved For Release 2000/08/10 : CIA-RDP96-00789R003800450001-1 UNCLASSIFIED III RESULTS (UV) (U) The results of the FY 1986 evaluation effort have been obtained primarily from two sources: (1) identification of inter-analyst reliability factors, based on analysis of figure of merit statistics, and (2) insights into descriptor list formulations and target pool composition, based on post hoc analysis and observations. Each of these areas is explored in turn below. A. (U) Inter-Analyst Reliability Factors (U) A method was developed in FY 1986 for rating the abilities of potential remote viewing analysts. The most direct method of accomplishing this was simply to ask a candidate to analyze a known series of remote viewings; the results could then be compared to those produced by a proven analyst, 374. (U) Three individuals, 432, 579, and 642, were asked separately to score first the targets and then the responses used in a remote viewing series from a novice viewer. They used a twenty-bit descriptor list (see Table 1) under a “blind” protocol. The procedure described in Section II.A. was used to calculate figures of merit, session p-values, and overall p-values for each analyst. (U) Novice remote viewing data, which have been collected under our stimulus/response protocol, contains two distinguishing characteristics: @ The data tend to be sparse and abstract. @ The data tend to be noisy (i.e., large amounts of incorrect information). (U) If the descriptor list contains mostly concrete items rather than abstract concepts (e.g., “Is there a waterfall?” versus “Are there vertical features?”), then an analyst who is unwilling or unable to interpret abstract and/or sparse data will miss whatever remote viewing information may be present. In the extreme case, a literal analyst may not answer any questions on the descriptor list affirmatively. If it is assumed that there was some remote viewing information present in the abstract response, then it is clear that the literal analyst will miss it. As the responses become less abstract and possibly more accurate, the difference between an interpretive and literal analyst becomes less important. 16 Approved For Release SWE LA 8:2 FED 00789R003800450001-1
e Approved For Release 2000/08/10 : CIA-RDP96-00789R003800450001-1 UNCLASSIFIED (U) Based upon these concepts, three hypotheses were formulated that could be tested with the three candidate analysts listed above: e® For an analyst to be sensitive to novice data, he/she must be willing to interpret abstract data. e An interpretive analyst cannot demonstrate a remote viewing effect where there is none. e® For literal analysts, the difference between their p-values and those of an interpretive analyst will correlate significantly, on a session—by—session basis, with their own session p-values. (U) The first hypothesis is true by inspection. Because the only remote viewing output that is analyzed is the one coded into the descriptor list, it follows that an analyst must interpret abstract data, or there are no data for analysis. (U) Given that the analyst must be interpretive, we must consider whether an artifact could be induced by being interpretive. This is not the case. Because the analyst is “blind” to the correct target for a given session, there is no reason to expect that the interpretation of the abstract response would be selective in such a way as to match the intended target better than any other target in the series. Because the probability assessment of a single session involves the MCE cross-matched figure of merit distribution, any “enhanced” effects are canceled by the differential comparison. (U) As a result, it was predicted that the difference between the means of the actual figure of merit and the MCE (AM) would reflect the remote viewing information content, and that the difference would decrease as the analyst tends to be more literal. Table 2 shows the results from four analysts in assessing the same 45 novice remote viewing sessions. Tt should be noted that p(AM) is the probability (derived from ANOVA) of observing AM under the MCE hypothesis. Table 2 also shows the probability correlation, [r], described above, its degrees of freedom, [df], its associated probability, [p(r)], the slope, [Slope(r)], of the regression line, and the overall p-value achieved by each analyst for the series of 45 remote viewings. 17 Approved For Release 2ddINSab AS SHEE} 0789R003800450001-1
oo Approved For Release 2000/08/10 : ej -00789R003800450001-1 UNCLASSIFIEI Table 2 (U) VIEWER NO. 454 RESULTS CODED BY FOUR ANALYSTS Overall Analyst (j)| AM | p(AM) (p,- Papa) VS. P, df | p(r) Slope(r) Series p a RS Ta eee Se ST SRE veofowr] [=] |_| eae 0.004 | 0.881 0.421 0.0040} 0.326 0.702 0.007 | 0.768 0.555 0.0001) 0.509 0.867 0.012 | 0.573 0.552 0.0001} 0.558 0.909 UNCLASSIFIED (U) All three analyst candidates produced highly significant positive correlations between their p-values and the difference between their p-values and 374’s p-value. This indicates that the literal and interpretive analysts will tend to provide more similar p-value estimates as the quality of the data improves. (U) Another indication of the difference between a literal and interpretive analyst is AM. Even the most interpretive analyst did not find significance in this data set--i.e., p(A4M) = 0.44. Figure 3 demonstrates the effect on p-value estimates of the same data for a literal and an interpretive analyst from a AM perspective. An analyst with a large and positive AM will observe a larger number of significant sessions (i.e., that portion of the curve labeled “Matches” above Mkx--the critical value of the figure of merit) than an analyst with a small, or negative, value of AM. Thus, an optimal way of selecting analysts is to choose those with larger values for AM and smaller values for Slope(r). 18 Approved For Release AGHA S 2 EP o0789R003800450001-1
w Approved For Release 2000/08/10 : CIA-RDP96-00789R003800450001-1 UNCLASSIFIED Interpretive Analyst AM Cross—matches Matches Figure of Merit Literal Analyst AM Cross—matches Matches Figure of Merit FIGURE 3 (U) COMPARISON BETWEEN TYPES OF ANALYSTS B. (U) Response Definition: Descriptor List Formulation 1, (U) Novice Response Descriptor List (U) A post hoc examination of the FY 1986 novice RV transcripts has resulted in a summary list of responses that were considered by the analysts to be the most troublesome to interpret within the highly specific framework of a twenty-bit descriptor list (see Table 1). In the RV training paradigm currently used by novices, interpretation by the viewer is largely discouraged. As a result, concrete words, such as city, lake, tree, or boat, are often labeled as analytical overlay (AOL) and must be discarded from the analysis by definition. Abstract 19 Approved For Release JIN GLA Sa LEED 00789R003800450001-1
Approved For Release 2000/08/10 : CIA-RDP96-00789R003800450001-1 UNCLASSIFIED (U) (and less interpretive) descriptions, such as rounded, vertical, and wavy, however, are typically encouraged and are commonly found in transcripts. The addition of abstract descriptors, therefore, would enable analysts to quantify the contents of a transcript literally, without having to make the considerable interpretive leap from nebulous responses to highly specific, concrete descriptors. Table 3 summarizes some of the more common abstract descriptions gleaned from novice responses, and provides suggestions for candidate abstract descriptors for incorporation into future lists. * (U) It has yet to be determined how abstract and concrete descriptors will be structured within a given list~-e.g., their interdependence could be either hierarchical in nature or configured along the lines of semantic networks. Whatever the mathematical formalism, it is anticipated that the addition of abstract descriptors will alleviate much of the burden of novice response interpretation for analysts. 2. (U) Advanced Response Descriptor List (U) An FY 1986 experiment consisting of 12 outbound sessions was performed in which an advanced remote viewer (No. 342) was permitted, in an unsupervised fashion, to create his own descriptor list in advance of the experiment. The viewer was told only that the experiment was to be of the outbound beacon variety using San Francisco Bay Area targets and that his list should consist of approximately 20 to 30 descriptors. He was given the novice RV descriptor list as a template (see Table 1). The hypothesis under test was that the viewer, himself, would be most knowledgeable about his internal perceptions and would therefore be most qualified to objectify these perceptions in the form of his own “personalized” descriptors. * (U) Table 3 is not meant to be an exhaustive list of potential abstract descriptors. It is merely meant to be illustrative by highlighting some of the more commonly encountered novice RV responses. 20 Approved For Release aM INGASSEELED) 0789R003800450001-1
Approved For Release UNCLASSIFIED ee onss000 -1 Table 3 (U) CANDIDATE ABSTRACT DESCRIPTORS FOR NOVICE RESPONSES Typical Novice Responses Suggested Abstract Descriptor Category of Response Actual Responses «FF --"«20-06-E°2—-22020Ee eee re Patterns~- curved, Curved, circular, circle, oval, circular, ellipse, round, wavy, rolling, rounded contours, contoured, sloping Are patterns of round, curved, or circular lines significant at the site? Patterns—— straight, Straight, angled, parallel, angled horizontal lines, verticality, vertical lines, vertical objects, diagonal Are patterns of straight, angled, or parallel lines significant at the site? Patterns—- combined curved and straight Are a mixture of curved and straight patterns significant at the site? rough, bumpiness, rugged terrain, clusters, irregular blobs, irregular shapes patterns at the site. Distinct boundaries Areas of light and dark, light and dark contrast Distinct boundaries between light and dark are significant at the site. or Contrasting areas of light and dark are significant at the site. Unspecified (generic) | Water, wavy, waves, rippling, Is water a significant part water water movement, water blue of the scene UNCLASSIFIED No discernible patterns} Irregular, shapeless, uneven, | There are no significant (U) Table 4 provides a comparison of target and response codings (assigned by an analyst on a blind basis) for the target matched with its correct response (see Figure 4).* This particular example was chosen because our subjective appraisal tells us that the quality o {the response does not seem to be reflected in its overall p-value (0.3289). The analyst’s coding of the response and target were evaluated on a post hoc basis and were found to be * (U) The abstract descriptors proposed by the viewer appear to hold promise for codifying some of the information contained in novice responses (see Table 3). ai ASSIF D Approved For Release NG : CIA-R ED l P96-00789R003800450001 -1
Approved For Release 2000/08/10 : CIA-RDP96-00789R003800450001-1 UNCLASSIFIED Table 4 (U) COMPARISON OF TARGET VS. RESPONSE CODING FOR “BAYLANDS” TARGET Descriptor 0 1 1 1 0 1 There are no significant patterns. 1 0 1 Are patterns of straight, parallel, or angled lines significant at the site? Are patterns of round, curved, or circular lines significant at the site? 2 3 0 4 0 Are a mixture of round and straight patterns significant at the site? 5 0 Is a significant part of the scene hectic, chaotic, congested, or cluttered? 6 1 Is a significant part of the scene clean, empty, or open? 7 0 0 Is a significant part of the scene inside? 8 1 1 Is a significant part of the scene outside? ) 1 1 Is water a significant part of the scene? Is sculptured water a significant part of the scene (fountains, etc.)? Is natural water a-significant part of the scene (lakes, ponds, streams, etc.)? Are buildings or other manmade structures a significant part of the scene? Is a single structure a significant part of the scene? Is/are functional (useful, moving parts, etc.) structure(s) at the site? Is/are artistic (there to look at) structure(s) at the site? Is a single color predominant at the scene? Is foliage a significant part of the scene? Is foliage natural in appearance at the scene? Is foliage significantly sculpted, manicured, or pruned at the scene? Is the scene predominantly void of foliage? Is motion significantly important at the site? Is ambient noise significant at the site? Is noise generated by the target? Is noise generated by people adjacent to the target? *1= yes, 0 =no. UNCLASSIFIED 22 Approved For Release 20M NGLASSIPLED, -soR003800450001-1
Approved For Release 2000/08/1 (U) reasonable~~i.e., only one or two bit assignments were arguable, and their reassignment would not have resulted in a significant p~value, which this session seemed to merit. (U) With analyst error seemingly eliminated, attention was focused on the efficacy of the descriptor list itself. It was concluded that the list was deficient in its ability to capture certain kinds of information, which largely accounted for the perceived accuracy of the response, including: The juxtaposition of elements (i.e., spatial relationships) The “novelty factor” of certain elements in the response The high specificity of named (or alluded to) target elements. (U) No mechanism exists, as yet, within descriptor list technology for capturing the information contained in the spatial relationships between elements. Clearly, as in the example cited in Figure 4, this type of information can be very significant—--i.e., the viewer drew his response from the beacon’s actual perspective on the target. Spatial relationships appear to be particularly significant in advanced RV responses, in which complex, composite drawings are much more prevalent than in novice responses. This type of information may eventually be accounted for by employing new technologies such as rule—based expert systems (see Section IV.B.), which lend themselves well to recognition of juxtaposed elements. (U) Another factor that is often thought to be important both in novice and advanced RV responses is——for want of a better term—-the novelty or strangeness of an element in a response. An example of this in Figure 4 might include the odd shape of the structure’s roof——i.e., a curved roof is a slight departure from normal expectation. The operative information here is embodied in the idea of “architectural oddity,” a concept that is quite central to the target and is higher in information content than what is expressed by the various combinations of pattern and structure descriptors alone. Another example is the viewer’s statement, “...like a fence present but not a fence...,” which is a somewhat odd and uncertain phrase, but actually describes the catwalk guardrails in the target quite well. An experienced analyst might consider this latter type of information to be qualitatively better, because it represents a viewer’s attempt to objectify his perception without succumbing to the pitfalls of analytical overlay (AOL). Analyst observations about response novelty can be systematically tested by devising an element—by-element analysis that can be applied across a wide qualitative range of responses. If the analyst “lore” is correct, it might be captured either by a “novelty” fuzzy weighting factor applied to descriptor lists or by expert systems capabilities. 23 Approved For Release MINGLA SS LEED 0 07s9R003800450001-1
Approved For Release 2000/08/10 : CIA-RDP96-00789R003800450001-1 UNCLASSIFIED per sor ID No, 342 \ wD oareers ' sine’ - Like LooKiny som Ar SomeTHIAG TVA RAS AS Ir oes Alay PRO eee you _— feasreennve : ‘ yet ok oe . — ATU LI APR ope KiAr- WATER Oe. Ae) ree ££ Locarea ar BOGE 0 WATER. " ED & . Che) sects UNCLASSIFIED FIGURE 4 (U) BAYLANDS NATURE INTERPRETIVE CENTER, WITH RV RESPONSE 24 Approved For Release SN EAS EER oo7seR003800450001-1
Approved For Release 2000/08/10 : CIA-RDP96-00789R003800450001-1 UNCLASSIFIED (U) The final factor that led to forfeiture of information entails the degree of specificity of the descriptor list. A highly detailed response of advanced RV quality will suffer if the descriptor list ‘quantifying it cannot register detail. In the Figure 4 example, pertinent (and possibly unique) pieces of data--like poles being used in the structure’s construction-—are relegated to relatively nonunique bit categories such as “patterns of straight, oi parallel, or angled lines...” Additional highly specific, concrete descriptors (e.g., poles, pathways, etc.), therefore, are essential for descriptor lists quantifying high-quality RV data. (U) The question of whether a viewer is better able to devise his own descriptor list remains unanswered. Subjectively, it is felt that analyst-derived lists have tended to be more concrete in nature and that the advanced series described here would have benefited from that kind of emphasis. There is also the additional fact that the viewer was not aware of the logical consistency rules governing descriptor formulation and application, and that an analyst’s awareness of these procedural mechanics would have been beneficial to the construction of the list. The viewer’s insights into abstract descriptor composition, however, were quite invaluable and hold important implications for novice list construction in particular. C. (U) Target Definition: Implications for Target Pool Composition (U) A few preliminary guidelines governing target pool composition have been distilled from two sources: (1) the opinions of RV monitors about the appropriateness of various kinds of RV targets for novice viewers, and (2) RV analysts’ assessments about the difficulties encountered in using the twenty-bit descriptor list to score the 412 targets currently in the novice target pool. (U) As a general rule, the current subjective consensus is that targets are inappropriate for training purposes if they exhibit any of the following qualities: @ They are contrary to the viewer’s expectations. @ They are imbued with negative emotional impact. * @ They violate the “spirit” of the descriptor list’s intended use. *(U) Laboratory anecdotal evidence suggests that targets having negative emotional impact often result in psi-missing responses. 25 Approved For Release MING AS SHEE Doo7s9R003800450001-1
Approved For Release UNCLASSIFIED (U) A wide range of target types was used in the FY 1986 novice RV training series and have been subjectively determined on a post hoc basis to be of varying degrees of appropriateness for this task. Table 5 provides specific examples of how the current novice target pool may be problematic, given the following assumptions: (1) novice viewers had anticipated that targets for this series would consist of pictures taken from National Geographic Magazine featuring large, outdoor, gestalt scenes (e.g. cities, mountains, lakes) of roughly the same dimensionality; and (2) the twenty—bit descriptor list was appropriate for coding targets of this type only and nothing else--e.g., use of the list for technical sites or for targets featuring “unnatural” expressions of the bits was inappropriate. (U) While it has yet to be determined empirically (i.e., by systematically examining figures of merit) whether these target types are actually problematic, it is currently the subjective opinion of the evaluation team that these kinds of targets would pose the greatest difficulties for novice viewers. Table 5 (U) POTENTIAL PROBLEM AREAS FOR NOVICE TARGETS Condition Violated Target Type See Viewer Emotional | Intended expectation| impact juse of list Close-up photo of a small feature (e.g., a flower, tree trunk, etc.) ations in a still pool Moon--off-earth photos Standing dead trees (without foliage) Photos with people and/or animals Ornate interior of the Vatican during a ceremony Specific Target Problem Dimensionality Unusual perspective Underwater photos Technical site Significant target element for which there was no descriptor Significant target element for which there was no descriptor Too complex for novices aa UNCLASSIFIED 26 Approved For Release HINGLA SRIF IED 00720R003800450001-1
Approved For Release 2000/08/10 : CIA-RDP96-00789R003800450001-1 UNCLASSIFIED IV RECOMMENDATIONS (VU) (U) The results of the FY 1986 evaluation effort have illuminated several areas of investigation that may hold promise for improving RV evaluation procedures. These areas include (1) identification of new descriptor lists that more accurately reflect target and response information, (2) implementation of enhancement techniques (e.g., fuzzy set theory) for attaining greater accuracy from descriptor lists, (3) systematic examination of inter-analyst reliability factors, and (4) development of new technologies (e.g., expert systems) for capturing analysts’ insights with greater efficiency. Several parallel approaches, which address various aspects of these areas, have been targeted for preliminary research in FY 1987. These include: © A “similarity” experiment (proposed by S. J. P. Spottiswoode) in which an attempt will be made to identify underlying semantic structures in remote viewing descriptions of target materials. @ An approach using artificial intelligence (AI) techniques (proposed by J. Vallee) for recognizing, analyzing, and describing target materials. @ In-house approaches directed at ~ Improvement of existing descriptor lists by incorporating more abstract descriptors into novice lists and more concrete descriptors into advanced lists. - Implementation of fuzzy set mathematical weighting factors into existing descriptor lists in an attempt to decrease their granularity. - Assessment of the relative merits of analyst-derived versus percipient-derived descriptor lists. - Identification of possible percipient-specific “ideogramic dictionaries,” which might serve as prescriptive guides for the RV analyst. - Development of mission-specific descriptor lists (e.g., for technical sites). Each of these approaches is outlined briefly below. A. (U) Similarity Experiment (U) According to the proposal submitted by consultant S. J. P. Spottiswoode, the proposed similarity experiment is aimed at improving existing evaluation techniques through 27 Approved For Release MINGLA S23 HED 00789R003800450001-1
Approved For Release 2000/08/10 : CIA-RDP96-00789R003800450001-1 UNCLASSIFIED (U) “...quantification of the informational content of transcripts on a small set of underlying semantic dimensions which might serve as basis vectors for the viewer’s internal representation of the target. If such basis vectors can be found, complex constructs in the viewing data might be assembled by combining sets of data so expressed.”9 (U) Using well-established techniques from the area of environmental psychology (which have been used to solve analogous problems in “normal” perception), the proposed experiment will attempt to isolate underlying semantic structures in RV perceptions. In essence, percipients will be asked to remote view--in sessions of two targets each——all possible pairs of targets and to estimate the similarity between targets in each pair. The data will be analyzed for important factors such as intersubject reliability, presentation order effect, and target pair effects. Multidimensional scaling (MDS) analysis* will then be applied to identify the underlying semantic dimensions. Identification of semantic structures would hold important implications for descriptor list development and possibly for identification of fundamental commonalities of perception across viewers. B. (U) ATI Techniques (U) According to the following excerpt from the letter proposal submitted by consultant J. Vallee, the proposed AI approach “...will seek to build an expert system for target recognition by analyzing the process that enables a human expert monitor to provide an interpretation of a remote viewing session, or a judge to match a given description with an actual target. It is expected that a rule~based expert system can be developed in a series of iterations starting with the simple “twenty questions” framework already used in the project. Later this will lead to a fully-developed interactive model. We envision this “smart monitor” taking the analyst from simple scene and “gestalt” recognition to the detection of breaks, contradictions and, possibly, analytical overlays as well.” 10 (U) Assuming promising results with the initial task of target definition, it is anticipated that the expert system will ultimately be expanded to play a more active role in operational remote viewing sessions through on-line capture of respondents’ ideograms and interactive analysis of their features. wo (U) It is beyond the scope of this discussion to describe the details of the MDS analysis. 28 Approved For Release UINGLASSIRBED 10789R003800450001-1
Approved For Release 2000/08/10: CIA 96-00789R003800450001-1 CIA-RDP UNCLASSIFIED On (U) In-House Effort (U) SRI in-house approaches will focus on improvement and refinement of existing technologies. One potential line of inquiry would focus on identifying an appropriate set of abstract descriptors to add to the current novice list. This effort would be aimed at mitigating some of the inter—analyst reliability problems that have been encountered with novice data (see Section III.A.). An attempt would also be made to incorporate additional concrete descriptors into advanced lists. (U) A second approach would endeavor to complete work begun in FY 1986-- namely, a reanalysis of novice data using fuzzy mathematical weighting factors with the current list of twenty descriptors. The hypothesis is that the greater latitude afforded by fuzzy set membership values (as opposed to the “all or nothing” capability of the descriptors in their current configuration) will significantly decrease the granularity of the current list--i.e., will allow capture of more information. If the post hoc reanalysis yields promising results, the fuzzy set approach would be benchmarked on a blind basis against the current binary ~ approach. (U) A third effort would systematically evaluate the efficacy of viewer—derived versus analyst-derived descriptor lists. While viewers are sensitive to their internal perceptions, they are not cognizant of the requirements of the analytical procedures; conversely, the analyst is privy to the linguistic/analytical aspects of descriptor lists, but may be unaware of how to optimize a viewer’s perceptions using descriptors. The hypothesis is that the combined insights of analyst and percipient will synergistically result in the optimal formulation of descriptor lists. One way to test this hypothesis would be to compare statistics across analyst-derived versus percipient-derived versus combined analyst/percipient-derived descriptor lists on the same set of RV data. (U) A fourth approach would endeavor to take a retrospective look at viewers’ ideograms and their possible range of meaning. If, for example, a viewer typically draws a “tic-tac-toe,” cross—hatch-style ideogram, and post hoc analysis reveals that the drawing correctly corresponds to the presence of a city in targets 80 percent of the time, this information might be used to assign an a priori fuzzy weighting factor of 0.8 when the ideogram is encountered on a blind basis. An in-depth examination of a viewer's ideograms, therefore, might result in the development of viewer-specific, prescriptive guides for the assignment of fuzzy weighting factors in assessing RV responses. This type of information could also conceivably be automated and updated through iterative expert system capabilities. 29 Approved For Release UNGLASSIRIER 00789R003800450001-1
Approved For Release 2000/08/10 : CIA-RDP96-00789R003800450001-1 SECRET (S) Finally, work will be initiated to develop mission-specific ( descriptor lists for technical site applications. It is anticipated that efforts along these lines will enable us to better address intelligence community requirements. 30 Approved For Release 2000/00 BER RpP96-00789R003800450001-1
Approved For Release 2000/08/10 : CIA-RDP96-00789R003800450001-1 SECRET V CONCLUSIONS (U) (U) The FY 1986 evaluation effort has resulted in (1) refinement and extension of current techniques, and (2) identification of candidate new technologies for preliminary research. (U) The mathematical formalism for the current evaluation procedure--the figure of merit analysis——is well understood and stable. In addition to the system’s ability to provide a reasonable assessment of remote viewing data, it has also provided a mechanism for systematic examination of inter-analyst reliability factors. (U) The descriptor lists that currently form the basis for the figure of merit analysis have been evaluated on a post hoc basis. Preliminary observations indicate that lists designed for novice responses require greater abstract descriptor capability, whereas lists designed for advanced responses (i.e., higher-quality data) require greater concrete descriptor capability. It is anticipated that fuzzy set technology will assist in formalizing the interdependence between abstract and concrete descriptors, by providing a mathematical framework through which basis vector descriptors can be combined to form concrete descriptors. (U) Research into new technologies for RV evaluation will begin in FY 1987. One of these approaches, the proposed “similarity” experiment, shows promise for identifying basis vector descriptors. A second approach, using rule-based expert systems, will explore a different dimension by endeavoring to capture RV analysts’ expertise in codifying targets. Should this initial effort in artificial intelligence prove successful, it will be expanded to address the more difficult problem of response interpretation. (S) It is hoped that this multifaceted approach to the refinement of RV evaluation procedures will result in increased capabilities for addressing the more complex problems of mission-oriented, operational RV. 31 Approved For Release 2000/6846, IrpP96-00789R003800450001-1
Approved For Release 2000/08/10 : CIA-RDP96-00789R003800450001-1 UNCLASSIFIED REFERENCES (UV) 1. Puthoff, H. E. and Targ, R., “A Perceptual Channel for Information Transfer Over Kilometer Distances: Historical Perspective and Recent Research,” Proceedings of the IEEE, Vol. 64, No. 3 (March 1976) UNCLASSIFIED. 2. Targ, R., Puthoff, H. E., and May, E. C., 1977 Proceedings of the International Conference of Cybernetics and Society, pp. 519-529 UNCLASSIFIED. 3. May, E. C., “A Remote Viewing Evaluation Protocol (U),” Final Report (revised), SRI Project 4028, SRI International, Menlo Park, California (July 1983) SECRET. 4, May, E. C., Humphrey, B. S., and Mathews, C., “A Figure of Merit Analysis for Free-Response Material,” Proceedings of the 28th Annual Convention of the , Parapsychological Association, pp. 343-354, Tufts University, Medford, Massachusetts (August 1985) UNCLASSIFIED. 5. Fisher, R. A., “Statistical Methods for Research Workers,” Oliver & Boyd (7th ed.), London, England (1938) UNCLASSIFIED. 6. Edgington, E. S., “A Normal Curve Method for Combining Probability Values from Independent Experiments,” Journal of Psychology, Vol. 82, pp. 85-89 (1972) UNCLASSIFIED. 7. Cooper, B. E., “Statistics for Experimentalists,” pp. 219-223, Pergamon Press, Oxford, England (1969) UNCLASSIFIED. 8. Zadeh, L. A., “Fuzzy Sets versus Probability,” Proceedings of the IEEE, Vol. 68, No. 3, p. 421 (March 1980) UNCLASSIFIED. 9. Spottiswoode, S. J. P., “Proposals for Experimental Studies on Semantic Structure and Target Probability in Remote Viewing,” Private Communication (July 1986) UNCLASSIFIED. 10. Vallee, J., “Applications of Artificial Intelligence Techniques to Remote Viewing: Building an Expert System for Target Description,” Private Communication (August 1986) UNCLASSIFIED. 32 Approved For Release 2bdBNGLOAG § Holisb)o0789R003800450001-1
ee te tere ee nseettaeae Approved For Release 2000/08/10 : CIA-RDP96-00789R003800450001-1 SECRET WARNING NOTICE RESTRICTED DISSEMINATION TO THOSE WITH VERIFIED ACCESS SG1A NOT RELEASABLE TO FOREIGN NATIONALS Approved For Release 2000/SeHe R&T Rpp96-00789R003800450001 -1