Approved For Release 2000/08/10 : .CIA-RDP96- -0078 UNCLASSIFIED ABSTRACT (U) (U) A simplified automated procedure is suggested for the analysis of free-response material. As in earlier similar procedures, the target and response materials are coded as yes/no answers to a set of questions (descriptors). By definition, this coding defines the complete target and response information. The accuracy of the response is defined as the percent of the target material that is correctly described (i.e., the number of correct response bits divided by the number of target bits = 1). The reliability of the response is defined as the percent of the response that is correct (i.e., the number of correct response bits divided by the total number of response bits = 1). The figure of merit is the product of the accuracy and reliability. The advantages and weaknesses of the figure of merit are discusseck with examples. Mean chance expectations (MCE) are calculated for the figure of merit, and recommendations are made to extend current techniques and to explore new technologies. — at va il IH . ; li Approved For Release 200QNIC Ic, § §1F4ED003800440001-2 /
: pproved For Release OINCLASSIFIE 9R003800440001-2 TABLE OF CONTENTS (VU) bee e eee eeee sete eee eee eee eee eee tee eee eee teense ereseseereeeceee Ti LIST OF ILLUSTRATIONS ...........6: Seen eee eee eee eee eee nese nee eees iv ‘LIST OF TABLES 2... ccc ccc cece ee cee eee eee een eee eee n ener eeeees Seve eeeeees v INTRODUCTION «2... -0e eee eee eees enn eee eee enn ene teeta 1 II METHOD OF APPROACH .... ccc eee cece een e nee e nrc eee eeeneenennnees 3 A. Figure of Merit Analysis ......... eee e cece eneeeeeeaees Cece eens 3 1. Overview ........... beeeee cece ce eee eee e eee eeeeeneeennnenees 3 2. Mathematical Formalism ........0ccccceeeeeece nec esereneeees 3 a. Definitions 0.0.0... ccc cee ccc cece eset teen nee eenedeenaees 3 b. Linear Least-Squares Analysis ......0 cs ec scecer eee enenrees 6 c. Mean Chance Expectation ....... ccc cece eect ere e rete eens 7 d. Probability Assessment and Analysis ..........0ece eee eerees 8 B. Fuzzy Set Theory--An Enhancement Technique for Descriptor List Technology ..... ccc ccc e rere eee eee cee etn e ence eee ete ea ees 11 ~ A. Overview co ccc ccc ce cee er cece een eee e erie eee eneeeee 11 2 A Tutorial oo... ccc ccc eee renee eee eens eee eeee 12 3. Initial Application to RV Evaluation .......0 ccs c ener cence eens 14 4. Potential Future Applications ............e cece eee eee e eee e eens 14 % II RESULTS woo cece eee eee eee eee ete e teeta nent eee e eee? 16 A.  Inter-Analyst Reliability Factors ....... 0. cece ence eee eee een eenaes 16 B. Response Definition: Descriptor List Formulation ......... beeen nee 19 1. Novice Response Descriptor List ........ 00: sce e eee e eee eee eene 19 2. Advanced Response Descriptor List 1.1.0... . cee cere eect neces 20 C. Target Definition: Implications for Target Pool Composition ........... 25 IV. RECOMMENDATIONS ...........04: ved e ee eee eee e eee eenaeeeeeee es 27 A. Similarity Experiment 0... .. 00: e cece ee recente ten eeees 27 B. AT Techniques 0.0... . cece ee ccc tcc eee eter tee teen eee ee nee 28 C. In-House Effort 2.0... cece eee eee eee eee tenes 29 Vv CONCLUSIONS 2... cee ccc cece eee renee een een eee sent aeees 31 REFERENCES 00.00. e cece cece renee nee nee een ened eden eee enone een E Ene 32 iii CLASSIFIE Approved For Release 200 UN : CIA-RDP96-00789R003800440001-2
Approved For Release 2000/08/10 : CIA-RDP96-00789R003800440001-2 UNCLASSIFIED / LIST OF ILLUSTRATIONS (VU) An MCE Figure of Merit Distribution 2.0... 0c cece cetera reece eee nnteees 9 The Fuzzy Set “Very Young” ...... cece cece erence cere renee anne een eneeoeens 13 Comparison Between Types of Analysts 0.0... 6c eee eee ee cee e nee e ene e enn eeenes 19 Baylands Nature Interpretive Center, With RV Response .........-eseeeeeeceeees 24 < \ 3 iv Approved For Release 200WINCLASSIBVEB003800440001-2 \
Approved For. Release 2000/08/10 : CIA- RDP96- 00789R003800440001 -2 UNCLASSIFIED LIST OF TABLES (U) 1. Descriptor~Bit Definition .......... Sect e nee e en eees Ledeen scene seeeceeneses 4 2. Viewer No. 454 Results Coded by Four Analysts ..... cee eeee de cueeueeuesaceens 18 3. Candidate Abstract Descriptors for Novice Responses .......- see ece cere eeeeenaes “21 4. Comparison of Target vs. Response Coding for “Baylands” Target .......... seoeee 22 5. Potential Problem Areas for Novice Targets ....... cece eceee rec teueeneveneenes 26 ¥ ‘ ~ Approved For Release 2000 NCLASSIBIER03800440001-2
Approved For Release 2000/08/10 : CIA-RDP96-00789R003800440001-2 —-— I INTRODUCTION (VU) ' Since the publication of the initial remote viewing (RV) effort at SRI International*', two basic questions have remained in evaluating remote viewing data: @® What is the definition of the target site? @ What is the definition of the RV response? In the development of meaningful evaluation procedures, we must address these two questions, whether the RV task is a research-oriented one (in which the target pool is known), or ‘mission (in which the target may not be known). (U) In the older, [EEE-style, outbound experiment, definitions of target and response were particularly difficult to achieve. The protocol for such an experiment dictated that an experimenter travel to some randomly chosen location at a prearranged time; a viewer's task was to describe that location. In trying to assess the quality of the RV descriptions (in a series of trials, for example), an analyst visited each of the sites and attempted to match responses to them. While standing at a site, the analyst had to determine not only the bounds of the site, but also the site details that were to be included in the analysis. To cite a specific example using this protocol: if the analyst were to stand in the middle of the Golden Gate Bridge, he/she would have to determine whether the buildings of downtown San Francisco, which are clearly and prominently visible, were to be considered part of the Golden Gate Bridge target. The RV response to the Golden Gate Bridge target could be equally troublesome, because responses of this sort were typically 15 pages of dream-like free associations. A reasonable description of the bridge might be contained in the response—-it might be obfuscated, however, by a large amount of unrelated material. How was an analyst to approach this problem of response definition? (U) The first attempt at quantitatively defining an RV response involved reducing the raw transcript to a series of declarative statements called concepts.? Initially, it was “(U) References are listed in order of appearance at the end of this report. Approved For Release 2000/08/10 : CIA-RDP96-00789R003800440001-2 f ~/
Approved For Release 2000/08/10 : CIA-RDP96-00789R003800440001-2 © (U) c determined that a coherent concept should not be reduced to its component palts. For example, a small red VW car would be considered a single concept rather than four separate concepts, small, red, VW, and car. Once a transcript had been “conceptualized,” the list of concepts constituted, by definition, the RV response. The analyst rated the concept lists against the sites. Although this represented a major advance over previous methods, no attempt was made to define the target site. During an FY 1982 program, a procedure was developed to define 4 — both the target and response material.? It became evident that before a site can be quantified, the overall remote viewing goal must be clearly defined. If the goal is simply to demonstrate the existence of the RV phenomena, then anything that is perceived at the site is important. But if the goal is to gain information that is useful » then specific items at the site are important while others remain insignificant. For example, let us assume that an office is a hypothetical target and that a single computer in that office is of specific interest. Let us also assume, hypothetically, that a viewer gives an accurate description of the shape of the office, provides the serial number of the typewriter, and gives a complete description of the owner of the office. Although this kind of a response might provide excellent evidence for remote viewing, the target of interest (the computer) is completely missed--this response, therefore, is of no interestt -What is needed is a specific technique to allow assessments that are mission-oriented. 4 ~ | This report describes a computerized RV evaluation procedure that was initially developed in FY 19844 and has been expanded and refined in FY 1986.* In its current evolution, it is an analysis that has been aimed primarily at simpler, research-oriented tasks using a known target pool. It is anticipated, however, that future refinements to existing procedures, in addition to the advances of proposed new technologies, will allow evaluation techniques to begin to address the more complex issue of a \ ; , RV ycollection. *(U) This report constitutes Objective A, Task 4, “Remote Viewing Evaluation Techniques.” 2 Approved For Release 2000/08/10 : CIA-RDP96-00789R003800440001-2
_ Approved For Release 2000/08/10 : CIA- RDP96- 00789R00 800440004 “ UNCLASSIFIED. Il METHOD OF APPROACH (U) A. (U) Figure of Merit Analysis 1. . (U) Overview (U) Current approaches in evaluation technology have focused on the refinement and extension of the figure of merit analysis. Defined in general terms, this procedure generates a figure of merit (M) between 0 and 1, which provides an accurate assessment of an RV response. The M is the product of the accuracy and reliability with which an RV response describes its correct target, as determined by an analyst’s coding of RV targets and responses according to a “descriptor list.” Table 1 provides a representative example of such a list, which was used in an FY 1986 novice RV training program. Each of the items in a descriptor list requires a binary decision from the analyst as to the item’s presence or absence in each of the targets and responses. The mathematical formalism for converting the analyst’s binary codes into Ms and their controls is detailed in Section A.2. below. 2. (U) Mathematical Formalism { a. (U) Definitions (U) For a single viewer, the overall method of analysis consists of calculating a figure of merit, M, for each viewing session, and then comparing these Ms to a control set of figures of merit. 3 Approved For Release oo0dJdNCLASSIGHEDo03800440001-2
fap (U) DESCRIPTOR-BIT DEFINITION Descriptor Is any significant part of the scene hectic, chaotic, congested, orcluttered? © Does a single major object or structure dominate the scerie? Is the central focus or predominant ambience of the scene primarily natural rather than artificial or manmade? ‘ Do the effects of the weather appear to be a significant part of the scene? (e.g., as in the presence of snow or ice, evidence of erosion, etc.) Is the scene predominantly colorful, characterized by a profusion of color, by a strikingly contrasting combination of colors, or by outstanding, brightly- colored objects (e.g., flowers, stained-glass windows, etc.--not normally. blue sky, green grass, or usual building color)? Is a mountain, hill, or cliff, or a range of mountains, hills, or cliffs a significant feature of the scene? wf 2 Is a volcano a significant part of the scene? Are buildings or other manmade structures a significant part of the scene? Is a city a significant part of the scene? Is a town, village, or isolated settlement or outpost a significant feature of the scene Are ruins a significant part of the scene? Is a large expanse of water——specifically an ocean, sea, gulf, lake, or bay-—a significant aspect of the scene * : Is a land/water interface a significant part of the scene? Is a river, canal, or channel a significant part of the scene? Is a waterfall a significant part of the scene? Is a port or harbor a significant part of the scene? Is an island a significant part of the scene? Is a swamp, jungle, marsh, or verdant of heavy foliage a significant part of the scene Is a flat aspect to the landscape a significant part of the scene? Is a desert a significant part of the scene, or is the scene predominately dry to the point of being arid? UNCLASSIFIED 4 “Approved For Release ood ClASSd BIER soss00440001-2
Approved For Release 2000/08/10: CIA-RDP96- 00789R003800440001-2 UNCLASSIFIED (U) Let [n] be the number of sessions - to be ee Suppose also that the descriptor list contains [m] bits. We then define the total number of bits in a specific oo response, [k], as . . D> R= R k jel ik 7) where R,, = 1 if bit [j] were answered affirmatively and equal to 0 otherwise. Likewise jk the total number of bits in a specific target, [kK], as where T, ‘= 1 if bit [j] were answered affirmatively and equal to 0 otherwise. (U) The accuracy of response, [k], (the percent of target [i] that is described correctly) is given by y (U) Finally, the figure of merit for response [k] matched against target [i] is given by w= Fg XK Pag (U) The analysis can be considered from two perspectives: matches—-i.e., the figure of merit is calculated by matching a response against its intended target (i.e., k = i), and cross—matches-—-i.e., the figure of merit is calculated by matching a response against some target other than its intended one (i.e., k 5 i). 5 Approved For Release 2000kbIN CHA SSA EER oss00s40001-2
Approved For Release 2000/08/10 : ‘CIA- RDP9 003800440001 -2: _ UNCLASSIFIED - b.  (U) Linear Least-Squares Analysis (U) After {n] remote viewing sessions have been completed and the analysis described above performed using k = i, there are [n] figures of merit, one for each RV session, in order of session number. To examine if there are any systematic variations within this data, a best-fit straight line is fitted through the figures of merit using standard techniques. If [x] is the session number (x=1, ses, n), consider a straight line defined as —M@).= a+b (x-x) where a = M(x) and x = constant, and [a] and [b] are the intercept and slope, respectively. Suppose there are [n] pairs of points, (x,M,). Then the slope, which is calculated by a standard least-squares technique, is given by 2 DxM, - Sx 3M, b = et et et : whereA is given by. n | rn 2 A= ion > x’ - ea . , x=1 x=1 The intercept is given by ¢ = ‘, e as a+ bx,. where a is given by n x 2x M, =1 Me “n nn ax DM, - = x= ~ * ce —_ * If we set x = to the average value of the session number, then’ M*? x=+>x = 4 ; 1 it and [a] becomes the average value of the figure of merit. Thus Approved For Release 2000 KIN CA SSI REED 03800420001-2