~ Approved For Release ~ anil einem -9 -_ ; The Qppivcation analyst, as opposed to an RV analyst, should construct such _ a list for each mission. While there may be considerable similarities between element lists for different missions, undoubtedly the lists will require specialization. In Section II~C below, we show the construction of one element list and how it can be applied to a set of 65; _ [ L targets. - 2. (U) Analysis of Complete Responses | Once an appropriate universal set of elements has been created, and fuzzy sets that define the target and the response have been specified, the comparison between them is straightforward. We have defined accuracy as the percent of the target material that is described correctly by a response. Likewise, we have defined reliability (of the viewer) as the percent of the response that is correct.6 Although in the laboratory it is required to provide a posterior probability estimates of the target-response match, in an aglicat'ms setting, this may be less important. All that is usually necessary is to describe the accuracy and reliability for complete responses, and for individual target elements of interest. These quantities for the jth sessions are » WelRy 1) Tie k= rps 7A (1) > WLR) x Ke=1 and Aa > WR OY THe qj =f > WeTj.t k=1 (2) where the sum over & is called the sigma count in fuzzy-set terminology, and is defined as the sum of the membership values (1) for the elements of the response, the target, or their intersection, and n is the number of possible elements as defined by the element list. A fuzzy intersection is defined as the minimum of the intersecting fuzzy set membership values. In this version of the definitions, we have allowed for the possibility of weighting the membership values, Ws, to =} 7 nl i i provide mission-defined relevances. (U) For the above calculation to be meaningful, the membership values for the targets must be similar in kind to those for the responses. For most mission-dependent specifications, this is generally not the case. The target membership values represent the degree to which a particular element is characteristic of the target, and the response membership values represent the degree to which the analyst is convinced that the given element is represented in the response. 3 Approved For Release ovo /oarhe: clA-RIpb9s-0078eR002200530001-6
EBEREEEEE EE HE Approved For Release 2000/038N6L ASSIRRTS 6-00789R002200530001-9 (U) Until RV abilities can encompass the recognition of elements as well as their degree of target characterization, we are required to modify the target fuzzy set. An analyst must decide upon a threshold above which an element is considered to be completely characteristic of the target site. In fuzzy set theory, this is called an a-cut: a technique to apply a threshold to the pL values such that if the original value exceeds it, reassign the value to 1, otherwise set it to 0. In this way, the analyst’s subjectivity can be encoded in the response fuzzy set, and Equations 1 and 2 remain valid. 3. (U) Analysis of an Individual Element (U) Equations 1 and 2 can be simplified to provide an accuracy and reliability on an individual element basis instead for a complete response. For example, let N be the number of sessions against different targets that exist in a current archive for a specified viewer. Let € be an element in question (e.g., airport). Then the empirical probability that element ¢ is in the target, given that the viewer said it was, is given by N R@©= =, (3) N, af where Nc is the number of times that the individual was correct, and Nr is the number of times that element € was mentioned in the response. R(e) is also the reliability of the viewer for that specified element. (U) To compute what chance guessing would be, we must know the occurrence rate of element ¢ in the N sessions. Let No be the actual number of times element € was contained in the N targets. Then the chance—guessing empirical probability is given by t R Ni Ro(€) can also be considered as the guessing reliability (i.e., the reliability that would be observed if the viewer guessed e during every session). The more R(e) > Ro(e), the more reliable the individual is for the specified element. , (U) The empirical probability that the viewer said element e, given that it was in the target, is given by A(e) is also the accuracy of the viewer for that specified element. (U) As a numerical example, suppose a single viewer participated in N = 25 sessions. Let « = “airport.” Further suppose that No = 5 of the targets actually contained an airport. Approved For Release 2000 Gs ASB Ro6-00789R002200530001-9
Approved For Release 2000/0 : CIA- Rif96-00780R002200830001-8 (U) Then, Ro(airport) = 0.20 is the chance probability (i.e., guessing airport during every session would only by 20 percent reliable). Assume that the viewer mentioned airport Nr = 6 times and was correct Nc = 4 times. Then this viewer’s reliability for airports is computed as R(airport) = 0.67 > Ro(airport) = 0.20. The viewer’s accuracy for airports is computed as A (airport) = Nc/No = 0.80. Thus in this example, we can conclude that this viewer is reasonably accomplished. at remote viewing an airport. B. (U) Prototype Analysis System ( , We assume that an‘ _ andlyst has constructed a mission-dependent universal set of elements. We further assume ‘that there are a number of competing interpretations of the target site in question. 1 1. (U) Target Templates The first step in our prototype ‘analysis system is to define templates (i.e., general descriptions of classes of target types) of all competing target interpretations from the . universal set of elements. ( i - “Sy Exactly what the templates should represent is entirely dependent upon what kind of information is sought. Both the underlying universal set of elements and the templates must be constructed to be rich enough to allow for the encoding of all the information ~ That is, if neither the set of elements nor the templates can meaningfully represent information about, othe tage+ __\> ~- then it will be unreasonable to consider asking, “(tievant queshms abddt ; ’ the site. Furthermore, a certain amount of atomization is necessary because such division into small units provides the potential for interactions within the universal set of elements. If the profile of a t “facility consists of a single element, the template would be useless unless the response directly: stated that particular element; rather, the profile should be constructed from groups of elemental features LL ( There are two different ways to generate target templates. The most straightforward te technique is also likely to be the most unreliable, because it relies on the analyst’s judgment of a single target type. With this method, the analyst, who is familiar with the intelligence problem at hand, simply generates membership values for elements from the universal set of elements based upon his or her general knowledge. ‘Given the time and resources, the best way to generate template membership values is to encode known targets that are closely related. - Each template pis the average value across targets, and thus is more reliable. If it is known that some targets are more Approved For Release 2000/08/08 : CIA-RDP96-00789R002200530001-9