Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 ¢ a Chapter Three: Research Reviews of remote viewing. Another issue where there seems to be some remaining disagreement is a variant of the causation problem. Methodological problems are, of course, not the only reason a statistically significant effect might occur but still fail to provide evidence for the existence of a phenomenon. It is quite possible that an effect occurred for reasons other than the proposed cause. In other words, normal psychological processes rather than paranormal processes might account for the effects obtained in the remote viewing studies. Recognition of this fact led both reviewers to stress the point that more attention needs to be given to the identification of underlying causes in remote viewing research. The reviewers however, differ somewhat in the way they would approach this issue. Dr. Utts believes that the existence of the phenomenon has been adequately demonstrated; therefore, the focus of future research should be on causative mechanisms — how the ability works, Dr. Hyman takes the position that competing hypotheses have not yet been eliminated; thus, he stresses the continued need for replication in different laboratories, with different investigators, etc., so that causal mechanisms, specifically paranormal mechanisms, can be identified. Our conclusion about this issue is direct: If laboratory research is to continue, this research must be conducted in such a way that causal mechanisms can be articulated, and that also serves to rule out competing explanations. A final point of disagreement has to do with the nature of the remote viewing phenomenon itself. Dr. Hyman argues that the effect has not been readily replicated in other laboratory settings. Instead, it appears inconsistently, occurring only for certain people at certain times. He argues that the size of the observed effects are small, they do not consistently emerge under certain specified conditions, and that a coherent pattern of results has not been obtained in studies of remote viewing. At the risk of misrepresenting his position, the implication is that unless more is uncovered about the phenomenon, it is not — likely to be a fruitful area for further research, Dr. Utts counters with a compelling argument, noting that many effects are statistical in nature, occurring only rarely and under certain conditions. This should not preclude further research into the causes and operation of the phenomenon. American Institutes for Research 3-80 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 Chapter Three: Research Reviews We agree with Dr. Utts that the conditions under which a phenomenon occurs should not determine whether or not it is worth exploring and pursuing scientifically. On the other hand, Dr. Hyman's observations point to a broader problem. If weak, inconsistent effects are obtained in a laboratory-based research program, it is open to question whether the phenomenon under investigation has any practical value. The anomaly may occur from time to time but it occurs in such an erratic fashion that it cannot be used to guide ones’ actions. In this regard, we find some agreement between the two reviewers, both of whom question whether the kind of strong consistent effects needed for practical applications have been demonstrated. Conclusions from the Expert Reviews In the preceding section we noted the points of agreement and disagreement among the reviewers. We tried furthermore to clarify and reconcile these points of agreement and. disagreement. With this background in mind, we now return to the basic questions presented to the reviewers and attempt to draw some firm conclusions about the implications to be drawn from this research review. The first question presented to the reviewers was whether the evidence indicated the presence of a statistically significant effect. This question was answered in a straightforward fashion: the reviewers agrecd that, considered broadly, statistically significant effects have been obtained in these studies. It appears that viewers’ descriptions produce hits more frequently than would be expected by chance. The second question presented to the reviewers considers the nature of these effects. The question to be answered was whether the effects could be attributed to paranormal phenomena. In this regard, the reviewers disagreed, with Dr. Utts arguing positively and Dr. Hyman negatively. Our conclusion from the discussions is that direct evidence has not been provided indicating that this paranormal ability of the remote viewers is the source of these effects. Attribution in general is difficult to demonstrate; for the present set of laboratory experiments, a primary concern for us is that the same viewers, the same judge, the same target set, and the same scoring procedures were repetitively used. This makes it difficult or impossible to localize the source of the phenomenon. American institutes for Research 3-81 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
ee Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 ne Chapter Three: Research Reviews The third question presented to the panel asked whether we have obtained an adequate understanding of the phenomenon. Do we know how the ability, if it exists, works? Here it is clear that the present research program has failed to identify mechanisms explaining the source of these effects. The fourth and final question presented to the reviewers was whether the research provides support for intelligence gathering operations. Here the magnitude of the observed effects, their consistency and replicability, and the need for subjective interpretation all seem to argue against potential applications. Taken as a whole, these answers lead to relatively straightforward general conclusions: ° The laboratory research conducted as part of the present program has identified a Statistically significant "anomaly." ° However, the experiments have not provided a convincing demonstration that a paranormal ability is involved. * The research studies have not identified the nature and source of the effect. ° There is no evidence that the phenomenon would prove useful in intelligence gathering. American Institutes for Research 3-82 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
enn Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 4. Evaluating the Utility of Remote Viewing in Intelligence Operations In the preceding section, we presented several conclusions bearing on evidence for the existence of remote viewing and on the nature of the phenomenon. In this section, we will, for the moment, assume that the phenomenon does indeed exist. This leads to questions of the utility of the phenomenon for intelligence applications. As noted earlier, our evaluation of remote viewing in terms of its value to the intelligence community had three components: , assessment of task requirements in relation to the boundary conditions believed to influence application of the phenomenon, . interviews with various participants in the program, and ° user assessments of the information provided by the remote viewing process. In this section, we consider each of these components. Research on Boundary Conditions Historically, the literature on paranormal phenomena, and on remote viewing in particular, has paid relatively little attention to establishing boundary conditions —- more specifically, the conditions under which effects are and are not observed. This point has been made in numerous earlier critiques of parapsychological research (Druckman & Swets, 1988; Hyman, 1994; Swets & Bjork, 1990), In response to these critiques, several investigators have initiated research intended to provide more clear-cut evidence bearing on the conditions influencing when effects are and are not observed in parapsychological studies (Bem & Honorton, 1994; Honorton, 1994). Research on remote viewing reflects this same general trend. In fact, many of the studies conducted in recent years have expressly examined this issue (May, Luke, & James, 1994; May, Luke & Lantz, 1993). merican Institutes for Research 4- Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 Chapter Four: Evaluating Utllity in intelligence Operations Senders and Distance. When one considers the remote viewing paradigm, one boundary condition immediately comes to the fore with regard to intelligence applications. During information gathering, it is unlikely that a “beacon" or sender will be at the site. In fact, the interviews with end users and the remote viewers indicate that only in the case of missing persons is a sender likely to be involved in information collection. If senders are necessary for the phenomenon to occur, its utility would be dramatically reduced. The laboratory experiments conducted as part of the current program avoid this issue; since the targets are photographs kept in another part of the laboratory, the equivalent of a “sender” would be the laboratory assistant who retrieves the photograph (or the computer that selects it). If one considers a "sender" as either a transmitter of information or an active participant in the remote viewing, this paradigm essentially does not have one. However, a recent study by May, Spottswood, and James (1994) has specifically addressed this issue. Their findings indicate that significant remote reviewing effects are obtained even when senders are not available. Another boundary condition that might influence the utility of remote viewing in intelligence collection is the distance of the target from the viewer. More specifically, for many of the tasks involved in intelligence applications, it may be impossible to place the viewer anywhere near the target. Thus, if the accuracy of remote viewing is limited by distance, this condition may, in turn, restrict its value. However, the work of May and his colleagues (May, Luke, & James, 1994; May, Luke, & Lantz, 1994) indicates that remote viewing may not be related to distance. Targets. A set of studies conducted by Lantz, Luke, and May (1994) examined another boundary condition of particular importance when assessing potential applications of the remote viewing phenomenon. Most recent research on remote viewing has been based on a fixed set of "static" targets, drawn from National Geographic photographs. Lantz, Luke, and May varied target type by including a new set of targets covering a wider range of novel stimuli, referred to as “dynamic" targets. Perhaps another relevant aspect of these new targets is that the remote viewers in the standard paradigm are very familiar with the National Geographic photographs, but were unfamiliar with the dynamic targets. The researchers found that significant effects were obtained for static but not dynamic targets. American Institutes for Research 4-2 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
scenes mein SE A Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 . Chapter Four: Evaluating Utility In Intelligence Operations nena This finding is of some importance from an operational intelligence perspective. In Appendix C, the interview reports, end users and remote viewers describe the operational targets presented to remote viewers. These targets represent a diverse set of potentially novel stimuli ranging from the location of ships to the likely background characteristics of a person. The findings obtained in the Lantz, Luke, and May study suggest that accurate viewings are less likely to be obtained for these dynamic "real-world" targets, a finding which, if replicated, represents a severe constraint on the utility of remote viewing to the intelligence community. Even if, as May, Spottswood, and James (1994) point out, those effects can be explained in terms of target "handwidth," there is no reason to assume that intelligence targets will conform to a limited range of predefined bandwidths. Information Requirements and Specificity. Another set of boundary conditions examined in recent studies considers the nature of the targets and the information to be assessed by judges. Intelligence consumers place a premium on the availability of specific, concrete, potentially verifiable information. On the other hand, the information obtained from remote viewings tends to be stated in broad, vague terms; a critical role must be played by judges or intelligence analysts, who must interpret the remote viewers’ reports. Recognizing this problem, a series of studies (May, Luke, & Lantz, 1993) were initiated intended to provide more concrete judgments framed in terms of the presence or absence of specific categories of objects, a technique referred to as “binary coding.” Results from these studies indicated that when relatively specific information was used in judging matches, weak or insignificant effects were obtained. » Specificity, as a boundary condition, can also be assessed in terms of targets. This issue was examined in a meta-analysis (May, Luke, & Lantz, 1993) contrasting the effects obtained for different types of targets. They found that effect sizes were related to target complexity, with more dynamic targets showing larger effect sizes. Incidentally, and seemingly in contradiction to the Lantz, Luke, and May (1994) study cited above, they found insignificant effects for static photograph targets. This pattern of effects, if intrinsic to the remote viewing process, would seem to limit applications to those involving more complex targets requiring less specific descriptions. Furthermore, it may also represent a significant limitation on applications of the remote viewing phenomenon where specific information is required. American Institutes for Research 4-3 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 Chapter Four: Evaluating Utility In Intelligence Operations ee Another aspect of specificity has to do with the degree of accuracy of the remote viewers' reports. Most remote viewing reports contain a large number of potentially interpretable components, an unknown percentage of which may not be related to the target. The end users cannot know which components are or are not related. This makes it difficult for end users, particularly if they are not highly trained, to separate valid information from irrelevant information, a problem that may prohibit éffective operational applications of the results of the viewing process. Feedback. Perhaps the most widely accepted boundary condition applying to remote viewing is the need for feedback. It is commonly held that the accuracy of viewings depends on the viewer receiving feedback following production of the viewing. In fact, one explanation of the remote viewing phenomenon is that viewings represent a form of precognition where the viewers identify future outcomes or events that they themselves will experience; in effect, they "communicate" with themselves across time (May, Lantz, & Piantineda, 1994). The important point here is that in "real-world" information gathering situations, it may be impossible for viewers to acquire the kind of feedback needed for accurate viewings. Remote Viewer Training. A final boundary condition frequently noted in the literature on remote viewing pertains to the nature and development of the presumptive remote viewing ability. Typically, remote viewing is held to represent a relatively rare talent not widely distributed in the population. Recent studies have suggested that associative learning techniques may contribute to improved performance on remote viewing tasks (May, Luke & Lantz, 1993). However, little evidence is available indicating that the capacity for producing accurate viewings can be systematically developed. This has implications for information gathering operations, since the burden would fall on the ability of the sponsoring agency to recruit and/or select individuals who already possess remote viewing skills. Other Boundary Conditions. In the preceding discussion, we have limited commentary to those boundary conditions directly addressed in the literature. However, a number of other boundary conditions exist that might limit potential applications of the remote viewing phenomenon. One such limitation on potential applications of the phenomenon involves the assessments of the information provided by the remote viewers. Typically, when judgmental data, such as target matches, are to be used in decision making, American institutes for Research 4-4 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 Chapter Four: Evaluating Utility in Intelligence Operations evidence is required indicating that end-user analysts arrive at much the same decisions. If analysts (or trained judges) cannot arrive at similar decisions, it is impossible to know what kind of actions should be taken. Unfortunately, evidence is not available indicating the degree of interrater, or cross-judge, agreement. Accordingly, it is difficult to assess whether different analysts or judges will reach similar conclusions given the same data. These differences in assessments, if indeed they exist, may set another important limitation on potential applications of remote viewings. Another boundary condition that has potential importance for remote viewing operational applications pertains to the degree of prior practice. In the remote viewing research studies conducted as part of the present program, both judges and the viewers have had years working together on a rather limited target set. Putting aside for the moment potential cueing and rule-based learning effects resulting from long periods of practice, in most field settings viewers and judges or analysts will have only limited opportunities to work together. As a result, it is open to question whether the findings obtained in these laboratory experiments can be extended to field settings. Boundary Conditions: Conclusions The issue of boundary conditions is part of a broader problem in generalizing research findings to operational settings. Typically, generalization from laboratory to operational settings is contingent on three requirements (Cook & Campbell, 1979). First, there is a need to demonstrate that an observed effect is sufficiently robust that it can be observed using different methods. Second, explicit verified causal explanations, reflecting an understanding of tlie source of observed effects, need to be provided. Third, alternative causal explanations, for example judges and viewers learning implicit rules, must be ruled out, The laboratory experiments in the current research program conducted to date have relied on one method, judgmental matching; plausible causal mechanisms has not been identified; and studies have not been conducted drawing out competing explanations. Lacking this evidence, drawing strong conclusions concerning the potential operational applications of remote viewing is at best tenuous, and at worst misleading. Taken as a whole, prior laboratory experiments examining the boundary conditions related to remote viewing have clearly provided an important foundation for establishing the American Institutes for Research 4-5 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 Chapter Four: Evaluating Utility in intelligence Operations TS nature of the phenomenon. On the other hand, however, the findings obtained in these studies actually argue against operational applications in the intelligence community. Broadly speaking, it appears that the conditions under which intelligence information is gathered, the nature of the targets, the unavailability of feedback, and the inconsistency with which accurate viewings are obtained may all limit the usefulness of the phenomenon in intelligence operations. Further, many significant boundary conditions have not been examined and the scientific basis for generalizing from laboratory to field settings has not been provided. Given these observations, it appears that the existing research does not justify operational applications and, in fact, vis a vis the known boundary conditions, argues against operational applications. Interview Findings The second component of the operational evaluation was the interviews conducted with the various parties who had direct involvement in the remote viewing program in various intelligence operations. A detailed description of the procedures used in interviewing program participants and the results obtained in individual interviews is presented in Appendix C; here, we briefly summarize the interview procedures before turning to the principal findings which they generated. Structured interviews were conducted with the three principal types of program participants: recent users of the remote viewing service, the remote viewers themselves, and the Program Manager. Separate interview protocols were developed for each type of respondent to capture their unique perspectives. The interview questions were written to obtain objective information about characteristics of the program as opposed to more subjective evaluative assessments of parapsychological phenomena. All questions were reviewed by AIR psychologists. Interview Results: End Users. The interviews conducted with representatives of end user groups were particularly noteworthy: these interviews directly examined the accuracy and utility of the information being provided by the remote viewers and provided user assessments of the operational value of the viewings. American institutes for Research 4-6 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 Chapter Four: Evaluating Utility in Intelligence Operations As can be seen in the individual interview reports (Appendix C), initial interest in the potential uses of remote viewing was linked to contact with the Program Manager. Typically, users decided to try out the viewings on an exploratory basis. The primary reasons users were willing to explore the potential value of this technique were that it might (a) provide information otherwise difficult or impossible to obtain, and (b) provide information more rapidly and economically than other sources. All of the users indicated that viewings might be especially attractive when other, more traditional options had been exhausted. The tasks assigned to viewers involved a number of different types of targets. Although all of these targets were relevant to operational intelligence issues, the targets were not the same as those reported in the laboratory experiments. The targets included people, their backgrounds, and their actions, as well as their locations. Likewise, the typical tasks asked of the remote viewers were dissimilar to laboratory conditions: they were asked to identify objects at specified locations and where certain objects might be found. When responding to these task demands, the remote viewers typically had at least some background knowledge; in some cases,this background knowledge was substantial. Typically, three independent viewings were obtained. The results of each viewing were summarized in a three or four page report which included drawings and verbal descriptions. With regard to the information provided in each report, five general observations emerged across all interviews: . The information provided in the reports was stated in broad, vague terms. ° The reports were most likely to prove accurate with regard to general stereotypical characteristics of the situation. Such results might be attributable to the background information available to the viewers. ° The reports were most likely to prove inaccurate with regard to concrete specifics of the task. For example, the reports often were not consistent with key known facts nor did they provide information about the unique features of a location. American Institutes for Research 4-7 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 Chapter Four: Evaluating Uillity in Intelligence Operations ° A large amount of irrelevant and often inaccurate information was contained in the reports, thus making them difficult to apply without substantial interpretive effort. ° The reports independently provided by different remote viewers displayed many inconsistencies. Because the viewings were not consistent with each other, because inaccuracies were observed, and because the information lacked the specificity needed for intelligence operations, the viewings were never used as a primary source of evidence in making decisions. In fact, even the most favorable of the user groups found the information inadequate for operational decisions. Instead, it was used to fill in background information on people that could not be readily obtained through available assets. All of the users noted that viewings should only be considered as providing supplemental information, and should be judiciously interpreted if used at all. The perceived utility of the information varied with the nature of the organization using the viewing services. Typically, viewings were seen as more valuable when the organization involved did not have adequate assets available and when there was no other convenient way of obtaining requisite information. When other information sources were available, the users found the viewings to be less valuable. In all cases, the users remarked that they would continue to consider working with the viewers on an exploratory basis. However, in most cases the users were not willing to invest their own operational funds to obtain viewings. In part this was an issue of cost; in part, however, the users felt that because the remote viewing process was controversial, official acknowledgement of its value was essential for routine use in intelligence gathering. This point was illustrated by the manager of the most favorable user group, who noted that many of the analysts were critical of the technique and unwilling to apply it unless specifically directed to do so. Remote Viewers. The three remote viewers we interviewed were all working in the intelligence community when they were recruited to be viewers. Although one viewer was subjected to a formal screening process, all were primarily selected on the basis of their interest in parapsychology. The viewers all received formal training as described in Appendix C and continued to extend this initial training through active exploration of various American Institutes for Research 4-8 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 Chapter Four: Evaluating Utility in Intelligence Operations parapsychological techniques. The viewers used a variety of techniques to produce viewings including automatic writing, meditation, and channelling. With regard to the procedures used in generating viewings, a number of factors were noted that influenced the nature and outcomes of the viewers’ efforts. First, the viewers disagreed as to how useful it was to have background information concerning the nature of their tasking. Second, the viewers noted that the usefulness of viewings depended on having sophisticated, knowledgeable users who accepted the technique and were willing to actively work with what was necessarily somewhat ambiguous information. Third, they uniformly agreed that the types of information requested — such as activities of individuals and specific locations of specific people or objects — were not particularly compatible with their remote viewing experiences. In addition to these observations, the viewers indicated a number of other considerations that might contribute to program outcomes. Some of these considerations, such as balancing work load and the repetitive nature of specific tasks, related to better management. Other considerations, such as management support and the availability of managers familiar with the process, related to the day-to-day management of the program. From a parapsychological perspective, a potentially disturbing aspect of the interview was that the remote viewers commented that their written reports were occasionally inconsistent with their viewings. They themselves considered it appropriate to make the written reports consistent with known characteristics of the target, even if the report was different from their viewing. Similarly, the remote viewers commented that previous managers also changed viewers’ reports to make them more consistent. The Program Manager. Appendix C presents a summary of the interview with the most recent program manager. Although the manager did not have any prior background in parapsychology, he did bring to the program a background in operational intelligence. This background played an important role in allowing the manager to recruit clients. Most clients, however, had been interested in the program in the past — an interest maintained in part because the program provided a rapid turnaround of otherwise difficult-to-obtain information. American Institutes for Research 4-9 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 a Chapter Four: Evaluating Utility in intelligence Operations The manager noted that none of the user groups had used the viewings as a primary basis for operational decisions. Instead, they were using the service to explore potential applications of the technique or, alternatively, as a source of supplemental information. Roughly half of the groups contacted agreed to participate in the program on an exploratory basis. During his interview, the manager stated that recruitment of clients was in large measure due to the Congressional mandate. He noted however, that acceptance of the program was limited by the controversy surrounding remote viewing. The research component was seen as potentially helpful in addressing this issue, although it did not contribute much to operational management. The manager felt that the program's long-term results, particularly in foreign assessment, would be enhanced by declassifying the research work and assigning responsibility for it to another government organization such as the Department of Justice or the National Science Foundation. Along similar lines, he suggested that remote viewing services be provided on a contract basis whenever the need for them arose. Conclusions Before turning to the broader conclusions flowing from these interviews, certain limitations inherent in this particular evaluative effort should be mentioned. First, the evaluation was limited to recent operations. Although this factor reduces and focuses the range of information considered, it does not help to ensure the accuracy of the information reported. Indeed, if previous managers or the viewers themselves changed viewer reports, then relying on this body of information could potentially be misleading with respect to the remote viewing phenomenon or to the operational use of remote viewing. Second, it should be recognized that interviews generate qualitative information that does not permit us to draw strong quantitative conclusions. Third, it is clear that this report necessarily focuses on actual operations and not on operations as they might occur under ideal conditions. Even bearing these points in mind, we believe that the interviews have a number of significant implications, The most important implication pertains to the nature of the information provided by the remote viewing process and the potential applications of this information in intelligence operations. The information provided by viewings: merican Institutes for Research 4-7 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 Chapter Four: Evaluating Utility in Intelligence Operations SL ° is vague and general in nature; . is not consistent across independent viewings; . lacks specific content consistent with known facts of the case; and ° includes a large amount of irrelevant, often erroneous, information. Intelligence operations are contingent on the availability of relatively specific information which is reliable, consistent, and potentially verifiable. The lack of specifics apparent in the viewings may well be an intrinsic characteristic of the process — a point noted by the remote viewers. Nonetheless, this vagueness and the limited agreement evident in viewers’ reports make it impossible to apply this information in decision making. One potential strategy for addressing this issue would be to seek more specific information in viewings. However, existing laboratory research provides little support for the likely success of this approach. Alternatively, one might explicitly train users to work with this kind of broad, rather vague information in such a way that they would attend to both its strengths and its weaknesses. However, this kind of effort would require a strong commitment from sponsoring organizations and unambiguous, high-level support for the program. In addition to the problems associated with the reliability and vagueness of the information provided, the viewings typically contain a large amount of irrelevant information and often fail to capture key aspects of the case. These characteristics of the viewings reduce user confidence in the information provided. Further, the nature of this information is such that.the burden is effectively placed on the user in sorting out what is and is not relevant. Under these conditions it is quite possible that the information provided by viewings will lead users down a number of blind alleys, thereby resulting in misallocation of intelligence resources. Alternatively, it may do little more than reinforce existing preconceptions and stereotypes about the targets — a trend evident in the user interviews, This observation brings us to the accuracy issue. Although the viewings tended not to be accurate with regard to concrete specifics, a characteristic which held true regardless of the type of tasking, the users felt that they were more accurate in describing broad background factors involved in the case. The problem here, of course, is that this outcome may simply merican Institutes for Researc 4-47 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 t Chapter Four: Evaluating Utility in Intelligence Operations reflect the availability of background information and the logical use of relevant analytic cues. This problem was, in fact, noted by the viewers when they indicated that they modified reports to make them consistent with know background factors. This observation suggests that the viewers may not be providing any more information than could be obtained from perceptive analysis working without the aid of remote viewing techniques. Still another limitation on the operational value of the information being provided pertains to its source. Remote viewing is a controversial phenomenon. Accordingly, routine operational use of the resulting information across multiple units in the intelligence community may well be contingent on a compelling demonstration of the phenomenon and broad acceptance of the resulting findings. Without this acceptance, users will discount the information provided and attempts to force analysts to use this information are likely to cause conflict. If, however, comparable information were provided by the viewings, it could be argued that the program was more cost-effective than conventional techniques. The fact that the information being provided is too vague and unreliable to permit operational application suggests that the viewing program is of limited value to the intelligence community. This conclusion, however, pertains to the current program. It is, of course, always possible that a substantially restructured and redirected research effort along with a substantial additional investment of resources might yield better results. Given the costs involved, however, and the need for specific, reliable information, any investment along those lines should be contingent on successful demonstration of not only the existence of the phenomenon but also its ability to consistently produce specific accurate information. Our foregoing observations of the interview results can be summarized as follows: ° The viewing process has not provided information of adequate specificity and reliability for use in intelligence gathering. ° The process is not widely accepted and is not seen as essential to intelligence gathering. * The process is often used experimentally because the needed information is difficult to obtain. American institutes for Research 4-12 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 Chapter Four: Evaluating Utility In Intelligence Operations ° Accuracy is limited to broad superficial characteristics of the case and may simply reflect good logical analysis. ° The available research does not directly support operations and new types of research are needed to justify current operational use. User Assessments A third source of evidence used to evaluate operational utility consisted of analyses of user feedback. This feedback was in the form of assessments of product (i.¢., the remote viewers! reports) accuracy and value collected from end users. This information was collected by a representative of the U.S. Government in the Spring and Summer of 1995, The results of these analyses have been compiled in a report; this report is included as Appendix D; we summarize the findings and implications below. The program office asked each user submitting tasks to evaluate the accuracy and value of the viewings. Evaluations were available for forty viewings obtained in 1994 and 1995. Accuracy (i.e., "Is the information accurate?") was rated on a six-point scale with 1 indicating high degrees of accuracy. Value (i.e., "What is the value of the source's information?") was rated on a five-point scale where a 1 indicated major significance, while a 5 indicated no value. It should be noted that separate ratings were obtained for each viewer's report on each tasking requested by an organization. The average accuracy rating was 3.0 across viewers and tasks, indicating that there might "possibly" be some accurate information provided, The average value rating was 3.5 (out of 5) indicating that the information was of relatively low value. This pattern of results is consistent with the interview findings. Remote viewings were better at capturing broad background information rather than the concrete, specific information needed for intelligence operations. Bearing this general conclusion in mind, a number of other questions about the remote viewings can be addressed by these data. For example, one important question is whether any particular remote viewer did particularly well. The accuracy scores of all three viewers were within a tenth of a standard deviation from the mean, while value scores were within a American Institutes for Research 4-7 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 Chapter Four: Evaluating Utility In Intelligence Operations quarter of a standard deviation. Thus, it appears that no one viewer performed exceptionally well or poorly. Another question is whether the remote viewers performed better on some tasks than others. Because tasks differed by the nature of the organization, this question can be addressed by contrasting the evaluations obtained from different organizations. Two organizations, one concerned with tracking people and one concerned with combat intelligence, indicated that the information provided was of substantially greater accuracy and value than was reported by the other user organizations. In the case of the two organizations providing positive evaluations, however, the viewers had substantial general background information concerning the targets, a finding suggesting that the ratings of outcomes of viewings may be dependent on the amount of pre-existing background information. These findings are noteworthy because they confirm a conclusion derived from the interviews. More specifically, viewings apparently were of limited accuracy, with accuracy being linked to the availability of general background information. Such background information, however, was not believed to have much operational value. Information Gathering Applications: Operational Conclusions The findings obtained in this evaluation of the intelligence applications of remote viewing lead us to the conclusion that remote viewing as used in the present program has limited value for the intelligence community as an information gathering technique. The basi¢ considerations that lead us to this conclusion are: ° Conditions under which significant effects are observed in experimental laboratory research are, for the most part, unlikely to occur in intelligence operations. Not only will feedback be unavailable, but the target pool will typically be unconstrained. . Information provided by the remote viewing technique tends to be vague and ambiguous and it appears difficult, if not impossible, to consistently obtain accurate information from the remote viewers across a range of targets. American institutes for Research 4-14 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 Chapter Four: Evaluating Utility In Intelligence Operations These problems with the consistency and accuracy of viewings were also apparent in the end user interviews. In these interviews, viewings were found to be too broad and vague; the viewings failed to provide the concrete, specific information needed for actionable intelligence. Further, there were indications that its potential usefulness was limited to information which could be acquired in other ways. Thus, the evidence accrued from research, interviews, and user assessments all indicate that the remote viewing phenomenon has no real value for intelligence operations at present. In fact, given the findings obtained to date and the nature of intelligence operations, one must question whether any further applications can be justified without major theoretical and practical advances in our understanding of the phenomenon, assuming it exists at all. American Institutes for Research 4-15 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 5. Conclusions In the preceding sections of this report, we have presented a variety of evaluative data bearing on the existence of the paranormal phenomenon known as remote viewing and its potential applications in intelligence gathering. This multifaceted evaluation effort was structured to ensure a fair, unbiased evaluation of both the research program and its intelligence applications. As is the case with any objective, relatively sophisticated program evaluation effort, many pieces of evidence bearing on different aspects of the program have been presented. In this section, we summarize the basic conclusions flowing from this evaluation effort. Summary of Key Findings Two expert reviewers, one known to be a sophisticated advocate of the study of paranormal phenomena and one viewed as a fair-minded skeptic, reviewed the laboratory experiments conducted as part of the current program that bear on the existence of the remote viewing phenomenon. They focused primarily on recent, better-controlled laboratory studies, drawing from other sources as needed to ensure a comprehensive evaluation of the research literature. Although the reviewers disagreed on some points, on many points they reached substantial agreement. The first important point of agreement concerns the existence of a statistically significant effect, which leads to the following finding: . A statistically significant effect has been observed in the recent laboratory experiments of remote viewing. However, the existence of a statistically significant effect did not lead both reviewers to the conclusion that this research program has provided an unequivocal demonstration that remote viewing exists. A statistically significant effect might result either from the existence of the phenomenon, or, alternatively, to methodological artifacts or other alternative explanations for the observed effects. merican institutes for Hesearc. §-7 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 ne em Chapter Five: Conclusions It is with regard to the explanation for these effects that the two reviewers differ most clearly, One reviewer argues that the procedures used in recent studies rule out many, but not all, methodological explanations. The other reviewer argues that the consistency of the results obtained across experiments strongly suggests the existence of the paranormal phenomenon. We concluded that ° The experimental research conducted as part of the current program does not unambiguously support the interpretation of the results in terms of a paranormal phenomenon. Both reviewers agreed that one important methodological problem has not yet been addressed. Specifically, only one judge — apparently the Principal Investigator — was used in assessing matches throughout these experimental studies. As a consequence, there is no evidence for agreement across independent judges as to the accuracy of the remote viewings. Failure to provide evidence that independent judges arrive at similar conclusions makes it difficult to unambiguously determine whether the observed effects can be attributed to the remote viewers' (paranormal) ability, to the ability of the judge to interpret ambiguous information, or to the combination or interaction of the viewers and the judge. Furthermore, given the Principal Investigator's familiarity with the viewers, the target set, and the experimental procedures, it is possible that subtle, unintentional factors may have influenced the results obtained in these studies. Thus, until it can be shown that independent judges agree, and similar effects are obtained in studies using independent judges, it cannot be said that adequate evidence has been provided for existence of the remote viewing phenomenon. Both reviewers agree that no compelling explanation has been provided for the observed effects, One reviewer considers the investigation of the determinants of remote viewing as the next necessary step, but does not see it as essential to continue to conduct experiments designed solely to demonstrate the existence of the phenomenon. The other reviewer argues that without identifying causal mechanisms and explicitly providing evidence that alternative explanations cannot account for the observed effects, we cannot say we have convincing evidence for the existence of a phenomenon. Essentially, this position holds that an observed effect may arise for many reasons; to say a phenomenon has been demonstrated we must know the reasons for its existence. American Institutes for Research 5-2 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
St Sth eh Se SS Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 Rt Chapter Five: Conclusions Our conclusion is that at this juncture it would be premature to assume that we have a convincing demonstration of a paranormal phenomenon. In fact, until a plausible causal mechanism has been identified, and competing explanations carefully investigated, we cannot interpret the set of anomalous observations localized to one laboratory with one set of methods, Given these observations, and the methodological problems noted above, we must conclude that ° Adequate experimental and theoretical evidence for the existence of remote viewing as a parapsychological phenomenon has not been provided by the research component of current program, A significant change in focus and methods would be necessary to justify additional laboratory research within the current program. This is not to say definitively that paranormal phenomena do not exist. At some point in time, adequate evidence might be provided for the existence of remote viewing. With this point in mind, we considered the potential applications of remote viewing in intelligence gathering. The first consideration involves the conditions under which remote viewing occurs and if those conditions constrain its application for intelligence purposes. Prior research suggests that distance is not a constraint and indeed that a sender or "beacon" may not be necessary. However, other characteristics of intelligence gathering indicate that remote viewing is of little value. Intelligence operations do not provide targets of a fixed bandwidth; rather, targets and target types are highly variable. Moreover, the apparent necessity for feedback to the remote viewers would preclude its use in intelligence gathering operations. Finally, intelligence information is most valuable if it is concrete and specific, and reliably interpretable. Unfortunately, the research conducted to date indicates that the remote viewing phenomenon fails to meet those preconditions. Therefore, we conclude that . Remote viewing, as exemplified by the efforts in the current program, has not been shown to have value in intelligence operations. This point was also graphically illustrated in the user interviews, where it was found that remote viewings have never provided an adequate basis for "actionable" intelligence merican institutes tor Researc. 5-3 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 a Chapter Five: Conclustons operations — that is, information sufficiently valuable or compelling enough so that action was taken as a result. If a phenomenon does not contribute to intelligence operations, it is difficult to see what justification exists for its continued application. This is particularly true in the case of remote viewing, where a large amount of irrelevant, erroneous information is provided and little agreement is observed among viewers’ reports. Particularly troublesome from the perspective of the application of paranormal phenomena is the fact that the remote viewers and project managers reported that remote viewing reports were changed to make them consistent with know background cues. While this was appropriate in that situation, it makes it impossible to interpret the role of the paranormal phenomenon independently. Also, it raises some doubts about some well- publicized cases of dramatic hits, which, if taken at face value, could not easily be attributed to background cues. In at least some of these cases, there is reason to suspect, based on both subsequent investigations and the viewers' statement that reports had been "changed" by previous program managers, that substantially more background information was available than one might at first assume. Give these observations, it is difficult to argue that available evidence justifies application of remote viewing in intelligence operations. In summary, two clear-out conclusion emerge from our examination of the operational component of the current program. First, as stated above, evidence for the operational value of remote viewing is not available, even after a decade of attempts. Second, it is unlikely that remote viewing — as currently understood — even if existence can be unequivocally demonstrated, will prove of any use in intelligence gathering due to the conditions and constraints applying in intelligence operations and the suspected characteristics of the phenomenon, We conclude that: ° Continued support for the operational component of the current program is not justified. American institutes for Research 5-4 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 REFERENCES Bem, D.J. & Honorton, C. (1994) Does PSI Exist? Replicable Evidence for an Anomalous Process of Information Transfer, Psychological Bulletin, 115, 4-18. Cook, T. D., & Campbell, P.T. (1979) Quasi-experimentation: Design and Analysis Issues for Field Settings. Chicago, Il; Rand Mc Nally Druckman, D., & Swets, G.A. (Eds.) (1988) Enhancing Human Performance: Issues, Theories, and Techniques. Washington, DC: National Academy Press. Honorton, C. (1985) Meta-Analysis of PSI Ganzfeld Research: A Response to Hyman. Journal of ParaPsychology, 49, 51-91. Hyman, R., (1994) Anomaly or Artifact? Comments on Bem and Honorton. Psychological Bulletin, 115, 19-24. James, L.R., Muliak, S.A., & Brett, J.M. (1982). Causal Analysis: Assumptions, Models, and Data. Beverly Hills, CA: SAGE Lantz, N. D., Luke, W.L., & May, E.C. (1994) Target Sender Dependencies in Anomalous Cognition Experiments. Menlo Park, CA: Science Applications International Corporation. May, E.C., Lantz, N.D., & Piantineda, J. (1994) Feedback Considerations in Anomalous Cognition Experiments. Menlo Park, CA: Science Applications International Corporation. May, E.C., Luke, W.L., & James, C.C. (1994). Phenomological Research and Analysis. Menlo Park, CA: Science Applications International Corporation. May, E.C., Luke, W.L., & Lantz, N.D. (1993), Phenomological Research and Analysis, Menlo Park, CA: Science Applications International Corporation. May, EC,, Spottiswoode, S.J. & James Cc. Cc. (1994) Bandwidth: Noise ts. Menlo Park, CA: Science Applications International Corporation. Messick, S. (1989) Validity. In R.L. Linn (Ed)_Educational Measurement (PP 13-104). New York, Macmillian Swets, J.A., & Bjork, R.A. (1990) Enhancing Human Performance: An Evaluation of New Age Techniques Considered by the U.S. Army. Psychological Science, 1, 85-96. merican Institutes for Hesearc R- Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 APPENDIX A: REVIEWER VITAS CURRICULUM VITAE' September 1994 Name: Ray Hyman Present position: Professor of Psychology Department of Psychology University of Oregon Eugene, Oregon 97403 Date and place of birth: June 23, 1928 Chelsea, Massachusetts Academic degrees: 1950 A.B., Boston University Phi Beta Kappa Honors in Psychology 1952 M.A., The Johns Hopkins University 1953(FEB) PhD., The Johns Hopkins Univ. Positions held: Jul 1953-Jun 1961 Assistant Professor of Social Psychology, Harvard University Jul 1958-Jun 1961 Consultant in Behavioral Research, General Electric Company Sep 1961-Aug 1964 Associate Professor of Psychology, University of Oregon Sep .1964- Professor of Psychology, University of Oregon Sep 1967-Aug 1968 Fulbright-Hays Research Scholar, University of Bologna, Italy Sep 1976-Dec 1976/Jun 1977-Sep !977/Jun 1978-Sep 1978 NSF Faculty Fellowship in Science Applied to Societal Problems Sep 1982-Jun 1983 Visiting Professor of Psychology, Stanford University (Thomas Welton Stanford Chair for Psychical Research) 'This is a very abbreviated and somewhat modified version of my full resume. merican institutes for Researc. -7 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 rn Appendix A: Reviewer Vitas Mar 1949-Jul 1949 Statistician for antihistamine study, Allergy Fund of Boston Jan 1953-Jul 1953 Consultant to Systems Coordination Division, Naval Research Laboratory, Washington, D.C. Jul 1974-Aug 1974 Subcommittee on Preparing Problems and Examples for Committee on the Mathematical Training of Social Scientists, Social Science Research Council Oct 1955-Jan 1956/Oct 1956-Jan 1957/Oct 1957-Jan 1958 Director of Statistical Workshop for Psychology Department, Clark University Oct 1957-Jun 1958 Statistical Consultant, Age Center of New England 1970-1975 Statistical Consultant, Roseburg VA Hospital Some Current or Recent Positions: Executive Council, Committee for the Scientific Investigation of Claims of the Paranormal Editorial Board, The Skeptical Inquirer Member of National Research Council's Committee on Techniques for the Enhancement of Human Performance (1985-1991) Publications: BOOKS 7 MONOGRAPHS: Bush, R.R., Abelson, R.P., & Hyman, R. (1956). Mathematics for psychologists: examples and problems. New York: Social Science Research Council. Vogt E.Z., & Hyman, R. (1979, Revised Edition). Water Witching ULS.A. Chicago: Univ. of Chicago Press. Hyman, R. (1960). Some experiments in creativity. New York: General Electric Company (101 pages). 7A Partial listing. American Institutes for Research A-2 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 a  Appeniix A: Reviewer Vitas Hyman, R. (1960). ivity: a New York: General Electric Company (204pp). Hyman, R. (1964). The nature of psychological inquiry. Englewood Cliffs, New Jersey: Prentice-Hall. Hyman, R. (1965). Creativity and the prepared mind: preconceptions in creative achievement and creativity research. National Art Education Association, Hyman, R. (1989). The elusive quamv:_a scientific appraisal of psychical research. Buffalo, NY: Prometheus Books. Hyman, R. (in preparation). How smart people go wrong: cognition and human error, ARTICLES: I have published over 200 articles in professional journals on perception, pattern recognition, creativity, problem-solving and critiques of the paranormal. I consider all my publications as dealing with aspects of my major interest in human error and deception. I have also published articles in magic journals and have won awards for inventing new conjuring effects. k d ia Pr During this period I have given talks to public schools, civic groups, and other organizations. I also have appeared on several television and radio programs. I serve as a resource to the media on various topics related to the paranormal, deception, mysticism, the occult, and human error, In this connection I have appeared on all the major networks, on Cable Network News, the Larry King Live show, Italian Television, Canadian Television, BBC television, and Nova. I see these appearances as part of my program to help educate the public about how human cognition both enables us to cope with problems and makes us susceptible to illusion and deception. ‘perience ; denti V. e ic Phenome Since 1953, I have been called upon by various governmental agencies to investigate or evaluate paranormal claims, Some examples would be the evaluation of a lady who allegedly could read with her finger tips, the evaluation of the claim by a group of engineers that they could teach psychics to gather information from photographs; the assessment of parapsychological research by American and foreign investigators, and the observation of metal bending and other alleged miracles by Uri Geller. I have also served as consultant and expert witness in court cases involving psychics or related paranormal claims. I have American institutes for Research A-3 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 ee Appendix A: Reviewer Vitas appeared on several radio and television programs in the United States, Great Britain, Canada and Italy to comment upon paranormal claims and to demonstrate how alleged paranormal phenomena can be duplicated by simple trickery and psychological manipulation. I earned my way through college performing mentalism and reading palms. I have demonstrated psychic readings on several television and radio shows. I have done research and written articles on why people can falsely believe that their psychic readings were accurate and depended upon occult information. I have been invited to the Euroskeptics' conference in Ostende, Belgium to conduct a workshop on psychic readings at the end of September, 1994. American institutes for Research A-4 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 Appendix A: Reviewer Vitas JESSICA M. UTTS Current Employment: Professor, Division of Statistics 916-752-6496 (office phone) University of California, Davis 916-752-7099 (FAX) Davis, CA 95616 jmutts@ucdavis.edu Education BA State University of New York at Binghamton, Math and Psychology, 1973 MA Pennsylvania State University, Statistics, 1975 PhD. Pennsylvania State University, Statistics, 1978 vious Ti Visiti Visiting Senior Research Fellow, Department of Psychology, University of Edinburgh, Scotland, Winter 1994. Director, Statistical Laboratory, University of California, Davis, 7/88-6/93 SRI International, Cognitive Sciences Program, Menlo Park, CA, Visiting Scientist, 6/87-8/88 Stanford University, Dept. of Statistics, Visiting Professor, 6/83-9/83 and 9/84-6/85 University of California, Davis, Division of Statistics, Assistant and Associate Professor, 7/79- 6/79 University of California, Dais, Dept. of Mathematics, Assistant Professor, 7/78-6/79 Penn State University, Dept. of Statistics, Instructor, 3/78-8/78 Academic Honors: Fellow, American Association for the Advancement of Science, 1992 Fellow, Institute of Mathematical Statistics, 1991 Fellow, American Statistical Association, 1990 Academic Senate Distinguished Teaching Award, University of California, Davis, 1984 Magnar Ronning Award for Teaching Excellence, University of California, Davis, 1981 Phi Beta Kappa, State University of New York at Binghamton, 1973 merican institutes for Researc A-5 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 a Appendix A: Reviewer Vitas Professional Affiliations and Offices (alphabetical order): American Association for the Advancement of Science: various roles in Section U (Statistics) American Statistical Association: President, State College PA Chapter, 1977-78 Biometric Society, western North American Region: President, 1986; Reg. Comm., 1982-84; ‘Program Chair, 1983 Caucus for Women in Statistics: President, 1988 Institute of Mathematical Statistics: Treasurer, 1988-1994; Assistant Program Secretary, 1980, 1989 Parapsychological Association: Representative to AAAS Section X, 1989-Phi Beta Kappa: President of UC Davis Chapter, 1984-85, Vice President, 1983-84 Society for Scientific Exploration: Council Member, 1987-93 Maior Consultati Panels: National Academy of Sciences, Panel on the Evaluation of AIDS Interventions Congressional Office of Technology Assessment, Panel to Assess Defense Technologies National Park Service, Statistics Short Course for Resource Management Trainees California Department of Health Services, Course on Statistics for Groundwater SRI International Cognitive Sciences Program, Consultant California Public Utilities Commission, Consultant Hershey Medical Center, Sudden Infant Death Syndrome Study, Consultant ABC News 20/20 Program, Interview (appeared July 4, 1985); various other national television shows Editorial Positions: Associate Editor, Journal of the American Statistical Association (Theory & Method) Statistical Editor, Journal of the American Society for Psychical Research American Institutes for Research A-6 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 Appendix A: Reviewer Vitas A JESSICA M. UTTS PUBLICATIONS l. 1976 Utts, J.M. and T.A. Ryan, Jr. Lack of fit in regression. American Statistical Association Proceedings of the Statistical Computing Section,285-287 2, 1976 Varner, L. and J. Utts. Parallel prediction lines: a test for interaction. The Journal of Educational Research, 70(2):63-66 3, 1977 Hettmansperger, T.P. and J.M. Utts. Robustness properties for a simple class of rank estimates. Communication in Statistics - Theory and Methods, A6(9): 855-868 4, 1977 Naeye, R.L., W.L. Harkness and J. Utts. Abruptio placentae and perinatal death: a prospective study. American Journal of Obstetrics and Gynecology, 128(7): 740-746 5. 1980 Uvtts, J.M. and T.P. Hettmansperger. A robust class of tests and estimates for multivariate location. Journal of the American Statistical Association, 75(372): 939-946 6. 1981 Ainley, D.G., C.R. Grau, T.E. Roudybush, 8.H. morrell and J.M. Utts. Petroleum ingestion reduces reproduction i Cassin's anklets. Marine Pollution Bulletin, 12(9): 314-317. 7. 1982 Utts, J.M. The rainbow test for lack of fit in regression. Communications in Statistics-Theory and Methods, 11(24): 2801- 2815 8. 1983 Samaniego, F.J. and J.M. utts. Evaluating performance in continuous experiments with feedback to subjects. Psychometrika, 48(2): 195-209 9. 1985 Rucker, M.K. McGee, M. Hopkins, A. Harrison and J. Utts. Effects © of similarity and consistency of style of dress on impression formation. In The Psychology of Fashion, ED. M.R. Solomon, Lexington Books, Lexington, MASS, 309-318 10. 1985 May, E.C.D.I. Radin, G.S. Hubbard, B.S. Humphrey and J.M. Utts. Psi experiments with random number generators: an informational model. American Institutes for Research A-7 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
LR 11. 12. 13. 14, 15. 16. 17, 18, 19. 20. 21. Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 1985 1986 1986 1986 1986 1987 1987 1988 1988 1988 1988 Appendix A: Fleviewer Vitas Proceedings of the Parapsychological Association Annual Convention, 235-266 Johnson, W.O., J.M. Utts, and L.M. Pearson. Bayesian robust estimation of the mean. The Journal of the Royal Statistical Society Series C (Applied Statistics), 35(1):63-72 Utts, J.M. The Ganzfel Debate: a statistician's perspective. Journal of Parapsychology, 50: 393-402 Utts, JM. Comment on "Computers in statistical research." Statistical Sciuence, 1(4): 437-439 Johnson, W.O., L.M. Pearson, and J.M. Utts. A Monte Carlo comparison of Bayesian estimators and trimmed means. Journal of Statistical Computation and Simulation, 25:167-192 Utts, J.M. and E.C. May. An exact method for combining P-values. Research in Parapsycholgy, 99-103, Scarecrow Press, Metuchen, N.J. G.P. Hansen and J. Utts. Use of both sum of ranks and direct hits in free-response psi experiments. Journal of Parapsycholgy, 51:321-335 Utts, J. Psi, statistics, and society. Behavioral and Braiun Sciences, 10:615-616 Utts, J. Successful replication versus statistical significance. Proceedings of the Parapsychological Association, 31:148-162 Humphrey, B.S., E.C. May and J. Utts. Fuzzy set technology in the analysis of remote viewing, Proceedings of the Parapsychology Association, 31:378-394 Palmer, J.A., C. Honorton and J. Utts. Reply to the National Research Council study on parapsychology, Journal of the American Society for Psychical Research, 83(1):31-49. (Also published in Proceedings of the Parapsychology Association, 31:424-451 Utts, J. Successful replication versus statistical significance. American Institutes for Research A-8 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 a Appendix A: Reviewer Vitas 22. 23. 24, 25. 26. 27. 28, 29. - 30. 31. 32. 1989 1989 1989 1990 1990 1991 1992 1992 1992 1993 1993 Journal of Parapsychology, 52{4):305-320 Radin, D. and J. Utts. Experiments investigating the influence of intention on random and pseudorandom events. Journal of Scientific Exploraiton, 3(1):65-79 Flay, B.R., R.C. Kessler, and J.M. Utts, Evaluating media campaigns. In Evaluating AIDS Prevention Programs, Coyle, Boruch, and Turner, Eds., National Academy Press, Washington, DC Utts, J. Randomness and randomization tests: A reply to Gilmore. Journal of Parapsychology, 53(4):345-35 Utts, J. Use hammers for nails and corkscrews for wine: In defence of defendable statistical methods. The Skeptic, 4(5): SEpt./Oct. 1990, 16-18 May, E.C., J.M. Utts, B.S. Humphrey, W. Luke, T.J. Frivold, and V.V. Trask. Advances in Remote Viewing, Journal of Parapsychology, 54(3), 193-228 Utts, JM. Replication and meta-analysis in parapsychology (with discussion). Statistical Science, 6(4), 363-403 Christensen, R. and J.M. Utts Testing for nonadditivity in log-linear and logit models. Journal of Statistical Planning and Inference, 33, 333-343 Christensen, R. and J.M. Utts. Bayesian resolution of the exchange paradox. The American Statistician, November 1992, 274-276 Hansen G.P., J.M. Utts and B. Markwick. Critique of the PEAR remote viewing experiments. Journal of Parapsychology, 56(2), 97- 113 Utts, J. Analyzing free-response data-a pregress report. In PSI Research Methodology-A Reexamination, ed. L. Coly and J.D.S. McMahon, Parapsychology Foundation, New York, 71-83 Utts, J. Honorton the Meta-Analyst. Journal of Parapsychology, merican Institutes for Hesearc A-9 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 rT 33. 1993 34. 1993 35. 1994 36. 1994 Appendix A: Reviewer Vitas 57(1), 89-100 Utts, J. Obituary: Florence Nightengale David, 1909-1993. Biometrics, 49, 1289-1291 Krippner, S., W. Braud, LL. Child, J. Palmer, K.R. Rao, M. Schlitz, R.A. White and J.M. Utts. Demonstration research and meta- analysis in parapsychology. Journal of Parapsychology, 57(3), 275- 286 Utts, J. Social, Institutional, and Cultural Influenmces of Gender on Science. In Women and Parapsychology, ed. L. Coly and R.A. White, Parapsychology Foundation, New York, 28-44 Murphy, T.M. and J.M. Utts. A retrospective analysis of peer review at Physiologia Plantarum. Physiologia Plantarum, 92, 535-542 1. 1996 1. 1975 2. : 1979 3. 1980 4. 1982 BOOKS Utts, Jessica Seeing Through Statistics, Belmont, CA: Wadsworth(In Press) LIMITED DISTRIBUTION Utts, J.M. A test for lack of fit in regression models, Technical Report No. 29, Department of Statistics, Teh Pennsylvania State University. Utts, J.M. and T.P. Hettmansperger. A robust class of tests and estimates for multivariate location, Technical Report No. 1, Division of Statistics, University of California, Davis. (see Publ. #5A.) Utts, J.M. and L.L. Varner. A preocedure for analyzing a large data set using log-linear models. Technical Report No. 14, University of California, Davis. Samaniego, F.J. and Utts, J.M. Evaluating performance in continuous experiments with feedback to subjects. Technical Report No, 30, University of California, Davis. (See Publ. #8) American Institutes for Research A-10 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 Appendix A: Reviewer Vitas 5. 1982 Utts, J.M. A Regional test for fit in regression. Technical Report No. 38, University of California, Davis. 6. 1982 Johnson, W.O., L.M. Pearson and J.M, Utts. Assessing the performance of some robust point and interval estimators in the presence of maean-shift contamination. Technical Report NO. 44, Division of Statistics, University of California, Davis. 7. 1983 Utts, J.M. A sensible approach to hypothesis testing. Technical Report No. 51, University of California, Davis. 8. 1985 Utts, J.M. Inference for an experiment based on repeated majority vote. Technical Report, Dept. of Statistics, Stanford University 9, 1987 Utts, J.M., E.C, May and T.J. Frivold. Intuitive data sorting. Technical Report, SRI International Cognitive Sciences Program, Menlo Park, CA, December 1987. 10, 1987 Hubbard, G.S., J.M. Utts and W.W. Braud. Experimental protocol for hemolysis: Confirmation experiment. Technical Report, SRI International Cognitive Sciences Program, Menlo Park, CA, December 1987, 11. 1987 Humphrey, B.S., E.C. May, J.M. Utts, T.J. Frivold, W.W. Luke, and V.V. Trask. Fuzzy set applications in remote viewing analysis. Technical Report, SRI International Cognitive Sciences Program, Menlo Park, CA, December 1987, 12.° 1991 Christensen, R. and J. Utts. Bayesian resolution of classical paradoxes: two examples. Technical Report #220, Division of Statistics, University of California, Davis. BOOK REVIEWS 1, 1987 Utts, J.M. Nonparametric Methods for Quantitative Analysis(2nd ed.) by Jean Dickinson Gibbons. Journal of the American Statistical Association 82(397):364-367 2. 1988 Utts, J.M. Siac Wan, VoL -Pisuee Mates . G.P. Patil, M.T. Boswell and American institutes for Research A-1 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 Se Appendix A: Reviewer Vitas _ MLV. Ratnaparkhi; Vol, 2:_ Continuous Univariate Models, by G.P. Ratnaparkhi; Vol. 3: Multivariate Models, by G.P. Patil, M.T. Boswell, M.V. Ratnaparkhi, and J.J.J. Roux. Journal of the American Statistical Association, 83:562 3. 1993 Utts, J. Perspectives on Contemporary Statistics, ed. by D.C. Hoaglin and D.S. Moore. Journal of the American Statistical Association, 88, 695, American Institutes for Research A-12 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 APPENDIX B: BIBLIOGRAPHY DeGraff, E., May, E.C. (1993). Phenomenological Research and Analysis. Periodic Status Report -- Covering 12/30/92 - 04/23/93, Contract MDA908-93-C-0004, SAIC, Menlo Park, CA. , DeGraff, E., May E.C. (1993). Phenomenological Research and Analysis. Periodic Status Report -- Covering 4/23/93 - 6/30/93, Contract MDA908-93-C-0004, SAIC, Menlo Park, CA. DeGraff, E., May, E.C. (1993). Phenomenological Research and Analysis. Periodic Status Report -- Covering 6/22/93 - 9/10/93, Contract MDA908-93-C-0004, SAIC, Menlo Park, CA. Hubbard, G.S. (1987). Initial Protocol for Remote Action Interactions with c-Particles. Interim Report -- Objective H, Task 1, Deliverable a, SRI International, Menlo Park, CA. Hubbard, G.S., Bentley, P.P., Pasturel, P.K., Isaacs, J. (1987). A Remote Action Experiment With a Piezoelectric Transducer. Final Report -- Objective H, Tasks 3 and 3a, SRI International, Menlo Park, CA. , Hubbard, G.S., Isaacs, J.D. (1986). An Experiment to Examine the Possible Existence of Remote Action Effects in Piezoelectric Strain Gauges. Final Report -- Objective E, Task 8, SRI International, Menlo Park, CA. Hubbard, G.S., Langford, G.O. (1986). A Suggested Remote Viewing Training Procedure (U). Final Report -- Objective D, Task 1, SRI International, Menlo Park, CA. Hubbard, G.S., Utts, J.M., Braud, W.W. (1987). Experimental Protocol for Hemolysis: Confirmation Experiment. Final Report -- Objective E, Task 2, SRI International, Menlo Park, CA. Lantz, N.D. (1989). An Effort to Improve Remote Viewing Quality Using Hypnosis. Final Report -- Task 6.0.6, Covering Period 1 October 1988 to September 1989, SRI Project 1291, SRI International, Menlo Park, CA. Lantz, N.D. (1987). Review of the Personality Assessment System. Final Report -- Objective C, Tasks 2 and 3, SRI International, Menlo Park, CA. merican institutes for Researec. es = = | Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 a Appendix B: Bibliography Lantz, N.D., Kierman, R.J. (1986). Neuropsychological Assessment of Participants in _ Psychoenergetic Tasks, Final Report -- Objective C, Task 5, SRI International, Menlo Park, CA. Lantz, N.D., Luke, W.L.W., May, E.C. (1994). Target and Sender Dependencies in Anomalous Cognition Experiments. SAIC, Menlo Park, CA. * Lantz, N.D., Luke, W.L.W., May, E.C. (1994), Target and Sender Dependencies in Anomalous Cognition Experiments. Journal of Parapsychology, 58, 285-302. Lantz, N.D., May, E.C. (1988). Mass Screening for Psychoenergetic Talent Using a Remote Viewing Task. Final Report -- Objective B, Task 1, Covering the Period 1 October 1987 to 30 September 1988, SRI Project 1291, SRI International, Menlo Park, CA. Luke, W.L.W., Frivold, T.J., May, E.C., Trask, V.V. (1989). A Prototype Analysis System For Special Remote Viewing Tasks. Final Report -- Task 6.0.3, Covering the Period 1 October 1988 to 30 September 1989, SRI Project 1291, SRI International, Menlo Park, CA. * Luke, W.L.W., May, E.C. (1993). Phenomenological Research and Analysis. Interim Technical Report -- Covering 30 December 1992 thru 30 April 1993, Contract MDA908-93-C0004, SAIC, Menlo Park, CA. Luke, W.L.W., May, E.C. (1993). Travel Report: The Third Meeting of The Chinese Society of Somatic Science (CSSS), Bejing China. Contract MDA908-91-C-0037, SAIC, Menlo Park, CA. * May, E.C. (1989). An Application Oriented Remote Viewing Experiment. Final Report -- Covering the Period 1 May 1988 to April 1989, SRI Project 2740, SRI International, Menlo Park, CA. May, E.C. (1986). Enhanced Human Performance Investigation. Final Technical Report -- Covering the Period 1 July 1986 to 15 November 1986, SRI Project 1291, SRI International, Menlo Park, CA. May, E.C. (1988). Enhanced Human Performance Investigation (U). Final Technical Report -- Covering the Period 1 October 1987 to 30 September 1988, SRI Project 1291, SRI International, Menlo Park, CA. May, E.C. (1993). Phenomenological Research and Analysis. Periodic Status Report merican Institutes for Research B- Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 EEE Appendix B: Bibliography -- Covering 12/16/92 - 04/05/93, Contract MDA908-91-C-0037, SAIC, Menlo Park, CA. May, E.C, (1992). Phenomenological Research and Analysis. Periodic Status Report -- Covering 6/19/92 - 09/11/92, Contract MDA908-91-C-0037, SAIC, Menlo Park, CA. May, E.C. (1992). Phenomenological Research and Analysis. Periodic Status Report -- Covering 9/11/92 - 11/16/92, Contract MDA908-91-C-0037, SAIC, Menlo Park, CA. May, E.C. (1992). Phenomenological Research and Analysis. Periodic Status Report -- Covering 3/31/92 - 06/09/92, Contract MDA908-91-C-0037, SAIC, Menlo Park, CA. May, E.C. (1992). Phenomenological Research and Analysis. Periodic Status Report -- Covering 06/11/92 - 12/16/92, Contract MDA908-91-C-0037, SAIC, Menlo Park, CA. May, E.C. (1992). Phenomenological Research and Analysis. Periodic Status Report -- Covering 10/01 - 11/30/91, Contract MDA908-91-C-0037, SAIC, Menlo Park, CA. May, E.C. (1991). Research of Anomalous Mental Phenomena. Periodic Status Report -- Covering 2/4 - 9/30/91, Contract MDA908-91-C-0037, SAIC, Menlo Park, CA. May, E.C., Lantz, N.D., Piantineda, J. (1994). Feedback Considerations in Anomalous Cognition Experiments. SAIC, Menlo Park, CA. * May, E.C., Luke W.L.W. (1992), Phenomenological Research and Analysis. Final Report: Support, Contract MDA908-91-C-0037, SAIC, Menlo Park, CA. * May, E.C., Luke, W.L.W., James, C.L. (1994). Phenomenological Research and Analysis. Final Report -- Contract MDA908-93-C-0004, SAIC, Menlo Park, CA. * May, E.C., Luke, W.L.W., Lantz, N.D. (1993). Phenomenological Research and Analysis. Final Report: 6.2 and 6.3, Contract MDA908-91-C-0037, SAIC, Menlo Park, CA. May, E.C., Luke W.L.W., Lantz, N.D. (1992). Phenomenological Research and Analysis. Interim Technical Report -- Contract MDA908-91-C-0037, SAIC, Menlo Park, CA. May, E.C., Luke, W.L.W. (1991). Technical Protocol for the MEG Investigation. SAIC, Menlo Park, CA. * May, E.C., Luke W.L.W. (1992). Phenomenological Research and Analysis. Interim Amerlean Institutés for Research B-3 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 a Appendix B: Bibliography Report: 6.2, 6.3, 6.4, Contract MDA908-91-C-0037, SAIC, Menlo Park, CA. * May, E.C., Luke W.L.W. (1992). Phenomenological Research and Analysis. Final Report 6.5: Support, Contract MDA908-91-C-0037, SAIC, Menlo Park, CA, * May, E.C., Luke W.L.W. (1992). Phenomenological Research and Analysis. Interim Report Covering 2/4/91 - 12/31/91, Contract MDA908-91-C-0037, SAIC, Menlo Park, CA. May, E.C., Luke, W.L.W., Trask, V.V., Frivold, T.J. (1989). Observation of Neuromagnetic Fields in Response to Remote Stimuli. Final Report -- Task 6.0.2, Covering the Period 1 October 1988 to 30 September 1989, SRI Project 1291, SRI, Menlo Park, CA. May, E.C., Luke, W.L.W., Trask, V.V., Frivold, T.J. (1990). Observation of Neuromagnetic Fields in Response to Remote Stimuli, Final Report (Rev.) -- Task 6.0.2, SRI Project 1291, SRI International, Menlo Park, CA. May, E.C., MacGowan, D., DeGraff, E. (1993). Phenomenological Research and Analysis. Periodic Status Report -- Covering 9/11/93 to 12/31/93, Contract MDA908-93-C-0004, SAIC, Menlo Park, CA. May, E.C., Pleass, C.M. (1987). A Remote Action Investigation with Marine Animals. Final Report -- Objective E, Task 2, SRI International, Menlo Park, CA. May, E.C., Puthoff, H.E. (1981). Feasibility Study on the Use of RV Detection Techniques to Determine Location of Military Targets. SRI International, Menlo Park, CA. May, E.C., Spttiswoode, S.J., James, C.C. (1994). Managing the Target Pool Bandwidth: Noise Reduction for Anomalous Cognition Experiments. SAIC, Menlo Park, CA. * May, E.C., Trask, V.V. (1988). Forced Choice Remote Viewing. Final Report -- Objective E, Task 4, Covering the Period'1 October 1987 to 30 September 1988, SRI Project 1291, SRI International, Menlo Park, CA. * May, E.C., Utts, J.M., Trask, V.V., Luke, W.W., Frivold, T.J., Humphrey, B.S. (1989). Review of the Psychoenergetic Research Conducted at SRI International (1973-1988). Final Report -- Task 6.0.1, Covering the Period 1 October to 15 February 1989, SRI Project 1291, SRI International, Menlo Park, CA. American Institutes for Research B-4 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 a Appeoneilx B: Bibliography Puthoff, H.E., Targ, R. (1978). Standard Remote-Viewing Protocol (Local Targets). SRI International, Menlo Park, CA. Trask, V.V., Lantz, N.D., Luke, W.W., May, E.C. (1989). Screening for Remote Viewing Talent. Final Report -- Task 6.0.5, Covering the Period 1 October 1988 to 30 September 1989, SRI Project 1291, SRI International, Menlo Park, CA. Trask, V.V., Lantz, N.D., Luke, W.L.W., May, E.C. (1989). Screening for Remote Viewing Talent. Final Report -- Task 6.0.5, Covering the Period 1 October 1988 to 30 September 1989, SRI Project 1291, SRI, Menlo Park, CA. Utts, J.M., May, E.C., Frivold, T.J. (1987). Intuitive Data Sorting. Final Report -- Objective I, Task 2, SRI International, Menlo Park, CA. Listings with '*" are from Ed May's top 10 list. American institutes for Research B-5 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 a SAIC Papers ts Phenomenological Research and Analysis Technical Proposal (U). (1993). MDA908-93-C-0004, Contract Modification Number P00002, SAIC, Menlo Park, CA. Phenomenological Research and Analysis Technical Proposal (U). (1993). MDA908-93-C-0004, Contract Modification Number P00003, SAIC, Menlo Park, CA. American Institutes for Research B-6 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 a Un SRI Papers I A Remote Viewing Evaluation Protocol (U). (1983). Final Report, SRI International, Menlo Park, CA. An Application Oriented Remote Viewing Experiment (U). (1988). Final Report, Covering the Period 1 May 1987 to April 1988, SRI Project 8339, SRI International, Menlo Park, CA. An Automated RV Evaluation Procedure (U). (1985). Final Report -- Covering the Period October 1983 to October 1984, SRI Project 7408-12, SRI International, Menlo Park, CA. Applications of Fuzzy Sets to Remote Viewing Analysis (U), (1988). Final Report -- Objective F, Task 1, Covering the Period 1 October 1987 to 30 September 1988, SRI Project 1291, SRI International, Menlo Park, CA. Audiolinguistic Correlations With The Quality Of Remote Viewing Sessions (U). (1982). Final Report -- Covering the Period October 1980 to October 1981, SRI International, Menlo Park, CA. Computer-Assisted Search (U). (1987). Final Report -- Objective D, Task 1 and Objective G, Task 1, Covering the Period 1 October 1985 to 30 September 1987, SRI Project 1291, SRI International, Menlo Park, CA. Co-Ordinate Remote Viewing (CRV) Technology 1981 - 1983 Three-Year Project. (1983). Draft Report. Countermeasures: A Survey and Evaluation (U). (1982). Final Report -- Covering the Period October 1980 to September 1981, SRI International, Menlo Park, CA. Enhanced Human Performance Investigation (U). (1988). Final Report, Covering the Period November 1983 to October 1985, SRI International, Menlo Park, CA. Enhanced Human Performance Investigation (U). (1987). Final Technical Report -- Covering the Period 1 October 1986 to 30 September 1987, SRI Project 1291, SRI International, Menlo Park, CA. American Institutes for Research B-7 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 So Appendix 8: Bibliography Feedback and Precognition Dependent Remote Viewing Experiments (U). (1987). Final Report -- Objective F, Tasks la and 1b, Covering the Period 1 October 1986 to 30 September 1987, SRI Project 1291, SRI International, Menlo Park, CA. Feedback and Target Dependencies in Remote Viewing Experiments (U). (1988). Final Report -- Objective E, Tasks 1 and 2, Covering the Period 1 October 1987 to 30 September 1988, SRI International, Menlo Park, CA. - Free World Psychoenergetics Research Survey (U). (1983). Final Report -- Covering the Period October 1982 to September 1983, SRI Project 4028-2, SRI International, Menlo Park, CA. Geophysical Effects Study (U). (1984). Interim Report -- Covering the Period 15 November 1983 to 15 July 1984, SRI Project 6600, SRI International, Menlo Park, CA. Geophysical Effects Study (U). (1984). Final Report -- Covering the Period 15 November 1983 to 15 December 1984, SRI Project 6600, SRI International, Menlo Park, CA. Location of Target in Space and Time. (1986). Interim Report -- Covering the Period 1 October 1985 to 30 September 1986, SRI International, Menlo Park, CA. Neurophysiological Correlates to Remote Viewing (U). (1988). Final Report -- Objective D, Task 1, Covering the Period 1 October 1987 to 30 September 1988, SRI Project 1291, SRI International, Menlo Park, CA. NIC Techniques (U). (1980). Quarterly Progress Report -- Covering the Period 1 October to 31 December 1979, SRI Project 7560, SRI International, Menlo Park, CA. Personnel Identification and Selection (U). (1984). Final Report -- Covering the Period 15 November 1983 to 15 December 1984, SRI Project 6600, SRI International, Menlo Park, CA. Photon Production (Chinese Replication) (U). (1985). Final Report -- Covering the Period October 1983 to October 1984, SRI International, Menlo Park, CA. Remote Viewing Evaluation Techniques (U). (1986). Final Report -- Covering the Period 1 October 1985 to 30 September 1986, SRI International, Menlo Park, CA. RV Reliability, Enhancement , and Evaluation (U). (1982). Final Report -- Covering American Institutes for Research B-8 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 nan Appendix 8: Bibliography the Period October 1981 to September 1982, SRI International, Menlo Park, CA. RV Reliability, Enhancement, and Evaluation (U). (1982). Final Report -- Covering the Period October 1980 to September 1981, SRI International, Menlo Park, CA. Special Orientation Techniques (U). (1980). Final Report -- Covering the Period 1 May 1979 to 31 March 1980, SRI International, Menlo Park, CA. Special Orientation Techniques: S-I, S-II, S-III (VU). (1984). Final Report -- Covering the Period 15 November 1983 to 15 December 1984, SRI Project 6600, SRI International, Menlo Park, CA. Special Orientation Techniques: S-V,S-VI (U). (1984). Final Report -- Covering the Period 15 November 1983 to 15 December 1984, SRI Project 6600, SRI International, Menlo Park, CA. Special Orientation Techniques: S-IV (U). (1984). Final Report -- Covering the Period 1 February 1983 to 30 April 1984, SRI Project 5590, SRI International, Menlo Park, CA. Targeting Requirements Tasks (U). (1981). Final Report -- Covering the Period October 1980 to October 1981, SRI International, Menlo Park, CA. Target Search Techniques (U). (1984). Final Report -- Covering the Period 15 November 1983 to 15 December 1984, SRI Project 6600, SRI International, Menlo Park, CA. American institutes for Research B-9 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 APPENDIX C ATTACHMENT 1 INTERVIEWER INSTRUCTIONS In this interview, you will be attempting to gather information bearing on the use of remote viewing in the intelligence community. Attached you will find the questions to be asked in the interview. You should begin the interview by noting that all information, particularly the respondents’ identity, will be treated as confidential. You should begin the interview by stating your name, clearance, and affiliation. After ensuring confidentiality, you should provide a brief description of the reasons for conducting these interviews, describing both the origins of the work, including program transfer, and the ensuing congressional mandate. You should also note the program evaluation is being conducted by the CIA in conjunction with a not-for-profit research organization. Once you have provided this background information, you may proceed to the interview questions. Twelve general questions are presented. Please try to work through each of these questions in turn. The early questions focus on background, the middle questions on process, and the final questions on evaluation. To help you follow this structure one intended to feed the discussion objective, it might be useful to ask the person being interviewed to hold their evaluation until the end of the interview. Once you ask a question, allow the persons being interviewed to answer the question in their own terms. After they have finished you might want to follow-up on their answers. You should only ask follow-up questions when the answer provided initially was not clear. Please sketch out the answers provided in the space below the question and prompts. Note, you should record only what the interviewee says not you impressions. If you are unsure about a point, ask the interviewee only after you have finished working through the question and prompts. It should take you about ninety minutes to complete this interview. All material should be returned to Dr. Michael Mumford at the American Institutes for Research. merican Institutes for Researe. =f Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 Appendix C: Interviewer Material nnn ATTACHMENT 2 INTERVIEW QUESTIONS Users: 1. Have you used the support of remote viewers? 2. Had you used remote viewers before? 3. What information did you request from the remote viewer? 4. What information did you receive from the remote viewer? 5. How did you use the remote viewing products? 6. Did the remote viewing product seem to confirm your initial approach? 7. Did you receive any subsequent information that confirmed/disconfirmed the remote viewing information (e.g., other intelligence sources)? 8. Could the remote viewing products be used without information from other sources? 9. Were the remote viewing products accurate? 10. How much relevant information was included in the remote viewing product? 11. Would you use this again and if so, under what circumstances? 12. Would you pay for the services of a remote viewer? Viewers: 1. What lead you to become interested in the remote viewing program? 2. How were you selected to become a remote viewer? 3. Were you provided with any training and if so, was it helpful? 4, How much familiarity did you have with the remote viewing research? 5. Was this research helpful in doing your work? American Institutes for Research C-2 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Ee Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 Appendix C: Interviewer Material 6. What were the major types of taskings you were assigned? 7. How much background information did you have about the nature of the target? 8. Was this background information useful in producing a viewing? 9. How did you typically go about generating viewings? 10. | When multiple viewers worked on a tasking how were you viewings similar to or different from each other? 11. What could have been done to improve the accuracy and usefulness of viewings? 12. What were the conditions that led to the best viewings? 13. What were the conditions that led to poor viewings? 14, How could the program be managed differently to improve the accuracy and usefulness of viewings? 15, What actions could be taken to make you a better viewer? Manager: 1. Why were you selected to manage the remote viewing program? 2. How did you identify potential users of the remote viewing services? 3. Why did users request viewings? 4, What were the major types of taskings requested by the users? 5. How many user groups request multiple viewings? 6. How did you assign viewers to taskings? 7. How much information did the viewers have about the nature or background of the tasking? 8. How useful was the research background in structuring the viewers’ activities? 9. What kind of information was typically provided to the viewers about the tasking? American nstltules for Research OSOSOSOSOSSOOTTCOC“C‘“CSC Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 nn Appendix C: Interviewer Material 10. 11. 12. 13. 14. 15. 16. How accurate was the information provided by the viewers relative to other sources of information? When were viewers particularly likely to produce useful and accurate products? What influenced users' acceptance of or use of the remote viewings? How often did viewings have a major influence on operational decisions? What organizational influences contributed to the successes of the program? What organizational influences limited the success of the program? How should the program be managed differently in the future? American Institutes for Research C-4 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 a Appendix C: interviewer Material ATTACHMENT 3 INTERVIEW REPORT STAR GATE OPERATIONAL USE INTERVIEW #1 DATE: JULY 7, 1995 CONTEXT: The Unit Commander in the organization had not previously used the services of remote viewers. A single tasking was his only experience in using remote viewing information. The decision to use remote viewers was in part based on contact with the Star Gate Program Director and awareness of publicized incidents where remote viewings were used in police cases. The primary motivation for use of the remote viewers appears to simply have been to try out a low cost approach that might pay off. TARGET: The target in this case was a person rather than a site. The target person was suspected of potential involvement in espionage. The primary evidence bearing on this assessmnent was access, finances, and reported comments of which the most important evidence was financial data. REQUEST: The remote reviews were asked to provide a variety of information about the target person. The requested information included descriptions of the person, likely travel locations, and events occurring during travel. Four, sequential, apparently "independent" remote viewings were obtained. NATURE OF INFORMATION: The four sequential viewings were provided and accompanied by reports, The information provided in these reports included both verbal descriptions and drawings. A high degree of agreement was not observed among the four remote viewing reports. Furthermore, the narrative descriptive information was provided in broad, highly ambiguous terms. USE OF INFORMATION: The information provided by these viewings was not held to be useful in any operational sense. The reasons stated for reaching this conclusion were: 1) the information was too broad and too vague to direct relevant observations: 2) crucial elements of the case, particularly financial concerns, did not appear in any of the reports; 3) the information provided could be interpreted in too many different ways; 4) hits were often stereotypic given the available cues in the tasking; 5) there were a large number of demonstrably wrong conclusions. Given the preceding reactions of the user, no attempt was made to use the information provided by the remote viewers. It did not in any way contribute to the actions taken in the case of interpretation of other available data. It is of note that the data apparently were used objectively without forcing them into preconceptions about the casc. American Institutes for Research C-5 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 Appendix C: Interviewer Material MERITS RELATED TO OTHER SOURCES: It was noted that the users relied more on other sources of information (e.g., financial, human, etc...) than the reports of the remote viewers. The viewings were, apparently, discredited due to the number of inaccuracies and failure to identify known key aspects of the case, UTILITY: The remote viewings were not held to be of substantial value due to the inaccurate described above. The user noted however, that would be given further consideration use of remote viewers if the "situation were desperate, no costs were entailed to the user, and the viewers were more intimately involved in the case for some period of time." It was further noted that remote viewers could be viewed as another source of manpower. In the case at hand, however, the viewings proved of no practical value, American institutes for Research C-6 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
a eeeeeesesesesei‘“Cs__N Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 Appendix C: Interviewer Material ATTACHMENT 3 INTERVIEW REPORT STAR GATE OPERATIONAL USER INTERVIEW #2 DATE: JULY 26, 1995 CONTEXT: The manager of this group has used the services of remote viewers a number of times. In the course of this interview, the unit manager stressed the importance of keeping an open mind. However, the primary motivation for using remote viewing was that it provided a way to obtain a simple, clear terms information that could not be easily obtained from other sources, particularly human intelligence, archival records, or financial records. This manager required analysis to provide targets on a regular basis. TARGET: Three major types of targets were involved, One set of targets consisted of people and their roles in criminal organizations. The second set of targets were ships or the location of material on ships. The third and final type of target was the site of manufacturing operations or shipping. Targets were selected by analysis based on current concerns and generally were described in terms of names and locations. REQUESTS: The type of information remote viewers were asked to provide was consistent with the nature of the targets. When the target was a person the viewers were given a name and asked to indicate the person's role and position in a criminal organization. When the target was a ship, they might be asked to indicate its location or type of cargo. When the target was a plant they might be asked to indicate what was being produced. NATURE OF INFORMATION: Multiple viewings were made of each target. The results of these viewings were provided in reports presenting both verbal descriptions and drawings. The report material was typically stated in rather broad terms and was synthesized by the analysts, The viewers had some knowledge of the target organizations and thcir operations but not the background of the particular tasking at hand. USE OF INFORMATION: The manager indicated that the information provided by the viewers was uscful for filling in background information about a person and his or her likely role in a criminal organization (e.g., who is Mr. Chin?). The information provided by the viewers was not deemed useful when they were required to address specific operational issues such as the location of objects on a ship. As a result, the information was not used to guide operations. Instead, it was used to identify the roles of individuals within targeted criminal organizations, Generally, this background information was felt to be accurate, although it did not permit immediate action. However, the manager being interviewed offered that some degree of accuracy could be expected if the viewers had a knowledge of the sponsoring organization American institutes for Researc. =f Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
eee Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 Appendix C: Interviewer Material and its areas of interest. The manager also noted that some analysts felt far more comfortable than others about using this type of information implying that some analysts had to be persuaded to use the information. MERITS RELATED TO OTHER SOURCES: The nature of the sponsoring organization was such that relatively few alternative sources of information were available. Much of the intelligence information used was indirect, such as shipping patters or records of business ownership. In comparison to this type of indirect information, the remote viewings were viewed as useful, particularly in identifying backgrounds and roles of people in criminal organizations when direct information was not available. UTILITY: Although the viewings were deemed useful for filling in background information, they were not deemed useful for operations. It was also pointed out that analysts doubts about the value of the source might limit use of the resulting information. Further, it was noted that a viable program could only be maintained if it were an in house Government activity. If this provision was met, the manager was willing to devote operational resources to pay for the services of remote viewers. American Institutes for Researc Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 mn Appendix C: Interviewer Maierlal ATTACHMENT 3 INTERVIEW REPORT STAR GATE OPERATIONAL USER INTERVIEW #3 DATE: AUGUST 3, 1995 CONTEXT: This interview consisted of a group manager and three analysts responsible for identifying lost or missing Department of Defense personnel, The group was interested in remote viewing as another possible tool for identifying the location of lost or missing personnel when more conventional methods (e.g., radios) could not be used. The remote viewings had not been used operationally. Instead, the group was using the services of the remote viewers in a series of experiments to determine whether it had any potential operational value. They had found out about the potential applications of remote viewing through research reports acquired by a fellow manager. These experiments were conducted with the cooperation of the remote viewing program manager, Both viewers and the program manager had met with members of this operational group of a number of times. TARGET: In these experiments, two major types of targets were involved: 1) the location of a person at a predetermined time; 2) the nature of the location or its physical characteristics. Targets were selected on the basis of convenience, typically in proximity to office, although more distant locations were sometimes selected. In the last experiment, the viewers were asked to indicate the map coordinates of a person who had moved to an area around the office site. REQUEST: The viewers were asked to indicate the location of the person serving as beacon and describe the location. The person playing the role of beacon evaluated accuracy. NATURE OF INFORMATION: Three viewers provided independent viewings at the designated time. The material was presented in a three to four-page report which included drawings and a verbal description of the location. Information contained in the reports often included a number of vague general statements (e.g., "There is water nearby"). The nature of the information was sufficiently broad and vague that the analysts typically had to force an interpretation. These interpretations often involved actions and locations occurring at another time in the day. It was only with this extended interpretation that hits were identified by the analysts. In the experiment where a specific map location was identified by the viewers, none of the viewers correctly identified the target site. In fact, most of the locations were not in the vicinity of the target site which was a park near the group's office. However, one viewer located the beacon at another nearby park and did describe general characteristics of the park, including trash cans, wooden steps, etc. American Institutes for Research C-9 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 an ee Appendix C: Interviewer Material USE OF INFORMATION: As noted above, these experiments were not intended to provide information for operational use. However, all members of the group stated that the information was too vague and ambiguous for operations, noting that unless specific map locations could be identified, the information could not be used in operational decision making. MERITS RELATED TO OTHER SOURCES: Because viewings were being used on an experimental basis, it was not possible to obtain direct comparisons of the viewings with other potential sources of information. However, all members of the group took the position that the information provided was so vague that it could not be used without other sources of information. If used at all, it would be treated as supplemental information that could not by itself provide a basis for operational decisions. UTILITY: Because the viewers could not identify specific locations in unambiguous terms, the group did not feel the procedures had any current operational utility. They did note, however, that if further research were to yield more specific consistent information from viewers, it might eventually have some utility. Even under these conditions, however, the information would have to be obtained under the auspices of a formal Government program to ensure that it was viewed as credible. American Institutes for Research C-76 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 nn Appendix C: Interviewer Material ATTACHMENT 4 INTERVIEW REPORT STAR GATE VIEWERS INTERVIEW DATE: AUGUST 4, 1995 CONTEXT: This interview was conducted with the three viewers employed in the Star Gate program at the time of its suspension. All three of the viewers attended the group interview. The viewers had worked in the program for at least five years. Generally, all viewers had in one way or another been affiliated with the sponsoring organization prior to becoming viewers.. They were recruited by more senior officials based on background factors indicating an interest in parapsychology or psychology more generally. SELECTION AND TRAINING: Systematic selection procedures or pretesting were not used in selecting two of the three viewers. The third viewer, however, was selected based on a series of tests given by a contractor organization. This viewer reported scoring relatively high in a relatively large pool of candidates. When viewers entered the program they were provided with formal training. The training included three component stages moving from simple to complex targets. They began with beacon-based viewing on site and progressed to operational targets. This initial training was six to 18 months long. Follow-up or refresher training did not occur. However, all of these viewers had initiated and maintained personal self-development program which included attempts to keep up with the literature in parapsychology in general and remote viewing in particular. With regard to the remote viewing research, the viewers noted that the research paradigm did not directly correspond to operational assignments in terms of conditions and requirements (e.g., the availability of a beacon). Thus a variety of different sources and techniques was used in self development and operational assignments aside form those found in the remote viewing research literature. TECHNIQUES: Each of the three viewers used different techniques to generate viewings. One viewer used coordinate viewing coupled with meditation. The second viewer relied on relaxation techniques and meditation. The third viewer relied on channelling and automatic writing. Although the viewers tended to rely on different techniques, they noted that these techniques were not necessarily incompatible. Further, it was pointed out that all of these techniques were physically and emotionally draining and resulted in products which were somewhat ambiguous or vague in nature. As a result, the ability of users to correctly interpret the results of a viewing was considered essential to successful application. Generally, the viewers felt that successful use was influenced by acceptance of the phenomenon. American Institutes for Research C-T1 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
a AO arma eee eA TSE EE a: 2 A le: eT: PPS crane Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 a Appendix D: Stargate Operational User Intertviews One of the RV's expressed a concern to the author that the protocols that were followed in conducting the RV process in response to the operational tasking were not consistent with those that are generally specified for the study of paranormal phenomena. Whether the claimed discrepancy was detrimental to the information derived by the RV's,or to its subsequent evaluation or use cannot be determined from the available data. The operational tasking noted earlier concerning activities in North Korea which required precognitive abilities on the part of the RV's provides an opportunity for a post-analysis by comparing the RV predictions against subsequent realities. Additional comparative data of this type is available from operational tasking during the period 11/90 through 1/91 regarding the Middle East situation (this tasking was not numerically evaluated). 6.0 SUMMARY FROM USER INTERVIEWS (U) Subsequent to the review and analysis of the numerically scored tasking described in the previous sections of this report, the author participated in interviews with representatives of all of the tasking organizations presented in Table 4-1 except Org. D. Only a brief summary of the results from those interviews is presented here; more detailed synopses are presented in Appendix B. In all cases except for Org. C, the interviewees were the actual personnel who had participated directly in the tasking and evaluation of the Star Gate program. For Org. C, the sole interviewee was the Chief of the Analysis Branch; the staff who defined the tasking and performed the evaluations was comprised of his lead analysts. American Institutes for Research ~ ~~ -17 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5
(eran Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5 a Appendix D: Stargate Operational User interviews information’, 2 however, the SG data provided was either not specific enough and/or not timely enough to task other assets for additional information. Some SG data was included in “target packages" given to field operatves- however, there was no audit trail so there is no evidence regarding the accuracy or use of these data. User also noted that classification prohibited data dissemination as did concerns about skepticism of others regarding the source and the potential for a subsequent negative impact on his organization. FUTURE USE OF SG SERVICES: User desires to continue using SG-type service if the program continues. In addition, the user stated that he would be willing to pay for the service if necessary. However, subsequent discussion indicated that his management would not fund the activity unless the credibility could be demonstrated better and the phenomenology legitimized. User went on to claim that only the sponsorship of a government agency could "legibmize " this activity and its application to operational problems. User believes that RV's working directly with his analysts on specific problems would not be beneficial due to the skepticism of his analysts and the deleterious impact that would have on the RV's. The views provided by the user - note none of the actual evaluators were present - appeared to be unique to him and his belief in the phenomenology, i.e., his remarks indicated that the use of this process was not actively supported by anyone else in his organization. The numerical evaluations of the 19 tasks performed in 1994/95 certainly do not indicate, on the average, either a high degree of accuracy or value of the data provided. I, 2User was unaware that the tasking organization and its primary mission were known to the RV's. Portions of the data provided by the RV's could have been predicted from this knowledge. American Institutes for Research D-22 Approved For Release 2002/05/22 : CIA-RDP96-00791R000200180005-5