Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
'
Chapter Four:
Evaluating Utility In Intelligence Operations
The manager noted that none of the user groups had used the viewings as a primary
basis for operational decisions.
Instead, they were using the service to explore potential
applications of the technique or, alternatively, as a source of supplemental information.
Roughly half of the groups contacted agreed to participate in the program on an exploratory
basis.
During his interview, the manager stated that recruitment of clients was in large
measure due to the Congressional mandate. He noted however, that acceptance of the
program was limited by the controversy surrounding remote viewing.
The research
component was seen as potentially helpful in addressing this issue, although
it did not
contribute much to operational management.
The manager felt that the program s long-term
results, particularly in foreign assessment, would be enhanced by declassifying the research
work and assigning responsibility for it to another government organization such as the
Department of Justice or the National Science Foundation.
Along similar lines, he suggested
that remote viewing services be provided on a contract basis whenever the need for them
arose.
Conclusions
Before turning to the broader conclusions flowing from these interviews, certain
limitations inherent in this particular evaluative effort should be mentioned.
First, the
evaluation was limited to recent operations.
Although this factor reduces and focuses the
range of information considered,
it does not help to ensure the accuracy of the information
reported.
Indeed, if previous managers or the viewers themselves changed viewer reports,
then relying on this body of information could potentially be misleading with respect to the
remote viewing phenomenon or to the operational use of remote viewing.
Second,
it should
be recognized that interviews generate qualitative information that does not permit us to draw
strong quantitative conclusions.
Third,
it is clear that this report necessarily focuses on actual
operations and not on operations as they might occur under ideal conditions.
Even bearing these points in mind, we believe that the interviews have a number of
significant implications.
The most important implication pertains to the nature of the
information provided by the remote viewing process and the potential applications of this
information in intelligence operations.
The information provided by viewings:
TUnerlcan Institutes Tor Research
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
•
Chapter Four: Evaluating Utility In Intelligence Operations
•
is vague and general in nature;
•
is not consistent across independent viewings;
•
lacks specific content consistent with known facts of the case; and
•
includes a large amount of irrelevant, often erroneous, information.
Intelligence operations are contingent on the availability of relatively specific information
which is reliable, consistent, and potentially verifiable.
The lack of specifics apparent in the
viewings may well be an intrinsic characteristic of the process — a point noted by the remote
viewers.
Nonetheless, this vagueness and the limited agreement evident in viewers' reports
make it impossible to apply this information in decision making.
One potential strategy for
addressing this issue would be to seek more specific information in viewings.
However,
existing laboratory research provides little support for the likely success of this approach.
Alternatively, one might explicitly train users to work with this kind of broad, rather vague
information in such a way that they would attend to both its strengths and its weaknesses.
However, this kind of effort would require a strong commitment from sponsoring
organizations and unambiguous, high-level support for the program.
In addition to the problems associated with the reliability and vagueness of the
information provided, the viewings typically contain a large amount of irrelevant information
and often fail to capture key aspects of the case.
These characteristics of the viewings reduce
user confidence in the information provided.
Further, the nature of this information is such
that, the burden is effectively placed on the user in sorting out what is and is not relevant.
Under these conditions
it is quite possible that the information provided by viewings will lead
users down a number of blind alleys, thereby resulting in misallocation of intelligence
resources.
Alternatively,
it may do
little more than reinforce existing preconceptions and
stereotypes about the targets — a trend evident in the user interviews.
This observation brings us to the accuracy issue.
Although the viewings tended not to
be accurate with regard to concrete specifics, a characteristic which held true regardless of the
type of tasking, the users felt that they were more accurate in describing broad background
factors involved in the case.
The problem here, of course,
is that this outcome may simply
"American Institutes Tor Research
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
'
Chapter Four:
Evaluating Utility In Intelligence Operations
reflect the availability of background information and the logical use of relevant analytic cues.
This problem was, in fact, noted by the viewers when they indicated that they modified
reports to make them consistent with know background factors.
This observation suggests
that the viewers may not be providing any more information than could be obtained from
perceptive analysis working without the aid of remote viewing techniques.
Still another limitation on the operational value of the information being provided
pertains to its source.
Remote viewing is a controversial phenomenon.
Accordingly, routine
operational use of the resulting information across multiple units in the intelligence
community may well be contingent on a compelling demonstration of the phenomenon and
broad acceptance of the resulting findings.
Without this acceptance, users will discount the
information provided and attempts to force analysts to use this information are likely to cause
conflict.
If, however, comparable information were provided by the viewings,
it could be
argued that the program was more cost-effective than conventional techniques.
The fact that the information being provided is too vague and unreliable to permit
operational application suggests that the viewing program is of limited value to the
intelligence community.
This conclusion, however, pertains to the current program.
It is, of
course, always possible that a substantially restructured and redirected research effort along
with a substantial additional investment of resources might yield better results.
Given the
costs involved, however, and the need for specific, reliable information, any investment along
those lines should be contingent on successful demonstration of not only the existence of the
phenomenon but also
its ability to consistently produce specific accurate information.
Our foregoing observations of the interview results can be summarized as follows.
•
The viewing process has not provided information of adequate specificity and
reliability for use in intelligence gathering.
•
The process is not widely accepted and is not seen as essential to intelligence
gathering.
•
The process is often used experimentally because the needed information is
difficult to obtain.
American Institutes for "Research
4m1
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
i
Chapter Four:
Evaluating Utility In Intelligence Operations
•
Accuracy is limited to broad superficial characteristics of the case and may
simply reflect good logical analysis.
•
The available research does not directly support operations and new types of
research are needed to justify current operational use.
User Assessments
A third source of evidence used to evaluate operational utility consisted of analyses of
user feedback.
This feedback was in the form of assessments of product (i.e., the remote
viewers' reports) accuracy and value collected from end users.
This information was collected
by a representative of the U.S. Government in the Spring and Summer of 1995.
The results
of these analyses have been compiled in a report; this report is included as Appendix D, we
summarize the findings and implications below.
The program office asked each user submitting tasks to evaluate the accuracy and
value of the viewings.
Evaluations were available for forty viewings obtained in 1994 and
1995.
Accuracy (i.e., "Is the information accurate?") was rated on a six-point scale with
indicating high degrees of accuracy.
Value (i.e., "What is the value of the source's
information?")
was rated on a
five-point scale where a
1 indicated major significance, while
a 5 indicated no value.
It should be noted that separate ratings were obtained for each
viewer's report on each tasking requested by an organization.
The average accuracy rating was 3.0 across viewers and tasks, indicating that there
might "possibly" be some accurate information provided.
The average value rating was 3.5
(out of 5) indicating that the information was of relatively low value.
This pattern of results
is consistent with the interview findings.
Remote viewings were better at capturing broad
background information rather than the concrete, specific information needed for intelligence
operations.
Bearing this general conclusion in mind, a number of other questions about the remote
viewings can be addressed by these data.
For example, one important question is whether
any particular remote viewer did particularly well.
The accuracy scores of all three viewers
were within a tenth of a standard deviation from the mean, while value scores were within a
American THstitutes Tor 'Research
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Chapter Four:
Evaluating Utility In Intelligence Operations
quarter of a standard deviation.
Thus,
it appears that no one viewer performed exceptionally
well or poorly.
Another question is whether the remote viewers performed better on some tasks than
others.
Because tasks differed by the nature of the organization, this question can be
addressed by contrasting the evaluations obtained from different organizations. Two
organizations, one concerned with tracking people and one concerned with combat
intelligence, indicated that the information provided was of substantially greater accuracy and
value than was reported by the other user organizations.
In the case of the two organizations
providing positive evaluations,
however, the viewers had substantial general background
information concerning the targets, a finding suggesting that the ratings of outcomes of
viewings may be dependent on the amount of pre-existing background information.
These findings are noteworthy because they confirm a conclusion derived from the
interviews.
More specifically, viewings apparently were of limited accuracy, with accuracy
being linked to the availability of general background information.
Such background
information, however, was not believed to have much operational value.
Information Gathering Applications:
Operational Conclusions
The findings obtained in this evaluation of the intelligence applications of remote
viewing lead us to the conclusion that remote viewing as used in the present program has
limited value for the intelligence community as an information gathering technique.
The
basic considerations that lead us to this conclusion are:
•
Conditions under which significant effects are observed in experimental
laboratory research are, for the most part, unlikely to occur in intelligence
operations.
Not only will feedback be unavailable, but the target pool will
typically be unconstrained.
•
Information provided by the remote viewing technique tends to be vague and
ambiguous and
it appears difficult, if not impossible, to consistently obtain
accurate information from the remote viewers across a range of targets.
"American institutes lor Research
-
*
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Chapter Five: Conclusions
It is with regard to the explanation for these effects that the two reviewers differ most
clearly.
One reviewer argues that the procedures used in recent studies rule out many, but not
all, methodological explanations.
The other reviewer argues that the consistency of the results
obtained across experiments strongly suggests the existence of the paranormal phenomenon.
We concluded that
•
The experimental research conducted as part of the current program does not
unambiguously support the interpretation of the results in terms of a
paranormal phenomenon.
Both reviewers agreed that one important methodological problem has not yet been
addressed.
Specifically, only one judge — apparently the Principal Investigator — was used
in assessing matches throughout these experimental studies.
As a consequence, there is no
evidence for agreement across independent judges as to the accuracy of the remote viewings.
Failure to provide evidence that independent judges arrive at similar conclusions makes
it
difficult to unambiguously determine whether the observed effects can be attributed to the
remote viewers
1 (paranormal) ability, to the ability of the judge to interpret ambiguous
information, or to the combination or interaction of the viewers and the judge. Furthermore,
given the Principal Investigator's familiarity with the viewers, the target set, and the
experimental procedures,
it is possible that subtle, unintentional factors may have influenced
the results obtained in these studies.
Thus, until
it can be shown that independent judges
agree, and similar effects are obtained in studies using independent judges,
it cannot be said
that adequate evidence has been provided for existence of the remote viewing phenomenon.
.
Both reviewers agree that no compelling explanation has been provided for the
observed effects.
One reviewer considers the investigation of the determinants of remote
viewing as the next necessary step, but does not see it as essential to continue to conduct
experiments designed solely to demonstrate the existence of the phenomenon.
The other
reviewer argues that without identifying causal mechanisms and explicitly providing evidence
that alternative explanations cannot account for the observed effects, we cannot say we have
convincing evidence for the existence of a phenomenon,
Essentially, this position holds that
an observed effect may arise for many reasons; to say a phenomenon has been demonstrated
we must know the reasons for its existence.
American Institutes for Research
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Chapter Five: Conclusions
Our conclusion is that at this juncture it would be premature to assume that we have a
convincing demonstration of a paranormal phenomenon.
In fact, until a plausible causal
mechanism has been identified, and competing explanations carefully investigated, we cannot
interpret the set of anomalous observations localized to one laboratory with one set of
methods.
Given these observations, and the methodological problems noted above, we must
conclude that
Adequate experimental and theoretical evidence for the existence of remote
viewing as a parapsychological phenomenon has not been provided by the
research component of current program, A significant change in focus and
methods would be necessary to justify additional laboratory research within
the current program.
This is not to say definitively that paranormal phenomena do not exist.
At some point
in time, adequate evidence might be provided for the existence of remote viewing.
With this
point in mind, we considered the potential applications of remote viewing in intelligence
gathering.
The first consideration involves the conditions under which remote viewing occurs and
if those conditions constrain its application for intelligence purposes.
Prior research suggests
that distance is not a constraint and indeed that a sender or "beacon" may not be necessary.
However, other characteristics of intelligence gathering
indicate that remote viewing is of
little value.
Intelligence operations do not provide targets of a fixed bandwidth, rather, targets
and target types are highly variable.
Moreover, the apparent necessity for feedback to the
remote viewers would preclude its use in intelligence gathering operations.
Finally,
intelligence information is most valuable if it is concrete and specific, and reliably
interpretable.
Unfortunately, the research conducted to date indicates that the remote viewing
phenomenon fails to meet those preconditions.
Therefore, we conclude that
•
Remote viewing, as exemplified by the efforts in the current program,
has
not been shown to have value in intelligence operations.
This point was also graphically illustrated in the user interviews, where
it was found that
remote viewings have never provided an adequate basis for "actionable" intelligence
American Institutes lor Research
~~
5 ‘‘-
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
|
—
Chapter Five: Conclusions
operations — that is, information sufficiently valuable or compelling enough so that action
was taken as a result.
If a phenomenon does not contribute to intelligence operations,
it is
difficult to see what justification exists for its continued application.
This is particularly true
in the case of remote viewing, where a large amount of irrelevant, erroneous information is
provided and little agreement is observed among viewers' reports.
Particularly troublesome from the perspective of the application of paranormal
phenomena is the fact that the remote viewers and project managers reported that remote
viewing reports were changed to make them consistent with know background cues.
While
this was appropriate in that situation,
it makes
it impossible to interpret the role of the
paranormal phenomenon independently.
Also,
it raises some doubts about some well-
publicized cases of dramatic hits, which, if taken at face value, could not easily be attributed
to background cues.
In at least some of these cases, there is reason to suspect, based on both
subsequent investigations and the viewers' statement that reports had been "changed" by
previous program managers, that substantially more background information was available
than one might at first assume.
Give these observations,
it is difficult to argue that available
evidence justifies application of remote viewing in intelligence operations.
In summary, two clear-out conclusion emerge from our examination of the operational
component of the current program.
First, as stated above, evidence for the operational value
of remote viewing is not available, even after a decade of attempts.
Second,
it is unlikely
that remote viewing — as currently understood — even if existence can be unequivocally
demonstrated, will prove of any use in intelligence gathering due to the conditions and
constraints applying in intelligence operations and the suspected characteristics of the
phenomenon. We conclude that:
•
Continued support for the operational component of the current program is
not justified.
American Institutes lor Research
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
REFERENCES
Bern, D.J. & Honorton, C. (1994) Does PSI Exist?
Replicable Evidence for an Anomalous Process of
Information Transfer, Psychological Bulletin. 115, 4-18.
Cook, T. D., & Campbell, P.T. (1979) Quasi-experimentation:
Design and Analysis Issues for Field
Settings.
Chicago, 11; Rand Me Nally
Druckman, D., & Swets, G.A. (Eds.) (1988) Enhancing Human Performance;
Issues* Thsoriesumd
Techniques .
Washington, DC:
National Academy Press.
Honorton, C. (1985) Meta-Analysis of PSI Ganzfeld Research: A Response to Hyman.
Journal of
ParaPsychology, 49, 51-91.
Hyman, R., (1994) Anomaly or Artifact? Comments on Bern and Honorton.
Psychological Bulletin,
115. 19-24.
James, L.R., Muliak, S.A., & Brett, J.M. (1982).
Causal Analysis:
Assumptions, Models, and Data.
Beverly Hills, CA: SAGE
Lantz, N. D., Luke, W.L., & May, E.C. (1994) Target Sender Dependencies
in AnomalouS-Cognitign
Experiments .
Menlo Park, CA:
Science Applications International Corporation.
May, E.C., Lantz, N.D., & Piantineda, J. (1994) Feedback Considerations in Anomal ous . Cogn ition
Experiments .
Menlo Park, CA:
Science Applications International Corporation.
May, E.C., Luke, W.L., & James, C.C. (19941 Phenomological Research and Analysis-
Menlo Park,
CA:
Science Applications International Corporation.
May, E.C., Luke, W.L., & Lantz, N.D. H9931 Phenomological Research and An alysis. Menlo Park,
CA:
Science Applications International Corporation.
May, E.C., Spottiswoode, S.J. & James C. C. (1994) Managing the Target..P.QC>l Bandwidth;. Noise
Reduction for Anomalous Cognition Experimen ts.
Menlo Park, CA:
Science Applications
International Corporation.
Messick, S. (1989) Validity.
In R.L. Linn (Ed). Educational Measurement (PP 13-104). New York,
Macmillian
Swets, J.A., & Bjork, R.A. (1990) Enhancing Human Performance: An Evaluation of New Age
Techniques Considered by the U.S. Army.
Psychological Scien ce, 1, 85-96.
American Institutes Tor Research
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
APPENDIX A: REVIEWER VITAS
CURRICULUM VITAE
September 1994
Name :
Ray Hyman
Present position;
Professor of Psychology
Department of Psychology
University of Oregon
Eugene, Oregon
97403
Date and place of birth:
June 23, 1928
Chelsea, Massachusetts
Academic degrees'.
Positions held;
1950 A.B., Boston University
Phi Beta Kappa
Honors in Psychology
1952 M.A., The Johns Hopkins University
1953(FEB) PhD., The Johns Hopkins Univ.
Jul 1953-Jun 1961
Assistant Professor of Social Psychology, Harvard University
Jul 1958-Jun 1961
Consultant in Behavioral Research, General Electric Company
Sep 1961 -Aug 1964
Associate Professor of Psychology, University of Oregon
Sep
1 964-
Professor of Psychology, University of Oregon
Sep 1967-Aug
1968 Fulbright-Hays Research Scholar, University of Bologna, Italy
Sep 1976-Dec 1976/Jun 1977-Sep !977/Jun 1978-Sep 1978 NSF Faculty Fellowship
in
Science Applied to Societal Problems
Sep 1982-Jun 1983
Visiting Professor of Psychology, Stanford University (Thomas Welton
Stanford Chair for Psychical Research)
'This
is a very abbreviated and somewhat modified version of my full resume.
American Jnstltutes for Research
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5