Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Chapter Three: Research Reviews
Despite better controls and careful use of statistical inference, the investigators seem to be
getting significant results that do not appear to derive from the more obvious flaws of
previous research.
I have argued that this does not justify concluding that anomalous
cognition has been demonstrated.
However,
it does suggest that it might be worthwhile to
allocate some resources toward seeing whether these findings can be independently replicated.
If so, then it will be time to reassess if it is worth pursuing the task of determining if these
effects do indeed reflect the operation of anomalous cognition.
This latter quest will involve
finding lawful relationships between attributes of this hypothesized phenomenon and different
independent variables.
Both the scientific and operational value of such an alleged
phenomenon will depend upon how well the conditions for its occurrence can be specified
and how well
its functioning can be brought under control.
Both Professor Utts and I agree that the very first consideration is to see if the SAIC remote
viewing results will
still be significant when independent judges are used.
I understand Ed
May's desire to use a judge who
is very familiar with the response styles of the experienced
viewers.
However, if remote viewing is real, then conscientious judges, who are blind to the
actual targets, should
still be able to match the verbal descriptions to the actual targets better
than chance.
If this cannot be done, the viability of the case for remote viewing becomes
problematical. On the other hand, assuming that independent judges can match the
descriptions to the correct targets reasonably well, then
it becomes worthwhile to try to
independently replicate the SAIC experiments.
At this point we face some interesting questions.
Should we try to replicate the remote
viewing studies by using the same viewers, the same targets, and the same protocol? Perhaps
change only the experimenters, the judge, and the laboratory? At some point we would also
want to change the targets.
For completeness, we would also want to search for new, talented
viewers.
If independent replications confirm the SAIC findings, we still have a long way to go.
However, at this stage in the proceedings, the scientific community at large might be willing
to acknowledge that an anomaly of some sort has been demonstrated.
Before the scientific
community will go beyond this acknowledgment, the parapsychologists will have to devise a
positive theory of anomalous communication from which they can make testable predictions
about relationships between anomalous communication and other variables.
.American Institutes Jo? Research
~
3-70
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Chapter Three: Research Reviews
CONCLUSIONS
THE SCIENTIFIC STATUS OF THE SAIC RESEARCH PROGRAM
1 .
The SAIC experiments on anomalous mental phenomena are statistically and
methodologically superior to the earlier SRI remote viewing research as well as to
previous parapsychological studies.
In particular, the experiments avoided the major
flaw of non-independent trials for a given viewer.
The investigators also made sure to
avoid the problems of multiple stastical testing that was characteristic of much
previous parapsychological research.
2.
From a scientific viewpoint, the SAIC program was hampered by its secrecy and the
multiple demands placed upon
it.
The secrecy kept the program from benefiting from
the checks and balances that comes from doing research in a public forum.
Scrutiny
by peers and replication in other laboratories would accelerated the scientific
contributions from the program.
The multiple demands placed on the program meant
that too many things were being investigated with too few resources.
As a result, no
particular finding was followed up in sufficient detail to pin it down scientifically.
Ten experiments, no matter how well conducted, are insufficient to fully resolve one
important question, let alone the several that were posed to the SAIC investigators.
3.
Although,
I cannot point to any obvious flaws in the experiments, the experimental
program is too recent and insufficiently evaluated to be sure that flaws and biases have
been eliminated.
Historically, each new paradigm in parapsychology has appeared to
its designers and contemporary critics as relatively flawless.
Only subsequently did
previously unrecognized drawbacks come to light.
Just as new computer programs
require a shakedown period before hidden bugs come to light, each new scientific
program requires scrutiny over time in the public arena before its defects emerge.
Some possible sources of problems for the SAIC program are its reliance on
experienced viewers, and the use of the same judge—one who is familiar to the
viewers, for all the remote viewing.
4.
The statistical departures from chance appear to be too large and consistent to attribute
to statistical flukes of any sort.
Although I cannot dismiss the possibility that these
American Institutes Tor Research
3-7/
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Chapter Three: Research Reviews
rejections of the null hypothesis might reflect limitations in the statistical model as an
approximation of the experimental situation,
I tend to agree with Professor Utts that
real effects are occurring in these experiments.
Something other than chance
departures from the null hypothesis has occurred in these experiments.
5.
However, the occurrence of statistical effects does not warrant the conclusion that
psychic functioning has been demonstrated.
Significant departures from the null
hypothesis can occur for several reasons.
Without a positive theory of anomalous
cognition, we cannot say that these effects are due to a single cause, let alone claim
they reflect anomalous cognition. We do not yet know how replicable these results
will be, especially in terms of showing consistent relations to other variables.
The
investigators report findings that they believe show that the degree of anomalous
cognition varies with target entropy and the 'bandwidth' of the target set.
These
findings are preliminary and only suggestive at this time.
Parapsychologists, in the
past, have reported finding other correlates of psychic functioning such as
extroversion, sheep/goats, altered states only to find that later studies could not
replicate them.
6.
Professor Utts and the investigators point to what they see as consistencies between
the outcome of contemporary ganzfeld experiments and the SAIC results.
The major
consistency is similarity of average effect sizes across experiments.
Such consistency
is problematical because these average effect sizes, in each case, are the result of
arbitrary combinations from different investigators and conditions.
None of these
averages can be justified as estimating a meaningful parameter.
Effect size, by itself,
says nothing about its origin.
Where parapsychologists see consistency,
I see
inconsistency.
The ganzfeld studies are premised on the idea that viewers must be in
altered state for successful results.
The remote viewing studies use viewers in a
normal state.
The ganzfeld experimenters believe that the viewers should judge the
match between their ideation and the target for best results; the remote viewers believe
that independent judges provide better evidence for psi than viewers judging their own
responses.
The recent autoganzfeld studies found successful hitting only with dynamic
targets and only chance results with static targets.
The SAIC investigators, in one
study, found hitting with static targets and not with dynamic ones.
In a subsequent
study they found hitting for both types of targets.
They suggest that they may have
American Institutes Tor Research
3-72
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Chapter Three: Research Reviews
solution to this apparent inconsistency in terms of their concept of bandwidth.
At this
time, this is only suggestive.
7.
The challenge to parapsychology, if it hopes to convincingly claim the discover, of
anomalous cognition, is to go beyond the demonstration of significant effects.
The
parapsychologists need to achieve the ability to specify conditions under which one
can reliably witness their alleged phenomenon.
They have to show that they can
generate lawful relationships between attributes of this alleged phenomenon and
independent variables.
They have to be able to specify boundary conditions that will
enable us to detect when anomalous cognition is and is not present.
SUGGESTIONS FOR FUTURE RESEARCH
1.
Both Professor Utts and
I agree that the first step should be to have the SAIC
protocols rejudged by independent judges who are blind to the actual target.
2.
Assuming that such independent judging confirms the extra-chance matchings, the
findings should be replicated in independent laboratories.
Replication could take
several forms.
Some of the original viewers from the SAIC experiments could be
used.
However,
it seems desirable to use a new target set and several independent
judges.
OPERATIONAL IMPLICATIONS
.
Tthe current default assessment of the operational effectiveness of remote viewing is
fraught with hazards.
Subjective validation is well known to generate compelling, but
false, convictions that a description matches a target in striking ways.
Better, double
blind, ways of assessing operational effectiveness can be used.
I suggest at least one
way in the report.
2.
The ultimate assessment of the potential utility of remote viewing for intelligence
gathering cannot be separated from the findings of laboratory research.
American TnstlEiies Tor ’Research
3-73
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
'
Chapter Three: Research Reviews
THE REPLY
Response to Ray Hyman’s Report of September 11, 1995^
"Evaluation of Program in Anomalous Mental Phenomena"
Jessica Utts
Division of Statistics
University of California, Davis
Ray Hyman's report of September 11, 1995, written partially in response to my written report
of September
1, 1995 elucidates the issues on which he and I agree and disagree.
I basically
concur with his assessment, but there are three issues he raises with regard to the scientific
status of parapsychology to which I would like to respond.
1 .
"Only parapsychology, among the fields of inquiry claiming scientific status, lacks a
cumulative database (p.6)."
It is simply not true that parapsychology lacks a cumulative database.
In fact, the
accumulated database is truly impressive for a science that has had so few resources.
While
critics are fond of relating, as Professor Hyman does in his report, that there has been "more
than a century of parapsychological research (p. 7)" psychologist Sybo Schouten (1993,
p.316) has noted that the total human and financial resources devoted to parapsychology since
1882 is at best equivalent to the expenditures devoted to fewer than two months of research
in conventional psychology in the United States.
On pages 4 and 5 of their September 29, 1994 SAIC final report, May, Luke and James
summarize four reports that do precisely what Professor Hyman claims is not don in
parapsychology; they put forth the accumulated evidence for anomalous cognition in a variety
of formats.
Rather than dismissing the former experiments, parapsychologists build on them.
As in any area of science,
it is of course the most recent experiments that receive the most
attention, but that does not mean that the field would divorce itself from past work.
Quite to
the contrary, past experimental results and methodological weaknesses are used to design
better and more efficient experiments.
American Institutes for Research
~
3-7%
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
*•
Chapter Three: Research Reviews
As an example of the normal progress of inquiry expected in any area of science, the
autoganzfeld experiments currently conducted by parapyscholgists did not simply spring out
of thin air.
The original ganzfeld experiments followed from Honorton's observation at
Maimondies Medical Center, that anomalous cognition seemed to work well in dreams. He
investigated ways in which a similar state could be achieved in normal waking hours, and
found the ganzfeld regime in another area of psychology.
The automated ganzfeld followed
from a critical evaluation of the earlier ganzfeld experiments, and a set of conditions agreed
upon by Honorton and Professor Hyman.
The current use of dynamic targets in autoganzfeld
experiments follows from the observation that they were more successful than static targets in
the initial experiments.
The investigation of entropy at SAIC follows from this observation as
well.
This is just one example of how current experiments are built from past results.
2.
"Only parapsychology claims to be a science on the basis of phenomena (or a
phenomenon) whose presence can be detected only by rejecting a null hypothesis
(p.8)."
While
it is true that parapsychology has not figured out all the answers,
it does not differ
from normal science in this regard.
It is the norm of scientific progress to make observations
first, and then to attempt to explain them. Before quantum mechanics was developed there
were a number of anomalies observed in physics that could not be explained.
There are many
observations in physics and in the social and medical sciences that can be observed, either
statistically or deterministically, but which cannot be explained.
As a more recent example, consider the impact of electromagnetic fields on health. An article
in Science (Vol. 269, 18 August, p. 911) reported that "After spending nearly a decade
reviewing the literature on electromagnetic fields (EMFs), a panel of the National Council on
Radiation Protection and Measurements (NCRP) has produced a draft report concluding that
some health effects linked to EMFs—such as cancer and immune deficiencies—appear real and
warrant steps to reduce EMF exposure. „
Biologists have failed to pinpoint a convincing
mechanism of action."
In other words, a statistical effect has been convincingly established
and
it is not the responsibility of science to attempt to establish its mechanism, just as in
parapsychology.
American Institutes for Research
3-75
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Chapter Three: Research Reviews
As yet another example, consider learning and memory, which have long been studied in
psychology. We know they exist, but brain researchers are just beginning to understand how
they work by using sophisticated brain imaging techniques. Psychologists do not understand
these simple human capabilities, and they certainly do not understand other observable human
phenomena such as what causes people to fall in love.
Yet, no one would deny the existence
of these phenomena just because we do not understand them.
In any area involving the natural inherent in humans, science progresses by first observing a
statistical difference and then attempting to explain
it.
At this stage, I believe parapsychology
has convincingly demonstrated that an effect is present, and future research attempts should
be directed at finding an explanation.
In this regard parapsychology in on par with scientific
questions like the impact of electromagnetic fields on health, or the cross-cultural differences
in memory that have been observed by psychologists.
3.
"Parapsychology is the only field of scientific inquiry that does not have even one
exemplar that can be assigned to students with the expectation that they will observe
the original results (p.
18)."
I disagree with this statement for two reasons.
- First, I can name other phenomena for which
students could not be expected to do a simple experiment and observe a result, such as the
connection between taking aspirin and preventing heart attacks or the connection between
smoking and getting lung cancer.
What differentiates these phenomena from simple
experiments like splitting light with a prism is that the effects are statistical in nature and are
not expected to occur every single time.
Not everyone who smokes gets lung cancer, but we
can predict the proportion who will. Not everyone who attempts anomalous cognition will be
successful, but I think we can predict the proportion of time success should be achieved.
Since I believe the probability of success has been established in the autoganzfeld
experiments,
I would offer them as the exemplar Professor Hyman requests. The problem is
that to be relatively assured of a successful outcome requires several hundred trials, and no
student has the resources to commit to this experiment.
As
I have repeatedly tried to explain
to Professor Hyman and others, when dealing with a small to medium effect it takes hundreds
or sometimes thousands of trials to establish "statistical significance.
In fact, the Physicians
Health Study that initially established the link between taking aspirin and reducing heart
American Institutes for Research
—
~
3-76
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
:
Chapter Three: Research Reviews
attacks studies over 22,000 men.
Had
it been conducted on only 2,200 men with the same
reduction in heart attacks, it would not have achieved statistical significance.
Should[students
be required to recruit 22,000 participants and conduct such an experiment before we believe
the connection between aspirin and heart attacks is real?
Despite Professor Hyman’s continued protests about parapsychology lacking repeatability, I
have never seen a skeptic attempt to perform an experiment with enough trials to even come
close to insuring success.
The parapsychologists who have recently been willing to take on
this challenge have indeed found success in their experiments, as described m my original
report.
REFERENCE:
Schouten, Sybo (1993).
"Are we making progress?" In Psi Research Methodology. A Re-
examination, Proceedings of an International Conference, Oct 29-30, 1988, edited by
L. Coly and J. McMahon, NY:
Parapsychology Foundation, Inc., pgs. 295-322.
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Chapter Three: Research Reviews
Points of Agreement and Disagreement
The exchanges between Dr. Utts and Dr. Hyman has converged upon a number of
issues bearing on the remote viewing research, and parapsychological research in general.
In
this section of the report, we summarize the major points of agreement and disagreement
particularly as they pertain to the remote viewing laboratory experiments conducted as part of
the current program. By adopting this more narrow focus, we are intentionally bypassing the
evidence for remote viewing or other paranormal phenomena that have arguably been
demonstrated in other paradigms (e.g., the autoganzfeld studies) or prior to the present set of
laboratory studies.
Our position is that the charter for the present evaluation is to examine
the current program; evidence obtained by other researchers in other laboratories is not a
direct component of this program.
Before beginning this discussion, we feel
it important to recognize that, in our
opinion, both review papers are of exceptional quality.
One of Dr. Hyman's first comments
about Dr. Utts' review was that he considered it perhaps the best defense of parapsychological
research he had come across. We concur; likewise, we feel that Dr. Hyman's paper represents
one of the clearest expressions of the skeptic position we have seen.
At the outset, it should be noted that the two reviewers agreed far more than they
disagreed.
One central point of agreement concerns the existence of a statistically significant
effect:
Both reviewers note that the evidence accrued to date in the experimental laboratory
studies of remote viewing indicate that a statistically significant effect has been obtained.
Likewise, they agree that the current (e.g., post-NRC review) experimental procedures contain
significant improvements in methodology and experimental control.
The major remaining area of disagreement concerns attribution of causality.
It is Dr.
Hyman's position that existence of a statistically significant effect does not allow one to infer
that these laboratory experiments have provided an unequivocal demonstration of remote
viewing. A statistically significant effect might arise from many sources; more simply stated,
the results could have occurred for many different reasons.
Until the reasons for the results
can be pinpointed, one can not say that fully adequate evidence has been obtained for the
existence of any phenomenon, including paranormal phenomena such as remote viewing.
American Institutes lor "Research
——
3-7/
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
•
Chapter Three: Research Reviews
Typically, the process of identifying causes involves both eliminating competing explanations
and developing and testing, in Dr, Hyman's terms, "positive" hypotheses.
Dr. Hyman states
that these competing explanations have not been adequately eliminated; Dr. Utts believes that
they have.
Our conclusion is that, although the remote viewing research has made substantial
methodological progress in recent years, a central problem remains in the laboratory
experiments conducted as part of the present program.
The problem is that most if not all
significant findings have been obtained by using the same remote viewers, the same judge,
the same target set, and the same scoring procedures.
This characteristic of the remote
viewing research in the program raises the possibility of what has been termed
monomethod
bias" — a factor limiting the confidence we can have in drawing conclusions about the
generality utility of the phenomenon.
Basically, the concept here is that a specific method
could have some very subtle influences on performance that only appear over many
repetitions or trials; these influences could have nothing to do withh the phenomenon being
investigated. Recognition of this point is why both reviewers stress the need for independent
replication.
Another consequence of this experimental situation (the same remote viewers, etc.) is
that
it makes attribution of causation difficult. For example, one interpretation of the
significant outcomes obtained is that the judge has the ability to influence the results of the
remote viewing.
As unlikely as this seems, this interpretation cannot be ruled out unless
different judges are used.
Although this general problem was of some concern to the reviewers, both were more
concerned about one specific aspect of the problem:
The methods used in the current
program to study remote viewing involve having a judge assess the degree of correspondence
between the viewings and a set of targets.
In the current studies, this judge has typically been
the Principal Investigator, a person who has substantial familiarity with both the viewers and
the research paradigm. In addition to the potential confounding noted above, this kind of
procedure can potentially to result in "criterion contamination"— some unconscious,
nondeliberate influence on the results in a particular direction.
As a result, both reviewers
stress the point that cross-judge agreement needs to be demonstrated: the studies need to be
replicated using independent judges before strong statements can be made about the existence
American Institutes for Research
~
5-73
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Chapter Three: Research Reviews
of remote viewing.
Another issue where there seems to be some remaining disagreement is a variant of
the causation problem.
Methodological problems are, of course, not the only reason a
statistically significant effect might occur but still fail to provide evidence for the existence of
a phenomenon.
It is quite possible that an effect occurred for reasons other than the proposed
cause.
In other words, normal psychological processes rather than paranormal processes
might account for the effects obtained in the remote viewing studies.
Recognition of this fact
led both reviewers to stress the point that more attention needs to be given to the
identification of underlying causes in remote viewing research.
The reviewers however, differ somewhat in the way they would approach this issue.
Dr. Utts believes that the existence of the phenomenon has been adequately demonstrated;
therefore, the focus of future research should be on causative mechanisms — how the ability
works.
Dr. Hyman takes the position that competing hypotheses have not yet been
eliminated; thus, he stresses the continued need for replication in different laboratories, with
different investigators, etc., so that causal mechanisms, specifically paranormal mechanisms,
can be identified.
Our conclusion about this issue is direct:
If laboratory research is to continue, this
research must be conducted in such a way that causal mechanisms can be articulated, and that
also serves to rule out competing explanations.
A final point of disagreement has to do with the nature of the remote viewing
phenomenon itself.
Dr. Hyman argues that the effect has not been readily replicated in other
laboratory settings.
Instead,
it appears inconsistently, occurring only for certain people at
certain times.
He argues that the size of the observed effects are small, they do not
consistently emerge under certain specified conditions, and that a coherent pattern of results
has not been obtained in studies of remote viewing.
At the risk of misrepresenting his
position, the implication is that unless more is uncovered about the phenomenon,
it is not
likely to be a fruitful area for further research.
Dr. Utts counters with a compelling argument,
noting that many effects are statistical in nature, occurring only rarely and under certain
conditions.
This should not preclude further research into the causes and operation of the
phenomenon.
American Institutes Tor Research
—
3-80
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
•
Chapter Three: Research Reviews
We agree with Dr. Utts that the conditions under which a phenomenon occurs should
not determine whether or not it is worth exploring and pursuing scientifically. On the other
hand, Dr. Hyman's observations point to a broader problem.
If weak, inconsistent effects are
obtained in a laboratory-based research program,
it is open to question whether the
phenomenon under investigation has any practical value.
The anomaly may occur from time
to time but it occurs in such an erratic fashion that it cannot be used to guide ones' actions.
In this regard, we find some agreement between the two reviewers, both of whom question
whether the kind of strong consistent effects needed for practical applications have been
demonstrated.
Conclusions from the Expert Reviews
In the preceding section we noted the points of agreement and disagreement among
the reviewers. We tried furthermore to clarify and reconcile these points of agreement and
disagreement.
With this background in mind, we now return to the basic questions presented
to the reviewers and attempt to draw some firm conclusions about the implications to be
drawn from this research review.
The first question presented to the reviewers was whether the evidence indicated the
presence of a statistically significant effect.
This question was answered in a straightforward
fashion: the reviewers agreed that, considered broadly, statistically significant effects have
been obtained in these studies.
It appears that viewers' descriptions produce hits more
frequently than would be expected by chance.
;
The second question presented to the reviewers considers the nature of these effects.
The question to be answered was whether the effects could be attributed to paranormal
phenomena.
In this regard, the reviewers disagreed, with Dr. Utts arguing positively and Dr.
Hyman negatively.
Our conclusion from the discussions is that direct evidence has not been
provided indicating that this paranormal ability of the remote viewers is the source of these
effects.
Attribution in general
is difficult to demonstrate; for the present set of laboratory
experiments, a primary concern for us is that the same viewers, the same judge, the same
target set, and the same scoring procedures were repetitively used.
This makes
it difficult or
impossible to localize the source of the phenomenon.
American Jnstlulies Tor Research
3-81
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
•
Chapter Three: Research Reviews
The third question presented to the panel asked whether we have obtained an adequate
understanding of the phenomenon. Do we know how the ability, if it exists, works?
Here
it
is clear that the present research program has failed to identify mechanisms explaining the
source of these effects.
The fourth and final question presented to the reviewers was whether the research
provides support for intelligence gathering operations.
Here the magnitude of the observed
effects, their consistency and replicability, and the need for subjective interpretation all seem
to argue against potential applications.
Taken as a whole, these answers lead to relatively straightforward general conclusions:
•
The laboratory research conducted as part of the present program has identified
a statistically significant "anomaly."
•
However, the experiments have not provided a convincing demonstration that a
paranormal ability is involved.
.
The research studies have not identified the nature and source of the effect.
•
There is no evidence that the phenomenon would prove useful in intelligence
gathering.
American TnstlEiies Tor Researcn
3-82
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
4.
Evaluating the Utility of Remote Viewing in
Intelligence Operations
In the preceding section, we presented several conclusions bearing on evidence for the
existence of remote viewing and on the nature of the phenomenon.
In this section, we will,
for the moment, assume that the phenomenon does indeed exist.
This leads to questions of
the utility of the phenomenon for intelligence applications. As noted earlier, our evaluation of
remote viewing in terms of its value to the intelligence community had three components:
.
assessment of task requirements in relation to the boundary conditions believed
to influence application of the phenomenon,
•
interviews with various participants in the program, and
.
user assessments of the information provided by the remote viewing process.
In this section, we consider each of these components.
Research on Boundary Conditions
Historically, the literature on paranormal phenomena, and on remote viewing in
particular, has paid relatively little attention to establishing boundary conditions — more
specifically, the conditions under which effects are and are not observed.
This point has been
made in numerous earlier critiques of parapsychological research (Druckman & Swets, 1988;
Hyman, 1994; Swets & Bjork, 1990).
In response to these critiques, several investigators
have initiated research intended to provide more clear-cut evidence bearing on the conditions
influencing when effects are and are not observed in parapsychological studies (Bern &
Honorton, 1994; Honorton, 1994).
Research on remote viewing reflects this same general
trend.
In fact, many of the studies conducted in recent years have expressly examined this
issue (May, Luke, & James, 1994; May, Luke & Lantz, 1993).
American Institutes for Research
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Chapter Four:
Evaluating Utility in intelligence Operations
Senders and Distance. When one considers the remote viewing paradigm, one
boundary condition immediately comes to the fore with regard to intelligence applications.
During information gathering,
it is unlikely that a "beacon" or sender will be at the site.
In
fact, the interviews with end users and the remote viewers indicate that only in the case of
missing persons is a sender likely to be involved in information collection.
If senders are
necessary for the phenomenon to occur, its utility would be dramatically reduced.
The laboratory experiments conducted as part of the current program avoid this issue;
since the targets are photographs kept in another part of the laboratory, the equivalent of a
"sender" would be the laboratory assistant who retrieves the photograph (or the computer that
selects
it).
If one considers a "sender" as either a transmitter of information or an active
participant in the remote viewing, this paradigm essentially does not have one.
However, a
recent study by May, Spottswood, and James (1994) has specifically addressed this issue.
Their findings indicate that significant remote reviewing effects are obtained even when
senders are not available.
Another boundary condition that might influence the utility of remote viewing in
intelligence collection is the distance of the target from the viewer.
More specifically, for
many of the tasks involved in intelligence applications,
it may be impossible to place the
viewer anywhere near the target.
Thus, if the accuracy of remote viewing is limited by
distance, this condition may, in turn, restrict its value.
However, the work of May and his
colleagues (May, Luke, & James, 1994; May, Luke, & Lantz, 1994) indicates that remote
viewing may not be related to distance.
•
Targets. A set of studies conducted by Lantz, Luke, and May (1994) examined
another boundary condition of particular importance when assessing potential applications of
the remote viewing phenomenon.
Most recent research on remote viewing has been based on
a fixed set of "static" targets, drawn from National Geographic photographs.
Lantz, Luke,
and May varied target type by including a new set of targets covering a wider range of novel
stimuli, referred to as "dynamic" targets.
Perhaps another relevant aspect of these new
targets is that the remote viewers in the standard paradigm are very familiar with the National
Geographic photographs, but were unfamiliar with the dynamic targets.
The researchers
found that significant effects were obtained for static but not dynamic targets.
American TnsUtutes Tor Research
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
•
Chapter Four:
Evaluating Utility In Intelligence Operations
This finding is of some importance from an operational intelligence perspective.
In
Appendix C, the interview reports, end users and remote viewers describe the operational
targets presented to remote viewers.
These targets represent a diverse set of potentially nove
stimuli ranging from the location of ships to the likely background characteristics of a person.
The findings obtained in the Lantz, Luke, and May study suggest that accurate viewings are
less likely to be obtained for these dynamic "real-world" targets, a finding which, if
replicated, represents a severe constraint on the utility of remote viewing to the intelligence
community.
Even
if, as May, Spottswood, and James (1994) point out, those effects can
e
explained in terms of target "bandwidth," there is no reason to assume that intelligence targets
will conform to a limited range of predefined bandwidths.
Information Requirements and Specificity.
Another set of boundary conditions
examined in recent studies considers the nature of the targets and the information to be
assessed by judges.
Intelligence consumers place a premium on the availability of specific,
concrete, potentially verifiable information. On the other hand, the information obtained from
remote viewings tends to be stated in broad, vague terms; a critical role must be played by
judges or intelligence analysts, who must interpret the remote viewers’ reports.
Recognizing this problem, a series of studies (May, Luke, & Lantz, 1993) were
initiated intended to provide more concrete judgments framed in terms of the presence or
absence of specific categories of objects, a technique referred to as "binary coding."
Results
from these studies indicated that when relatively specific information was used in judging
matches, weak or insignificant effects were obtained.
>
Specificity, as a boundary condition, can also be assessed in terms of targets.
This
issue was examined in a meta-analysis (May, Luke, & Lantz, 1993) contrasting the effects
obtained for different types of targets.
They found that effect sizes were related to target
complexity, with more dynamic targets showing larger effect sizes.
Incidentally, and
seemingly in contradiction to the Lantz, Luke, and May (1994) study cited above, they found
insignificant effects for static photograph targets.
This pattern of effects, if intrinsic to the
remote viewing process, would seem to limit applications to those involving more complex
targets requiring less specific descriptions.
Furthermore,
it may also represent a significant
limitation on applications of the remote viewing phenomenon where specific information is
required.
American Institutes for Research
”
~
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
:
chapter Four:
Evaluating Utility In Intelligence Operations
Another aspect of specificity has to do with the degree of accuracy of the remote
viewers' reports.
Most remote viewing reports contain a large number of potentially
interpretable components, an unknown percentage of which may not be related to the target.
The end users cannot know which components are or are not related.
This makes
it difficult
for end users, particularly if they are not highly trained, to separate valid information from
irrelevant information, a problem that may prohibit effective operational applications of the
results of the viewing process.
Feedback.
Perhaps the most widely accepted boundary condition applying to remote
viewing is the need for feedback.
It is commonly held that the accuracy of viewings depends
on the viewer receiving feedback following production of the viewing.
In fact, one
explanation of the remote viewing phenomenon is that viewings represent a form of
precognition where the viewers identify future outcomes or events that they themselves will
experience; in effect, they "communicate" with themselves across time (May, Lantz, &
Piantineda, 1994).
The important point here is that in "real-world" information gathering
situations,
it may be impossible for viewers to acquire the kind of feedback needed for
accurate viewings.
Remote Viewer Training. A final boundary condition frequently noted in the
literature on remote viewing pertains to the nature and development of the presumptive
remote viewing ability.
Typically, remote viewing is held to represent a relatively rare talent
not widely distributed in the population.
Recent studies have suggested that associative
learning techniques may contribute to improved performance on remote viewing tasks (May,
Luke & Lantz, 1993).
However,
little evidence is available indicating that the capacity for
producing accurate viewings can be systematically developed.
This has implications for
information gathering operations, since the burden would fall on the ability of the sponsoring
agency to recruit and/or select individuals who already possess remote viewing skills.
Other Boundary Conditions.
In the preceding discussion, we have limited
commentary to those boundary conditions directly addressed in the literature.
However, a
number of other boundary conditions exist that might limit potential applications of the
remote viewing phenomenon.
One such limitation on potential applications of the
phenomenon involves the assessments of the information provided by the remote viewers.
Typically, when judgmental data, such as target matches, are to be used in decision making,
American THstiUHes Tor Research
”
*
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
;
Chapter Four:
Evaluating Utility In Intelligence Operations
evidence is required indicating that end-user analysts arrive at much the same decisions.
If
analysts (or trained judges) cannot arrive at similar decisions,
it is impossible to know what
kind of actions should be taken.
Unfortunately, evidence is not available indicating the
degree of interrater, or cross-judge, agreement.
Accordingly,
it is difficult to assess whether
different analysts or judges will reach similar conclusions given the same data.
These
differences in assessments, if indeed they exist, may set another important limitation on
potential applications of remote viewings.
Another boundary condition that has potential importance for remote viewing
operational applications pertains to the degree of prior practice.
In the remote viewing
research studies conducted as part of the present program, both judges and the viewers have
had years working together on a rather limited target set.
Putting aside for the moment
potential cueing and rule-based learning effects resulting from long periods of practice, in
most field settings viewers and judges or analysts will have only limited opportunities to work
together. As a result,
it is open to question whether the findings obtained in these laboratory
experiments can be extended to field settings.
Boundary Conditions: Conclusions
The issue of boundary conditions is part of a broader problem in generalizing research
findings to operational settings.
Typically, generalization from laboratory to operational
settings is contingent on three requirements (Cook & Campbell, 1979).
First, there is a need
to demonstrate, that an observed effect is sufficiently robust that
it can be observed using
different methods.
Second, explicit verified causal explanations, reflecting an understanding
of the source of observed effects, need to be provided.
Third, alternative causal explanations,
for example judges and viewers learning implicit rules, must be ruled out.
The laboratory
experiments in the current research program conducted to date have relied on one method,
judgmental matching; plausible causal mechanisms has not been identified; and studies have
not been conducted drawing out competing explanations.
Lacking this evidence, drawing
strong conclusions concerning the potential operational applications of remote viewing is at
best tenuous, and at worst misleading.
Taken as a whole, prior laboratory experiments examining the boundary conditions
related to remote viewing have clearly provided an important foundation for establishing the
American Institutes Tor "Research
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Chapter Four:
Evaluating Utility In Intelligence Operations
These problems with the consistency and accuracy of viewings were also apparent
in the end
user interviews.
In these interviews, viewings were found to be too broad and vague; the
viewings failed to provide the concrete, specific information needed for actionable
intelligence.
Further, there were indications that its potential usefulness was limited to
information which could be acquired in other ways.
Thus, the evidence accrued from research, interviews, and user assessments all indicate
that the remote viewing phenomenon has no real value for intelligence operations at present.
In fact, given the findings obtained to date and the nature of intelligence operations, one must
question whether any further applications can be justified without major theoretical and
practical advances in our understanding of the phenomenon, assuming
it exists at all.
American Tnstltutes Tor Research
4-75
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
5.
Conclusions
In the preceding sections of this report, we have presented a variety of evaluative data
bearing on the existence of the paranormal phenomenon known as remote viewing and its
potential applications in intelligence gathering.
This multifaceted evaluation effort was
structured to ensure a fair, unbiased evaluation of both the research program and its
intelligence applications.
As is the case with any objective, relatively sophisticated program
evaluation effort, many pieces of evidence bearing on different aspects of the program have
been presented.
In this section, we summarize the basic conclusions flowing from this
evaluation effort.
Summary of Key Findings
Two expert reviewers, one known to be a sophisticated advocate of the study of
paranormal phenomena and one viewed as a fair-minded skeptic, reviewed the laboratory
experiments conducted as part of the current program that bear on the existence of the remote
viewing phenomenon.
They focused primarily on recent, better-controlled laboratory studies,
drawing from other sources as needed to ensure a comprehensive evaluation of the research
literature.
Although the reviewers disagreed on some points, on many points they reached
substantial agreement.
The first important point of agreement concerns the existence of a statistically
significant effect, which leads to the following finding:
•
A statistically significant effect has been observed in the recent laboratory
experiments of remote viewing.
However, the existence of a statistically significant effect did not lead both reviewers to the
conclusion that this research program has provided an unequivocal demonstration that remote
viewing exists. A statistically significant effect might result either from the existence of the
phenomenon, or, alternatively, to methodological artifacts or other alternative explanations for
the observed effects.
American Institutes Jo7 Research
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Appendix A: Reviewer Vitas
Additional Positions and Exnerienfig!
Mar
1 949-Jul 1949
Statistician for antihistamine study, Allergy Fund of Boston
Jan 1953-Jul 1953
Consultant to Systems Coordination Division, Naval Research
Laboratory, Washington, D.C.
jul 1974-Aug 1974
Subcommittee on Preparing Problems and Examples for Committee on
the Mathematical Training of Social Scientists, Social Science Research
Council
Oct 1955-Jan 1956/Oct 1956-Jan 1957/Oct 1957-Jan 1958
Director of Statistical Workshop
for Psychology Department, Clark University
Oct 1957-Jun 1958
Statistical Consultant, Age Center of New England
1970-1975
Statistical Consultant, Roseburg VA Hospital
Some Current or Recent Positions;
Executive Council, Committee for the Scientific Investigation of Claims of the Paranormal
Editorial Board, The Skeptical Inquirer
Member of National Research Council's Committee on Techniques for the Enhancement of
Human Performance (1985-1991)
Publications;
BOOKS 7 MONOGRAPHS:
Bush, R.R., Abelson, R.P., & Hyman, R. (1956).
Mathematics. for psychologists;
examples
and problems . New York:
Social Science Research Council.
Vogt E.Z., & Hyman, R. (1979, Revised Edition).
Water Witching U.S.A, Chicago:
Univ. of
Chicago Press.
Hyman, R. (1960).
Some experiments in creativity . New York:
General Electric Company
(101 pages).
2A Partial listing.
American Jnstitutes Tor 'Research
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Appendix A: Reviewer Vitas
Proceedings of the Porapsychological Association Annual Convention, 235-266
11.
1985
Johnson, W.O., J.M. Utts, and L.M. Pearson.
Bayesian robust
estimation of the mean.
The Journal of the Royal Statistical Society
Series C (Applied Statistics), 35(l):63-72
12.
1986
Utts, J.M. The Ganzfel Debate:
a statistician's perspective.
Journal ofParapsychology, 50:
393-402
13.
1986
Utts, J.M.
Comment on "Computers in statistical research."
Statistical Sciuence,
1(4):
437-439
14.
1986
Johnson, W.O., L.M. Pearson, and J.M. Utts. A Monte Carlo
comparison of Bayesian estimators and trimmed means.
Journal
of Statistical Computation and Simulation, 25:167-192
15.
1986
Utts, J.M. and E.C. May. An exact method for combining P-values.
Research
in Parapsycholgy, 99-103, Scarecrow Press, Metuchen, N.J.
16.
1987
G.P. Hansen and J. Utts.
Use of both sum of ranks and direct hits
in free-response psi experiments.
Journal ofParapsycholgy,
51:321-335
17.
1987
Utts, J.
Psi, statistics, and society.
Behavioral and Braiun
Sciences, 10:615-616
18.
1988
Utts, J.
Successful replication versus statistical significance.
Proceedings of the Porapsychological Association, 31:148-162
19.
1988
Humphrey, B.S., E.C. May and J. Utts.
Fuzzy set technology in
the analysis of remote viewing, Proceedings of the Parapsychology
Association, 3 1
: 3 7 8 -394
20.
1988
Palmer, J.A., C. Honorton and J. Utts.
Reply to the National
Research Council study on parapsychology. Journal of the
American Society for Psychical Research, 83(1):3 1-49. (Also
published in Proceedings of the Parapsychology Association,
31:424-451
21.
1988
Utts,
J.
Successful replication versus statistical significance.
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
••
Appendix B: Bibliography
Lantz, N.D., Kierman, R.J.
(1986).
Neuropsychological Assessment of Participants m
Psychoenergetic Tasks.
Final Report - Objective C, Task 5, SRI International, Menlo Par
:,
CA.
Lantz, N.D., Luke, W.L.W., May, E.C.
(1994).
Target and Sender Dependencies in
Anomalous Cognition Experiments.
SAIC, Menlo Park, CA.
*
Lantz, N.D., Luke, W.L.W., May, E.C.
(1994).
Target and Sender Dependencies in
Anomalous Cognition Experiments.
Journal of Parapsychology ,
285-302.
Lantz, N.D., May, E.C.
(1988).
Mass Screening for Psychoenergetic Talent Using a
Remote Viewing Task.
Final Report - Objective B, Task 1, Covering the Period
1 October
1987 to 30 September 1988, SRI Project 1291, SRI International, Menlo Park, CA.
Luke W.L.W., Frivold, T.J., May, E.C., Trask, V.V.
(1989). A Prototype Analysis
System For Special Remote Viewing Tasks.
Final Report - Task 6.0.3, Covering the Period
1 October 1988 to 30 September 1989, SRI Project 1291, SRI International, Menlo Park, CA.
*
Luke, W.L.W., May, E.C.
(1993).
Phenomenological Research and Analysis.
Interim
Technical Report - Covering 30 December 1992 thru 30 April 1993, Contract
MDA908-93-C0004, SAIC, Menlo Park, CA.
Luke, W.L.W., May, E.C.
(1993).
Travel Report: The Third Meeting of The Chinese
Society of Somatic Science (CSSS), Bejing China.
Contract MDA908-91-C-0037, SAIC,
Menlo Park, CA.
*
May, E.C.
(1989). An Application Oriented Remote Viewing Experiment.
Final
Report — Covering the Period
1 May 1988 to April 1989, SRI Project 2740, SRI
International, Menlo Park, CA.
May, E.C.
(1986).
Enhanced Human Performance Investigation.
Final Technical
Report - Covering the Period
1 July 1986 to 15 November 1986, SRI Project 1291, SRI
International, Menlo Park, CA.
May, E.C.
(1988).
Enhanced Human Performance Investigation (U).
Final Technical
Report ~ Covering the Period
1 October 1987 to 30 September 1988, SRI Project 1291, SRI
International, Menlo Park, CA.
May, E.C.
(1993).
Phenomenological Research and Analysis.
Periodic Status Report
American Tnstiiuies Tor Research
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
'
Appendix B: Bibliography
- Covering 12/16/92
- 04/05/93, Contract MDA908-91-C-0037, SAIC, Menlo Park, CA.
May, E.C,
(1992).
Phenomenological Research and Analysis.
Periodic Status Report
- Covering 6/19/92
- 09/11/92, Contract MDA908-91-C-0037, SAIC, Menlo Park, CA.
May, E.C.
(1992).
Phenomenological Research and Analysis.
Periodic Status Report
- Covering 9/11/92 - 11/16/92, Contract MDA908-91-C-0037, SAIC, Menlo Park, CA.
May, E.C.
(1992).
Phenomenological Research and Analysis.
Periodic Status Report
- Covering 3/31/92 - 06/09/92, Contract MDA908-91-C-0037, SAIC, Menlo Park, CA.
May, E.C.
(1992).
Phenomenological Research and Analysis.
Periodic Status Report
- Covering 06/11/92 - 12/16/92, Contract MDA908-91-C-0037, SAIC, Menlo Park, CA.
May, E.C.
(1992).
Phenomenological Research and Analysis.
Periodic Status Report
- Covering 10/01
- 11/30/91, Contract MDA908-91-C-0037, SAIC, Menlo Park, CA.
May, E.C.
(1991).
Research of Anomalous Mental Phenomena.
Periodic Status
Report - Covering 2/4
- 9/30/91, Contract MDA908-91-C-0037, SAIC, Menlo Park, CA.
May, E.C., Lantz, N.D., Piantineda,
J.
(1994).
Feedback Considerations in
Anomalous Cognition Experiments.
SAIC, Menlo Park, CA.
*
May, E.C., Luke W.L.W. (1992).
Phenomenological Research and Analysis.
Final
Report:
Support, Contract MDA908-91-C-0037, SAIC, Menlo Park, CA.
*
May, E.C., Luke, W.L.W., James, C.L.
(1994).
Phenomenological Research and
Analysis.
Final Report — Contract MDA908-93-C-0004, SAIC, Menlo Park, CA.
*
May, E.C., Luke, W.L.W., Lantz, N.D.
(1993).
Phenomenological Research and
Analysis.
Final Report:
6.2 and 6.3, Contract MDA908-91-C-0037, SAIC, Menlo Park, CA.
May, E.C., Luke W.L.W., Lantz, N.D. (1992).
Phenomenological Research and
Analysis.
Interim Technical Report -- Contract MDA908-91-C-0037, SAIC, Menlo Park, CA.
May, E.C., Luke, W.L.W.
(1991).
Technical Protocol for the MEG Investigation.
SAIC, Menlo Park, CA.
*
May, E.C., Luke W.L.W.
(1992).
Phenomenological Research and Analysis.
Interim
American Institutes Jo7 Research
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
•
Appendix B:
Bibliography
Report:
6.2, 6.3, 6.4, Contract MDA908-91-C-0037, SAIC, Menlo Park, CA.
*
May, E.C., Luke W.L.W.
(1992).
Phenomenological Research and Analysis.
Final
Report 6.5:
Support, Contract MDA908-91-C-0037, SAIC, Menlo Park, CA.
*
May EC
Luke W.L.W.
(1992).
Phenomenological Research and Analysis.
Interim
Report Covering 2/4/91
- 12/31/91, Contract MDA908-91-C-0037, SAIC, Menlo Park, CA.
May, E.C., Luke, W.L.W., Trask, V.V., Frivold, T.J.
(1989).
Observation of
Neuromagnetic Fields in Response to Remote Stimuli.
Final Report - Task 6.0.2, Covering
the Period
1 October 1988 to 30 September 1989, SRI Project 1291, SRI, Menlo Park, CA.
May, E.C., Luke, W.L.W., Trask, V.V., Frivold, T.J.
(1990).
Observation of
Neuromagnetic Fields in Response to Remote Stimuli.
Final Report (Rev.) - Task 6.0.2, SRI
Project 1291, SRI International, Menlo Park, CA.
May, E.C., MacGowan, D., DeGraff, E.
(1993).
Phenomenological Research and
Analysis.
Periodic Status Report - Covering 9/11/93 to 12/31/93, Contract
MDA908-93-C-0004, SAIC, Menlo Park, CA.
May, E.C., Pleass, C.M.
(1987). A Remote Action Investigation with Marine
Animals.
Final Report - Objective E, Task 2, SRI International, Menlo Park, CA.
May, E.C., Puthoff, H.E.
(1981).
Feasibility Study on the Use of RV Detection
Techniques to Determine Location of Military Targets.
SRI International, Menlo Park, CA.
May, E.C., Spttiswoode, S.J., James, C.C.
(1994).
Managing the Target Pool
Bandwidth:
Noise Reduction for Anomalous Cognition Experiments.
SAIC, Menlo Park,
CA.
*
May, E.C., Trask, V.V.
(1988).
Forced Choice Remote Viewing.
Final Report
Objective E, Task 4, Covering the Period
1 October 1987 to 30 September 1988, SRI Project
1291, SRI International, Menlo Park, CA.
*
May, E.C., Utts, J.M., Trask, V.V., Luke, W.W., Frivold, T.J., Humphrey, B.S.
(1989).
Review of the Psychoenergetic Research Conducted at SRI International (1973-1988).
Final Report - Task 6.0.1, Covering the Period
1 October to 15 February 1989, SRI Project
1291, SRI International, Menlo Park, CA.
American institutes for 'Research
-m
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Appendix D: Stargate Operational User Intertvlews
OPERATIONAL TASKS THAT HAVE
NUMERICAL EVALUATIONS
Tasking
valuatoi Remote Viewer & Scores
Totals
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5
Appendix D: Stargate Operational User Intertviews
OPERATIONAL TASKS THAT HAVE
NUMERICAL EVALUATIONS
i
Doc.
Date
Tasker
Evaluator Remote Viewer & Scores
Totals
]
Org.
No.
1A
IV
2A
2V
3A
3V
_yj
Si
M
258 8/3/94
Org. C
1.0
3.0
2.0
3.0
I—
257 8/1/94
Org. C
WKWM 3.0
5.0
3.0
5.0
256 7/28/94
Org. C a 2.0
3.0
rani 4.0
252 7/18/94
Org. C
WM3MEEH 4.0
2.0 wm93
251
7/15/94
Org. C
mmmmmoi 3.0
1.0
3.0
Elf 3.0
131
1-1
243 5/31/94
Org. C
3.0
3.0
5.0
4.0 umEE9I
242 5/25/94
Org. C
1.5 EM
1.5
3.0
240 5/17/94
Org. C
BM
BRif 4.0
235 4/18/94
Org. C
3.0
4.0
3.0
3.0
RIM 4.0
234 4/14/94
Org. C
2.0
3.0
5.0
3.0 wm
233 4/11/94
Org. C
3.0
3.0
3.0
To~]Rif 3.0
229 3/29/94
Org. C
kliMI 4.0
2.0
4.0
5.0
4.0
228 3/28/94
Org. C
1.0
3.0
4.0
E2B 3.0
227 3/24/94
iszMsm
3.0
3.0
4.0
5.0
Rif 3.0
226 3/22/94
Org. C
5.0 OS 5.0
4.0
2.0
3.0
225 3/21/94
Org. C
HMWM 3.0
3.0 HEM 3.0
259 7/15/94
Org. C
5.0 mm
2.0
2.0
262 8/23/94
Org. C
4(7)
6.0
4.0
ism 5.0
287 4/3/95
Org. C
2.0
4.0
1.0
4.0 i
Score sums*
52.5
Kail
EM 65.0
KEIilmi
No. of entries*
_
ii
Him
Avq score*
2.8
3.5
3.5 mm 3.6
3.5
Ora. B
270
11/3/94 mmm
I3WESB
Em 4.0
271
11/3/94
Org. B
3.0 um
5.0
4.0
273
11/3/94
4.0
5.0
gif 5.0
267
11/3/94 msnm
3.0
ism
Rif 4.0
268
11/3/94
Org. B
3.0
4.0
4.0
3,0
5.0
4.0
269
11/3/94
Org. B
3.0
3.0
5.0 am
272
11/3/94
Org. B
3.0
Score sums*
18.0
21.0
12.0
13.0 Ram 26.0 wmK»1
No. of entries-
KQmm
Avp score* mmESI 4.0
3.2
4.5 3ESIg)Bl
Table 4-2
American Institutes for Research
-m
Approved For Release 2002/05/22
: CIA-RDP96-00791R0002001 80005-5