Approved For Release 2001/03/07
: CIA-RDP96-00791R0002001 00003-3
t
Journal of Parapsychology, Vol. 54, September 1990
ADVANCES IN REMOTE-VIEWING
ANALYSIS
By Edwin C. May, Jessica M. Utts, Beverly S. Humphrey,
Wanda L. W. Luke, Thane J. Frivold, and Virginia V. Trask
ABSTRACT: Fuzzy set technology is applied to the ongoing research question of
how to automate the analysis of remote-viewing data. Fuzzy sets were invented to
describe, in a formal way, the subjectivity inherent in human reasoning. Applied
to remote-viewing analysis, the technique involves a quantitative encoding of target
and response material and provides a formal comparison. In this progress report,
the accuracy of a response is defined as the percent of the intended target material
that is described correctly. The reliability is defined as the percent of the response
that was correct. The assessment of the remote-viewing quality
is defined as the
product of accuracy and reliability, called the figure of merit. The procedure
is
applied to a test set of six remote-viewing trials. A comparison of the figures of
merit with
the
subjective assessments of 37 independent
analysts, shows good
agreement. The fuzzy set technology is also used to provide a quantitative defini-
tion of target orthogonality.
Human analysts are commonly used
to evaluate free-response
data. Although there are many variations, the basic idea
is that an
analyst, who
is blind
to the
actual
result,
is presented with a re-
sponse and a number of target possibilities, one of which
is the in-
tended
target. The analyst’s task
is
to decide what
is the best
re-
sponse/target match, and frequently includes rank-ordering the
targets from best to worst correspondence with the response.
It
is
beyond the scope of this report to provide a critical review of the
extensive literature on this topic.
One aspect, however, of this type of evaluation
is that analysts
are required to make global judgments about the overall match be-
tween a complex target (e.g., a photograph of a natural scene) and
an equally complex response (e.g., written words and drawings). In
a recent book, Dawes (1988) has discussed various decision algo-
rithms in general and the difficulty with global techniques, such as
those used
in rank-order evaluation,
in
particular.
1 According
to
Dawes, the research results suggest that global decisions of this type
are not as good as those based on smaller subelements that are later
1 We are indebted
to Professor D. Bern, Cornell University, for directing us to
this valuable source of information.
Approved For Release 2001/03/07
: CIA-RDP96-00791R0002001 00003-3
! Approved For Release 2001/03/07
: CIA-RDP96-00791R0002001 00003-3
I
194
The Journal of Parapsychology
\
!
combined.
(See Dawes,
1988, chap.
10,
for references
to the
re-
search.) Humans appear to be capable of deciding what the appro-
priate variables should be in complex decision processes, but they
have proved to be unreliable at combining these variables to arrive
at a single decision. Linear algorithms are consistently better at this
latter task. Therefore,
it seems prudent to develop evaluation tech-
niques that are less sensitive to global decision processes and rely on
combinations of more restrictive decisions.
Honorton (1975) has pointed out an additional difficulty inher-
ent
in
a
global rank-order approach. Asking an
analyst
to rank-
order
a small
set of target
possibilities converts the free-response
experiment into a forced-choice one, at least on the part of the an-
alyst.
It
is obvious that in doing so, much quantitative information
is
lost.
For example,
a near
perfect correspondence between
re-
sponse and target will receive only as much “credit” as one that just
barely allowed an analyst to discriminate among the possibilities.
If multiple analysts are used, addition problems arise concerning
interanalyst
reliability.
If an individual analyst judges a number of
responses in a series, within-analyst consistency becomes an individ-
ual problem.
To address these
difficulties, various computer-automated pro-
cedures have been suggested
in an attempt
to reduce the
inter-
analyst reliability while increasing within-analyst consistency. For ex-
amples,
see Honorton (1975), Humphrey, May, Trask, and
Thomson (1986), Humphrey, May, and Utts (1988), Jahn, Dunne,
and Jahn
(1980), May
(1983), May, Humphrey, and Mathews
(1985), and Targ, Puthoff, and May (1977).
In
this paper we present the current status of an ongoing re-
search topic. We are not yet ready to propose that the techniques
described here be used for free-response analysis; however, we hope
to inspire the community to develop a proper set of subvariables so
that
the
problems
inherent
in
global
decision
processes
can
be
avoided.
Finally, we present a successful application of the mathematical
techniques for quantifying target orthogonality for a complex target
pool.
i
|
Background
\
Substantial progress has been made
in methods for evaluating
remote-viewing experiments since the publication of the
initial re-
mote-viewing (RV)
effort
at SRI
International
(Puthoff & Targ,
Approved For Release 2001/03/07
: CIA-RDP96-00791R0002001 00003-$
Approved For Release 2001/03/07
: CIA-RDP96-00791R0002001 00003-3
*
Advances in Remote-Viewing Analysis
195
1976). This paper outlines some of the progress and presents the
details for one particular method.
Two basic questions are inherent in the analysis of any remote-
viewing data, namely, how
is the target defined, and how
is the re-
sponse defined.
In a typical outbound RV experiment, definitions of target and
response are particularly
difficult to achieve. The protocol for such
an experiment
dictates
that an experimenter
travel
to some ran-
domly chosen location at a prearranged time; a viewer’s task
is to
describe that location. One method of trying to assess the quality of
the RV descriptions in a series of trials
is to require that an analyst
visit each of the sites and attempt to match responses to them. While
standing at a site, the analyst has to determine not only the bounds
of the
site, but also the
site
details that are
to be included
in the
analysis. For example,
if the target location was the Golden Gate
Bridge, the analyst would have to determine whether the buildings
of downtown San Francisco, which are clearly and prominently vis-
ible from the bridge, were to be considered part of the target. The
RV response
to the Golden Gate Bridge
target could be equally
troublesome, because responses of this sort are typically 15 pages of
dream-like free associations. A reasonable description of the bridge
might be contained in the response;
it might be obfuscated, how-
ever, by a large amount of unrelated material. How
is an analyst to
approach this problem of response definition?
The
first attempt
at SRI
at quantitatively defining an RV
re-
sponse involved reducing the raw transcript to a series of declarative
statements
called concepts (Targ et
al.,
1977).
Initially,
it was de-
cided that a coherent concept should not be reduced to
its compo-
nent parts. For example, a small red VW car would be considered a
single concept rather than four separate concepts, small,
red, VW,
and
car . Once a
transcript had been “conceptualized,” the
list of
concepts constituted, by definition,
the RV response. The analyst
rated the concept
lists against the
sites. Although the response was
well defined by this method, no attempt was made to define the tar-
get site.
In
1982, a procedure was developed to define both the target
and response material (May,
1983). It became evident that before a
site can be qualified, the overall remote-viewing goal must be clearly
defined.
If the goal
is simply to demonstrate the existence of the
2 Although the term remote viewing
is used throughout
this paper, the
analysis
techniques can easily be applied to any free-response data.
Approved For Release 2001/03/07
: CIA-RDP96-00791R0002001 00003-3
I
Approved For Release 2001/03/07
: CIA-RDP96-OO791ROOO2O01 00003-3
196
The Journal of Parapsychology
RV phenomenon, then anything that
is perceived at the site
is im-
portant. But if the goal is to gain specific information about the RV
process, then
possibly
specific items
at the
site are important
whereas others remain insignificant.
In
1984, work began on a computerized evaluation procedure
(May et
al.,
1985), which underwent significant expansion and
re-
finement during 1986 (Humphrey et
al.,
1986). The mathematical
formalism underlying
this procedure
is known
as
the “figure of
merit” (FM)
analysis. This method
is predicated on descriptor
list
technology, which represented a significant improvement over ear-
lier “conceptual analysis” techniques, both in terms of “objectifying”
the analysis of RV data and in increasing the speed and efficiency
with which evaluation can be accomplished. Humphrey’s technique,
which was based on the pioneering work of Honorton (1975) and
its expansion by Jahn, Dunne, and Jahn (1980), was to encode tar-
get and response material in accordance with the presence or ab-
\
sence of specific elements.
It became increasingly evident, however, that this particular ap-
plication of descriptor lists was inadequate in providing discrimina-
tors that were “fine” enough to describe a complex target accurately,
and unable to exploit fully the more subtle or abstract information
content of the RV response. To decrease the granularity of the RV
evaluation system, therefore, a new technology would have to allow
the analyst a gradation of judgment about target and response fea-
tures rather than the hard-edged (and rather imprecise) all-or-noth-
I
ing binary determinations. Requiring an analyst to restrict subjective
judgment to single elements rather than
to complete responses
is
I
consistent with the research reported by Dawes (1988).
A preliminary survey of various disciplines and their evaluation
methods (spanning such diverse fields as
artificial intelligence,
lin-
guistics, and environmental psychology) revealed a branch of math-
!
ematics, known as “fuzzy set theory.”
j
i
Fuzzy Set Concepts
j
Fuzzy set theory was chosen as the focal point of the RV analyt-
ical techniques because
it provides a mathematical framework for
i
modeling situations that are inherently imprecise. Because
it
is such
j
an important component in the analysis, a brief tutorial will be pre-
sented to highlight its major concepts.
s We wish to thank S. James P. Spottiswoode and D. Graff, CE, for directing us
to the fuzzy set literature and for many helpful discussions.
j
Approved For Release 2001/03/07
: CIA-RDP96-00791R0002001 00003-3
Approved For Release 2001/03/07
: CIA-RDP96-00791R0002001 00003-3
Advances in Remote-Viewing Analysis
197
12345678
•••
•••
Population
in
100 Thousands
Figure
1. The fuzzy set “kind-of-small” cities.
In traditional set theory
(i.e., crisp sets), an element either
is or
is not a member of a
set. For example, the crisp set of cities with
population equal to or greater than
1,000,000 includes New York
City, but not San Francisco. This set would also not include a city
with a population of 999,999. The problem
is obvious. There
is no
real difference between
cities with
populations of 1,000,000 and
999,999, yet one
is in the set and the other is not. Humans do not
reason
this way;
therefore, something other than crisp
sets
is
re-
quired to capture the subjectivity inherent in RV analysis.
Fuzzy set theory introduces the concept of degree of membership.
Herein lies the essence of its applicability to the modeling of impre-
cise concepts.
For example,
if we consider the
size of a
city, we
might define certain fuzzy sets , such as very small cities or kind-of-small
cities. Using kind-of-small cities as a fuzzy set example, we might sub-
jectively assert that a city with a population of 100,000
is definitely
such a city, but a city with a population of 400,000
is only a little bit
like a kind-of-small city. As depicted in Figure
1, fuzzy set theory al-
lows us to assign a membership value between 0 and
that repre-
sents our best subjective estimate as to how much each of the pos-
sible
city
populations embodies
the
concept
kind-of-smalL
In
this
example, a population of 700,000 assigned a membership value of
0.3.
Clearly, a different set of membership values would be assigned
to the populations for the fuzzy
sets very small cities, medium
cities,
large
cities, and so forth; a population of 100,000 might receive a
value of 0.2 for very small
cities, but a value for
1.0 for kind-of-
small
cities, depending on
context, consensus, and
the
particular
Approved For Release 2001/03/07
: CIA-RDP96-00791R0002001 00003-3
Approved For Release 2001/03/07
: CIA-RDP96-OO791ROOO2O01 00003-3
198
The Journal of Parapsychology
application. These membership values can be obtained through con-
sensus opinion, a mathematical formula, or by several other means.
j
Crisp
sets are special cases of fuzzy
sets, in which
all membership
values are either zero or one. By using membership values, we are
able to provide manipulatable numerical values for imprecise natu-
ral language expressions; in addition, we are no longer forced into
I
making inaccurate binary decisions such as, "Is the city of San Fran-
j
cisco large—yes or no?”
In this example, the crisp set of all cities defines the universal set
i
of elements (USE). The crisp set of cities with populations of one
million or more
is a subset of USE. The fuzzy sets very small, kind-
\
of-small, medium, and large cities are fuzzy subsets of USE.
j
Universal Set of Elements
Since targets and the responses will be defined as fuzzy sets, we
j
must specify a USE. The universal set of elements can be quite gen-
eral and include
all aspects of a given target pool, or
it can be tai-
lored to a specific experiment to test a given concept (e.g., include
only geometric shapes). Since the method of fuzzy set analysis crit-
ically depends on the choice of USE, we provide one example that
was derived from a target pool used in earlier experiments. What
follows
is only an example of how one might construct a USE. The
one we use
is not generally applicable to other target pools or other
experiments.
We constructed our USE by including a
list of features present
in photographs from
the National Geographic magazine
with
ele-
j
ments obtained from the RV responses in earlier experiments. This
USE
is presented in Appendix A as the actual coding forms. For
the target features, we focused on direct visual elements.
(In the
|
general case, other perceptual dimensions can be considered.)
In
the case of the RV response-derived elements, an effort was made
to preserve the vocabulary used by the viewers. Some of the
ele-
j
ments,
therefore, are
either response-dependent or target-depen-
dent or both, whereas others, particularly at the more abstract lev-
els, appear to be more universal across possible USEs.
This
universal
set of elements
is structured
in
levels, ranging
j
from the relatively abstract, information poor (such as vertical lines),
to the relatively complex, information rich (such as churches). The
;
current system is structured into seven primary and three secondary
levels of elements; the main intent of this structure
is to serve as a
j
heuristic device for guiding the analyst into making judicious con-
j
Approved For Release 2001/03/07
: CIA-RDP96-00791R0002001 00003-3
Approved For Release 2001/03/07
: CIA-RDP96-00791R0002001 00003-3
'
Advances in Remote-Viewing Analysis
199
crete element assignments based on
rather abstract commentary.
The use of levels
is advantageous in that each element level can be
weighted separately and used or not, as the case may be. This ena-
bles various combinations of levels
to be deployed to identify the
optimal mix of concrete versus
abstract elements. Of course, any
such weighting scheme must be determined in advance of any ex-
periment.
The determination as to which elements belonged on which level
was made after consideration of two primary factors:
(1) the appar-
ent ability of the viewers to be able to resolve certain features, cou-
pled with
(2) the amount of pure information thought to be con-
tained
in any given element. Some of these “factor one”
determinations were based on the combined anecdotal experience
of analysts and monitors in the course of either analyzing or con-
ducting numerous RV experiments; some were determined empir-
ically from post hoc analyses of viewers’ abilities to perceive various
elements in previous experiments.
The “factor two” determinations were made primarily by arrang-
ing the elements such that an element at any given level represents
the sum of
its constituent elements at lower levels. For example, a
port element (Level 7) could be considered to include canal (Level 6)
and partially bounded expanse of water (Level
5). The world
is not a
very crisp place and not all its elements are amenable to hierarchical
structuring. Certain violations of the “factor two”
rule appear,
therefore, throughout the USE example.
It should be noted, how-
ever, that some of the more glaring violations were largely driven
by the “factor one” determinations
(i.e., the viewers’ abilities to dis-
cern certain elements) enumerated above.
To emphasize once again,
it is very important to realize that this
universal
set of elements was constructed
to match our particular
special targets, viewers, and requirements. They are shown here to
illustrate the procedure. Any particular application of fuzzy set tech-
nology to the analysis of free-response material requires an a priori
construction of an individualized, and improved, USE specific to the
target pool and the goals of the experiment.
Target Fuzzy Sets
Each target
is defined as a fuzzy set constructed by assigning a
membership value to each of the elements in the USE (see Appen-
dix A), In general, membership values can vary continuously on the
interval
[0,1].
In
this application they represent human judgment
Approved For Release 2001/03/07
: CIA-RDP96-00791R0002001 00003-3
!
Approved For Release 2001/03/07
: CIA-RDP96-00791R0002001 00003-3
200
The Journal of Parapsychology
:
I
and, thus, were constrained to vary in steps of 0. 1
. In addition, they
must represent the perceptual dimension used to construct the USE.
In our example, membership values were assigned to each element
for each of the targets, according to a consensus (on an element-by-
j
element basis) reached by three analysts. This approach was used to
mitigate the potential influence of any single coder’s biases and idio-
syncrasies. A numerical assignment, p (0
*£ p
1, in steps of 0.1),
was made for each element in response to the following question:
How visually important is this element to this photograph?
j
Encoded by this method, the fuzzy sets served as a formal defi-
j
nition of the targets for the
analysis.
It should be noted that our
j
USE defined targets in terms of visual importance.
4 If other dimen-
sions are of interest
(e.g., conceptual,
functional,
allegorical), the
USE would have to be revised to incorporate them.
In an actual experimental series,
it is critical that the target fuzzy
sets be defined by analysts before the series begins. Because of the
potential information leakage owing to bias on the part of the ana-
lyst,
it is an obvious mistake to attempt to define the target fuzzy set
on a target-by-target basis in real time or post hoc.
Response Fuzzy Sets
To define RV response fuzzy sets, membership values p are as-
signed for each element in the USE by asking: To what degree am
I (the analyst) convinced that this element is represented in this re-
sponse? For example, if a response explicitly states “water,” then the
|
membership value for the water-element should be
1.
If, however,
j
the response
is a rough sketch of what might be waves, then the
I
membership value for the water-element might be only 0.3, depend-
I
ing on the specificity of the drawing. This definition of membership
value is quite general and can be used in most applications.
In our example, responses were coded according to this defini-
tion (but
still using the USE in Appendix A). The assigned
p’s for
,
the targets and responses were one-digit fuzzy numbers on the in-
terval
[0,1]
(e.g.,
0.1,
0.2,
0.3,
etc.). In some rare cases, two-digit
assignments
(e.g., 0.05, 0.15, 0.25, 0.35,
etc.) were made; any finer
;
assignments, however, were deemed to be meaningless. Thus, the
response was defined as
its fuzzy subset of the USE.
j
4 Implied visual importance was ignored. For example, in a photograph of the
Grand Canyon that did not show the Colorado River, water, river, and so on would
-j
be scored as zero. By definition the target was only what was visible in the photo-
graph.
Approved For Release 2001/03/07
: CIA-RDP96-00791R0002001 00003-3
Approved For Release 2001/03/07
: CIA-RDP96-00791R0002001 00003-3
“
Advances in Remote-Viewing Analysis
201
In an actual experimental series, each response fuzzy set
is cre-
ated by analysts who are blind to the intended target.
Fuzzy Set Definition of Figure of Merit
Once the fuzzy sets that define the target and the response have
been specified, the comparison between them to provide a figure of
merit (FM)
is straightforward. In previous work (Humphrey et al,
1986), we have defined accuracy as the percent of the target material
that was described correctly by a response. Likewise, we have de-
fined
reliability
(of the viewer)
as the percent of the response that
was correct. The FM is the product of the two; to obtain a high FM,
a response must be a comprehensive description of the target and
be devoid of inaccuracies. The mathematical definitions for accuracy
and
reliability
for the 7th target/response
pair are
as
follows.
Let
|xk(Ri) and
|xk(Tj) be the membership values for the Mi element in
USE for the
s'th response and the 7th target, respectively. Then the
accuracy and reliability for the ith response applied to the 7th target
are given by:
accuracy t> = atj
reliability^ =
r
2W*min{p*(/0>M<*(7})}
k
k
2 w*min{p*(Ii j),p*(
,
r-)}
k
2>*p*(fl,)
where the sum over k
is called the sigma count in fuzzy set terminol-
ogy, and
is defined as the sum of the membership values. We have
allowed for the possibility of weighting the membership values with
weights WA in order to examine various level/element contributions
to the FM. The index,
k, ranges over the entire USE.
For the above calculation to be meaningful, the p’s for the tar-
gets must be similar in meaning to the p’s for the responses. As we
noted above, in our definition of the membership values, this
is not
the case. The target p’s represent the visual importance of the ele-
ment relative to the scene, and the response
p’s represent the de-
gree
to which an
analyst
is convinced
that the element
is repre-
sented in the response regardless of its relevance to that response.
With advanced viewers
it might be possible to change the defi-
nition of the response p’s to match the definition of the target
p’s.
In that case, the viewer must not only recognize that an element
is
Approved For Release 2001/03/07
: CIA-RDP96-00791R0002001 00003-3
I
Approved For Release 2001/03/07
: CIA-RDP96-00791R0002001 00003-3
202
The Journal of Parapsychology
'
|
present in the target, but must also provide information as to how
visually important
it
is. This ability
is currently beyond the
skill of
j
most novice viewers. Alternatively, we have opted to modify the tar-
get
(x definition by using the fuzzy set technique of a-cuts. In our
example, an a-cut is a way to set a threshold for visual importance.
All target elements possessing that threshold value or higher are
;
considered to be
full members of the target
set. In fuzzy
set par-
lance, an a-cut converts a fuzzy set to a crisp one. The result is that
j
the target
set: is now devoid of detailed visual information: a poten-
j
tial target element
is either present or absent in the target
set, re-
gardless of
its actual visual importance. Even with this conceptual
change in the target definition, the FM formalism described above
remains applicable, because a crisp set can be considered as a fuzzy
|
set with
all membership values equal to 0 or
1.
It
is important to
recognize that the a-cut
is only applied
to the target
set;
the
re-
j
sponse set remains fuzzy.
|
Assessment of Quality of the Remote Viewing
It is difficult to arrive at a general assessment of how well a given
j
response matches a specified target. The ideal situation
is to obtain
|
some absolute measure of goodness of match. Although the FM
is
an approximation to this measure,
it is impossible to assess the like-
j
lihood of a particular FM value because
it requires knowledge of the
viewer’s specific response bias for the session.
It
is possible to deter-
j
mine general response biases (May et al., 1985), but that knowledge
is only useful on the average. For example, a viewer may love rock
j
climbing and may spend most of his free time involved in that ac-
tivity. Thus, the general response bias would probably entail aspects
of mountains, rocks, ropes, and so forth. Suppose, however, that the
viewer spent the evening previous to a given RV session on a ro-
j
mantic moonlight
sail on San Francisco Bay. For
this
specific RV
,
session,
the
response
bias might include romantic images of the
moonlit water, lights of the city, and bridges.
The current solution to the problem
is to provide a relative as-
sessment of FM likelihood. A relative assessment addresses the fol-
|
lowing question: “How good
is the response matched against
its in-
tended target, when compared to all possible targets that could have
been chosen for the session?” This is not ideal, since the answer de-
pends on the nature of the remaining targets in the pool. An ex-
ample of the worst-case scenario
illustrates the problem. Suppose
Approved For Release 2001/03/07
: CIA-RDP96-00791R0002001 00003-$
Approved For Release 2001/03/07
: CIA-RDP96-00791R0002001 00003-3
Advances in Remote-Viewing Analysis
203
that the target pool consisted of 100 photographs of waterfalls, and
the viewer gave a near-perfect description of a waterfall. (We
as-
sume that this description
is not fortuitous,) An absolute assessment
of the resulting FM should be good, whereas a relative assessment
will be low. The worst-case scenario can be avoided, to a large de-
gree, by carefully selecting the target pool. (See the later section “A
Quantitative Definition of Target Orthogonality.”)
To provide a relative assessment of the likelihood of a given FM,
we define the score for one session to be the number of targets, n,
out of a total, N, that have an FM equal to or higher than the FM
achieved by the correct match.5 The answer to the question: “Given
this response, what is the probability of selecting a target that would
match
it as well as or better than the target selected?”
is n/N.
Consecutive RV responses by the same viewer are not statistically
independent, nor can the responses be considered to be random in
any sense. The statistically independent random element in the ses-
sion is the target. Since targets are selected with replacement, under
the null hypothesis of no psi, the collection of scores derived over a
series of m trials constitutes a set of independent random variables,
each with a discrete uniform distribution. Under the null hypothe-
sis,
the mean chance expectation for the score
in each session
is
given by (N +
l)/2 and the variance
is given by
(N
2 -
1)/12. If A
is the sum of scores from a series of remote viewings, then the prob-
ability of K, under the null hypothesis, can be obtained from the
exact distribution for the sum of ranks given by Solfvin, Kelly, and
Burdick (1978):
If m
is
large, then the sum-of-ranks distribution
is approximately
normal and Kim
has
a mean
of
(
N
+
l)/2
and
a
variance
of
(N
-
1)1 12m. Thus, a z score can be computed from:
z(K or less)
0.5(iV +
1) - -
m
(2 )
N must be the
size of the
target pool from which each
target was randomly
selected, and for this theoretical discussion, we assume no ties.
Approved For Release 2001/03/07
: CIA-RDP96-00791R0002001 00003-3
Approved For Release 2001/03/07
: CIA-RDP96-00791R000200100()03-3
204
The Journal of Parapsychology
Ground Truth
To determine whether the new analytical approach was effective,
a standard had to be developed against which
it could be measured.
It was determined that this standard—known as “ground truth —
should consist of a “real-world” normalized consensus about the de-
gree of correspondence between RV responses and their intended
targets.
To achieve this objective, we presented analysts (chosen from the
general SRI staff) with the same test case of six remote-viewing re-
sponses and their associated targets. The test case was the data from
a single viewer (177) taken from an experimental series in a 1986
photomultiplier tube experiment (Hubbard, May, & Fnvold, 1987).
The responses
(i.e., two to five pages of rudimentary drawings with
some
associated
descriptive words)
were
fairly
typical
of novice
viewer output and represented a broad range of response quality.
The targets consisted of six photographs of outdoor scenes selected
from a National Geographic magazine target pool of 200. Thus, this
data set was ideally suited for an analysis testbed. Appendix B con-
tains the “best” and “worst”
trials (Sessions 9005 and 9004, resPec"
lively) from this series in the form of their responses, their intended
targets, and their fuzzy set encodings (see the next section).
Each analyst was asked individually for his subjective judgment
about the degree of correspondence between the remote-viewing re-
sponses and their respective intended targets. The
degree o
cor-
respondence” was purposely undefined; the analysts had to formu-
late
their own
criteria. The only
information provided was
that
responses typically begin with small bits of information and even-
tually culminate
in a composite drawing at the end. Appendix C
contains the coding form that was used to obtain “ground trut
.
Each analyst was instructed to examine all of the responses and
their intended targets. Then, on a session-by-session basis, he was
asked:
(1)
to
assess
the
degree
of correspondence
between
the
remote-viewing response and its intended target, and (2) to register
this correspondence
assessment by making
a
vertical hash mark
across a 10 cm scale ranging from “none” to “complete.
To perform the ground truth analysis, distance measurements
were taken from the left end point of each scale to the vertical slash
mark for each assessment. Let the distance obtained for the kth ses-
Approved For Release 2001/03/07
: CIA-RDP96-00791R0002001 00003-3
Approved For Release 2001/03/07
: CIA-RDP96-00791R0002001 00003-3
Advances m Remote-Viewing Analysis
223
Figure B8. Page three of response (Session 9005, Target 1005).
Approved For Release 2001/03/07
: CIA-RDP96-00791R0002001 00003-3
Approved For Release 2001/03/07
: CIA-RDP96-00791R0002001 00003-3
j
224
The Journal of Parapsychology
Figure B9. Page four of response (Session 9005, Target 1005).
Approved For Release 2001/03/07
: CIA-RDP96-00791R0002001 00003-3
Approved For Release 2001/03/07
: CIA rRDP96-00791R0002001 00003-3
Advances m Remote-Viewing Analysis
225
Approved For Release 2001/03/07
: CIA-RDP96-00791R0002001 00003-3