Approved For Release 2000/08/10
: CIA-RDP96-00789R003800440001-2
UNCLASSIFIED
(U)
This analysis produces the average figure of merit for a series and an
indicator, the slope, of learning.
c.
(U)
Mean Chance Expectation
(U)
The calculation for the mean chance expectation (MCE) must be
sensitive to a number of possible artifacts or confounding factors:
•
Viewer variations
(e.g., viewers’ different response biases).
•
General knowledge of the target pool
(e.g., targets are known
to be National Geographic magazine photographs).
•
Specific knowledge of the target pool resulting from
trail-by-trial feedback.
•
Methodological considerations
(e.g., viewers are asked to
respond with more data at the end of the series compared to
what was asked of them at the beginning of the series)
All of these factors will affect the expected average figure of merit and any session-to-session
systematic variation that may be present.
(U) A method for determining the figure of merit MCE, which requires the
fewest number of assumptions about the structure of the data or the response biases, involves
the cross-matching of all the responses to the same target set used in the series in question.
A cross-match
is defined as a comparison between a response and a target other than the one
used in the session.
If a figure of merit distribution is calculated for a large number of
cross-matches, a number of the confounding factors listed above will be addressed.
To
determine the session-torsession dependencies of the MCE, however, the session order must
be preserved.
By preserving the order of the responses and by calculating [n] sets of
cross-matches at a time, MCE figure of merit, slope, and intercept distributions can be
calculated.
(U)
As before,
let
[n] be the number of sessions in a series for a single
viewer.
Also,
let the order of the responses, Rk be preserved.
Define [N] as the number of
cross-match cycles through the ordered set of [n] responses.
The MCE calculation proceeds
as follows:
1.
Randomly choose a target order,
i = l,n
, such that k
i
where k =
1, n
is the preserved response order.
2.
Calculate the figure of merit for the kth response/target
cross-matches as
Approved For Release 200(UNCtiASSINEfio03800440001-2
Approved For Release 2000/08/10,: Q
'-p.-
-*-0078
UNCLASSIFIED
(U)
M ki =
a
ki X
3.
4.
5.
' ?rfJJHw IP'Sb
ki
,
k #
i,
•.
j
'T*"'
\8ffl
,
i''-Tv
where a
and r
are the accuracy and reliability for response
[k]
ki
ki
,
"
cross-matched against target
[i], respectively.
Do step 2 above for all
[n]
sessions.
Calculate a slope and intercept for the resulting figures of
merit by the linear least-squares analysis described above.
Repeat steps
1 through 4 above for [N] cycles to produce
MCE figure of merit, slope, and intercept distributions.
(U)
It is important to note that MCE distributions are generated for each
viewer and are not summed across viewers.
Therefore, individual viewer differences in
response “biases” are accounted for by definition.
(U)
This procedure also accounts for general knowledge of the target pool
by the viewer, because information learned by this method in a given session will not
necessarily be associated with the intended target for that session.
The net effect of this type
of artifact will be to “bias” the MCE figure of merit distribution toward larger values.
,
(U)
Because the order of viewings is preserved, any knowledge of the target
pool that is learned by the viewer as a result of trial-by-trial feedback is accounted for in two
ways:
1 .
2 .
Information resulting from increasing knowledge of the target
plol will “bias” the MCE figure of merit distribution toward
larger values.
Information resulting from increasing knowledge of the target
pool as a function of session number will “bias” the MCE
slope distribution toward larger values.
(U)
Similarly, any artifact caused by methodological considerations as a
function of session,
will also “bias” the MCE figure of merit and slope distributions.
d.
(U)
Probability Assessment and Analysis
(U)
There are a number hypotheses that can be tested using the various
MCE distributions described above:
•
An individual remote viewing is statistically beyond MCE.
Approved For Release 2000/1(UNGLASSIF/lEO3800440001-2
Approved For Release 2000/08/10
: CIA-RDP96-00789R003800440001 -2
UNCLASSIFIED
(U)
•
The series generated by a single viewer shows statistical
evidence for remote viewing.
•
There
is evidence above MCE for remote viewing “learning.”
•
The mean of the observed figure of merit distribution
is
significantly larger than the mean of the MCE distribution.
•
The observed figure of merit distribution is significantly
different than the MCE distribution.
(U)
Using the MCE figure of merit distribution, a straightforward calculation
of areas will determine whether a particular figure of merit from a single session is significant.
Figure
1 shows an example of an MCE figure of merit distribution described above.
Mkk is
the figure of merit resulting from session
[k].
The probability of obtaining a figure of merit of
Mkk or larger caused by artifact
is the patterned area divided by the total area under the curve
shown in Figure
1.
This technique can be used to assess the chance likelihood for all sessions
in a series by a single viewer.
FIGURE
(U) AN MCE FIGURE OF MERIT DISTRIBUTION
(U)
To determine whether there
is statistical evidence of remote viewing
within a given series for a single viewer, the p-values for the individual sessions must be
combined.
The primary method used for combining p-values was developed by Fisher.6 A
X2 with two degrees of freedom
is computed for each p-value and summed.
The resulting X2
is evaluated with 2n degrees of freedom where
[n]
is the number of p-values that were
combined.
If
[k]
is the session number, the appropriate total X2
is given by
Approved For Release 2000(0^|Q|j^gj4FtED>038004^0001 ~2
Approved For Release 2000/08/10
: CIA-RDP96-00789R003800440001-2
UNCLASSIFIED
X
2 = 2 "2 ‘° ln Pk*
df = 2n
k=l
where the p
are the p-values for each of the
[k]
sessions. A second technique involves
testing the significance of the average p-value across
all sessions. A standard z-score is
calculated by
Z
=
V 12 n
(0.5 - p)
,
where p
is the average p-value and
[n]
is number of sessions.
(U)
These two measures are sensitive to different aspects of the remote
viewing series.
For approximately 20 or more sessions, the two techniques will yield similar
probability estimates
if there
is slight, but consistent evidence of remote viewing.
On the other
hand,
if there are a few very good results
(i.e., individual p-values < 0.001), then the X2
technique more accurately reflects the series as a whole.
(U)
As an example of consistency, suppose 20 sessions having individual
p-values of 0.35 each are analyzed.
Then the z-score for the average p-value
is 2.32,
corresponding to a combined p-value of 0.01.
The X2 technique yields a total X2 of 42.0
with df = 40, corresponding to combined p-value of 0.40.
To illustrate the X2 technique’s
sensitivity to “good” remote viewings, consider the following p-values for 5 individual sessions:
0.45, 0.72, 0.55, 0.00^, and 0.00005.
The average p-value technique yields a combined
p-value of 0.11, while the X2 technique yields a combined p-value of approximately 0.0005.
(U)
To determine whether there
is evidence of “learning” and whether the
means of the actual and MCE figure of merit distributions are significantly different, an
ANOVA technique
is used. 7
By transforming the data about the average value of the session
number, the slope and intercept hypothesis testing may be done separately.
The F-ratios
(from the ANOVA) for the two tests are given below.
Approved For Release 200
003800440001-2
Approved For Release 2000/08/10
: CIA-RDP96-00789R0Q3800440001-2
UNCLASSIFIED
F (slope)
=
-
*
)
,
dfl =
1; df2 = (n-2)
/
f \ 2
F (intercept)
=
—
>
dfl =
1; df2 = (n-2)
,
where
[a] and
[b] are the intercept and slope from the remote viewing figure of merit data,
and
[a’] and
[b’]
are the intercept slope from the MCE figure of merit data. A
is given by
u
n
«
n
n
2 M k + n a 2 + b
2 ]^k
2 + 2 ab
k - 2a ]>}M k -
^ b 2 k M
fc
k=l
v
(U)
Because the F-ratio for the slope for the figure of merit data
is a
statistical test between the observed slope and that computed from the MCE,
it constitutes an
estimate of the probability that remote viewing “learning” occurred over and above any
contribution that might have occurred because of some artifact.
The F-ratio for the intercept
constitutes an estimate pf the probability that the mean of the figure of merit distribution
is
different from the mean of the MCE. We use a standard X2 measure, a more sensitive test,
to determine if the observed figure of merit distribution
is statistically identical to that of the
MCE distribution.
B.
(U)
Fuzzy Set Theory—An Enhancement Technique for Descriptor List
Technology
1.
(U)
Overview
(U)
The figure of merit analysis
is predicated on descriptor list technology, which
represents a significant improvement over earlier “conceptual analysis” techniques, both in
terms of “objectifying” the analysis of free response data and in increasing the speed and
efficiency with which evaluation can be accomplished.
It has become increasingly evident.
Approved For Release 20
003800440001-2