Approved For Reiy^Sj20|/^0$(|p)§)D6-OO792ROOO5OO35OOO1-4
2.
(U) Database Design
(U) The
database
schema
that was
used
consists
of
four
basic
tables
(people,
documents, experiments, and units), and two basic relationships (author and parameter).
See
Figure 2 for an illustration of this schema. The units-table contains information about the lowest
level of statistical analysis in a given experiment.
For example,
if 6 viewers participated in 20
trials each, the database would contain 6 unit entries—one for the overall result for each viewer.
(U) Although
our
database
management
system
is
a
relational
database,
our
requirements were
inherently
hierarchical.
That
is, each of the documents contains
several
experiments, and each
of our experiments
contain
several
trials.
In order to minimize the
redundancy within the database, we attempted to include all pertinent information as high in the
hierarchy as possible.
That
is,
if a parameter or condition applied to an entire experiment, we
would record that data
at the experiment
level.
If, on the other hand, the parameter varied
across units within a given experiment, we made provision to record those data as a function of
unit instead.
(U) The
analyses
of most
of our experiments
contain both individual and group
statistics.
In order to prevent any trial from being “counted” multiple times, we required that all
experiments be broken up into the
“units” which represent the basic grouping of trials upon
which a hypothesis was being tested.
Thus, any given trial appears only once in the database yet
we can reconstitute the group statistics at a later time.
(U)
This approach offers two advantages.
First, any arbitrary parameter which does
not have an explicit slot in the database can be stored, thus providing flexibility.
Second, we can
distinguish between
“independent
variables”
and
“incidental
variables.”
The
former
are
variables which are intentionally manipulated by the experimenter, and the
latter are actually
parameters which the experimenter either could not control or treated as insignificant.
(U) Some of the documents detail multiple analyses for a given experiment in order
to compare and evaluate standard and new analytic techniques.
For this
effort, however, we
required that only one analysis be recorded for each experiment, since our primary focus was to
evaluate the parameters that
effect psychoenergetic
functioning and not to compare
different
evaluation techniques.
In determining which analysis to enter into the database, we always chose
a blind method over a post hoc method.
If a choice
still remained, we then always chose the
technique that had been developed
first.
UNCLASSIFIED
Approved For Release 2003/09/09
: CIA-RDP96-00792R000500350001-4
Approved For Rel|^p0|/j|^^J(|pjF^06-OO792ROOO5OO35OOO1-4
UNCLASSIFIED
FIGURE 2
(U) DATABASE SCHEMA DESIGN FOR META-ANALYSIS
(U) The Appendix contains examples of the DBMS input sheets that were used to
encode psychoenergetic data for the database, and the instructions that were given to analysts.
They are included in the Appendix for completeness; there is no further discussion about them in
this report.
C.
(U)
Statistical Methods
1.
(U)
Effect Size Calculations
(U)
Effect sizes were calculated for each experiment or condition using the formula
given by Rosenthal: 2
where n
is the number of trials and z
is the usual normalized output score. If no z score was given
for an experiment, but a p value was, the z that would have given that p value was computed and
used
in
the
formula.
The
exception
to
this
procedure
was
for
experiments
based
on
a
sum-of-rank statistic.
For those, a more appropriate effect size formula was used and is given by
UNCLASSIFIED
Approved For Release 2003/09/09
; CIA-RDP96-00792R000500350001-4
Approved For Reiy^Ja^/^^tPI1^^6 '00792^00500350001 '4
(U)
d =
.
CjgjLl)
’
//? 2 -l
’
V
where 5
is the average rank and R
is the number of choices for each rank.
2.
(U) Comparisons Across Classes
(U) Experiments
can
be
categorized
in
accordance
with
a
number
of
specific
variables
(e.g., type of feedback, type of target,
distance between the viewer and the target).
Effect sizes can be examined within a given category and compared across categories.
For each
categorization, the following questions are of interest:
(1)
Question
:
Is there any evidence of psychoenergetic functioning within
each of the individual categories?
(2)
Question 2
:
Is the level of psychoenergetic functioning constant across
all experiments within a category?
(3)
Question 3
:
Is the level of psychoenergetic functioning constant across
categories?
(4)
Question 4
:
If
there
are
differences
across
categories,
what
is
the
relative size of the effect in each category?
(U) Table
1 shows the notation that
is used
in the
formalism
that answers these
questions.
(U) To answer question
1, compare the average z score in each category with the
standard normal tables.
(U) To answer question
2, compute
k
m
i
Qw=
, nij(djj ~ dj.) 2
.
i =
i ; = i
If effect sizes are homogeneous within categories, the distribution of Qw will be approximately X2
with v = (Xmi - k) degrees of freedom.
The hypothesis of homogeneity
is rejected
if £>w is large
compared to the chi-square table entry with v degrees of freedom.
To
test
for homogeneity
within a single category,
i, compute
m
i
Qwi =
tiij(dij - di .)
7 =
UNCLASSIFIED
Approved For Release 2003/09/09
: CIA-RDP96-00792R000500350001-4
Approved For Re^J^la^/^/^StPI®©6 -00792*000500350001-4
(U)
Similarly,
the
distribution
of Qwi
will be approximately X2 with v =
(mi - k)
degrees of freedom, and can be examined as above.
Table
(U) DEFINITIONS AND META-ANALYSIS FORMALISM
Basic Definitions
k = number of categories
mi = number of experiments in category
/;
i =
1,
...
, k
dij=
effect size for experiment; in category
/;
i =
,
...
,
k; j =
1,
...
, m (
-
riij = number of trials in experiment j in category
i
Zij =
z score for experiment j in category
i
ComDUted Quantities
/ f
m. = ^ntj = number of trials
J
Svi/
j
di.
= average effect size
\
m .
Within Category
i
-V
Z j. =
-
t
= dj. Vn/T = average z score
vn/.
*T
n,. =
= total number of trials
t
Across Categories
«J
2 X n ‘jdiJ
\
d.. =“
= overall average effect size
n ..
Vsw
z.. =Jn~,d..
= overall average
z score
UNCLASSIFIED
(U) To answer question
3, compute
*
Qb = 'y. n i- {di- ~d..) 2
.
i =
l
If effect sizes are homogeneous across categories, the distribution of 0b will be approximately X2
with
v = k-1 degrees of freedom.
Therefore, the hypothesis of homogeneity across categories
is
rejected if 0b is large compared to the appropriate entry in the chi-square table with v degrees of
freedom.
UNCLASSIFIED
Approved For Release 2003/09/09
: CIA-RDP96-00792R000500350001-4
Approved For Rel<y^(21iA<S&JE4€O6-00792R000500350001-4
(U)
Finally,
to answer question
4, approximate 95% confidence
intervals may be
computed for the average effect size within a category using
UNCLASSIFIED
Approved For Release 2003/09/09
: CIA-RDP96-00792R000500350001-4
Approved For Release 2®;EM7® C»-RDP96-00792R000500350001-4
V
RESULTS AND DISCUSSION (U)
(U) The results of the meta-analysis are presented here, where possible,
in quantitative
analytic terms, and various interpretations are discussed in detail,
In addition, items that cannot
be analyzed are discussed from a qualitative perspective.
(U) The analysis proceeds in a top-down fashion in accordance with the hierarchy shown
in Figure
1.
A.
(U)
Overall Results
(U) The overall analysis was conducted from three different perspectives:
(1)
All of the data, regardless of the purported
skill of the subjects,
(2) A subset of the data contributed by an experienced group of viewers, G1
(i.e,, long-term, generally accepted expert viewers—002, 009,
131, 372,
414, and 504)
(3)
All of the data except for the group G1
(i.e., All-Gl).
Table 2 shows the number of trials n, total
z score, p value, and effect size d for informational
and putative causal processes and for the combination of the two.
Table 2
(U) STATISTICAL RESULTS FOR MAJOR CLASSES
Class
Perspective
n
z
P*
d
Psychoenergetics
All
25,449
9.37
3.69
(-21)
0.059
G1
9,825
6.86
3.46 (-12)
0.069
All-Gl
15,624
6.53
3.46 (-11)
0.052
Informational
All
24,450
9.07
5.83
(-20)
0.058
G1
9,702
6.69
1.14
(-11)
0.068
All-Gl
14,748
6.25
1.96 (-10)
0.052
Causal
All
999
2.42
6.39
(-03)
0.077
G1
123
2.06
1.99
(-02)
0.171
All-Gl
876
1.89
2.95
(-02)
0.064
SECRET
*
(U) Powers-of-ten are shown
in parentheses.
SECRET
Approved For Release 2003/09/09
: CIA-RDP96-00792R000500350001-4
Approved For Release 2(raCMr9f»CqiA-RDP96-00792R000500350001-4
(U) The number of trials shown in Table 2 differs slightly from those shown in Figure
1.
A few trials in each category were analyzed from a post-hoc point of view and therefore have not
been included in the formal analysis.
(S/NF)
The heterogeneity of effect size within each group for all classes
is very large
(i.e.,
the chi-squares for within-groups were large)
.
This
is to be expected for such a global analysis
and
is frequently seen in meta-analyses of psychological data. 5
The sources of the within-group
variation
include the psychoenergetic
skill
level
of the
subjects and
fundamental
differences
between psychoenergetic tasks.
(S/NF)
The data, regardless of subjects or process, show strongly significant evidence for
psychoenergetic functioning
(p < 3.69 X
10 -21 ).
Both the informational and putative causal
processes show significant evidence of psychoenergetic functioning,
as well.
(S/NF)
Since p values are
strongly dependent upon the number of
trials, the modern
trend
in meta-analysis
is to consider the trial-independent measure of effect
size. From
this
point of view, the magnitude of the psychoenergetic functioning appears roughly constant for all
the data shown
in Table
2, and, according to Cohen’s
criteria
for the interpretation of effect
size,*
corresponds
to small
effects. 6 The method of calculating overall effect
size, however,
involves a weighted average
(see Table
1) and thus may not provide an accurate picture of the
size of the psychoenergetic functioning within a given category.
To obtain more insight into the
nature of the functioning, we must examine the data within each category.
B.
(U)
Results for Categories Within the Informational Process
(S/NF)
Table
3 shows the number of
trials,
total
z
score, p value, and
effect
size
for
categories within the informational process.
The data show
strongly
significant evidence
for
psychoenergetic functioning for
all categories regardless of subjects.
The effect
size, however,
begins to demonstrate category differences.
(S/NF)
The forced-choice effect size
(
d
= 0.052)
is equivalent to the overall effect size
shown in Table 2
(d = 0.059).
Since the forced-choice category accounts for 77% of the total
number of trials, the effect-size averaging technique biases the overall result.
For example, the
effect size
(
d
= 0.209) for the RV-Lab category is significantly larger than for the Forced-Choice
case (X2 = 22.70, v =
1; p < 6.63 X
10' 6 ). The RV-Lab effect sizes meet Cohen’s criterion for
a medium-sized behavioral effect.
*
(U)
Values of 0.1, 0.3, and 0.5 correspond to small, medium, and large effects, respectively.
SECRET
Approved For Release 2003/09/09
: CIA-RDP96-00792R000500350001-4
Approved For Release 2So&G/REIa-RDP96-00792R000500350001-4
Table
(U) STATISTICAL RESULTS FOR INFORMATIONAL CATEGORIES
Category
Perspective
n
z
P*
d
Forced-Choice
All
19,675
7.42
6.12
(-14)
0.052
G1
9,487
5.82
2.92 (-09)
0.060
All-Gl
10,188
4.69
1.39
(-06)
0.046
RV-Lab
All
966
6.49
4.33 (-11)
0.209
G1
196
5.39
3.49
(-08)
0.385
All-Gl
770
4.55
2.71
(-06)
0.164
RV-Ops
All
3.98
3.45
(-05)
1.326
G1
3.98
3.45
(-05)
1.326
All-Gl
—
—
—
—
RV-Search
All
3,790
2.61
4.53
(-03)
0.042
G1
—
—
—
—
All-Gl
3,790
2.61
4.53
(-03)
0.042
SECRET
*
(U) Powers-of-ten are shown in parentheses.
(S/NF)
For the RV-Lab
category,
the
experienced
group,
Gl
t
performs
significantly
better than the novice,
larger group (X2 = 7.63,
v =
1; p < 0.0057).
(S/NF)
As in the overall analysis, the data analyzed in Table 3 show a large heterogeneity
of effect
size within each category.
The heterogeneity of effect
size, however,
is
significantly
reduced for the experienced subjects in the RV-Lab category.
This reduction may result from a
more uniform
skill
level
of the
subjects
in
group Gl\
this
is
in
general agreement
with
our
qualitative assessment of their abilities.
(S/NF)
Only 8.5%
of the remote viewing operational
trials were analyzed
as
a
formal
experiment.
The
effect
size
for
these
exceeds Cohen’s
definition
of
a
large
effect.
The
intelligence requirements of operational remote viewing, however, are less dependent upon the
quality of the viewing than they may be on other
factors.
Excellent remote viewing does not
necessarily imply good intelligence information.
(S/NF)
Because
of
the
usual
sensitivities
associated
with
intelligence
data,
obtaining
evaluations of the operational remote viewing has continued to be
difficult.
SECRET
Approved For Release 2003/09/09
: CIA-RDP96-00792R000500350001-4
Approved For Release 2&E<£REcTa-RDP96-00792R000500350001-4
(S/NF)
In the RV-Search category, 91.3% of the data were collected under laboratory
conditions
by
novice
subjects.
The
remaining
trials
were
conducted
under
operational
conditions, and
intelligence
analysis
is not
available.
The
small
effect
size
(d =
0.042)
is
commensurate with that found in other laboratories, and may reflect our lack of understanding
about how to
elicit this form of psychoenergetic functioning.
C.
(U)
Specific Results for Remote Viewing
(U)
In this section we address the specific questions posed in the SOW.
In any kind of an
investigation where the general results fall under a statistical regime
(i.e., z scores less than about
5), no hard definitions exist for definitive conclusions. The problem is confounded in behavioral
science because many
factors, beyond
the
particular independent
variable
in
question, may
significantly alter the outcome of an experiment.
In trying to assess a large body of literature, as
more constraints are placed on the outcomes, fewer within-group trials are available for analysis;
thus, statistical conclusions become more difficult.
This is also true for psychoenergetic research.
Yet,
it is possible to describe trends, to suggest ways of improving experiments based upon earlier
results, and to obtain clear insights into factors that may affect psychoenergetic functioning.
(S/NF)
To ensure the most reliable interpretations of results in what follows below, group
G1 has been used for the quantitative discussion.
As was shown in Section V.-B,
this group
possessed the most homogeneous
set of data
for the RV-Lab category and demonstrated
a
significant amount of remote viewing ability.
1.
(U)
Selection/Screening
(S/NF)
The selection of individuals who are able to accomplish remote viewing both
in an operational
setting and
in the laboratory
is of paramount importance.
As
is shown
in
Section V.-B.,
above,
group G1
provides the
best
results
for both
types
of remote
viewing.
Throughout the
history of the program
at
SRI,
individuals have been able
to demonstrate
consistent
functioning over
a long period
of time.
This does not mean
that,
after vigorous
searching, only 6 have been found.
Rather, given our applications-oriented charter for most of
the time period
in
question, we had
little impetus
to
find
other viewers.
During
fiscal years
1986-1988,
it became
clear that a
greater number
of talented viewers was needed
for both
applications and research.
SECRET
Approved For Release 2003/09/09
: CIA-RDP96-00792R000500350001-4
Approved For Release 200^B@R<E4-RDP96-00792R000500350001 -4
(S/NF)
Prior to FY 1986,
little was known about how to select good viewers.
There
was
little systematic research either at SRI or within the field in general, and what was available
was inconclusive
or
contradictory.
The
effort that began
in FY
1986 encompassed
a broad
approach
to
the
problem.
We
initiated
three
different
types
of
quantitative
approaches:
self-report
personality
tests,
neuropsychological
testing,
and
behavioral
testing
(i.e.,
the
Personality Assessment System—PAS).
In
addition, we used one
heuristic approach,
which
simply asked individuals to try remote viewing.
(S/NF)
The heuristic approach has been quite successful.
The efficiency
(i.e., the
number of talented viewers found divided by the total number screened)
is approximately 1% in
the general population
(i.e.,
groups of self-selected volunteers).
Based upon the
results of a
mass screening effort, two individuals have been asked to be regular contributors to the project.
(S/NF)
One other heuristic source of good viewers
is individuals who have noticed a
psychoenergetic
ability
in
their
lives.
Many viewers
in group G1 came
to the project
in
this
manner, and
a new viewer,
claiming similar experiences, was identified
in
a
recent screening
effort.
This viewer produced an effect size of 0.440 in 6 remote viewing trials, which contained
many striking qualitative correspondences between targets and responses.
(S/NF)
Of the
quantitative
techniques,
the
neuropsychological approach was
not
successful at predicting performance.
The PAS, however, predicted performance of 9 viewers to
a significant degree.
(S/NF)
By
far, the best way to select viewers as of this writing
is to use individuals
who
either
have
abilities measured
in
other
laboratories,
or who have had
strong personal
experiences.
(S/NF)
One technique
not mentioned above
holds
great promise
for the
future.
Three individuals from group G1 who participated in a neurophysiological study of correlates with
remote
viewing produced
unusually
large
central
nervous
system
responses
to
light
stimuli
directed
at the
eyes.
More work
is needed to determine
if this simple test might be the most
effective way to screen for individuals with excellent remote viewing ability.
SECRET
Approved For Release 2003/09/09
: CIA-RDP96-00792R000500350001-4