Approved For Release 2000/08/08
: CIA-RDP96-00789R002200450001 -8
o
use a "tripolar protocol,"
i.e,,
collect data under three interspersed
conditions
(e.g.,
intention to raise
the number of events,
intention to
lower the number of events,
and baseline);
and
o
within the experimental program,
maintain equal respect for the
aesthetic/impressionistic and rigorous/analytical aspects of the research on
this unusual topic
One additional point that was made repeatedly in the workshop regarded the
necessity of distinguishing between exploratory and confirmatory
(proof-
oriented)
experiments.
The former are generally shorter,
more broadly defined,
and not reported.
There was general agreement that the explorational objectives
should be clear and the temptation to report positive results as
"proof"
of a
hypothesis should be resisted.
Instead,
exploratory studies should be used to
suggest and develop hypotheses
for later proof- oriented experiments which should
be highly defined and constrained.
Exploratory studies can also be used to
develop and refine experimental procedures.
III.
ANALYSIS OF DATA
As was noted earlier,
the scientific evidence for psi consists chiefly of
statistically significant deviations from a theoretical chance expectation or
experiential baseline.
Consequently,
the validity of statistical analyses
employed
is crucial
to
the strength and acceptability of parapsychological
claims.
According to Jessica Utts,
a statistician participating in the OTA
workshop,
parapsychologists in general tend to be more careful with statistical
methodology than researchers
in other fields.
They are generally more
responsive
to criticism on this score than other researchers
are,
she
said.
There
is general understanding among them,
Utts said,
that human responses
cannot be assumed to be statistically random.
It
is also well understood that
selection of targets with replacement
is preferable because
it allows
independent trials
.
She also referred to the formal policy of parapsychology
for reporting negative results.
However,
it was noted,
not all statistical
analysis by parapsychologists
(or by their critics)
is of high caliber.
Much of the discussion centered around a single broad topic:
how to inject
the maximum robustness and power into the empirical data base.
It was
recommended that parapsychologists should confine themselves
to methodologies
that yield such robust data that the outcome does not depend largely on subtle
statistical points.
There was general agreement on the various suggestions that
were made
to this
end.
For example,
there was considerable discussion of the traditional notion
that a "successful" experiment has
a significance level
(e value)
of
.05 or less
(meaning that there
is no more than a
5 percent chance that the results are due
to chance).
Partly as
a result of this
concept,
Utts noted,
nonsignificant
results
(that
is, where e >.05)
are too often interpreted erroneously by critics
as insignificant results.
An example was given to demonstrate that sample size
(n) has
a substantial impact on the statistical outcome;
if n is small
there
is
very little chance for a "successful"
outcome,
even if relatively high actual
probabilities of success based on ESP are assumed.
Meta-analysis of all
the
forced-choice precognition experiments conducted from the 1930s on showed that
Approved For Release 2000/08/08
: CIA-RDP96-00789R002200450001-8
Approved For Release 2000/08/08
: CIA-RDP96-00789R002200450001-8
Wi
f
h siSfificant results had,
on average,
two and one-half times
larger
size
f
than those with nonsignificant results.
Thus,
large values of n
o?ohflh^^
S1^
rabl
?
greater
"P°wer "
the analysis.
Power
is,
essentially “the
now
f
the
®xPeriment
is going to succeed.
(This concept of power ’is
being taught at the graduate level
in statistics.)
Table
2 illustrates how
power increases with increasing
n.
statistician recommended that parapsychologists stop focusing on the
meacnv
y ’
E <- 05 and begin focusing,
instead,
on more meaningful
measures such as estimating the magnitude of the observed effect,
looking for
“r7
’ Tf
StUdylnS
?
atterns across many studies.
Meta-analysis
if a ^ay
-f^
C^
large
^ !ample si2es and
"hits -"
In an analysis of ganzfeld t>si
"f
2?
hit Pr°bability of
.25 but which were nonsignificant
(and thus*
ilures
),
combining
the studies
gave
a p of
.02
("success")
.
This
recommendation met with general
agreement.
Approved For Release 2000/08/08
: CIA-RDP96-00789R002200450001-8
Approved For Release 2000/08/08
: CIA-RDP96-00789R002200450001-8
TABLE
"POWER" AS A FUNCTION OF SAMPLE SIZE
FOR DIRECT HIT GANZFELD ANALYSIS
WITH CHANCE p-.25
Power
True p
n-10
n-100
.30
.047
.29
.33
.073
.54
.40
.166
.94
.50
.377
.9998
(
-
.02)
(
.04)
Approved For Release 2000/08/08
: CIA-RDP96-00789R002200450001-8