Approved For Release 2003/01/09
: CIA-RDP96-00789R002200480001-5
2.
Database Design
The
database
schema
that
was
used
consists
of
four
basic
tables
(people,
documents, experiments, and units), and two basic relationships (author and parameter).
See
Figure 2 for an illustration of this schema. The units-table contains information about the lowest
of statistical analysis
in
a given experiment.
For example,
if 6 viewers participated
in 20
tnals each, the database would contain 6 unit entries-one for the overall result for each viewer.
Although
our
database
management
system
is
a
relational
database,
our
requirements were
inherently
hierarchical.
That
is,
each of the documents
contains
several
experiments, and each
of our experiments
contain
several
trials.
In
order to minimize
the
redundancy within the database, we attempted to include
all pertinent information as high in the
hierarchy as possible.
That
is,
if a parameter or condition applied to an entire experiment, we
would record that data
at the experiment
level.
If, on the other hand, the parameter varied
across units within
a given experiment, we made provision to record those data
as a function of
unit instead.
The
analyses
of most
of our experiments
contain both
individual and group
statistics.
In order to prevent any trial from being “counted" multiple times, we required that all
experiments be broken up into the
“units”
which represent the basic grouping of
trials upon
which a hypothesis was being tested.
Thus, any given trial appears only once in the database yet
we can reconstitute the group
statistics
at a later time.
This approach offers two advantages.
First, any arbitrary parameter which does
not have an explicit slot in the database can be stored, thus providing flexibility.
Second, we can
distinguish
between
“independent
variables"
and
"incidental
variables.”
The
former
are
variables which are
intentionally manipulated by the experimenter, and the
latter are actually
parameters which the experimenter either could not control or treated as insignificant.
Some of the documents detail multiple analyses for a given experiment in order
to compare and evaluate standard and new analytic techniques.
For
this
effort, however, we
required that only one analysis be recorded for each experiment, since our primary focus was to
evaluate
the parameters
that
effect psychoenergetic
functioning and
not
to compare
different
evaluation techniques.
In determining which analysis to enter into the database, we always chose
a blind method over a post hoc method.
If a choice
still remained, we then always chose the
technique that had been developed
first.
Approved For Release 2003/01/09
: CIA-RDP96-00789R002200480001-5
Approved For Release 2003/01/09
: CIA-RDP96-00789R002200480001-5
FIGURE 2
DATABASE SCHEMA DESIGN FOR META-ANALYSIS
The Appendix contains examples of the DBMS input sheets that were used to
encode psychoenergetic data
for the database, and the instructions that were given to analysts.
They are included in the Appendix for completeness; there is no further discussion about them in
this
report.
C.
Statistical Methods
1.
Effect Size Calculations
Effect sizes were calculated for each experiment or condition using the formula
given by Rosenthal: 2
where n
is the number of trials and z
is the usual normalized output score.
If no z score was given
for an experiment, but a p value was, the z that would have given thatp value was computed and
used
in
the
formula.
The
exception
to
this
procedure
was
for
experiments based
on
a
sum-of-rank statistic.
For those,
a more appropriate effect size formula was used and is given by
Approved For Release 2003/01/09
: CIA-RDP96-00789R002200480001-5
Approved For Release 2003/01/09
: CIA-RDP96-00789R002200480001-5
[r 2 -
V
where 5
is the average rank and R
is the number of choices
for each rank.
2.
Comparisons Across Classes
Experiments
can
be
categorized
in
accordance
with
a
number
of
specific
variables
(e.g., type
of feedback, type of target, distance between the viewer and the target).
Effect sizes can be examined within a given category and compared across categories.
For each
categorization, the following questions are
of interest:
(1)
Question
:
Is there any evidence of psychoenergetic functioning within
each of the individual categories?
(2)
Question
:
Is the
level of psychoenergetic functioning constant across
all experiments within
a category?
(3)
Question
:
Is the
level of psychoenergetic functioning constant across
categories?
(4)
Question 4
:
If
there
are
differences
across
categories,
what
is
the
relative
size of the
effect in each category?
Table
shows the
notation
that
is used
in
the
formalism
that answers these
questions.
To answer question
1, compare the average
z score
in each category with the
standard normal tables.
To answer question
2, compute
k
m
i
Qw-
^ nijtfij-di ,)
.
<» iy«i
If effect sizes are homogeneous within categories, the distribution of Qw will be approximately X 2
with v = (Imi - k) degrees of freedom.
The hypothesis of homogeneity
is rejected
if Qw
is large
compared to the chi-square table entry with
v degrees of freedom.
To
test
for homogeneity
within
a
single category,
/, compute
m
i
Qw/ = ^ n ij{dij - dj.)
Approved For Release 2003/01/09
: CIA-RDP96-00789R002200480001-5