Approved For Release JdtfoKsloAS&ll'tED-00789R002200460001 -7
2.
(U) Database Design
(U) The
database schema
that was used
consists
of
four
basic
tables
(people,
documents, experiments, and units) , and two basic relationships (author and parameter)
.
See
Figure 2 for an illustration of this schema. The units-table contains information about the lowest
level of statistical analysis in a given experiment.
For example,
if 6 viewers participated in 20
trials each, the database would contain 6 unit entries—one for the overall result for each viewer.
(U) Although
our
database
management
system
is
a
relational
database,
our
requirements were inherently
hierarchical.
That
is, each of the documents contains several
experiments, and each of our experiments contain
several
trials.
In order to minimize the
redundancy within the database, we attempted to include all pertinent information as high in the
hierarchy as possible.
That
is,
if a parameter or condition applied to an entire experiment, we
would record that data
at the experiment level.
If, on the other hand, the parameter varied
across units within a given experiment, we made provision to record those data as a function of
unit instead.
(U) The
analyses
of most of our experiments contain both
individual and group
statistics.
In order to prevent any trial from being "counted” multiple times, we required that all
experiments be broken up
into the
“units” which represent the basic grouping of
trials upon
which a hypothesis was being tested.
Thus, any given trial appears only once in the database yet
we can reconstitute the group statistics at a later time.
(U)
This approach offers two advantages.
First, any arbitrary parameter which does
not have an explicit slot in the database can be stored, thus providing flexibility.
Second, we can
distinguish between
“independent
variables”
and
“incidental
variables.”
The
former
are
variables which are intentionally manipulated by the experimenter, and the
latter are actually
parameters which the experimenter either could not control or treated as insignificant.
(U) Some of the documents detail multiple analyses for a given experiment in order
to compare and evaluate standard and new analytic techniques.
For this effort, however, we
required that only one analysis be recorded for each experiment, since our primary focus was to
evaluate the parameters that effect psychoenergetic functioning and not to compare different
evaluation techniques.
In determining which analysis to enter into the database, we always chose
a blind method over a post hoc method.
If a choice
still remained, we then always chose the
technique that had been developed
first.
UNCLASSIFIED
Approved For Release 2000/08/08
: CIA-RDP96-00789R002200460001-7
Approved For Release &N<€iiAS&IFi£G-00789R002200460001 -7
UNCLASSIFIED
FIGURE 2
(U) DATABASE SCHEMA DESIGN FOR META-ANALYSIS
(U) The Appendix contains examples of the DBMS input sheets that were used to
encode psychoenergetic data for the database, and the instructions that were given to analysts.
They are included in the Appendix for completeness; there is no further discussion about them in
this report.
C.
(U)
Statistical Methods
1.
(U)
Effect Size Calculations
(U)
Effect sizes were calculated for each experiment or condition using the formula
given by Rosenthal: 2
A
Z
where n
is the number of trials and z is the usual normalized output score. If no z score was given
for an experiment, but a p value was, the z that would have given that p value was computed and
used
in
the
formula.
The
exception
to
this
procedure
was
for
experiments based
on
a
sum-of-rank statistic.
For those, a more appropriate effect size formula was used and is given by
UNCLASSIFIED
Approved For Release 2000/08/08
: CIA-RDP96-00789R002200460001-7
UNCLASSIFIED
Approved For Release 2000/08/08
: CIA-RDP96-00789R002200460001-7
(U)
~
(*+i)
V
where S
is the average rank and R
is the number of choices for each rank.
2.
(U) Comparisons Across Classes
(U)
Experiments
can be
categorized
in
accordance
with
a number
of
specific
variables
(e.g., type of feedback, type of target, distance between the viewer and the target).
Effect sizes can be examined within a given category and compared across categories.
For each
categorization, the following questions are of interest:
(1)
Question
:
Is there any evidence of psychoenergetic functioning within
each of the individual categories?
(2)
Question 2
:
Is the level of psychoenergetic functioning constant across
all experiments within a category?
(3)
Question
3 :
Is the level of psychoenergetic functioning constant across
categories?
(4)
Question 4
:
If
there
are
differences
across
categories,
what
is
the
relative size of the effect in each category?
(U) Table
1 shows the notation that
is used
in the
formalism that answers these
questions.
(U) To answer question
1, compare the average z score in each category with the
standard normal tables.
(U) To answer question 2, compute
k
m
i
Qw='Z,^ nij(dij-di .)
.
If effect sizes are homogeneous within categories, the distribution of Qw will be approximately X2
with v = (Xwi - k) degrees of freedom. The hypothesis of homogeneity is rejected if Qw is large
compared to the chi-square table entry with v degrees of freedom.
To test
for homogeneity
within a single category,
/, compute
m
i
Qwi - ^ ftijidij - dj.) 2
.
/'i
UNCLASSIFIED
Approved For Release 2000/08/08
: CIA-RDP96-00789R002200460001-7