112
Approved For Release 2000/08/11
: CIA-RDP96-00792R000700660002-7
Papers
star "makes or breaks" the results.
By using large "anonymous"
source groups the incentive for any one individual to create
false
anomalies might be
greatly reduced.
STATISTICAL ISSUES AND METHODS*
WHEN WILL WE BEGIN TO REDUCE ALPHA AND BETA ERRORS
IN STATISTICAL PSI EXPERIMENTS?
Ulrich Timm
(Institut
fur Grenzgebiete. der Psychologie und
Psychohygiene, Eichhalde
12,
7800 Freiburg
i .Br.
,
West
Germany)
In many psi experiments some
statistical selection errors are
made,
after whose correction the
initial
statistical significance dis-
appears.
These are Type
I errors,
more simply
called alpha errors.
That does not necessarily mean, however,
that in these experiments
real psi effects do not exist, since the usual methods,
if utilized
correctly,
are often so ineffective --with regard to the rareness,
instability, and inconsistency of psi effects- -that they can only
seldom
lead to
statistical significance.
This inefficiency of statisti-
cal methods
creates Type
II errors, or beta errors.
Therefore,
our objective should not only be the reduction of alpha errors and
the related decrease of spurious significances but
also the reduction
of beta errors and the related increase of real significance.
First
I
give an overview of those alpha errors that
I
call
statistical selection errors.
These show,
simply stated, the
follow-
ing three qualities
(Timm,
ZP_,
1983,
195-229):
(1)
From
. a set of statistical results
a single result
is
se-
lected and evaluated by some significance test.
(2)
The selection
is not performed randomly but according
to a criterion that
is related to the level of the single
result
in that
it directly or indirectly favors positive
results
(3)
Despite
this
success -dependent
selection, the significance
test
is carried out and interpreted in the usual manner
without any correction.
*Chaired by Martin
U.
Johnson.
Approved For Release 2000/08/11
: CIA-RDP96-00792R000700660002-7
Approved For Release 2000/08/11
: CIA-RDP96-00792R000700660002-7
ffl
o m s
g 4
g
2 ^
» £ C
S’ » &
#j 3
E 0
553
“ O
52 d ^
y
y y
y
O
P
<+ ^
C3 m
m
.5
© ©
a y
rr
u'
r+ w
P* ©
tj?
w
(D
C$
Ha & 0
C
,
** 3
o
§ a £
3- s
”*
-
S’
3- S
"
TO ° S ^ >
S'
B
» ^
o' ?
W
® ' i
r+ 4
r+ y 0
.j P
,/
c7 O
r. U4
t/l
^
*4
w
M. (JQ
0-
h-
i ^
n,
o 3 ^>0
h
Uffi
m ®
h-
q K
E ©
<<i o
o ©
?2S
3. P S.^o?o^
a*q
CD
a-™
ffi ^ a 1 § *g
,
.—
hi
3' -1
.
h*
y
^ y y y aa
°
§
o &
a U
aa* ?e
» s
“
3 o
It7 3
p; O
_
h. 0
S
§.<&' £
ffi §
C! ®
’O g
«> Jfi SIS S 2.2
3 „
5 s
g.
CD
g. P M
S- m
O'
to
, P
—
“
cr
*•
FL
(a
CD
*S Lie
..
~
“
p. o
CD
c+
0- O
CD
2,^
e
p
rS
1 — 3 ^
3.
L 8 O I
--
3 3
© g
o' K
(D o
& 2
o
CD
CD
•
^
n-
m.
ts*
CD4
O ©
B
cp
®{SS JJw J3
0 S6
&
™
C0 K_.Ol:S
Oi
p
a)
!3i-j'
IH
p
r+
Q. Q
£ °
Oi
y'
4 |y ! 8
s *
o
S'
o
ft
O' 3
a
^ ® & ®
ro
e S
P3 ^
CO
*
OQ
g
“
m
-g
(D
1~~ ^
0 OQo§f“fJ-b?;nf+P
O ®
® " ? ^ o .§ 8 ®
C 2 ® "
£ n
ffr
*es.s
„ !>'®^!'E3 '*» » 2-S 2
•g
®
e. §
“ x a s “
e. s ?
3 SSoj g*« I Q
‘
-g
S •
&8s BV| 38j<3g
1 §a|
'f £
ro
3 TO S
1 g
..
<« 8
as S ^ |
CD
CD
m. "
J3
P3
3 E 5
=- w
O
tj
w
I—
I
H- 2
CD
P
to
2.
^ 5
*
«
CD
t. »
>
C/q
CD
TJ
p
a
oq
ftl
CD
£7 ©
^
y
r+
r+
S'
r+
CD
tY
X
•0
CD
3^
cn
« 5
'
g «a
CD
S S
-a
s
CD
O'
«
CDy
yf
CD
o' H
E7
°
W
4»
CD
»
CD
1:2 a
LJ
I-**
§
T3
g
1 O
CD
Wj
PO
B.
O
S
GO
j-ii
CD
OMj
CD
03 ^
g
tJ
O
(D
CD
h»
"1
O'
(D
CD
on
H*
r+
i_J
y
cd 3
r+
CO
3T
CD
<-+
CD
W
y-
m
y
^
P-
CD
f]1
a
§ § §
§
O' 3
^
S' o
to
Cfq
l“4
3 a
®.
1=1 y
°
K?ctq
0: 4 ©
to ©
r+ y
3. CSCQa
g.
<+ »
H' O
O
O
p
eS y
cn
y'
cd
•
©
©
K
4 ^
P
CD
rf
to
r+
2. w
H3
I* 3
£ 3
O'
o
a %
O
p
H
y'
p
c»0
CD
a
o
o
B*
CTQ
&
flo
•g
El 3 H
O
y
y 0
3 o
yj
Ha
to
„
l-f*
y
to
oq
M*
<D
©
= 43
O
B
P ^
to
to
O
to
y
to
cn
a a
to
*»s
i£8
c
®
p
oo g
r+
E
to0
oo
oy
0y
r+
E
y
rfE
to
to
>§
p
0OE
W
E
r+
4400
g
'—
pH-
a >
i 0O o3
r+ B
44O4
to
M
P
“ ?
to
yr
OQ
„
h
_
o
a § a a *
gsr*! a a
m- o
_
O
p
0 3
B X
'
hhv{
HH ^
©
p7 0
Ha
&
P3
^ 0
„
>«
<!
f+
CD
«s ||
111 -
R
: X
3*
0y
r-^
e.
0 log
«5
c-t“
p
tr 2a B- *s
5 0 ^ o
to
rn
h.
«<J
M,
S*
4'
^
y- 4
S. s
1 ^
^
-
o
y*
to
y
<?q
s
/-N
0 y
p
g *
FJ.
’lj
O' «
S*
CD
Ct
g*
~
O
Jd
m
B
r+ aq
O
V3
III
*<1
It;
ffi
©
a
X
§ 3
& & a
g
2.
to a
S —
O
x
g. ©
M
<<j
—
3' 3.
y a
H
<J
y*
ff>
_
O
C
s.^
P5
to
y
r+
go
O
cn
to
to
0 g
° |
w
3 R 5* S
O
to
oq
O'
y
»y
y
o
r+ o 3;^0
y
o*
p
o*
S'
toy
t__j o
a y
p
H .
O' 3
y
&E
m
O-
C7
to
EL
to
|crq
P‘
3. ®
g*
oo
CD
r+
§ a
at)
X
g ^ 8
O'
r+
r+
°
g &
H-
y
p
o
0 4
S
4 O
© 0
p
3 g S
'O. £
CD
Ha
y
-
y
•
orq
PJ
r+
ID
S
to
2 ^
0}
Cl tj
_
©
s* & a
a
o. a
ffi. orq
to
rrn
CC
H-
sl | a
?&8 “
S' g
s 3
p. a g.-o
TO
to
•„ y
c+ O
P
to
H-
O'
s*
to0
to
Jf 8
a s
S*
to
to qP
r+y
©
p
4 B
g ©
©
©
TO W
S ^ o
y
p
Ma
a | S
g e®
£ ^ O
^
?
w
M-
r+
g P
.L
to
to
fff
0Oy
U
H
4 ^
0 ^
h» <<;
th
p
y
•
to
r+
Hi P
TO*
§ S &
0 d
_ y
o
y 4
to
4-
e
£
p p
3. ©
p
K
g
1 p
y
a.
©
3 m
tJ
P
4 o
U
y ^
Cl
to
i ^
-
S
r+
to
° y
y* ©
y
-
P- 0<0y
*o
p
t3
ss^a^s
s
h
3 y
y 0 <^S2 . 0 pq
w
S
W. & y
to y p 0 y
eg*
n* P .« p
3 -
S'
Ha 0
y
a'd ^
©
E?
to
o
g* y
p-
y
:
as Ig-
•o &5
1 a
p
r+ ^
a _
to
£j* m p
tt*
*
S'
H§
.
y ©
H* 0
^ g X
o
p
*d
y
©
o
® 2 .
Ml
3 y w
©
G
«
o
1 *d
H ^d P
y © y
y
3. &
mg
•o
p
& ^d
y p
to
p
;* »d
. 0
to
h-
to
0 ^
Pa ©
©
H
©
SS
r+ 0
to' 0
<-f 0
f§0
M-
to
S
p
y
<j
p4
to
to
m 3 4 ^
0 nd y ©
,4* »
to
P
to
p
§ I
cn
® ^
e.
~
1 a
R
O'
y a
p
-
a y
_ _
o.
oy
©
to
cn
44O4
y
§
^d 0
O' y0
<1
Q
B
X!
y ©
S
cd
p
y
©
o
•
y y
H- o «
O
Hs q
4 4 X
0 y *d
to
4-3
3.
*
3_
to
o&p
y
to
'{Q
tg'
y o
® a a
Egg'
m
*2
©
4^3
0 4
y
m
o 4 y
^ 4
g,
0 3©
Ha ©
«
“
to
CD
P
HJ
g B *a
NO
y
cd
p
0 ,y
P*
0P
to
r,
'P
M<
^d
p
5*
o
§
•8 a
tr
vj
r+
h<
OHa
to
i—' 0
o
a.
Ha
to
P
*
B.
I*
Approved For Release 2000/08/11
: CIA-RDP96-00792R000700660002-7
y P
0 y
-
9*5
cr
y
g*
2.
©
© 4
to 4
H* m
g a ®
®
Cfq
„
y
5 e
P
g g
r+ ^
*d
i o
0 p
* *
4 2
°
£ E
CD «
O' P*
CD
O
4 O
00- y
- 0
<y a
©
sr
y £
to
»
s
4*
CD
o
o-
Ha
.
»
y “
y
^d04O4Oy y
-
p
to
O-
O'
©
O'
Is
& 3
to
o
y E
p
Approved For Release 2000/08/11
: CIA-RDP96 -00792R000700660002-7
Q.*d £
CD
O
2-e
a * a
• e H) 4
.§ 8
§
!
“
:
O O
|
! 9 * e
.
O
O'
3,
!
CD
(D
po
i
' g a &
-
b. P
r+
,
^
hH
O
p
*P E
O
TO ^
^T*o
o g
52.
„P*
f+ «
MS B
..
crq
CD
e
®
H 3
"
.4
'S
g*
J
CD
2. o
O
'
ft
' a
S. «
s* crq
<
» |«,e
' O
0 ^
I* g
§? 3
i
CR
<“+ g
>
(o 5 a
5 a s
>
S- S
g. o
p 2 .
3 <a o
m »
H*
^
^
<
ft
P*
CD
H
°
'
a - 2 a
w o
t+
S’
tr 4
o
cl
.
(D
CD
m
(-*•
e-
rrn
TO
s E &
TO
q
ft.
eg a
* 1 *
p
‘
,
B S 3
L W ^
ft
f.
Qj
r+ O
f
P
cl
if!
§' ®
>
H.
O
r.
p
c
*> 2
P
M
to
lias
!
M-
M.
* 3 © 3
r.§
g
1 *
>
ff &a
. s
IQ
5 S S 2
r &<& a
: ! t i
j
TO
CO £
at |
A g» o
r “
g
«
>
ct o
(D
(tl
Hj
W
r+
ffl
•
(D 4 ^
TO
OS
°
CL crq
i-b
to ^ 03 .
hO
H. <
f!
3 ®
:
CD
O
. P
“
H
CD
-
p
CD
•
P
TO
s
s. &
tr
ss- Z
CD
ffl
pj
$ '!£
p
r+ 4
*
TO
ft
CO
CD
CD
P*
O
TO ^
,
M> r+
^
©
ct 3
__
TO
W
f+
r+
o
o £
tt a
TO
, p ^
'
TO
r+ 4
ffl
£
cd
g-
W U CR
j
03 ^
*0
o
P
P.
H
CD
© 3 ^
4 <R
Jj
*+
CD
o ^ o
£
S' g g
& E 3 S
' ».w?
g:
w “
, p
pi 2 . a
O
§
(D
• 2 § E p
O
(6
®
CD
TO
3 r+%%
e s a §
ff. E
05*
Pu
•P
ft
U
O
ft
TO p ^ P
P
_.
r+
-
C* P
P
Mj
’Sg
s a§
< 3. g-5
:£g r
C
M
TO
CD
r+trt
o
p
h3
TO E
s
rt 8
* g
o
g E <
p
p
p
C
«
CD
O
o ®
e
<T
°
g. g a
'^I^o
to
p
s* E
CD
P
S* g
1 a w
a 4
CD
CL
• S
. o
4 H 3
. o
tJ* «
O
<D
^
: 4
a*
CD
o
o
TO C
rf
r+ M
a
a s
-
3 3 ^
g
s a
i
ago
o
o
o
,
, p
tt p
•
TO
I
' o
P
S*
I
h4
4 E
!
CD
CD
|
I-K
^3
I
r+ P
j
4 tOi
o
re
S
O
CO
M.
*!
; 3
p
i
Ct
S-
p* «
s
p.
r+ 4
P
o" p
crq
p 5
2. ^
CD
:
P
-
r+
'
O
CD
ft
r+
3 P
<D E 0
2 . o ^
O
l-J- M
^
3 3
P
CD
CL
P
c;
P? g*
CD
CD
CD
P
P
O
S
h.
1-4
§
w
TO
3 P S 3
O
CD
f+
P
ft
P
TO
CD
CD
O g
P
p
h p a
s
s %
i ?
tp
p*
P
P
P<
CL
TO
w
{r
1 >
cn
pj
•
oo
rt
^ o
f'3
O 4 O
& ?
c « &
rt ^3
re
g
-
act
i_3 P
L; 4 n
ft
oq
sr
® » s-
a ° o
ft ^ 2
CD
rH 3
a S
K*
CD
cl E 3
ro H
cd
!’ P ^
st
*
a
p o
v
.
to
ccq
„
c
s g 5
<
M 9: §
|
O
P
p- ^
I § ^ o
- 4
P
(
)
TO
P
M-
<
s*
p
N
;
;• O
&.
CD
t 3 ^ 5
i
r *
®
:
’
,2 h
CS.
j
j
“
®
ffi
!
' as
n*
CD
>
S
°
E
CD
t3
j
u, „
1-1 .
; p
a
;
tcrq a o
t
p
,
l
ci'
CL
g B
c
J
M- P
P
1 p a s
*2 IS
l B i
? p p 2
*
i
CL
CL P
#
t
p
<
TO P
qT
P
P
p; p
o
3 S
§
P
*
?
g
?
g- cr^
1 I ® 8
S'* OP
§ & p
TO
p
ggS
p
“ 8 & g &
a s
" S g
f=r 2
<d
p
m
§
3 P
TO
5 P
CD
o
p
I
p
O- JQ
g; p
i
!• : i
tff)
Q
o)
<D
2 . § &
•»
i
i-j. a
C
rS
0X5
m-
crq
w
o
3 .
co
to o
P 2
P
o
CO
S*
g &o S
U. u O
4j
5 3 S3
r+ P
hi
P- 5
< 4
o 3
S
p
p E
^
p E
CD
cr §
h- 2
8 s &s
p
p
p
o p
p E
§ a
B 2.^
b-
a
CD
m
B g 5 P- a
s-
% a b
a o a g S
cd & p E 3
ft
TO
§. P g
? ^ ? hi g
ro
o 2 P 4
a E
2. o
E S O
CD
H,
^
^ P
H,
& p
4 < 5 g
p
cd
p ^ E
o
3 §
5* °
g 2 ^ ®
<S
r+
ro
ft
a*
TO
g’
TO
P
vl
O
P
n ^
i~^ P
P
^2
&
hrj
TO
$ -
g- ® 3.
® g
p, o
°
^ s ^
§
»rS5'
S & - n n
S' 5
ro
S9 p
§;” m ^
g>
o
? a
§.
P
P*
ft
a 1 1 § J
p & & 1
a s
">
s a §
'8 aS
8 2 . ”
.
aa 5
P
P
r+
rap'
o
§
P
m
<
' ©
*
p
M. O ^
TO
CD
v.
hi
P
ft
CL
p e p
CD
W 2
O- & P
!
,+ 5.4
E
CD
“
CD
_
| E
TO
P
! I-
a
!
3* 8
B-
S &E
Approved For Release 2000/08/11
: CIA-RDP96-00792R000700660002-7