UNCLASSIFIED
Approved For Release 2000/08/08
: CIA-RDP96-00788R001 800050001 -8
Table
A-5
(continued)
Approved For Release 200WfcldELASSl|)FIE-O788R001 800050001 -8
rtK
s
fl
to
o
Sf
1-4P
I
©
CO
AP
V
l00
O
HI
N
CO
lO
CO
N
O)
o
H
CO
H
H
H
H
r**4
?
*
O
H
N
n
«
CO
O)
o
N
to
CO
o
in
.
_r—
,
H
H
H
H
N
oa
IS a
rl §
a*
1 w
<p
&0
td
0)
><
T3
4-)
\JS
to
-H
0)5
1—
f
CO
rH
1—
CM
pH
O
i—
oo
LO
LO
CM
vt
oo
oo
CM
LO
r—4
i—
o
1—)
NOIXVmVAa XdlHDSMVHX
<n
CO
•W
O
a
o
0)
CO
u
coa
LH
o
UNCLASSIFIED
Approved For Release 2000/08/08
: CIA-RDP96-00788R001 800050001 -8
Table
A-b
(contmuea
;
Approved For Release 200MJfe&LASSAPJ&B788R001 800050001 -8
Koixvrrma mihosmvhx
UNCLASSIFIED
Approved For Release 2000/08/08
: CIA-RDP96-00788R001 800050001 -8
Table
A-5
(continued)
UNCLASSIFIED
Approved For Release 2000/08/08
: CIA-RDP96-00788R001 800050001 -8
Approved For Release 2009 PECL Ac BSHIFH FD 0788R001800050001-8 vo a fs] et 3) P PEE EPEC EREEE g nN € < 8] 3 3] 2 3 =} : & om oY he nfl = 2 3 3 wt | on) — eee eee eee Oe, OO, ee > co} a es ee es Es ee ee a Oo ~~ a | ot =] & g 7 Oo art q Cd 1 PS EE EE SE EE EE Es Ee eee eee or Oo) “| & BR es 3 2 38 a n PR 3 xe RQ 2 3 =| ° 8 a & & 2 2 7 g zl . a 8 x & bp + nN Sion Ay Nd E ae ol al dl» s| 3 7- § ti i) f=] > fe zB zg be g NSN f=] “N a “4 i] fal é | El 8} 8] gf zl 3 z P zl el g z al zl &| 3] 2 z S [o) E E vt re Length of Session: rei, 10 pn | Begin Time: End Time i) Ww A. ra) fs) =] fe) oO ed fe) Ere NOILVOQTIVAT IdTYOSNVEL 37 UNCLASSIFIED Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8 (continued ) Table A-5
Approved For Release 2000/bW6d: Ai HFM [90788R001800050001-8 et a) a] s By MA A g pa M ei] | LE sJelels[s|  [s ¢| 8 z 3} Pe < a 0 sv fe] al + = 60 ot oe («)) a. Og = 8 £ 3 28s F n o » E a} « yu 8 R £ < | PF Ey a a a vn 20 2 2) iy ® ie al a a é =| # “A » F a . | ct wal = » oO PEG ee 3 a 7“ et | wy t roa oe g Qo 6 N E N i+} B : Ell = a m ia et a aD or] a} |e : jo | a FI : i é a os . $3 ag °9 O @ a a $ 2 7 s r= ey FF S) aa 3s G4 NOILVMIVAT LETHOSNVUL ° as) 38 UNCLASSIFIED Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8 Table A-5 (continued)
Approved For Release 20(UM£M;AS£tKBb788R001 800050001 -8
UNCLASSIFIED
Approved For Release 2000/08/08
: CIA-RDP96-00788R001 800050001 -8
Table
A-5
(continued)
Approved For Release 204g/Dd GL: AS SHFHS6100788R001800050001-8 (Pepnptouos) = G-y eTqBL Tey Peet ET PLE Ee EF] erceee 2° | 1z7z | aseroay pe dystay . mer fer [rr for [s fe fe ]o [s | > fe] ze] | of os pete} (ete pep pete} {ete pet mr ete} { [ PT po ley (ete pep mer ete} (ete pet mer Pete} ye Pe pep mr OE a efet pet pep edetfe pe pepe ET FO a De ee 4 jp} SaaS eA — a Is8SBIO P tuo Fssag :uoTsseg jo 47WueT NOILVMTIVAT LATYOSNVEL T86T raqusceq FT :938q Sazsutptocg ?2uy.eSZrey, Jo poq zen [emEL pug anne :TaMoTA nf reyexqung $7e3r8y 9F:2T femtL UpZeqg 40 UNCLASSIFIED Approved For Release 2000/08/08 : CIA-RDP96-00788R001800050001-8
Approved For Release 2oUMfiiAftSUH6Do788R001 800050001 -8
(U)
Two types
of
overall
assessment were chosen
to emphasize
the
versatility
of
the evaluation procedure,
(1)
a simple rank ordering based
on weighted
average
scores,
and
(2)
a concept-by-concept,
non-parame tr ic
,
statistical
technique.
Table A-6
shows
the results
of
the
first method,
the rank ordering.
For convenience,
the correct matches
are underlined.
Table A-6
(U)
A RANK ORDERING OF WEIGHTED AVERAGES
(U)
Session/Target
Inverness
Hango
Fernando
de Noronha
Punkaharju
*>v
4.13
2.72
0.22
2.11
1.12
2.65
2.49
2.42
1 .21
1.38
2.22
2.21
1.53
2.52
2.36
0.90
^Scores
computed with non-uniform target relevance factors.
(U)
From Table A-6, we
see
that
there were
3 first-place matches
and
fourth-place match.
The probability
of obtaining
of
4 possible
first-place matches
from chance fluctuations
alone
are
less
than 0.051.
The point
spread between
the best match
(Inverness)
and
the worst match
(Punkaharju)
are
in qualitative agreement with
a subjective "first
look
at
the quality
of
the
transcripts
as well.
(U)
The
second analysis determines
the
significance
of
the difference
between
the
correct concept / target matches
and
a control
set
of matches.
All
concept/ target matches
that
are not
the correct matches
act
as
an
internal
control
set.
To avoid
any
invalid assumptions
as
to
the
correct
parent distribution,
a non-parame trie
statistical
test,
the Mann-Whitney
U-Test, was
chosen
for
the analysis.
UNCLASSIFIED
Approved For Release 2000/08/08
: CIA-RDP96-00788R001 800050001 -8
Approved For Release 2oW/N^A&$tfflfeQo788R001 800050001 -8
(U)
It
is beyond the
scope
of this report
to review the details
of
the Mann-Witney U-Test;
thus,
only the results are quoted here.
The
probability that the
set
of
correct
concept/target
matches
is statistically
indistinguishable from the control
concept/target matches
is
less than
0.071.
(U)
There are
a number
of additional
statistical procedures that
could be used to analyze the results
of
this evaluation technique.
The
two cited above,
however,
represent
a spread in complexity that demonstrates
the internal consistency
of
the basic evaluation procedure.
With only four
similar RV sessions,
the evaluation technique nearly reached the 0.05
level
of
statistical
significance with each of
the two
statistical procedures,
a
result indicating
a successful
outcome with regard to the overall
assessment procedure.
UNCLASSIFIED
Approved For Release 2000/08/08
: CIA-RDP96-00788R001 800050001 -8