Approved For Release 2002/05/17
: CIA-RDP96-00789R002900290001-9
-6CGRCT
REVIEW OF PROJECT PROCEDURES
Date
of
Publication
5 April
1991
This
is
a Department
o-f
Defense
Intelligence Document prepared
by the
Technology
Assessment and Support
O-f-fice,
Directorate for Scientific
and Technical
Intelligence,
Defense
Intelligence Agency.
PREPARED BY
Technology Assessment and Support Office
(DT-S)
NOT RELEASABLE TO FOREIGN NATIONALS
REPRODUCTION REQUIRES APPROVAL OF
ORIGINATOR OR HIGHER DOD AUTHORITY.
FURTHER DISSEMINATION ONLY
AS DIRECTED
BY DT OR HIGHER DOD AUTHORITY.
CLASSIFIED
BY:
DIA/DT
DECLASSIFY
ON:
OADR
GCGR EPlBOFflil UMDIS
Approved For ReleaSMITE
RDP96-00789R002900290001 -9
Approved For Release 2002/05/17
: CIA-RDP96-00789R002900290001-9
UNCLASSIFIED
table of contents
Page
I.
PURPOSE
j
II.
SCOPE
j
III.
SUMMARY
j
IV.
BACKGROUND
t
V.
SHORTFALLS/REVISIONS
APPENDIX
A.
EVALUATION FLOW CHART
B.
REPORT FORMAT
C.
SUMMARY OF
INFORMATION FORMAT
UNCLASSIFIED
Approved For Release 2002/05/17
: CIA-RDP96-00789R002900290001-9
Approved For Release 2002/05/17
: CIA-RDP96-00789R002900290001-9
UNCLASSIFIED
I.
(U)
PURPOSE;
(U)
Purpose
of
this report
is
to
review initial
STAR
GATE
procedures and
to
provide
recommendations
for
revisions
based
on
recent experience including DESERT STORM tasking.
II.
(U)
SCOPE ;
(S/NF)
This
report examines
data
development,
evaluation
and
assessment procedures for
data generated
by STAR GATE sources.
III.
(U)
SUMMARY;
A.
(S/NF)
Lessons
learned
-from recent DESERT STORM
tasking have
led
to several
.immediate changes
in STAR GATE procedures
as discussed
in
Section
IV.
Some recommendations
-for
-future
revisions are also
identified.
B.
(S/NF)
Evaluations
of
future STAR
GATE reporting
will
be
prepared
in accordance with the criteria and provisions
of
DIAM 58-13,
Subject:
The Department
of
Defense HUMINT Management System,
dated
March
1988.
IV.
(U)
BACKGROUND ;
A.
(S/NF)
In December
1990
and January
1991,
initial
efforts
were made
to provide
an effective and viable system
to evaluate STAR
GATE reporting.
These procedures are
illustrated
in Figure
1.
The
INCLASSI FI ED
- 1 -
Approved For Release 2002/05/17
: CIA-RDP96-00789R002900290001-9
Approved For Release 2002/05/17
: CIA-RDP96-00789R002900290001-9
initial
proposals resulted
in
the
implementation
of procedures that
attempted
to assess
both the
utility
and the
value
of
STAR
GATE
reporting
.
B.
(S/NF)
During
the course
of
an off-site conference held
on
*
and
8 March
1991
at
Fort
Dietrich,
senior analysts from DT-5A,
DT-3B,
and
DT-S,
identified shortfalls
in current evaluation procedures
and
prescribed revisions
that
would allow
for
the
implementation
of
an
objective
and workable
system.
The senior
analysts
believed
that
assessments
relating
to the accuracy
of
STAR
GATE reporting could
best
be addressed
by
applying applicable scientific methodologies
to
STAR GATE
material.
Procedures already exist
and
can be implemented
available project
ADF' methods.
Specific accuracy assessments would be
determined by
intelligence analysts
in conjunction
with appropriate
outside scientific
review.
The same analysts were
of
the opinion that
the intelligence value
or utility
of
STAR GATE reporting
could best
be
determined
by select
intelligence
analysts directly
involved
with
specific
task
areas.
C.
(U)
What
follows are
the results
of
a comprehensive group
effort conducted
to identify
procedural
shortfalls and recommend
the
implementation
of
the following
revisions.
V.
(S/NF/SG/LIMDI5)
SHORTFALLS AND REVISIONS ;
^
-
SHORTFALL;
Need
to
identify action officers with
whom DT-3B
can
deal
directly to expedite the evaluation
process and
insure that
such evaluations are accomplished
in
the most successful
manner.
GATE
/
Approved For Release 2002/05/1 7^"CIA-RDP96-00789R00290029000 1-9
Approved For Release 20Q§^fif£^l^RDP96-00789R002900290001-9
SGI J
analytical
evaluation
process.
Direct
involvement with
these
(and
other)
scientific analysts should help
in
better
task definition
and
in
identifying
ways for improving potential
utility.
SHORTFALL ;
Procedures
to date
appear burdensome
and
need
streamlining.
“
REVISI ON;
Procedures developed
by the R&D community
should
be
used
for
assessing
the
accuracy
of
project
reporting.
The
intelligence
value
of
STAR
GATE
reporting
must
be
determined
by
intelligence
community
analysts
working
from
an
all
source
intelligence perspective and environment.
This should be based
on an
agreed-upon definition
of
utility
and what
target
data
would
be
required
to satisfy the utility criteria.
SHORTFALL ;
Need
to assess
whether
our
quantifiable answer
categories would be expanded
to reflect the range
of
uncertainty when
one
is evaluating
a DT-S report.
-
REVISION ;
Specificity
in
the judicious selection
of
mission
requirements
will
go
a
long
way towards
streamlining the
process.
Target selection must
be based
on targets
against
which STAR GATE has
NO FORN/ L I MD I
STAR GATE
-3-
Approved For Release 2002/05/17
: CIA-RDP96-00789R002900290001-9
Approved For Release 20^05g7^I^DP96-00789R002900290001-9
a
-favorable track
record.
To this extent,
limits
to STAR BATE
data
(or
project
type)
,
need
to be
provided
in
advance
to the
tasking
el ements.
SHORTFALL;
Need
to
re-accompl i sh
certain
original
tasking,
updating
it
as necessary
to the current
situation.
REVISION;
Tasking
should be
levied
within
specific
time
windows
and
not
be
left
open-ended.
Requirement originators should be
reminded
of
their responsibility to update requirements and changes
of
collection emphasis.
SHORTFALL ;
Senior
analysts
need
to
insure
that
the
intelligence
evaluations
(ie,
utility
assessments)
of
project
reporting
are
timely,
objective,
substantive
and
prepared
in
consonance with established intelligence community procedures.
-
REVISI ON;
STAR
GATE
reporting
needs
to
be
evaluated
in
consonance
with
the
DIA
HUMINT
evaluation
scheme
paraphrased
substantially as follows;
RATING
VALUE
Of
Major Significance
Having
Impact
on National
Policy and Decision-Making
Of
High Value
Best Effort
- First Report
in
a Field
of
Endeavor
Of
Value
Supplements
- Updates
-
Confirms Known Data
FORN/ L I MD I
STAR GATE
-4-
Approved For Release 2002/05/17
; CIA-RDP96-00789R002900290001-9
Approved For Release 2fflmi5U^yA-RDP96-00789R002900290001 .9
Ik
L
Of
Low Value
Poor Quality
- Not
Timely
f
No Value
Erroneous
- Misleading
There
is
another
consideration.
Proper
assessment procedures
•for
either accuracy
or utility
require that
criteria for
evaluation be
specific
in
advance.
Consequently,
it
should
be
appropriate for
a
candidate
user
(or
task originator)
to
clearly define
what type
of
data would
have
potential
utility,
and
how
it
would
be
used
if
provided
and how
it
would
be evaluated.
This
type
of
data could
include specific
essential
elements
of
information
(EEI)
and required
timing.
A troublesome problem may be encountered
if
the specific
EEI
(if
posed
as
a question)
provides too much background
information for
the source
that could adversely influence session
results.
To avoid
such
difficulty,
it may be necessary to conduct
the session
in several
phases,
time
permitting.
This
issue
is
not easy
to
resolve and
remains
as
a future
action
item.
<A
basic review
of
this overall
issue
can
be found
in DT-S-1010-B,
13 Dec
1990,
Evaluation Methods).
SHORTFALL;
Need
to reconsider acceptance
of
tasking based
on
availability and timeliness
of
known ground
truth.
“
REVISION ;
The STAR SATE collection management system
must be
selective
in
its acceptance
of
requirements for
which there
is
a
likelihood
of
learning
ground truth
within
a
reasonable amount
of
time.
A procedure
needs
to be
established that
allows for
rapid
follow-up
for task
refinement.
L IMD I
S
STAR GATE
- 5 -
Approved For Release 2002/05/17
: CIA-RDP96-00789R002900290001-9
Approved For Release 2002/05/1 7
:
P96-00789R002900290001 -9
SHORTFALL :
Multiple source
reporting
in
a combined summary
•format
is
confusing
precludes
affixing
responsibility
to
the
reporting.
REVISION :
The
principle
of
"one
source one report"
must
be
strictly
adhered
to
so
that
reporting
may be
attributable
to
a
specific source.
Consequently,
data from any source
will
be reported
separately
or
in
easily
identifiable and
separated sections
of
an
overall
summary report.
SHORTFALL:
Current
STAR GATE
reporting
is inconsistent with
other established HUMINT reporting formats.
-
REVISION :
Reports
need
to be prepared
in accordance
with the
criteria used
in the
preparation
of
Intelligence
Information
Reports
(IIR's).
Each
report will
bear
a numerical
source designator
that
will
protect the identity
of
the source.
An example of
the reporting
format
is
at
Figure
2.
A
Summary of
Information
(SOI)
will
be
prepared
by
the source.
It
will
reflect
a project
number
,
source
number,
session
number,
and date
of
report.
The
SOI
will
be attached
as
an addendum
to the
formal
report.
An example
of
an
SOI
is at
Figure
3.
-
SHORTFALL :
STAR GATE predictions are rarely accurate.
REVISION:
Predictive reporting
is
indeed
the project's
weakest
feature.
The results
of
searches are conducted
at
a slightly
higher
rate
of
success.
Hi stor i cal ly
,
the project
s
track record
GATE
/
- 6 -
Approved For Release 2002/05/17
: CIA-RDP96-00789R002900290001-9
Approved For Release 2002/05/17
: CIA-RDP96-00789R002900290001-9
reveals
a relatively
high
rate
of
success
when mission
tasking
is
targeted against
fixed,
static targets located
in past
or current time
windows.
Consequently,
project
tasking
should
be
prioritised
according
to estimated success probability.
~
SHORTFALL S
STAR GATE project reporting
should
not stand alone.
REVISION S
Senior analysts need
to insure
that
STAR GATE
is
an
integral
part
of
the
all -source collection management
system,
a
key
tip-off
player acting
in concert
with
other
intelligence disciplines.
SHORTFALL S
Highlights
of
singularly outstanding
reporting
should be recorded
and
made readily retrievable.
-
REVISION S
A
"Kudo File"
will
be established
to facilitate
the
retrieval
of
examples
of
high quality
reporting.
Factors that may
have
contributed
to
these
results
will
be
identified
whenever
possible.
\I0F0RN/LIMDIS
STAR
1 GATE
-7-
Approved For Release 2002/05/17
: CIA-RDP96-00789R002900290001-9
Approved For Release 2002/05/17
: CIA-RDP96-00789R002900290001-9
DATA DEVELOPMENT, EVALUATION AND ASSESSMENT
REQUIREMENTS
i
wmmmmi
liANA^lli
OUTPUT
yX«!^j^>W^<WWMWiWW8WMMM8MW»aWWW0WMWMWMWMMWW0WW8»^^
iI|,;JNDEPEN DENTlll®
lllllRESPOKsilllll
to ®igi
TASK REQUIREMENTS?
I
lASSESgMElill
100MR|RiSQS|
ran
OVERSIGHT
lS?PANElS-I
C6487.06JP
Approved For Release 2002/05/17
: CIA-RDP96-00789R002900290001-9
Approved For Release 2002£0^/1£
:
f/jC|A1R^P96-00789R002900290001-9
PROJECT STAR GATE
PROJECT NUMBER
::
DATE OF
SESSION::
SESSION STARTs
METHOD
::
SOURCE
NUMBER'S
DATE OF REPORTS
SESSION
ENDS
SESSION
NUMBER::
( S / IMF )
BACKGROUND s
2.
(S/IMF)
ORIGINATOR
REQUIREMENT'S
3.
(S/IMF)
SOURCE
TASK:
4.
(S/NF)
OPS OFFICER COMMENTS:
Address
inclemencies.
Include any info not contained
in Summary
o-f
information
(SOI)
p r ov i d ed
by
sourc
e
5.
(S/IMF)
EVALUATION:
Enter numerical
designator
IAW DIAM 58-
13.
6.
(S/NF)
SEARCH EVALUATIONS
Not Releasable to
foreign nationals
L. i m i t e cl
d i s t ribution
CLASSIFIED
BY:
DIA/DT
DECLASSIFY
ONs
OADR
dmmmm imoforn / 1. 1 md i s
APPENDIX
^
Figure
Approved For Release 2002/05/17
: CIA-RDP96-00789R002900290001-9
Approved For Release 2002/05/17 : CIA-RDP96-00789R002900290001-9
EXAMPLE
PROJECT:
SOURCE
!
DATE:
sess
:i: ON
:
SUMMARY OF
INFORMATION
THE SOI
IS ATTACHED TO THE REPORT
SKETCHES PREPARED BY
THE SOURCE
I N
sol
SUPPORT OF FINDINGS FOLLOW
*
APPENDIX
C| Figure
Approved For Release 2002/05/17
: CIA-RDP96-00789R002900290001-9