Approved For Release 2000/0ae= CTE RDP 96-00789R000700260002-5 known). Consequently, complete evaluation of the viewer’s data cannot be made until a later time when ground truth does become available. In the interim, reasonable estimates of the overall validity of the viewer’s data can be made for many of the operational projects worked, based on what is generally known or suspected about the target. These interim evaluation results GA yveuld=be updated whenever new ground truth is received. A (S/NF/SS-2) The operational projects pursued by SUN STREAK are approved by the program manager and are, in part, based on the program manager’s familarity with IC needs and on solicitation from others within the IC who have been briefed into the SUN STREAK program. (S/NF/SS~-2) Bho Evaluation; performed for this report covenff all the operational and operational simulation projects (approximately Feu that have been worked by SUN STREAK personnel o GRUm OF Samp Ro cable pr Since 1986. Heyer, @few special operations A MAS Witte Av alas ff consultants from the SRI talent pool. These ORS —S teow overall evaluations prescrbed—-in—this—7 Sports. wx a Approved For Release 2000/08/08 : CIA-RDP96-00789R000700260002-5
Approved For Release 2000/08/Gpe=@HMGOP 96-00789R000700260002-5 II EVALUATION (U) 1. DATA BASE (U) (S/NF/SS-2) The SUN STREAK project maintainga an extensive record of all project activity. Details include project timing, people involved({i.e., viewers, interviewers, and possibly observers), and a variety of other data considered essential for good record keeping and for evaluating project results. This data, along with project summaries, are maintained in an automated data base for convenient retrieval. Copi of project summaries are also sent to the frogram flanagen,for hia review. In addition, all raw data (4b) (i.e., sketches, viewer’s notes) are maintained ina separate file that is available for review and analysis (Additional project record details are in appendix I). (S/NF/SS-2) Evaluations conducted for this report involved a complete reexamination of the entire SUN STREAK operational data base. Many of the earlier projects had only been partially evaluated, or not evaluated at all, due to lack of ad Vas bit Undre C edo suitable ground truth ects were reevaluated at “. this time to adjust, <r new intelligence data that has Aenceé “ > me Ciggaeerteiny become available fer—som—nf—the projects. (S/NF/SS-2) For this evaluation, the data base was subdivided into 6 main project types: (1) Gcientific and ee Approved For Release 2000/08/08 : CIA-RDP96-00789R000700260002-5
WP OE Approved For Release 2000/88TSs CIA. RDP96-00789R000700260002-5 fochnological (S/T)3 (2) counterterrorist (CT); (3) Gounternarcotics (CN) ; Hbcument contents (Doc Total projects worked SY figure 1. (S/NF/SS-2) Of the ne approximately one-half truth is not sufficien approximately one-four totally known (or high fourth, ground truth i asufficient for making (S/NF/SS-2) Some of t example, prediction da CN, many of the CT and prediction category in of a political/militar data will be evaluated categories. 2, EVALUATION TECHNIQUES (S/NF/SS-2) Techniques use operational and simulated o nature of the task and type most difficult to evaluate. (4) counterintelligence (CI); (5) v) Cont); and (6) predictive (pred). for these categories are shown pn a of Cap A —— arly 200 projects worked, cannot be evaluate since ground an tly known at this time. For th of the projects, ground truth is ly certain), and for the other one- s only partially known but considered a reasonable interim evaluation. 0d nead categories can overlap. For ta is also an aspect of most of the a few of the S/T projects. The figure 1 refers mostly o predictions ‘¢~ y nature. 4 future analysis, predictive “ plese C as a separate aspect Of ther S gol (UW) d for evaluating the SUN STREAK perational data base depend on the of project. S/T projects are the This difficulty arises from the ere Approved For Release 2000/08/08 : CIA-RDP96-00789R000700260002-5
Approved For Release 2000/08/08 : a oe corsorooorai 002-5 general complexity existing at most 3/T target sites, a possible ambiguous aspects of known ground truth about the target gite, from the nature of the information desired, and in a few poonate cases, possibly from the (RV targeting method employed. It is om easier to evaluate data or S/T targets if only a single issue, such as presence or absence (of a particular system, for example) is desired, then it is to evaluate how well a viewers’ detailed but possibly fragmentary description correlates with aspects of a complex site. In this case a considerable amount of subjectivity can be involved in evaluating the degree of data /target correlation. (S/NF/SS-2) To assist in reducing overall subjectivity of evaluating complex S/T targets, the viewers’ data is examined and. compared to ground truth with several data categories in mind. These categories are shown in f{gure 2, and include geographic descriptions, large and small scale objects, large and small scale functions, personality data, and predictive data. Not all these categories may be relevant to a specific project, and in some cases may even be part of the RV targeting procedure (e.g. when a photo of target building is used as an RV targeting reference for accessing its unknown contents). (S/NF/SS-2) After identifying the appropriate data category, the next step is to examine the viewers’ raw (or summarized) data for comparison to known or estimated ground truth and to make a best judgement on what approximate degree of data correlation actually a a Approved For Release 2000/08/08 : CIA-RDP96-00789R000700260002-5
Approved For Release 2000 ASST CIA-RDP96-00789R000700260002-5 ¢ exists, Figure 3 defines the scale ratings used along with their - approximate degree of data correlation. (Apy ix II contains detailed instructions for analyst consideration when reviewing the data). (S/NF/SS-2) Final evaluations and summaries are prepared by the q program Wlanager anaf‘his project representative $iumo is not part - Aaptorn ot of SUN STREAK staff) in conjunction with the responsible area analyst or Intelligence Community point-of-contact. Latest intelligence data and reports on that target site are also reviewed during this process. In some cases, area analysta and the IC points—of-contact provide written appraisals to assist in the final evaluation process. These evaluations are recorded on summary forms and are maintained in the brogram fraser’ files. (S/NFPSS-2) An example of an S/T target evaluation is in f 4. In this case the target site was idéwe project was completed in June 1987 and involved four viewers (2 proven and 2 novice). In this example, the bracket ( ) dela molidati=n” den that deita cotagots— indicates a best estinate was he ground truth is not yet SGTA totally known . dash “means that data category was not L“ present in the viewers’ data. One of the viewers 101) attempted to describe the site 6 months in the future. In this case, SG1C SCC sci SG1B Po Some of the data categories (i.e., pan wok Lhige-ee gotta 4 geographic features, large scale objects) are not impo vant sinee—— eee CC pneup ict A oan . Approved For Release 2000/08/08 : CIA-RDP96-00789R000700260002-5