Approved For Release 249RG COE ASS PPFP B°0789R002200380001-6 SOC Reviewers’ Comments, Objective I, Task 1 (Meta-Analysis of Forced-Choice Precognition Experiments) (verbatim transcription—-not edited) NAME: Robert Morris Comments: 1. This report is a valuable application of meta-analytic techniques to an important body of data. Its extension to assessing favorable conditions and flaw analysis is very important. 2. The author should describe the basis for declaring that a given study is a precognition study and warrants its inclusion. He should also clarify on what basis he decided that experimenter’s expectation was for psi missing as opposed to chance or merely less positive results for condition B versus condition A. There are some additional analyses that now could be done on these data, and they are noted in the margins. The data amendable to IDS interpretation could also be added. Recommendation: Yes November 4, 1988 NAME: James Press Comments: Ll. A great deal of effort has been extended in this endeavor. Mean 3 scores are reported, and Std Dev from the mean which is more than three times the mean, so the mean doesn’t look very impressive. Moreover, no standard deviations are given for the various studies, nor other measures of the distribution of results in the various studies. Only the mean 3 score is given. An effort was made to study “quality” of the studies (page 13), and each study was giving a quality rating. But all studies were jointly evaluated in the meta analysis, good and bad, so what’s the conclusion? What does it mean to merge results of good and bad studies? Hard to evaluate this in available time. Recommendation: Yes November 4, 1988 NAME: Brian Skyrms Comments: 1. A careful review of the literature, but I’m not sure what it proves. What is the real theoretical status of the “quality rating”? Recommendation: Yes November 3, 1988 A-24 UNCLASSIFIED Approved For Release 2000/08/08 : CIA-RDP96-00789R002200380001-6
Approved For Release 2009 EL A'S SPE PE pp 789R002200380001 -6 NAME: Mike Wartell Comments: 1. Good study. Seems carefully done. Interesting result. Recommendation: Yes November 4, 1988 NAME: Phil Zimbardo Comments: 1. This is a superb example of meta-analysis that helps to clarify some of the central issues in F.C. precog experiments (what variables co-vary with study outcome, etc.) and to yield some strong support for significant effect of directional hitting. 2. Excellent attempt to quantify study quality. 3. Also this analysis dispels several myths promoted by critics of parapsychology, e.g., that effects disappear as methodological rigor increases, or that selective reporting affects the cumulative significance of precog. studies, 4, Page 4 ~ type - of omitted. Recommendation: Yes November 4, 1988 A-25 UNCLASSIFIED Approved For Release 2000/08/08 : CIA-RDP96-00789R002200380001-6
Approved For Release 2ppAPE67@8A SS FEE) 00789R002200380001 -6 REVIEWER’S COMMENTS The attached report titled: has been reviewed by the undersigned. My assessment of the research design, statistical protocols employed, the analyses of the data, and conclusions reached in this report is as follows: Additional comments: SIGNED DATE A-2 UNCLASSIFIED Approved For Release 2000/08/08 : CIA-RDP96-00789R002200380001-6
Approved For Release 2P9NPEESA SS FRIED 00789R002200380001-6 General Response to the Scientific Oversight Committee’s Comments NAME: Comments: 1. Edwin C. May, SRI International All the technical suggestions by the SOC have been incorporated in the FY 1988 final reports. All the positive comments are appreciated, and require no response from SRI. Two specific items have been addressed in response to Professor Zimbardo’s comments. Both of his comments reflect a general feeling of the Committee at large. Therefore, SRI provides a general response. Given that SRI is researching a controversial area, it is always tempting to think of each experiment as an attempt at proof-of-principle. In all cases except the forced-choice remote viewing, the protocols were designed to answer specific questions assuming the existence of an anomaly. The formal protocols are designed to eliminate all sources of known sensory leakage, but are not as complete as they would be in a proof-of-principle experiment. Many of the suggestions for improvement of protocols toward proof-of-principle, are being incorporated in the design of the FY 1989 experiments (e.g., multiple judges and across~analyst reliability measures). A-3 UNCLASSIFIED Approved For Release 2000/08/08 : CIA-RDP96-00789R002200380001-6
Approved For Release 200)NR(OB ACYSRP PED 0789R002200380001-6 SOC Reviewers’ Comments, Objective A, Task 3 (Final Technical Report, Enhanced Human Performance Investigation) (verbatim transcription——not edited) NAME: Robert Morris Comments: 1. This report essentially is a summary of the other reports. It seems a fair, accurate representation of the information contained in them. Recommendation: Yes November 4, 1988 NAME: Michael Wartell Comments: 1, Straightforward oversight description. I still believe that even more emphasis needs to be placed on screening and focus should be brought to bear on that in the main. Recommendation: Yes November 3, 1988 NAME: Chris Zarafonetis Comments: 1. This is a good summary/overview and review of progress to date. The Program objectives and resources are delineated and progress noted briefly since the study findings are given in detailed, specific reports which are also available and reviewed by me. Recommendation: Yes November 4, 1988 NAME: Phil Zimbardo Comments: 1. This is an overview only; it is a clear, concise summary of research progress. 2. See page 6 for grammar error (last paragraph, line 4, data does should be changed to data do). 3. There should be added a_ section on problems of interpretation, conceptual/empirical problems facing the staff. Recommendation: Yes November 4, 1988 A-4 UNCLASSIFIED Approved For Release 2000/08/08 : CIA-RDP96-00789R002200380001-6
Approved For Release 209 NG ROE ACYSRPTE £07 89R002200380001 -6 SOC Reviewers’ Comments, Objective B, Task 1 (Mass Screening For Psychoenergetic Talent Using A Remote Viewing Task) NAME: Comments: 1. (verbatim transcription——-not edited) Herb Ley Research design and statistical protocol were appropriate to the goal of the study. As noted under “Method of Approach” screening of large groups of people poses both logistical and support problems, but those problems must be faced, perhaps with an accepted penalty of missing a group of persons with RV talent in order to identify the small proportion of persons from the group who have significant RV potential. The analysis of the results of screening was clear, and the conclusions, i.e., the selection of two persons, followed from the analysis. The “yield” from this screening effort was two of 154 persons, or 1.3%. This seems to be a small yield, but lacking any good measures of the distribution of RV talent in the general population, I have to accept it. I have two questions which I doubt can be answered, but, rather, than be bashful, I’ll ask then anyway. Do you have any measure of whether the four groups of subjects were more sympathetic to the RV concept than the general population? Have you considered taking two more persons of the selected group of 18 with the best scores for further tests? I understand and admit my motivation: I would like to see the “yield” increased, if possible, and not discard persons with RV talent that could be further developed. Recommendation: Yes November 1, 1988 NAME: Comments: 1. Robert Morris The research design seems clean. I have noted some places where additional procedural descriptions would help clarify things. Data analysis and statistical protocols seem fine. I think the overall procedure is probably conservative and may exclude some potentially good RV participants. I think group testing should be eliminated where possible. It is not known as a psi-conducive environment (see Honorton Meta-analysis report) and does not resemble the later testing or performance circumstances. Note that for the one group where individual testing was used, the yield is higher. Recommendation: Yes November 4, 1988 A-5 UNCLASSIFIED Approved For Release 2000/08/08 : CIA-RDP96-00789R002200380001-6
Approved For Release 20DPQROB JOSS] BH fF90789R002200380001-6 NAME: James Press Comments: 1. Explain what a “carefully constructed target pool” is (page 2., A.). 2. What were the “independent trials,” were they different subjects or different trials with the same subject (page 2., A.). 3. The 28 combined trials (page 4) are not mutually independent. The 20 are correlated (since they are performed by the same subject) and the 8 are correlated. 4. The p-value of 0.007 (page 4) is problematical because of both the dependence and “the visual correspondence method of judging.” 5. It was not clear how many viewers total we started with from all groups, and successively eliminated until 2 were left with “significant level of ability at p = 0.05.” The rate of finding good viewers should be a fundamental output of this research, and you didn’t report it. Recommendation: Yes October 31, 1988 NAME: Brian Skyrms Comments: 1. This is not really an experiment but rather a search for subjects for experiments. Two were found. (It had been hoped that a larger population could be screened.) They should now be used as subjects in tightly controlled experiments, to see if the effect persists and if it can be enhanced. Recommendation: Yes November 3, 1988 NAME: Yervant Terzian Comments: 1. Page 1 - See underlines in first paragraph. 2. Page 1, paragraph 2 — Tested in controlled experiments? Hard to talk about “natural” given first sentence! 3. Page 4, first paragraph - “Of the 28 combined...” More. 4, Page 5, first paragraph - Pool of people should not matter if phenomena is “human” related. 5. Page 5, last paragraph - Re. “...some distance from the screening auditorium,...” How far? 6. Page 7, Table 2 - Ok. More quantitative needed?? Maybe not. 7. Page 8, 4. - Did individuals in second stage do as well (better? worse?) as during stage one? A-6 UNCLASSIFIED Approved For Release 2000/08/08 : CIA-RDP96-00789R002200380001-6
Approved For Release 2000}¢@@L AlS§ HPPE BD) 789R002200380001-6 8. Page 9, paragraph 1 — Re. “...16 showed qualitative evidence...” At what Table 2 level? 1%, 9. Page 9, last paragraph, last full line -— Define “...first place matches...” 10. Page 11, paragraph 2 - Define “complete misses” 11. Page 11 - Use systematic “group” of people, easily accessible — such as the “prisons,” or the “military,” or “schools” and “colleges.” Has any “age” discrimination taken place? What about “children”? If RV is independent of human prior experience, then children should show this ability equally - or better if “experience clutters RV, and vice-versa. Recommendation: Yes November 7, 1988 NAME: Michael Wartell Comments: 1. The research design, stat protocols, analyses of data, and conclusions are all appropriate. The relatively loose first level screening seems adequate since it is, in fact, a screening device. Then second level Screening is accomplished with proven protocols. Much more screening needs to be done in order to accomplish the goals of this work. Recommendation: Yes November 3, 1988 NAME: Chris Zarafonetis Comments: 1. The study overview, objectives, approach, etc., were appropriate for this mass screening - which required a needle in the haystack effort. 5 groups, ranging 6 to 139 in size were tested. 16 gave qualitative evidence of ability to report target related material, and 9 of 16 participated in Stage II screening. 2 tested to p = 0.05, and other 2 with strong tendency for scoring in right direction. 2. It will be of interest as more persons are found with this “talent’ to note the age and gender of the subjects. Recommendation: Yes November 4, 1988 NAME: Phil Zimbardo Comments: 1. See text comments pages 1, 2, 3, 4, 6, 7, 8, 9, 11. Major method problems. Page 1 comment: The sentence read, “Some of these persons have had spontaneous experiences that lead them to be more or less aware of an extrasensory potential while others do not.” Dr. Zimbardo’s comment is, “With A-7 UNCLASSIFIED Approved For Release 2000/08/08 : CIA-RDP96-00789R002200380001-6