: ARBSRUAE OF REAP R GOLA NB/OR., GIA-RDP96-00789R003700210001-3. Introduction May, Utts, and Spottiswoode (1994) proposed Decision Augmentation Theory (DAT) as a general model of anomalous mental phenomena (AMP).” DAT holds that anomalous cognition (AC) informa- tion is included along with the usual inputs that result in a final human decision that favours a “desired” outcome. In statistical parlance, DAT says that a slight, systematic bias is introduced into the decision process by AC. This concept has the advantage of being quite general. We know of no experiment that is devoid of at least one human decision; thus, DAT might be the underlying basis for AMP. May et al. (1994) mathe- matically developed this concept and constructed a retrospective test algorithm than can be applied to existing databases. In this paper, we summarize the theoretical predictions of DAT, review the criteria for valid retrospective tests, and analyze the historical random number generator (RNG) database. In addition, we summarize the findings from one prospective test of DAT (Radin and May, 1985) and com- ment on the published criticisms of an earlier formulation, which was then called Intuitive Data Sorting. We conclude with a discussion of RNG results that provide a strong circumstantial argument against a causal explanation. As part of this review, we show that one biological-AP experiment is better de- scribed by an influence model (Braud, 1990), Review of Decision Augmentation Theory Since the formal discussion of DAT is statistical, we will describe the overall context for the development of the model from that perspective. Consider a random variable, X, that can take on continuous values (e.g., the normal distribution), Examples of X might be the hit rate in an RNG experiment when the number of binary bits in the sequence is large, the swimming velocity of cells, or the mutation rate of bacteria. Let Y be the average computed over values of X, where n is the number of items that are collectively subjected to an AMP influence as the result of a single decision—one trial, and let Z be the appropriate Z-score corresponding to Y. Often this may be equivalent to a single effort period, but it also may include repeated efforts. The key point is that, regardless of the effort style, the average value of the dependent variable is computed over the n values resulting from one decision point. In the exam- ples above, 7 is the sequence length of a single run in an RNG experiment, the number of swimming cells measured during the trial, or the number of bacteria-containing test tubes present during the trial. Under DAT, we assume that the underlying parent distribution of a physical system remains unper- turbed; however, the measurements of the physical system are systematically biased by an AC-mediated informational process. Since the deviations seen in actual experiments tend to be small in magnitude, it is safe to assume that the measurement biases are small and that the sampling distribution will remain normal; therefore, we assume the bias appears as small shifts of the mean and variance of the sampling distribution as: Z ~ N(uz,97}. * The Cognitive Sciences Laboratory has adopted the term anomalous mentalphenomena instead of the more widely known psi. Likewise, we use the terms anomalous cognition and anamatous perturbation for ESP and PK, respectively. We have done so because we believe that these terms are more naturally descriptive of the observables and are neutral in that they do not imply mechanisms. These new terms will be used throughout this paper. Approved For Release 2000/08/08 : CIA-RDP96-00789R003200210001-3 2
Approved For Release 2000/08/08 : CIA-RDP96- 00789R003700210001-3 lications of Decision Augmentation Theory August 1994 This notation means that Z is distributed as a normal distribution with a mean of 4, and a standard deviation of o,. Under the null hypothesis, 4, = 0.0 and a, = 1.0. Review of a Causal Model For comparison sake, we summarize a model for AP interactions. We begin with the assumption that a putative AP force would give rise to a perturbational interaction. What we mean is that given an en- semble of entities (e.g., random binary bits), a small force perturbs, on the average, each member of the ensemble. We call this type of interaction perturbational AP (PAP). In the simplest PAP model, the perturbation induces a change in the mean of the parent distribution but does not effect its variance. We parameterize the mean shift in terms of a multiplier of the initial stan- dard deviation. Thus: 1 = Ho + Exp %; where 1; and jg are the means of the perturbed and unperturbed distributions, respectively, and where Oo is the standard deviation of the unperturbed distribution. ¢4p can be considered the AP effect size. Under the null hypothesis for binary RNG experiments, “; = ug = 0.5, o9 = 0.5, and egp = 0. The expected value and variance of Z? under the mean-chance-expectation (MCE), PAP and DAT as- sumptions for the normal distribution are shown in Table 1. The details of the calculations can be found in May, Utts, and Spottiswoode (1994), Table 1. Normal Parent Distribution Mechanism Figure 1 displays these theoretical calculations for the three mechanisms graphically. Figure 1. Predictions of MCE, PAP and DAT. Approved For Release 2000/08/08 : CIA-RDP96-00789R003200210001-3 3
ARBIRMAGLE SF RELEASE 2000/08/08, : CIA-RDP96-00789R003709210004-3. This formulation predicts grossly different outcomes for these models and, therefore, is ultimately ca- pable of separating them, even for very small perturbations. Retrospective Tests It is possible to apply DAT retrospectively to any body of data that meets certain constraints. It is critical to keep in mind the meaning of n—the number of measures of the dependent variable over which to compute an average during a single trial following a single decision In terms of their predictions for experimental results, the crucial distinction between DAT and the PAP model is the dependence of the results upon n; therefore, experiments which are used to test these theories ideally should be those in which experiment participants are blind to n, and where the distribution of n does not contain extreme outliers. Aside from these considerations, the application of DAT is straight forward. Having identified the unit of analysis and n, simply create a scatter diagram of points (Z7, n) and compute a weighted least square fit to a straight line. Table 1 shows that for the PAP model, the slope of the resulting fit is the square of the AP-effect size. A student’s t-test may be used to test the hypothesis that the AP-effect size is zero, and thus test for the validity of the PAP model. If the slope is zero, these same tables show that the intercept may be interpreted as an AC strength parameter for DAT. In other words, an intercept larger than one would support the DAT model, while a slope greater then zero would support he PAP model. Monte Carlo Verification The expressions shown in Table 1 are representations which arise from simple algebraic manipulations of the basic mathematical assumptions of the PAP and DAT models. To verify that these expressions give the expected results, we used a published pseudo random number generator (Lewis, 1975) with well-understood properties to produced data that mimicked the results under three models (i.e., MCE, PAP and DAT). Our standard implementation of the PRNG allows the integers in the range (0,2!5—1] as potential seeds. For the sequence lengths 100, 500, 1000, and 5000, we computed Z-scores for all possible seeds with an effect size of 0.0 to simulate MCE and an effect size of 0.03 to simulate PAP. We used the fact that if the effect size varies as / //n, DAT and PAP are degenerate to simulate DAT. For this case we used effect sizes of 0.030, 0.0134, 0.0095, and 0.0042 for the above sequence lengths, respec- tively. Figures 2a-c show the results of 100 trials, which were chosen randomly from the appropriate Z-score data sets, at each of the sequence lengths for each of the models. In each Figure, MCE is indi- cated by a horizontal solid line a Z? = The slope of a least squares fit computed under the MCE simulation was —(2.8/-£2.49) x 1076, which corresponded to a p-value of 0.812 when tested against zero, and the intercept was 1.007-L0.005, which corresponds to a p-value of 0.131 when tested against one. Under the PAP model, an estimate of the effect size using the expression in Table 1 was gap = 0.02880.002, which is in good agreement with 0.03, the value that was used to create the data. Similarly, under DAT the slope was —(2.44+-57.10) x 10-8, which corresponded to a p-value of 0.515 when tested against zero, and the intercept was 1.050-L0.001, which corresponds to a p-value of 2.4 x 1074 when tested against one. Thus, we are able to say that the Monte Carlo simulations confirm the simple formulation shown in Table 1. Approved For Release 2000/08/08 : CIA-RDP96-00789R003200210001-3 4
ARRIRYESHE ADRALRASE GROAN BHF coy A-RDP96-00789RO03Z0024 NOT sb. 1200 2400 3600 4800 6000 (a) por ar poor oe | an One See Os Ce CO I o 1200 2400 3600 4800 6000 (b) 6.0 1 on nee eel Figure 2. Z?vsn for Monte Carlo Simulations of MCE, PAP and DAT. Historical Binary RNG Database Radin and Nelson (1989) analyzed the complete literature (i.e., over 800 individual studies) of con- sciousness-related anomalies in random physical systems. They eloquently demonstrated that a robust statistical anomaly exists in that database. Although they analyzed this data from a number of perspec- tives, they report an average Z/./n effect size of approximately 3 x 10-4, regardless of the analysis type. Radin and Nelson did not report p-values, but they quote a mean Z of 0.645 and a standard devi- ation of 1.601 for 597 studies. We compute a single-mean t-score of 9.844, df = 596 (p< 3.7 x 10723), We returned to the original publications of all the binary RNG studies from those listed by Radin and Nelson and identified 128 studies in which we could compute, or were given, the average Z-score, the number of runs, N, and the sequence length, x, which ranged from 16 to 10,000. For each of these stud- ies we computed: Since we were unable to determine the standard deviations of the Z-scores from the literature, we as- sumed that s, = 1.0 for each study. In addition, if aim directions were reported, we inverted the Z-score Approved For Release 2000/08/08 : CIA-RDP96-00789R003200210001-3 °
ARRIRYRTE A DRALRASE GROMARBAOF, ade A-RDP96-00789RO0I20R A NAD reb4 for the negative aim since the DAT analysis is fundamentally two-tailed. We see from Table 1 that under MCE the expected variance is 2.0 so that the estimated standard deviations for the Z? from Equation 1 is \/2.01N. Figure 3 shows a portion of the 128 data points (Z4,n). MCE is shown as a solid line (i.e., Z? = 7), and the expected best-fit lines for two assumed AP effect sizes of 0.07 and 0.003, respectively, are shown as short dashed lines. We calculated a weighted (i.e., using N/2.0 as the weights) least squares fit to ana + bn straight line for the 128 data points and display it as a long-dashed line. For clarity, we have offset and limited the Z axis and have not shown the error bars for the individual points, but the weights and all the data were used in the least squares fit. We found an intercept of a = 1.036-+0.004. The 1-stan- dard error for the intercept is small and is shown in Figure 3 in the center of the sequence range. The t-score for the intercept being different from 1.0 (ie, ¢ = 9.1, df = 126, p< 48 x 10-9) is in good agreement with that derived from Radin and Nelson’s analysis. Since we set standard deviations for all the Z’s equal to one; and since Radin and Nelson showed that the overall standard deviation was 1.6, we would expect that our analysis would be more conservative than theirs. The important result, however, was that the slope of the best-fit line was b = (1.734319) x 10-6 (t = 0.543, df = 126, p < 0.295), which is not significantly different from zero, The 1-o standard error for the slope encompasses zero. Even though avery small AP effect size might fit the data at large sequence lengths, it is clear in Figure 3 what happens at small sequence lengths; an e4p = 0.003, suggests a linear fit that is significantly below the actual fit. Best Fit Intercept dlope 1.036+0.004 b: — (1.7303.194) «1076 9.10, df= 126 t 0.543, df = 126 48x 10-20 p: 0.295 ~~ “ < eo] =| 2 a N < N fap = 0.003 ~ oT TL LO «Best Fit 1 4000 6000 8000 10000 Sequence Length (n) Figure 3. Binary RNG Database: Slope and Intercept for Best Fit Line The sequence lengths from this database are not symmetric nor are they distributed and do contain ourliers (i.c., median = 64, average = 566). Figure 4 shows that the lower half of the data, however, is symmetric and nearly uniformly distributed (i.e., median = 35, average = 34). Since the criteria for a valid retrospective test is that n should be uniform, we analyzed the two median halves independently. The intercept for the weighted best-fit line for the uniform lower half is a = 1.022-£0.006 (t = 3.63, df = 62, p<2.9 x 10-4), and the slope is b = (—0.0343.70) x 1074 (t = -0.010, df = 62, p <0.504). The Approved For Release 2000/08/08 : CIA-RDP96-00789R003200210001-3 8
Approved For Release 2000/08/08 ; CIA-RDP96-00789R0G32N02 100013 fits for the upper half yield a = 1.064-40.005 (t = 13.47, df = 62, p< 1.2 x 10~*') and b = (-4.522.38) x 107° (t = —1.903, df = 62, p < 0.969), for the intercept and slope, respectively. Since the best retrospective test for DAT is one in which the distribution of n contains no outliers, the statistically zero slope for the fit to the lower half of the data is inconsistent with a simple AP model. Although the same conclusion could be reached from the fits to the database in its entirety (i.e., Figure 3), we suggest caution in that this fit could possibly be distorted by the distribution of the sequence lengths. That is, a few points at large sequence Jengths can easily influence the slope. Since the slope for the upper half of the data is statistically slightly negative, it is problematical to assign an imaginary AP effect size to these data. More likely, the results are distorted by a few outliers in the upper half of the data. PPE See OR OOS CS RS GOES BOON ERED WRENS ONENOG: OODENTS ENE? IEEE Wor 40 60 380 100 Sequence Length (n) Figure 4. Historical Database: Distribution of Sequence Lengths < 64. From these analyses, it appears that Z? does not linearly depend upon the sequence length; however, since the scatter is so large, even a linear model is not a good fit (i.e., X? = 171.2, df = 125, p < 0.0038), where X? is a goodness-of-fit measure in general given by: 2 2 j=l i where the gj are the errors associated with data point yj, f is the falue of the fitted function at point), and v is the number of data points. A “good” fit to a set of data should lead to a non-significant X2. The fit is not improved by using higher order polynomials (i.c., X2 = 170.8, df = 124; X? = 174.1, df = 123; for quadratics and cubics, respective- ly). If, however, the AP effect size was any monotonic function of n other than the degenerate case where the AP effect size is exactly proportional to 1 /./n, it would manifest as a non-zero slope in the regression analysis. Within the limits of this retrospective analysis, we conclude for RNG experiments that we must reject all causal models of AP which propose a shift of the mean of the parent distribution. Approved For Release 2000/08/08 : CIA-RDP96-00789R003200210001-3’
ications of Decision Augmentation Approved Fo Release 2000/08/08; GIA-RDP96-00789R00920021000 1a, Princeton Engineering Anomalies Research Laboratory RNG Data The historical database we analyzed does not include the extensive RNG data from the Princeton Engi- neering Anomalies Research (PEAR) laboratory since their total number of bits exceeds the total amount in the entire historical database. For example, in a recent report Nelson, Dobyns, Dunne, and Jahn (1991) analyze 5.6 x 106 trials all atn = 200. In this section, we apply DAT retrospectively to their published work where they have examined other sequence lengths; however, even in these cases, they report over five times as much data as in the historical database. Jahn (1982) reported an initial RNG data set with a single operator at n = 200 and 2,000. Data were collected both in the automatic mode (i.e., a single button press produced 50 trials at x) and the manual mode (i.e., a single button press produced one trial at). From a DAT perspective, data were actually collect at four values of n (i.e., 200, 2000, 200 x 50 = 10,000, and 2000 x50 = 1 00,000). Unfortunately data from these two modes were grouped together and reported only at 200 and 2,000 bit/trial. It would seem, therefore, we would be unable to apply DAT to these data. Jahn, however, reports that the differ- ent modes “...give little indication of importance of such factors in the overall performance.” This qual- itative statement suggests that the PAP model is indeed not a good description for these data, because, under PAP, we would expect stronger effects at the longer sequence lengths. Nelson, Jahn, and Dunne (1986) describe an extensive RNG and pseudorandom RNG (PRNG) data- base in the manual mode only (i.e., over 7 x 10° trials); however, whereas Jahn provide the mean and standard deviations for the hits, Nelson et al. report only the means. We are unable to apply DAT to these data, because any assumption about the standard deviations would be highly amplified by the massive data set. As part of a cooperative agreement in 1987 between PEAR and the Cognitive Sciences Program at SRI International, we analyzed a set of RNG data from a single operator.” Since they supplied the raw data for each button press, we were able to analyze this data at two extreme values of n. We combined the individual trial Z-scores for the high and low aims by inverting the sign of the low-aim scores, because our analysis is two-tailed, in that we examine Z?. Given that the data sets at n = 200 and 100,000 were independently significant (Stouffer’s Z of 3.37 and 2.45, respectively), and given the wide separation between the sequence lengths, we used DAT as a ret- rospective test on these two data points. Because we are examining only two values of, we do not compute a best-fit slope. Instead, as outlined in May, Utts, and Spottiswoode (1994), we compare the PAP prediction to the actual data at a single value of n. Atn = 200, 5918 trials yielded Z = 0.044 + 1.030 and Z? = 1.063 + 0.019, We compute a proposed AP effect size Z /\/200 = 3.10 x 10-3. With this effect size, we computed what would be expected under the PAP model atn = 100,000. Using the theoretical expressions in Table 1, we computed Z? = 1.961 + 0.099. The 1-sigma error is derived from the theoretical variance divided by the actual number of trials (597) at n = 100,000. The observed values were Z = 0.100 = 0.997 and Z? = 1.002 + 0.050. A t-test between the observed and expect values of Z? gives t = 8.643, df = 1192. Considering this t as equivalent to a Z, the data at n = 100,000 fails to meet what would be expected under the causal PAP model by * We thank R. Jahn, B. Dunne, and R. Nelson for providing this raw data for our analysis in 1987. Approved For Release 2000/08/08 : CIA-RDP96-00789R003200210001-3 ®
Approved For Release 2000/08/08 ., GIA-RDP96-00789R00920024 000123, cations of Decision Augmentation ugus 86-0. Suppose, however, that the effect size observed at nz = 100,000 (3.18 x 1 0-4) better represents the AP effect size. We computed the predicted value of Z? = 1.00002 + 0.018 forn = 200. Using at-test for the difference between the observed value and this predicted one gives t = 2.398, df=11,834. The PAP model fails in this direction by more than 2.4-0. DAT predicts that Z? would be statistically equiva- lent at the two sequence lengths, and we find that to be the case (¢ = 1.14, df = 6513, p 0.127). Jahn (1982) indicates in their 1982 PEAR RNG data that “Traced back to the elemental binary samples, these values imply directed inversion from chance behavior of about one or one and a half bits in every one thousand...” Assuming /.5 excess bits/1000, we calculate an AP effect size of 0.003, which is consis- tent with the observed value in their = 200 data set. Thus, we are forced to conclude that this data set from PEAR is inconsistent with the simple PAP model, and that Jahn’s statement is not a good descrip- tion of the anomaly. We urge caution in interpreting these calculations. As is often the case in a retrospective analysis, some of the required criteria for a meaningful test are violated. These data were not collected when the oper- ators were blind to the sequence length. Secondly, these data represent only a fraction of PEAR’s RNG database. A Prospective Test of DAT In developing a methodology for future tests, Radin and May (1986) worked with two operators who had previously demonstrated strong AP ability in RNG studies. They used a pseudorandom number generator (PRNG), which was based on a shift-register algorithm by Kendell and has been shown to meet the general criteria for “randomness” (Lewis, 1975), to create the binary sequences so that timing considerations could be examined. The operators were blind to which of nine different sequences (i.e., n = 101, 201, 401, 701, 1001, 2001, 4001, 7001, 10001 bits)" were used in any given trial, and the program was such that the trials lasted for a fixed time period and feedback was presented only after the trial was complete. Thus, the criteria for a valid test of DAT had been met, except that the source of the binary bits was a PRNG. We re-analyzed the combined data from this experiment with the current Z-score formalism of DAT. For the 200 individual runs (i.e. 10 at each of the sequence lengths for each of the two participants) we found the best fit line to yield a slope = 43x 1078 + 1.6x10~° (t = 0.028, df = 8, p < 0.489) and an intercept = 1.16 + 0.06 (t = 2.89, df = 8, p < 0.01). The slope easily encompasses zero and is not significantly different from zero, the significance level is consistent with what Radin and May reported earlier. Since the PRNG seeds and bit streams were saved for each trial, it was possible to determine if the ex- periment sequences exactly matched the ones produced by the shift register algorithm; they did. Since our UNIX-based Sun Microsystems workstations were synchronized to the system clock, any momen- tary interruption of the clock would “crash” the machine, but no such crashes occurred. Therefore, we believe mo casual interaction occurred. To explore the timing aspects of the experiment Radin and May reran each run with PRNG seeds rang- ing from —5 to +5 clock ticks (i-e., 20 ms/tick) from the actual seed used in the run. We plot the resulting * The original JDS analysis required the sequence lengths to odd because of the logarithmic formalism. Approved For Release 2000/08/08 : CIA-RDP96-00789R003200210001-3 9
ApprOnedshobRAGAAE ARAUHR/ Reo IA-RDP96-00789R003290 £19901 38a run effect sizes, which we computed from the experimental F-ratios (Rosenthal, 1991), for operator 531 in Figure 5. The estimated 1-standard errors are the same for each seed shift and equal 0.057, oO 3 ny -_ 3 a 1) ao} 3) 0) n Relative Seed Position Figure 5. Seed Timing for Operator 531 (298 Runs). Radin and May erroneously concluded that the significant differences between zero and adjacent seed positions was meaningful, and that the DAT ability was effective within 20 milliseconds. In fact, the situation shown in Figure 5 is expected. Differing from true random number generators in which slight changes in timing produce essentially the same sequence, PRNGs produced totally different sequences as a function of single digit seed changes. Thus, it would be surprising if the seed-shift display produced anything but a spike at seed shift zero, We will return to this point in our analysis of some of the pub- lished remarks on our theory. From this prospective test of DAT, we conclude that for PRNGs it is possible to select a proper entry point into a bit stream to produce significant deviations from MCE that are independent of sequence length. The Literature: Review and Comment We have identified six published articles that have commented upon the Intuitive Data Sorting (IDS) theory, the earlier name for DAT. In this section, we chronologically summarize and comment on each report. Walker — September 1987 In his first of two criticisms of IDS, Walker (1987) suggested that his Monte Carlo simulations did not fit the predictions of the IDS model. He generated a single deviant set of 100 bits (i.e., Z = 2.33, Pp <0.01), and he inserted this same sequence as the first 100 bits of 400 otherwise randomly generated sequences ranging from 100 to 10° bits in length. Walker’s analysis of these sequences did not yield a least square’s slope of —0.5 as predicted under the /DS formalism. Walker concluded that the DS theory was incor- rect, Walker’s sequences, however, are not the type that are generated in AP experiments or the type for which the DAT/IDS model is valid. Approved For Release 2000/08/08 : CIA-RDP96-00789R003200210001 -9