“Approved For Release 2000/08/07 : CIA-RDP96-00787R000700090011-6 Under the null hypothesis of random binomial choices with probability 1/4 and no learning, the probability of observing 2k successes inn trials is approximated by the probability of a normal distribution value, > (« - 7 - yore The design objective was to build a four-state machine, with each state equally likely to occur on each trial, independent of the past sequence of states. “te the machine meets this objective, it should not be possible to devise a rule for future play that significantly differs “from chance. A simple example of such a rule would be to select the machine state observed in the preceding trial; if this strategy were to produce scores significantly above chance (25 percent hits), we would reject the hypothesis of randomness of the machine under test. Before experimentation machines purchased from Aquarius Electronics, Albion, California, were extensively tested for randomness. Data were analyzed on a CDC-6400 computer, and the machine finally selected for use met established criteria for randomness. In devéloping randomness tests, we are guided in part by a knowledge of the machine logic. When one of the four choice keys or the pass key is depressed, the current machine state is displayed; then a brief time after release of the key, a new machine state is established (but not shown to the subject!) by sampling the instantaneous state of a high- Approved For Release 2000/08/07 : CIA-RDP96-00787R000700090011-6
ere | “Approved For Release 2000/08/07 : CIA-RDP96-00787R000700090011-6 speed four-state electronic counter. For the machine to be random, the times of dwell of the counter in. each of the four states must be precisely equal; otherwise, the distribution of outcomes will be biased. The first randomness test is thus based on tallying the number of occurrences of each of the four states. This test should detect a stable bias, yet may miss a drifting bias. To test for this second possibility we also tally the distribution of outcomes in each group of 100 trials, then compute a likelihood ratio test statistic (see below) for each group. Under the null hypothesis of equal likelihood of the four states, these statistic values are distributed approxi- mately as chi-square with three degrees of freedom and their sum for m groups distributed approximately as chi-square with three m degrees of freedom. This test may also detect stable bias, but is not as powerful for this purpose as the first test. Variable bias of still a shorter period, if substantial, can be tested for by tallying the frequency with which the previous machine state is repeated; an overall repeat ratio ("all") significantly above 0.25 is indicative of such bias, If for any reason the machine were to fail to sample the counter to establish a new state, the previous machine state would be repeated. To test for this possibility, we tally the number of repeats following the depression of each key. A repeat ratio significantly greater than 9.25 should be considered a danger signal. Approved For Release 2000/08/07 : CIA-RDP96-00787R000700090011-6
bans Approved For. Release 2000/08/07 : CIA-RDP96-00787R000700090011-6 . We also tally the initial machine states following reset and the transitions between states. In each case, the number of occurrences of each of the four possible outcomes should be approximately equal. When repeats are deleted from the sequence of trials ("nondiagonal transi- tions"), the four states should also be approximately equal in frequency. In testing the null hypothesis of four equally likely outcomes of a trial, a likelihood ratio test is used. The statistic if \- n/4 -2 n. &n () iL. n, i i=1 under the null hypothesis is distributed approximately as chi-square with three degrees of freedom, with rejection for large values of this statistic.* The computer program used in testing randomness includes a subroutine for computing the probability of a chi-square value as large or larger than that observed. In testing the null hypothesis that the probability of a.repeat is 0.25, the binomial probability of obtaining the observed number K or more repeats in N trials is computed. For K greater than 1000, a normal distribution approximation is computed, assuming the statistic K - 1/2 N (28 ~ 0.29 = N a i i i a ie a ee *Alexander Mood, Introduction to the Theory of Statistics (McGraw Hill, New York, 1950). . Approved For Release 2000/08/07 : CIA-RDP96-00787R000700090011-6
beg ‘Approved For Release 2000/08/07 : CIA-RDP96-00787R000700090011-6 ~ The typical test pattern used was six passes followed by 25 choices of one color, repeating this for each of the four colors. In this way each of the five keys other than resetwere given approximately equal use. Typically, 2000 to 6000 trials were made in cach sitting. In the absence of any unusual results in the randomness tests, a minimum of 10,009 trials were made before using a machine with experimental subjects, With 10,000 trials, the expected fraction of repeats is 0.25 with a standard deviation of 3/200 =~ 0.00866. | A computer listing of the results of randomness tests is included in Table 1. No significant departures from randomness were observed. Subject Data Data was collected from subjects Sl through SG. Each subject was asked to complete 100 25-trial runs (i.e., a total of 2500 trials each). The results are tabulated in Table 2. (One subject, S3, declined to complete the 2500~trial run, indicating a lack of rapport with the machine and, hence, a lack of motivation for the task.) For the six subjects, only one (S2) scored significantly above chance. For the 2500 trials that subject averaged 29.36 hits/100 trials rather than the expected 25/100, a result whose a priori probability under the null hypothesis is p = 3x10", His scores are plotted in Figure 12. The statistics of the machine during the successful run of subject S2 were tabulated for the entire 3488 machine transitions (2509 choices, Approved For Release 2000/08/07 : CIA-RDP96-00787R000700090011-6
Gre " Approved. For Release 2000/08/07, GIA-RDP96-00787R000700090011-6 departure from random expectation during the successful run, and therefore, the significant result cannot be attributed to machine malfunction, At a later time, subject $2 was asked to repeat the entire exper- iment, and he was able to replicate successfully a high mean scoring rate (27.88/100 average over 2509 trials, a result whose a priori oo. -4 probability under the null hypothesis is p = 4.8x10 ). We thus conclude from this part of the study that of the ‘six sub- jects tested, one subject (S2) generated a significant result replicable and not attributable to machine malfunction. Finally, the study taken as a whole (15,750 trials) was significant, yielding an average scoring rate 26.47 hits/100 trials, a result whose a priori probability under the null hypothesis is p = 1.1x10 7 The bit rate associated with the information channel can be cal-~ culated from me R= H(x) - H (x) , ue ¥ where H(x) is the uncertainty of the source message containing symbols with a& priori probability P a 4 H(x) = ; Plog P, i 2 oi r i=l and H (x) is the conditional entropy based on the a posteriori pro- y , ; I babilities that a received symbol was actually transmitted 4 H (x) = + > P(i,j) log. P.(j). y 2 iF ° i,j=1 : Approved For Release 2000/08/07 : CIA-RDP96-00787R000700090011-6
bere Approved For | Release 2000/08/07 ; CIA- RDP96- 00787R000700090011 -6 ‘st run, with Pe = 1/4, P(k,k) = 2936, and an average of 30 seconds per choice, we have a source “uncertainty H(x) = 2 bits and a calculated bit rate R w 0.007 bits/symbol or 4 R/T x 2x10 bits/sec. Approved For Release 2000/08/07 : CIA-RDP96-00787R000700090011-6
fre “Approved For Release 2000/08/07 : CIA-RDP96-00787R000700090011-6 Table 1 PRE-EXPERIMENT RANDOMNESS TESTS * Buttons Initial states 107 116 113 1.996 Transitions Y 728 | 764° 765 2.973 1 G ) 777 | 784 773 6.745 Oo Fron B 776 796 810 1.158° R 787 852 803 2.877 All states | 3175 | 3312 3264 5.667 Nondiagonal , transitions 2340 2412 2341 2.630 Diagonal transitions 728 784 810 5.414 CO 7 . Bionomial Key N-Trials Repeats fo oe Prob. Diagonal Yellow |. 2774 705 0.2541 0.313 transitions as a function Green _ 2755 674 0.2446 0.748 of key press Blue 2761 706 0.2557 0.250 Red 2742 667 0.2433 0.793 Pass 1614 375 0.2323 0.953 All 12646 3127 0.2473 0.763 ie a nr re rs ee Randomness in groups of 100 trials: Chi-sq. = 299.6141 D.F, = 345 Prob, = 0.9628 ne ee ee rt ae A A A RAR ee et Approved For Release 2000/08/07 : CIA-RDP96-00787R000700090011-6
: CIA-RDP96-00787R000700090011-6 Approved For Release 2000/08/07 pabLe era Test Description Scaring, ee - _ -—_ 1 Sl $2 \ $3 | s4 | $5 | $6 7 _—-- ee ce ee ee oe a ne | ee eee | Halstead Category Nonverbal test requiring abstraction of conceptual relation- j rv 33 36 6 Test ships. Score: Total errors. “ 28 —_—+ ___-| _ Tactual Performance Requires placement of 10 geometrically shaped blocks in Test their correct locations on 4 formboard while blindfolded. 16,4 11.8 7.7 7.7 11.4 6.9 Separate RT, LT, and bimanual trials. Score: Total time (min.)- Speech Perception Discrimination of non-word speech sounds. b 2 0 Test Score: Total errors. 2 5 3 Seashore Rhythm Test Discrimination of nonverbal rhythms. Score: Number correct. 27 25 28 29 26 29 —y_ Finger Tapping Test Measure of finger oscillation rate for 10-sec. period, both RT/LT  RT/LT RT/LT  RT/LT RT/LT RT/LT RT and LT hand trials. Score: No. taps/10 sec. 53/50 53/49 48/47 54/53 47/47 = 48/43 Trail Making Test Requires connecting numbered circles in order from 1 to 25. 40 16 18 19 (Part A) Paper and pencil task. Score: Total times (sec) 30 27 ——— ——— Trail Making Test Requires connecting alphabetic and numbered circles by 56 50 55 50 54 5 (Part B) alternating l+A*2~B, etc. Score: Total time (sec) 3 Knox Cube Test Measure of attention span and immediate visual memory. Score: Number correct. Raven Progressive Nonverbal intelligence test involving spatial matrices. Matrices Score: Number correct. 39 33 49 33 0 54 Verbal’ Concept Requires abstraction of verbal conceptual relationships. / Attainment Test Score: Number correct. 22 24 27 23 / 24 24 : : . . . : . Total: Buschke Memory Test Requires learning a 20-word list in a maximum of 12 trials with 5 repetition of words omitted after each trial. Score: Max. no. mee 17/20 18/20 19/20 20/20 20/20 words correctly remembered; List: no- words consistently remembered 8/20. 14/20 11/20 16/20 15/20 16/20 L (8 trials) (7 trials Grooved Pegboard Test Requires insertion of 25 pegs in their holes in a pegboard. Both RT/LT RI/LT RT/LT RT/LT  RI/LT 9 RT/LT RT and LT hand trials. Score: Total time (sec). 76/74 69/70 58/67 59/67 BR 48/50 a ‘o Spatial Relations 7] Requires mental rotation and identification of figures _ Subtest of the PMA rotated in 2 dimensions. Score: no. correct — m0 errors. ~ ~ ~ 60 52 Requires tracing outline of simple figure hidden tithin Poor Avg. - v.good outst. outst. Gottschaldt Hidden Figures Test lines of more complex figure. Score: Time and no correct. wen ae : CIA-RDP96-00787R000700090011-6 Approved For Release 2000/08/07