Institutional Members: CEPR, NBER and Università Bocconi WORKING PAPER SERIES Surprised by the Gambler’s and Hot Hand Fallacies? A Truth in the Law of Small Numbers Joshua B. Miller and Adam Sanjurjo Working Paper n. 552 This Version: September 15, 2015 IGIER – Università Bocconi, Via Guglielmo Röntgen 1, 20136 Milano –Italy The opinions expressed in the working papers are those of the authors alone, and not those of the Institute, which takes non institutional policy position, nor those of CEPR, NBER or Università Bocconi.
.£ = 1 and, if i < n, then xi+1 = 0 and, if i > j, then xi−j = 0}|. R1 (x) := |{i ∈ {1, . . . , n} : xi = 1 and, if i < n, then xi+1 = 0}|.
min{n1 ,n0 +1} ) n1 − 1 n0 n1 − r1 = n1 n − 1 min{n1 ,n0 +1} ) n1 − 1 n0 r r1 =1 r1 − 1 r1 − 1 n1 − 1 n1 − 1 n0 − n1 − 1 r1 =1 r1 − 1 r1 − 1 = n − 1 n0 = n − 1 n0 1 − n1 − 1 n0 − n1 − 1 min{n1 ,n0 +1} ) r1 =1 min{n1 ,n0 +1} ) r1 =2 n1 − 1 r1 − 1 n1 − 1 r1 − 1 n0 r1 − 1 n0 − 1 r1 − 2 (r1 − 1) = n − 1 n0 = n − 1 n0 n0 − n1 − 1 n0 − n1 − 1 min{n1 −2,n0 −1} ) n − 2 n1 − 2 n1 − 1 x + 1 n0 − 1 x Combining both cases we have: 1 n − 1 n0 n − 2 n − 1 n0 n − 2 l E[P11|N1 = n1] = ( n ) n1 n0 − 1 − n1 + n1 − 1 n0 n1 − 1 n1 − 2 1 n = ( n ) n1 0 n1 − 1
xi < k . In case 1i, with f1k = 0, the number of terms, which we denote 49 Note that Lk−1 jr1j < n1 − k implies f1k > s1k ≥ 1, which guarantees f1k ≥ 2.
1k , satisfies: U1k := ) C1k r11 ,...,r1k−1 ,s1k j=1 jr1j =n1 s1k =0 = ) r11 ,...,r1k−1 j=1 jr1j =n1 min{n1 ,n0 +1} r1! k−1 j=1 1j + 1 r1 = ) r1 =1 n0 + 1 r1 ) r11 j=1 jr1j =n1 j=1 r1j =r1 n1 −r1 k−1 j=1 1j min{n1 ,n0 +1} = ) r1 =1 n0 + 1 r1 l k−1 ) =0 r1 (−1) £ − 1 − £(k − 1) r1 − 1 where the last line follows by first noting that the inner sum of the third line is the number of compositions (ordered partitions) of n1 − k into r1 − 1 parts, which has generating function (x + x2 + · · · xk−1)r1 (Riordan 1958, p. 124). Therefore the inner sum can be generated as the coefficient on xn1 in the multinomial expansion of (x + x2 + · · · xk−1)r1 . The inner sum of binomial coefficients in the fourth line corresponds to the coefficient on xn1 in the binomial expansion of of an equivalent representation of the generating function xr1 (1 − xk−1)r1 /(1 − x)r1 = (x + x2 + · · · xk−1)r1 . The coefficient in the binomial expansion must agree with the coefficient in the multinomial expansion.50 In case 1ii, with f1k = 1 and Ln xi = k, in which case Pˆ1k (x) is also undefined, all sequences that satisfy this criteria can be constructed by first forming a distinguishable permutation of the r1 − 1 runs of 1s not including the final run of k 1s, which can be done in r1!/( k−1 r1j !) ways, and second placing these runs to the left or the right of the available n0 0s, not including the right end point, which can be done in ( n0 r1 −1 n
) ways with the n0 positions. Summing over all possible 50 The binomial expansion is given by: k−1 r1 r r1 ( \ r +∞ (r 1 + t \ xr1 (1 − x ) = xr1 r1 (−1)t1 xt1 (k−1) · 1 − 2 xt2 (1 − x)r1 t1 t1 =0 t2 =0 r1 − 1 therefore the coefficient on xn1 is L(−1)t1 (r1 )(r1 −1+t2 ) where the sum is taken over all t1 , t2 such that r1 + t1 (k − 1) + t2 = n1 . t1 r1 −1
this criteria is: (r1 − 1)! (s1k − 1)! k−1 r1j ! n0 r1 − 1 f1k − 1 s1k − 1 = n0 + 1 C1k In case 3, in which Pˆ1k (x) is defined with Ln xi < k and f1k ≥ 1, it must be the case that Lk−1 jr1j ≤ n1 − k, as before all sequences that satisfy this criteria can be constructed in
three steps analogously to Lemma 6, and we consider two subcases, sequences that terminate in a 1 (i.e. a run of 1s of length less than k) and sequences that terminate in a (i.e. a run of 0s). For those sequence that terminate in a 1, for a given (r11, . . . , r1k−1, s1k ) the total number of sequences satisfying this criteria is: r1! (r1 − 1)! n0 f1k − 1 = r1 − s1k C1k − j=1 r1j ! (s1k − 1)! k−1 j=1 1j r1 − 1 s1k − 1 n0 + 1 with (r1 − 1)!/((s1k − 1)! k−1 r1j !) being the number of sequences that terminate in a run of 1s of length k or more. For those sequences that terminate in a 0, for a given (r11, . . . , r1k−1, s1k ) the total number of sequences satisfying this criteria is: r1! n0 f1k − 1 n0 + 1 − r1 1k j=1 r1j ! r1 s1k − 1 n0 + 1 therefore, the sum of the k/1-streak momentum statistic across all sequences for which it is defined satisfies: n l ) s1k f1k − s1k E[P1k |N1 = n1] − U1k 1 = r11 ,...,r1k−1 ,s1k j=1 jr1j <n1 −k s1k ≥1 C1k 0 + 1 f1k − 1 + ) C1k r1 − s1k f1k − s1k r11 ,...,r1k−1 ,s1k j=1 jr1j ≤n1 −k s1k ≥1 n0 + 1 f1k + ) C1k
n0 + 1 − r1 f1k − s1k r11 ,...,r1k−1 ,s1k j=1 jr1j ≤n1 −k s1k ≥1 n0 + 1 f1k and this reduces to the formula in the theorem because the final two terms can be combined, and then can be summed over only runs that satisfy Lk−1 jr1j < n1 − k and combined with the first term (because f1k − s1k = 0 if Lk−1 jr1j = n1 − k).51 51 While the first term has the closed form representation L C1k s1k f1k −s1k = (n1 −1)/(n−1), this does not appear to n0 +1 f1k −1 k k be so for the other terms. E[P1k |N1 = n1 ] cannot have a closed form representation in any case, as the term U1k does not allow one.
• B.2 Expected Difference in Proportions The exact formula for the expected difference between the relative frequency of 1 for k/1-streak successor trials and the relative frequency of 1 for k/0-streak successor trials can be obtained with an approach similar to the previous section. The difference satisfies Dk := P1k − (1 − P0k ), and there are three categories of sequences for which Dk is defined: (1) a sequence that ends in a run of 0s of length k or more with f0k ≥ 2 and f1k ≥ 1, with the difference equal to D1 = (f1k − s1k )/f1k − (s0k − 1)/(f0k − 1), (2) a sequence that ends in a run of 1s of length k or more with f0k ≥ 1 and f1k ≥ 2, with the difference equal to D2 := (f1k − s1k )/(f1k − 1) − s0k /f0k (3) a sequence that ends in a run of 0s of length k − 1 or less or a run of 1s of length k − 1 or less, with f0k ≥ 1 and f1k ≥ 1, with the difference equal to D3 := (f1k − s1k )/f1k − s0k /f0k . For all other sequences the difference is undefined. Theorem 8 For n > 0 and n1, n0, k with n0 + n1 = n and 1 < k ≤ n0, n1 ≤ n, the expected difference in the relative frequency of 1 for k/1-streak successors that are 1 and the relative frequency
of 1 for k/0-streak successors, Dk := P1k − (1 − P0k ), satisfies ) E[Dk | N1 = n1] = ( n ) s0k k D1 + r0 − s0k 3 l k n1 − Uk r01 ,...,r0k 1 ,s r0 r0 r ,...,r − ,s0k 11 1k−1 1k j=1 jr0j <n0 −k,s0k ≥1 j=1 jr1j ≤n1 −k,s1k ≥1 r0 ≥r1 + ) Ck r01 ,...,r0k−1 ,s0k r11 ,...,r1k−1 ,s1k j=1 jr0j ≤n0 −k,s0k ≥1 j=1 jr1j <n1 −k,s1k ≥1 r1 ≥r0 s1k r1 D2 + − r k where D1 = (f1k − s1k )/f1k − (s0k − 1)/(f0k − 1), D2 := (f1k − s1k )/(f1k − 1) − s0k /f0k , D3 := (f1k − s1k )/f1k − s0k /f0k , and Ck := r0!
to a sum of binomial coefficients: Uk = min{n1 ,n0 +1} ) n0 + 1 n1 −r1 l k−1 ) r1 (−1) n1 − 1 − £(k − 1) (*) r1 r1 =1 =0 £ n1 −k−r1 +1 r1 − 1 min{n1 −k+1,n0 +1} ) 0 l k−1 ) r1 − 1 n1 − k − 1 − £(k − 1) + δn1 k + r1 =2 r1 − 1 (−1) £ =0 r1 − 2 min{n0 ,n1 +1} + ) r0 =1 n1 + 1 r0 n0 −r0 l k−1 ) =0 r0 (−1) £ − 1 − − 1) r0 − 1 min{n0 −k+1,n1 +1} ) 1 n0 −k−r0 +1 l k−1 ) r0 − 1 n0 − k − 1 − £(k − 1) + δn0 k + r0 =2 r0 − 1 (−1) £ =0 r0 − 2 ,n0 +1} ) − r1 =1 n0 −r0 min{r1 +1,n0 } ) r0 =max{r1 −1,1} (2 · 1 {r1 =r0 } 1 ) −r0 |=1} n1 −r1 l k−1 ) × 0 =0 r0 ( 1) 0 £0 n0 − 1 − £0(k − 1) r0 − 1 l k−1 ) 1 =0 r1 ( 1) 1 £1 n1 − 1 − £1(k − 1) r1 − 1 min{n0 −k+1,n1 +1} min{r0 ,n1 } l k 1 n0 −k−r0 +1 − − r0 − 1 n0 − k − 1 − £0(k − 1) k + l n1 −r1 k−1 r0 =2 r1 =max{r0 −1,1} 0 =0 (−1) 0 r0 − 2 × ) × 1 =0 r1 ( 1) 1 £1
n1 − 1 − £1(k − 1) r1 − 1 min{n1 −k+1,n0 +1} min{r1 ,n0 } l k 1 n1 −k−r1 +1 − − r1 − 1 n1 − k − 1 − £1(k − 1) k + l n0 −r0 k−1 r1 =2 r0 −1,1} 1 =0 (−1) 1 r1 − 2 × ) × 0 =0 r0 ( 1) 0 £0 n0 − 1 − £0(k − 1) r0 − 1 •
covariate that indicates if a trial is a k/1-streak successor. Theorem 9 Let x ∈ {0, 1 n with I1k (x) /= ∅. If βk (x) = (β0k (x), β1k (x)) is defined to be the solution to the least squares problem, βk (x) ∈ argminβ R2 x − [1 d] β 2 defined so that di := 1I1k (x)(i) for i = 1, . . . , n, then where d ∈ {0, 1 n is 55 β0k (x) + β1k (x) = Pˆ1k (x) Proof: If βk (x) minimizes that sum of squares then β1k (x) = Ln working with the numerator, letting I1k ≡ I1k (x) we have (xi − − Ln (di − First, n )(xi − − = ) (xi − − + ) (xi − − i=1 i∈I1k = 1 |I1k | n ) (xi i∈I1k i∈I C − − |I1k | ) (xi i∈I C − = 1 |I1k | ) xi i∈I1k |I1k | − n ) xi i∈I C I n + |I1k | (n − |I |)x¯ = |I1k | 1 − | I1k | i∈I1k xi Li∈I C xi − n |I1k | n − |I1k | second, with the denominator of β1k (x) we have n )(di − = ) |I1k | 2 − |I1k | 2 i=1 i∈I1k n I1k 2
i∈I C n I1k 2 = |I1k | 1 − | = |I1k | 1 − | | n I1k | n + (n − |I1k |) | | 55 When I1k (x) = ∅ the solution set of the least squares problem is infinite, i.e. argmin β∈R2 x − β0 2 = {(β0 , β1 ) ∈ R2 : β0 = (1/n) Ln i k (x) as undefined in this case, then the bias from using the conditional relative i=1 x }. If we treat β frequency is equal to the finite sample bias in the coefficients of the associated linear probability model. If instead we define β1k (x) = 0, then the bias in the coefficients of the associated linear probability model will be less than the bias in the conditional relative frequency.