SlowGuess/NeurIPS_2023_OCR huggingface dataset:
[46] D. Lazer and A. Friedman. The network structure of exploration and exploitation. Administrative Science Quarterly, 52:667-694, 2007.
[47] R. D. Luce. Individual Choice Behavior: A Theoretical Analysis. Wiley, New York, NY, USA, 1959.
[48] Y. Mansour, A. Slivkins, and V. Syrgkanis. Bayesian incentive-compatible bandit exploration. Operations Research, 68(4):1132-1161, 2020. Preliminary version in ACM EC 2015.
[49] J. L. Myers. Probability learning and sequence learning. In W. Estes, editor, Handbook of Learning and Cognitive Processes: Approaches to Human Learning and Motivation, pages 171-205. Erlbaum, Hillsdale, NJ, 1976.
[50] M. Puri and D. T. Robinson. Optimism and economic choice. Journal of Financial Economics, 86(1):71-99, 2007.
[51] M. Raghavan, A. Slivkins, J. W. Vaughan, and Z. S. Wu. Greedy algorithm almost dominates in smoothed contextual bandits. SIAM J. on Computing (SICOMP), 52(2):487-524, 2023. Preliminary version at COLT 2018. Working paper available at arxiv.org/abs/2005.10624.
[52] D. Russo, B. Van Roy, A. Kazerouni, I. Osband, and Z. Wen. A tutorial on thompson sampling. Foundations and Trends in Machine Learning, 11(1):1-96, 2018. Published with Now Publishers (Boston, MA, USA). Also available at https://arxiv.org/abs/1707.02038.
[53] M. Sellke and A. Slivkins. The price of incentivizing exploration: A characterization via thompson sampling and sample complexity. Operations Research, 71(5), 2022. Preliminary version in ACM EC 2021.
[54] A. Slivkins. Introduction to multi-armed bandits. Foundations and Trends in Machine Learning, 12(1-2):1-286, Nov. 2019. Published with Now Publishers (Boston, MA, USA). Also available at https://arxiv.org/abs/1904.07272. Latest online revision: Jan 2022.
[55] A. Slivkins. Exploration and persuasion. In F. Echenique, N. Immorlica, and V. Vazirani, editors, Online and Matching-Based Market Design. Cambridge University Press, 2023. Also available at http://slivkins.com/work/ExplPers.pdf.
[56] L. Smith and P. Sørensen. Pathological outcomes of observational learning. *Econometrica*, 68: 371-398, 2000.
[57] W. R. Thompson. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika, 25(3-4):285-294, 1933.
[58] J. Ville. Etude critique de la notion de collectif. Bull. Amer. Math. Soc, 45(11):824, 1939.
[59] N. Vulkan. An economist's perspective on probability matching. Journal of Economic Surveys, 14:101-118, 2000.
[60] I. Welch. Sequential sales, learning, and cascades. The Journal of finance, 47:695-732, 1992.
[61] A. R. Zhang and Y. Zhou. On the non-asymptotic and sharp lower tail bounds of random variables. Stat, 9(1):e314, 2020.
SUPPLEMENTARY MATERIALS FOR "BAYESIAN BANDIT SOCIAL LEARNING UNDER MYOPIC BEHAVIOR"
The supplement contains the proofs, as well as detailed discussion of related work on social learning.
A Related Work on Social Learning 15
B Preliminaries: Reward-Tape 15
C Proofs from Section 3: Learning Failures 16
D Proofs from Section 4: Upper Bounds for Optimistic Agents 22
E Proofs from Section 5: Learning Failures for Bayesian Agents 25
F The corollary from Section 6 26
A Related Work on Social Learning
A vast literature on social learning studies agents that learn over time in a shared environment. A prominent topic is the presence or absence of learning failures such as ours. Models vary across several dimensions, such as: which information is acquired or transmitted, what is the communication network, whether agents are long-lived or only act once, how they choose their actions, etc. Below we discuss several lines of work that are most relevant.
In "sequential social learning", starting from [9, 60, 14, 56], agents observe private signals, but only the chosen actions are observable in the future; see Golub and Sadler [27] for a survey. The social planner (who chooses agents' actions given access to the knowledge of all previous agents) only needs to exploit, i.e., choose the best action given the previous agents' signals, whereas in our model it also needs to explore. Learning failures are (also) of primary interest, but they occur for an entirely different reason: restricted information flow, i.e., the fact that the private signals are not observable in the future.
"Strategic experimentation", starting from Bolton and Harris [16] and Keller et al. [39], studies long-lived learning agents that observe both actions and rewards of one another; see Hörner and Skrzypacz [33] for a survey. Here, the social planner also solves a version of multi-armed bandits, albeit a very different one (with time-doubling, "safe" arm that is completely known, and "risky" arm that follows a stochastic process). The main difference is that the agents engage in a complex repeated game where they explore but prefer to free-ride on exploration by others.
Bala and Goyal [8] and Lazer and Friedman [46] consider a network of myopic learners, all faced with the same bandit problem and observing each other's actions and rewards. The interaction protocol is very different from ours: agents are long-lived, act all at once, and only observe their neighbors on the network. Other specifics are different, too. Bala and Goyal [8] makes strong assumptions on learners' beliefs, which would essentially cause the greedy algorithm to work well in BSL. In Lazer and Friedman [46], each learner only retains the best observed action, rather than the full history. Both papers study social learning under different network topologies.
Prominent recent work, e.g., [31, 15, 25, 44], targets agents with misspecified beliefs, i.e., beliefs whose support does not include the correct model. The framing is similar to BSL with Bayesian-unbiased agents: agents arrive one by one and face the same decision problem, whereby each agent makes a rational decision after observing the outcomes of the previous agents.[18] Rational decisions under misspecified beliefs make a big difference compared to BSL, and structural assumptions about rewards/observations and the state space tend to be very different from ours. The technical questions being asked tend to be different, too. E.g., convergence of beliefs is of primary interest, whereas the chosen arms and agents' beliefs/estimates trivially converge in our setting.[19]
B Preliminaries: Reward-Tape
It is convenient for our analyses to interpret the realized rewards of each arm as if they are written out in advance on a "tape". We posit a matrix  $\left(\mathsf{Tape}_{a,i} \in [0,1] : a \in [2], i \in [T]\right)$ , called reward-tape, such that each entry  $\mathsf{Tape}_{a,i}$  is an independent Bernoulli draw with mean  $\mu_{a}$ . This entry is returned as reward when and if arm  $a$  is chosen for the  $i$ -th time. (We start counting from the initial samples, which comprise entries  $i \in [N_0]$ .) This is an equivalent (and well-known) representation of rewards in stochastic bandits.
We will use the notation for the UCBs/LCBs defined by the reward-tape. Fix arm  $a \in [2]$  and  $n \in [T]$ . Let  $\widehat{\mu}_{a,n}^{\mathrm{tape}} = \frac{1}{n}\sum_{i \in [n]} \mathsf{Tape}_{a,i}$  be the average over the first  $n$  entries for arm  $a$ . Now, given  $\eta \geq 0$ , define the appropriate confidence bounds:
$$
\mathrm {U C B} _ {a, n} ^ {\text {t a p e}, \eta} := \min  \left\{1, \widehat {\mu} _ {a, n} ^ {\text {t a p e}} + \sqrt {\eta / n} \right\} \quad \text {a n d} \quad \mathrm {L C B} _ {a, n} ^ {\text {t a p e}, \eta} := \max  \left\{0, \widehat {\mu} _ {a, n} ^ {\text {t a p e}} - \sqrt {\eta / n} \right\}. \tag {B.1}
$$
18 The original framing in this work posits a single learner that makes (possibly) myopic decisions over time and observes their outcomes. An alternative interpretation is that each decision is made by a new myopic agent who observes the history.
Essentially, if an arm is chosen infinitely often then the agents beliefs/estimates converge on its true mean reward; else, the agents eventually stop receiving any new information about this arm.
C Proofs from Section 3: Learning Failures
Our proofs rely on two tools from Probability (proved in Section C.4 and C.5): a sharp anticoncentration inequality for Binomial distribution and a lemma that encapsulates a martingale argument.
Lemma C.1 (anti-concentration). Let  $(X_{i})_{i\in \mathbb{N}}$  be a sequence of independent Bernoulli random variables with mean  $p\in [c,1 - c]$ , for some  $c\in (0,1 / 2)$  interpreted as an absolute constant. Then
$$
(\forall n \geq 1 / c, q \in (c / 8, p)) \quad \Pr \left[ \frac {1}{n} \sum_ {i = 1} ^ {n} X _ {i} \leq q \right] \geq \Omega \left(e ^ {- O \left(n (p - q) ^ {2}\right)}\right), \tag {C.1}
$$
where  $\Omega (\cdot)$  and  $O(\cdot)$  hide the dependence on  $c$ .
Lemma C.2 (martingale argument). In the setting of Lemma C.1,
$$
\forall q \in [ 0, p) \quad \Pr \left[ \forall n \geq 1: \quad \frac {1}{n} \sum_ {i = 1} ^ {n} X _ {i} \geq q \right] \geq \Omega_ {c} (p - q). \tag {C.2}
$$
The overall argument will be as follows. We will use Lemma C.1 to upper-bound the average reward of arm 1, i.e., the good arm, by some threshold  $q_{1}$ . This upper bound will only be guaranteed to hold when this arm is sampled exactly  $N$  times, for a particular  $N \geq N_{0}$ . Lemma C.2 will allow us to uniformly lower-bound the average reward of arm 2, i.e., the bad arm, by some threshold  $q_{2} \in (q_{1}, \mu_{2})$ . Focus on the round  $t^{*}$  when the good arm is sampled for the  $N$ -th time (if this ever happens). If the events in both lemmas hold, from round  $t^{*}$  onwards the bad arm will have a larger average reward by a constant margin  $q_{2} - q_{1}$ . We will prove that this implies that the bad arm has a larger index, and therefore gets chosen by the agents. The details of this argument differ from one theorem to another.
Lemma C.1 is a somewhat non-standard statement which follows from the anti-concentration inequality in [61] and a reverse Pinsker inequality in [28]. More standard anti-concentration results via Stirling's approximation lead to an additional factor of  $1 / \sqrt{n}$  on the right-hand side of (C.1). For Lemma C.2, we introduce an exponential martingale and relate the event in (C.2) to a deviation of this martingale. We then use Ville's inequality (a version of Doob's martingale inequality) to bound the probability that this deviation occurs.
C.1 Proof of Theorem 3.1:  $\eta$ -confident agents
Fix thresholds  $q_{1} < q_{2}$  to be specified later. Define two "failure events":
Fail<sub>1</sub>: the average reward of arm 1 after the  $N_0$  initial samples is below  $q_1$ ;
Fail2: the average reward of arm 2 is never below  $q_{2}$ .
In a formula, using the reward-tape notation from Appendix B, these events are
$$
\operatorname {F a i l} _ {1} := \left\{\widehat {\mu} _ {1, N _ {0}} ^ {\text {t a p e}} \leq q _ {1} \right\} \quad \text {a n d} \quad \operatorname {F a i l} _ {2} := \left\{\forall n \in [ T ]: \widehat {\mu} _ {2, n} ^ {\text {t a p e}} \geq q _ {2} \right\}. \tag {C.3}
$$
We show that event  $\text{Fail} := \text{Fail}_1 \cap \text{Fail}_2$  implies the 0-sampling failure, as long as the margin  $q_2 - q_1$  is sufficiently large.
Claim C.3. Assume  $q_{2} - q_{1} > 2 \cdot \sqrt{\eta / N_{0}}$  and event Fail. Then arm 1 is never chosen by the agents.
Proof. Assume, for the sake of contradiction, that some agent chooses arm 1. Let  $t$  be the first round when this happens. Note that  $\operatorname{Ind}_{1,t} \geq \operatorname{Ind}_{2,t}$ . We will show that this is not possible by upper-bounding  $\operatorname{Ind}_{1,t}$  and lower-bounding  $\operatorname{Ind}_{2,t}$ .
By definition of round  $t$ , arm 1 has been previously sampled exactly  $N_0$  times. Therefore,
$$
\begin{array}{l} \operatorname {I n d} _ {1, t} \leq \widehat {\mu} _ {1, N _ {0}} ^ {\text {t a p e}} + \sqrt {\eta / N _ {0}} \quad (b y \\ \leq q _ {1} + \sqrt {\eta / N _ {0}} \quad (b y F a i l _ {1}) \\ <   q _ {2} - \sqrt {\eta / N _ {0}} \quad (b y \\ \end{array}
$$
Let  $n$  be the number of times arm 2 has been sampled before round  $t$ . This includes the initial samples, so  $n \geq N_0$ . It follows that
$$
\operatorname {I n d} _ {2, t} \geq \widehat {\mu} _ {2, n} ^ {\text {t a p e}} - \sqrt {\eta / n}
$$
$$
(b y
$$
$$
\geq q _ {2} - \sqrt {\eta / N _ {0}}
$$
$$
(b y \operatorname {F a i l} _ {2} a n d n \geq N _ {0}).
$$
Consequently,  $\mathrm{Ind}_{2,t} > \mathrm{Ind}_{1,t}$ , contradiction.
In what follows, let  $c$  be the absolute constant from assumption (3.1).
Let us lower bound  $\operatorname*{Pr}[\text{Fail}]$  by applying Lemmas C.1 and C.2 to the reward-tape.
Claim C.4. Assume  $c / 4 < q_1 < q_2 < \mu_2$ . Then
$$
\Pr [ \text {F a i l} ] \geq q _ {\text {f a i l}} := \Omega_ {c} \left(\mu_ {2} - q _ {2}\right) \cdot e ^ {- O _ {c} \left(N _ {0} \left(\mu_ {1} - q _ {1}\right) ^ {2}\right)}. \tag {C.4}
$$
Proof. To handle  $\text{Fail}_1$ , apply Lemma C.1 to the reward-tape for arm 1, i.e., to the random sequence  $(\text{Tape}_{1,i})_{i\in [T]}$ , with  $n = N_0$  and  $q = q_{1}$ . Recalling that  $N_0\geq 1 / c$  by assumption (3.2),
$$
\Pr \left[ \text {F a i l} _ {1} \right] \geq \Omega_ {c} \left(e ^ {- O _ {c} \left(N _ {0} \left(\mu_ {1} - q _ {1}\right) ^ {2}\right)}\right).
$$
To handle  $\text{Fail}_2$ , apply Lemma C.2 to the reward-tape for arm 2, i.e., to the random sequence  $(\text{Tape}_{2,i})_{i\in [T]}$ , with threshold  $q = q_2$ . Then
$$
\Pr \left[ \text {F a i l} _ {2} \right] \geq \Omega_ {c} \left(\mu_ {2} - q _ {2}\right).
$$
Events  $\text{Fail}_1$  and  $\text{Fail}_2$  are independent, because they are determined by, resp., realized rewards of arm 1 and realized rewards of arm 2. The claim follows.
Finally, let us specify suitable thresholds that satisfy the preconditions in Claims C.3 and C.4:
$$
q _ {1} := \mu_ {2} - 4 \cdot \sqrt {\eta / N _ {0}} - c \Delta / 4 \quad \text {a n d} \quad q _ {2} := \mu_ {2} - \sqrt {\eta / N _ {0}} - c \Delta / 4.
$$
Plugging in  $\mu_2 \geq c$  and  $N_0 \geq 64 \cdot \eta / c^2$ , it is easy to check that  $q_1 \geq c / 4$ , as needed for Claim C.4.
Thus, the preconditions in Claims C.3 and C.4 are satisfied. It follows that the 0-failure happens with probability at least  $q_{\mathrm{fail}}$ , as defined in Claim C.4. We obtain the final expression in Eq. (3.3) because  $\mu_{a} - q_{a} = \Theta_{c}(\Delta + \sqrt{\eta / N_{0}})$  for both arms  $a \in [2]$ .
C.2 Proof of Theorem 3.10: pessimistic agents
We reuse the machinery from Section C.1: we define event Fail :=  $\text{Fail}_1 \cap \text{Fail}_2$  as per Eq. (C.3), for some thresholds  $q_1 < q_2$  to be specified later, and use Claim C.4 to bound  $\text{Pr}[\text{Fail}]$ . However, we need a different argument to prove that Fail implies the 0-sampling failure, and a different way to set the thresholds.
Claim C.5. Assume  $q_{1} > \sqrt{\eta / N_{0}}$  and event Fail. Then arm 1 is never chosen by the agents.
Proof. Assume, for the sake of contradiction, that some agent chooses arm 1. Let  $t$  be the first round when this happens. Note that  $\mathrm{Ind}_{1,t} \geq \mathrm{Ind}_{2,t}$ . We will show that this is not possible by upper-bounding  $\mathrm{Ind}_{1,t}$  and lower-bounding  $\mathrm{Ind}_{2,t}$ .
$$
\operatorname {I n d} _ {1, t} = \max  \left\{0, \widehat {\mu} _ {1, N _ {0}} ^ {\text {t a p e}} - \sqrt {\eta / N _ {0}} \right\}
$$
$$
\leq \max  \{0, q _ {1} - \sqrt {\eta / N _ {0}} \}
$$
$$
(b y F a i l _ {1})
$$
$$
= q _ {1} - \sqrt {\eta / N _ {0}}
$$
$$
(b y \text {a s s u m p t i o n}).
$$
$$
\operatorname {I n d} _ {2, t} \geq \hat {\mu} _ {2, n} ^ {\text {t a p e}} - \sqrt {\eta / n}
$$
Consequently,  $\operatorname{Ind}_{2,t} > \operatorname{Ind}_{1,t}$ , contradiction.
Now, set the thresholds  $q_{1}, q_{2}$  as follows:
$$
q _ {1} := \mu_ {2} - c \Delta / 4 \quad \text {a n d} \quad q _ {2} := \mu_ {2} - c \Delta / 8.
$$
Plugging in  $\mu_2 \geq c$  and  $N_0 \geq 64 \cdot \eta / c^2$ , it is easy to check that the preconditions in Claims C.4 and C.5 are satisfied. So, the 0-failure happens with probability at least  $q_{\mathrm{fail}}$  from Claim C.4. The final expression in Eq. (3.3) follows because  $\mu_a - q_a = \Theta_c(\Delta)$  for both arms  $a \in [2]$ .
C.3 Proof of Theorem 3.9: small  $N_0$
We focus on the case when  $N_0 \leq N^* := \lceil 64\eta / c^2 + 1 / c \rceil$ . We can now afford to handle the initial samples in a very crude way: our failure events posit that all initial samples of the good arm return reward 0, and all initial samples of the bad arm return reward 1.
$$
\operatorname {F a i l} _ {1} := \left\{\forall i \in [ 1, N ^ {*} ]: \operatorname {T a p e} _ {1, i} = 0 \right\},
$$
$$
\operatorname {F a i l} _ {2} := \left\{\forall i \in [ 1, N ^ {*} ]: \operatorname {T a p e} _ {2, i} = 1 \quad \text {a n d} \quad \forall i \in [ T ]: \widehat {\mu} _ {2, i} ^ {\mathrm {t a p e}} \geq q _ {2} \right\}.
$$
Here,  $q_{2} > 0$  is the threshold to be defined later.
On the other hand, our analysis given these events becomes more subtle. In particular, we introduce another "failure event"  $\text{Fail}_3$ , with a more subtle definition: if arm 1 is chosen by at least  $n := N^* - N_0$  agents, then arm 2 is chosen by  $n$  agents before arm 1 is.
We first show that  $\text{Fail} \coloneqq \text{Fail}_1 \cap \text{Fail}_2 \cap \text{Fail}_3$  implies the  $n$ -sampling failure.
Claim C.6. Assume that  $q_{2} \geq c / 4$  and Fail holds. Then at most  $n = N^{*} - N_{0}$  agents choose arm 1.
Proof. For the sake of contradiction, suppose arm 1 is chosen by more than  $n$  agents. Let agent  $t$  be the  $(n + 1)$ -th agent that chooses arm 1. In particular,  $\mathrm{Ind}_{1,t} \geq \mathrm{Ind}_{2,t}$ .
By definition of  $t$ , arm 1 has been previously sampled exactly  $N^*$  times before (counting the  $N_0$  initial samples). Therefore,
$$
\operatorname {I n d} _ {1, t} \leq \widehat {\mu} _ {1, N ^ {*}} ^ {\text {t a p e}} + \sqrt {\eta / N ^ {*}}
$$
$$
= \sqrt {\eta / N ^ {*}}
$$
$$
\leq c / 8
$$
$$
(b y \eta - c o n f i d e n c e)
$$
$$
(b y \quad e v e n t \quad F a i l _ {1})
$$
$$
(b y \quad d e f i n i t i o n \quad N ^ {*}).
$$
Let  $m$  be the number of times arm 2 has been sampled before round  $t$ . Then
$$
\operatorname {I n d} _ {2, t} \geq \widehat {\mu} _ {2, m} ^ {\mathrm {t a p e}} - \sqrt {\eta / m}
$$
$$
\geq q _ {2} - \sqrt {\eta / m}
$$
$$
\geq q _ {2} - \sqrt {\eta / N ^ {*}}
$$
$$
\geq q _ {2} - c / 8
$$
$$
> c / 8
$$
$$
(b y e v e n t F a i l _ {2})
$$
$$
(s i n c e m \geq N ^ {*} b y e v e n t F a i l _ {3})
$$
$$
(b y \quad d e f i n i t i o n \quad N ^ {*})
$$
$$
(s i n c e q _ {2} \geq c / 2).
$$
Therefore,  $\operatorname{Ind}_{2,t} > \operatorname{Ind}_{1,t}$ , contradiction.
Next, we lower bound the probability of  $\text{Fail}_1 \cap \text{Fail}_2$  using Lemma C.2.
Claim C.7. If  $q_{2} < \mu_{2}$  then  $\operatorname{Pr}\left[\mathsf{Fai1}_1 \cap \mathsf{Fai1}_2\right] \geq \Omega_c(\mu_2 - q_2) \cdot c^{2N^*}$ .
Proof. Instead of analyzing  $\text{Fail}_2$  directly, consider events
$$
\mathcal {E} := \left\{\forall i \in [ 1, N ^ {*} ]: \mathrm {T a p e} _ {2, i} = 1 \right\} \text {a n d} \mathcal {E} ^ {\prime} := \left\{\forall m \in [ N ^ {*} + 1, T ]: \frac {1}{m - N ^ {*}} \sum_ {i = N ^ {*} + 1} ^ {m} \mathrm {T a p e} _ {2, i} \geq q _ {2} \right\}.
$$
Note that  $\mathcal{E} \cap \mathcal{E}'$  implies  $\operatorname{Fail}_2$ . Now,  $\operatorname{Pr}\left[\operatorname{Fail}_1\right] \geq \mu_1^{N^*} \geq c^{N^*}$  and  $\operatorname{Pr}\left[\mathcal{E}\right] \geq (1 - \mu_2)^{N^*} \geq c^{N^*}$ . Further,  $\operatorname{Pr}\left[\mathcal{E}'\right] \geq \Omega_c(\mu_2 - q_2)$  by Lemma C.2. The claim follows since these three events are mutually independent.
To bound  $\operatorname*{Pr}[\text{Fail}]$ , we argue indirectly, assuming  $\text{Fail}_1 \cap \text{Fail}_2$  and proving that the conditional probability of  $\text{Fail}_3$  is at least  $1/2$ . While this statement feels natural given that  $\text{Fail}_1 \cap \text{Fail}_2$  favors arm 2, the proof requires a somewhat subtle inductive argument. This is where we use the symmetry and monotonicity properties from the theorem statement.
Claim C.8.  $\operatorname*{Pr}\left[\text{Fail}_3\mid \text{Fail}_1\cap \text{Fail}_2\right]\geq \frac{1}{2}.$
Now, we can lower-bound  $\operatorname*{Pr}[\text{Fail}]$  by  $\Omega_c(\mu_2 - q_2) \cdot c^{2N^*}$ . Finally, we set the threshold to  $q_2 = c / 2$  and the theorem follows.
Proof of Claim C.8. Note that event  $\text{Fail}_t$  is determined by the first  $N^*$  entries of the reward-tape for both arms, in the sense that it does not depend on the rest of the reward-tape.
For each arm  $a$  and  $i \in [T]$ , let agent  $\tau_{a,i}$  be the  $i$ -th agent that chooses arm  $a$ , if such agent exists, and  $\tau_i = T + 1$  otherwise. Then
$$
\operatorname {F a i l} _ {3} = \left\{\tau_ {2, n} \leq \tau_ {1, n} \right\} = \left\{\tau_ {1, n} \geq 2 n \right\} \tag {C.5}
$$
Let  $\mathcal{E}$  be the event that the first  $N^{*}$  entries of the reward-tape are 0 for both arms. By symmetry between the two arms (property (P1) in the theorem statement) we have
$$
\Pr \left[ \tau_ {2, n} <   \tau_ {1, n} \mid \mathcal {E} \right] = \Pr \left[ \tau_ {2, n} > \tau_ {1, n} \mid \mathcal {E} \right] = 1 / 2,
$$
$$
\Pr [ \text {F a i l} _ {3} \mid \mathcal {E} ] = \Pr [ \tau_ {2, n} \leq \tau_ {1, n} \mid \mathcal {E} ] \geq 1 / 2. \tag {C.6}
$$
Next, for two distributions  $F, G$ , write  $F \succeq_{\mathrm{fosd}} G$  if  $F$  first-order stochastically dominates  $G$ . A conditional distribution of random variable  $X$  given event  $\mathcal{E}$  is denoted  $(X|\mathcal{E})$ . For each  $i \in [T]$ , we consider two conditional distributions for  $\tau_{1,i}$ : one given  $\mathrm{Fail}_1 \cap \mathrm{Fail}_2$  and another given  $\mathcal{E}$ , and prove that the former dominates:
$$
\left(\tau_ {1, i} \mid \operatorname {F a i l} _ {1} \cap \operatorname {F a i l} _ {2}\right) \succeq_ {\text {f o s d}} \left(\tau_ {1, i} \mid \mathcal {E}\right) \quad \forall i \in [ T ]. \tag {C.7}
$$
Applying (C.7) with  $i = n$ , it follows that
$$
\begin{array}{l} \Pr \left[ \text {F a i l} _ {3} \mid \text {F a i l} _ {1} \cap \text {F a i l} _ {2} \right] = \Pr \left[ \tau_ {1, n} \geq 2 n \mid \text {F a i l} _ {1} \cap \text {F a i l} _ {2} \right] \\ \geq \Pr \left[ \tau_ {1, n} \geq 2 n \mid \mathcal {E} \right] = 1 / 2. \\ \end{array}
$$
(The last equality follows from (C.6) and Eq. (C.6).) Thus, it remains to prove (C.7).
Let us consider a fixed realization of each agents' behavioral type, i.e., a fixed, deterministic mapping from histories to arms. W.l.o.g. interpret the behavioral type of each agent  $t$  as first deterministically mapping history  $\mathsf{hist}_t$  to a number  $p_t \in [0,1]$ , then drawing a threshold  $\theta_t \in [0,1]$  independently and uniformly at random, and then choosing arm 1 if and only if  $p_t \geq \theta_t$ . Note that  $p_t = \operatorname*{Pr}\left[a_t = 1 \mid \mathsf{hist}_t\right]$ . So, we pre-select the thresholds  $\theta_t$  for each agent  $t$ . Note the agents retain the monotonicity property (P2) from the theorem statement. (For this property, the probabilities on both sides of Eq. (3.6) are now either 0 or 1.)
Let us prove (C.7) for this fixed realization of the types, using induction on  $i$ . Both sides of (C.7) are now deterministic; let  $A_{i}, B_{i}$  denote, resp., the left-hand side and the right-hand side. So, we need to prove that  $A_{i} \geq B_{i}$  for all  $i \in [n]$ . For the base case, take  $i = 0$  and define  $A_{0} = B_{0} = 0$ . For the inductive step, assume  $A_{i} \geq B_{i}$  for some  $i \geq 0$ . We'd like to prove that  $A_{i+1} \geq B_{i+1}$ . Suppose, for the sake of contradiction, that this is not the case, i.e.,  $A_{i+1} < B_{i+1}$ . Since  $A_{i} < A_{i+1}$  by definition of the sequence  $(\tau_{a,i} : \in [T])$ , we must have
$$
B _ {i} \leq A _ {i} <   A _ {i + 1} <   B _ {i + 1}.
$$