SlowGuess/NeurIPS_2023_OCR huggingface dataset:
where  $\operatorname {kl}(1 - \delta ,\delta)\triangleq x\log \frac{x}{y} + (1 - x)\log \frac{1 - x}{1 - y}$  for  $x,y\in (0,1)$ .
Proof sketch. We use Sequential Karwa-Vadhan Lemma [AB22, Lemma 2] with a data-processing inequality in the BAI canonical model. Extra care is needed to deal with the stopping times in the coupling, compared to a fixed horizon  $T$  in regret minimization. The proof is deferred to Appendix B.
Leveraging Lemma 1, we derive a sample complexity lower bound for any  $\epsilon$ -DP-FC-BAI strategy.
Theorem 2 (Sample complexity lower bound for  $\epsilon$ -DP-FC-BAI). Let  $\delta \in (0,1)$  and  $\epsilon > 0$ . For any  $\delta$ -correct  $\epsilon$ -global DP BAI strategy, we have that
$$
\mathbb {E} _ {\boldsymbol {\nu}} [ \tau ] \geq T ^ {\star} (\boldsymbol {\nu}; \epsilon) \log (1 / 3 \delta), \tag {3}
$$
where  $(T^{\star}(\pmb {\nu};\epsilon))^{-1}\triangleq \sup_{\omega \in \Sigma_K}\inf_{\pmb {\lambda}\in \mathrm{Alt}(\pmb {\nu})}\min \left(\sum_{a = 1}^{K}\omega_a\mathrm{KL}(\nu_a\parallel \lambda_a),6\epsilon \sum_{a = 1}^{K}\omega_a\mathrm{TV}(\nu_a\parallel \lambda_a)\right).$
Comments on the lower bound. Similar to the lower bound for the non-private BAI [GK16], the lower bound of Theorem 2 is the value of a two-player zero-sum game between a MIN player and MAX player. MIN plays an alternative instance  $\lambda$  close to  $\nu$  in order to confuse MAX. The latter plays an allocation  $\omega \in \Sigma_K$  to explore the different arms, with the purpose of maximising the divergence between  $\nu$  and the confusing instance  $\lambda$  that MIN played. On top of the KL divergence present in the non-private lower bound, our bound features the TV distance that appears naturally when incorporating the  $\epsilon$ -global DP constraint. The proof is deferred to Appendix B. In order to compare the lower bound of an  $\epsilon$ -global BAI strategy with the non-private lower bound of [GK16], we relax Theorem 2 to further derive a simpler bound, as in Corollary 1.
Corollary 1. For any  $\delta$ -correct  $\epsilon$ -global DP BAI strategy, we have that
$$
\mathbb {E} _ {\boldsymbol {\nu}} [ \tau ] \geq \max  \left(T _ {\mathrm {K L}} ^ {\star} (\boldsymbol {\nu}), \frac {1}{6 \epsilon} T _ {\mathrm {T V}} ^ {\star} (\boldsymbol {\nu})\right) \log (1 / 3 \delta),
$$
where  $(T_{\pmb{d}}^{\star}(\pmb {\nu}))^{-1}\triangleq \sup_{\omega \in \Sigma_K}\inf_{\pmb {\lambda}\in \mathrm{Alt}(\pmb {\nu})}\sum_{a = 1}^{K}\omega_a\pmb {d}(\nu_a,\lambda_a)$ , and  $\pmb{d}$  is either KL or TV.
Proof. The proof is direct by observing that  $T^{\star}(\nu ;\epsilon)\geq T_{\mathrm{KL}}^{\star}(\nu)$  and  $T^{\star}(\nu ;\epsilon)\geq \frac{1}{6\epsilon} T_{\mathrm{TV}}^{\star}(\nu)$ .
Comparison with the non-private lower bound.  $T_{\mathrm{KL}}^{\star}$  is the characteristic time in the non-private lower bound [GK16], and we refer to Section 2.2 of [GK16] for a detailed discussion on its properties. The sample complexity lower bound suggests the existence of two hardness regimes depending on  $\epsilon$ ,  $T_{\mathrm{KL}}^{\star}$  and  $T_{\mathrm{TV}}^{\star}$ . (1) Low-privacy regime: When  $\epsilon > T_{\mathrm{TV}(\nu)}^{\star} / (6T_{\mathrm{KL}}^{\star}(\nu))$ , the lower bound retrieves the non-private lower bound, i.e.  $T_{\mathrm{KL}}^{\star}(\nu)$ , and thus, privacy can be achieved for free. (2) High-privacy regime: When  $\epsilon < T_{\mathrm{TV}}^{\star}(\nu) / (6T_{\mathrm{KL}}^{\star}(\nu))$ , the lower bound becomes  $T_{\mathrm{TV}}^{\star} / (6\epsilon)$  and  $\epsilon$ -global DP  $\delta$ -BAI requires more samples than non-private ones.
In the following proposition, we characterise  $T_{\mathrm{TV}}^{\star}$  for Bernoulli instances.
Proposition 1 (TV characteristic time for Bernoulli instances). Let  $\nu$  be a bandit instance, i.e. such that  $\nu_{a} = \text{Bernoulli}(\mu_{a})$  and  $\mu_{1} > \mu_{2} \geq \dots \geq \mu_{K}$ . Let  $\Delta_{a} \triangleq \mu_{1} - \mu_{a}$  and  $\Delta_{\min} \triangleq \min_{a \neq 1} \Delta_{a}$ . We have that
$$
T _ {\mathrm {T V}} ^ {\star} (\boldsymbol {\nu}) = \frac {1}{\Delta_ {m i n}} + \sum_ {a = 2} ^ {K} \frac {1}{\Delta_ {a}}, \qquad \text {a n d} \qquad \frac {1}{\Delta_ {m i n}} \leq T _ {\mathrm {T V}} ^ {\star} (\boldsymbol {\nu}) \leq \frac {K}{\Delta_ {m i n}}.
$$
Proof sketch. The proof is direct by solving the optimisation problem defining  $T_{\mathrm{TV}}^{\star}$  and using that TV (Bernoulli(p) || Bernoulli(q)) = |p - q|. We refer to Appendix B for details.
Comment. The aforementioned bound on TV characteristic time for Bernoulli instances is  $\epsilon$ -global DP parallel of the KL-characteristic time bound  $T_{\mathrm{KL}}^{\star}(\nu) \leq \sum_{a=1}^{K} \Delta_a^{-2}$  [GK16]. Using Pinsker's inequality, one can connect the TV and KL characteristic times by  $T_{\mathrm{TV}}^{\star}(\nu) \geq \sqrt{2 T_{\mathrm{KL}}^{\star}(\nu)}$ .
4 Algorithm design: Private Top Two with adaptive episodes (AdaP-TT)
In this section, we propose AdaP-TT, an  $\epsilon$ -global DP version of the TTUCB algorithm [JD22]. We show that AdaP-TT satisfies  $\epsilon$ -global DP, is  $\delta$ -correct, and has an asymptotic sample complexity that matches the high privacy lower bounds up to multiplicative constants.
TTUCB belongs to the family of Top Two algorithms [Rus16,  $\mathrm{SHM}^{+}20$ ,  $\mathrm{JDB}^{+}22$ ], which selects at each time two arms called leader and challenger, and sample among them. After initialisation, TTUCB uses a UCB-based leader and a Transportation Cost (TC) challenger, expressed by
$$
B _ {n} = \underset {a \in [ K ]} {\arg \max } \{\hat {\mu} _ {n, a} + \sqrt {6 \log (n) / N _ {n , a}} \}, \quad \text {a n d} \quad C _ {n} = \underset {a \neq B _ {n}} {\arg \min } \frac {\hat {\mu} _ {n , B _ {n}} - \hat {\mu} _ {n , a}}{\sqrt {1 / N _ {n , B _ {n}} + 1 / N _ {n , a}}}.
$$
Here,  $(\hat{\mu}_n,N_n)$  are the empirical means and counts on the whole history. The theoretical motivation behind the TC challenger comes from the theoretical lower bound in FC-BAI, which involves the KL-characteristic time  $T_{\mathrm{KL}}^{\star}(\pmb {\mu}) = \min_{\beta \in (0,1)}T_{\mathrm{KL},\beta}^{\star}(\pmb {\mu})$  . For Gaussian distributions, we have
$$
2 T _ {\mathrm {K L}, \beta} ^ {\star} (\boldsymbol {\mu}) ^ {- 1} = \max  _ {\omega \in \Sigma_ {K}, \omega_ {a \star} = \beta} \frac {\left(\mu_ {a ^ {\star}} - \mu_ {a}\right) ^ {2}}{1 / \beta + 1 / \omega_ {a}} \quad \text {a n d} \quad T _ {\mathrm {K L}, 1 / 2} ^ {\star} (\boldsymbol {\mu}) \leq 2 T _ {\mathrm {K L}} ^ {\star} (\boldsymbol {\mu}),
$$
The maximiser of the above equation is denoted by  $\omega_{\mathrm{KL},\beta}^{\star}(\pmb {\mu})$ , and is further referred to as the  $\beta$ -optimal allocation as it is unique. Let  $N_{n,b}^{a}$  denote the number of times arm  $b$  was pulled when  $a$  was the leader, and  $L_{n,a}$  denotes the number of times arm  $a$  was the leader. In order to select the next arm to sample  $I_{n}$ , TTUCB relies on  $K$  tracking procedures, i.e. set  $I_{n} = B_{n}$  if  $N_{n,B_n}^{B_n}\leq \beta L_{n + 1,B_n}$ , else  $I_{n} = C_{n}$ . This ensures that  $\max_{a\in [K],n > K}|N_{n,a}^{a} - \beta L_{n,a}|\leq 1$ . Standing on this premise, we now describe how we design an  $\epsilon$ -global DP extension of TTUCB.
Private algorithm design. As illustrated in Algorithm 2, AdaP-TT relies on three ingredients: adaptive episodes with doubling, forgetting, and adding calibrated Laplacian noise. (1) AdaP-TT maintains  $K$  episodes, i.e. one per arm. The private empirical estimate of the mean of an arm is only updated at the end of an episode, that means when the number of times that a particular arm was played doubles (Line 5). (2) For each arm  $a$ , AdaP-TT forgets rewards from previous phases of arm  $a$ , i.e. the private empirical estimate of arm  $a$  is only computed using the rewards collected in the last phase of arm  $a$  (Line 8). This assures that the means of each arm are estimated using a non-overlapping sequence of rewards. (3) Thanks to this doubling and forgetting, AdaP-TT is  $\epsilon$ -global DP as soon as each empirical mean (Line 9) is made  $\epsilon$ -DP, and thus, avoiding any use of privacy composition. This is achieved by adding Laplace noise. We formalise this intuition in Lemma 2 of Appendix C.
Remark 2. The aforementioned generic wrapper can be used to construct a near-optimal differentially private version of any existing FC-BAI algorithm that deploys a sampling rule with the empirical means of rewards. In this work, we consider and rigorously analyse the Top Two algorithms since they demonstrate both good theoretical guarantees and empirical performance.
Theorem 3 (Privacy analysis). For rewards in  $[0,1]$ , AdaP-TT satisfies  $\epsilon$ -global DP.
Proof sketch. A change in one user only affects the empirical mean calculated at one episode of an arm, which is made private using the Laplace Mechanism and Lemma 2. Since the sampled actions, recommended action, and stopping time are computed only using the private empirical means, AdaP-TT satisfies  $\epsilon$ -global DP thanks to post-processing lemma. We refer to Appendix C for details.
Private GLR stopping rule. We consider the private GLR stopping rule based on the private means and on the pulling counts from the last phase (Line 12), and recommend the arm with the highest private mean (Line 11). Lemma 4 yields a threshold function ensuring that any sampling rule is  $\delta$ -correct, when using the private GLR stopping rule.
Theorem 4 (δ-correctness). Let  $\delta \in (0,1)$ ,  $\epsilon > 0$ . Let  $s > 1$  and  $\zeta$  be the Riemann  $\zeta$  function. Let  $c_{k}(n,m,\delta) = 2\mathcal{C}_{G}(\log ((K - 1)\zeta(s)k^{s} / \delta) / 2) + 2\log (4 + \log n) + 2\log (4 + \log m)$  be the threshold without privacy. Given any sampling rule, the following threshold
$$
c _ {\epsilon , k _ {1}, k _ {2}} (n, m, \delta) = 2 c _ {k _ {1} k _ {2}} (n, m, \delta / 2) + \frac {1}{n \epsilon^ {2}} \log \left(\frac {2 K k _ {1} ^ {s} \zeta (s)}{\delta}\right) ^ {2} + \frac {1}{m \epsilon^ {2}} \log \left(\frac {2 K k _ {2} ^ {s} \zeta (s)}{\delta}\right) ^ {2} \tag {4}
$$
with the GLR stopping rule yields a  $\delta$ -correct algorithm for sub-Gaussian distributions. The function  $\mathcal{C}_G$  is defined in (15). It satisfies  $\mathcal{C}_G(x) \approx x + \ln(x)$ .
Remark. We observe that approximately  $c_{\epsilon, k_1, k_2}(n, m, \delta) \approx 2 \log(1/\delta) + (1/n + 1/m) \log(1/\delta)^2/\epsilon^2$ .
Proof sketch. Proving  $\delta$ -correctness of a GLR stopping rule is done by leveraging concentration results. Specifically, we start by decomposing the failure probability  $\mathbb{P}_{\mu}(\tau_{\delta} < +\infty, \hat{a} \neq a^{\star})$  into a non-private and a private part using the basic property of  $\mathbb{P}(X + Y \geq a + b) \leq \mathbb{P}(X \geq a) + \mathbb{P}(Y \geq b)$ . The two-factor in front of  $c_{k_1 k_2}$  originates from the looseness of this decomposition. To remove it, we would need a tighter stopping threshold that jointly controls both the non-private and the private parts. We conclude using concentration results from sub-Gaussian random variables for the non-private part, and Laplace random variables for the private part.
Theorem 5 (Asymptotic upper bound on expected sample complexity). Let  $(\delta, \beta) \in (0,1)^2$  and  $\epsilon > 0$ . Combined with the private GLR stopping rule using threshold as in (4), AdaP-TT is  $\delta$ -correct and satisfies that, for all  $\mu \in \mathbb{R}^K$  such that  $\min_{a \neq b} |\mu_a - \mu_b| > 0$ ,
$$
\limsup_{\delta \to 0}\frac{\mathbb{E}_{\mu}[\tau_{\delta}]}{\log(1 / \delta)}\leq 4T_{\mathrm{KL},\beta}^{\star}(\boldsymbol {\mu})\left(1 + \sqrt{1 + \frac{\Delta_{\max}^{2}}{2\epsilon^{2}}}\right)  .
$$
Proof sketch. We adapt the asymptotic proof of the TTUCB algorithm, which is based on the unified analysis of Top Two algorithms from  $\mathrm{[JDB^{+}22]}$ . Below, we present high-level ideas of the proof and specify the effect of different elements of AdaP-TT on the expected sample complexity.
Consequences of Theorem 5. (1) The non-private TTUCB algorithm [JD22] achieves a sample complexity of  $T_{\mathrm{KL},\beta}^{\star}(\mu)$  for sub-Gaussian random variables. The proof relies on showing that the empirical pulling counts are converging towards the  $\beta$ -optimal allocation  $\omega_{\mathrm{KL},\beta}^{\star}(\mu)$ . (2) The effect of doubling and forgetting is a multiplicative four-factor, i.e.  $4T_{\mathrm{KL},\beta}^{\star}(\mu)$ . The first two-factor is due to forgetting since we throw away half of the samples. The second two-factor is due to doubling since we have to wait for the end of an episode to evaluate the stopping condition. (3) The Laplace noise only affects the empirical estimate of the mean. Since the Laplace noise has no bias and a sub-exponential tail, the private means will still converge towards their true values. Therefore, the empirical counts of AdaP-TT will also converge to  $\omega_{\mathrm{KL},\beta}^{\star}(\mu)$  asymptotically. (4) While the Laplace noise has little effect on the sampling rule itself, it changes drastically the dependency in  $\log(1/\delta)$  of the threshold used in the GLR stopping rule. The private threshold  $c_{\epsilon,k_1,k_2}$  has an extra factor  $\mathcal{O}(\log^2(1/\delta))$  compared to the non-private one  $c_k$ . Using the convergence towards  $\omega_{\mathrm{KL},\beta}^{\star}(\mu)$ , the stopping condition is met as soon as  $\frac{n}{T_{\mathrm{KL},\beta}^{\star}(\mu)} \lesssim 2\log(1/\delta) + \frac{\Delta_{\max}^2}{2\epsilon^2}\frac{T_{\mathrm{KL},\beta}^{\star}(\mu)}{n}\log^2(1/\delta)$ . Solving the inequality for  $n$  concludes the proof while adding a multiplicative four-factor.
Algorithm 2 AdaP-TT. Private statistics are in red. Changes due to privacy are in blue.
1: Input:  $\beta \in (0,1)$ , risk  $\delta \in (0,1)$ , privacy budget  $\epsilon$ , thresholds  $c_{\epsilon,k_1,k_2} : \mathbb{N}^2 \times (0,1) \to \mathbb{R}^+$   
2: Output: Recommendation  $\hat{a}$  and Stopping time  $\tau$  satisfying  $\epsilon$ -global DP  
3: Initialization:  $\forall a \in [K]$ , pull arm  $a$ , set  $k_a = 1$ ,  $T_1(a) = K + 1$ ,  $L_{n,a} = 0$ ,  $N_{n,a} = 1$ ,  $n = K + 1$ .  
4: for  $n > K$  do  
5: if there exists  $a \in [K]$  such that  $N_{n,a} \geq 2N_{T_ka(a),a}$  then ▷ Per-arm doubling  
6: Change phase  $k_a \gets k_a + 1$  for this arm  $a$   
7: Set  $T_{k_a}(a) = n$  and  $\tilde{N}_{k_a,a} = N_{T_{k_a}(a),a} - N_{T_{k_a-1}(a),a}$  ▷ Pulls of  $a$  in its last phase  
8: Set  $\hat{\mu}_{k_a,a} = \tilde{N}_{k_a,a}^{-1} \sum_{s=T_{k_a-1}(a)}^{T_{k_a}(a)-1} X_s \mathbb{1}\{I_s = a\}$  ▷ Empirical mean of  $a$  in its last phase  
9: Set  $\tilde{\mu}_{k_a,a} = \hat{\mu}_{k_a,a} + Y_{k_a,a}$  where  $Y_{k_a,a} \sim \mathrm{Lap}((\epsilon \tilde{N}_{k_a,a})^{-1})$  ▷ Make it private  
10: end if  
11: Set  $\hat{a}_n = \arg \max_{b \in [K]} \tilde{\mu}_{k_b,b}$  ▷ Arm with highest private mean  
12: if  $\frac{(\tilde{\mu}_{k_{\hat{a}_n},\hat{a}_n} - \tilde{\mu}_{k_b,b})^2}{1/\tilde{N}_{k_{\hat{a}_n},\hat{a}_n} + 1/\tilde{N}_{k_b,b}} \geq 2c_{\epsilon,k_{\hat{a}_n},k_b} (\tilde{N}_{k_{\hat{a}_n},\hat{a}_n}, \tilde{N}_{k_b,b}, \delta)$  for all  $b \neq \hat{a}_n$  then  
13: return ( $\hat{a}_n, n$ ) ▷ If GLR stopping condition is met, recommend the empirical best arm  
14: end if  
15: Set  $B_n = \arg \max_{a \in [K]} \{\tilde{\mu}_{k_a,a} + \sqrt{k_a / \tilde{N}_{k_a,a}} + k_a / (\epsilon \tilde{N}_{k_a,a})\}$  ▷ Private UCB leader  
16: Set  $C_n = \arg \min_{a \neq B_n} \frac{\tilde{\mu}_{k_{B_n},B_n} - \tilde{\mu}_{k_a,a}}{\sqrt{1/N_{n,B_n} + 1/N_{n,a}}}$  ▷ Private TC challenger  
17: Set  $I_n = B_n$  if  $N_{n,B_n}^{B_n} \leq \beta L_{n+1,B_n}$ , else  $I_n = C_n$  ▷ Tracking  
18: Pull  $I_n$  and observe  $X_n \sim \nu_{I_n}$   
19: Set  $N_{n+1,I_n} \gets N_{n,I_n} + 1$ ,  $N_{n+1,I_n}^{B_n} \gets N_{n,I_n}^{B_n} + 1$  and  $L_{n+1,B_n} \gets L_{n,B_n} + 1$ . Set  $n \gets n+1$   
20: end for
Discussion. For  $\beta = 1/2$ , it is well known that  $T_{\mathrm{KL},1/2}^{\star}(\pmb{\mu}) \leq 2T_{\mathrm{KL}}^{\star}(\pmb{\mu}) \leq 8\sum_{a \neq a^{\star}} \Delta_a^{-2}$ . We consider Bernoulli instances  $(0 < \Delta_{\min} \leq \Delta_{\max} < 1)$ , where the gaps have the same order of magnitude, i.e. Condition 1: there exists a constant  $C \geq 1$  such that  $\Delta_{\max} / \Delta_{\min} \leq C$ . For such instances, there exists a universal constant  $c$ , such that
$$
\limsup_{\delta \to 0}\frac{\mathbb{E}_{\boldsymbol{\mu}}[\tau_{\delta}]}{\log(1 / \delta)}\leq c\max \left\{T_{\mathrm{KL},1 / 2}^{\star}(\boldsymbol {\mu}),C\epsilon^{-1}\sum_{a\neq a^{\star}}\Delta_{a}^{-1}\right\} .
$$
Without privacy, i.e.  $\epsilon \rightarrow +\infty$ , AdaP-TT yields a multiplicative eight-factor. On top of the four-factor due to doubling and forgetting, another multiplicative two comes from  $2c_{k_1k_2}$  in Equation (4).
Comparison to the lower bound. For Bernoulli bandits verifying Condition 1, the upper bound of Theorem 5 matches the  $T_{\mathrm{TV}}^{\star}(\pmb {\mu}) / \epsilon$  lower bound of Corollary 1 up to constants in the high-privacy regime, i.e. when  $\epsilon \preceq T_{\mathrm{TV}}^{\star}(\pmb {\mu}) / T_{\mathrm{KL}}^{\star}(\pmb {\mu})$ . In the low-privacy regime, the upper bound reduces to  $T_{\mathrm{KL},1 / 2}^{\star}(\pmb {\mu})$ . In Appendix E.7, we discuss in-depth why this difference is necessary for private BAI algorithms based on the GLR stopping rule, which poses an interesting open problem.
Comparison to DP-SE. DP-SE is a private version of the successive-elimination algorithm studied in [SS19] for the regret minimisation setting. The algorithm samples active arms uniformly during phases of geometrically increasing length. Based on the private confidence bounds, DP-SE eliminates provably sub-optimal arms at the end of each phase. Due to its phased-elimination structure, DP-SE can be easily converted into an  $\epsilon$ -DP-FC-BAI algorithm, where we stop once there is only one active arm left. In particular, the proof of Theorem 4.3 of [SS19] shows that with high probability any suboptimal arm  $a \neq a^{\star}$  is sampled no more than  $\mathcal{O}(\Delta_a^2 + (\epsilon \Delta_a)^{-1})$ . From this result, it is straightforward to extract a sample complexity upper bound for DP-SE, i.e.  $\mathcal{O}(\sum_{a \neq a^{\star}} \Delta_a^{-2} + \sum_{a \neq a^{\star}} (\epsilon \Delta_a)^{-1})$ . This shows that DP-SE too achieves (ignoring constants) the high-privacy lower bound  $T_{\mathrm{TV}}^{\star}(\mu) / \epsilon$  for Bernoulli instances. However, due to its uniform sampling within the phases, DP-SE is less adaptive than AdaP-TT. Inside a phase, DP-SE continues to sample arms that might already be known to be bad, while AdaP-TT adapts its sampling rule based on the transportation costs that reflect the amount of evidence collected in favour of the hypothesis that the leader is the best arm. Finally, AdaP-TT has the advantage of being anytime, i.e. its sampling strategy does not depend on the risk  $\delta$ .
Figure 1: Evolution of the stopping time  $\tau$  (mean  $\pm$  std. over 100 runs) of AdaP-TT, DP-SE, and TTUCB with respect to the privacy budget  $\epsilon$  for  $\delta = 10^{-2}$  on Bernoulli instance  $\mu_{1}$  (left) and  $\mu_{2}$  (right). The shaded vertical line separates the two privacy regimes. AdaP-TT outperforms DP-SE.
We perform experiments to show that: (i) AdaP-TT has better empirical performance compared to DP-SE, and (ii) the transition between high and low-privacy regimes is reflected empirically.
Experimental setup. We compare the performances of AdaP-UCB and DP-SE for FC-BAI in different Bernoulli instances as in [SS19]. The first instance has means  $\mu_{1} = (0.95, 0.9, 0.9, 0.9, 0.5)$  and the second instance has means  $\mu_{2} = (0.75, 0.7, 0.7, 0.7, 0.7)$ . As a benchmark, we also compare to the non-private TTUCB. We set the risk  $\delta = 10^{-2}$ . We implement all the algorithms in Python (version 3.8) and on an 8-core 64-bits Intel i5@1.6 GHz CPU. We run each algorithm 100 times, and plot corresponding average and standard deviations of stopping times in Figure 1. We also test the algorithms on other Bernoulli instances and report the results in Appendix F.
Result analysis. a. Efficiency in performance. AdaP-TT requires less samples than DP-SE to provide a  $\delta$ -correct answer. In the high privacy regime, i.e. small  $\epsilon$ , AdaP-TT outperforms DP-SE in all the instances tested. In the low privacy regimes, i.e. large  $\epsilon$ , both algorithms have similar performance that in the worst case is four times the samples required of TTUCB, as shown theoretically.
b. Impact of privacy regimes. As indicated by the theoretical sample complexity lower bounds and upper bounds, the experimental performance of AdaP-TT demonstrates two regimes: a high-privacy regime (for  $\epsilon < 0.2$ ), where the stopping time of AdaP-TT depends on the privacy budget  $\epsilon$ , and a low privacy regime (for  $\epsilon > 0.2$ ), where the performance of AdaP-TT does not depend on  $\epsilon$ .
We study FC-BAI with  $\epsilon$ -global DP. We derive a sample complexity lower bound that quantifies the additional samples needed by a  $\delta$ -correct BAI strategy in order to ensure  $\epsilon$ -global DP. The lower bound further suggests the existence of two privacy regimes. In the low-privacy regime, no additional samples are needed, and privacy can be achieved for free. For the high-privacy regime, the lower bound reduces to  $\Omega(\epsilon^{-1}T_{\mathrm{TV}}^{\star})$ , and more samples are required. We also propose AdaP-TT, an  $\epsilon$ -global DP variant of the Top Two algorithms, that runs in adaptive phases and adds Laplace noise. AdaP-TT achieves the high privacy regime lower bound up to multiplicative constants.
The upper bound matches the lower bound by a multiplicative constant in the high privacy regime, and is also loose in some instances in the low privacy regime, due to the mismatch between the KL divergence of Bernoulli distributions and that of Gaussian. It would be an interesting technical challenge to merge this gap. One possible direction to solve this issue is to use transportation costs tailored to Bernoulli for both the Top Two Sampling and the stopping. Another interesting direction would be to extend the proposed technique to other variants of pure DP, namely  $(\epsilon, \delta)$ -DP and Rényi-DP [Mir17], or other trust models, namely local DP [DJW13] and shuffle DP [Che21, GDD+21].
This work has been partially supported by the THIA ANR program "AI_PhD@Lille". A. Al-Marjani acknowledges the support of the Chaire SeqALO (ANR-20-CHIA-0020). D. Basu acknowledges the Inria-Kyoto University Associate Team "RELIANT" for supporting the project, and the ANR JCJC for the REPUBLIC project (ANR-22-CE23-0003-01). We thank Emilie Kaufmann and Aurélien Garivier for the interesting conversations. We also thank Philippe Preux for his support.