SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We also fix a sequence of sampled actions  $\underline{a}^T = \{a_1, \dots, a_T\} \in [K]^T$  and a recommended action  $\hat{a} \in K$ .
We refer to AdaP-TT BAI strategy by  $\pi$ .
We want to show that:  $\pi (\underline{a}^T,\widehat{a},T\mid \underline{\mathbf{d}}^T)\leq e^\epsilon \pi (\underline{a}^T,\widehat{a},T\mid \underline{\mathbf{d}}^T)$
The main idea is that the change of reward in the  $j$ -th reward only affects the empirical mean computed in one episode, which is made private using the Laplace Mechanism and Lemma 2.
Step 1. Sequential decomposition of the output probability
We observe that due to the sequential nature of the interaction, the output probability can be decomposed to a part that depends on  $\underline{\mathbf{d}}^{j - 1} \triangleq \{\mathbf{x}_1, \ldots, \mathbf{x}_{j - 1}\}$ , which is identical for both  $\underline{\mathbf{d}}^T$  and  $\underline{\mathbf{d}}^{\prime T}$  and a second conditional part on the history.
Specifically, we have that
$$
\begin{array}{l} \pi \left(\underline {{a}} ^ {T}, \widehat {a}, T \mid \underline {{\mathbf {d}}} ^ {T}\right) \triangleq \operatorname {R e c} _ {T + 1} \left(\widehat {a} \mid \mathcal {H} _ {T}\right) \mathrm {S} _ {T + 1} (\top \mid \mathcal {H} _ {T}) \prod_ {t = 1} ^ {T} \mathrm {S} _ {t} \left(a _ {t} \mid \mathcal {H} _ {t - 1}\right) \\ \triangleq \mathcal {P} _ {\mathbf {d} ^ {j - 1}} ^ {\pi} (\underline {{a}} ^ {j}) \mathcal {P} _ {\underline {{\mathbf {d}}}} ^ {\pi} (a _ {> j}, \hat {a}, T \mid \underline {{a}} ^ {j}) \\ \end{array}
$$
$$
\begin{array}{l} \bullet a _ {> j} \triangleq (a _ {j + 1}, \dots , a _ {T}) \\ \cdot \mathcal {P} _ {\mathbf {d} ^ {j - 1}} ^ {\pi} (\underline {{a}} ^ {j}) \triangleq \prod_ {t = 1} ^ {j} S _ {t} (a _ {t} \mid \mathcal {H} _ {t - 1}) \\ \cdot \mathcal {P} _ {\underline {{\mathbf {d}}}} ^ {\pi} (a _ {> j}, \hat {a}, T \mid \underline {{a}} ^ {j}) \triangleq \operatorname {R e c} _ {T + 1} (\widehat {a} \mid \mathcal {H} _ {T}) \mathrm {S} _ {T + 1} (\top \mid \mathcal {H} _ {T}) \prod_ {t = j + 1} ^ {T} \mathrm {S} _ {t} (a _ {t} \mid \mathcal {H} _ {t - 1}) \\ \end{array}
$$
$$
\pi (\underline {{a}} ^ {T}, \widehat {a}, T \mid \underline {{\mathbf {d}}} ^ {\prime T}) \triangleq \mathcal {P} _ {\underline {{\mathbf {d}}} ^ {j - 1}} ^ {\pi} (\underline {{a}} ^ {j}) \mathcal {P} _ {\underline {{\mathbf {d}}} ^ {\prime}} ^ {\pi} (a _ {> j}, \widehat {a}, T \mid \underline {{a}} ^ {j})
$$
since  $\underline{\mathbf{d}}^{rj - 1} = \underline{\mathbf{d}}^{j - 1}$
$$
\frac {\pi \left(\underline {{a}} ^ {T} , \hat {a} , T \mid \underline {{\mathbf {d}}} ^ {T}\right)}{\pi \left(\underline {{a}} ^ {T} , \hat {a} , T \mid \underline {{\mathbf {d}}} ^ {\prime T}\right)} = \frac {\mathcal {P} _ {\underline {{\mathbf {d}}}} ^ {\pi} \left(a _ {> j} , \hat {a} , T \mid \underline {{a}} ^ {j}\right)}{\mathcal {P} _ {\underline {{\mathbf {d}}} ^ {\prime}} ^ {\pi} \left(a _ {> j} , \hat {a} , T \mid \underline {{a}} ^ {j}\right)} \tag {9}
$$
Step 2. The adaptive episodes are the same, before step  $j$
Let  $\ell$  such that  $t_{\ell} \leq j < t_{\ell + 1}$  when  $\pi$  interacts with  $\underline{\mathbf{d}}^T$ . Let us call it  $\psi_{\mathbf{d}^T}^\pi(j) \triangleq \ell$ .
Similarly, let  $\ell'$  such that  $t_{\ell'} \leq j < t_{\ell'+1}$  when  $\pi$  interacts with  $\underline{\mathbf{d}}'_T$ . Let us call it  $\psi_{\underline{\mathbf{d}}'_T}^{\pi}(j) \triangleq \ell'$ .
Since  $\psi_{\underline{\mathbf{d}}^T}(j)$  only depends on  $\underline{\mathbf{d}}^{j - 1}$ , which is identical for  $\underline{\mathbf{d}}^T$  and  $\underline{\mathbf{d}}^{\prime T}$ , we have that  $\psi_{\underline{\mathbf{d}}^T}^\pi (j) = \psi_{\underline{\mathbf{d}}^{\prime T}}^\pi (j)$  with probability 1.
We call  $\xi_{j}$  the last time-step of the episode  $\psi_{\underline{\mathbf{d}}^T}(j)$ , i.e.  $\xi_{j} \triangleq t_{\psi_{\underline{\mathbf{d}}^T}(j) + 1} - 1$ .
Step 3. Private sufficient statistics
Let  $r_t \triangleq \underline{\mathbf{d}}_{t,a_t}^T$ , be the reward corresponding to the action  $a_t$  in the table  $\underline{\mathbf{d}}^T$ . Similarly,  $r_t' \triangleq \underline{\mathbf{d}}_{t,a_t}'^T$  for  $\underline{\mathbf{d}}'_T$ .
Let us define  $L_{j} \triangleq \mathcal{G}_{\{r_{1}, \ldots, r_{\xi_{j}}\}}$  and  $L_{j}' \triangleq \mathcal{G}_{\{r_{1}', \ldots, r_{\xi_{j}}'\}}$ , where  $\mathcal{G}$  is defined as in Eq. 8, using the same episodes for  $d$  and  $d'$ . In other words,  $L_{j}$  is the list of private empirical means computed on a non-overlapping sequence of rewards before step  $\xi_{j}$ .
Using the forgetting structure of AdaP-TT, there exists a randomised mapping  $f_{\underline{\mathbf{d}}_{> \xi_j}}$  such that  $\mathcal{P}_{\underline{\mathbf{d}}}^{\pi}(.| \underline{a}^{j}) = f_{\underline{\mathbf{d}}_{> \xi_j}} \circ L_j$  and  $\mathcal{P}_{\underline{\mathbf{d}}'}^{\pi}(.| \underline{a}^{j}) = f_{\underline{\mathbf{d}}_{> \xi_j}} \circ L_j'$ .
In other words, the interaction of  $\pi$  with  $\underline{\mathbf{d}}$  and  $\underline{\mathbf{d}}^{\prime}$  from step  $\xi_{j} + 1$  until  $T$  only depends on the sufficient statistics  $L_{j}$ , which summarises what happened before  $\xi_{j}$ , and the new inputs  $\underline{\mathbf{d}}_{> \xi_j}$ , which are the same for  $\underline{\mathbf{d}}$  and  $\underline{\mathbf{d}}^{\prime}$ .
Step 4. Concluding with Lemma 2 and the post-processing lemma
Since rewards are in  $[0,1]$ , using Lemma 2, we have that  $\mathcal{G}$  is  $\epsilon$ -DP.
Since  $\mathcal{P}_{\mathbf{d}}^{\pi}(.|\underline{a}^{j})$  is just a post-processing of the output of  $\mathcal{G}$ , we have that
$$
\frac {\mathcal {P} _ {\underline {{\mathbf {d}}}} ^ {\pi} (a _ {> j} , \hat {a} , T \mid \underline {{a}} ^ {j})}{\mathcal {P} _ {\underline {{\mathbf {d}}} ^ {\prime}} ^ {\pi} (a _ {> j} , \hat {a} , T \mid \underline {{a}} ^ {j})} \leq e ^ {\epsilon},
$$
and Eq. (9) concludes the proof.
D AdaP-TT: A private Top Two algorithm with adaptive episodes
We propose a generic wrapper to adapt existing BAI algorithms to tackle private BAI (Appendix D.1). Then, we show how to instantiate our wrapper with an instance of Top Two algorithm (Appendix D.2), namely TTUCB [JD22].
D.1 Generic wrapper on existing BAI algorithms
We propose a generic wrapper to adapt existing BAI algorithms to tackle the  $\epsilon$ -DP-FC-BAI problem.
1. It uses adaptive episodes with doubling and forgetting (Appendix D.1.1). This builds on the idea used to make the UCB algorithm private for regret minimisation [AB22].
2. It relies on a private GLR (generalised likelihood ratio) stopping rule (Appendix D.1.2). The GLR stopping rule [GK16] is widely used in the BAI literature, since it ensures  $\delta$ -correctness of the stopping rule regardless of the sampling rule.
For each arm  $a \in [K]$ , we maintain a phase at time  $n$  whose index will be denoted by  $k_{n,a} \in \mathbb{N}$ . We switch phase as soon as the number of times that the arm was played is doubled. We only evaluate the stopping condition when we switch phase for an arm.
D.1.1 Adaptive episodes with doubling and forgetting
As initialisation, we start by pulling each arm once, and set  $k_{n,a} = 1$ ,  $T_1(a) = K + 1$  and  $N_{n,a} = 1$  for all  $a \in [K]$ . In the following, we will consider  $n > K$  and we denote the global pulling count of arm  $a$  before time  $n$  by  $N_{n,a} \triangleq \sum_{t \in [n-1]} \mathbb{1}(I_t = a)$ . For each arm  $a \in [K]$ , the random stopping time denoting the end of  $k_a - 1$  and the beginning of phase  $k_a > 1$  is denoted by
$$
T _ {k _ {a}} (a) \triangleq \inf  \left\{n \in \mathbb {N} \mid N _ {n, a} \geq 2 N _ {T _ {k _ {a} - 1} (a), n} \right\}. \tag {10}
$$
At the beginning of phase  $k_{a}$  for an arm  $a \in [K]$ , we update the empirical mean based on the observations on arm  $a \in [K]$  collected during this last phase, i.e.
$$
\hat {\mu} _ {k _ {a}, a} \triangleq \frac {1}{\tilde {N} _ {k _ {a} , a}} \sum_ {s = T _ {k _ {a} - 1} (a)} ^ {T _ {k _ {a}} (a) - 1} X _ {s} \mathbb {1} \left\{I _ {s} = a \right\}, \tag {11}
$$
where the local pulling count pf arm  $a$  is denoted by  $\tilde{N}_{k_a,a} \triangleq N_{T_{k_a}(a),a} - N_{T_{k_a-1}(a),a}$ , meaning it is the number of collected samples during the phase  $k_a - 1$ . To ensure privacy, we add a Laplace
noise to define the private empirical mean, i.e.
$$
\tilde {\mu} _ {k _ {a}, a} \triangleq \hat {\mu} _ {k _ {a}, a} + Y _ {k _ {a}, a} \quad \text {w h e r e} \quad Y _ {k _ {a}, a} \sim \operatorname {L a p} \left(\frac {1}{\epsilon \tilde {N} _ {k _ {a} , a}}\right). \tag {12}
$$
We emphasise that only the private version the estimator  $\hat{\mu}_{k_a,a}$ , i.e.  $\tilde{\mu}_{k_a,a}$ , is used by the algorithm until the end of phase  $k_{a}$  for arm  $a$ . Since  $(k_{n,a})_{a\in [K]}$  denotes the current phases at time  $n$ , our algorithm relies on  $(\hat{\mu}_{k_{n,a},a},\tilde{\mu}_{k_a,a},\tilde{N}_{k_{n,a},a},N_{n,a})_{a\in [K]}$ .
Due to the doubling, the growth of the global and local pulling counts is exponential (Lemma 3).
Lemma 3. For all  $a \in [K]$  and all  $k \in \mathbb{N}$  such that  $\mathbb{E}_{\nu}[T_k(a)] < +\infty$ , we have
$$
N _ {T _ {k} (a), a} = 2 ^ {k - 1} \quad a n d \quad \tilde {N} _ {k, a} = 2 ^ {k - 2}.
$$
Proof. Let  $a \in [K]$ . After initialisation, we have  $k = 1$ ,  $T_{1}(a) = K + 1$  and  $N_{T_{1}(a),a} = 1$ . Using the definition of the adaptive phase switch (Equation (10)), it is direct to see that  $N_{T_{2}(a),a} = 2$  and  $\tilde{N}_{2,a} = 1$  when  $\mathbb{E}_{\nu}[T_2(a)] < +\infty$ .
Now, we proceed by recurrence. Suppose that  $N_{T_k(a),a} = 2^{k - 1}$  and  $\tilde{N}_{k,a} = 2^{k - 2}$  when  $\mathbb{E}_{\nu}[T_k(a)] < +\infty$ . If  $\mathbb{E}_{\nu}[T_{k + 1}(a)] < +\infty$ , then it means that the phase  $k$  ends for arm  $a$  almost surely. Since we sample only one arm at each round, at the beginning of phase  $k + 1$  for arm  $a$ , we have  $N_{T_{k + 1}(a),a} = 2N_{T_k(a),a} = 2^k$  by using the definition of the adaptive phase switch (10). Then, we have directly that  $\tilde{N}_{k + 1,a} = N_{T_{k + 1}(a),a} - N_{T_k(a),a} = 2^k - 2^{k - 1} = 2^{k - 1}$ .
D.1.2 GLR stopping rule
Non-private GLR stopping rule with phases. Given a set of non-private threshold functions  $(c_k)_{k\in \mathbb{N}}$ , such that  $c_{k}:\mathbb{N}\times \mathbb{N}\times (0,1)\to \mathbb{R}_{+}$  for all  $k\in \mathbb{N}$ , the non-private GLR stopping rule can be evaluated at the beginning of each phase for each arm, namely
$$
\tau_ {\delta} ^ {\mathrm {N P}} = \inf  \left\{n \in \mathbb {N} \mid \forall b \neq \hat {a} _ {n} ^ {\mathrm {N P}}, \frac {\left(\hat {\mu} _ {k _ {n , \hat {a} _ {n} ^ {\mathrm {N P}}} , \hat {a} _ {n} ^ {\mathrm {N P}} - \hat {\mu} _ {k _ {n , b} , b}\right) ^ {2}}{1 / \tilde {N} _ {k _ {n , \hat {a} _ {n} ^ {\mathrm {N P}}} , \hat {a} _ {n} ^ {\mathrm {N P}}} + 1 / \tilde {N} _ {k _ {n , b , b}}} \geq 2 c _ {k _ {n, \hat {a} _ {n} ^ {\mathrm {N P}}} k _ {n, b}} \left(\tilde {N} _ {k _ {n, \hat {a} _ {n} ^ {\mathrm {N P}}} , \hat {a} _ {n} ^ {\mathrm {N P}}}, \tilde {N} _ {k _ {n, b}, b}, \delta\right) \right\}, \tag {13}
$$
where  $\hat{a}_n^{\mathrm{NP}} = \arg \max_{a\in [K]}\hat{\mu}_{k_{n,a},a}$  is the non-private candidate answer until we switch phase again (for any arm). We emphasise that this stopping condition is only evaluated at the beginning of each phase for each arm since it involves quantities that are fixed until we switch phase again.
Lemma 4 gives non-private threshold functions ensuring that the non-private GLR stopping rule is  $\delta$ -correct for all  $\delta \in (0,1)$ , independently of the sampling rule.
Lemma 4. Let  $\delta \in (0,1)$ . Let  $s > 1$  and  $\zeta$  be the Riemann  $\zeta$  function. Given any sampling rule, the non-private GLR stopping rule (Equation (13)) with non-private threshold functions
$$
c _ {k} (n, m, \delta) = 2 \mathcal {C} _ {G} \left(\frac {1}{2} \log \left(\frac {(K - 1) \zeta (s) ^ {2} k ^ {s}}{\delta}\right)\right) + 2 \log (4 + \log n) + 2 \log (4 + \log m) \tag {14}
$$
ensures  $\delta$ -correctness for 1-sub-Gaussian distributions. The function  $\mathcal{C}_G$  is defined in Equation (15). It satisfies  $\mathcal{C}_G \approx x + \log x$ .
Proof. The non-private GLR stopping rule matches the one used for Gaussian bandits. Proving  $\delta$ -correctness of a GLR stopping rule is done by leveraging concentration results. In particular, we build upon Theorem 9 of [KK21], which is restated below.
Lemma 5. Consider sub-Gaussian bandits with means  $\pmb{\mu} \in \mathbb{R}^{K}$ . Let  $S \subseteq [K]$  and  $x > 0$ .
$$
\mathbb {P} _ {\nu} \left[ \exists n \in \mathbb {N}, \sum_ {k \in S} \frac {N _ {n , k}}{2} (\mu_ {n, k} - \mu_ {k}) ^ {2} > \sum_ {k \in S} 2 \log \left(4 + \log (N _ {n, k})\right) + | S | \mathcal {C} _ {G} \left(\frac {x}{| S |}\right) \right] \leq e ^ {- x},
$$
where  $\mathcal{C}_G$  is defined in [KK21] as
$$
\mathcal {C} _ {G} (x) \triangleq \min  _ {\lambda \in ] 1 / 2, 1 ]} \frac {g _ {G} (\lambda) + x}{\lambda} \quad a n d \quad g _ {G} (\lambda) \triangleq 2 \lambda - 2 \lambda \log (4 \lambda) + \log \zeta (2 \lambda) - \frac {1}{2} \log (1 - \lambda). \tag {15}
$$
Here,  $\zeta$  is the Riemann  $\zeta$  function and  $\mathcal{C}_G(x) \approx x + \log (x)$ .
Let  $\hat{a} = \hat{a}_n^{\mathrm{NP}} = \arg \max_{b\in [K]}\hat{\mu}_{k_{n,a},a}$ . Standard manipulations yield that for all  $b\neq \hat{a}$
$$
\frac {(\hat {\mu} _ {k _ {n , \hat {a}} , \hat {a}} - \hat {\mu} _ {k _ {n , b} , b}) ^ {2}}{1 / \tilde {N} _ {k _ {n , \hat {a}} , \hat {a}} + 1 / \tilde {N} _ {k _ {n , b} , b}} = \inf  _ {y \geq x} \left\{\tilde {N} _ {k _ {n, \hat {a}}, \hat {a}} (\hat {\mu} _ {k _ {n, \hat {a}}, \hat {a}} - x) ^ {2} + \tilde {N} _ {k _ {n, b}, b} (\hat {\mu} _ {k _ {n, b}, b} - y) ^ {2} \right\}.
$$
Using the non-private GLR stopping rule (13) with non-private threshold functions  $(c_k)_{k\in \mathbb{N}}$  and the above manipulations, we obtain
$$
\mathbb {P} _ {\boldsymbol {\nu}} \left(\tau_ {\delta} ^ {\mathrm {N P}} <   + \infty , \hat {a} \neq a ^ {\star}\right)
$$
$$
\leq \mathbb {P} _ {\boldsymbol {\nu}} \left(\exists n \in \mathbb {N}, \exists a \neq a ^ {\star}, a = \underset {b \in [ K ]} {\arg \max } \hat {\mu} _ {k _ {n, b}, b}, \forall b \neq a, \right.
$$
$$
\inf  _ {y \geq x} \left\{\tilde {N} _ {k _ {n, a}, a} (\hat {\mu} _ {k _ {n, a}, a} - x) ^ {2} + \tilde {N} _ {k _ {n, b}, b} (\hat {\mu} _ {k _ {n, b}, b} - y) ^ {2} \right\} \geq 2 c _ {k _ {n, a} k _ {n, b}} (\tilde {N} _ {k _ {n, a}, a}, \tilde {N} _ {k _ {n, b}, b}, \delta)
$$
$$
\leq \mathbb{P}_{\nu}\left(\exists n\in \mathbb{N},  \exists a\neq a^{\star},  a = \operatorname *{arg  max}_{b\in [K]}\hat{\mu}_{k_{n,a},a}, \right.
$$
$$
\tilde {N} _ {k _ {n, a}, a} \left(\hat {\mu} _ {k _ {n, a}, a} - \mu_ {a}\right) ^ {2} + \tilde {N} _ {k _ {n, a ^ {\star}}, a ^ {\star}} \left(\hat {\mu} _ {k _ {n, a ^ {\star}}, a ^ {\star}} - \mu_ {a ^ {\star}}\right) ^ {2} \geq 2 c _ {k _ {n, a} k _ {n, a ^ {\star}}} \left(\tilde {N} _ {k _ {n, a}, a}, \tilde {N} _ {k _ {n, a ^ {\star}}, a ^ {\star}}, \delta\right)
$$
$$
\stackrel {\leq} {(\mathbb {b})} \mathbb {P} _ {\boldsymbol {\nu}} \left(\exists a \neq a ^ {\star}, \exists (k _ {a}, k _ {a ^ {\star}}) \in \mathbb {N} ^ {2}, \right.
$$
$$
\tilde {N} _ {k _ {a}, a} \left(\hat {\mu} _ {k _ {a}, a} - \mu_ {a}\right) ^ {2} + \tilde {N} _ {k _ {a ^ {\star}}, a ^ {\star}} \left(\hat {\mu} _ {k _ {a ^ {\star}}, a ^ {\star}} - \mu_ {a ^ {\star}}\right) ^ {2} \geq 2 c _ {k _ {a} k _ {a ^ {\star}}} \left(\tilde {N} _ {k _ {a}, a}, \tilde {N} _ {k _ {a ^ {\star}}, a ^ {\star}}, \delta\right)
$$
$$
\leq \sum_ {(c)} \sum_ {a \neq a ^ {\star}} \sum_ {(k _ {a}, k _ {a ^ {\star}}) \in \mathbb {N} ^ {2}} \mathbb {P} _ {\nu} \left(\tilde {N} _ {k _ {a}, a} (\hat {\mu} _ {k _ {a}, a} - \mu_ {a}) ^ {2} + \tilde {N} _ {k _ {a ^ {\star}}, a ^ {\star}} (\hat {\mu} _ {k _ {a ^ {\star}}, a ^ {\star}} - \mu_ {a ^ {\star}}) ^ {2} \right.
$$
$$
\left. \geq 2 c _ {k _ {a} k _ {a ^ {\star}}} \left(\tilde {N} _ {k _ {a}, a}, \tilde {N} _ {k _ {a ^ {\star}}, a ^ {\star}}, \delta\right)\right),
$$
The inequality (a) is obtained with  $(b,x,y) = (a^{\star},\mu_{a},\mu_{a^{\star}})$ . The inequality (b) drops the condition  $a = \arg \max_{b\in [K]}\hat{\mu}_{kb,b}$ , hence we can restrict to  $(k_{a},k_{a^{\star}})\in \mathbb{N}^{2}$  since it doesn't depend on other phase indices. The inequality (c) relies on a direct union bound. For all  $a\neq a^{\star}$  and all  $(k_{a},k_{a^{\star}})\in \mathbb{N}^{2}$ , the estimators  $\hat{\mu}_{k_a,a}$  (resp.  $\hat{\mu}_{k_{a^{\star}},a^{\star}}$ ) are based solely on the observations collected for arm  $a$  (resp. arm  $a^{\star}$ ) between times  $n\in \{T_{k_a - 1}(a),\dots ,T_{k_a}(a) - 1\}$  (resp.  $n\in \{T_{k_{a^{\star}} - 1}(a^{\star}),\dots ,T_{k_{a^{\star}}}(a^{\star}) - 1\}$ ) with local counts  $\tilde{N}_{k_a,a}$  (resp.  $\tilde{N}_{k_{a^{\star}},a^{\star}}$ ), i.e. dropping past observations. Using Lemma 5 for all  $a\neq a^{\star}$  and all  $(k_{a},k_{a^{\star}})\in \mathbb{N}^{2}$ , we obtain
$$
\mathbb {P} _ {\boldsymbol {\nu}} \left(\tau_ {\delta} <   + \infty , \hat {i} = i ^ {\star}\right) \leq \frac {\delta}{K - 1} \frac {1}{\zeta (s) ^ {2}} \sum_ {a \neq a ^ {\star}} \sum_ {(k _ {a}, k _ {a ^ {\star}}) \in \mathbb {N} ^ {2}} \frac {1}{(k _ {a} k _ {a ^ {\star}}) ^ {s}} = \delta .
$$
Private GLR stopping rule with phases. Since we want to ensure privacy, the non-private GLR stopping rule cannot be used since it relies on the empirical means  $\hat{\mu}_{k_{n,a},a}$ , which are not private. To alleviate this problem, we propose the private GLR stopping rule, which is based on the private empirical means  $\tilde{\mu}_{k_{n,a},a}$ .
Given a set of private threshold functions  $(c_{\epsilon, k_1, k_2})_{(\epsilon, k_1, k_2) \in \mathbb{R}_+^* \times \mathbb{N}^2}$ , such that  $c_{\epsilon, k_1, k_2}: \mathbb{N} \times \mathbb{N} \times (0, 1) \to \mathbb{R}_+$  for all  $(\epsilon, k_1, k_2) \in \mathbb{R}_+^* \times \mathbb{N}^2$ , the non-private GLR stopping rule is evaluated at the beginning of each phase for each arm, namely
$$
\tau_ {\delta} = \inf  \left\{n \in \mathbb {N} \mid \forall b \neq \hat {a} _ {n}, \frac {\left(\tilde {\mu} _ {k _ {n , \hat {a} _ {n}} , \hat {a} _ {n}} - \tilde {\mu} _ {k _ {n , b} , b}\right) ^ {2}}{1 / \tilde {N} _ {k _ {n , \hat {a} _ {n}} , \hat {a} _ {n}} + 1 / \tilde {N} _ {k _ {n , b} , b}} \geq 2 c _ {\epsilon , k _ {n, \hat {a} _ {n}}, k _ {n, b}} \left(\tilde {N} _ {k _ {n, \hat {a} _ {n}}, \hat {a} _ {n}}, \tilde {N} _ {k _ {n, b}, b}, \delta\right) \right\}, \tag {16}
$$
where  $\hat{a}_n = \arg \max_{a\in [K]}\tilde{\mu}_{k_{n,a},a}$  is the private candidate answer until we switch phase again (for any arm). We emphasise that this stopping condition is only evaluated at the beginning of each phase for each arm since it involves quantities that are fixed until we switch phase again.
Theorem 4 gives private threshold functions ensuring that the private GLR stopping rule is  $\delta$ -correct for all  $\delta \in (0,1)$  and all  $\epsilon \in \mathbb{R}_{+}^{\star}$ , independently of the sampling rule.
Theorem 4 ( $\delta$ -correctness of the private GLR stopping rule). Let  $\delta \in (0,1)$  and  $\epsilon \in \mathbb{R}_{+}^{\star}$ . Let  $s > 1$  and  $\zeta$  be the Riemann  $\zeta$  function and non-private threshold function  $(c_k)_{k \in \mathbb{N}}$  as in (14). Given any sampling rule, the private GLR stopping rule (Equation (16)) with private threshold functions
$$
c _ {\epsilon , k _ {1}, k _ {2}} (n, m, \delta) = 2 c _ {k _ {1} k _ {2}} (n, m, \delta / 2) + \frac {1}{n \epsilon^ {2}} \log \left(\frac {2 K k _ {1} ^ {s} \zeta (s)}{\delta}\right) ^ {2} + \frac {1}{m \epsilon^ {2}} \log \left(\frac {2 K k _ {2} ^ {s} \zeta (s)}{\delta}\right) ^ {2} \tag {17}
$$
ensures  $\delta$ -correctness for 1-sub-Gaussian distributions.
Proof. Let  $\epsilon \in \mathbb{R}_+^*$ . Since  $Y_{k_{n,a},a} \sim \mathrm{Lap}\left((\epsilon \tilde{N}_{k_{n,a},a})^{-1}\right)$ , we have that  $\tilde{N}_{k_{n,a},a}|Y_{k_{n,a},a}| \sim \mathcal{E}(\epsilon)$  for all  $a \in [K]$  and all  $n \in \mathbb{N}$ , where  $\mathcal{E}(\cdot)$  denotes the exponential distribution. Using concentration results for exponential distribution, a direct union bound yields that
$$
\mathbb {P} \left(\exists n \in \mathbb {N}, \exists a \in [ K ], \tilde {N} _ {k _ {n, a}, a} \left| Y _ {k _ {n, a}, a} \right| \geq \frac {1}{\epsilon} \log \left(\frac {K k _ {n , a} ^ {s} \zeta (s)}{\delta}\right)\right) \leq \delta . \tag {18}
$$
Let us denote  $\tilde{c}_{\epsilon ,k_1,k_2}(n,m,\delta)$  the threshold associated to the Laplace noise, i.e.
$$
\tilde {c} _ {\epsilon , k _ {1}, k _ {2}} (n, m, \delta) = \frac {1}{n \epsilon^ {2}} \log \left(\frac {K k _ {1} ^ {s} \zeta (s)}{\delta}\right) ^ {2} + \frac {1}{m \epsilon^ {2}} \log \left(\frac {K k _ {2} ^ {s} \zeta (s)}{\delta}\right) ^ {2}.
$$
Using the private GLR stopping rule (Equation (16)) with private threshold functions  $(c_{\epsilon ,k_1,k_2})_{(\epsilon ,k_1,k_2)\in \mathbb{R}_+^*\times \mathbb{N}^2}$ , similar manipulations as above yields
$$
\begin{array}{l} \mathbb {P} _ {\boldsymbol {\nu}} \left(\tau_ {\delta} <   + \infty , \hat {a} \neq a ^ {\star}\right) \\ \leq \mathbb {P} _ {\nu} \left(\exists n \in \mathbb {N}, \exists a \neq a ^ {\star}, \tilde {N} _ {k _ {n, a}, a} \left(\hat {\mu} _ {k _ {n, a}, a} - \mu_ {a} + Y _ {k _ {n, a}, a}\right) ^ {2} + \tilde {N} _ {k _ {n, a ^ {\star}}, a ^ {\star}} \left(\hat {\mu} _ {k _ {n, a ^ {\star}}, a ^ {\star}} - \mu_ {a ^ {\star}} + Y _ {k _ {n, a ^ {\star}}, a ^ {\star}}\right) ^ {2} \right. \\ \geq 4 c _ {k _ {n, a} k _ {n, a ^ {\star}}} \left(\tilde {N} _ {k _ {n, a}, a}, \tilde {N} _ {k _ {n, a ^ {\star}}, a ^ {\star}}, \delta / 2\right) + 2 \tilde {c} _ {\epsilon , k _ {n, a}, k _ {n, a ^ {\star}}} \left(\tilde {N} _ {k _ {n, a}, a}, \tilde {N} _ {k _ {n, a ^ {\star}}, a ^ {\star}}, \delta / 2\right)) \\ \end{array}
$$
$$
\underset {(a)} {\leq} \mathbb {P} _ {\boldsymbol {\nu}} \left(\exists a \neq a ^ {\star}, \exists (k _ {a}, k _ {a ^ {\star}}) \in \mathbb {N} ^ {2}, \right.
$$
$$
\tilde {N} _ {k _ {a}, a} (\hat {\mu} _ {k _ {a}, a} - \mu_ {a}) ^ {2} + \tilde {N} _ {k _ {a ^ {\star}}, a ^ {\star}} (\hat {\mu} _ {k _ {a ^ {\star}}, a ^ {\star}} - \mu_ {a ^ {\star}}) ^ {2} \geq 2 c _ {k _ {a} k _ {a ^ {\star}}} (\tilde {N} _ {k _ {a}, a}, \tilde {N} _ {k _ {a ^ {\star}}, a ^ {\star}}, \delta / 2) \Big)
$$
$$
\begin{array}{l} + \mathbb {P} _ {\boldsymbol {\nu}} \left(\exists n \in \mathbb {N}, \exists a \neq a ^ {\star}, \tilde {N} _ {k _ {n, a}, a} Y _ {k _ {n, a}, a} ^ {2} + \tilde {N} _ {k _ {n, a ^ {\star}}, a ^ {\star}} Y _ {k _ {n, a ^ {\star}}, a ^ {\star}} ^ {2} \right. \\ \left. \geq \tilde {c} _ {\epsilon , k _ {n, a}, k _ {n, a ^ {*}}} \left(\tilde {N} _ {k _ {n, a}, a}, \tilde {N} _ {k _ {n, a ^ {*}}, a ^ {*}}, \delta / 2\right)\right) \\ \end{array}
$$
$$
\begin{array}{l} \leq \delta / 2 + \mathbb {P} \left(\exists n \in \mathbb {N}, \exists a \in [ K ], \tilde {N} _ {k _ {n, a}, a} | Y _ {k _ {n, a}, a} | \geq \frac {1}{\epsilon} \log \left(\frac {2 K k _ {n , a} ^ {s} \zeta (s)}{\delta}\right)\right) \\ \begin{array}{l} \underset {(c)} {\leq} \delta / 2 + \delta / 2 = \delta  . \end{array} \\ \end{array}
$$
The inequality (a) uses that  $\mathbb{P}(X + Y\geq a + b)\leq \mathbb{P}(X\geq a) + \mathbb{P}(Y\geq b)$  and  $(x - y)^2\leq 2x^2 +2y^2$  The inequality (c) leverages Lemma 4 and a direct inclusion of event. The inequality (c) deploys Equation (18) to conclude.
D.2 AdaP-TT: Instantiating our wrapper with a Top Two sampling rule
A blueprint of Top Two algorithm design. At time  $n > K$ , a Top Two sampling rule defines a leader  $B_{n}$  and a challenger  $C_n$ . Then, it selects among them the next arm to sample  $I_{n}$ . Given a proportion  $\beta \in (0,1)$  fixed beforehand, the choice of  $I_{n} \in \{B_{n},C_{n}\}$  should ensure that the leader is sampled close to  $\beta$  of the times where it was chosen as leader. In early works on Top Two algorithms, this choice is randomised. Following [JD22], we use  $K$  independent tracking procedures.