SlowGuess/NeurIPS_2023_OCR huggingface dataset:
[AB22] Achraf Azize and Debabrota Basu. When privacy meets partial information: A refined analysis of differentially private bandits. Advances in Neural Information Processing Systems, 35:32199-32210, 2022.
[ACBF02] Peter Auer, Nicolo Cesa-Bianchi, and Paul Fischer. Finite-time analysis of the multiarmed bandit problem. Machine learning, 47(2-3):235-256, 2002.
[AKR21] Maryam Aziz, Emilie Kaufmann, and Marie-Karelle Riviere. On multi-armed bandit designs for dose-finding clinical trials. The Journal of Machine Learning Research, 22(1):686-723, 2021.
[ASZ21] Jayadev Acharya, Ziteng Sun, and Huanyu Zhang. Differentially private assouad, fano, and le cam. In Algorithmic Learning Theory, pages 48-78. PMLR, 2021.
[AYBG+18] Yasin Abbasi-Yadkori, Peter Bartlett, Victor Gabillon, Alan Malek, and Michal Valko. Best of both worlds: Stochastic & adversarial best-arm identification. In Conference on Learning Theory, pages 918-949. PMLR, 2018.
[BDT19] Debabrota Basu, Christos Dimitrakakis, and Aristide Tossou. Differential privacy for multi-armed bandits: What is it and what is its cost? arXiv preprint arXiv:1905.12298, 2019.
[Bec54] Robert E Bechhofer. A single-sample multiple decision procedure for ranking means of normal populations with known variances. The Annals of Mathematical Statistics, pages 16-39, 1954.
[Bec58] Robert E Bechhofer. A sequential multiple-decision procedure for selecting the best one of several normal populations with a common unknown variance, and its use with various experimental designs. Biometrics, 14(3):408-429, 1958.
[Che21] Albert Cheu. Differential privacy in the shuffle model: A survey of separations. arXiv preprint arXiv:2107.11839, 2021.
[CL16] Alexandra Carpentier and Andrea Locatelli. Tight (lower) bounds for the fixed budget best arm identification bandit problem. In Conference on Learning Theory, pages 590-604. PMLR, 2016.
$\left[\mathrm{CLK}^{+}14\right]$  Shouyuan Chen, Tian Lin, Irwin King, Michael R Lyu, and Wei Chen. Combinatorial pure exploration of multi-armed bandits. Advances in neural information processing systems, 27, 2014.
[CWZ21] T Tony Cai, Yichen Wang, and Linjun Zhang. The cost of privacy: Optimal rates of convergence for parameter estimation with differential privacy. The Annals of Statistics, 49(5):2825-2850, 2021.
[DJW13] John C Duchi, Michael I Jordan, and Martin J Wainwright. Local privacy and statistical minimax rates. In 2013 IEEE 54th Annual Symposium on Foundations of Computer Science, pages 429-438. IEEE, 2013.
[DKM19] Rémy Degenne, Wouter M Koolen, and Pierre Ménard. Non-asymptotic pure exploration by solving games. Advances in Neural Information Processing Systems, 32, 2019.
[SS19] Touqir Sajed and Or Sheffet. An optimal private stochastic-mab algorithm based on optimal private stopping rule. In International Conference on Machine Learning, pages 5579-5588. PMLR, 2019.
[TBD+16] Katherine Tucker, Janice Branson, Maria Dilleen, Sally Hollis, Paul Loughlin, Mark J Nixon, and Zoë Williams. Protecting patient privacy when sharing patient-level data from clinical trials. BMC medical research methodology, 16(1):5-14, 2016.
[TD16] Aristide CY Tossou and Christos Dimitrakakis. Algorithms for differentially private multi-armed bandits. In Thirtieth AAAI Conference on Artificial Intelligence, 2016.
[ZCL14] Yuan Zhou, Xi Chen, and Jian Li. Optimal pac multiple arm identification with applications to crowdsourcing. In International Conference on Machine Learning, pages 217-225. PMLR, 2014.
A Outline 16
B Lower bounds on sample complexity 16
B.1 Canonical model for BAI 16
B.2 Transportation lemma under  $\epsilon$ -global DP: Proof of Lemma 1 17
B.3 Lower bound on sample complexity: Proof of Theorem 2. 17
B.4 TV characteristic time for Bernoulli instances: Proof of Proposition 1 18
B.5 On the total variation distance and the hardness of privacy 19
C Privacy analysis 20
C.1 Privacy lemma for non-overlapping sequences 20
C.2 Privacy analysis of AdaP-TT: Proof of Theorem 3 21
D AdaP-TT: A private Top Two algorithm with adaptive episodes 22
D.1 Generic wrapper on existing BAI algorithms 22
D.2 AdaP-TT: Instantiating our wrapper with a Top Two sampling rule 25
E Analysis of AdaP-TT: Proof of Theorem 5 27
E.1 Technical results 28
E.2 Sufficient exploration 29
E.3 Convergence towards  $\beta$ -optimal allocation 33
E.4 Cost of doubling and forgetting 36
E.5 Asymptotic expected sample complexity 37
E.6 Connection to the lower bound 40
E.7 Limitation and open problem 42
E.8 Analysis of non-private AdaP-TT: Proof of Theorem 6 43
F Extended experimental analysis 44
- The proof of our lower bound is detailed in Appendix B (Theorem 2).
- The proof of the privacy of AdaP-TT is done in Appendix C (Theorem 3).
- The AdaP-TT algorithm is presented in more details in Appendix D, and we show the  $\delta$ -correctness of the non-private GLR stopping rule (Theorem 4).
- The asymptotic upper bound on the expected sample complexity of AdaP-TT is proven in Appendix E (Theorem 5).
- Extended experiments are presented in Appendix F.
B Lower bounds on sample complexity
In this section, we provide the proofs for the sample complexity lower bounds. First, we present the canonical model for BAI to introduce the relevant quantities. Then, we prove an  $\epsilon$ -global version of the transportation lemma, i.e. Lemma 1. Using this lemma, we prove Theorem 2. Finally, we prove the formula expressing the TV characteristic time for Bernoulli instances.
B.1 Canonical model for BAI
Let  $\nu \triangleq \{\nu_{a}: a \in [K]\}$  be a bandit instance, consisting of  $K$  arms with finite means  $\{\mu_{a}\}_{a \in [K]}$ . Now, we recall the interaction between a BAI strategy  $\pi$  and the bandit instance  $\nu$  in the Protocol 1. The BAI strategy  $\pi$  halts at  $\tau$ , samples a sequence of actions  $\underline{A}^{\tau}$ , and recommends the action  $\hat{A}$ . Let  $\mathbb{P}_{\nu,\pi}$  be the probability distribution over the triplets  $(\tau, \underline{A}^{\tau}, \hat{A})$ , when the BAI strategy  $\pi$  interacts with the bandit instance  $\nu$ .
For a fixed  $T > 1$ , a sequence of actions  $\underline{a}^T = (a_1, \dots, a_T) \in [K]^T$  and a recommendation  $\hat{a} \in [K]$ , we define the event  $E = \{\tau = T, \underline{A}^\tau = \underline{a}^T, \hat{A} = \hat{a}\}$ . We have that
$$
\mathbb {P} _ {\boldsymbol {\nu}, \pi} (E) = \int_ {\underline {{r}} ^ {T} = (r _ {1}, \dots , r _ {T}) \in \mathbb {R} ^ {T}} \pi (\underline {{a}} ^ {T}, \hat {a}, T \mid \underline {{r}} ^ {T}) \prod_ {t = 1} ^ {T} \mathrm {d} \nu_ {a _ {t}} (r _ {t}) d r _ {t}
$$
$$
\pi (\underline {{a}} ^ {T}, \widehat {a}, T \mid \underline {{r}} ^ {T}) \triangleq \operatorname {R e c} _ {T + 1} (\widehat {a} \mid \mathcal {H} _ {T}) \mathrm {S} _ {T + 1} (\top \mid \mathcal {H} _ {T}) \prod_ {t = 1} ^ {T} \mathrm {S} _ {t} (a _ {t} \mid \mathcal {H} _ {t - 1})
$$
and  $\mathcal{H}_t = (a_1, r_1, \ldots, a_t, r_t)$ .
Remark on the bandit feedback. Let  $\pi$  be an  $\epsilon$ -DP BAI strategy. Let  $T \geq 1$ ,  $\underline{a}^T \in [K]^T$  a sequence sampled actions and  $\widehat{a} \in [K]$  a recommended actions. This time, let  $\underline{r}^T = \{r_1, \dots, r_T\} \in \mathbb{R}^T$  and  $\underline{r}'^T \in \mathbb{R}^T$  two neighbouring sequence of rewards, i.e.  $d_{\mathrm{Ham}}(\underline{r}^T, \underline{r}'^T) \triangleq \sum_{t=1}^{T} \mathbb{1}\{r_t \neq r_t'\} = 1$ . Consider the table of rewards  $\underline{d}^T$  consisting of concatenating  $\underline{r}^T$  colon-wise  $K$  times, i.e.  $\underline{d}_{t,i}^T = \underline{r}_t^T$  for all  $i \in [K]$  and all  $t \in [T]$ . Define  $\underline{d'}^T$  similarly with respect to  $\underline{r}'^T$ .
In this case, by definition of  $\pi$ ,  $\underline{d}^T$  and  $\underline{d}'^T$ , it is direct that
$$
\pi (\underline {{a}} ^ {T}, \widehat {a}, T \mid \underline {{r}} ^ {T}) = \pi (\underline {{a}} ^ {T}, \widehat {a}, T \mid \underline {{d}} ^ {T})
$$
and  $d_{\mathrm{Ham}}(\underline{d}^T, \underline{d}'^T) = 1$ .
Which means that
$$
\pi (\underline {{a}} ^ {T}, \widehat {a}, T \mid \underline {{r}} ^ {T}) \leq e ^ {\epsilon} \pi (\underline {{a}} ^ {T}, \widehat {a}, T \mid \underline {{r}} ^ {\prime T}).
$$
In other words, if  $\pi$  is  $\epsilon$ -pure DP for neighbouring table of rewards  $\underline{d}^T$ , then  $\pi$  is also  $\epsilon$ -pure DP for neighbouring sequence of observed rewards  $\underline{r}^T$ .
B.2 Transportation lemma under  $\epsilon$ -global DP: Proof of Lemma 1
Lemma 1 (Transportation lemma under  $\epsilon$ -global DP). Let  $\delta \in (0,1)$  and  $\epsilon > 0$ . Let  $\pmb{\nu}$  be a bandit instance and  $\lambda \in \mathrm{Alt}(\pmb{\nu})$ . For any  $\delta$ -correct  $\epsilon$ -global DP BAI strategy, we have that
where  $\operatorname {kl}(x,y)\triangleq x\log \frac{x}{y} + (1 - x)\log \frac{1 - x}{1 - y}$  for  $x,y\in (0,1)$ .
Proof. Step 1: Distinguishability due to  $\delta$ -correctness. Let  $\pi$  be a  $\delta$ -correct  $\epsilon$ -global DP BAI strategy. Let  $\nu$  be a bandit instance and  $\lambda \in \mathrm{Alt}(\nu)$ .
Let  $\mathbb{P}_{\nu, \pi}$  denote the probability distribution of  $(\underline{A}, \widehat{A}, \tau)$  when the BAI strategy  $\pi$  interacts with  $\nu$ . For any alternative instance  $\lambda \in \mathrm{Alt}(\nu)$ , the data-processing inequality gives that
$$
\begin{array}{l} \operatorname {K L} \left(\mathbb {P} _ {\boldsymbol {\nu}, \pi} \| \mathbb {P} _ {\boldsymbol {\lambda}, \pi}\right) \geq \operatorname {k l} \left(\mathbb {P} _ {\boldsymbol {\nu}, \pi} \left(\widehat {A} = a ^ {\star} (\boldsymbol {\nu})\right), \mathbb {P} _ {\boldsymbol {\lambda}, \pi} \left(\widehat {A} = a ^ {\star} (\boldsymbol {\nu})\right)\right) \\ \geq \operatorname {k l} (1 - \delta , \delta). \tag {5} \\ \end{array}
$$
where the second inequality is because  $\pi$  is  $\delta$ -correct i.e.  $\mathbb{P}_{\pmb{\nu},\pi}\left(\widehat{A} = a^{\star}(\pmb{\nu})\right) \geq 1 - \delta$  and  $\mathbb{P}_{\pmb{\lambda},\pi}\left(\widehat{A} = a^{\star}(\pmb{\nu})\right) \leq \delta$ , and the monotonicity of the  $\mathrm{kl}$ .
Step 2: Connecting KL and TV under  $\epsilon$ -global DP. On the other hand, by the definition of the KL, we have that
$$
\mathrm {K L} \left(\mathbb {P} _ {\boldsymbol {\nu}, \pi} \parallel \mathbb {P} _ {\boldsymbol {\lambda}, \pi}\right) = \mathbb {E} _ {\tau , \underline {{A}} ^ {\tau}, \hat {A} \sim \mathbb {P} _ {\boldsymbol {\nu}, \pi}} \left[ \log \left(\frac {\mathbb {P} _ {\boldsymbol {\nu} , \pi} (\tau , \underline {{A}} ^ {\tau} , \hat {A})}{\mathbb {P} _ {\boldsymbol {\lambda} , \pi} (\tau , \underline {{A}} ^ {\tau} , \hat {A})}\right) \right]
$$
$$
\mathbb {P} _ {\boldsymbol {\nu}, \pi} (\tau = T, \underline {{A}} ^ {\tau} = \underline {{a}} ^ {T}, \hat {A} = \hat {a}) = \int_ {\underline {{r}} \in \mathbb {R} ^ {T}} \pi (\underline {{a}} ^ {T}, \hat {a}, T \mid \underline {{r}}) \prod_ {t = 1} ^ {T} \mathrm {d} \nu_ {a _ {t}} (r _ {t}).
$$
Since  $\pi$  is  $\epsilon$ -global DP, using the sequential Karwa-Vadhan lemma [AB22, Lemma 2], we get that
$$
\begin{array}{l} \log \left(\frac {\mathbb {P} _ {\boldsymbol {\nu} , \pi} (\tau = T , \underline {{A}} ^ {\tau} = \underline {{a}} ^ {T} , \hat {A} = \hat {a})}{\mathbb {P} _ {\boldsymbol {\lambda}, \pi} (\tau = T , \underline {{A}} ^ {\tau} = \underline {{a}} ^ {T} , \hat {A} = \hat {a})}\right) \leq 6 \epsilon \sum_ {t = 1} ^ {T} \mathrm {T V} \left(\nu_ {a _ {t}} \| \lambda_ {a _ {t}}\right) \\ = 6 \epsilon \sum_ {a = 1} ^ {K} N _ {a} (T) \mathrm {T V} \left(\nu_ {a} \| \lambda_ {a}\right) \\ \end{array}
$$
Which gives that
$$
\operatorname {K L} \left(\mathbb {P} _ {\boldsymbol {\nu}, \pi} \| \mathbb {P} _ {\boldsymbol {\lambda}, \pi}\right) \leq 6 \epsilon \mathbb {E} _ {\boldsymbol {\nu}, \pi} \left[ \sum_ {a = 1} ^ {K} N _ {a} (\tau) \mathrm {T V} \left(\nu_ {a} \| \lambda_ {a}\right) \right]. \tag {6}
$$
Combining Inequalities 5 and 6 concludes the proof.
B.3 Lower bound on sample complexity: Proof of Theorem 2
$$
\mathbb {E} _ {\nu} [ \tau ] \geq T ^ {\star} (\nu ; \epsilon) \log (1 / 3 \delta),
$$
where  $(T^{\star}(\pmb {\nu};\epsilon))^{-1}\triangleq \sup_{\omega \in \Sigma_K}\inf_{\pmb {\lambda}\in \mathrm{Alt}(\pmb {\nu})}\min \Big(\sum_{a = 1}^{K}\omega_a\mathrm{KL}(\nu_a\parallel \lambda_a),6\epsilon \sum_{a = 1}^{K}\omega_a\mathrm{TV}(\nu_a\parallel \lambda_a)\Big).$
Proof. Let  $\pi$  be a  $\delta$ -correct  $\epsilon$ -global DP BAI strategy. Let  $\pmb{\nu}$  be a bandit instance and  $\lambda \in \mathrm{Alt}(\pmb{\nu})$ .
Let  $\mathbb{E}$  denote the expectation under  $\mathbb{P}_{\nu ,\pi}$ , ie  $\mathbb{E} \triangleq \mathbb{E}_{\nu ,\pi}$ .
By Lemma 1, we have that  $6\epsilon \sum_{a = 1}^{K}\mathbb{E}\left[N_a(\tau)\right]\mathrm{TV}\left(\nu_a\parallel \lambda_a\right)\geq \mathrm{kl}(1 - \delta ,\delta)$
Lemma 1 from [KCG16] gives that  $\sum_{a=1}^{K} \mathbb{E}\left[N_a(\tau)\right] \mathrm{KL}\left(\nu_a \parallel \lambda_a\right) \geq \mathrm{kl}(1 - \delta, \delta)$ .
Since these two inequalities hold for all  $\lambda \in \mathrm{Alt}(\nu)$ , we get
$$
\begin{array}{l} \operatorname {k l} (1 - \delta , \delta) \leq \inf  _ {\boldsymbol {\lambda} \in \operatorname {A l t} (\boldsymbol {\nu})} \min  \left(6 \epsilon \sum_ {a = 1} ^ {K} \mathbb {E} \left[ N _ {a} (\tau) \right] \operatorname {T V} \left(\nu_ {a} \| \lambda_ {a}\right), \sum_ {a = 1} ^ {K} \mathbb {E} \left[ N _ {a} (\tau) \right] \operatorname {K L} \left(\nu_ {a} \| \lambda_ {a}\right)\right) \\ \stackrel {(a)} {=} \mathbb {E} [ \tau ] \inf  _ {\boldsymbol {\lambda} \in \operatorname {A l t} (\boldsymbol {\nu})} \min  \left(6 \epsilon \sum_ {a = 1} ^ {K} \frac {\mathbb {E} \left[ N _ {a} (\tau) \right]}{\mathbb {E} [ \tau ]} \mathrm {T V} \left(\nu_ {a} \| \lambda_ {a}\right), \sum_ {a = 1} ^ {K} \frac {\mathbb {E} \left[ N _ {a} (\tau) \right]}{\mathbb {E} [ \tau ]} \mathrm {K L} \left(\nu_ {a} \| \lambda_ {a}\right)\right) \\ \stackrel {(b)} {\leq} \mathbb {E} [ \tau ] \left(\sup  _ {\omega \in \Sigma_ {K}} \inf  _ {\boldsymbol {\lambda} \in \operatorname {A l t} (\boldsymbol {\nu})} \min  \left(6 \epsilon \sum_ {a = 1} ^ {K} \omega_ {a} \mathrm {T V} \left(\nu_ {a} \| \lambda_ {a}\right), \sum_ {a = 1} ^ {K} \omega_ {a} \mathrm {K L} \left(\nu_ {a} \| \lambda_ {a}\right)\right)\right) . \\ \end{array}
$$
(a) is due to the fact that  $\mathbb{E}[\tau]$  does not depend on  $\lambda$ . (b) is obtained by noting that the vector  $(\omega_{a})_{a\in [K]}\triangleq \left(\frac{\mathbb{E}_{\nu,\pi}[N_{a}(\tau)]}{\mathbb{E}_{\nu,\pi}[\tau]}\right)_{a\in [K]}$  belongs to the simplex  $\Sigma_K$ .
The theorem follows by noting that for  $\delta \in (0,1)$ ,  $\mathrm{kl}(1 - \delta, \delta) \geq \log(1/3\delta)$ .
B.4 TV characteristic time for Bernoulli instances: Proof of Proposition 1
Proof. Step 1: Let  $\nu$  be a bandit instance, i.e. such that  $\nu_{a} \triangleq \operatorname{Bernoulli}(\mu_{a})$  and  $\mu_{1} > \mu_{2} \geq \dots \geq \mu_{K}$ .
For the alternative bandit instance  $\lambda$ , we refer to the mean of arm  $a$  as  $\rho_{a}$ , i.e.  $\lambda_{a} \triangleq \operatorname{Bernoulli}(\rho_{a})$ .
By the definition of  $T_{\mathbf{TV}}^{\star}$ , we have that
$$
\begin{array}{l} \left(T _ {\mathbf {T V}} ^ {\star} (\boldsymbol {\nu})\right) ^ {- 1} = \sup  _ {\omega \in \Sigma_ {K}} \inf  _ {\boldsymbol {\lambda} \in \operatorname {A l t} (\boldsymbol {\nu})} \sum_ {a = 1} ^ {K} \omega_ {a} \operatorname {T V} \left(\nu_ {a} \parallel \lambda_ {a}\right) \\ \stackrel {(a)} {=} \sup  _ {\omega \in \Sigma_ {K}} \min  _ {a \neq 1} \inf  _ {\boldsymbol {\lambda}: \rho_ {a} > \rho_ {1}} \omega_ {1} | \mu_ {1} - \rho_ {1} | + \omega_ {a} | \mu_ {a} - \rho_ {a} | \\ \stackrel {(b)} {=} \sup  _ {\omega \in \Sigma_ {K}} \min  _ {a \neq 1} \min  (\omega_ {1}, \omega_ {a}) \Delta_ {a} \\ \stackrel {(c)} {=} \sup  _ {\omega \in \Sigma_ {K}} \omega_ {1} \min  _ {a \neq 1} \min  (1, \frac {\omega_ {a}}{\omega_ {1}}) \Delta_ {a} \\ \stackrel {(d)} {=} \sup  _ {(x _ {2}, \ldots , x _ {K}) \in (\mathbb {R} ^ {+}) ^ {K - 1}} \frac {\min  _ {a \neq 1} g _ {a} (x _ {a})}{1 + x _ {2} + \cdots + x _ {K}}, \\ \end{array}
$$
where  $g_{a}(x_{a})\triangleq \min (1,x_{a})\Delta_{a}$
Equality (a) is obtained due to the fact that  $\mathrm{Alt}(\pmb{\nu}) = \bigcup_{a \neq 1} \{\lambda : \rho_a > \rho_1\}$ , and for Bernoullis,  $\mathrm{TV}(\nu_a \parallel \lambda_a) = |\mu_a - \rho_a|$ .
Equality (b) is true, since  $\inf_{\lambda : \rho_a > \rho_1} \omega_1 |\mu_1 - \rho_1| + \omega_a |\mu_a - \rho_a| = \min(\omega_1, \omega_a) \Delta_a$ .
Equality (c) holds true, since  $\omega_{1} \neq 1$  (if  $\omega_{1} = 0$ , the value of the objective is 0).
Equality (d) is obtained by the change of variable  $x_{a} \triangleq \frac{\omega_{a}}{\omega_{1}}$
Step 2: Let  $(x_{2},\ldots ,x_{K})\in (\mathbb{R}^{+})^{K - 1}$ . By the definition of  $g_{a}$ , we have that
$$
g _ {a} \left(x _ {a}\right) \leq x _ {a} \Delta_ {a} \quad \text {a n d} \quad g _ {a} \left(x _ {a}\right) \leq \Delta_ {a}.
$$
This leads to the inequalities
$$
\min  _ {a \neq 1} g _ {a} (x _ {a}) \leq g _ {a} (x _ {a}) \leq x _ {a} \Delta_ {a} \quad \text {a n d} \quad \min  _ {a \neq 1} g _ {a} (x _ {a}) \leq \Delta_ {\min }.
$$
$$
\begin{array}{l} \left(\min  _ {a \neq 1} g _ {a} \left(x _ {a}\right)\right) \left(\frac {1}{\Delta_ {\min }} + \sum_ {a = 2} ^ {K} \frac {1}{\Delta_ {a}}\right) = \frac {\min  _ {a \neq 1} g _ {a} \left(x _ {a}\right)}{\Delta_ {\min }} + \sum_ {a = 2} ^ {K} \frac {\min  _ {a \neq 1} g _ {a} \left(x _ {a}\right)}{\Delta_ {a}} \\ \leq 1 + \sum_ {a = 2} ^ {K} x _ {a}. \\ \end{array}
$$
This means that for every  $(x_{2},\ldots ,x_{K})\in (\mathbb{R}^{+})^{K - 1}$
$$
\frac {\operatorname* {m i n} _ {a \neq 1} g _ {a} (x _ {a})}{1 + x _ {2} + \cdots + x _ {K}} \leq \frac {1}{\frac {1}{\Delta_ {\mathrm {m i n}}} + \sum_ {a = 2} ^ {K} \frac {1}{\Delta_ {a}}}.
$$
Here, the upper bound is achievable for  $x_{a}^{\star} = \frac{\Delta_{\min}}{\Delta_{a}}$ , since  $g_{a}(x_{a}^{\star}) = \Delta_{\min}$  for all  $a \neq 1$ .
This concludes that
$$
\left(T _ {\mathbf {T V}} ^ {\star} (\boldsymbol {\nu})\right) ^ {- 1} = \frac {1}{\frac {1}{\Delta_ {\min }} + \sum_ {a = 2} ^ {K} \frac {1}{\Delta_ {a}}} \quad \Longrightarrow \left(T _ {\mathbf {T V}} ^ {\star} (\boldsymbol {\nu})\right) = \Delta_ {\min } + \sum_ {a = 2} ^ {K} \frac {1}{\Delta_ {a}}.
$$
Step 3: The lower and upper bounds on  $(T_{\mathbf{TV}}^{\star}(\pmb {\nu}))$  follow from the fact that  $\frac{1}{\Delta_a}\geq 0$  for all  $a$ , and  $\frac{1}{\Delta_a}\leq \frac{1}{\Delta_{\mathrm{min}}}$  for all  $a\neq 1$ .
Hence, we conclude the proof.
B.5 On the total variation distance and the hardness of privacy
Our lower bound suggests that the hardness of the DP-FC-BAI problem is characterized by  $T_{TV}^{\star}$ , which is a total variation counterpart of the classic KL-based characteristic time  $T_{KL}^{\star}$  in FC-BAI [GK16]. The total variation distance appears to be the natural measure to quantify the hardness of privacy in other settings such as regret minimization [AB22], Karwa-Vadhan lemma [KV17] and Differentially Private Assouad, Fano, and Le Cam [ASZ21]. The high-level intuition is that: Pure DP can be seen as a multiplicative stability constraint of  $e^{\epsilon}$  when one data point changes. With group privacy, if two datasets differ in  $d_{ham}$  points, then one incurs a factor  $e^{d_{ham} \epsilon}$ . Now, by sampling  $n$  i.i.d points from a distribution  $P$  and  $n$  i.i.d points from a distribution  $Q$ , the Karwa-Vadhan lemma states that the incurred factor is  $e^{(nTV(P,Q)) \epsilon}$ . This is proved by building a maximal coupling, which is the coupling that minimizes the Hamming distance in expectation. In brief, the total variation naturally appears in lower bounds since it is the quantity that characterizes the hardness of the optimal transport problem minimizing the hamming distance, i.e  $TV(P,Q) = \inf_{(X,Y) \sim (P,Q)} E(1_{X \neq Y})$ . However, it is possible that the problem can be characterized by other f-divergences. Finally, one can always go from TV to KL using Pinsker's inequality, though that would always be less tight than the TV-based lower bound.
On the relation between  $T_{TV}^{\star}$  and  $T_{KL}^{\star}$  A direct application of Pinsker's inequality gives that  $T_{TV}^{\star}(\nu) \geq \sqrt{2T_{KL}^{\star}(\nu)}$ . For completeness, we present here the exact calculations:
For every alternative mean parameter  $\lambda$  and every arm  $a$ , using Pinkser's inequality, we have that  $d_{TV}(\mu_a,\lambda_a)\leq \sqrt{\frac{1}{2}d_{KL}(\mu_a,\lambda_a)}$ . Therefore, for every allocation over arms  $\omega$ , we have
$$
\sum_ {a} \omega_ {a} d _ {T V} (\mu_ {a}, \lambda_ {a}) \leq \sum_ {a} \omega_ {a} \sqrt {\frac {1}{2} d _ {K L} (\mu_ {a} , \lambda_ {a})} \leq \sqrt {\frac {1}{2} \sum_ {a} \omega_ {a} d _ {K L} (\mu_ {a} , \lambda_ {a})}.
$$
Taking the supremum over the simplex and the infimum over the set of alternative mean parameters yields  $(T_{TV}^{\star}(\nu))^{-1} \leq \sqrt{\frac{1}{2}(T_{KL}^{\star}(\nu))^{-1}}$ . This concludes the proof.
C Privacy analysis
In this section, we prove that AdaP-TT satisfies  $\epsilon$ -DP. We first provide the privacy lemma that justifies using doubling and forgetting in AdaP-TT. Using the privacy lemma and the post-processing property of DP, we conclude the privacy analysis of AdaP-TT.
C.1 Privacy lemma for non-overlapping sequences
Lemma 2 (Privacy of non-overlapping sequence of empirical means). Let  $\mathcal{M}$  be a mechanism that takes a set as input and outputs the private empirical mean, i.e.
$$
\mathcal {M} \left(\left\{r _ {i}, \dots , r _ {j} \right\}\right) \triangleq \frac {1}{j - i} \sum_ {t = i} ^ {j} r _ {t} + L a p \left(\frac {1}{(j - i) \epsilon}\right). \tag {7}
$$
Let  $\ell < T$  and  $t_1, \ldots, t_\ell, t_{\ell+1}$  be in  $[1, T]$  such that  $1 = t_1 < \dots < t_\ell < t_{\ell+1} - 1 = T$ . Let's define the following mechanism
$$
\mathcal {G}: \{r _ {1}, \dots , r _ {T} \} \rightarrow \bigotimes_ {i = 1} ^ {\ell} \mathcal {M} _ {\left\{r _ {t _ {i}}, \dots , r _ {t _ {i + 1} - 1} \right\}} \tag {8}
$$
In other words,  $\mathcal{G}$  is the mechanism we get by applying  $\mathcal{M}$  to the non-overlapping partition of the sequence  $\{r_1,\ldots ,r_T\}$  according to  $t_1 < \dots < t_\ell < t_{\ell +1}$ , i.e.
$$
\left( \begin{array}{c} r _ {1} \\ r _ {2} \\ \vdots \\ r _ {T} \end{array} \right) \xrightarrow {\mathcal {G}} \left( \begin{array}{c} \mu_ {1} \\ \vdots \\ \mu_ {\ell} \end{array} \right)
$$
where  $\mu_{i}\sim \mathcal{M}_{\{r_{t_i},\dots,r_{t_{i + 1} - 1}\}}$
For  $r_t \in [0,1]$ , the mechanism  $\mathcal{G}$  is  $\epsilon$ -DP.
Proof. Let  $r^T \triangleq (r_1, \ldots, r_T)$  and  $r'^T \triangleq (r'_1, \ldots, r'_T)$  be two neighbouring reward sequences in [0,1]. This implies that  $\exists j \in [1, T]$  such that  $r_j \neq r'_j$  and  $\forall t \neq j, r_t = r'_t$ .
Let  $\ell'$  be such that  $t_{\ell'} \leq j \leq t_{\ell'+1} - 1$ , and follows the convention that  $t_0 = 1$  and  $t_{\ell+1} = T + 1$ .
Let  $\mu \triangleq (\mu_1, \ldots, \mu_\ell)$  a fixed sequence of outcomes. Then,
$$
\frac {\mathbb {P} (\mathcal {G} (r ^ {T}) = \mu)}{\mathbb {P} (\mathcal {G} (r ^ {\prime T}) = \mu)} = \frac {\mathbb {P} \left(\mathcal {M} \left(\left\{r _ {t _ {\ell^ {\prime}}}, \dots , r _ {t _ {\ell^ {\prime} + 1} - 1} \right\}\right) = \mu_ {\ell^ {\prime}}\right)}{\mathbb {P} \left(\mathcal {M} \left(\left\{r _ {t _ {\ell^ {\prime}}}, \dots , r _ {t _ {\ell^ {\prime} + 1} - 1} \right\}\right) = \mu_ {\ell^ {\prime}}\right)} \leq e ^ {\epsilon},
$$
where the last inequality holds true because  $\mathcal{M}$  satisfies  $\epsilon$ -DP following Theorem 1.
C.2 Privacy analysis of AdaP-TT: Proof of Theorem 3
Theorem 3 (Privacy analysis). For rewards in  $[0,1]$ , AdaP-TT satisfies  $\epsilon$ -global  $DP$ .
Proof. Let  $T \geq 1$ . Let  $\mathbf{d}^T = \{\mathbf{x}_1, \dots, \mathbf{x}_T\}$  and  $\underline{\mathbf{d}}'^T = \{\mathbf{d}_1', \dots, \mathbf{d}_T'\}$  two neighboring reward tables in  $(\mathbb{R}^K)^T$ . Let  $j \in [1, T]$  such that, for all  $t \neq j$ ,  $d_t = d_t'$ .