SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Upper-lower bound discussion for the two privacy regimes. In the low privacy regime, our upper bound retrieves  $T_{\mathrm{KL},1/2}^{\star}(\mu)$  for Gaussian distributions. Since the rewards in the analysis are supposed Bernoulli, the mismatch from exact optimality is coming from the mismatch between the KL divergence of Bernoulli distributions and that of Gaussian, which is generally controllable in most instances where the means are far from the borders, i.e., 0, and 1. This is in essence, similar to the mismatch between UCB and KL-UCB in the regret-minimization literature (Chapter 10 in [LS20]). To overcome this mismatch, it is necessary to adapt the transportation costs to the family of distributions considered. In our setting, this can be done by using the Bernoulli KL rather than Gaussian KL in lines 12 and 16 of the algorithm. While the Top Two algorithms for Bernoulli distributions have been studied in [JDB+22], the analysis is more involved. Therefore, it would obfuscate where and how privacy is impacting the expected sample complexity.
In the high privacy regime, and for instances verifying Condition 1, our upper bound matches the lower bound  $\epsilon^{-1}T_{\mathrm{TV}}^{\star}(\pmb {\nu})$  up to a constant. Having matching upper and lower bounds only for high privacy regimes is an interesting phenomenon that appears in different settings of differential privacy literature, such as regret minimization [AB22], parameter estimation [CWZ21] and hidden probabilistic graphical models [NKS19]. We speculate two facets of this phenomenon:
1. Explicit bounds in high and low privacy regimes: We have matching bounds only in the high or low privacy regimes because the lower bounds are generally harder to explicit and understand in transitional phases. Thus, it is harder to claim optimality in those phases.
2. Information-theoretic roots: There might be a more profound information-theoretic reason in relation to [NKS19]. Indeed, there seems to be a link between the privacy budget  $\epsilon$  and the information thresholds introduced in [NKS19]. Specifically, if the randomized mapping  $\mathcal{F}$  in [3] satisfies DP, then the noisy information threshold and noiseless information threshold can be similarly written as a function of  $\epsilon$  and the total variation. Finding a rigorous link between these two quantities is an interesting question to explore.
E.7 Limitation and open problem
In the previous section (Appendix E.6), we argue that the upper and lower bounds match up to multiplicative constants in both privacy regimes, provided we restrict ourselves to instances satisfying Condition 1 (i.e. there exists  $C \geq 1$  such that  $\Delta_{\max} \leq C \Delta_{\min}$ ).
While this holds for numerous instances, it does not account for instances in which the gaps have different orders of magnitude. One example would be the regime where  $\Delta_{\mathrm{min}} \to 0$  while  $\Delta_{\mathrm{max}}$  is fixed, hence yielding  $T_{\mathrm{KL}}^{\star}(\pmb{\mu}) \to +\infty$ .
In Appendix E.5 (see Lemma 20 and 21), we show that combining the private GLR stopping rule (Equation (16)) with any BAI algorithms whose empirical proportions converge towards the  $\beta$ -optimal allocation  $\omega_{\mathrm{KL},\beta}^{\star}(\pmb{\mu})$  for Gaussian bandits will incur a problem dependent multiplicative cost
$$
1 + \sqrt {1 + \frac {\Delta_ {\operatorname* {m a x}} ^ {2}}{2 \epsilon^ {2}}} \tag {32}
$$
in addition to the multiplicative four-factor due to doubling and forgetting.
In order to have matching upper and lower bounds, we would need to have  $\Delta_{\mathrm{min}}$  instead of  $\Delta_{\mathrm{max}}$  in Equation (32). Unfortunately, our results show that it is not possible when using the private GLR stopping rule (Equation (16)) and a sampling rule that is tailored to asymptotic ( $\beta$ -)optimality for Gaussian bandits. Therefore, this impossibility result holds for a large class of BAI sampling rules when adapting them to tackle private BAI by using the private GLR stopping rule (Equation (16)).
Origin of this limitation. The term  $\Delta_{\mathrm{max}}$  appears due to the private stopping threshold that ensures  $\delta$ -correctness of the private GLR stopping rule (Equation (16)).
For asymptotically  $(\beta)$ -optimal algorithms, the additive term due to privacy in the threshold is of the order
$$
\frac {1}{\epsilon^ {2}} \left(\frac {1}{N _ {n , a}} + \frac {1}{N _ {n , a ^ {\star}}}\right) \log (1 / \delta) ^ {2} \approx_ {n \to + \infty} \frac {T _ {\mathrm {K L}} ^ {\star} (\pmb {\mu})}{n} \frac {\Delta_ {a} ^ {2}}{2 \epsilon^ {2}} \log (1 / \delta) ^ {2}.
$$
Therefore, the private GLR stopping rule (Equation (16)) will stop when, for all  $a \neq a^{\star}$ ,
$$
\frac {n}{T _ {\mathrm {K L}} ^ {\star} (\pmb {\mu})} \geq 2 \log (1 / \delta) + \frac {T _ {\mathrm {K L}} ^ {\star} (\pmb {\mu})}{n} \frac {\Delta_ {a} ^ {2}}{2 \epsilon^ {2}} \log (1 / \delta) ^ {2},
$$
which yields the problem-dependent multiplicative cost  $1 + \sqrt{1 + \frac{\Delta_{\max}^2}{2\epsilon^2}}$
Open problem. This impossibility result is specific to the way we derive an  $\epsilon$ -DP version of the GLR stopping rule. Therefore, a natural question is whether it is possible to derive a better private GLR stopping rule to match the lower bound for all Bernoulli instances.
A two-phase algorithm. AdaP-TT tracks the non-private lower bound (i.e.  $T_{KL}^{\star}$ ) as a non-private algorithm would do, and the additional cost tracking that privately is shown to be  $T_{TV}^{\star} / \epsilon$  (up to constants for bandits verifying Condition 1), where the additional cost comes from the added Laplace noise. Another approach would be to first perform a test to determine in which privacy regime the policy resides, and then track (privately) the corresponding characteristic time. Such an algorithm would empirically estimate both  $T_{TV}^{\star}$  and  $T_{KL}^{\star}$  in the first phase. If the privacy budget  $\epsilon$  is bigger than the empirical estimate of  $T_{TV}^{\star} / T_{KL}^{\star}$ , then this means that we are in the low privacy regime and the algorithm tracks (privately) the KL characteristic time in the second phase (as AdaP-TT does). However, if the privacy budget  $\epsilon$  is smaller than the empirical estimate of  $T_{TV}^{\star} / T_{KL}^{\star}$ , then it is the high privacy regime and the algorithm tracks (privately) the TV characteristic time in the second phase. For such a two-phase algorithm to achieve optimality, properly tuning the amount of time spent in its first phase is a crucial step. Whether it is possible to analyze such an algorithm and quantify the proper tuning (if it even exists) is an interesting direction for future work.
Non-asymptotic sample complexity of AdaP-TT. In the non-private FC-BAI literature (i.e.  $\epsilon = +\infty$ ), there is no tight lower bound in the non-asymptotic regime (i.e. for any value of  $\delta$ ). This is the main open problem in FC-BAI, and hence in DP-FC-BAI too. In the class of asymptotically ( $\beta$ -)optimal algorithms, TTUCB [JD22] is one of the few to have non-asymptotic guarantees. Adapting
the non-asymptotic analysis of [JD22] to the private AdaP-TT is an interesting direction for research. We conjecture that such an adaptation is possible (up to technicalities) by adding concentration terms linked to Laplace distribution and losing at least a multiplicative four-factor compared to TTUCB due to doubling and forgetting.
E.8 Analysis of non-private AdaP-TT: Proof of Theorem 6
The proofs detailed in Appendices E.2 and E.3 can easily be adapted to provide guarantees on the non-private AdaP-TT, which relies on the non-private leader/challenger defined in Equation (20) and Equation (21).
The key difference between the non-private AdaP-TT and the private AdaP-TT lies in the definition of the stopping threshold. While the private GLR stopping rule (Equation (16)) has an additive terms in  $\mathcal{O}(\log (1 / \delta)^2 /\epsilon)$  to cope for the uncertainty due to the Laplace noise, the non-private GLR stopping rule (Equation (13)) scales simply as  $\log (1 / \delta)$ . This has drastic consequences in terms of asymptotic upper bound on the expected sample complexity.
It is direct to see that most manipulations from Appendix E.5 still hold for the non-private AdaP-TT. Therefore, we use the notations and conditions defined therein. We show that the non-private GLR stopping rule (Equation (13)) yields
$$
\begin{array}{l} \min \left\{\tau_ {\delta}, T \right\} - \kappa T \\ \leq \sum_ {T \geq T _ {k _ {n}} ^ {+} \geq \kappa T} (T _ {k _ {n} + 2} ^ {+} - T _ {k _ {n} + 1} ^ {+}) \mathbb {1} \left(\tau_ {\delta} > T _ {k _ {n} + 1} ^ {+}\right) \\ \leq \sum_ {T \geq T _ {k _ {n}} ^ {+} \geq \kappa T} (T _ {k _ {n} + 2} ^ {+} - T _ {k _ {n} + 1} ^ {+}) \mathbb {1} \left(\exists a \neq a ^ {\star}, \frac {(\hat {\mu} _ {k _ {n , a ^ {\star}} + 1 , a ^ {\star}} - \hat {\mu} _ {k _ {n , a} + 1 , a}) ^ {2}}{1 / \tilde {N} _ {k _ {n , a ^ {\star}} + 1 , a ^ {\star}} + 1 / \tilde {N} _ {k _ {n , a} + 1 , a}} \right. \\ <   2 c _ {\left(k _ {n, a ^ {*}} + 1\right) \left(k _ {n, a} + 1\right)} \left(\tilde {N} _ {k _ {n, a ^ {*}} + 1, a ^ {*}}, \tilde {N} _ {k _ {n, a} + 1, a}, \delta\right) \\ \leq \sum_ {T \geq T _ {k _ {n}} ^ {+} \geq \kappa T} \left(T _ {k _ {n} + 2} ^ {+} - T _ {k _ {n} + 1} ^ {+}\right) \mathbb {1} \left(\left(T _ {k _ {n}} ^ {-} - 1\right) \frac {1 - \zeta}{T _ {\mathrm {K L} , \beta} ^ {\star} (\boldsymbol {\mu})} <   4 \log (4 + \log T _ {k _ {n}} ^ {-} + \log (2 + \eta)) \right. \\ \left. + 2 \mathcal {C} _ {G} \left(\log (1 / \delta) / 2 + s \log (2 + \log_ {2} T _ {k _ {n}} ^ {-} + \log_ {2} (2 + \eta)) + \log ((K - 1) \zeta (s) ^ {2}) / 2\right)\right), \\ \end{array}
$$
$$
\lim _ {\delta \to 0} \frac {\mathbb {E} _ {\pmb {\nu}} [ \tau_ {\delta} ]}{\log (1 / \delta)} \leq \lim _ {\delta \to 0} \frac {\mathbb {E} _ {\pmb {\nu}} [ T _ {k _ {\delta} + 2} ^ {+} ]}{\log (1 / \delta)} \leq (2 + \eta) ^ {2} \lim _ {\delta \to 0} \frac {\mathbb {E} _ {\pmb {\nu}} [ T _ {k _ {\delta} + 1} ^ {+} ]}{\log (1 / \delta)} \leq (2 + \eta) ^ {2} \lim _ {\delta \to 0} \frac {T _ {\zeta} (\delta)}{\log (1 / \delta)},
$$
First-order terms. Since we are only interested in upper bounding  $\lim_{\delta \to 0} \frac{T_{\zeta}(\delta)}{\log(1 / \delta)}$ , we can safely drop the second orders terms in  $T$  and  $\log(1 / \delta)$ . This allows us to remove the terms in  $\mathcal{O}(\log \log T)$  and in  $\mathcal{O}(\log \log (1 / \delta))$ . Using that  $\mathcal{C}_G(x) = x + \mathcal{O}(\log x)$ , tedious manipulations yield that
$$
\limsup_{\delta \to 0}\frac{T_{\zeta}(\delta)}{\log(1 / \delta)}\leq \frac{T_{\mathrm{KL},\beta}^{\star}(\boldsymbol {\mu})}{(1 - \kappa)(1 - \zeta)}.
$$
$$
\limsup_{\delta \to 0}\frac{\mathbb{E}_{\boldsymbol{\nu}}\left[ \tau_{\delta}\right]}{\log(1 / \delta)}\leq 4T_{\mathrm{KL},\beta}^{\star}(\boldsymbol {\mu}).
$$
F Extended experimental analysis
In this section, we perform additional experiments to compare AdaP-TT and DP-SE for FC-BAI. We test the two algorithms in six bandit environments with Bernoulli distributions, as defined by [SS19], namely
$$
\mu_ {1} = (0. 9 5, 0. 9, 0. 9, 0. 9, 0. 5), \quad \mu_ {2} = (0. 7 5, 0. 7, 0. 7, 0. 7, 0. 7),
$$
$$
\mu_ {3} = (0, 0. 2 5, 0. 5, 0. 7 5, 1), \quad \mu_ {4} = (0. 7 5, 0. 6 2 5, 0. 5, 0. 3 7 5, 0. 2 5) \},
$$
$$
\mu_ {5} = (0. 7 5, 0. 5 3 1 2 5, 0. 3 7 5, 0. 2 8 1 2 5, 0. 2 5), \quad \mu_ {6} = (0. 7 5, 0. 7 1 8 7 5, 0. 6 2 5, 0. 4 6 8 7 5, 0. 2 5) \}.
$$
For each Bernoulli instance, we implement the algorithms with
$$
\epsilon \in \{0. 0 0 1, 0. 0 0 5, 0. 0 1, 0. 0 5, 0. 1, 0. 2, 0. 3, 0. 4, 0. 5, 0. 6, 0. 7, 0. 8, 0. 9, 1, 1 0 \},
$$
and a risk level  $\delta = 0.01$ . We verify empirically that the algorithms are  $\delta$ -correct by running each algorithm 100 times. In Figure 2, we plot the evolution of the average stopping time and standard deviation with respect to the privacy budget  $\epsilon$ . All the algorithms are implemented in Python (version 3.8) and are tested with an 8-core 64-bits Intel i5@1.6 GHz CPU.
All the experiments validate the same conclusions as the ones reached in Section 5, i.e.
1. AdaP-TT requires fewer samples to provide a  $\delta$ -correct answer,
2. there exists two privacy regimes, and in the low-privacy regime, the sample complexity is independent of the privacy budget.
(a)  $\mu_{1}$
(b)  $\mu_{2}$
(c)  $\mu_{3}$
(d)  $\mu_4$
(e)  $\mu_{5}$
(f)  $\mu_6$
Figure 2: Evolution of the stopping time  $\tau$  (mean  $\pm$  std. over 100 runs) of AdaP-TT, DP-SE, and TTUCB with respect to the privacy budget  $\epsilon$  for  $\delta = 10^{-2}$  on different Bernoulli instances. The shaded vertical line separates the two privacy regimes. AdaP-TT outperforms DP-SE.