Title: Optimal Regret of Bandits under Differential Privacy
Venue: NeurIPS 2025
Accepted: yes
Keywords: Differential Privacy

Multi-armed Bandits

Regret Analysis

Lower Bound
Abstract: As sequential learning algorithms are increasingly applied to real life, ensuring data privacy while maintaining their utilities emerges as a timely question. In this context, regret minimisation in stochastic bandits under $\epsilon$-global Differential Privacy (DP) has been widely studied. 
The present literature poses a significant gap between the best-known regret lower and upper bound in this setting, though they ``match in order''.
Thus, we revisit the regret lower and upper bounds of $\epsilon$-global DP bandits and improve both. 
First, we prove a tighter regret lower bound involving a novel information-theoretic quantity characterising the hardness of $\epsilon$-global DP in stochastic bandits. This quantity smoothly interpolates between Kullback–Leibler divergence and Total Variation distance, depending on the privacy budget $\epsilon$.
Then, we choose two asymptotically optimal bandit algorithms, i.e., KL-UCB and IMED, and propose their DP versions using a unified blueprint, i.e., (a) running in arm-dependent phases, and (b) adding Laplace noise to achieve privacy. For Bernoulli bandits, we analyse the regrets of these algorithms and show that their regrets asymptotically match our lower bound up to a constant arbitrary close to 1.
At the core of our algorithms lies a new concentration inequality for sums of Bernoulli variables under Laplace mechanism, which is a new DP version of the Chernoff bound.
Finally, our numerical experiments validate that DP-KLUCB and DP-IMED achieve lower regret than the existing $\epsilon$-global DP bandit algorithms.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates regret minimization in Bernoulli bandits under $\\epsilon$-global Differential Privacy (DP). The authors first define DP in the context of bandit settings, formalizing privacy constraints for sequential decision-making. They then derive a tighter regret lower bound for the general problem. Next, they develop a new concentration inequality for sums of Bernoulli random variables with added Laplace noise, serving as a DP analog of the Chernoff bound. Based on this, they propose two algorithms, DP-KLUCB and DP-IMED, which operate in arm-dependent phases with geometrically increasing batch sizes. Finally, the authors prove the DP properties of these algorithms and establish regret upper bounds that asymptotically match the lower bound up to a constant arbitrarily close to 1, with numerical experiments validating their superior performance.\n\nSTRENGTHS AND WEAKNESSES:\n(a) Quality:  The technical contributions of this paper include the proposed algorithms and the proofs for both the lower and upper regret bounds. The lower bound proof is particularly compelling, introducing a novel term (\\mathrm{d}_\\epsilon(\\mu_a, \\mu^*)) and employing a double change of environment technique to achieve a tighter bound. However, the algorithms appear as straightforward extensions of existing methods like , DP-KLUCB and DP-IMED,, lacking significant innovation in their formulation. Similarly, the upper bound proof adheres to standard methodologies in differentially private bandit literature, offering no novel technical insights.\n\n\n(b) Clarity: The paper is clearly written and well-organized, with a logical progression from motivation to empirical validation. However, it assumes substantial prior knowledge of differential privacy and bandit algorithms, such as KL-UCB and IMED, which may hinder accessibility for readers less familiar with these areas. To improve clarity, the authors could provide additional background on these concepts, perhaps in the introduction or an appendix, to broaden the paper’s accessibility.\n\n(c) Significance： This paper addresses regret minimization in differentially private bandits, building on prior work such as Azize and Basu [2022], and achieves a tighter lower bound, enhancing theoretical understanding. However, the theorem’s restriction to Bernoulli bandits limits its applicability to other reward distributions, potentially reducing its impact on the broader field. The algorithms’ primary novelty lies in using rewards from previous phases for parameter estimation, but the resulting improvement in regret performance is marginal, further constraining the paper’s practical significance.\n\n\n(d) Originality: The paper contributes a tighter lower bound, deepening the understanding of the current differentially private bandit model, and clearly distinguishes its contributions from prior work with appropriate citations. However, the originality of the algorithms is limited, as their design heavily relies on existing frameworks like DP-KLUCB, with minimal novel components beyond the use of historical rewards.\n\nQUESTIONS:\n(a) The current results, particularly the regret lower bound and algorithms, are specific to Bernoulli bandits. Could the authors discuss the potential for extending these results to other reward distributions, such as sub-Gaussian or heavy-tailed distributions? What specific challenges do they anticipate in such extensions, and are there any preliminary ideas or approaches to address these challenges? Additionally, can the strategy of using rewards from previous phases be applied to bandits with other distributions?\n\n(b) The design of the algorithms heavily relies on Proposition 1, a new concentration inequality. Could the authors provide intuition behind this proposition, explaining how it facilitates differentially private bandit algorithms? Additionally, how did the concentration inequality from Azize and Basu [2022] inspire the development of this new, tighter inequality?\n\n(c) The introduction highlights adaptive clinical trials as a motivating example. Could the authors provide a detailed description of how the proposed algorithms, such as DP-KLUCB and DP-IMED, could be applied in this context? Specifically, how do these algorithms ensure patient privacy while optimizing treatment allocation?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies the $K$-armed bandit problem under $\\epsilon$-global differential privacy (DP) and establishes matching lower and upper regret bounds up to a constant $\\alpha$ that can be made arbitrarily close to one. The contributions include: a tighter lower bound (Theorem 1) based on a new divergence that interpolates between KL-divergence and total variation distance; an improved concentration inequality (Proposition 1); and differentially private variants of two asymptotically optimal bandit algorithms (KL-UCB and IMED), which are shown to match the regret lower bound. The paper also includes experimental results comparing the performance of the proposed algorithms to prior methods (DP-SE, AdaP-KLUCB, Lazy-DP-TS); see Figure 1.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper is well written.\n\nIt closes a gap in the literature regarding the performance of differentially private bandit algorithms. In particular, Table 1 reports the lower and upper bounds from Azize and Basu [2022], which exhibit a constant gap (assuming the KL divergence is within constant factors of the squared reward gap). This paper improves that gap to an arbitrarily close-to-1 constant by introducing a new information-theoretic quantity that interpolates between KL divergence and total variation distance. See Theorem 1, Corollary 1, and Theorem 2. The lower bound is proven using standard change-of-measure techniques from bandit theory (e.g., Lai & Robbins, 1985). While I did not go through all the proofs in full detail, the results appear sound and consistent with prior techniques. The regret analysis for the proposed algorithms builds on the corresponding non-private analyses, leveraging the new concentration inequality in Proposition 1.\n\nA limitation is that the results apply only to Bernoulli reward distributions, which restricts the generality of the findings. While the technical content is solid and compelling, the paper addresses a rather niche problem (closing the constant gap between lower and upper bounds for differentially private multi-armed bandits) in a narrow setting (Bernoulli rewards). The limited scope of the contribution is my main reservation about the paper.\n\nMinor comment:\nTypo on line 85: “uppe bound” → “upper bound”\n\nQUESTIONS:\n1. What is the limitation in extending the concentration inequality for Bernoulli plus Laplace noise to more general reward distributions beyond Bernoulli?\n\n2. Shouldn’t the limitation to Bernoulli reward distributions be reflected in the title of the paper?\n\n3. Is it correct to say that the analysis of DP-KLUCB and DP-IMED follows similar lines as the analyses of their non-private variations, with the key difference being the use of the concentration inequality in Proposition 1?\n\n4. In the definition of $d_\\epsilon(x, y)$ (Eq. (6)), the infimum is taken over $z \\in [x \\wedge y, x \\vee y]$. Is this infimum always attained for Bernoulli distributions, and how computationally efficient is it to evaluate this in practice when applying the algorithm?\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

comments: SUMMARY:
This paper studies differentially private (DP) multi-armed bandits. It first provides an improved lower bound of the regret of bandits algorithms under the constraint of differential privacy. Then, it proposes two algorithms for DP bandits, and proves that their regret bounds match with the lower bound up to a constant factor. Finally, the synthetic experiments validates the theoretical results.

STRENGTHS AND WEAKNESSES:
Strengths:
1. The theoretical results seem solid and important. Both lower and upper bounds are provided and they are matching up to a constant factor.
2. The presentation is clear because it lists the key improvements in the proofs.

Weaknesses:
1. The setting is limited to multi-armed bandits and Bernoulli distribution, while linear or contextual bandits are more general models. In particular, Proposition 1 is limited to Bernoulli distribution. It is unclear that Gaussian or sub-Gaussian distribution can also have similar results so that the algorithm is still optimal.
2. The experiments could be benefit from some real-world dataset.

QUESTIONS:
1. Why the authors would like to propose two algorithms? Any specific reasons, for example, their design ideas are fundamentally different?

LIMITATIONS:
Yes.
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the classical DP MAB problem, with an objective of closing the instance-dependent upper and lower bound that existed in previous works, namely $\\Delta_a^{-2}$ v.s. $\\text{kl}(\\mu_a,\\mu_\\ast)^{-1}$. The authors proposed a new measure $d_\\epsilon$ such that 1) a new lower bound of $d_\\epsilon^{-1}$ is given, and 2) a new algorithm achieving $d_\\epsilon^{-1}$ is also yielded.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n1. The exact match of two $d_\\epsilon^{-1}$ is exciting. Previously the upper and lower bounds only agree in an asymptotical setting (but now still this case due to $\\alpha$; nevertheless the $\\alpha$ here can be arbitrarily close to $1^+$ in contrast to the previous instance-dependent approximation).\n2. The algorithm design illustrates that it is possible to get rid of \"forgetting\", thus offering more algorithmic tools in the future. However, I have to admit that I'm not super familiar with DP literature, and therefore I am not super certain whether this innovation is significant enough.\n3. Numerical illustraions are provided, which supports the superiority of proposed algorithms.\n\nWeakness:\n1. The match between upper and lower bounds only holds in asymptotic settings where one can reduce $\\alpha\\to 1^+$ as much as they want. \n2. It is unclear how much improvement is really yielded by the new $d_\\epsilon$ measure, given that $\\text{kl}$ and $\\Delta_a^{-2}$ are already approx. equal in asymp. cases.\n\nOverall, I feel the improvements may look kind of \"marginal\" given the previously already-good (but non-perfect) upper and lower bounds. However, given the perfect matching coefficient (if I did not misunderstand), I feel this \"perfect\" result deserves to be presented.\n\nQUESTIONS:\nLine 95--96: \"However, this approximation can be arbitrarily bad, exposing a gap between the state-of-the-art upper and lower bounds in DP bandits.\" What does \"arbitrarily bad\" exactly mean?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}