Original: When Privacy Meets Partial Information: A Refined Analysis of Differentially Private Bandits
Revised: When Privacy Meets Partial Information: A Refined Analysis of Differentially Private Bandits

Original: Abstract
Revised: Abstract

Original: We study the problem of multi-armed bandits with ϵ -global Differential Privacy (DP).
Revised: We study the problem of multi-armed bandits with ϵ -global Differential Privacy (DP).

Original: First, we prove the minimax and problem-dependent regret lower bounds for stochastic and linear bandits that quantify the hardness of bandits with ϵ -global
Revised: First, we prove the minimax and problem-dependent regret lower bounds for stochastic and linear bandits that quantify the hardness of bandits with ϵ -global

Original: DP.
Revised: DP.

Original: These bounds suggest the existence of two hardness regimes depending on the privacy budget ϵ .
Revised: These bounds suggest the existence of two hardness regimes depending on the privacy budget ϵ .

Original: In the high-privacy regime (small ϵ ), the hardness depends on a coupled effect of privacy and partial information about the reward distributions.
Revised: In the high-privacy regime (small ϵ ), the hardness depends on a coupled effect of privacy and partial information about the reward distributions.

Original: In the low-privacy regime (large ϵ ), bandits with ϵ -global DP are not harder than the bandits without privacy.
Revised: In the low-privacy regime (large ϵ ), bandits with ϵ -global DP are not harder than the bandits without privacy.

Original: For stochastic bandits, we further propose a generic framework to design a near-optimal ϵ -global DP extension of an index-based optimistic bandit algorithm .
Revised: For stochastic bandits, we further propose a generic framework to design a near-optimal ϵ -global DP extension of an index-based optimistic bandit algorithm .

Original: The framework consists of three ingredients: the
Revised: The framework consists of three ingredients: the

Original: Laplace mechanism, arm-dependent adaptive episodes, and usage of only the rewards collected in the last episode for computing private statistics.
Revised: Laplace mechanism, arm-dependent adaptive episodes, and usage of only the rewards collected in the last episode for computing private statistics.

Original: Specifically, we instantiate ϵ -global DP extensions of UCB and KL-UCB algorithms, namely
Revised: Specifically, we instantiate ϵ -global DP extensions of UCB and KL-UCB algorithms, namely

Original: AdaP - UCB and AdaP - KLUCB .
Revised: AdaP - UCB and AdaP - KLUCB .

Original: AdaP - KLUCB is the first algorithm that both satisfies ϵ -global DP and yields a regret upper bound that matches the problemdependent lower bound up to multiplicative constants.
Revised: AdaP - KLUCB is the first algorithm that both satisfies ϵ -global DP and yields a regret upper bound that matches the problemdependent lower bound up to multiplicative constants.

Original:  Introduction
Revised:  Introduction

Original: Multi-armed bandit problems, in short bandits , are a model for sequential decision-making with partial information (Lattimore and Szepesvári, 2018).
Revised: Multi-armed bandit problems, in short bandits , are a model for sequential decision-making with partial information (Lattimore and Szepesvári, 2018).

Original: In bandits, a learner sequentially interacts with an environment, which is a set of unknown distributions (or arms or actions), over T ∈ N steps.
Revised: In bandits, a learner sequentially interacts with an environment, which is a set of unknown distributions (or arms or actions), over T ∈ N steps.

Original: T is referred to as the horizon.
Revised: T is referred to as the horizon.

Original: At each step t ∈ { 1 , . . .
Revised: At each step t ∈ { 1 , . . .

Original: , T } , the learner chooses an arm A t from { 1 , . .
Revised: , T } , the learner chooses an arm A t from { 1 , . .

Original: , K } and the environment reveals a reward r t from the distribution ν
Revised: , K } and the environment reveals a reward r t from the distribution ν

Original: A t .
Revised: A t .

Original: The learner’s objective is to maximise its cumulative reward (cid:80)
Revised: The learner’s objective is to maximise its cumulative reward (cid:80)

Original: Tt =1 r t .
Revised: Tt =1 r t .

Original: An equivalent performance metric for a bandit algorithm π is regret .
Revised: An equivalent performance metric for a bandit algorithm π is regret .

Original: Regret is the difference between the expected cumulative reward collected by pulling the optimal arm a ∗ for T times and the expected cumulative reward obtained by using π .
Revised: Regret is the difference between the expected cumulative reward collected by pulling the optimal arm a ∗ for T times and the expected cumulative reward obtained by using π .

Original: Regret is the price paid by the bandit algorithm due to partial information about the reward distributions.
Revised: Regret is the price paid by the bandit algorithm due to partial information about the reward distributions.

Original: The goal of a bandit algorithm is to minimise its regret .
Revised: The goal of a bandit algorithm is to minimise its regret .

Original: Bandits, introduced in (Thompson, 1933) for medical trials, are widely studied and deployed in real-life applications, such as online advertising (Schwartz et al., 2017), recommendation systems (Silva et al., 2022) and investment portfolio design (Silva et al., 2022).
Revised: Bandits, introduced in (Thompson, 1933) for medical trials, are widely studied and deployed in real-life applications, such as online advertising (Schwartz et al., 2017), recommendation systems (Silva et al., 2022) and investment portfolio design (Silva et al., 2022).

Original: Sensitive data of individuals, such as health conditions, personal preferences, financial status etc., are utilised in these applications, which raises the concern about privacy.
Revised: Sensitive data of individuals, such as health conditions, personal preferences, financial status etc., are utilised in these applications, which raises the concern about privacy.

Original: Example 1.
Revised: Example 1.

Original: In bandit-based recommendation systems, the learning algorithm selects an item for each user.
Revised: In bandit-based recommendation systems, the learning algorithm selects an item for each user.

Original: The user decides whether or not to click the recommendation.
Revised: The user decides whether or not to click the recommendation.

Original: Based on the click feedback, the bandit algorithm improves its subsequent recommendations.
Revised: Based on the click feedback, the bandit algorithm improves its subsequent recommendations.

Original: Here, an action corresponds to an 36th Conference on Neural Information Processing Systems (NeurIPS 2022).
Revised: Here, an action corresponds to an 36th Conference on Neural Information Processing Systems (NeurIPS 2022).

Original: item and a reward corresponds to whether a user clicks on the item ( 1 ) or not ( 0 ).
Revised: item and a reward corresponds to whether a user clicks on the item ( 1 ) or not ( 0 ).

Original: A change in a user’s preference prompts changes in the algorithm’s output.
Revised: A change in a user’s preference prompts changes in the algorithm’s output.

Original: As a result, even if the click feedback is kept private, the user’s private information, i.e. their preference over items, is revealed.
Revised: As a result, even if the click feedback is kept private, the user’s private information, i.e. their preference over items, is revealed.

Original: Privacy protection aims to prevent the algorithm’s output from revealing the preferences of any specific user.
Revised: Privacy protection aims to prevent the algorithm’s output from revealing the preferences of any specific user.

Original: This example demonstrates the need for privacy in bandits.
Revised: This example demonstrates the need for privacy in bandits.

Original: In this paper, we use Differential Privacy (DP) (Dwork and Roth, 2014) as the framework for privacy.
Revised: In this paper, we use Differential Privacy (DP) (Dwork and Roth, 2014) as the framework for privacy.

Original: DP ensures that an algorithm’s output is unaffected by changes in input at a single data point.
Revised: DP ensures that an algorithm’s output is unaffected by changes in input at a single data point.

Original: By limiting the amount of sensitive information that an adversary can deduce from the output, DP renders an individual corresponding to a data point ‘indistinguishable’.
Revised: By limiting the amount of sensitive information that an adversary can deduce from the output, DP renders an individual corresponding to a data point ‘indistinguishable’.

Original: A calibrated amount of noise is injected into an algorithm to ensure DP.
Revised: A calibrated amount of noise is injected into an algorithm to ensure DP.

Original: The noise scale is set to be proportional to the algorithm’s sensitivity and inversely proportional to the privacy budget ϵ .
Revised: The noise scale is set to be proportional to the algorithm’s sensitivity and inversely proportional to the privacy budget ϵ .

Original: To address the privacy issues in bandit applications, the problem of Differentially Private Bandits is coined and studied under different settings, such as stochastic bandits (Mishra and Thakurta, 2015;
Revised: To address the privacy issues in bandit applications, the problem of Differentially Private Bandits is coined and studied under different settings, such as stochastic bandits (Mishra and Thakurta, 2015;

Original: Tossou and Dimitrakakis, 2016; Sajed and Sheffet, 2019; Hu et al., 2021), adversarial bandits (Tossou and Dimitrakakis, 2017), and linear contextual bandits (Shariff and Sheffet, 2018; Neel and Roth, 2018).
Revised: Tossou and Dimitrakakis, 2016; Sajed and Sheffet, 2019; Hu et al., 2021), adversarial bandits (Tossou and Dimitrakakis, 2017), and linear contextual bandits (Shariff and Sheffet, 2018; Neel and Roth, 2018).

Original: Also, multiple adaptations of DP, namely local and global, are proposed for bandits (Basu et al., 2019).
Revised: Also, multiple adaptations of DP, namely local and global, are proposed for bandits (Basu et al., 2019).

Original: Local DP aims to preserve the privacy of the sequence of rewards obtained by sending noisy rewards to the algorithm (Duchi et al., 2013).
Revised: Local DP aims to preserve the privacy of the sequence of rewards obtained by sending noisy rewards to the algorithm (Duchi et al., 2013).

Original: Global DP allows the algorithm to access rewards without noise and aims to keep the sequence of rewards private while only the sequence of actions taken by the algorithm is produced to the public (Basu et al., 2019).
Revised: Global DP allows the algorithm to access rewards without noise and aims to keep the sequence of rewards private while only the sequence of actions taken by the algorithm is produced to the public (Basu et al., 2019).

Original: Though local DP provides stronger privacy as the data curator has no access to the original reward stream, it injects too much noise that leads to higher regret.
Revised: Though local DP provides stronger privacy as the data curator has no access to the original reward stream, it injects too much noise that leads to higher regret.

Original: Also, the fundamental hardness of local DP in bandits in terms of regret lower bound and also corresponding optimal algorithms are well-understood (Zheng et al., 2020).
Revised: Also, the fundamental hardness of local DP in bandits in terms of regret lower bound and also corresponding optimal algorithms are well-understood (Zheng et al., 2020).

Original: Thus, in this paper, we focus on the bandit problems with ϵ -global DP.
Revised: Thus, in this paper, we focus on the bandit problems with ϵ -global DP.

Original: We aim to address two questions:
Revised: We aim to address two questions:

Original: What is the fundamental hardness of differentially private bandits with global DP expressed in terms of the regret lower bound?
Revised: What is the fundamental hardness of differentially private bandits with global DP expressed in terms of the regret lower bound?

Original: How to design an algorithmic framework that converts an optimistic and near-optimal bandit algorithm into a near-optimal bandit algorithm satisfying global DP?
Revised: How to design an algorithmic framework that converts an optimistic and near-optimal bandit algorithm into a near-optimal bandit algorithm satisfying global DP?

Original: Our Contributions.
Revised: Our Contributions.

Original: These questions have led to the following contributions:
Revised: These questions have led to the following contributions:

Original: Hardness as Lower Bounds: We derive both the minimax (worst-case) and problem-dependent lower bounds on the regret of bandits with ϵ -global DP for stochastic bandits (Thm. 2 and 3, Sec. 3).
Revised: Hardness as Lower Bounds: We derive both the minimax (worst-case) and problem-dependent lower bounds on the regret of bandits with ϵ -global DP for stochastic bandits (Thm. 2 and 3, Sec. 3).

Original: Both the bounds show that the hardness depends on a trade-off between the privacy budget ϵ and the distinguishability gap of a bandit environment.
Revised: Both the bounds show that the hardness depends on a trade-off between the privacy budget ϵ and the distinguishability gap of a bandit environment.

Original: If the ϵ is bigger than the distinguishability gap, a bandit with global DP is not harder than a non-private bandit problem.
Revised: If the ϵ is bigger than the distinguishability gap, a bandit with global DP is not harder than a non-private bandit problem.

Original: Additionally, our problem-dependent regret lower bound (Thm. 3) provides a novel observation that the difficulty of a bandit problem with global DP depends on the TV-indistinguishability gap ( t inf , Thm. 3).
Revised: Additionally, our problem-dependent regret lower bound (Thm. 3) provides a novel observation that the difficulty of a bandit problem with global DP depends on the TV-indistinguishability gap ( t inf , Thm. 3).

Original: This was not known in the regret lower bound in (Shariff and Sheffet, 2018), where the privacy dependent term in regret, i.e. K log( T ) ϵ is independent of the hardness of the bandit instance.
Revised: This was not known in the regret lower bound in (Shariff and Sheffet, 2018), where the privacy dependent term in regret, i.e. K log( T ) ϵ is independent of the hardness of the bandit instance.

Original: Our lower bounds explicates this missing link between the interaction of privacy and partial information.
Revised: Our lower bounds explicates this missing link between the interaction of privacy and partial information.

Original: We also extend our techniques to derive the lower bounds for linear bandits with a finite number of arms (Thm. 4 and 5).
Revised: We also extend our techniques to derive the lower bounds for linear bandits with a finite number of arms (Thm. 4 and 5).

Original: These lower bounds also reflect the same transition of hardness
Revised: These lower bounds also reflect the same transition of hardness

Original: depending on
Revised: depending on

Original: ϵ
Revised: ϵ

Original: and
Revised: and

Original: distinguishability
Revised: distinguishability

Original: gaps.
Revised: gaps.

Original: Algorithm Design: Optimistic bandit algorithms used the empirical mean and variance of observed rewards to compute indexes and use them to select an action.
Revised: Algorithm Design: Optimistic bandit algorithms used the empirical mean and variance of observed rewards to compute indexes and use them to select an action.

Original: We propose three fundamental strategies to design a private bandit algorithm from an optimistic algorithm (Sec. 4)– a) add Laplacian noise to the empirical mean of each arm calibrated by the corresponding sensitivity, b) use adaptive episodes to compute the private empirical mean less number of times, and c) use only the
Revised: We propose three fundamental strategies to design a private bandit algorithm from an optimistic algorithm (Sec. 4)– a) add Laplacian noise to the empirical mean of each arm calibrated by the corresponding sensitivity, b) use adaptive episodes to compute the private empirical mean less number of times, and c) use only the

Original: observed rewards of the arm’s last active episode 1 and forget everything before to keep the sensitivity of the empirical mean low.
Revised: observed rewards of the arm’s last active episode 1 and forget everything before to keep the sensitivity of the empirical mean low.

Original: We deploy these techniques with UCB (Auer et al., 2002) and KL - UCB (Garivier and
Revised: We deploy these techniques with UCB (Auer et al., 2002) and KL - UCB (Garivier and

Original: Cappé, 2011) algorithms for non-private bandits to propose two near-optimal and ϵ -global DP bandit algorithms, AdaP - UCB and AdaP - KLUCB (Sec. 4).
Revised: Cappé, 2011) algorithms for non-private bandits to propose two near-optimal and ϵ -global DP bandit algorithms, AdaP - UCB and AdaP - KLUCB (Sec. 4).

Original: Both of these algorithms achieve near-optimal problem-dependent regret and also reflect a transition in hardness from a low to a high privacy regime
Revised: Both of these algorithms achieve near-optimal problem-dependent regret and also reflect a transition in hardness from a low to a high privacy regime

Original: (Thm. 7 and 8).
Revised: (Thm. 7 and 8).

Original: Both theoretical (Table 1) and experimental (Sec. 5) results demonstrate optimality of AdaP - KLUCB than existing algorithms.
Revised: Both theoretical (Table 1) and experimental (Sec. 5) results demonstrate optimality of AdaP - KLUCB than existing algorithms.

Original: Technical Tools: (a) To derive the lower bound, we extend the Karwa-Vadhan lemma (Lemma 6.1, (Karwa and Vadhan, 2017)) to the sequential setting (Lemma 2).
Revised: Technical Tools: (a) To derive the lower bound, we extend the Karwa-Vadhan lemma (Lemma 6.1, (Karwa and Vadhan, 2017)) to the sequential setting (Lemma 2).

Original: (b) We present a novel sequential information processing lemma under ϵ -global DP (Thm. 10) that controls the difference between the outcome streams of a differentially private policy when interacting with two different bandit instances.
Revised: (b) We present a novel sequential information processing lemma under ϵ -global DP (Thm. 10) that controls the difference between the outcome streams of a differentially private policy when interacting with two different bandit instances.

Original: (c) This leads to a generic proof structure, utilised to generate refined regret lower bounds under ϵ -global DP for different settings.
Revised: (c) This leads to a generic proof structure, utilised to generate refined regret lower bounds under ϵ -global DP for different settings.

Original: (d) To derive the regret upper bound, we develop a novel and general analysis of optimistic algorithms with adaptive episodes.
Revised: (d) To derive the regret upper bound, we develop a novel and general analysis of optimistic algorithms with adaptive episodes.

Original: Our regret lower and upper bounds close the open question posed in (Tenenbaum et al., 2021), i.e. the problem-dependent regret bounds of differentially private bandits should depend on the
Revised: Our regret lower and upper bounds close the open question posed in (Tenenbaum et al., 2021), i.e. the problem-dependent regret bounds of differentially private bandits should depend on the

Original: KL-divergence between the reward distributions.
Revised: KL-divergence between the reward distributions.

Original: Related Works: Lower Bound.
Revised: Related Works: Lower Bound.

Original: (Shariff and Sheffet, 2018) first proposed a problem-dependent lower bound on regret, Ω(max { (cid:80) a ̸ = a ∗ log T ∆ a , K log( T ) ϵ } ) 2 , for stochastic bandits with ϵ -global DP.
Revised: (Shariff and Sheffet, 2018) first proposed a problem-dependent lower bound on regret, Ω(max { (cid:80) a ̸ = a ∗ log T ∆ a , K log( T ) ϵ } ) 2 , for stochastic bandits with ϵ -global DP.

Original: But this bound is restricted to Bernoulli distributions of reward.
Revised: But this bound is restricted to Bernoulli distributions of reward.

Original: In this paper, we provide the first problem-dependent regret lower bound that is valid for any reward distribution.
Revised: In this paper, we provide the first problem-dependent regret lower bound that is valid for any reward distribution.

Original: This lower bound shows that the effect of privacy does not only depend on ϵ but also on the total variation distance corresponding to the environment .
Revised: This lower bound shows that the effect of privacy does not only depend on ϵ but also on the total variation distance corresponding to the environment .

Original: Thus, it explicates the coupled effect of privacy and partial information in a high-privacy regime , which was not observed before.
Revised: Thus, it explicates the coupled effect of privacy and partial information in a high-privacy regime , which was not observed before.

Original: To derive this tighter lower bound, we extend the Karwa-Vadhan lemma (Lemma 6.1, (Karwa and Vadhan, 2017)) to a sequential setting and also propose a generic proof structure leading to problem-dependent and minimax lower bounds for stochastic and linear bandits .
Revised: To derive this tighter lower bound, we extend the Karwa-Vadhan lemma (Lemma 6.1, (Karwa and Vadhan, 2017)) to a sequential setting and also propose a generic proof structure leading to problem-dependent and minimax lower bounds for stochastic and linear bandits .

Original: To our knowledge, these are the first regret lower bounds for linear bandits with ϵ -global DP .
Revised: To our knowledge, these are the first regret lower bounds for linear bandits with ϵ -global DP .

Original: Basu et al.
Revised: Basu et al.

Original: (2019) also proposed a minimax regret lower bound for stochastic bandits with ϵ -global DP.
Revised: (2019) also proposed a minimax regret lower bound for stochastic bandits with ϵ -global DP.

Original: However, our minimax lower bound (Thm. 2) is tighter and does not need to assume that the reward distributions are Lipschitz continuous.
Revised: However, our minimax lower bound (Thm. 2) is tighter and does not need to assume that the reward distributions are Lipschitz continuous.

Original: Related Works: Bandit Algorithms with ϵ -global DP.
Revised: Related Works: Bandit Algorithms with ϵ -global DP.

Original: DP - UCB (Mishra and Thakurta, 2015;
Revised: DP - UCB (Mishra and Thakurta, 2015;

Original: Tossou and Dimitrakakis, 2016) was the first global DP version of UCB.
Revised: Tossou and Dimitrakakis, 2016) was the first global DP version of UCB.

Original: DP - UCB uses the treebased mechanism (Dwork et al., 2010a; Chan et al., 2011) to compute the sum of rewards.
Revised: DP - UCB uses the treebased mechanism (Dwork et al., 2010a; Chan et al., 2011) to compute the sum of rewards.

Original: The tree mechanism maintains a binary tree of depth log( T ) over the T streaming observations, where each node in this tree holds an i.i.d sample from a Laplace distribution with zero mean and scale (log( T ) /ϵ ) .
Revised: The tree mechanism maintains a binary tree of depth log( T ) over the T streaming observations, where each node in this tree holds an i.i.d sample from a Laplace distribution with zero mean and scale (log( T ) /ϵ ) .

Original: At each step t , the mechanism yields the sum of the first t observations and the log( T ) nodes on the root-to-the-leaf path in the binary tree as the private empirical mean.
Revised: At each step t , the mechanism yields the sum of the first t observations and the log( T ) nodes on the root-to-the-leaf path in the binary tree as the private empirical mean.

Original: As a result, the noise added to the UCB index per time-step is O (cid:0) log( T ) 2 .
Revised: As a result, the noise added to the UCB index per time-step is O (cid:0) log( T ) 2 .

Original: 5 /ϵ (cid:1) , which is responsible for the extra multiplicative factor log( T ) 1 . 5 in regret compared to the lower bound.
Revised: 5 /ϵ (cid:1) , which is responsible for the extra multiplicative factor log( T ) 1 . 5 in regret compared to the lower bound.

Original: DP - SE (Sajed and
Revised: DP - SE (Sajed and

Original: Sheffet, 2019) was the first ϵ -global DP algorithm to eliminate the additional multiplicative factor log( T ) 1 . 5 .
Revised: Sheffet, 2019) was the first ϵ -global DP algorithm to eliminate the additional multiplicative factor log( T ) 1 . 5 .

Original: DP - SE is an ϵ -global DP version of the Successive Elimination algorithm
Revised: DP - SE is an ϵ -global DP version of the Successive Elimination algorithm

Original: (Even-Dar et al., 2002).
Revised: (Even-Dar et al., 2002).

Original: However, the drawbacks were that the algorithm was not anytime, and was optimal only asymptotically, i.e. the horizon should be big enough.
Revised: However, the drawbacks were that the algorithm was not anytime, and was optimal only asymptotically, i.e. the horizon should be big enough.

Original: On the other hand, a careful analysis of the algorithm suggests that what made the algorithm optimal was the fact that the algorithm was run in doubling episodes, where the private means computed at the end of the episode to decide which arms to eliminate, were only computed using the samples collected from that episode.
Revised: On the other hand, a careful analysis of the algorithm suggests that what made the algorithm optimal was the fact that the algorithm was run in doubling episodes, where the private means computed at the end of the episode to decide which arms to eliminate, were only computed using the samples collected from that episode.

Original: We detect these ingredients and generalise them to propose a general framework to make any optimist index-based policy optimal followed by upper bounds matching the lower bounds (Table 1).
Revised: We detect these ingredients and generalise them to propose a general framework to make any optimist index-based policy optimal followed by upper bounds matching the lower bounds (Table 1).

Original: Similar techniques have been deployed to design a global DP extension of UCB in (Hu et al., 2021) but they have an additive factor log( T ) extra in regret than AdaP - UCB and do not match our proposed regret lower bound.
Revised: Similar techniques have been deployed to design a global DP extension of UCB in (Hu et al., 2021) but they have an additive factor log( T ) extra in regret than AdaP - UCB and do not match our proposed regret lower bound.

Original: Also, DP-TS (Hu and Hegde, 2022) aims to achieve ϵ -global DP with a Thompson sampling based approach.
Revised: Also, DP-TS (Hu and Hegde, 2022) aims to achieve ϵ -global DP with a Thompson sampling based approach.

Original: The “lazy" version of DP-TS achieves a similar regret as AdaP - UCB , but does not achieve the refined lower bound of Thm.
Revised: The “lazy" version of DP-TS achieves a similar regret as AdaP - UCB , but does not achieve the refined lower bound of Thm.

Original: 3 with the indistinguishability gaps.
Revised: 3 with the indistinguishability gaps.

Original: Even in non-private bandits, Thompson Sampling (Agrawal and Goyal, 2013) is not known to achieve the
Revised: Even in non-private bandits, Thompson Sampling (Agrawal and Goyal, 2013) is not known to achieve the

Original: Kullback-Leibler (KL) indistinguishability gap while KL-UCB (Garivier and Cappé, 2011) is known to achieve it.
Revised: Kullback-Leibler (KL) indistinguishability gap while KL-UCB (Garivier and Cappé, 2011) is known to achieve it.

Original: Our work leads to the first near-optimal bandit algorithm with ϵ -global DP, namely
Revised: Our work leads to the first near-optimal bandit algorithm with ϵ -global DP, namely

Original: AdaP - KLUCB .
Revised: AdaP - KLUCB .

Original: [Table]
Revised: [Table]

Original:  Background: Differential Privacy and Bandits
Revised:  Background: Differential Privacy and Bandits

Original: Differential Privacy (DP).
Revised: Differential Privacy (DP).

Original: DP renders an individual corresponding to a datum indistinguishable by constraining the output of an algorithm to be almost the same under a change in one input datum
Revised: DP renders an individual corresponding to a datum indistinguishable by constraining the output of an algorithm to be almost the same under a change in one input datum

Original: [Table]
Revised: [Table]

Original: Definition 1 ( ( ϵ, δ ) -DP
Revised: Definition 1 ( ( ϵ, δ ) -DP

Original: (Dwork and Roth, 2014)) .
Revised: (Dwork and Roth, 2014)) .

Original: A randomised algorithm
Revised: A randomised algorithm

Original: A satisfies ( ϵ, δ ) -
Revised: A satisfies ( ϵ, δ ) -

Original: Differential Privacy (DP) if for any two neighbouring datasets 5 D and D ′ , and for all sets of output O ⊆
Revised: Differential Privacy (DP) if for any two neighbouring datasets 5 D and D ′ , and for all sets of output O ⊆

Original: Range( A )
Revised: Range( A )

Original: [Equation]
Revised: [Equation]

Original: (1) where the probability space is over the coin flips of the mechanism A and for some ( ϵ, δ ) ∈ R ≥ 0 × R ≥ 0 .
Revised: (1) where the probability space is over the coin flips of the mechanism A and for some ( ϵ, δ ) ∈ R ≥ 0 × R ≥ 0 .

Original: If δ = 0 , we say that A satisfies ϵ -differential privacy .
Revised: If δ = 0 , we say that A satisfies ϵ -differential privacy .

Original: The Laplace mechanism (Dwork et al., 2010b; Dwork and Roth, 2014) ensures ϵ -DP by injecting a random noise to the output of the algorithm that is sampled from a calibrated Laplace distribution (as specified in Theorem 1).
Revised: The Laplace mechanism (Dwork et al., 2010b; Dwork and Roth, 2014) ensures ϵ -DP by injecting a random noise to the output of the algorithm that is sampled from a calibrated Laplace distribution (as specified in Theorem 1).

Original: We use Lap ( b ) to denote the Laplace distribution with mean 0 and variance 2 b 2 .
Revised: We use Lap ( b ) to denote the Laplace distribution with mean 0 and variance 2 b 2 .

Original: Theorem 1 ( ϵ -DP of Laplace Mechanism (Theorem 3.6, Dwork and Roth (2014))) .
Revised: Theorem 1 ( ϵ -DP of Laplace Mechanism (Theorem 3.6, Dwork and Roth (2014))) .

Original: Let us consider an algorithm f : X → R d with sensitivity s ( f ) ≜ max D , D ′ s.t ∥D−D ′
Revised: Let us consider an algorithm f : X → R d with sensitivity s ( f ) ≜ max D , D ′ s.t ∥D−D ′

Original: ∥
Revised: ∥

Original: Hamming =1 ∥ f ( D )
Revised: Hamming =1 ∥ f ( D )

Original: − f ( D ′ )
Revised: − f ( D ′ )

Original: ∥ 1 .
Revised: ∥ 1 .

Original: Here, ∥·∥ 1 is the L 1 norm on R d .
Revised: Here, ∥·∥ 1 is the L 1 norm on R d .

Original: If d noise samples { N i } di =1 are generated independently from Lap ( s ( f ) ϵ ) , the output injected with the noise, i.e. f ( D ) + [ N 1 , . . .
Revised: If d noise samples { N i } di =1 are generated independently from Lap ( s ( f ) ϵ ) , the output injected with the noise, i.e. f ( D ) + [ N 1 , . . .

Original: , N d ] satisfies ϵ -DP.
Revised: , N d ] satisfies ϵ -DP.

Original: The Laplace mechanism is originally proposed to ensure DP when the input database is static.
Revised: The Laplace mechanism is originally proposed to ensure DP when the input database is static.

Original: In a sequential setting like bandits, a mechanism must update the published statistics as new data items arrive, and thus DP definitions are extended accordingly (Basu et al., 2019).
Revised: In a sequential setting like bandits, a mechanism must update the published statistics as new data items arrive, and thus DP definitions are extended accordingly (Basu et al., 2019).

Original: Stochastic Bandits with Global DP.
Revised: Stochastic Bandits with Global DP.

Original: Now, we describe the general canonical bandit model (Section 4.6, (Lattimore and Szepesvári, 2018)) and define global Differential Privacy (global DP) in this setting.
Revised: Now, we describe the general canonical bandit model (Section 4.6, (Lattimore and Szepesvári, 2018)) and define global Differential Privacy (global DP) in this setting.

Original: A bandit algorithm (or policy) interacts with an environment ν consisting of K arms with reward distributions { ν a } Ka =1 for a given horizon T and produces a history H T ≜ { ( A t , R t ) }
Revised: A bandit algorithm (or policy) interacts with an environment ν consisting of K arms with reward distributions { ν a } Ka =1 for a given horizon T and produces a history H T ≜ { ( A t , R t ) }

Original: Tt =1 .
Revised: Tt =1 .

Original: At each step t , the choice of the arm depends on the previous history H t − 1 .
Revised: At each step t , the choice of the arm depends on the previous history H t − 1 .

Original: The reward R t is sampled from the reward distribution ν
Revised: The reward R t is sampled from the reward distribution ν

Original: A t and is conditionally independent of the previous history H t − 1 .
Revised: A t and is conditionally independent of the previous history H t − 1 .

Original: A bandit algorithm (or policy) π can be represented by a sequence ( π t ) T t =1 , where π t : H t − 1 →
Revised: A bandit algorithm (or policy) π can be represented by a sequence ( π t ) T t =1 , where π t : H t − 1 →

Original: [ K ] is a probability kernel.
Revised: [ K ] is a probability kernel.

Original: Thus, we denote the probability of occurrence of a sequence of actions a T ≜ [ a 1 , . . .
Revised: Thus, we denote the probability of occurrence of a sequence of actions a T ≜ [ a 1 , . . .

Original: , a T ] given a sequence of rewards r T ≜ [ r 1 , . . .
Revised: , a T ] given a sequence of rewards r T ≜ [ r 1 , . . .

Original: , r T ] as π ( a T | r T ) ≜ (cid:81) Tt =1 π t ( a t | a 1 , r 1 , . . .
Revised: , r T ] as π ( a T | r T ) ≜ (cid:81) Tt =1 π t ( a t | a 1 , r 1 , . . .

Original: , a t − 1 , r t − 1 ) .
Revised: , a t − 1 , r t − 1 ) .

Original: If we consider a T as the output of a bandit algorithm and r T as the input, π ( a T | r T ) defines the corresponding probability distribution on the output space required for defining DP (Eq.
Revised: If we consider a T as the output of a bandit algorithm and r T as the input, π ( a T | r T ) defines the corresponding probability distribution on the output space required for defining DP (Eq.

Original: In order to define DP in the sequential setup of bandits, the event-level privacy under continuous observations (Dwork et al., 2010b) framework is adopted.
Revised: In order to define DP in the sequential setup of bandits, the event-level privacy under continuous observations (Dwork et al., 2010b) framework is adopted.

Original: In this framework, if two data sequences differ on a single entry at a single time-step t and are identical on all other time-steps, they are called neighbouring data sequences.
Revised: In this framework, if two data sequences differ on a single entry at a single time-step t and are identical on all other time-steps, they are called neighbouring data sequences.

Original: A sequential algorithm is differentially private if its output sequence is not distinguishable from two neighbouring input sequences.
Revised: A sequential algorithm is differentially private if its output sequence is not distinguishable from two neighbouring input sequences.

Original: This framework leads to the global DP definition for bandits (Mishra and Thakurta, 2015; Basu et al., 2019).
Revised: This framework leads to the global DP definition for bandits (Mishra and Thakurta, 2015; Basu et al., 2019).

Original: Definition 2 (Global DP for Bandits) .
Revised: Definition 2 (Global DP for Bandits) .

Original: A bandit algorithm π ∈
Revised: A bandit algorithm π ∈

Original: Π
Revised: Π

Original: ϵ satisfies ϵ -global DP, if π ( a T | r T ) ≤
Revised: ϵ satisfies ϵ -global DP, if π ( a T | r T ) ≤

Original: e ϵ π ( a T | r ′ T ) for every action sequence a T ∈ [ K ] T and every two neighbouring reward sequences r T , r ′ T ∈ R T , which means that ∃ j ∈ [1 , T ]
Revised: e ϵ π ( a T | r ′ T ) for every action sequence a T ∈ [ K ] T and every two neighbouring reward sequences r T , r ′ T ∈ R T , which means that ∃ j ∈ [1 , T ]

Original: such that r j ̸ = r ′
Revised: such that r j ̸ = r ′

Original: j
Revised: j

Original: and ∀ t ̸ =
Revised: and ∀ t ̸ =

Original: j r t = r ′ t .
Revised: j r t = r ′ t .

Original: The set of all bandit algorithms satisfying ϵ -global DP, i.e. Π ϵ , is called the ϵ -global DP policy class.
Revised: The set of all bandit algorithms satisfying ϵ -global DP, i.e. Π ϵ , is called the ϵ -global DP policy class.

Original: [Table]
Revised: [Table]

Original: Now, we formally define the regret of a bandit algorithm with ϵ -global DP, namely π ϵ , as
Revised: Now, we formally define the regret of a bandit algorithm with ϵ -global DP, namely π ϵ , as

Original: [Equation]
Revised: [Equation]

Original: N a ( T ) ≜ (cid:80)
Revised: N a ( T ) ≜ (cid:80)

Original: Tt =1 1 { A t = a } is the number of times the arm a is played till T .
Revised: Tt =1 1 { A t = a } is the number of times the arm a is played till T .

Original: ∆ a ≜ µ ∗ ( ν ) − µ a is the suboptimality gap of the arm a .
Revised: ∆ a ≜ µ ∗ ( ν ) − µ a is the suboptimality gap of the arm a .

Original: The expectation is taken with respect to the probability measure on action-reward sequences induced by the interaction of π ϵ and ν .
Revised: The expectation is taken with respect to the probability measure on action-reward sequences induced by the interaction of π ϵ and ν .

Original: The objective of the bandit algorithm π ϵ is to satisfy ϵ -global DP while minimising the regret over horizon T .
Revised: The objective of the bandit algorithm π ϵ is to satisfy ϵ -global DP while minimising the regret over horizon T .

Original: A regret lower bound of any ϵ -global DP bandit algorithm in Π ϵ quantifies the fundamental hardness of this problem.
Revised: A regret lower bound of any ϵ -global DP bandit algorithm in Π ϵ quantifies the fundamental hardness of this problem.

Original:  Regret Lower Bounds for Bandits with ϵ -global DP
Revised:  Regret Lower Bounds for Bandits with ϵ -global DP

Original: The first question is how much additional regret we have to endure in bandits with ϵ -global DP than the bandits without privacy.
Revised: The first question is how much additional regret we have to endure in bandits with ϵ -global DP than the bandits without privacy.

Original: The lower bound on regret provides insight into the intrinsic hardness of the problem and serves as a target for the optimal algorithm design.
Revised: The lower bound on regret provides insight into the intrinsic hardness of the problem and serves as a target for the optimal algorithm design.

Original: In this section, we prove minimax and problem-dependent lower bounds for stochastic and linear bandits under ϵ -global DP.
Revised: In this section, we prove minimax and problem-dependent lower bounds for stochastic and linear bandits under ϵ -global DP.

Original: The proposed lower bounds are summarised in Table 2.
Revised: The proposed lower bounds are summarised in Table 2.

Original: We defer the proof details to the supplementary.
Revised: We defer the proof details to the supplementary.

Original: Stochastic Bandits.
Revised: Stochastic Bandits.

Original: First, we consider the stochastic bandit problem with K -arms (as in Section 2).
Revised: First, we consider the stochastic bandit problem with K -arms (as in Section 2).

Original: Minimax Regret.
Revised: Minimax Regret.

Original: We derive the minimax regret lower bound in this bandit setting with ϵ -global DP.
Revised: We derive the minimax regret lower bound in this bandit setting with ϵ -global DP.

Original: Minimax regret is the lowest achievable regret by any bandit algorithm under the worst environment among a family of environments E K with K arms under consideration.
Revised: Minimax regret is the lowest achievable regret by any bandit algorithm under the worst environment among a family of environments E K with K arms under consideration.

Original: [Equation]
Revised: [Equation]

Original: Theorem 2 (Minimax lower bound) .
Revised: Theorem 2 (Minimax lower bound) .

Original: For any K > 1 and T ≥ K − 1 , and ϵ > 0 , the minimax regret of stochastic bandits with ϵ -global DP satisfies
Revised: For any K > 1 and T ≥ K − 1 , and ϵ > 0 , the minimax regret of stochastic bandits with ϵ -global DP satisfies

Original: [Equation]
Revised: [Equation]

Original: Consequences of Theorem 2.
Revised: Consequences of Theorem 2.

Original: Equation (3) shows two regimes of hardness: high-privacy , corresponding to lower values of ϵ , and low-privacy , corresponding to higher values of ϵ .
Revised: Equation (3) shows two regimes of hardness: high-privacy , corresponding to lower values of ϵ , and low-privacy , corresponding to higher values of ϵ .

Original: In the high-privacy regime, specifically for ϵ < 13127 (cid:113) ( K − 1) T , the hardness depends only on the number of the arms
Revised: In the high-privacy regime, specifically for ϵ < 13127 (cid:113) ( K − 1) T , the hardness depends only on the number of the arms

Original: K and the privacy budget ϵ and is higher than the lower bound for bandits without privacy.
Revised: K and the privacy budget ϵ and is higher than the lower bound for bandits without privacy.

Original: In the low-privacy regime, i.e. for ϵ ≥ 13127 (cid:113) ( K − 1) T , the lower bound coincides with that of the bandits without privacy.
Revised: In the low-privacy regime, i.e. for ϵ ≥ 13127 (cid:113) ( K − 1) T , the lower bound coincides with that of the bandits without privacy.

Original: This indicates the phenomena that for higher values of ϵ , signifying lower privacy, bandits with global DP is not harder than the bandits without privacy.
Revised: This indicates the phenomenon that for higher values of ϵ , signifying lower privacy, bandits with global DP are not harder than the bandits without privacy.

Original: Specially, for significantly large T , the threshold between low and high privacy regime is smaller than most of the practically used privacy budget values.
Revised: Especially, for significantly large T , the threshold between low and high privacy regimes is smaller than most of the practically used privacy budget values.

Original: For example, if T = 10 6 and K = 100 , the bandits with and without global DP are equivalently hard for any privacy budget ϵ ≥ 0 .
Revised: For example, if T = 10 6 and K = 100 , the bandits with and without global DP are equivalently hard for any privacy budget ϵ ≥ 0 .

Original: This shows that for stochastic bandits, we can deploy very low privacy budgets ( ϵ ) without losing anything in performance.
Revised: This shows that for stochastic bandits, we can deploy very low privacy budgets ( ϵ ) without losing anything in performance.

Original: Problem-dependent Regret: Similar to the lower bounds specific to the Bernoulli reward distributions (Shariff and Sheffet, 2018)
Revised: Problem-dependent Regret: Similar to the lower bounds specific to the Bernoulli reward distributions (Shariff and Sheffet, 2018)

Original: , the minimax bound indicates a separation between hardness due to global DP and partial information.
Revised: , the minimax bound indicates a separation between hardness due to global DP and partial information.

Original: To understand whether this observation is an artefact of the
Revised: To understand whether this observation is an artefact of the

Original: Bernoulli distribution and the worst-case environment considered in minimax bounds, we derive a lower bound on the problem-dependent regret (Equation (2)) for general reward distributions.
Revised: Bernoulli distribution and the worst-case environment considered in minimax bounds, we derive a lower bound on the problem-dependent regret (Equation (2)) for general reward distributions.

Original: Before deriving the lower bound, we define two information-theoretic terms that quantify the distinguishability of the specific bandit environment ν where the algorithm is operating from all other bandit environments with K arms and finite means.
Revised: Before deriving the lower bound, we define two information-theoretic terms that quantify the distinguishability of the specific bandit environment ν where the algorithm is operating from all other bandit environments with K arms and finite means.

Original: If M is a set of distributions with finite means, and µ ( P ) is the mean of the reward distribution P ∈ M , we define the KL-distinguishability gap as d inf ( P, µ ∗ , M ) ≜
Revised: If M is a set of distributions with finite means, and µ ( P ) is the mean of the reward distribution P ∈ M , we define the KL-distinguishability gap as d inf ( P, µ ∗ , M ) ≜

Original: inf P ′ ∈M { D KL ( P ∥ P ′ ) : µ ( P ′ ) > µ ∗ } and the TV-distinguishability gap as t inf ( P, µ ∗ , M ) ≜ inf P ′ ∈M { TV ( P ∥ P ′ ) : µ ( P ′ ) > µ ∗ } .
Revised: inf P ′ ∈M { D KL ( P ∥ P ′ ) : µ ( P ′ ) > µ ∗ } and the TV-distinguishability gap as t inf ( P, µ ∗ , M ) ≜ inf P ′ ∈M { TV ( P ∥ P ′ ) : µ ( P ′ ) > µ ∗ } .

Original: These two quantities indicate the minimum dissimilarity, in terms of KL-divergence and Total Variation (TV) distance, of the reward distribution of the optimal arm a ∗ with any other distribution with  finite mean higher than µ ∗ .
Revised: These two quantities indicate the minimum dissimilarity, in terms of KL-divergence and Total Variation (TV) distance, of the reward distribution of the optimal arm a ∗ with any other distribution with a finite mean higher than µ ∗ .

Original: For bandits without DP,  inverse of the KL-distinguishability gap quantifies the hardness to identify the optimal arm of the environment ν under partial information (Lai and Robbins, 1985).
Revised: For bandits without DP, the inverse of the KL-distinguishability gap quantifies the hardness to identify the optimal arm of the environment ν under partial information (Lai and Robbins, 1985).

Original: Theorem 3 (Problem-dependent Regret Lower Bound) .
Revised: Theorem 3 (Problem-dependent Regret Lower Bound) .

Original: Let the environment E be a set of K reward distributions with finite means and a policy π ϵ ∈ Π cons ( E )
Revised: Let the environment E be a set of K reward distributions with finite means and a policy π ϵ ∈ Π cons ( E )

Original: ∩ Π ϵ be a consistent policy 6 over E satisfying ϵ -global DP .
Revised: ∩ Π ϵ be a consistent policy 6 over E satisfying ϵ -global DP .

Original: Then, for all ν =
Revised: Then, for all ν =

Original: ( P i )
Revised: ( P i )

Original: Ki =1 ∈ E , it holds that
Revised: Ki =1 ∈ E , it holds that

Original: [Equation]
Revised: [Equation]

Original: Consequences of Theorem 3.
Revised: Consequences of Theorem 3.

Original: Now, we summarise the interesting observations led by the lower bound
Revised: Now, we summarise the interesting observations led by the lower bound.

Revised: Universality: The lower bound of

Revised: Theorem

Revised: holds

Original: Universality: The lower bound of Theorem 3 holds for any environment with K arms and reward distributions with finite means.
Revised: for any environment with K arms and reward distributions with finite means.

Original: This is the first general lower bound for bandits with ϵ -global DP. 2.
Revised: This is the first general lower bound for bandits with ϵ -global DP. 2.

Original: For Bernoulli distributions of reward: TV-distinguishability gap t inf ( P a , µ ∗ , M a ) =
Revised: For Bernoulli distributions of reward: TV-distinguishability gap t inf ( P a , µ ∗ , M a ) =

Original: ∆ a , i.e. the suboptimality gap of arm a , and the KL-distinguishability gap d inf ( P a , µ ∗ , M a ) ≥ 2∆ 2 a . 
Revised: ∆ a , i.e. the suboptimality gap of arm a , and the KL-distinguishability gap d inf ( P a , µ ∗ , M a ) ≥ 2∆ 2 a . Thus

Original: Thus, our problem-dependent lower bound reduces to Ω( (cid:80) a ̸ = a ∗ log T min { ∆ a ,ϵ } ) .
Revised: , our problem dependent lower bound reduces to Ω( (cid:80) a ̸ = a ∗ log T min { ∆ a ,ϵ } ) .

Revised: For Bernoulli

Revised: rewards,

Original: For Bernoulli rewards, our lower bound is able to retrieve the lower bound of (Shariff and Sheffet, 2018) with explicit constants. 
Revised: our lower bound is able to retrieve the lower bound of Shariff and Sheffet, 2018) with explicit constants. High- and Low-privacy Regimes:

Revised: Like

Original: High- and Low-privacy Regimes: Like the minimax regret bound, the problem-dependent regret also indicates two clear regimes in regret due to high and low privacy (resp.
Revised: the minimax regret bound , the problem- dependent regret also indicates two clear regimes in regret due to high and low privacy (resp.

Original: small and large privacy budgets ϵ ).
Revised: small and large privacy budgets ϵ ).

Original: In the low-privacy regime, i
Revised: In the low-privacy regime, i.e. for ϵ ≥ d inf P a ,µ

Revised: ∗

Original: e. for ϵ ≥ d inf ( P a ,µ ∗ , M a ) 6 t inf( P a ,µ ∗ , M a ) , the regret achievable by bandits with global DP and without global DP are same.
Revised: , M a ) 6 t inf P a ,µ ∗ , M a ) , the regret achievable by bandits with global DP and without global DP are same.

Original: Thus, there is no loss in performance due to privacy in this regime.
Revised: Thus, there is no loss in performance due to privacy in this regime.

Original: In the high-privacy regime, 
Revised: In the high-privacy regime, i.e. for ϵ < d inf ( P a ,µ ∗ , M a ) 6 t

Original: i
Revised: inf

Original: e. for ϵ < d inf ( P a ,µ ∗ , M a ) 6 t inf ( P a ,µ ∗ , M a ) , the regret depends on a coupled effect of privacy and partial information.
Revised: ( P a ,µ ∗ , M a ) , the regret depends on a coupled effect of privacy and partial information.

Original: This effect is quantified by the inverse of the privacy budget times  inverse of the TV-distinguishability gap.
Revised: This effect is quantified by the inverse of the privacy budget times the inverse of the TV-distinguishability gap.

Original: This interaction between privacy and partial information was not explicated by any of the existing lower bounds.
Revised: This interaction between privacy and partial information was not explicated by any of the existing lower bounds.

Original: In Section 4, we propose the AdaP - KLUCB algorithm that demonstrates the same interactive effect in its regret upper bound.
Revised: In Section 4, we propose the AdaP - KLUCB algorithm that demonstrates the same interactive effect in its regret upper bound.

Original: Stochastic Linear Bandits.
Revised: Stochastic Linear Bandits.

Original: In order to illustrate the generality of our results, and also to investigate how partial information and privacy in structured bandits, we derive minimax and problem-dependent regret lower bounds for stochastic linear bandits (Lattimore and Szepesvari, 2017).
Revised: In order to illustrate the generality of our results, and also to investigate how partial information and privacy in structured bandits, we derive minimax and problem-dependent regret lower bounds for stochastic linear bandits (Lattimore and Szepesvari, 2017).

Original: To perform the regret analysis, we consider a simple linear model with parameter θ ∈ R d and Gaussian noise.
Revised: To perform the regret analysis, we consider a simple linear model with parameter θ ∈ R d and Gaussian noise.

Original: It implies that for an action A t ∈
Revised: It implies that for an action A t ∈

Original: A ⊆ R d the reward is R t = ⟨ A t , θ ⟩ + η t , where η t ∼ N (0 , 1) is a sequence of independent Gaussian noises.
Revised: A ⊆ R d the reward is R t = ⟨ A t , θ ⟩ + η t , where η t ∼ N (0 , 1) is a sequence of independent Gaussian noises.

Original: The regret of a policy is Reg T ( A , θ ) ≜
Revised: The regret of a policy is Reg T ( A , θ ) ≜

Original: E θ (cid:104)(cid:80) Tt =1 ∆
Revised: E θ (cid:104)(cid:80) Tt =1 ∆

Original: A t (cid:105) , where suboptimality gap ∆ a ≜ max a ′ ∈A ⟨ a ′ − a, θ ⟩ , and E θ [ · ] is the expectation with respect to the measure on outcomes induced by the interaction of the policy and the linear bandit determined by θ .
Revised: A t (cid:105) , where suboptimality gap ∆ a ≜ max a ′ ∈A ⟨ a ′ − a, θ ⟩ , and E θ [ · ] is the expectation with respect to the measure o outcomes induced by the interaction of the policy and the linear bandit determined by θ .

Original: Given this structure, we state the minimax and problem-dependent regret lower bounds for stochastic linear bandits.
Revised: Given this structure, we state the minimax and problem-dependent regret lower bounds for stochastic linear bandits.

Original: Theorem 4 (Minimax regret lower bound) .
Revised: Theorem 4 (Minimax regret lower bound) .

Original: Let A =
Revised: Let A =

Original: [ − 1 , 1] d and Θ = (cid:8) − T − 1 / 2 , T − 1 / 2 (cid:9) d .
Revised: [ − 1 , 1] d and Θ = (cid:8) − T − 1 / 2 , T − 1 / 2 (cid:9) d .

Original: Then,  there exists a vector θ ∈ Θleading to a minimax regret of linear bandits with ϵ -global DP such that
Revised: Then, for any ϵ -global DP policy, there exists a vector θ ∈ Θ such that

Original: [Equation]
Revised: [Equation]

Original: Theorem 5 (Problem-dependent regret lower bound) .
Revised: Theorem 5 (Problem-dependent regret lower bound) .

Original: Let A ⊂ R d be a finite set spanning R d and θ ∈ R
Revised: Let A ⊂ R d be a finite set spanning R d and θ ∈ R

Original: d be such that there is a unique optimal action.
Revised: d be such that there is a unique optimal action.

Original: Then,for any consistent and ϵ -global DP bandit algorithm satisfies
Revised: Then, any consistent and ϵ -global DP bandit algorithm satisfies

Original: [Equation]
Revised: [Equation]

Original: where the structural distinguishability gap is the solution of a constraint optimisation
Revised: where the structural distinguishability gap is the solution of a constraint optimisation

Original: [Equation]
Revised: [Equation]

Original: such that∥
Revised: such that

Original: [Equation]
Revised: [Equation]

Original: for all a ∈ A with ∆ a > 0 , H α
Revised: for all a ∈ A with ∆ a > 0 , H α

Original: = (cid:80) a ∈A α ( a ) aa ⊤ , and a structure dependent constant ρ a ( A ) .
Revised: = (cid:80) a ∈A α ( a ) aa ⊤ , and a structure-dependent constant ρ a ( A ) .

Original: Remarks.
Revised: Remarks.

Original: Theorems 4 and 5 are the first minimax and problem-dependent regret lower bounds for linear bandits.
Revised: Theorems 4 and 5 are the first minimax and problem-dependent regret lower bounds for linear bandits.

Original: The minimax regret bound also shows a clear distinction between high- and low-privacy regimes for ϵ < 1 / (3d √ T ) and ϵ ≥ 1 / (3d √ T ) .
Revised: The minimax regret bound also shows a clear distinction between high- and low-privacy regimes for ϵ < 1 / (3 √ T ) and ϵ ≥ 1 / (3 √ T ) .

Original: Interestingly, the threshold between two regimes decreases with increase in dimension d and allows us to use quite low values of ϵ without losing any performance.

Original: The difference between high- and low-privacy regimes is more subtle.
Revised: For the problem-dependent bound, the difference between high- and low-privacy regimes is more subtle.

Original: The first constraint on ∥ a ∥ 2 H − 1 α is reminiscent of the Graves-Lai bound for structural bandits without privacy.
Revised: The first constraint on ∥ a ∥ 2 H − 1 α is reminiscent of the Graves-Lai bound for structural bandits without privacy.

Original: The second constraint, i.e. 3 ϵρ a ( A )∆ a , captures the interaction between privacy and partial information under linear structure and in the high-privacy regime.
Revised: The second constraint, i.e. 3 ϵρ a ( A )∆ a , captures the interaction between privacy and partial information under linear structure and in the high-privacy regime.

Original: Proof Technique.
Revised: Proof Technique.

Original: In order to prove the lower bounds, we adopt the general canonical bandit model.
Revised: In order to prove the lower bounds, we adopt the general canonical bandit model.

Original: The high level idea of proving bandit lower bounds is selecting two hard environments, which are hard to be statistically distinguished but are conflicting, i.e. actions that may be optimal in one is suboptimal in other.
Revised: The high-level idea of proving bandit lower bounds is selecting two hard environments, which are hard to be statistically distinguished but are conflicting, i.e. actions that may be optimal in one is suboptimal in other.

Original: This is quantified by upper bounding the per-step KL-divergence between action distributions for such hard environments, and then plugging in this upper bound in the BretagnolleHuber inequality to obtain a regret lower bound.
Revised: This is quantified by upper bounding the per-step KL-divergence between action distributions for such hard environments, and then plugging in this upper bound in the Bretagnolle-Huber inequality to obtain a regret lower bound.

Original: Though this proof technique works to quantify the hardness due to partial information, we need to upper bound the “confusion" created due to global DP.
Revised: Though this proof technique works to quantify the hardness due to partial information, we need to upper bound the “confusion" created due to global DP.

Original: Existing proofs use the information-processing lemma of Karwa-Vadhan (Karwa and Vadhan, 2017).
Revised: Existing proofs use the information-processing lemma of Karwa-Vadhan (Karwa and Vadhan, 2017).

Original: Though this works for Bernoulli bandits, to proof a general bound, we derive a sequential version of this lemma.
Revised: To prove a general bound, we derive a sequential version of this lemma.

Original: The sequential version leads to an upper bound of the KL-divergence dependent on both  total-variation distinguishability gap and  privacy budget, and enables us to show the coupled effect of privacy and partial-information in the high-privacy regimes.
Revised: The sequential version leads to an upper bound of the KL-divergence dependent on both the total-variation distinguishability gap and the privacy budget, and enables us to show the coupled effect of privacy and partial-information in the high-privacy regimes.

Original: For details of the technical results, we refer to the Supplementary Material.
Revised: For details of the technical results, we refer to the Supplementary Material.

Original:  Stochastic Bandit Algorithms with ϵ -global DP: AdaP - UCB & AdaP - KLUCB
Revised:  Stochastic Bandit Algorithms with ϵ -global DP: AdaP - UCB & AdaP - KLUCB

Original: In this section, we present a framework (Algorithm 1) to convert index-based optimistic algorithms into algorithms satisfying ϵ -global DP .
Revised: In this section, we present a framework (Algorithm 1) to convert index-based optimistic algorithms into algorithms satisfying ϵ -global DP .

Original: We instantiate this framework by proposing AdaP - UCB and
Revised: We instantiate this framework by proposing AdaP - UCB and

Original: AdaP - KLUCB , and derive corresponding regret upper bounds.
Revised: AdaP - KLUCB , and derive corresponding regret upper bounds.

Original: Algorithm  of  subsequences 
Revised: Algorithm 1 A framework for ϵ -global DP extension of an index-based bandit algorithm Adaptive episodes of reward subsequences Let t ℓ = t + Compute A ℓ = argmax a

Revised: I ϵa ( t ℓ − 1 , α ) ▷ Arm selection with private indexes (Eqn. (7)-(8)) 7:

Revised: Choose arm A ℓ until round t such that N A ℓ ( t ) =

Revised: 2 N A ℓ ( t ℓ − 1)

Revised: ▷ Doubling of episodes 8: end for

Original: Index-Based Algorithms and Private Empirical Mean.
Revised: Index-Based Algorithms and Private Empirical Mean.

Original: We focus on index-based bandit algorithms that compute the empirical mean of rewards of each arm at each step.
Revised: We focus on index-based bandit algorithms that compute the empirical mean of rewards of each arm at each step.

Original: Using the empirical means, they compute an optimistic index for each arm that serves as a high-probability upper confidence bound on the true mean of the corresponding arm.
Revised: Using the empirical means, they compute an optimistic index for each arm that serves as a high-probability upper confidence bound on the true mean of the corresponding arm.

Original: Examples of such algorithms with empirical mean-based indexes include UCB (Auer et al., 2002), MOSS (Audibert and Bubeck, 2010), KL-UCB (Garivier and Cappé, 2011), IMED (Honda and Takemura, 2015) etc.
Revised: Examples of such algorithms with empirical mean-based indexes include UCB (Auer et al., 2002), MOSS (Audibert and Bubeck, 2010), KL-UCB (Garivier and Cappé, 2011), IMED (Honda and Takemura, 2015) etc.

Original: Here, the empirical mean is the main statistics of reward sequences used by the algorithms.
Revised: Here, the empirical mean is the main statistics of reward sequences used by the algorithms.

Original: Thus, by post-processing lemma, designing an ϵ -global DP bandit algorithm reduces to computing the empirical means privately.
Revised: Thus, by post-processing lemma, designing an ϵ -global DP bandit algorithm reduces to computing the empirical means privately.

Original: By Theorem 1, we know that adding a Laplacian noise with scale s (ˆ µ t ) /ϵ
Revised: By Theorem 1, we know that adding a Laplacian noise with scale s (ˆ µ t ) /ϵ

Original: to each empirical mean turns it ϵ -DP, where s (ˆ µ t ) is the sensitivity of empirical mean ˆ µ t at step t .
Revised: to each empirical mean turns it ϵ -DP, where s (ˆ µ t ) is the sensitivity of empirical mean ˆ µ t at step t .

Original: For K arms and horizon
Revised: For K arms and horizon

Original: T , an index-based bandit algorithm computes the empirical mean KT times.
Revised: T , an index-based bandit algorithm computes the empirical mean KT times.

Original: Using the naïve composition of ϵ -DP, a first baseline is to make each computed empirical mean ϵKT -DP.
Revised: Using the naïve composition of ϵ -DP, a first baseline is to make each computed empirical mean ϵKT -DP.

Original: This needs us to add at each step noise scaled linearly to T .
Revised: This needs us to add at each step noise scaled linearly to T .

Original: This might lead the corresponding bandit algorithm to yield a linear regret.
Revised: This might lead the corresponding bandit algorithm to yield a linear regret.

Original: Thus, we need to add noise less number of times and with lower sensitivity.
Revised: Thus, we need to add noise less number of times and with lower sensitivity.

Original: Empirical Mean using Reward Sub-sequence.
Revised: Empirical Mean using Reward Sub-sequence.

Original: The improvement is invoked by the observation that a bandit algorithm does not need to calculate the empirical mean at each time-step using all the rewards observed till that step.
Revised: The improvement is invoked by the observation that a bandit algorithm does not need to calculate the empirical mean at each time-step using all the rewards observed till that step.

Original: Thus, motivated by the DP-SE (Sajed and Sheffet, 2019), we calculate the empirical mean less number of times by exploiting an episodic structure that we will explicit later.
Revised: Thus, motivated by the DP-SE (Sajed and Sheffet, 2019), we calculate the empirical mean less number of times by exploiting an episodic structure that we will explicit later.

Original: Specifically, we use only the rewards of the last episode.
Revised: Specifically, we use only the rewards of the last episode.

Original: We formally express this trick in Lemma 1.
Revised: We formally express this trick in Lemma 1.

Original: Lemma 1.
Revised: Lemma 1.

Original: Let us define the private empirical mean of the rewards between steps i and j ( i < j ) as f
Revised: Let us define the private empirical mean of the rewards between steps i and j ( i < j ) as f

Original: [Equation]
Revised: [Equation]

Original: { r i , . . .
Revised: { r i , . . .

Original: , r j } ≜
Revised: , r j } ≜

Original: [Equation]
Revised: [Equation]

Original: − i
Revised: − i

Original: [Equation]
Revised: [Equation]

Original: (cid:88) t
Revised: (cid:88) t

Original: = i
Revised: = i

Original: [Equation]
Revised: [Equation]

Original: Lap (cid:18) ( j − i ) ϵ (cid:19) .
Revised: Lap (cid:18) ( j − i ) ϵ (cid:19) .

Original: If 1 < t 1 < · · · < t ℓ < T and r t ∈ [0 , 1] , the mechanism g ϵ mapping the sequence of rewards ( r 1 , r 2 , . . .
Revised: If 1 < t 1 < · · · < t ℓ < T and r t ∈ [0 , 1] , the mechanism g ϵ mapping the sequence of rewards ( r 1 , r 2 , . . .

Original: , r T ) to ( ℓ + 1) -private empirical means ( f ϵ { r 1 , . .
Revised: , r T ) to ( ℓ + 1) -private empirical means ( f ϵ { r 1 , . .

Original: , r t 1 − 1 } , f ϵ { r t 1 , . . .
Revised: , r t 1 − 1 } , f ϵ { r t 1 , . . .

Original: , r t 2 − 1 } , . . .
Revised: , r t 2 − 1 } , . . .

Original: , f ϵ { r t ℓ , . . .
Revised: , f ϵ { r t ℓ , . . .

Original: , r T } ) satisfies ϵ -DP.
Revised: , r T } ) satisfies ϵ -DP.

Original: Lemma 1 implies that, if we calculate the empirical mean of each arm ℓ + 1 ( ≪ T ) times on nonoverlapping sub-sequences of the reward stream, we only need to ensure that each empirical mean is ϵ -DP with respect to the corresponding reward sub-sequence.
Revised: Lemma 1 implies that, if we calculate the empirical mean of each arm ℓ + 1 ( ≪ T ) times on nonoverlapping sub-sequences of the reward stream, we only need to ensure that each empirical mean is ϵ -DP with respect to the corresponding reward sub-sequence.

Original: These ( ℓ + 1) private empirical means together ensure that the sequence of ( ℓ + 1) computed indexes and the resultant action sequence of length T satisfy ϵ -global DP by the post-processing lemma (Dwork and Roth, 2014).
Revised: These ( ℓ + 1) private empirical means together ensure that the sequence of ( ℓ + 1) computed indexes and the resultant action sequence of length T satisfy ϵ -global DP by the post-processing lemma (Dwork and Roth, 2014).

Original: Thus, we divide the horizon into ( ℓ + 1) -episodes.
Revised: Thus, we divide the horizon into ( ℓ + 1) -episodes.

Original: This allows us to take T actions only by computing ℓ + 1 private empirical means with the rewards observed in the last active episode of the arm.
Revised: This allows us to take T actions only by computing ℓ + 1 private empirical means with the rewards observed in the last active episode of the arm.

Original: Specifically, for each t ∈ [ t i , t i +1 − 1] , we play the same arm that was decided at the beginning of the episode, i.e. at t i , with the private empirical mean computed from the last active episode of that arm.
Revised: Specifically, for each t ∈ [ t i , t i +1 − 1] , we play the same arm that was decided at the beginning of the episode, i.e. at t i , with the private empirical mean computed from the last active episode of that arm.

Original: Adaptive Episodes with Doubling.
Revised: Adaptive Episodes with Doubling.

Original: In order to set the episode lengths, we focus on the specific structure of the bandit process.
Revised: In order to set the episode lengths, we focus on the specific structure of the bandit process.

Original: Specifically, we know that a sub-optimal arm a should be sampled at least n a = O (cid:16) log( T ) ∆ 2 a (cid:17) times before discarding.
Revised: Specifically, we know that a sub-optimal arm a should be sampled at least n a = O (cid:16) log( T ) ∆ 2 a (cid:17) times before discarding.

Original: Since the empirical means are computed using only rewards of one episode, to yield a near-optimal regret, the length of the episode should be greater at any given time than n a .
Revised: Since the empirical means are computed using only rewards of one episode, to yield a near-optimal regret, the length of the episode should be greater at any given time than n a .

Original: To satisfy this criterion, we deploy adaptive episodes with doubling.
Revised: To satisfy this criterion, we deploy adaptive episodes with doubling.

Original: Example 2 (Illustration of Algortihm 1.) .
Revised: Example 2 (Illustration of Algortihm 1.) .

Original: To clarify the schematic, we illustrate a few steps of executing Algorithm 1 in Figure 1.
Revised: To clarify the schematic, we illustrate a few steps of executing Algorithm 1 in Figure 1.

Original: After playing each arm once, the first episode begins at t 1 .
Revised: After playing each arm once, the first episode begins at t 1 .

Original: To observe different ingredients, we focus on step t 4 = 7 .
Revised: To observe different ingredients, we focus on step t 4 = 7 .

Original: The index of Arm 1 at t 4 uses the private empirical mean r 4 + r 5+ Lap ( 1 2 ϵ ) to build a high probability upper bound of the real mean µ 1 with confidence t − α 4 .
Revised: The index of Arm 1 at t 4 uses the private empirical mean r 4 + r 5+ Lap ( 1 2 ϵ ) to build a high probability upper bound of the real mean µ 1 with confidence t − α 4 .

Original: The index of Arm 2 uses r 6 + Lap ( 1 ϵ ) .
Revised: The index of Arm 2 uses r 6 + Lap ( 1 ϵ ) .

Original: If we assume that the index of Arm 1 is higher at t 4 , Arm 1 is played for a full episode from t 4 until t 5 − 1 .
Revised: If we assume that the index of Arm 1 is higher at t 4 , Arm 1 is played for a full episode from t 4 until t 5 − 1 .

Original: The last time, when Arm 1 was played, the length of the episode was 2 .
Revised: The last time, when Arm 1 was played, the length of the episode was 2 .

Original: Thus, following t 4 , the length of the episode is doubled to 4 .
Revised: Thus, following t 4 , the length of the episode is doubled to 4 .

Original: Theorem 6 ( ϵ -global DP) .
Revised: Theorem 6 ( ϵ -global DP) .

Original: If I ϵa is computed using only the private empirical mean of the rewards collected in the last active episode of arm a , Algorithm 1 satisfies ϵ -global DP.
Revised: If I ϵa is computed using only the private empirical mean of the rewards collected in the last active episode of arm a , Algorithm 1 satisfies ϵ -global DP.

Original: Proof.
Revised: Proof.

Original: The main idea is that a change in reward will only affect the empirical mean calculated in one episode, which is made private using the Laplace Mechanism and Lemma 1.
Revised: The main idea is that a change in reward will only affect the empirical mean calculated in one episode, which is made private using the Laplace Mechanism and Lemma 1.

Original: Since the actions are only calculated using the private empirical means, the algorithm is ϵ -global DP following the post-processing lemma.
Revised: Since the actions are only calculated using the private empirical means, the algorithm is ϵ -global DP following the post-processing lemma.

Original: We refer to Appendix D for a complete proof.
Revised: We refer to Appendix D for a complete proof.

Original: Now, to concretise an algorithm, we only need to explicit how the indexes are calculated.
Revised: Now, to concretise an algorithm, we only need to explicit how the indexes are calculated.

Original: In other words, we need to build a high-probability upper confidence bound on the true mean of an arm only using a private empirical mean.
Revised: In other words, we need to build a high-probability upper confidence bound on the true mean of an arm only using a private empirical mean.

Original: We instantiate the design details for AdaP - UCB and AdaP - KLUCB .
Revised: We instantiate the design details for AdaP - UCB and AdaP - KLUCB .

Original: AdaP - UCB and AdaP - KLUCB .
Revised: AdaP - UCB and AdaP - KLUCB .

Original: Let ˆ µ ℓa be the empirical mean reward of arm a computed using the samples collected between t ψ a ( ℓ ) and t ψ a ( ℓ )+1 − 1 .
Revised: Let ˆ µ ℓa be the empirical mean reward of arm a computed using the samples collected between t ψ a ( ℓ ) and t ψ a ( ℓ )+1 − 1 .

Original: For an episode ℓ , ψ a ( ℓ ) =
Revised: For an episode ℓ , ψ a ( ℓ ) =

Original: ℓ a is the last active
Revised: ℓ a is the last active

Original: Step t
Revised: Step t

Original: Arm
Revised: Arm

Original: Armrrtrtrrtrtrrrrtrr episode of arm a .
Revised: Armrrtrtrrtrtrrrrtrr episode of arm a .

Original: In Example 2, ψ 1 (4) = 2 and ψ 2 (4) = 3 .
Revised: In Example 2, ψ 1 (4) = 2 and ψ 2 (4) = 3 .

Original: Thus, due to the doubling of episode length, the empirical mean corresponds to 12 N a ( t ℓ − 1) samples of arm a .
Revised: Thus, due to the doubling of episode length, the empirical mean corresponds to 12 N a ( t ℓ − 1) samples of arm a .

Original: Since the rewards are in [0 , 1] , the private empirical mean as ˜ µ ℓa,ϵ
Revised: Since the rewards are in [0 , 1] , the private empirical mean as ˜ µ ℓa,ϵ

Original: = ˆ µ ℓa + Lap (cid:16)
Revised: = ˆ µ ℓa + Lap (cid:16)

Original: [Equation]
Revised: [Equation]

Original: (cid:17) satisfies ϵ -DP (Theorem 1).
Revised: (cid:17) satisfies ϵ -DP (Theorem 1).

Original: Now, we want to ensure that I ϵa ( t ℓ − 1 , α ) , computed using only ˜ µ ℓa,ϵ , is a high-probability upper bound on the true mean.
Revised: Now, we want to ensure that I ϵa ( t ℓ − 1 , α ) , computed using only ˜ µ ℓa,ϵ , is a high-probability upper bound on the true mean.

Original: Here, we introduce two specific indexes that satisfy this criterion.
Revised: Here, we introduce two specific indexes that satisfy this criterion.

Original: For AdaP - UCB :
Revised: For AdaP - UCB :

Original: [Equation]
Revised: [Equation]

Original: For AdaP - KLUCB :
Revised: For AdaP - KLUCB :

Original: [Equation]
Revised: [Equation]

Original: where ˘ µ ℓ,αa,ϵ =
Revised: where ˘ µ ℓ,αa,ϵ =

Original: Clip 0 , 1 (cid:16) ˜ µ ℓa,ϵ + α log( t
Revised: Clip 0 , 1 (cid:16) ˜ µ ℓa,ϵ + α log( t

Original: [Equation]
Revised: [Equation]

Original: ϵ 12 N a ( t ℓ − 1) (cid:17)
Revised: ϵ 12 N a ( t ℓ − 1) (cid:17)

Original: =
Revised: =

Original: min { max { 0 , ˜ µ ℓa,ϵ +
Revised: min { max { 0 , ˜ µ ℓa,ϵ +

Original: [Equation]
Revised: [Equation]

Original: × 12 N
Revised: × 12 N

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: } , 1 } is the private empirical mean clipped between zero and one.
Revised: } , 1 } is the private empirical mean clipped between zero and one.

Original: Theorem 7.
Revised: Theorem 7.

Original: For rewards in [0 , 1] , AdaP - UCB satisfies ϵ -global DP, and for α > 3 , it yields a regret
Revised: For rewards in [0 , 1] , AdaP - UCB satisfies ϵ -global DP, and for α > 3 , it yields a regret

Original: [Equation]
Revised: [Equation]

Original: Theorem 8.
Revised: Theorem 8.

Original: When the rewards are sampled from Bernoulli distributions, AdaP - KLUCB satisfies ϵ -global DP, and for α > 3 and constants C 1 ( α ) , C 2 > 0 , it yields a regret
Revised: When the rewards are sampled from Bernoulli distributions, AdaP - KLUCB satisfies ϵ -global DP, and for α > 3 and constants C 1 ( α ) , C 2 > 0 , it yields a regret

Original: [Equation]
Revised: [Equation]

Original: Both the upper bounds show that for low-privacy regime, the regrets of AdaP - UCB and AdaP - KLUCB are independent of ϵ , and in high-privacy regime, they depend on the coupled effect.
Revised: Both the upper bounds show that for low-privacy regime, the regrets of AdaP - UCB and AdaP - KLUCB are independent of ϵ , and in high-privacy regime, they depend on the coupled effect.

Original: In Appendix E.4, we also derive problem-independent or minimax regret upper bounds for AdaP - UCB and
Revised: In Appendix E.4, we also derive problem-independent or minimax regret upper bounds for AdaP - UCB and

Original: AdaP - KLUCB , which are O ( (cid:112) KT log( T ) + K log( T ) ϵ ) .
Revised: AdaP - KLUCB , which are O ( (cid:112) KT log( T ) + K log( T ) ϵ ) .

Original: Remark 1.
Revised: Remark 1.

Original: α controls the width of the optimistic confidence bound.
Revised: α controls the width of the optimistic confidence bound.

Original: Specifically, it dictates that the real mean is smaller than the optimistic index with high probability, i.e. with probability 1 − 1 t α at step t .
Revised: Specifically, it dictates that the real mean is smaller than the optimistic index with high probability, i.e. with probability 1 − 1 t α at step t .

Original: The requirement that α > 3 is due to our analysis of the algorithm.
Revised: The requirement that α > 3 is due to our analysis of the algorithm.

Original: To be specific, the requirement that α > 3 is needed to use a sum-integral inequality in the proof of Theorem 11.
Revised: To be specific, the requirement that α > 3 is needed to use a sum-integral inequality in the proof of Theorem 11.

Original: Since the dominant terms in the regret upper bounds of both AdaP - UCB and AdaP - KLUCB are multiplicative in α , making α smaller will tighten the bound.
Revised: Since the dominant terms in the regret upper bounds of both AdaP - UCB and AdaP - KLUCB are multiplicative in α , making α smaller will tighten the bound.

Original: We leave it for future work.
Revised: We leave it for future work.

Original: We refer to Section G.3 for a detailed discussion on choosing α .
Revised: We refer to Section G.3 for a detailed discussion on choosing α .

Original: Summary of Algorithm Design.
Revised: Summary of Algorithm Design.

Original: We propose three ingredients to design an ϵ -global DP version of a bandit algorithm
Revised: We propose three ingredients to design an ϵ -global DP version of a bandit algorithm

Original: (Algo. 1).
Revised: (Algo. 1).

Original: Firstly, add Laplace noise to make the empirical means private (Lemma 1).
Revised: Firstly, add Laplace noise to make the empirical means private (Lemma 1).

Original: Secondly, compute the index of an arm using the private empirical mean of rewards collected from the last active episode of that arm and forget all the samples obtained before that.
Revised: Secondly, compute the index of an arm using the private empirical mean of rewards collected from the last active episode of that arm and forget all the samples obtained before that.

Original: Thirdly, use adaptive episodes with doubling.
Revised: Thirdly, use adaptive episodes with doubling.

Original: At the beginning of an episode choose the arm with the highest index and play it double number of times than the length of the last active episode of the arm.
Revised: At the beginning of an episode choose the arm with the highest index and play it double number of times than the length of the last active episode of the arm.

Original:  Experimental Analysis
Revised:  Experimental Analysis

Original: In this section, we perform empirical evaluations to test two hypotheses: (i) AdaP - KLUCB is the most optimal algorithm among the existing bandit algorithms with ϵ -global DP, and (ii)
Revised: In this section, we perform empirical evaluations to test two hypotheses: (i) AdaP - KLUCB is the most optimal algorithm among the existing bandit algorithms with ϵ -global DP, and (ii)

Original: the
Revised: the

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: Efficiency in Performance.
Revised: Efficiency in Performance.

Original: First, we compare performances of AdaP - UCB and AdaP - KLUCB with those of DP - SE and DP - UCB .
Revised: First, we compare performances of AdaP - UCB and AdaP - KLUCB with those of DP - SE and DP - UCB .

Original: We set α = 3 .
Revised: We set α = 3 .

Original: 1 to comply with the regret upper bounds of AdaP - UCB and AdaP - KLUCB .
Revised: 1 to comply with the regret upper bounds of AdaP - UCB and AdaP - KLUCB .

Original: We assign γ = 0 .
Revised: We assign γ = 0 .

Original: 1 for DP - UCB and β = 1 /T for DP - SE .
Revised: 1 for DP - UCB and β = 1 /T for DP - SE .

Original: We implement all the algorithms in Python (version 3 . 8 ) and on an 8 core 64-bits Intel i5@1.6 GHz CPU.
Revised: We implement all the algorithms in Python (version 3 . 8 ) and on an 8 core 64-bits Intel i5@1.6 GHz CPU.

Original: We test the algorithms for Bernoulli bandits with 5 -arms and means { 0 . 75 , 0 . 625 , 0 .
Revised: We test the algorithms for Bernoulli bandits with 5 -arms and means { 0 . 75 , 0 . 625 , 0 .

Original: 375 , 0 . 25 } (as in Sajed and Sheffet (2019)).
Revised: 375 , 0 . 25 } (as in Sajed and Sheffet (2019)).

Original: We run each algorithm 20 times for a horizon T = 10 7 , and plot corresponding average and standard deviations of regrets in Figure 2.
Revised: We run each algorithm 20 times for a horizon T = 10 7 , and plot corresponding average and standard deviations of regrets in Figure 2.

Original: AdaP - KLUCB achieves the lowest regret followed by AdaP - UCB .
Revised: AdaP - KLUCB achieves the lowest regret followed by AdaP - UCB .

Original: Both of them achieve 10 times lower regret than the competing algorithms.
Revised: Both of them achieve 10 times lower regret than the competing algorithms.

Original: Impact of Privacy Regimes.
Revised: Impact of Privacy Regimes.

Original: In Figure 3, we plot regret of AdaP - KLUCB at T = 10 7 for a Bernoulli bandit with mean rewards { 0 . 8 , 0 . 1 , 0 . 1 , 0 . 1 , 0 . 1 } .
Revised: In Figure 3, we plot regret of AdaP - KLUCB at T = 10 7 for a Bernoulli bandit with mean rewards { 0 . 8 , 0 . 1 , 0 . 1 , 0 . 1 , 0 . 1 } .

Original: We plot the average regret over 20 runs as a function of the privacy budget ϵ ∈ [0 .
Revised: We plot the average regret over 20 runs as a function of the privacy budget ϵ ∈ [0 .

Original: As indicated by the theoretical regret lower bounds and upper bounds, the experimental performance of AdaP - KLUCB demonstrates two regimes: a high-privacy regime (for ϵ < 0 . 3 ), where the regret of AdaP - KLUCB depends on the privacy budget ϵ , and a low privacy regime (for ϵ > 0 . 3 ), where the regret of AdaP - KLUCB does not depend on ϵ .
Revised: As indicated by the theoretical regret lower bounds and upper bounds, the experimental performance of AdaP - KLUCB demonstrates two regimes: a high-privacy regime (for ϵ < 0 . 3 ), where the regret of AdaP - KLUCB depends on the privacy budget ϵ , and a low privacy regime (for ϵ > 0 . 3 ), where the regret of AdaP - KLUCB does not depend on ϵ .

Original: In brief, our experimental results validate that AdaP - KLUCB is the most optimal algorithm for stochastic bandits that satisfies ϵ -global DP, and performance of AdaP - KLUCB transits from highto low-privacy regimes, where its performance turns independent of the privacy budget ϵ .
Revised: In brief, our experimental results validate that AdaP - KLUCB is the most optimal algorithm for stochastic bandits that satisfies ϵ -global DP, and performance of AdaP - KLUCB transits from highto low-privacy regimes, where its performance turns independent of the privacy budget ϵ .

Original:  Conclusion and Future Works
Revised:  Conclusion and Future Works

Original: We revisit bandits with ϵ -global DP.
Revised: We revisit bandits with ϵ -global DP.

Original: We prove the minimax and problem-dependent regret lower bounds for stochastic and linear bandits indicating two regimes of hardness.
Revised: We prove the minimax and problem-dependent regret lower bounds for stochastic and linear bandits indicating two regimes of hardness.

Original: We propose a framework to design an ϵ -global DP version of an index-based optimistic bandit algorithm in a near-optimal fashion by utilising three ingredients: adding noise with Laplace Mechanism, using the empirical mean of rewards collected in the last active episode of an arm, and adapting episodes with doubling.
Revised: We propose a framework to design an ϵ -global DP version of an index-based optimistic bandit algorithm in a near-optimal fashion by utilising three ingredients: adding noise with Laplace Mechanism, using the empirical mean of rewards collected in the last active episode of an arm, and adapting episodes with doubling.

Original: One limitation of our analysis that the lower and upper bounds defer by multiplicative constants.
Revised: One limitation of our analysis that the lower and upper bounds defer by multiplicative constants.

Original: It would be a technical challenge to merge this gap.
Revised: It would be a technical challenge to merge this gap.

Original: Another future direction is to derive regret lower bounds for other variants of DP, namely ( ϵ, δ ) -DP and Rényi-DP (Mironov, 2017), especially in the structured bandits compatible with Gaussian noise injection and also to extend the proposed proof techniques in these settings.
Revised: Another future direction is to derive regret lower bounds for other variants of DP, namely ( ϵ, δ ) -DP and Rényi-DP (Mironov, 2017), especially in the structured bandits compatible with Gaussian noise injection and also to extend the proposed proof techniques in these settings.

Original: For the algorithm design, it would be interesting to see how the proposed ingredients generalise to linear and contextual bandits (Shariff and Sheffet, 2018), and
Revised: For the algorithm design, it would be interesting to see how the proposed ingredients generalise to linear and contextual bandits (Shariff and Sheffet, 2018), and

Original: Markov Decision Processes (Vietri et al., 2020).
Revised: Markov Decision Processes (Vietri et al., 2020).

Original: Acknowledgments and Disclosure of Funding
Revised: Acknowledgments and Disclosure of Funding

Original: This work is supported by the AI_PhD@Lille grant.
Revised: This work is supported by the AI_PhD@Lille grant.

Original: We also thank Philippe Preux for his support.
Revised: We also thank Philippe Preux for his support.

Original: Table
Revised: Table

Original: of Contents
Revised: of Contents

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: F
Revised: F

Original: Existing Technical Results and Definitions
Revised: Existing Technical Results and Definitions

Original: G Extended Experimental Analysis
Revised: G Extended Experimental Analysis

Original: G.1 Experimental Setup . .
Revised: G.1 Experimental Setup . .

Original: G.2 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Revised: G.2 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Original: [Table]
Revised: [Table]

Original: A The Canonical Model of Bandits
Revised: A The Canonical Model of Bandits

Original: We extend the general canonical model of bandits (Chapter 4, Lattimore and Szepesvári (2018)) with ϵ -global differential privacy.
Revised: We extend the general canonical model of bandits (Chapter 4, Lattimore and Szepesvári (2018)) with ϵ -global differential privacy.

Original: The canonical model with ϵ -global DP consists of a privacy-preserving policy π ϵ and an environment ν .
Revised: The canonical model with ϵ -global DP consists of a privacy-preserving policy π ϵ and an environment ν .

Original: The policy interacts with the environment up to a given time horizon
Revised: The policy interacts with the environment up to a given time horizon

Original: T to produce a history H T ≜ { ( A t , R t ) } Tt =1 .
Revised: T to produce a history H T ≜ { ( A t , R t ) } Tt =1 .

Original: The iterative steps of this interaction process are:
Revised: The iterative steps of this interaction process are:

Original: the probability of choosing an action A t = a at time t
Revised: the probability of choosing an action A t = a at time t

Original: is dictated only
Revised: is dictated only

Original: by
Revised: by

Original: the policy
Revised: the policy

Original: π ϵ ( a |H t − 1 ) , 2.
Revised: π ϵ ( a |H t − 1 ) , 2.

Original: the distribution of reward R t is P A t and is conditionally independent of the previous observed history H t − 1 .
Revised: the distribution of reward R t is P A t and is conditionally independent of the previous observed history H t − 1 .

Original: Let us formalise this interaction by defining an ϵ -global DP policy, the environment and the probability space produced by this interaction.
Revised: Let us formalise this interaction by defining an ϵ -global DP policy, the environment and the probability space produced by this interaction.

Original: Let T ∈ N be the horizon.
Revised: Let T ∈ N be the horizon.

Original: Let ν =
Revised: Let ν =

Original: ( P a : a ∈ [ K ]) a bandit instance with K arms.
Revised: ( P a : a ∈ [ K ]) a bandit instance with K arms.

Original: For each t ∈ [ T ] , let Ω t =
Revised: For each t ∈ [ T ] , let Ω t =

Original: ([ K ] × R ) t ⊂ R 2 t and F t =
Revised: ([ K ] × R ) t ⊂ R 2 t and F t =

Original: B (Ω t ) with B being the Borel set.
Revised: B (Ω t ) with B being the Borel set.

Original: Definition 3.
Revised: Definition 3.

Original: A policy π is a sequence ( π t )
Revised: A policy π is a sequence ( π t )

Original: Tt =1 , where π t is a probability kernel from (Ω t , F t ) to ([ K ] , 2 [ K ] ) .
Revised: Tt =1 , where π t is a probability kernel from (Ω t , F t ) to ([ K ] , 2 [ K ] ) .

Original: Since [ K ] is discrete, we adopt the convention that for i ∈
Revised: Since [ K ] is discrete, we adopt the convention that for i ∈

Original: [ K ] , π t ( i | a 1 , r 1 , . . .
Revised: [ K ] , π t ( i | a 1 , r 1 , . . .

Original: , a t − 1 , r t − 1 ) =
Revised: , a t − 1 , r t − 1 ) =

Original: π t ( { i } | a 1 , r 1 , . . .
Revised: π t ( { i } | a 1 , r 1 , . . .

Original: , a t − 1 , r t − 1 ) and for a sequence of actions a T ≜ [ a 1 , . .
Revised: , a t − 1 , r t − 1 ) and for a sequence of actions a T ≜ [ a 1 , . .

Original: . , a T ] and a sequence of rewards r T ≜ [ r 1 , . .
Revised: . , a T ] and a sequence of rewards r T ≜ [ r 1 , . .

Original: , r T ] : π ( a T | r T ) =
Revised: , r T ] : π ( a T | r T ) =

Original: T (cid:89) t = π t ( a t | a 1 , r 1 , . . .
Revised: T (cid:89) t = π t ( a t | a 1 , r 1 , . . .

Original: , a t − 1 , r t − 1 )
Revised: , a t − 1 , r t − 1 )

Original: A policy π ϵ is ϵ -global DP, if
Revised: A policy π ϵ is ϵ -global DP, if

Original: [Equation]
Revised: [Equation]

Original: T | r ′ T ) for every sequence of actions a T and every two neighbouring reward streams r T , r ′ T : ∃ j ∈ [1 , T ] such that r j ̸ = r ′
Revised: T | r ′ T ) for every sequence of actions a T and every two neighbouring reward streams r T , r ′ T : ∃ j ∈ [1 , T ] such that r j ̸ = r ′

Original: j
Revised: j

Original: and ∀ t ̸ =
Revised: and ∀ t ̸ =

Original: j r t = r ′ t .
Revised: j r t = r ′ t .

Original: Let λ be a σ -finite measure on ( R , B ( R )) for which P a is absolutely continuous with respect to λ for all a ∈ [ K ] .
Revised: Let λ be a σ -finite measure on ( R , B ( R )) for which P a is absolutely continuous with respect to λ for all a ∈ [ K ] .

Original: Let p a = dP a
Revised: Let p a = dP a

Original: /dλ be the Radon–Nikodym derivative of P a with respect to λ , which is a function p a : R → R such that (cid:82) B p a dλ =
Revised: /dλ be the Radon–Nikodym derivative of P a with respect to λ , which is a function p a : R → R such that (cid:82) B p a dλ =

Original: P a ( B ) for all B ∈ B ( R ) .
Revised: P a ( B ) for all B ∈ B ( R ) .

Original: Letting ρ be the counting measure with ρ ( B ) =
Revised: Letting ρ be the counting measure with ρ ( B ) =

Original: | B | , the density p νπ ϵ : Ω T → R can now be defined with respect to the product measure ( ρ × λ ) T by
Revised: | B | , the density p νπ ϵ : Ω T → R can now be defined with respect to the product measure ( ρ × λ ) T by

Original: [Equation]
Revised: [Equation]

Original: and P νπ ϵ be defined by
Revised: and P νπ ϵ be defined by

Original: [Equation]
Revised: [Equation]

Original: Hence (Ω T , F T , P νπ ϵ ) is a probability space over histories induced by the interaction between π ϵ and ν .
Revised: Hence (Ω T , F T , P νπ ϵ ) is a probability space over histories induced by the interaction between π ϵ and ν .

Original: We define also a marginal distribution over sequence of actions by
Revised: We define also a marginal distribution over sequence of actions by

Original: [Equation]
Revised: [Equation]

Original: and for all C ∈ P ([ K ] T ) ,
Revised: and for all C ∈ P ([ K ] T ) ,

Original: [Equation]
Revised: [Equation]

Original: Hence, ([ K ] T , P ([ K ] T ) , M νπ ϵ ) is a probability space over sequence of actions produced when π ϵ interacts with ν for T time-steps.
Revised: Hence, ([ K ] T , P ([ K ] T ) , M νπ ϵ ) is a probability space over sequence of actions produced when π ϵ interacts with ν for T time-steps.

Original: B Distinguishing Environments with Partial Information and Global DP
Revised: B Distinguishing Environments with Partial Information and Global DP

Original: In this section, we first revisit the Karwa-Vadhan Lemma (Lemma 6.1, (Karwa and Vadhan, 2017)) that bounds the multiplicative distance between marginal distributions induced by a differentially private mechanism, when the datasets are generated using two different distributions P and Q .
Revised: In this section, we first revisit the Karwa-Vadhan Lemma (Lemma 6.1, (Karwa and Vadhan, 2017)) that bounds the multiplicative distance between marginal distributions induced by a differentially private mechanism, when the datasets are generated using two different distributions P and Q .

Original: We generalise this result to the setting where the inputs are not identically distributed.
Revised: We generalise this result to the setting where the inputs are not identically distributed.

Original: We call this
Revised: We call this

Original: Sequential Karwa-Vadhan Lemma (Lemma 2) and apply it to upper bound the Kullback-Leibler (KL) divergence between the marginal distributions M νπ ϵ and M ν
Revised: Sequential Karwa-Vadhan Lemma (Lemma 2) and apply it to upper bound the Kullback-Leibler (KL) divergence between the marginal distributions M νπ ϵ and M ν

Original: ′ π ϵ , when π ϵ is an ϵ -global DP policy, and ν and ν ′ are two different environments (Theorem 10).
Revised: ′ π ϵ , when π ϵ is an ϵ -global DP policy, and ν and ν ′ are two different environments (Theorem 10).

Original: Karwa-Vadhan Lemma.
Revised: Karwa-Vadhan Lemma.

Original: Let P and Q be two distributions, and TV ( P ∥ Q ) be the total variation distance between these two distributions.
Revised: Let P and Q be two distributions, and TV ( P ∥ Q ) be the total variation distance between these two distributions.

Original: Let M be an ( ϵ, δ ) -differentially private mechanism that runs on the set of samples { x 1 , . . .
Revised: Let M be an ( ϵ, δ ) -differentially private mechanism that runs on the set of samples { x 1 , . . .

Original: , x T } .
Revised: , x T } .

Original: For any event E in M ’s output space,
Revised: For any event E in M ’s output space,

Original: M ( E | X 1 = x 1 , . . .
Revised: M ( E | X 1 = x 1 , . . .

Original: , X T = x T ) denotes the probability that M outputs an element in E given the input x 1 , . . .
Revised: , X T = x T ) denotes the probability that M outputs an element in E given the input x 1 , . . .

Original: , x T , and
Revised: , x T , and

Original: [Equation]
Revised: [Equation]

Original: is the marginal distribution induced by the DP mechanism when the data is generated from the distribution P .
Revised: is the marginal distribution induced by the DP mechanism when the data is generated from the distribution P .

Original: Theorem 9 (Lemma 6.1, Karwa and Vadhan (2017)) .
Revised: Theorem 9 (Lemma 6.1, Karwa and Vadhan (2017)) .

Original: If a mechanism M satisfies ( ϵ, δ ) -DP, then for every event E in the output space of M , the marginal distributions induced by distributions P and Q satisfy
Revised: If a mechanism M satisfies ( ϵ, δ ) -DP, then for every event E in the output space of M , the marginal distributions induced by distributions P and Q satisfy

Original: [Equation]
Revised: [Equation]

Original: where ϵ ′ ≜ (6 ϵT )TV ( P ∥ Q ) and δ ′ ≜ (4 e ϵ ′
Revised: where ϵ ′ ≜ (6 ϵT )TV ( P ∥ Q ) and δ ′ ≜ (4 e ϵ ′

Original: Tδ )TV ( P ∥ Q ) .
Revised: Tδ )TV ( P ∥ Q ) .

Original: We extend this result for the setting where the data is not identically distributed.
Revised: We extend this result for the setting where the data is not identically distributed.

Original: B.1 Sequential Karwa-Vadhan Lemma
Revised: B.1 Sequential Karwa-Vadhan Lemma

Original: Let { P 1 , . .
Revised: Let { P 1 , . .

Original: . , P T } and { Q 1 , . . .
Revised: . , P T } and { Q 1 , . . .

Original: , Q T } two sets of independent distributions.
Revised: , Q T } two sets of independent distributions.

Original: Given the samples X 1 , . . .
Revised: Given the samples X 1 , . . .

Original: , X T generated from the distributions P 1 , . .
Revised: , X T generated from the distributions P 1 , . .

Original: , P T , we define the corresponding marginal distribution induced by M as
Revised: , P T , we define the corresponding marginal distribution induced by M as

Original: [Equation]
Revised: [Equation]

Original: Lemma 2 (Sequential Karwa-Vadhan Lemma) .
Revised: Lemma 2 (Sequential Karwa-Vadhan Lemma) .

Original: If M is a mechanism satisfying ( ϵ, δ ) -DP, then for every event E in the output space of M , the marginal distributions induced by the two sets of independent distributions { P 1 , . .
Revised: If M is a mechanism satisfying ( ϵ, δ ) -DP, then for every event E in the output space of M , the marginal distributions induced by the two sets of independent distributions { P 1 , . .

Original: . , P T } and { Q 1 , . . .
Revised: . , P T } and { Q 1 , . . .

Original: , Q T } satisfy
Revised: , Q T } satisfy

Original: [Equation]
Revised: [Equation]

Original: where ϵ ′ =
Revised: where ϵ ′ =

Original: 6 ϵ (cid:80)
Revised: 6 ϵ (cid:80)

Original: Ti =1 TV ( P i ∥ Q i ) and δ ′ =
Revised: Ti =1 TV ( P i ∥ Q i ) and δ ′ =

Original: 4 e ϵ ′
Revised: 4 e ϵ ′

Original: δ (cid:80)
Revised: δ (cid:80)

Original: Ti =1 TV ( P i ∥ Q i )
Revised: Ti =1 TV ( P i ∥ Q i )

Original: Proof.
Revised: Proof.

Original: We extend the proof proposed by (Karwa and Vadhan, 2017) to the non-identical distribution setting.
Revised: We extend the proof proposed by (Karwa and Vadhan, 2017) to the non-identical distribution setting.

Original: The main observation is that the proof follows naturally if the data is generated from different distributions by just adapting the coupling to the case of different distributions.
Revised: The main observation is that the proof follows naturally if the data is generated from different distributions by just adapting the coupling to the case of different distributions.

Original: For completeness, we present the whole proof with all the adapted changes.
Revised: For completeness, we present the whole proof with all the adapted changes.

Original: We construct a coupling between (cid:78) Ti =1 P
Revised: We construct a coupling between (cid:78) Ti =1 P

Original: i and (cid:78)
Revised: i and (cid:78)

Original: Ti =1 Q i that allows us to control the hamming distance between samples generated from this distributions.
Revised: Ti =1 Q i that allows us to control the hamming distance between samples generated from this distributions.

Original: Let us denote p i ≜
Revised: Let us denote p i ≜

Original: TV ( P i ∥ Q i ) , F i ≜ max( P i − Q i , 0) , G i ≜ max( Q i − P
Revised: TV ( P i ∥ Q i ) , F i ≜ max( P i − Q i , 0) , G i ≜ max( Q i − P

Original: i , 0) , and C i ≜ min( P i , Q i ) .
Revised: i , 0) , and C i ≜ min( P i , Q i ) .

Original: It is easy to see that P i
Revised: It is easy to see that P i

Original: = F i + C i
Revised: = F i + C i

Original: and
Revised: and

Original: Q
Revised: Q

Original: i =
Revised: i =

Original: G i + C i .
Revised: G i + C i .

Original: Given the aforementioned notations, we consider the following algorithm to generate 2 T samples:
Revised: Given the aforementioned notations, we consider the following algorithm to generate 2 T samples:

Original: For i = 1 to T , generate H i from Bernoulli ( p i )
Revised: For i = 1 to T , generate H i from Bernoulli ( p i )

Original: (a) If H i = 1 ,
Revised: (a) If H i = 1 ,

Original: sample X i ∝ F i and X ′ i
Revised: sample X i ∝ F i and X ′ i

Original: ∝ G i (b)
Revised: ∝ G i (b)

Original: If H i = 0 , sample X i ∝ C i and set X ′ i
Revised: If H i = 0 , sample X i ∝ C i and set X ′ i

Original: =
Revised: =

Original: X i .
Revised: X i .

Original: Here X i ∝ F i means that X i is generated from a distribution defined by normalizing F i .
Revised: Here X i ∝ F i means that X i is generated from a distribution defined by normalizing F i .

Original: This construction satisfies the following properties:
Revised: This construction satisfies the following properties:

Original: [Equation]
Revised: [Equation]

Original: Now, we introduce the following shorthand for the marginal distributions at step h
Revised: Now, we introduce the following shorthand for the marginal distributions at step h

Original: [Equation]
Revised: [Equation]

Original: for j ∈ { 0 , 1 } and p ( h )
Revised: for j ∈ { 0 , 1 } and p ( h )

Original: =
Revised: =

Original: P ( H = h ) .
Revised: P ( H = h ) .

Original: For j ∈ { 0 , 1 } and any event E , we have, by definition,
Revised: For j ∈ { 0 , 1 } and any event E , we have, by definition,

Original: [Equation]
Revised: [Equation]

Original: Fact 1: For j ∈ { 0 , 1 } , m j ( h ) ≤ e ϵ m j
Revised: Fact 1: For j ∈ { 0 , 1 } , m j ( h ) ≤ e ϵ m j

Original: ( h − 1)
Revised: ( h − 1)

Original: + δ for h = 1 , . . .
Revised: + δ for h = 1 , . . .

Original: , T , and m 1 (0)
Revised: , T , and m 1 (0)

Original: =
Revised: =

Original: m 0 (0) .
Revised: m 0 (0) .

Original: We defer the proof of Fact 1 to the end of this proof.
Revised: We defer the proof of Fact 1 to the end of this proof.

Original: By Fact 1, for j ∈ { 0 , 1 } , we have
Revised: By Fact 1, for j ∈ { 0 , 1 } , we have

Original: [Equation]
Revised: [Equation]

Original: Now, we obtain
Revised: Now, we obtain

Original: [Equation]
Revised: [Equation]

Original: T (cid:89) i =
Revised: T (cid:89) i =

Original: [Equation]
Revised: [Equation]

Original: p
Revised: p

Original: [Equation]
Revised: [Equation]

Original: p
Revised: p

Original: [Equation]
Revised: [Equation]

Original: ϵ
Revised: ϵ

Original: [Equation]
Revised: [Equation]

Original: + δ e
Revised: + δ e

Original: [Equation]
Revised: [Equation]

Original: (cid:32)
Revised: (cid:32)

Original: [Equation]
Revised: [Equation]

Original: (cid:89)
Revised: (cid:89)

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: p
Revised: p

Original: [Equation]
Revised: [Equation]

Original: p
Revised: p

Original: [Equation]
Revised: [Equation]

Original: · e ϵ )
Revised: · e ϵ )

Original: [Equation]
Revised: [Equation]

Original: (cid:33) (9)
Revised: (cid:33) (9)

Original: The last equality holds due to that fact that for any t > 0 , E [ e tH ] = (cid:81)
Revised: The last equality holds due to that fact that for any t > 0 , E [ e tH ] = (cid:81)

Original: Ti =1 (1 − p
Revised: Ti =1 (1 − p

Original: i + p i · e t ) .
Revised: i + p i · e t ) .

Original: Similarly, we obtain
Revised: Similarly, we obtain

Original: [Equation]
Revised: [Equation]

Original: Combining inequalities 9 and 10, we get
Revised: Combining inequalities 9 and 10, we get

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: From Lemma 6.1 of (Karwa and Vadhan, 2017), we know that
Revised: From Lemma 6.1 of (Karwa and Vadhan, 2017), we know that

Original: [Equation]
Revised: [Equation]

Original: Thus,
Revised: Thus,

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: Substituting Equations (12) and (16) in Equation 11, we obtain
Revised: Substituting Equations (12) and (16) in Equation 11, we obtain

Original: [Equation]
Revised: [Equation]

Original: where ϵ ′ =
Revised: where ϵ ′ =

Original: 6 ϵ ( (cid:80)
Revised: 6 ϵ ( (cid:80)

Original: Ti
Revised: Ti

Original: =
Revised: =

Original: p
Revised: p

Original: i ) and δ ′ =
Revised: i ) and δ ′ =

Original: 4 e ϵ ′
Revised: 4 e ϵ ′

Original: δ ( (cid:80)
Revised: δ ( (cid:80)

Original: Ti =1 p i ) .
Revised: Ti =1 p i ) .

Original: Now, we prove Fact 1.
Revised: Now, we prove Fact 1.

Original: Fact 1.
Revised: Fact 1.

Original: For j ∈ { 0 , 1 } , m j ( h ) ≤ e ϵ m j ( h − 1) + δ for h = 1 , . .
Revised: For j ∈ { 0 , 1 } , m j ( h ) ≤ e ϵ m j ( h − 1) + δ for h = 1 , . .

Original: , T , and m 1 (0)
Revised: , T , and m 1 (0)

Original: =
Revised: =

Original: m 0 (0) .
Revised: m 0 (0) .

Original: Proof.
Revised: Proof.

Original: We prove the claim for j = 0 , the other case is similar.
Revised: We prove the claim for j = 0 , the other case is similar.

Original: First, let us introduce some notations.
Revised: First, let us introduce some notations.

Original: Fix a ( h 1 , . .
Revised: Fix a ( h 1 , . .

Original: . , h T ) ∈ { 0 , 1 } T .
Revised: . , h T ) ∈ { 0 , 1 } T .

Original: Let I ′
Revised: Let I ′

Original: ≜
Revised: ≜

Original: { i : h i = 1 } ,
Revised: { i : h i = 1 } ,

Original: J ≜ { i : h i = 0 } , and r be any fixed index in I ′ .
Revised: J ≜ { i : h i = 0 } , and r be any fixed index in I ′ .

Original: Let I = I ′ / { r } and consider the following partition of X into three parts:
Revised: Let I = I ′ / { r } and consider the following partition of X into three parts:

Original: [Equation]
Revised: [Equation]

Original: where X I is the vector X specified by the indices in I .
Revised: where X I is the vector X specified by the indices in I .

Original: By definition of the coupling, X I ∼ (cid:78)
Revised: By definition of the coupling, X I ∼ (cid:78)

Original: i
Revised: i

Original: [Equation]
Revised: [Equation]

Original: F i ≜ F I , X r ∼ F r , X J ∼ (cid:78) i ∈
Revised: F i ≜ F I , X r ∼ F r , X J ∼ (cid:78) i ∈

Original: J C i ≜ C J .
Revised: J C i ≜ C J .

Original: Now, let X ′ r ∼ C r and
Revised: Now, let X ′ r ∼ C r and

Original: [Equation]
Revised: [Equation]

Original: Also, let h ′ 1 , . .
Revised: Also, let h ′ 1 , . .

Original: , h ′ T be the binary indicators corresponding to X ′ .
Revised: , h ′ T be the binary indicators corresponding to X ′ .

Original: By construction, we have the following properties:
Revised: By construction, we have the following properties:

Original: h i
Revised: h i

Original: =
Revised: =

Original: h ′ i for all
Revised: h ′ i for all

Original: i ̸
Revised: i ̸

Original: =
Revised: =

Original: r 2.
Revised: r 2.

Original: h r = 1 and h r =3.
Revised: h r = 1 and h r =3.

Original: (cid:80)
Revised: (cid:80)

Original: Ti =1 h
Revised: Ti =1 h

Original: i = h and (cid:80)
Revised: i = h and (cid:80)

Original: Ti =1 h
Revised: Ti =1 h

Original: ′ i
Revised: ′ i

Original: = h −4.
Revised: = h −4.

Original: D j
Revised: D j

Original: ( X | H 1 = h 1 , . .
Revised: ( X | H 1 = h 1 , . .

Original: , H T = h T ) =
Revised: , H T = h T ) =

Original: P F I ( X I )
Revised: P F I ( X I )

Original: P F r ( X r ) P C J ( X J ) 5.
Revised: P F r ( X r ) P C J ( X J ) 5.

Original: D j ( X ′ | H 1 = h ′ 1 , . .
Revised: D j ( X ′ | H 1 = h ′ 1 , . .

Original: , H T = h ′ T ) =
Revised: , H T = h ′ T ) =

Original: P F I ( X I ) P C r ( X ′ r ) P C J ( X J )
Revised: P F I ( X I ) P C r ( X ′ r ) P C J ( X J )

Original: Thus
Revised: Thus

Original: [Equation]
Revised: [Equation]

Original: we obtain
Revised: we obtain

Original: [Equation]
Revised: [Equation]

Original: Taking expectations on both sides with respect to ( H 1 , . .
Revised: Taking expectations on both sides with respect to ( H 1 , . .

Original: , H T ) proves the claim.
Revised: , H T ) proves the claim.

Original: B.2 KL-divergence Decomposition with ϵ -global DP
Revised: B.2 KL-divergence Decomposition with ϵ -global DP

Original: The Sequential Karwa-Vadhan Lemma (Lemma 2) allows us to show the maximum KL-divergence induced in the distributions of actions by a global DP policy π ϵ .
Revised: The Sequential Karwa-Vadhan Lemma (Lemma 2) allows us to show the maximum KL-divergence induced in the distributions of actions by a global DP policy π ϵ .

Original: The upper bound allows us to show how different the final distributions over actions induced by a global DP policy are for two different environments.
Revised: The upper bound allows us to show how different the final distributions over actions induced by a global DP policy are for two different environments.

Original: Thus, in turn, it provides an information-theoretic limit on distinguishability of two environments if π ϵ is played.
Revised: Thus, in turn, it provides an information-theoretic limit on distinguishability of two environments if π ϵ is played.

Original: Theorem 10 (Upper Bound on KL-divergence for Bandits with ϵ -global DP) .
Revised: Theorem 10 (Upper Bound on KL-divergence for Bandits with ϵ -global DP) .

Original: When an ϵ -global DP policy π ϵ interacts with two bandit instances ν =
Revised: When an ϵ -global DP policy π ϵ interacts with two bandit instances ν =

Original: ( P a : a ∈ [ K ]) and ν ′ =
Revised: ( P a : a ∈ [ K ]) and ν ′ =

Original: ( P
Revised: ( P

Original: [Equation]
Revised: [Equation]

Original: a : a ∈ [ K ]) we have:
Revised: a : a ∈ [ K ]) we have:

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: We define the marginal over the sequence of actions induced by π ϵ for a given environment ν as
Revised: We define the marginal over the sequence of actions induced by π ϵ for a given environment ν as

Original: [Equation]
Revised: [Equation]

Original: Since π ϵ is ϵ -global DP
Revised: Since π ϵ is ϵ -global DP

Original: [Equation]
Revised: [Equation]

Original: using Lemma
Revised: using Lemma

Original: [Equation]
Revised: [Equation]

Original: we obtain
Revised: we obtain

Original: [Equation]
Revised: [Equation]

Original: for every action sequence ( a 1 , . . .
Revised: for every action sequence ( a 1 , . . .

Original: , a T ) ∈
Revised: , a T ) ∈

Original: [ K ] T .
Revised: [ K ] T .

Original: Thus,
Revised: Thus,

Original: [Equation]
Revised: [Equation]

Original: This lemma explicates how the distinguishability of two environments ν and ν ′ under π ϵ is dictated by a joint effect of global DP, in terms of the privacy budget ϵ , and the partial information available in bandits, in terms of the total variation distance between the rewards of the arms
Revised: This lemma explicates how the distinguishability of two environments ν and ν ′ under π ϵ is dictated by a joint effect of global DP, in terms of the privacy budget ϵ , and the partial information available in bandits, in terms of the total variation distance between the rewards of the arms

Original: E νπ ϵ (cid:104)(cid:80) Tt =1 TV (cid:0) P a t (cid:13)(cid:13) P ′ a t (cid:1)(cid:105) .
Revised: E νπ ϵ (cid:104)(cid:80) Tt =1 TV (cid:0) P a t (cid:13)(cid:13) P ′ a t (cid:1)(cid:105) .

Original: We leverage this lemma further to construct the minimax and problem-dependent regret lower bounds for stochastic and linear bandits with ϵ -global DP.
Revised: We leverage this lemma further to construct the minimax and problem-dependent regret lower bounds for stochastic and linear bandits with ϵ -global DP.

Original: C Lower Bounds on Regret: Stochastic and Linear Bandits with ϵ -global DP
Revised: C Lower Bounds on Regret: Stochastic and Linear Bandits with ϵ -global DP

Original: In order to prove the lower bounds, we adopt the general canonical bandit model introduced in
Revised: In order to prove the lower bounds, we adopt the general canonical bandit model introduced in

Original: Section A.
Revised: Section A.

Original: The high level idea of proving bandit lower bounds is selecting two problem instances that are similar (the policy cannot statistically distinguish between them) but conflicting (actions that may be good in one instance are not good for the other).
Revised: The high level idea of proving bandit lower bounds is selecting two problem instances that are similar (the policy cannot statistically distinguish between them) but conflicting (actions that may be good in one instance are not good for the other).

Original: Under ϵ -global differential privacy, a new source of "confusion" is added to the problem, i.e. any sequence of actions induced by neighbouring reward streams must be ϵ -indistinguishable.
Revised: Under ϵ -global differential privacy, a new source of "confusion" is added to the problem, i.e. any sequence of actions induced by neighbouring reward streams must be ϵ -indistinguishable.

Original: In the canonical bandit framework, this is expressed by our Theorem 10.
Revised: In the canonical bandit framework, this is expressed by our Theorem 10.

Original: In the following, we plug this upper bound on KL-divergences in the classic proofs of regret lower bounds in bandits Lattimore and Szepesvári (2018) to derive our minimax and problem-dependent regret lower bounds.
Revised: In the following, we plug this upper bound on KL-divergences in the classic proofs of regret lower bounds in bandits Lattimore and Szepesvári (2018) to derive our minimax and problem-dependent regret lower bounds.

Original: Notations.
Revised: Notations.

Original: Let Π be the set of all policies, and Π ϵ be the set of all ϵ -global DP policies.
Revised: Let Π be the set of all policies, and Π ϵ be the set of all ϵ -global DP policies.

Original: C.1 Stochastic Bandits: Minimax Lower Bound
Revised: C.1 Stochastic Bandits: Minimax Lower Bound

Original: Theorem 2 (Minimax lower bound) .
Revised: Theorem 2 (Minimax lower bound) .

Original: For any K > 1 and T ≥ K − 1 , and ϵ > 0 , the minimax regret of stochastic bandits with ϵ -global DP satisfies
Revised: For any K > 1 and T ≥ K − 1 , and ϵ > 0 , the minimax regret of stochastic bandits with ϵ -global DP satisfies

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: We denote the environment corresponding to the set of K -Gaussian reward distributions with unit variance and means µ ∈ R K as E K N (1) ≜ (cid:110)
Revised: We denote the environment corresponding to the set of K -Gaussian reward distributions with unit variance and means µ ∈ R K as E K N (1) ≜ (cid:110)

Original: ( N ( µ i , 1))
Revised: ( N ( µ i , 1))

Original: Ki =1 : µ = ( µ 1 , . .
Revised: Ki =1 : µ = ( µ 1 , . .

Original: . , µ K ) ∈ R K (cid:111) .
Revised: . , µ K ) ∈ R K (cid:111) .

Original: Since Π ϵ ⊂ Π , we can have that
Revised: Since Π ϵ ⊂ Π , we can have that

Original: [Equation]
Revised: [Equation]

Original: The second inequality is due to Theorem 15.2 in (Lattimore and Szepesvári, 2018).
Revised: The second inequality is due to Theorem 15.2 in (Lattimore and Szepesvári, 2018).

Original: Step 1:
Revised: Step 1:

Original: Choosing the ‘Hard-to-distinguish’ Environments.
Revised: Choosing the ‘Hard-to-distinguish’ Environments.

Original: First, we fix a policy π ϵ in Π ϵ .
Revised: First, we fix a policy π ϵ in Π ϵ .

Original: Let ∆ be a constant (to be specified later), and ν be a Gaussian bandit instance with unit variance and mean vector µ = (∆ , 0 , 0 , ..., 0) .
Revised: Let ∆ be a constant (to be specified later), and ν be a Gaussian bandit instance with unit variance and mean vector µ = (∆ , 0 , 0 , ..., 0) .

Original: To choose the second bandit instance, let i ≜ arg min a> 1 E ν,π ϵ [ N a ( T )] be the least played arm in expectation other than the optimal arm 1.
Revised: To choose the second bandit instance, let i ≜ arg min a> 1 E ν,π ϵ [ N a ( T )] be the least played arm in expectation other than the optimal arm 1.

Original: The second environment ν ′ is then chosen to be a Gaussian bandit instance with unit variance and mean vector µ ′ =
Revised: The second environment ν ′ is then chosen to be a Gaussian bandit instance with unit variance and mean vector µ ′ =

Original: (∆ , 0 , 0 , . . . 0 , 2∆ , 0 . . .
Revised: (∆ , 0 , 0 , . . . 0 , 2∆ , 0 . . .

Original: , 0) , where µ ′ j = µ j for every j except for µ ′ i
Revised: , 0) , where µ ′ j = µ j for every j except for µ ′ i

Original: = 2∆ .
Revised: = 2∆ .

Original: The first arm is optimal in ν and the arm i is optimal in ν ′ .
Revised: The first arm is optimal in ν and the arm i is optimal in ν ′ .

Original: Since T = E νπ ϵ
Revised: Since T = E νπ ϵ

Original: [ N 1 ( T )]
Revised: [ N 1 ( T )]

Original: + (cid:80) a> 1 E νπ ϵ
Revised: + (cid:80) a> 1 E νπ ϵ

Original: [ N a ( T )] ≥ ( K − 1)
Revised: [ N a ( T )] ≥ ( K − 1)

Original: E νπ ϵ
Revised: E νπ ϵ

Original: [ N i ( T )] , we observe that
Revised: [ N i ( T )] , we observe that

Original: [Equation]
Revised: [Equation]

Original: Step 2: From Lower Bounding Regret to Upper Bounding KL-divergence.
Revised: Step 2: From Lower Bounding Regret to Upper Bounding KL-divergence.

Original: Now by the classic regret decomposition and Markov Inequality 6, we get
Revised: Now by the classic regret decomposition and Markov Inequality 6, we get

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: Reg T
Revised: Reg T

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Let us define the event A ≜ { N 1 ( T ) ≤ T/ 2 } = { ( a 1 , a 2 , . .
Revised: Let us define the event A ≜ { N 1 ( T ) ≤ T/ 2 } = { ( a 1 , a 2 , . .

Original: . , a T ) : card ( { j : a j = 1 } ) ≤ T/ 2 } .
Revised: . , a T ) : card ( { j : a j = 1 } ) ≤ T/ 2 } .

Original: By applying the Bretagnolle–Huber inequality, we have:
Revised: By applying the Bretagnolle–Huber inequality, we have:

Original: [Equation]
Revised: [Equation]

Original: Step 3: KL-divergence Decomposition with ϵ -global DP.
Revised: Step 3: KL-divergence Decomposition with ϵ -global DP.

Original: Now, we apply Theorem 10 to upperbound the KL-Divergence between the marginals.
Revised: Now, we apply Theorem 10 to upperbound the KL-Divergence between the marginals.

Original: [Equation]
Revised: [Equation]

Original: since ν and ν ′ only differ in the arm i .
Revised: since ν and ν ′ only differ in the arm i .

Original: Finally, using Pinsker’s Inequality 9, we obtain
Revised: Finally, using Pinsker’s Inequality 9, we obtain

Original: [Equation]
Revised: [Equation]

Original: Step 4: Choosing the Worst ∆ .
Revised: Step 4: Choosing the Worst ∆ .

Original: Plugging back in the regret expression, we find
Revised: Plugging back in the regret expression, we find

Original: [Equation]
Revised: [Equation]

Original: By optimising for ∆ , we choose ∆ =
Revised: By optimising for ∆ , we choose ∆ =

Original: K − 1 6 ϵT .
Revised: K − 1 6 ϵT .

Original: We conclude the proof by lower bounding exp( − 1) with 48131 , and using 2 max( a, b ) ≥ a + b .
Revised: We conclude the proof by lower bounding exp( − 1) with 48131 , and using 2 max( a, b ) ≥ a + b .

Original: C.2 Stochastic Bandits: Problem-dependent Lower Bound
Revised: C.2 Stochastic Bandits: Problem-dependent Lower Bound

Original: Theorem 3 (Problem-dependent Regret Lower Bound) .
Revised: Theorem 3 (Problem-dependent Regret Lower Bound) .

Original: Let the environment E be a set of K reward distributions with finite means and a policy π ϵ ∈ Π cons ( E )
Revised: Let the environment E be a set of K reward distributions with finite means and a policy π ϵ ∈ Π cons ( E )

Original: ∩ Π ϵ be a consistent policy 8 over E satisfying ϵ -global DP .
Revised: ∩ Π ϵ be a consistent policy 8 over E satisfying ϵ -global DP .

Original: Then, for all ν =
Revised: Then, for all ν =

Original: ( P i )
Revised: ( P i )

Original: Ki =1 ∈ E , it holds that
Revised: Ki =1 ∈ E , it holds that

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Let µ a be the mean of the a -th arm in ν , t a = t inf ( P a , µ ∗ , M a ) and π ϵ ∈ Π cons ( E ) ∩ Π ϵ .
Revised: Let µ a be the mean of the a -th arm in ν , t a = t inf ( P a , µ ∗ , M a ) and π ϵ ∈ Π cons ( E ) ∩ Π ϵ .

Original: Since π ϵ is consistent, by (Theorem 16.2, Lattimore and Szepesvári (2018)), it holds that
Revised: Since π ϵ is consistent, by (Theorem 16.2, Lattimore and Szepesvári (2018)), it holds that

Original: [Equation]
Revised: [Equation]

Original: The theorem will follow by showing, for every suboptimal arm a :
Revised: The theorem will follow by showing, for every suboptimal arm a :

Original: [Equation]
Revised: [Equation]

Original: Fix a suboptimal arm a , and let α > 0 be an arbitrary constant.
Revised: Fix a suboptimal arm a , and let α > 0 be an arbitrary constant.

Original: Step 1:
Revised: Step 1:

Original: Choosing the ‘Hard-to-distinguish’ Environment.
Revised: Choosing the ‘Hard-to-distinguish’ Environment.

Original: Let ν ′ ≜ (cid:0) P ′ j (cid:1)
Revised: Let ν ′ ≜ (cid:0) P ′ j (cid:1)

Original: K j =1 ∈ E be a bandit with
Revised: K j =1 ∈ E be a bandit with

Original: P ′ j = P j for j ̸ = a and P ′ a ∈ M a be such that TV ( P a ∥ P ′ a ) ≤ t a + α and µ ( P ′ a ) > µ ∗ , which exists by the definition of t a .
Revised: P ′ j = P j for j ̸ = a and P ′ a ∈ M a be such that TV ( P a ∥ P ′ a ) ≤ t a + α and µ ( P ′ a ) > µ ∗ , which exists by the definition of t a .

Original: Let µ ′ ∈
Revised: Let µ ′ ∈

Original: R K be the vector of means of distributions of ν ′ .
Revised: R K be the vector of means of distributions of ν ′ .

Original: Step 2: From Lower Bounding Regret to Upper Bounding KL-divergence.
Revised: Step 2: From Lower Bounding Regret to Upper Bounding KL-divergence.

Original: For simplicity of notations, we use Reg T =
Revised: For simplicity of notations, we use Reg T =

Original: Reg T ( π ϵ , ν ) , Reg ′ T =
Revised: Reg T ( π ϵ , ν ) , Reg ′ T =

Original: Reg T ( π ϵ , ν ) , and A = { ( a 1 , a 2 , . .
Revised: Reg T ( π ϵ , ν ) , and A = { ( a 1 , a 2 , . .

Original: . , a T ) : card ( { j : a j = 1 } ) ≤ T/ 2 } .
Revised: . , a T ) : card ( { j : a j = 1 } ) ≤ T/ 2 } .

Original: Then, by regret decomposition and Markov Inequality 6, we obtain
Revised: Then, by regret decomposition and Markov Inequality 6, we obtain

Original: [Equation]
Revised: [Equation]

Original: Step 3: KL-divergence Decomposition with ϵ -global DP.
Revised: Step 3: KL-divergence Decomposition with ϵ -global DP.

Original: By Theorem 10 and the construction of the ‘hard-to-distinguish’ environments, we obtain
Revised: By Theorem 10 and the construction of the ‘hard-to-distinguish’ environments, we obtain

Original: [Equation]
Revised: [Equation]

Original: Step 4: Rearranging and taking the limit inferior.
Revised: Step 4: Rearranging and taking the limit inferior.

Original: Thus, we get
Revised: Thus, we get

Original: [Equation]
Revised: [Equation]

Original: Now, taking the limit inferior on both sides leads to lim inf
Revised: Now, taking the limit inferior on both sides leads to lim inf

Original: [Equation]
Revised: [Equation]

Original: The last equality follows from the definition of consistency, which says that for any p > 0 , there exists a constant C p such that for sufficiently large T , Reg T + Reg ′ T ≤ C p T p .
Revised: The last equality follows from the definition of consistency, which says that for any p > 0 , there exists a constant C p such that for sufficiently large T , Reg T + Reg ′ T ≤ C p T p .

Original: This property implies that
Revised: This property implies that

Original: [Equation]
Revised: [Equation]

Original: which gives the result since p > 0 was an arbitrary constant.
Revised: which gives the result since p > 0 was an arbitrary constant.

Original: We arrive at the claimed result by taking the limit as α tends to zero.
Revised: We arrive at the claimed result by taking the limit as α tends to zero.

Original: Remark 2.
Revised: Remark 2.

Original: For Bernoulli distributions, t a is equal to ∆ a , so the private lower bound simplifies to:
Revised: For Bernoulli distributions, t a is equal to ∆ a , so the private lower bound simplifies to:

Original: [Equation]
Revised: [Equation]

Original: Thus, our problem-dependent regret lower bound retrieves as a special case the lower bound found in (Shariff and Sheffet, 2018) and established for Bernoulli distributions of rewards.
Revised: Thus, our problem-dependent regret lower bound retrieves as a special case the lower bound found in (Shariff and Sheffet, 2018) and established for Bernoulli distributions of rewards.

Original: C.3 Stochastic Linear Bandits: Minimax Lower Bound
Revised: C.3 Stochastic Linear Bandits: Minimax Lower Bound

Original: Theorem 4 (Minimax Regret Lower Bound) .
Revised: Theorem 4 (Minimax Regret Lower Bound) .

Original: Let A =
Revised: Let A =

Original: [ − 1 , 1] d and Θ = (cid:8) − T − 1 / 2 , T − 1 / 2 (cid:9) d .
Revised: [ − 1 , 1] d and Θ = (cid:8) − T − 1 / 2 , T − 1 / 2 (cid:9) d .

Original: Then, there exists a vector θ ∈ Θ leading to a minimax regret of linear bandits with ϵ -global DP such that
Revised: Then, there exists a vector θ ∈ Θ leading to a minimax regret of linear bandits with ϵ -global DP such that

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Due to Theorem 24.1,(Lattimore and Szepesvári, 2018), it holds that for any policy class Π ,
Revised: Due to Theorem 24.1,(Lattimore and Szepesvári, 2018), it holds that for any policy class Π ,

Original: [Equation]
Revised: [Equation]

Original: Now, we focus on proving the ϵ -global DP part of the lower bound.
Revised: Now, we focus on proving the ϵ -global DP part of the lower bound.

Original: For θ, θ ′ ∈ Θ , let ν and ν
Revised: For θ, θ ′ ∈ Θ , let ν and ν

Original: ′ be the bandit instances corresponding resp.
Revised: ′ be the bandit instances corresponding resp.

Original: to θ and θ ′ .
Revised: to θ and θ ′ .

Original: We denote M θ =
Revised: We denote M θ =

Original: M ν,π ϵ and M θ ′ =
Revised: M ν,π ϵ and M θ ′ =

Original: M ν ′ ,π ϵ .
Revised: M ν ′ ,π ϵ .

Original: Let E θ and E θ ′ the expectations under M θ and M θ ′ respectively.
Revised: Let E θ and E θ ′ the expectations under M θ and M θ ′ respectively.

Original: Step 1: From Lower Bounding Regret to Upper Bounding KL-divergence We begin with
Revised: Step 1: From Lower Bounding Regret to Upper Bounding KL-divergence We begin with

Original: [Equation]
Revised: [Equation]

Original: In this derivation, the first equality holds because the optimal action satisfies a ∗
Revised: In this derivation, the first equality holds because the optimal action satisfies a ∗

Original: i = sign ( θ i ) for i ∈
Revised: i = sign ( θ i ) for i ∈

Original: [ d ] .
Revised: [ d ] .

Original: The first inequality follows from an observation that (sign ( θ i ) − A ti ) θ
Revised: The first inequality follows from an observation that (sign ( θ i ) − A ti ) θ

Original: i ≥ | θ i | I { sign ( A ti )
Revised: i ≥ | θ i | I { sign ( A ti )

Original: ̸ = sign ( θ i ) } .
Revised: ̸ = sign ( θ i ) } .

Original: The last inequality is a direct application of Markov’s inequality 6.
Revised: The last inequality is a direct application of Markov’s inequality 6.

Original: For i ∈ [ d ] and θ ∈ Θ , we define
Revised: For i ∈ [ d ] and θ ∈ Θ , we define

Original: [Equation]
Revised: [Equation]

Original: Now, let i ∈
Revised: Now, let i ∈

Original: [ d ] and θ ∈ Θ be fixed.
Revised: [ d ] and θ ∈ Θ be fixed.

Original: Also, let θ ′
Revised: Also, let θ ′

Original: j
Revised: j

Original: = θ j for j ̸ =
Revised: = θ j for j ̸ =

Original: i and θ ′ i = − θ i .
Revised: i and θ ′ i = − θ i .

Original: Then, by the
Revised: Then, by the

Original: Bretagnolle-Huber inequality,
Revised: Bretagnolle-Huber inequality,

Original: [Equation]
Revised: [Equation]

Original: Step 2: KL-divergence Decomposition with ϵ -global DP.
Revised: Step 2: KL-divergence Decomposition with ϵ -global DP.

Original: From Theorem 10, we obtain that
Revised: From Theorem 10, we obtain that

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Here, the second inequality is a consequence of Pinsker’s inequality (Lemma 9).
Revised: Here, the second inequality is a consequence of Pinsker’s inequality (Lemma 9).

Original: The third inequality is due to Cauchy-Scwhartz.

Original: The last inequality holds true because A t ∈ [ − 1 , 1] d and θ, θ ′ ∈ (cid:8) − T − 1 / 2 , T − 1 / 2 (cid:9) d
Revised: The last inequality holds true because A t ∈ [ − 1 , 1] d and θ, θ ′ ∈ (cid:8) − T − 1 / 2 , T − 1 / 2 (cid:9) d

Original: Step 3:
Revised: Step 3:

Original: Choosing the ‘Hard-to-distinguish’ θ .
Revised: Choosing the ‘Hard-to-distinguish’ θ .

Original: We already have that
Revised: We already have that

Original: [Equation]
Revised: [Equation]

Original: Now, we apply an ‘averaging hammer’ over all θ ∈ Θ , such that | Θ | = 2 d , to obtain(cid:88)
Revised: Now, we apply an ‘averaging hammer’ over all θ ∈ Θ , such that | Θ | = 2 d , to obtain

Original: [Equation]
Revised: [Equation]

Original: This implies that there exists a θ ∈ Θ such that (cid:80) di
Revised: This implies that there exists a θ ∈ Θ such that (cid:80) di

Original: =1 p θ,i ≥ d exp( − 6 ϵd √ T ) / 4 .
Revised: =1 p θ,i ≥ d exp( − 6 ϵ √ T ) / 4 .

Original: Step 4: Plugging Back θ in the Regret Decomposition.
Revised: Step 4: Plugging Back θ in the Regret Decomposition.

Original: With this choice of θ , we conclude that
Revised: With this choice of θ , we conclude that

Original: [Equation]
Revised: [Equation]

Original: C.4 Stochastic Linear Bandits: Problem-dependent Lower Bound
Revised: C.4 Stochastic Linear Bandits: Problem-dependent Lower Bound

Original: Theorem 5 (Problem-dependent Regret Lower Bound) .
Revised: Theorem 5 (Problem-dependent Regret Lower Bound) .

Original: Let A ⊂ R d be a finite set spanning R d and θ ∈ R
Revised: Let A ⊂ R d be a finite set spanning R d and θ ∈ R

Original: d be such that there is a unique optimal action.
Revised: d be such that there is a unique optimal action.

Original: Then, any consistent and ϵ -global DP bandit algorithm π ϵ satisfies
Revised: Then, any consistent and ϵ -global DP bandit algorithm π ϵ satisfies

Original: [Equation]
Revised: [Equation]

Original: where the structural distinguishability gap is the solution of a constraint optimisation
Revised: where the structural distinguishability gap is the solution of a constraint optimisation

Original: [Equation]
Revised: [Equation]

Revised: α

Revised: [Equation]

Original:  , such that ∥ 
Revised: a , such that ∥ a ∥

Original: [Equation]
Revised: [Equation]

Original: for all a ∈ A with ∆ a > 0 , H α
Revised: for all a ∈ A with ∆ a > 0 , H α

Original: = (cid:80) a ∈A α ( a ) aa ⊤ , and an arm-structure dependent constant ρ a ( A ) .
Revised: = (cid:80) a ∈A α ( a ) aa ⊤ , and an arm-structure dependent constant ρ a ( A ) .

Original: Proof.
Revised: Proof.

Original: Let a ∗ = argmax a ∈A ⟨ a, θ ⟩ be the optimal action, which we assumed to be unique.
Revised: Let a ∗ = argmax a ∈A ⟨ a, θ ⟩ be the optimal action, which we assumed to be unique.

Original: By Theorem 25.1, Lattimore and Szepesvári (2018),
Revised: By Theorem 25.1, Lattimore and Szepesvári (2018),

Original: [Equation]
Revised: [Equation]

Original: Let M and M ′ be the measures on the sequence of outcomes
Revised: Let M and M ′ be the measures on the sequence of outcomes

Original: A 1 , . . .
Revised: A 1 , . . .

Original: A T induced by θ and θ ′ respectively.
Revised: A T induced by θ and θ ′ respectively.

Original: Let E [ · ] and E ′ [ · ] be the expectation operators of M and M ′ , respectively.
Revised: Let E [ · ] and E ′ [ · ] be the expectation operators of M and M ′ , respectively.

Original: Step 1:
Revised: Step 1:

Original: Choosing the ‘Hard to distinguish’ θ ′ .
Revised: Choosing the ‘Hard to distinguish’ θ ′ .

Original: Let θ ′ ∈
Revised: Let θ ′ ∈

Original: R
Revised: R

Original: d be an alternative parameter to be chosen subsequently.
Revised: d be an alternative parameter to be chosen subsequently.

Original: We follow the usual plan of choosing θ ′ to be close to θ , but also ensuring that the optimal action in the bandit determined by θ ′ is not a ∗ .
Revised: We follow the usual plan of choosing θ ′ to be close to θ , but also ensuring that the optimal action in the bandit determined by θ ′ is not a ∗ .

Original: Let ∆ min = min { ∆ a : a ∈ A , ∆ a > 0 } , α ∈ (0 , ∆ min ) and H be a positive definite matrix (to be chosen later) such that ∥ a − a ∗ ∥ 2 H > 0 .
Revised: Let ∆ min = min { ∆ a : a ∈ A , ∆ a > 0 } , α ∈ (0 , ∆ min ) and H be a positive definite matrix (to be chosen later) such that ∥ a − a ∗ ∥ 2 H > 0 .

Original: Given this setting, we define
Revised: Given this setting, we define

Original: [Equation]
Revised: [Equation]

Original: which is chosen such that ⟨ a − a ∗ , θ ′ ⟩ = ⟨
Revised: which is chosen such that ⟨ a − a ∗ , θ ′ ⟩ = ⟨

Original: [Equation]
Revised: [Equation]

Original: This means that a ∗ is α -suboptimal for the environment corresponding to θ ′ .
Revised: This means that a ∗ is α -suboptimal for the environment corresponding to θ ′ .

Original: Step 2: From Lower Bounding Regret to Upper Bounding KL-divergence.
Revised: Step 2: From Lower Bounding Regret to Upper Bounding KL-divergence.

Original: For simiplicity, we abbreviate Reg T =
Revised: For simiplicity, we abbreviate Reg T =

Original: Reg T ( A , θ ) and Reg ′ T =
Revised: Reg T ( A , θ ) and Reg ′ T =

Original: Reg T ( A , θ ′ ) .
Revised: Reg T ( A , θ ′ ) .

Original: Then, by applying the classic regret decomposition and Markov’s inequality 6, we obtain
Revised: Then, by applying the classic regret decomposition and Markov’s inequality 6, we obtain

Original: [Equation]
Revised: [Equation]

Original: Since a ∗ is α -suboptimal in bandit θ ′ , it implies that
Revised: Since a ∗ is α -suboptimal in bandit θ ′ , it implies that

Original: [Equation]
Revised: [Equation]

Original: Now, Bretagnolle–Huber inequality implies that
Revised: Now, Bretagnolle–Huber inequality implies that

Original: [Equation]
Revised: [Equation]

Original: Step 3: KL-divergence Decomposition with ϵ -global DP.
Revised: Step 3: KL-divergence Decomposition with ϵ -global DP.

Original: By Equation 18, we have that
Revised: By Equation 18, we have that

Original: [Equation]
Revised: [Equation]

Original: where we define
Revised: where we define

Original: [Equation]
Revised: [Equation]

Original: Thus, after re-arrangement, we get
Revised: Thus, after re-arrangement, we get

Original: [Equation]
Revised: [Equation]

Original: Step 4: Choosing H and Taking the Limit.
Revised: Step 4: Choosing H and Taking the Limit.

Original: The definition of consistency means that Reg T and
Revised: The definition of consistency means that Reg T and

Original: Reg ′ T are both sub-linear in T .
Revised: Reg ′ T are both sub-linear in T .

Original: This implies that the second term in Equation (20) tends to zero for large T .
Revised: This implies that the second term in Equation (20) tends to zero for large T .

Original: Thus, by tending T to ∞ and α to zero, we obtain
Revised: Thus, by tending T to ∞ and α to zero, we obtain

Original: [Equation]
Revised: [Equation]

Original: We now choose H to be a cluster point of the sequence (cid:0) ¯ G − 1 T / (cid:13)(cid:13) ¯ G − 1 T (cid:13)(cid:13)(cid:1) T ∈ S where (cid:13)(cid:13)
Revised: We now choose H to be a cluster point of the sequence (cid:0) ¯ G − 1 T / (cid:13)(cid:13) ¯ G − 1 T (cid:13)(cid:13)(cid:1) T ∈ S where (cid:13)(cid:13)

Original: ¯ G − 1 T (cid:13)(cid:13) is the spectral norm of the matrix ¯ G − 1 T .
Revised: ¯ G − 1 T (cid:13)(cid:13) is the spectral norm of the matrix ¯ G − 1 T .

Original: Fact 2 :
Revised: Fact 2 :

Original: For this choice of H ,
Revised: For this choice of H ,

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: Finally,
Revised: Finally,

Original: [Equation]
Revised: [Equation]

Original: Combined with Equation 19, we get that
Revised: Combined with Equation 19, we get that

Original: [Equation]
Revised: [Equation]

Original: Using that
Revised: Using that

Original: [Equation]
Revised: [Equation]

Original: from Theorem 25.1, Lattimore and Szepesvári (2018), we get that
Revised: from Theorem 25.1, Lattimore and Szepesvári (2018), we get that

Original: [Equation]
Revised: [Equation]

Original: Step 5: Getting Back to the Regret.
Revised: Step 5: Getting Back to the Regret.

Original: We conclude using the same steps as in the Corollary 2 (Lattimore and Szepesvari, 2017).
Revised: We conclude using the same steps as in the Corollary 2 (Lattimore and Szepesvari, 2017).

Original: Now, we prove Fact 2.
Revised: Now, we prove Fact 2.

Original: Fact 2.
Revised: Fact 2.

Original: If H is a cluster point of the sequence (cid:0) ¯ G − 1 T / (cid:13)(cid:13) ¯ G − 1 T (cid:13)(cid:13)(cid:1) T ∈ S and (cid:13)(cid:13) ¯ G − 1 T (cid:13)(cid:13) is the spectral norm of the matrix ¯ G − 1 T , then the following inequality holds true:
Revised: If H is a cluster point of the sequence (cid:0) ¯ G − 1 T / (cid:13)(cid:13) ¯ G − 1 T (cid:13)(cid:13)(cid:1) T ∈ S and (cid:13)(cid:13) ¯ G − 1 T (cid:13)(cid:13) is the spectral norm of the matrix ¯ G − 1 T , then the following inequality holds true:

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: − a ∗ ) (cid:12)(cid:12)
Revised: − a ∗ ) (cid:12)(cid:12)

Original: ∥ a j ∥
Revised: ∥ a j ∥

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: We let S be a subset so that ¯ G − 1 T / (cid:13)(cid:13) ¯ G − 1 T (cid:13)(cid:13) converges to H on T ∈ S .
Revised: We let S be a subset so that ¯ G − 1 T / (cid:13)(cid:13) ¯ G − 1 T (cid:13)(cid:13) converges to H on T ∈ S .

Original: Then, lim inf T →∞
Revised: Then, lim inf T →∞

Original: [Equation]
Revised: [Equation]

Original: =
Revised: =

Original: [Equation]
Revised: [Equation]

Original: Let j be such that ∥ a j ∥ ̸ = 0 .
Revised: Let j be such that ∥ a j ∥ ̸ = 0 .

Original: Now, we aim to upper bound the term (cid:12)(cid:12)
Revised: Now, we aim to upper bound the term (cid:12)(cid:12)

Original: [Equation]
Revised: [Equation]

Original: (cid:12)(cid:12)
Revised: (cid:12)(cid:12)

Original: First, we decompose a − a ∗ into two orthogonal components, which are aligned and orthogonal to a j respectively.
Revised: First, we decompose a − a ∗ into two orthogonal components, which are aligned and orthogonal to a j respectively.

Original: [Equation]
Revised: [Equation]

Original: where a ⊤ j b j = 0 and α j = a Tj ( a − a ∗ ) ∥ a j ∥ 2 .
Revised: where a ⊤ j b j = 0 and α j = a Tj ( a − a ∗ ) ∥ a j ∥ 2 .

Original: On the other hand, we have that
Revised: On the other hand, we have that

Original: [Equation]
Revised: [Equation]

Original: Since
Revised: Since

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: only the component of a − a ∗ in the direction of a j matters in the dot product a Tj ¯ G − 1 T ( a − a ∗ ) .
Revised: only the component of a − a ∗ in the direction of a j matters in the dot product a Tj ¯ G − 1 T ( a − a ∗ ) .

Original: Thus,
Revised: Thus,

Original: [Equation]
Revised: [Equation]

Original: Consequently, lim
Revised: Consequently, lim

Original: [Equation]
Revised: [Equation]

Original: Example 3 ( ρ a ( A ) for an orthogonal set of arms) .
Revised: Example 3 ( ρ a ( A ) for an orthogonal set of arms) .

Original: If the action space is the orthogonal basis, then ρ a ( A ) = 2 , because:
Revised: If the action space is the orthogonal basis, then ρ a ( A ) = 2 , because:

Original: [Equation]
Revised: [Equation]

Original:  and:
Revised:  and:

Original: [Equation]
Revised: [Equation]

Original: so:
Revised: so:

Original: [Equation]
Revised: [Equation]

Original: D Privacy Analysis of Algorithm
Revised: D Privacy Analysis of Algorithm

Original: In this section, we prove that any bandit algorithm designed using the framework of Algorithmsatisfies ϵ -global DP.
Revised: In this section, we prove that any bandit algorithm designed using the framework of Algorithmsatisfies ϵ -global DP.

Original: We establish the claim by proving ϵ -global DP for the set of private indices computed in Algorithm 1 and the final result is a consequence of the post-processing property of DP (Lemma 5).
Revised: We establish the claim by proving ϵ -global DP for the set of private indices computed in Algorithm 1 and the final result is a consequence of the post-processing property of DP (Lemma 5).

Original: Lemma 1 (Privacy of the ( l + 1) -means Computed in Algorithm 1) .
Revised: Lemma 1 (Privacy of the ( l + 1) -means Computed in Algorithm 1) .

Original: Let us define the private empirical mean of the rewards between steps i and j ( i < j ) as
Revised: Let us define the private empirical mean of the rewards between steps i and j ( i < j ) as

Original: [Equation]
Revised: [Equation]

Original: If 1 < t 1 < · · · < t ℓ < T and r t ∈ [0 , 1] , the mechanism g ϵ mapping the sequence of rewards ( r 1 , r 2 , . .
Revised: If 1 < t 1 < · · · < t ℓ < T and r t ∈ [0 , 1] , the mechanism g ϵ mapping the sequence of rewards ( r 1 , r 2 , . .

Original: , r T − 1 , r T ) to ( ℓ + 1) -private empirical means ( f ϵ { r 1 , . .
Revised: , r T − 1 , r T ) to ( ℓ + 1) -private empirical means ( f ϵ { r 1 , . .

Original: , r t 1 − 1 } , f ϵ { r t 1 , . . .
Revised: , r t 1 − 1 } , f ϵ { r t 1 , . . .

Original: , r t 2 − 1 } , . . .
Revised: , r t 2 − 1 } , . . .

Original: , f ϵ { r t ℓ − 1 , . . . , r t ℓ − 1 } , f ϵ { r t ℓ , . . .
Revised: , f ϵ { r t ℓ − 1 , . . . , r t ℓ − 1 } , f ϵ { r t ℓ , . . .

Original: , r T } ) satisfies ϵ -DP.
Revised: , r T } ) satisfies ϵ -DP.

Original: Proof.
Revised: Proof.

Original: Let r T ≜ ( r 1 , . .
Revised: Let r T ≜ ( r 1 , . .

Original: , r T ) and r ′ T ≜ ( r ′ 1 , . . .
Revised: , r T ) and r ′ T ≜ ( r ′ 1 , . . .

Original: , r ′ T ) be two neighbouring reward sequences in [0,1].
Revised: , r ′ T ) be two neighbouring reward sequences in [0,1].

Original: This implies that ∃ j ∈ [1 , T ] such that r j ̸ = r ′
Revised: This implies that ∃ j ∈ [1 , T ] such that r j ̸ = r ′

Original: j
Revised: j

Original: and ∀ t ̸ = j , r t = r ′ t .
Revised: and ∀ t ̸ = j , r t = r ′ t .

Original: Let ℓ ′ be such that t ℓ ′ ≤ j ≤ t ℓ ′ +1 − 1 , and follows the convention that t 0 = 1 and t ℓ +1 = T + 1 .
Revised: Let ℓ ′ be such that t ℓ ′ ≤ j ≤ t ℓ ′ +1 − 1 , and follows the convention that t 0 = 1 and t ℓ +1 = T + 1 .

Original: Let µ ≜ ( µ 0 , . .
Revised: Let µ ≜ ( µ 0 , . .

Original: . , µ ℓ ) a fixed sequence of outcomes obtained using Equation (21).
Revised: . , µ ℓ ) a fixed sequence of outcomes obtained using Equation (21).

Original: Then,
Revised: Then,

Original: [Equation]
Revised: [Equation]

Original: where the last inequality holds true because f ϵ satisfies ϵ -DP following Theorem 1.
Revised: where the last inequality holds true because f ϵ satisfies ϵ -DP following Theorem 1.

Original: Theorem 6 ( ϵ -global DP for Algorithm 1) .
Revised: Theorem 6 ( ϵ -global DP for Algorithm 1) .

Original: For any index I ϵa computed using the private empirical mean of the rewards collected in the last active episode of arm a , Algorithm 1 satisfies ϵ -global DP.
Revised: For any index I ϵa computed using the private empirical mean of the rewards collected in the last active episode of arm a , Algorithm 1 satisfies ϵ -global DP.

Original: Proof.
Revised: Proof.

Original: Fix two neighboring reward streams r T = { r 1 , . . .
Revised: Fix two neighboring reward streams r T = { r 1 , . . .

Original: , r T } and r ′ T = { r ′ 1 , . . .
Revised: , r T } and r ′ T = { r ′ 1 , . . .

Original: , r ′ T } .
Revised: , r ′ T } .

Original: This implies that ∃ j ∈ [1 , T ] such that r j ̸ = r ′
Revised: This implies that ∃ j ∈ [1 , T ] such that r j ̸ = r ′

Original: j
Revised: j

Original: and ∀ t ̸ = j , r t = r ′ t .
Revised: and ∀ t ̸ = j , r t = r ′ t .

Original: We also fix a sequence of actions a T = { a 1 , . .
Revised: We also fix a sequence of actions a T = { a 1 , . .

Original: . , a T } .
Revised: . , a T } .

Original: We want to show that: Pr ( π ( r T ) =
Revised: We want to show that: Pr ( π ( r T ) =

Original: a T ) ≤ e ϵ Pr ( π ( r ′ T )
Revised: a T ) ≤ e ϵ Pr ( π ( r ′ T )

Original: =
Revised: =

Original: a T ) .
Revised: a T ) .

Original: The main idea is that the change of reward in the j -th reward only affects the empirical mean computed in one episode, which is made private using the Laplace Mechanism and Lemma 1.
Revised: The main idea is that the change of reward in the j -th reward only affects the empirical mean computed in one episode, which is made private using the Laplace Mechanism and Lemma 1.

Original: • Since r j − 1 = r ′ j − 1 , Pr ( π ( r j − 1 ) =
Revised: • Since r j − 1 = r ′ j − 1 , Pr ( π ( r j − 1 ) =

Original: a j − 1 )
Revised: a j − 1 )

Original: =
Revised: =

Original: Pr ( π ( r ′ j − 1 ) =
Revised: Pr ( π ( r ′ j − 1 ) =

Original: a j − 1 ) .
Revised: a j − 1 ) .

Original: • Let t ℓ ≤ j < t ℓ +1 and t ℓ ′ ≤ j < t ℓ ′
Revised: • Let t ℓ ≤ j < t ℓ +1 and t ℓ ′ ≤ j < t ℓ ′

Original: +1 be the episodes corresponding to the j th reward in r T and r ′ T respectively.
Revised: +1 be the episodes corresponding to the j th reward in r T and r ′ T respectively.

Original: Since r j − 1 = r ′ j − 1 , we get that ℓ = ℓ ′ .
Revised: Since r j − 1 = r ′ j − 1 , we get that ℓ = ℓ ′ .

Original: Thus, Pr ( π ( r t ℓ +1 ) = a t ℓ +1 ) =
Revised: Thus, Pr ( π ( r t ℓ +1 ) = a t ℓ +1 ) =

Original: Pr ( π ( r ′ t ℓ +1 ) =
Revised: Pr ( π ( r ′ t ℓ +1 ) =

Original: a t ℓ +1 ) .
Revised: a t ℓ +1 ) .

Original: • Let ˜ µ ℓa j ,ϵ and ˜ µ ′ ℓa,ϵ be the private means of arm a j computed in the episode [ t ℓ , t ℓ +1 ] , by the Laplace mechanism, for every interval
Revised: • Let ˜ µ ℓa j ,ϵ and ˜ µ ′ ℓa,ϵ be the private means of arm a j computed in the episode [ t ℓ , t ℓ +1 ] , by the Laplace mechanism, for every interval

Original: I ∈ R , Pr (˜ µ ℓa,ϵ ∈ I ) ≤ e ϵ Pr (˜ µ ′ ℓa,ϵ ∈
Revised: I ∈ R , Pr (˜ µ ℓa,ϵ ∈ I ) ≤ e ϵ Pr (˜ µ ′ ℓa,ϵ ∈

Original: I ) .
Revised: I ) .

Original: • Finally, since { r j +1 , . . .
Revised: • Finally, since { r j +1 , . . .

Original: , r T } = { r ′ j +1 , . . .
Revised: , r T } = { r ′ j +1 , . . .

Original: , r ′ T } , Pr ( π ( r T ) =
Revised: , r ′ T } , Pr ( π ( r T ) =

Original: a T | ˜ µ ℓa,ϵ ∈ I ) =
Revised: a T | ˜ µ ℓa,ϵ ∈ I ) =

Original: Pr ( π ( r ′ T )
Revised: Pr ( π ( r ′ T )

Original: = a T | ˜ µ ′ ℓa,ϵ ∈ I )
Revised: = a T | ˜ µ ′ ℓa,ϵ ∈ I )

Original: Now, we conclude the argument by using a chain rule.
Revised: Now, we conclude the argument by using a chain rule.

Original: Since Theorem 6 holds for any index-based bandit algorithm that uses only private empirical means of rewards (Equation (21)) of the last active episode to compute the indices, it also implies that
Revised: Since Theorem 6 holds for any index-based bandit algorithm that uses only private empirical means of rewards (Equation (21)) of the last active episode to compute the indices, it also implies that

Original: AdaP - UCB and AdaP - KLUCB satisfy ϵ -global DP.
Revised: AdaP - UCB and AdaP - KLUCB satisfy ϵ -global DP.

Original: E Upper Bounds on Regret: AdaP - UCB and AdaP - KLUCB E.1 Concentration Inequalities
Revised: E Upper Bounds on Regret: AdaP - UCB and AdaP - KLUCB E.1 Concentration Inequalities

Original: Lemma 3.
Revised: Lemma 3.

Original: Assume that ( X i ) 1 ≤ i ≤ n are iid random variables in [0 , 1] , with E ( X i ) = µ .
Revised: Assume that ( X i ) 1 ≤ i ≤ n are iid random variables in [0 , 1] , with E ( X i ) = µ .

Original: Then, for any δ ≥ 0 ,
Revised: Then, for any δ ≥ 0 ,

Original: P
Revised: P

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: where ˆ µ n = 1 n (cid:80) nt =1 X t
Revised: where ˆ µ n = 1 n (cid:80) nt =1 X t

Original: Proof.
Revised: Proof.

Original: We have that
Revised: We have that

Original: [Equation]
Revised: [Equation]

Original: where the last inequality is due to Lemma 11 and Lemma 10.
Revised: where the last inequality is due to Lemma 11 and Lemma 10.

Original: Similarly,
Revised: Similarly,

Original: [Equation]
Revised: [Equation]

Original: where the last inequality is due to Lemma 11 and Lemma 10.
Revised: where the last inequality is due to Lemma 11 and Lemma 10.

Original: Lemma 4.
Revised: Lemma 4.

Original: Let X 1 , X 2 , . . .
Revised: Let X 1 , X 2 , . . .

Original: , X n be a sequence of independent random variables sampled from a
Revised: , X n be a sequence of independent random variables sampled from a

Original: Bernoulli distribution with mean µ , and let ˆ µ n = 1 n (cid:80) n t =1 X t be the sample mean.
Revised: Bernoulli distribution with mean µ , and let ˆ µ n = 1 n (cid:80) n t =1 X t be the sample mean.

Original: Let
Revised: Let

Original: [Equation]
Revised: [Equation]

Original: for δ > 0 be the clipped and private empirical mean.
Revised: for δ > 0 be the clipped and private empirical mean.

Original: Claim 1.
Revised: Claim 1.

Original: For any δ > 0 and α ∈ [0 , µ ] , the following inequality holds:
Revised: For any δ > 0 and α ∈ [0 , µ ] , the following inequality holds:

Original: [Equation]
Revised: [Equation]

Original: Claim 2.
Revised: Claim 2.

Original: Furthermore for δ ≥ 0 , we define
Revised: Furthermore for δ ≥ 0 , we define

Original: [Equation]
Revised: [Equation]

Original: Then,
Revised: Then,

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Here, we prove Claim 1 followed by Claim 2.
Revised: Here, we prove Claim 1 followed by Claim 2.

Original: Claim 1.
Revised: Claim 1.

Original: Since ˘ µ n ( δ ) =
Revised: Since ˘ µ n ( δ ) =

Original: min (cid:110) max (cid:110) 0 , ˆ µ n + Lap (cid:0) 1 nϵ (cid:1) + log( 1 δ ) nϵ
Revised: min (cid:110) max (cid:110) 0 , ˆ µ n + Lap (cid:0) 1 nϵ (cid:1) + log( 1 δ ) nϵ

Original: [Equation]
Revised: [Equation]

Original: , 1 (cid:111) , we have that
Revised: , 1 (cid:111) , we have that

Original: [Equation]
Revised: [Equation]

Original: (cid:18)
Revised: (cid:18)

Original: [Equation]
Revised: [Equation]

Original: nϵ (cid:19) + log(
Revised: nϵ (cid:19) + log(

Original: [Equation]
Revised: [Equation]

Original: ) nϵ (cid:19)(cid:27)
Revised: ) nϵ (cid:19)(cid:27)

Original: [Equation]
Revised: [Equation]

Original: (cid:18)
Revised: (cid:18)

Original: [Equation]
Revised: [Equation]

Original: (cid:19)
Revised: (cid:19)

Original: [Equation]
Revised: [Equation]

Original: nϵ ( since µ
Revised: nϵ ( since µ

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: or
Revised: or

Original: [Equation]
Revised: [Equation]

Original: (cid:18)
Revised: (cid:18)

Original: [Equation]
Revised: [Equation]

Original: It implies that
Revised: It implies that

Original: [Equation]
Revised: [Equation]

Original: The last inequality is due to Equation 38 of Lemma 13 and Lemma 10.
Revised: The last inequality is due to Equation 38 of Lemma 13 and Lemma 10.

Original: Claim 2.
Revised: Claim 2.

Original: We have that the sets
Revised: We have that the sets

Original: [Equation]
Revised: [Equation]

Original: Here, we chose an α > 0 such that d ( µ − α, µ ) = log( 1 δ ) n .
Revised: Here, we chose an α > 0 such that d ( µ − α, µ ) = log( 1 δ ) n .

Original: Step (a) holds because U n ( δ ) ≥ ˘ µ n ( δ ) by the definition of U n ( δ ) .
Revised: Step (a) holds because U n ( δ ) ≥ ˘ µ n ( δ ) by the definition of U n ( δ ) .

Original: Step (b) also holds true since d (˘ µ n ( δ ) , · ) is increasing on [˘ µ n ( δ ) , 1] .
Revised: Step (b) also holds true since d (˘ µ n ( δ ) , · ) is increasing on [˘ µ n ( δ ) , 1] .

Original: Since d (˘ µ n ( δ ) , U n ( δ ))
Revised: Since d (˘ µ n ( δ ) , U n ( δ ))

Original: =
Revised: =

Original: log( 1 δ ) n by the definition of U n ( δ ) , we obtain the equality in Step (c).
Revised: log( 1 δ ) n by the definition of U n ( δ ) , we obtain the equality in Step (c).

Original: Finally, Step (d) is obtained by inverting the relative entropy.
Revised: Finally, Step (d) is obtained by inverting the relative entropy.

Original: We conclude the proof by
Revised: We conclude the proof by

Original: [Equation]
Revised: [Equation]

Original: E.2 Generic Regret Analysis for Algorithm
Revised: E.2 Generic Regret Analysis for Algorithm

Original: Algorithm 1 is a generic framework to construct an extension of any optimistic index-based bandit algorithm, which would satisfy ϵ -global DP.
Revised: Algorithm 1 is a generic framework to construct an extension of any optimistic index-based bandit algorithm, which would satisfy ϵ -global DP.

Original: The algorithm is based on the index I ϵa of each arm.
Revised: The algorithm is based on the index I ϵa of each arm.

Original: I ϵa is computed using the private empirical mean of the last active episode of arm a and is a high probability upper bound of the real mean µ a .
Revised: I ϵa is computed using the private empirical mean of the last active episode of arm a and is a high probability upper bound of the real mean µ a .

Original: To explicate the two conditions on arm indexes, we introduce the notation I ϵa ( t − 1 , α, s ) , which is the index of arm a , at time-step t and computed using s reward samples from arm a .
Revised: To explicate the two conditions on arm indexes, we introduce the notation I ϵa ( t − 1 , α, s ) , which is the index of arm a , at time-step t and computed using s reward samples from arm a .

Original: Thus, we can express the index computed using just the last active episode as
Revised: Thus, we can express the index computed using just the last active episode as

Original: [Equation]
Revised: [Equation]

Original: Because I ϵa ( t − 1 , α ) only uses samples collected from the last active episode, and due to the doubling, the last active episode’s size is exactly half the number of times arm a was pulled since the beginning.
Revised: Because I ϵa ( t − 1 , α ) only uses samples collected from the last active episode, and due to the doubling, the last active episode’s size is exactly half the number of times arm a was pulled since the beginning.

Original: The optimism of the index is ensured by the fact that
Revised: The optimism of the index is ensured by the fact that

Original: [Equation]
Revised: [Equation]

Original: for every arm a , every sample size s and every time-step t , where α is the confidence level.
Revised: for every arm a , every sample size s and every time-step t , where α is the confidence level.

Original: Theorem 11.
Revised: Theorem 11.

Original: Let a be a suboptimal arm and ℓ ∈ N such that 2 ℓ < T .
Revised: Let a be a suboptimal arm and ℓ ∈ N such that 2 ℓ < T .

Original: Then, Algorithm 1 using an index
Revised: Then, Algorithm 1 using an index

Original: I ϵa satisfying Equations 28 and 29, also satisfies that for any α > 3 ,
Revised: I ϵa satisfying Equations 28 and 29, also satisfies that for any α > 3 ,

Original: [Equation]
Revised: [Equation]

Original: where G a,ℓ,T =
Revised: where G a,ℓ,T =

Original: { I ϵa ( T − 1 , α, 2 ℓ ) < µ ∗ } and G ca,ℓ,T is the complement of G a,ℓ,T
Revised: { I ϵa ( T − 1 , α, 2 ℓ ) < µ ∗ } and G ca,ℓ,T is the complement of G a,ℓ,T

Original: Proof.
Revised: Proof.

Original: Without loss of generality, we assume the first arm is the optimal one ( µ ∗ = µ 1 ) and denote a suboptimal arm by a ( 1 < a ≤ K ).
Revised: Without loss of generality, we assume the first arm is the optimal one ( µ ∗ = µ 1 ) and denote a suboptimal arm by a ( 1 < a ≤ K ).

Original: We leverage the standard idea of UCB-type proofs: if arm a is chosen at the beginning of an episode ℓ , then either its index at t ℓ is larger than the true mean of the first arm, or the true mean of the first arm is larger than the first arm’s index at t ℓ .
Revised: We leverage the standard idea of UCB-type proofs: if arm a is chosen at the beginning of an episode ℓ , then either its index at t ℓ is larger than the true mean of the first arm, or the true mean of the first arm is larger than the first arm’s index at t ℓ .

Original: Since decisions, i.e. playing the arm with the highest index, are only taken at the beginning of an episode, we introduce ϕ which takes as input a time step and outputs the time step corresponding to the beginning of an episode.
Revised: Since decisions, i.e. playing the arm with the highest index, are only taken at the beginning of an episode, we introduce ϕ which takes as input a time step and outputs the time step corresponding to the beginning of an episode.

Original: Formally, for each t ∈ [ K + 1 , T ] , let ϕ ( t ) =
Revised: Formally, for each t ∈ [ K + 1 , T ] , let ϕ ( t ) =

Original: t ℓ such that t ℓ ≤ t ≤ t ℓ +1 − 1 .
Revised: t ℓ such that t ℓ ≤ t ≤ t ℓ +1 − 1 .

Original: In Example 2, ϕ (5) = 4 and ϕ (9) = 7 .
Revised: In Example 2, ϕ (5) = 4 and ϕ (9) = 7 .

Original: Formally, ϕ ( t ) is a random variable such that
Revised: Formally, ϕ ( t ) is a random variable such that

Original: [Equation]
Revised: [Equation]

Original: Step 1: Decomposition of N a ( T ) .
Revised: Step 1: Decomposition of N a ( T ) .

Original: We observe that
Revised: We observe that

Original: N a ( T ) =
Revised: N a ( T ) =

Original: 1 +
Revised: 1 +

Original: T (cid:88) t = K +
Revised: T (cid:88) t = K +

Original: I { A t = a } = 1 +
Revised: I { A t = a } = 1 +

Original: T (cid:88) t = K +
Revised: T (cid:88) t = K +

Original: I { A t = a and I ϵ 1 ( ϕ ( t ) − 1 , α ) > µ 1 } + I { A t = a and I ϵ 1 ( ϕ ( t ) − 1 , α ) ≤ µ 1 } ≤ 1 + N ′ a ( T ) (cid:124) (cid:123)(cid:122) (cid:125) Term+
Revised: I { A t = a and I ϵ 1 ( ϕ ( t ) − 1 , α ) > µ 1 } + I { A t = a and I ϵ 1 ( ϕ ( t ) − 1 , α ) ≤ µ 1 } ≤ 1 + N ′ a ( T ) (cid:124) (cid:123)(cid:122) (cid:125) Term+

Original: T (cid:88) t = K +
Revised: T (cid:88) t = K +

Original: I { I ϵ 1 ( ϕ ( t ) − 1 , α ) ≤ µ 1 } (cid:124) (cid:123)(cid:122) (cid:125) Term
Revised: I { I ϵ 1 ( ϕ ( t ) − 1 , α ) ≤ µ 1 } (cid:124) (cid:123)(cid:122) (cid:125) Term

Original: We define N ′ a ( T ) ≜ (cid:80)
Revised: We define N ′ a ( T ) ≜ (cid:80)

Original: Tt =
Revised: Tt =

Original: K +

Revised: K +

Original: I
Revised: I

Original: A t = a
Revised: A t = a

Original: and I ϵ 1 ( t ℓ ′ − 1 , α ) > µ 1 }
Revised: and I ϵ 1 ( t ℓ ′ − 1 , α ) > µ 1 }

Original: Step 2: Decomposition of Term 1: N ′ a ( T ) .
Revised: Step 2: Decomposition of Term 1: N ′ a ( T ) .

Original: Let G a,ℓ,T be the ‘good’ event defined by
Revised: Let G a,ℓ,T be the ‘good’ event defined by

Original: G a,ℓ,T =
Revised: G a,ℓ,T =

Original: { I ϵa ( T − 1 , α, 2 ℓ ) < µ 1 } .
Revised: { I ϵa ( T − 1 , α, 2 ℓ ) < µ 1 } .

Original: The main part of the proof is decomposing N ′ a ( T ) among the ‘good’ and the ‘bad’ events, i.e.
Revised: The main part of the proof is decomposing N ′ a ( T ) among the ‘good’ and the ‘bad’ events, i.e.

Original: E [ N ′ a ( T )]
Revised: E [ N ′ a ( T )]

Original: =
Revised: =

Original: E [ I { G a,ℓ,T } N ′ a ( T )]
Revised: E [ I { G a,ℓ,T } N ′ a ( T )]

Original: + E
Revised: + E

Original: [ I { G ca,ℓ,T } N ′ a ( T )] ≤ 2 ℓ +1 + P ( G ca,ℓ,T ) T.
Revised: [ I { G ca,ℓ,T } N ′ a ( T )] ≤ 2 ℓ +1 + P ( G ca,ℓ,T ) T.

Original: G ca,ℓ,T denotes the complement of G a,ℓ,T .
Revised: G ca,ℓ,T denotes the complement of G a,ℓ,T .

Original: To prove the last inequality, we only need to prove that when G a,ℓ,T happens, N ′ a ( T ) ≤ 2 ℓ
Revised: To prove the last inequality, we only need to prove that when G a,ℓ,T happens, N ′ a ( T ) ≤ 2 ℓ

Original: +1 .
Revised: +1 .

Original: We prove it by contradiction.
Revised: We prove it by contradiction.

Original: Hence, let us assume that G a,ℓ,T holds but N ′ a ( T ) > 2 ℓ +1 .
Revised: Hence, let us assume that G a,ℓ,T holds but N ′ a ( T ) > 2 ℓ +1 .

Original: This assumption implies that the arm a is played more than 2 ℓ +1 times.
Revised: This assumption implies that the arm a is played more than 2 ℓ +1 times.

Original: Thus, there must exist a round t ℓ ′ , where N a ( t ℓ ′ − 1) = 2 ℓ +1 , A t ℓ ′ =
Revised: Thus, there must exist a round t ℓ ′ , where N a ( t ℓ ′ − 1) = 2 ℓ +1 , A t ℓ ′ =

Original: i and I ϵ 1 ( t ℓ ′ − 1 , α ) ≥ µ 1 .
Revised: i and I ϵ 1 ( t ℓ ′ − 1 , α ) ≥ µ 1 .

Original: Since indices are computed only using the samples from the last active episode, I ϵa ( t ℓ ′ − 1 , α ) is computed using exactly 2 ℓ reward samples from arm a .
Revised: Since indices are computed only using the samples from the last active episode, I ϵa ( t ℓ ′ − 1 , α ) is computed using exactly 2 ℓ reward samples from arm a .

Original: Thus, we obtain
Revised: Thus, we obtain

Original: [Equation]
Revised: [Equation]

Original: because t
Revised: because t

Original: [Equation]
Revised: [Equation]

Original: I
Revised: I

Original: [Equation]
Revised: [Equation]

Original: is increasing
Revised: is increasing

Original: [Equation]
Revised: [Equation]

Original: definition of G
Revised: definition of G

Original: [Equation]
Revised: [Equation]

Original: T
Revised: T

Original: [Equation]
Revised: [Equation]

Original: The last inequality contradicts the fact that A t ℓ ′ =
Revised: The last inequality contradicts the fact that A t ℓ ′ =

Original: i and thus, establishes the claim that N ′ a ( T ) ≤ 2 ℓ + under the ‘good’ event.
Revised: i and thus, establishes the claim that N ′ a ( T ) ≤ 2 ℓ + under the ‘good’ event.

Original: Step 3: Upper-bounding Term 2.
Revised: Step 3: Upper-bounding Term 2.

Original: To conclude,
Revised: To conclude,

Original: [Equation]
Revised: [Equation]

Original: Equation 29)
Revised: Equation 29)

Original: [Equation]
Revised: [Equation]

Original: (because ϕ ≥ t 2 )
Revised: (because ϕ ≥ t 2 )

Original: [Equation]
Revised: [Equation]

Original: for α > 3 and K ≥ 2 .
Revised: for α > 3 and K ≥ 2 .

Original: Here, the first inequality is due to an union bound on ϕ ( t ) ∈ [ t/ 2 , t ] (Equation 30), and the second inequality is due to a union bound on N 1 ( ϕ − 1) .
Revised: Here, the first inequality is due to an union bound on ϕ ( t ) ∈ [ t/ 2 , t ] (Equation 30), and the second inequality is due to a union bound on N 1 ( ϕ − 1) .

Original: Step 4:
Revised: Step 4:

Original: Combining the Bounds on Terms 1 and 2.
Revised: Combining the Bounds on Terms 1 and 2.

Original: [Equation]
Revised: [Equation]

Original: Now we design indexes that satisfy the conditions of Theorem 11, namely AdaP - UCB and
Revised: Now we design indexes that satisfy the conditions of Theorem 11, namely AdaP - UCB and

Original: AdaP - KLUCB .
Revised: AdaP - KLUCB .

Original: To obtain the final regret bounds, we only have to choose ℓ big enough such that P (cid:0) I a ( T, 2 ℓ ) ≥ µ 1 (cid:1) T is negligible.
Revised: To obtain the final regret bounds, we only have to choose ℓ big enough such that P (cid:0) I a ( T, 2 ℓ ) ≥ µ 1 (cid:1) T is negligible.

Original: This corresponds to the leading term in the regret upper-bounds, and this is where the regrets of AdaP - UCB and AdaP - KLUCB differ.
Revised: This corresponds to the leading term in the regret upper-bounds, and this is where the regrets of AdaP - UCB and AdaP - KLUCB differ.

Original: We explicate the issues of designing the indexes and choosing corresponding ℓ in the following section, which leads to the regret upper bounds of AdaP - UCB and AdaP - KLUCB .
Revised: We explicate the issues of designing the indexes and choosing corresponding ℓ in the following section, which leads to the regret upper bounds of AdaP - UCB and AdaP - KLUCB .

Original: E.3 Regret Analysis for AdaP - UCB and AdaP - KLUCB
Revised: E.3 Regret Analysis for AdaP - UCB and AdaP - KLUCB

Original: Theorem 7.
Revised: Theorem 7.

Original: For rewards in [0 , 1] , AdaP - UCB satisfies ϵ -global DP, and for α > 3 , it yields a regret
Revised: For rewards in [0 , 1] , AdaP - UCB satisfies ϵ -global DP, and for α > 3 , it yields a regret

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: The proof is constituted of three steps.
Revised: The proof is constituted of three steps.

Original: Step 1:
Revised: Step 1:

Original: Designing an Index satisfying Equation (28) , Equation (29) , and ϵ -global DP.
Revised: Designing an Index satisfying Equation (28) , Equation (29) , and ϵ -global DP.

Original: For
Revised: For

Original: AdaP - UCB , the index is defined as
Revised: AdaP - UCB , the index is defined as

Original: [Equation]
Revised: [Equation]

Original: where ˜ µ
Revised: where ˜ µ

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: is the private empirical mean of arm a computed using only samples from the last active episode, and ˆ µ a,s is the empirical mean of arm a calculated using s samples of reward from arm a .
Revised: is the private empirical mean of arm a computed using only samples from the last active episode, and ˆ µ a,s is the empirical mean of arm a calculated using s samples of reward from arm a .

Original: This index verifies the first condition (Equation 28) of Theorem 11.
Revised: This index verifies the first condition (Equation 28) of Theorem 11.

Original: The second condition (Equation 29) of Theorem 11 follows directly from Equation 23 of Lemma
Revised: The second condition (Equation 29) of Theorem 11 follows directly from Equation 23 of Lemma

Original: By Theorem 6, AdaP - UCB is ϵ -global DP.
Revised: By Theorem 6, AdaP - UCB is ϵ -global DP.

Original: By Theorem 11, for every suboptimal arm a , we have that
Revised: By Theorem 11, for every suboptimal arm a , we have that

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: Step 2: Choosing an ℓ .
Revised: Step 2: Choosing an ℓ .

Original: Now, we observe that
Revised: Now, we observe that

Original: [Equation]
Revised: [Equation]

Original: for γ =
Revised: for γ =

Original: [Equation]
Revised: [Equation]

Original: The idea is to choose ℓ big enough so that γ ≥ 0 .
Revised: The idea is to choose ℓ big enough so that γ ≥ 0 .

Original: Let us consider the contrary,
Revised: Let us consider the contrary,

Original: i
Revised: i

Original: [Equation]
Revised: [Equation]

Original: e.
Revised: e.

Original: [Equation]
Revised: [Equation]

Original: Thus, by choosing
Revised: Thus, by choosing

Original: [Equation]
Revised: [Equation]

Original: we ensure γ > 0 .
Revised: we ensure γ > 0 .

Original: This also implies that
Revised: This also implies that

Original: [Equation]
Revised: [Equation]

Original: The last inequality is due to Equation 22 of Lemma 3.
Revised: The last inequality is due to Equation 22 of Lemma 3.

Original: Step 3: The Regret Bound.
Revised: Step 3: The Regret Bound.

Original: Combining Steps 1 and 2, we get that
Revised: Combining Steps 1 and 2, we get that

Original: [Equation]
Revised: [Equation]

Original: Plugging this upper bound back in the definition of problem-dependent regret concludes the proof.
Revised: Plugging this upper bound back in the definition of problem-dependent regret concludes the proof.

Original: Remark 3.
Revised: Remark 3.

Original: The leading term of the regret is
Revised: The leading term of the regret is

Original: [Equation]
Revised: [Equation]

Original: , which is 4 times more than what we got from Equation 32.
Revised: , which is 4 times more than what we got from Equation 32.

Original: A multiplicative factor of 2 is introduced due to the doubling and another multiplicative factor of 2 is due to the forgetting.
Revised: A multiplicative factor of 2 is introduced due to the doubling and another multiplicative factor of 2 is due to the forgetting.

Original: Thus, the combined price of doubling and forgetting is a multiplicative constant 4 in the leading term of regret.
Revised: Thus, the combined price of doubling and forgetting is a multiplicative constant 4 in the leading term of regret.

Original: Theorem 8.
Revised: Theorem 8.

Original: When the rewards are sampled from Bernoulli distributions, AdaP - KLUCB satisfies ϵ -global DP, and for α > 3 and constants C 1 ( α ) , C 2 > 0 , it yields a regret
Revised: When the rewards are sampled from Bernoulli distributions, AdaP - KLUCB satisfies ϵ -global DP, and for α > 3 and constants C 1 ( α ) , C 2 > 0 , it yields a regret

Original: Reg T
Revised: Reg T

Original: [Equation]
Revised: [Equation]

Original: AdaP - KLUCB
Revised: AdaP - KLUCB

Original: [Equation]
Revised: [Equation]

Original: ≤ (cid:88)
Revised: ≤ (cid:88)

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: The proof is constituted of three steps.
Revised: The proof is constituted of three steps.

Original: Step 1:
Revised: Step 1:

Original: Designing an Index satisfying Equation (28) , Equation (29) , and ϵ -global DP.
Revised: Designing an Index satisfying Equation (28) , Equation (29) , and ϵ -global DP.

Original: For
Revised: For

Original: AdaP - KLUCB , the index is defined as
Revised: AdaP - KLUCB , the index is defined as

Original: [Equation]
Revised: [Equation]

Original: where ˘ µ ℓ,αa,ϵ =
Revised: where ˘ µ ℓ,αa,ϵ =

Original: Clip 0 ,
Revised: Clip 0 ,

Original: [Equation]
Revised: [Equation]

Original: µ
Revised: µ

Original: [Equation]
Revised: [Equation]

Original: ϵ + α log
Revised: ϵ + α log

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: t
Revised: t

Original: [Equation]
Revised: [Equation]

Original: =
Revised: =

Original: [Equation]
Revised: [Equation]

Original: as defined in Equation 24, ˜ µ ℓa,ϵ is the private empirical computed only using the samples from the last active episode (as defined for AdaP - UCB , and U a,s ( δ ) = max
Revised: as defined in Equation 24, ˜ µ ℓa,ϵ is the private empirical computed only using the samples from the last active episode (as defined for AdaP - UCB , and U a,s ( δ ) = max

Original: [Equation]
Revised: [Equation]

Original: as defined in
Revised: as defined in

Original: Equation
Revised: Equation

Original: This index verifies the first condition (Equation 28) of Theorem 11.
Revised: This index verifies the first condition (Equation 28) of Theorem 11.

Original: The second condition (Equation 29) of Theorem 11 follows directly from Equation 23 of Lemma
Revised: The second condition (Equation 29) of Theorem 11 follows directly from Equation 23 of Lemma

Original: By Theorem 6, AdaP - KLUCB also satisfies ϵ -global DP.
Revised: By Theorem 6, AdaP - KLUCB also satisfies ϵ -global DP.

Original: By Theorem 11, for every suboptimal arm a , we have that
Revised: By Theorem 11, for every suboptimal arm a , we have that

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: Step 2: Choosing an ℓ .
Revised: Step 2: Choosing an ℓ .

Original: We observe that
Revised: We observe that

Original: [Equation]
Revised: [Equation]

Original: by definition of U
Revised: by definition of U

Original: [Equation]
Revised: [Equation]

Original: where d + ( p, q ) ≜
Revised: where d + ( p, q ) ≜

Original: d ( p, q ) I p<q and d ( p, q ) is the relative entropy between Bernoulli distributions as stated in Definition 5.
Revised: d ( p, q ) I p<q and d ( p, q ) is the relative entropy between Bernoulli distributions as stated in Definition 5.

Original: Let β > 0 , and c ( β ) ∈
Revised: Let β > 0 , and c ( β ) ∈

Original: [0 , 1] such that: d ( µ a + c ( β )∆ a , µ 1 ) =
Revised: [0 , 1] such that: d ( µ a + c ( β )∆ a , µ 1 ) =

Original: d ( µ a ,µ 1 ) 1+ β .
Revised: d ( µ a ,µ 1 ) 1+ β .

Original: Since d ( · , µ 1 ) is a bijective function from [ µ a , µ 1 ] to [0 , d ( µ a , µ 1 )] , we get that c ( β ) always exists and is unique.
Revised: Since d ( · , µ 1 ) is a bijective function from [ µ a , µ 1 ] to [0 , d ( µ a , µ 1 )] , we get that c ( β ) always exists and is unique.

Original: In addition, c ( β ) verifies: lim β → 0 c ( β ) = 0 , lim β
Revised: In addition, c ( β ) verifies: lim β → 0 c ( β ) = 0 , lim β

Original: → + ∞ c ( β ) = 1 and c ( β ) is an increasing function of β .
Revised: → + ∞ c ( β ) = 1 and c ( β ) is an increasing function of β .

Original: First, we choose ℓ such that
Revised: First, we choose ℓ such that

Original: [Equation]
Revised: [Equation]

Original: This leads to
Revised: This leads to

Original: [Equation]
Revised: [Equation]

Original: (definition of
Revised: (definition of

Original: [Equation]
Revised: [Equation]

Original: is
Revised: is

Original: [Equation]
Revised: [Equation]

Original: on
Revised: on

Original: [Equation]
Revised: [Equation]

Original: definition of ˘ µ )
Revised: definition of ˘ µ )

Original: Let us consider γ ℓ,T such that d ( µ a + γ ℓ,T ∆ a , µ a ) = log( T ) 2 ℓ .
Revised: Let us consider γ ℓ,T such that d ( µ a + γ ℓ,T ∆ a , µ a ) = log( T ) 2 ℓ .

Original: We prove its existence and upper bound it later in Fact 3.
Revised: We prove its existence and upper bound it later in Fact 3.

Original: Thus, we obtain
Revised: Thus, we obtain

Original: P
Revised: P

Original: [Equation]
Revised: [Equation]

Original: G ca
Revised: G ca

Original: [Equation]
Revised: [Equation]

Original: ℓ
Revised: ℓ

Original: [Equation]
Revised: [Equation]

Original: ≤ P
Revised: ≤ P

Original: [Equation]
Revised: [Equation]

Original: ϵ
Revised: ϵ

Original: [Equation]
Revised: [Equation]

Original: ϵ
Revised: ϵ

Original: [Equation]
Revised: [Equation]

Original: Here, θ ≜ ( c ( β ) − γ ℓ,T )∆ a − (1+ α )log( T ) ϵ 2 ℓ .
Revised: Here, θ ≜ ( c ( β ) − γ ℓ,T )∆ a − (1+ α )log( T ) ϵ 2 ℓ .

Original: By choosing
Revised: By choosing

Original: [Equation]
Revised: [Equation]

Original: 35) we ensure that θ ≥ 0 .
Revised: 35) we ensure that θ ≥ 0 .

Original: Thus
Revised: Thus

Original: [Equation]
Revised: [Equation]

Original: we get
Revised: we get

Original: [Equation]
Revised: [Equation]

Original: The last inequality is due to Equation 37 of Lemma 13 and Lemma 10.
Revised: The last inequality is due to Equation 37 of Lemma 13 and Lemma 10.

Original: Fact 3.
Revised: Fact 3.

Original: B ≜ { β > 0 : c ( β ) > γ ℓ,T } ̸ = ∅ .
Revised: B ≜ { β > 0 : c ( β ) > γ ℓ,T } ̸ = ∅ .

Original: Combining both conditions 34 and 34 , we choose ℓ to be the smallest integer such that
Revised: Combining both conditions 34 and 34 , we choose ℓ to be the smallest integer such that

Original: [Equation]
Revised: [Equation]

Original: Step 3: The Regret Bound.
Revised: Step 3: The Regret Bound.

Original: Combining Steps 1 and 2, we get that
Revised: Combining Steps 1 and 2, we get that

Original: [Equation]
Revised: [Equation]

Original: Plugging this upper bound back in the definition of problem-dependent regret concludes the proof.
Revised: Plugging this upper bound back in the definition of problem-dependent regret concludes the proof.

Original: To conclude, we prove Fact 3.
Revised: To conclude, we prove Fact 3.

Original: Fact 3.
Revised: Fact 3.

Original: B ≜ { β > 0 : c ( β ) > γ ℓ,T } ̸ = ∅ .
Revised: B ≜ { β > 0 : c ( β ) > γ ℓ,T } ̸ = ∅ .

Original: Proof.
Revised: Proof.

Original: Step 1: Going from d ( · , µ a ) to d ( · , µ 1 ) .
Revised: Step 1: Going from d ( · , µ a ) to d ( · , µ 1 ) .

Original: The difficulty of the proof lies in the fact that γ ℓ,T is defined by inverting d ( · , µ a ) while c ( β ) is defined by inverting d ( · , µ 1 ) .
Revised: The difficulty of the proof lies in the fact that γ ℓ,T is defined by inverting d ( · , µ a ) while c ( β ) is defined by inverting d ( · , µ 1 ) .

Original: To handle that, we investigate the function g ( x ) ≜
Revised: To handle that, we investigate the function g ( x ) ≜

Original: d ( x, µ a ) − d ( x, µ 1 ) .
Revised: d ( x, µ a ) − d ( x, µ 1 ) .

Original: g satisfies the following properties:
Revised: g satisfies the following properties:

Original: • g is continuous and increasing in the interval [ µ a , µ 1 ] , • g ( µ a ) = − d ( µ a , µ 1 ) < 0 , and • g ( µ 1 ) =
Revised: • g is continuous and increasing in the interval [ µ a , µ 1 ] , • g ( µ a ) = − d ( µ a , µ 1 ) < 0 , and • g ( µ 1 ) =

Original: d ( µ 1 , µ a ) > 0 .
Revised: d ( µ 1 , µ a ) > 0 .

Original: This implies that there exists a unique root of g ( x ) , where it changes sign.
Revised: This implies that there exists a unique root of g ( x ) , where it changes sign.

Original: Specifically, there exists a unique z ∈ [ µ a , µ 1 ] such that:
Revised: Specifically, there exists a unique z ∈ [ µ a , µ 1 ] such that:

Original: • g ( z ) =
Revised: • g ( z ) =

Original: • ∀ x ∈ [ µ a , z [: g ( x ) <
Revised: • ∀ x ∈ [ µ a , z [: g ( x ) <

Original: • ∀ x ∈ ] z, µ 1 ] : g ( x ) >
Revised: • ∀ x ∈ ] z, µ 1 ] : g ( x ) >

Original: and consequently z verifies d ( z, µ a )
Revised: and consequently z verifies d ( z, µ a )

Original: [Equation]
Revised: [Equation]

Original: z, µ 1 )
Revised: z, µ 1 )

Original: Step 2: Choosing β .
Revised: Step 2: Choosing β .

Original: We choose β such that d ( µ a ,µ 1 ) 1+ β
Revised: We choose β such that d ( µ a ,µ 1 ) 1+ β

Original: [Equation]
Revised: [Equation]

Original: Step 3: Consequence of the choice of β on c ( β ) .
Revised: Step 3: Consequence of the choice of β on c ( β ) .

Original: Thus,
Revised: Thus,

Original: [Equation]
Revised: [Equation]

Original: which yields
Revised: which yields

Original: [Equation]
Revised: [Equation]

Original: by uniqueness of z .
Revised: by uniqueness of z .

Original: Step 4: Consequence of the choice of β on γ ℓ,T .
Revised: Step 4: Consequence of the choice of β on γ ℓ,T .

Original: On the other hand,
Revised: On the other hand,

Original: [Equation]
Revised: [Equation]

Original: As a consequence, we conclude that γ ℓ,T exists and γ ℓ,T < c ( β ) as d ( · , µ a ) is an increasing function in the interval [ µ a , 1]
Revised: As a consequence, we conclude that γ ℓ,T exists and γ ℓ,T < c ( β ) as d ( · , µ a ) is an increasing function in the interval [ µ a , 1]

Original: E.4 Problem-independent Regret Bounds
Revised: E.4 Problem-independent Regret Bounds

Original: In this section, we provide problem-independent (or minimax) regret upper bounds for AdaP - UCB .
Revised: In this section, we provide problem-independent (or minimax) regret upper bounds for AdaP - UCB .

Original: Theorem 12.
Revised: Theorem 12.

Original: For rewards in [0 , 1] , AdaP - UCB yields a regret
Revised: For rewards in [0 , 1] , AdaP - UCB yields a regret

Original: [Equation]
Revised: [Equation]

Original: which achieves the minimax lower bound of Thm 2 up to log( T ) factors.
Revised: which achieves the minimax lower bound of Thm 2 up to log( T ) factors.

Original: Proof.
Revised: Proof.

Original: Let ∆ be a value to be tuned later.
Revised: Let ∆ be a value to be tuned later.

Original: We have that
Revised: We have that

Original: [Equation]
Revised: [Equation]

Original: where the last step is by taking ∆ = 4 (cid:113) αK log( T ) T .
Revised: where the last step is by taking ∆ = 4 (cid:113) αK log( T ) T .

Original: Remark 4.
Revised: Remark 4.

Original: The same bound is achieved by AdaP - KLUCB (up to multiplicative constants) by using that d ( µ a , µ ∗ ) ≥ 2∆ 2 a and using the same steps in Thm 12.
Revised: The same bound is achieved by AdaP - KLUCB (up to multiplicative constants) by using that d ( µ a , µ ∗ ) ≥ 2∆ 2 a and using the same steps in Thm 12.

Original: F Existing Technical Results and Definitions
Revised: F Existing Technical Results and Definitions

Original: In this section, we summarise the existing technical results and definitions required to establish our proofs.
Revised: In this section, we summarise the existing technical results and definitions required to establish our proofs.

Original: Lemma 5 (Post-processing Lemma (Proposition 2.1, (Dwork and Roth, 2014))) .
Revised: Lemma 5 (Post-processing Lemma (Proposition 2.1, (Dwork and Roth, 2014))) .

Original: If a randomised algorithm A satisfies ( ϵ, δ ) -Differential
Revised: If a randomised algorithm A satisfies ( ϵ, δ ) -Differential

Original: Privacy and f is an arbitrary randomised mapping defined on A ’s output, then f ◦ A satisfies ( ϵ, δ ) -DP.
Revised: Privacy and f is an arbitrary randomised mapping defined on A ’s output, then f ◦ A satisfies ( ϵ, δ ) -DP.

Original: Lemma 6 (Markov’s Inequality) .
Revised: Lemma 6 (Markov’s Inequality) .

Original: For any random variable X and ε > 0 ,
Revised: For any random variable X and ε > 0 ,

Original: [Equation]
Revised: [Equation]

Original: Definition 4 (Consistent Policies) .
Revised: Definition 4 (Consistent Policies) .

Original: A policy π is called consistent over a class of bandits E if for all ν ∈ E and p > 0 , it holds that
Revised: A policy π is called consistent over a class of bandits E if for all ν ∈ E and p > 0 , it holds that

Original: [Equation]
Revised: [Equation]

Original: The class of consistent policies over E is denoted by Π cons ( E ) .
Revised: The class of consistent policies over E is denoted by Π cons ( E ) .

Original: Lemma 7 (Divergence decomposition) .
Revised: Lemma 7 (Divergence decomposition) .

Original: Let ν =
Revised: Let ν =

Original: ( P 1 , . . .
Revised: ( P 1 , . . .

Original: , P K ) and ν ′ =
Revised: , P K ) and ν ′ =

Original: ( P ′ 1 , . .
Revised: ( P ′ 1 , . .

Original: P ′ K ) be two bandit instances.
Revised: P ′ K ) be two bandit instances.

Original: Fix some policy π and let P νπ and P ν ′
Revised: Fix some policy π and let P νπ and P ν ′

Original: π be the probability measures on the canonical bandit model.
Revised: π be the probability measures on the canonical bandit model.

Original: Then,
Revised: Then,

Original: [Equation]
Revised: [Equation]

Original: Lemma 8 (Bretagnolle-Huber inequality) .
Revised: Lemma 8 (Bretagnolle-Huber inequality) .

Original: Let P and Q be probability measures on the same measurable space (Ω , F ) , and let A ∈ F be an arbitrary event.
Revised: Let P and Q be probability measures on the same measurable space (Ω , F ) , and let A ∈ F be an arbitrary event.

Original: Then,
Revised: Then,

Original: [Equation]
Revised: [Equation]

Original: where A c = Ω \
Revised: where A c = Ω \

Original: A is the complement of A .
Revised: A is the complement of A .

Original: Lemma 9 (Pinsker’s Inequality) .
Revised: Lemma 9 (Pinsker’s Inequality) .

Original: For two probability measures P and Q on the same probability space (Ω , F ) , we have
Revised: For two probability measures P and Q on the same probability space (Ω , F ) , we have

Original: [Equation]
Revised: [Equation]

Original: Lemma 10 (Tail Bounds for Laplacian Random Variables) .
Revised: Lemma 10 (Tail Bounds for Laplacian Random Variables) .

Original: For any a, b > 0 , we have
Revised: For any a, b > 0 , we have

Original: [Equation]
Revised: [Equation]

Original: Lap ( b ) >
Revised: Lap ( b ) >

Original: [Equation]
Revised: [Equation]

Original: ) =
Revised: ) =

Original: [Equation]
Revised: [Equation]

Original: (cid:16) −
Revised: (cid:16) −

Original: [Equation]
Revised: [Equation]

Original: b (cid:17) and P ( Lap ( b ) < − a ) = 1 2 exp (cid:16) − a b (cid:17) .
Revised: b (cid:17) and P ( Lap ( b ) < − a ) = 1 2 exp (cid:16) − a b (cid:17) .

Original: Lemma 11 (Hoeffding’s Bound) .
Revised: Lemma 11 (Hoeffding’s Bound) .

Original: Assume that ( X i ) 1 ≤ i ≤ n are iid random variables in [0 , 1] , with E ( X i )
Revised: Assume that ( X i ) 1 ≤ i ≤ n are iid random variables in [0 , 1] , with E ( X i )

Original: = µ .
Revised: = µ .

Original: For any δ, β ≥ 

Revised: For any δ, β ≥

Original: and, we have:
Revised: and, we have:

Original: [Equation]
Revised: [Equation]

Original: where ˆ µ n = 1 n (cid:80) nt =1 X t .
Revised: where ˆ µ n = 1 n (cid:80) nt =1 X t .

Original: Definition 5 (Relative entropy between Bernoulli distributions) .
Revised: Definition 5 (Relative entropy between Bernoulli distributions) .

Original: The relative entropy between
Revised: The relative entropy between

Original: Bernoulli distributions with parameters p, q ∈ [0 , 1] is
Revised: Bernoulli distributions with parameters p, q ∈ [0 , 1] is

Original: [Equation]
Revised: [Equation]

Original: where singularities are defined by taking limits: d (0 , q )
Revised: where singularities are defined by taking limits: d (0 , q )

Original: = log(1 / (1 − q )) and d (1 , q )
Revised: = log(1 / (1 − q )) and d (1 , q )

Original: = log(1 /q ) for q ∈ [0 , 1] and d ( p, 0)
Revised: = log(1 /q ) for q ∈ [0 , 1] and d ( p, 0)

Original: = 

Revised: =

Original: if p = 0 and ∞ otherwise and d ( p, 1) = 0 if p = 1 and ∞ otherwise.
Revised: if p = 0 and ∞ otherwise and d ( p, 1) = 0 if p = 1 and ∞ otherwise.

Original: Lemma 12 (Properties of the relative entropy between Bernoulli distributions (Lemma 10.2, (Lattimore and Szepesvári, 2018))) .
Revised: Lemma 12 (Properties of the relative entropy between Bernoulli distributions (Lemma 10.2, (Lattimore and Szepesvári, 2018))) .

Original: Let p, q, ε ∈ [0 , 1] .
Revised: Let p, q, ε ∈ [0 , 1] .

Original: The functions d ( · , q ) and d ( p, · ) are convex and have unique minimisers at q and p , respectively.
Revised: The functions d ( · , q ) and d ( p, · ) are convex and have unique minimisers at q and p , respectively.

Original: d ( p, · ) and d ( · , p ) are increasing in the interval [ p, 1] and decreasing in the interval [0 , p ] .
Revised: d ( p, · ) and d ( · , p ) are increasing in the interval [ p, 1] and decreasing in the interval [0 , p ] .

Original: Lemma 13 (Chernoff’s Bound) .
Revised: Lemma 13 (Chernoff’s Bound) .

Original: Let X 1 , X 2 , . . .
Revised: Let X 1 , X 2 , . . .

Original: , X n be a sequence of independent random variables that are Bernoulli distributed with mean µ , and let ˆ µ n = 1 n (cid:80) nt =1 X t be the sample mean.
Revised: , X n be a sequence of independent random variables that are Bernoulli distributed with mean µ , and let ˆ µ n = 1 n (cid:80) nt =1 X t be the sample mean.

Original: Then, for β ∈ [0 , 1 − µ ] , it holds that:
Revised: Then, for β ∈ [0 , 1 − µ ] , it holds that:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: and for β ∈ [0 , µ ] ,
Revised: and for β ∈ [0 , µ ] ,

Original: [Equation]
Revised: [Equation]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: G Extended Experimental Analysis G.1 Experimental Setup
Revised: G Extended Experimental Analysis G.1 Experimental Setup

Original: In this section, we perform additional experiments to compare AdaP - UCB and AdaP - KLUCB with respect to the existing bandit algorithms satisfying global DP, i.e. DP - SE (Sajed and Sheffet, 2019) and DP - UCB (Mishra and Thakurta, 2015).
Revised: In this section, we perform additional experiments to compare AdaP - UCB and AdaP - KLUCB with respect to the existing bandit algorithms satisfying global DP, i.e. DP - SE (Sajed and Sheffet, 2019) and DP - UCB (Mishra and Thakurta, 2015).

Original: We test the four algorithms in the four bandit environments with Bernoulli distributions, as defined by Sajed and Sheffet (2019), namely
Revised: We test the four algorithms in the four bandit environments with Bernoulli distributions, as defined by Sajed and Sheffet (2019), namely

Original: C 1 = { 0 . 75 , 0 .
Revised: C 1 = { 0 . 75 , 0 .

Original: 70 } , C 2 = { 0 .
Revised: 70 } , C 2 = { 0 .

Original: C 3 = { 0 . 75 , 0 . 53125 , 0 . 375 , 0 .
Revised: C 3 = { 0 . 75 , 0 . 53125 , 0 . 375 , 0 .

Original: 25 } , C 4 = { 0 .
Revised: 25 } , C 4 = { 0 .

Original: For each bandit environment, we implement the algorithms with ϵ ∈ { 0 . 1 , 0 . 25 , 0 .
Revised: For each bandit environment, we implement the algorithms with ϵ ∈ { 0 . 1 , 0 . 25 , 0 .

Original: We set α = 3 .
Revised: We set α = 3 .

Original: 1 to comply with the regret upper bounds of AdaP - UCB and AdaP - KLUCB .
Revised: 1 to comply with the regret upper bounds of AdaP - UCB and AdaP - KLUCB .

Original: We set γ = 0 .for
Revised: We set γ = 0 .for

Original: DP - UCB and β = 1 /T .
Revised: DP - UCB and β = 1 /T .

Original: All the algorithms are implemented in Python (version 3 . 8 ) and are tested with an 8 core 64-bits Intel i5@1.6 GHz CPU.
Revised: All the algorithms are implemented in Python (version 3 . 8 ) and are tested with an 8 core 64-bits Intel i5@1.6 GHz CPU.

Original: We run each algorithm 20 times, and plot their average regrets over the runs in Figures 4, 5, 6, and 7.In Section 5, we include Figure 2 to illustrate the evolution of the regret for the four algorithms with environment C 2 and ϵ = 1 .
Revised: We run each algorithm 20 times, and plot their average regrets over the runs in Figures 4, 5, 6, and 7.In Section 5, we include Figure 2 to illustrate the evolution of the regret for the four algorithms with environment C 2 and ϵ = 1 .

Original: G.2 Experimental Results
Revised: G.2 Experimental Results

Original: Here, we summarise the observations obtained from the experimental results.
Revised: Here, we summarise the observations obtained from the experimental results.

Original: Comparative Performance.
Revised: Comparative Performance.

Original: All the experiments validate that AdaP - KLUCB is the most optimal algorithm satisfying ϵ -global DP for stochastic bandits.
Revised: All the experiments validate that AdaP - KLUCB is the most optimal algorithm satisfying ϵ -global DP for stochastic bandits.

Original: Both AdaP - UCB and AdaP - KLUCB achieve similar regret, but AdaP - KLUCB is slightly better in all the cases studied.
Revised: Both AdaP - UCB and AdaP - KLUCB achieve similar regret, but AdaP - KLUCB is slightly better in all the cases studied.

Original: This observation matches the proven upper bounds, and also reflects similar improvement that KL - UCB brings over UCB in non-private bandits.
Revised: This observation matches the proven upper bounds, and also reflects similar improvement that KL - UCB brings over UCB in non-private bandits.

Original: Dependence of Regret on ϵ .
Revised: Dependence of Regret on ϵ .

Original: As predicted by the theoretical analysis, AdaP - UCB and AdaP - KLUCB have different regret depending on ϵ : the regret is smaller for low-privacy regimes.
Revised: As predicted by the theoretical analysis, AdaP - UCB and AdaP - KLUCB have different regret depending on ϵ : the regret is smaller for low-privacy regimes.

Original: This is also the case for DP - UCB .
Revised: This is also the case for DP - UCB .

Original: However, DP - SE have the same performance for different choices of ϵ and echoes the experimental results presented in the original paper (Sajed and Sheffet, 2019).
Revised: However, DP - SE have the same performance for different choices of ϵ and echoes the experimental results presented in the original paper (Sajed and Sheffet, 2019).

Original: [Figure]
Revised: [Figure]

Original: The Shapes of the Regrets.
Revised: The Shapes of the Regrets.

Original: DP - UCB has a regret shaped like the regret of the classic UCB algorithm.
Revised: DP - UCB has a regret shaped like the regret of the classic UCB algorithm.

Original: The algorithm chooses a different action at each time-step allowing it to still choose exploratory actions.
Revised: The algorithm chooses a different action at each time-step allowing it to still choose exploratory actions.

Original: On the other hand, due to the successive elimination, DP - SE "commits" at a certain step to one action (the optimal action with high probability).
Revised: On the other hand, due to the successive elimination, DP - SE "commits" at a certain step to one action (the optimal action with high probability).

Original: Thus, the shape of regret for DP - SE is piece-wise linear.
Revised: Thus, the shape of regret for DP - SE is piece-wise linear.

Original: On the other hand, AdaP - UCB and AdaP - KLUCB are a trade-off of both strategies: due to the doubling, both algorithms "commit" for long episodes to near-optimal actions , while still explore the sub-optimal actions for short episodes .
Revised: On the other hand, AdaP - UCB and AdaP - KLUCB are a trade-off of both strategies: due to the doubling, both algorithms "commit" for long episodes to near-optimal actions , while still explore the sub-optimal actions for short episodes .

Original: G.3 Choice of α
Revised: G.3 Choice of α

Original: α controls the width of the optimistic confidence bound.
Revised: α controls the width of the optimistic confidence bound.

Original: Specifically, it dictates that the real mean is smaller than the optimistic index with high probability, i.e. with probability 1 − 1 t α at step t .
Revised: Specifically, it dictates that the real mean is smaller than the optimistic index with high probability, i.e. with probability 1 − 1 t α at step t .

Original: The requirement that α > 3 is due to our analysis of the algorithm.
Revised: The requirement that α > 3 is due to our analysis of the algorithm.

Original: To be specific, the requirement that α > 3 is needed to use a sum-integral inequality to bound Term 2 of
Revised: To be specific, the requirement that α > 3 is needed to use a sum-integral inequality to bound Term 2 of

Original: Step 3 in the proof of Theorem 11.
Revised: Step 3 in the proof of Theorem 11.

Original: We leave it for future work to relax this requirement.
Revised: We leave it for future work to relax this requirement.

Original: The experiments are done with α = 3 . 1 to comply with the theoretical analysis.
Revised: The experiments are done with α = 3 . 1 to comply with the theoretical analysis.

Original: As shown in Figure 8, choosing α = 1 works better experimentally.
Revised: As shown in Figure 8, choosing α = 1 works better experimentally.

Original: This observation complies with the theoretical results, since the dominant terms in the regret upper bounds of both AdaP - UCB and AdaP - KLUCB are multiplicative in α .
Revised: This observation complies with the theoretical results, since the dominant terms in the regret upper bounds of both AdaP - UCB and AdaP - KLUCB are multiplicative in α .

Original: A tighter analysis might give us a bound for α = 1 and close the multiplicative gap between the regret’s lower and upper bound.
Revised: A tighter analysis might give us a bound for α = 1 and close the multiplicative gap between the regret’s lower and upper bound.

Original: Reflecting this phenomenon in the analysis will be an interesting future work to pursue.
Revised: Reflecting this phenomenon in the analysis will be an interesting future work to pursue.