Title: Private Online Learning via Lazy Algorithms
Venue: NeurIPS 2024
Accepted: yes
Keywords: Differential Privacy

Online Optimization
Abstract: We study the problem of private online learning, specifically, online prediction from experts (OPE) and online convex optimization (OCO). 
   We propose a new transformation that transforms lazy online learning algorithms into private algorithms. We apply our transformation for differentially private OPE and OCO using existing lazy algorithms for these problems. Our final algorithms obtain regret which significantly improves the regret in the high privacy regime $\varepsilon \ll 1$, obtaining $\sqrt{T \log d} + T^{1/3} \log(d)/\varepsilon^{2/3}$ for DP-OPE and $\sqrt{T} + T^{1/3} \sqrt{d}/\varepsilon^{2/3}$ for DP-OCO. We also complement our results with a lower bound for DP-OPE, showing that these rates are optimal for a natural family of low-switching private algorithms.

Private Online Learning via Lazy Algorithms

Abstract

We study the problem of private online learning, focusing on online prediction from experts (OPE) and online convex optimization (OCO). We propose a new transformation that translates lazy, low-switching online learning algorithms into private algorithms. We apply our transformation to differentially private OPE and OCO using existing lazy algorithms for these problems. The resulting algorithms attain regret bounds that significantly improve over prior art in the high privacy regime, where ε ≪ 1 , obtaining O ( √ T log d + T 1 / 3 log( d ) /ε 2 / 3 ) regret for DPOPE and O ( √ T + T 1 / 3 √ d/ε 2 / 3 ) regret for DP-OCO. We complement our results with a lower bound for DP-OPE, showing that these rates are optimal for a natural family of low-switching private algorithms.

1 Introduction

Online learning is a fundamental problem in machine learning, where an algorithm interacts with an oblivious adversary for T rounds. First, the oblivious adversary chooses T loss functions ℓ 1 , . . . , ℓ T : X → R over a fixed decision set X . Then, at any round t , the algorithm chooses a model x t ∈ X , and the adversary reveals the loss function ℓ t . The algorithm suffers loss ℓ t ( x t ) , and its goal is to minimize its cumulative loss compared to the best model in hindsight, namely its regret :

In this work, we study two different differentially private instances of this problem: differentially private online prediction from experts (DP-OPE) where the model x can be chosen from d experts ( X = [ d ] ); and differentially private online convex optimization (DP-OCO) where the model belongs to a convex set X ⊂ R d .

Both problems have been extensively studied recently [JKT12, ST13, JT14, AS17, KMS + 21] and an exciting new direction with promising results for this problem is that of designing private algorithms based on low-switching algorithms for online learning [AFKT23b, AFKT23a, AKST23a, AKST23b]. The main idea in these works is that the privacy cost for privatizing a low-switching algorithm can be significantly smaller as these algorithms do not update their models too frequently, allowing them to allocate a larger privacy budget for each update. This has been initiated by [AFKT23b], which used the shrinking dartboard algorithm to design new algorithms for DP-OPE, later revisited by [AKST23a] to design new algorithms for DP-OCO using a regularized follow-the-perturbedleader approach, and more recently by [AKST23b] which used a lazy and regularized version of the multiplicative weights algorithm to obtain improved rates for DP-OCO.

While all of these results build on lazy-switching algorithms for designing private online algorithms, each one of them has a different method for achieving privacy and, to a greater extent, a different analysis. Moreover, it is not clear whether these transformations from lazy to private algorithms in prior work have fulfilled the full potential of lazy algorithms for private online learning and whether better algorithms are possible through this approach. Indeed, the regret obtained in prior work [AFKT23b, AKST23b] is T 1 / 3 /ε (omitting dependence on d ) for DP-OPE, which implies that the normalized regret is 1 /T 2 / 3 ε : this is different than what exhibited in a majority of scenarios of private optimization, where the normalized error is usually a function of Tε .

1

0

0

.

2

0

.

4

0

.

6

0

.

8

1

0

.

4

0

.

6

0

.

8

1

α

β

L2P

[AKST23b]

[KMS

+

21]

non-private

(a) DP-OCO (

d

= log

T

)

0

0

.

2

0

.

4

0

.

6

0

.

8

1

0

.

4

0

.

6

0

.

8

1

α

β

L2P

[AKST23b]

[KMS

+

21]

non-private

(b) DP-OCO (

d

=

T

1

3

)

0

0

.

2

0

.

4

0

.

6

0

.

8

0

.

4

0

.

6

0

.

8

1

α

β

L2P

[AFKT23b]

non-private

(c) DP-OPE

1.1 Our contributions

Our main contribution in this work is a new transformation that converts lazy online learning algorithms into private algorithms with similar regret guarantees, resulting in new state-of-the-art rates for DP-OPE and DP-OCO. We provide a summary in Table 1 and Figure 1.

L2P : a transformation from lazy to private algorithms (Section 3). Our main contribution is a new transformation, we call L2P , that allows converting any lazy algorithm into a private one with only a slight cost in regret. This allows us to use a long line of work on lazy online learning [KV05, GVW10, AT18, CYLK20, SK21, AKST23a] to design new algorithms for the private setting. Our transformation builds on two new techniques: first, we design a new switching rule that only depends on the loss at the current round, so as to minimize the privacy cost of each switching and mitigate the accumulation of privacy loss. Second, we rely on a simple, key observation that by grouping losses in a large batch, we can minimize the effect on the regret of lazy online learning algorithms. We introduce a new analysis for the regret of lazy online algorithms with a large batch size that improves over the existing analysis in [AFKT23b]; this allows us to reduce the total number of 'fake switches' needed to guarantees privacy, improving the final regret.

Faster rates for DP-OPE (Section 3.1). As a first application, we use our transformation in the DP-OPE problem on the multiplicative weights algorithm [AHK12]. This results in a new algorithm for DP-OPE that has regret √ T log( d ) + T 1 / 3 log( d ) /ε 2 / 3 , improving over the best existing results for the high-dimensional regime in which the regret is √ T log( d ) + T 1 / 3 log( d ) /ε [AFKT23b]. 2

The improvement is particularly crucial in the high-privacy regime, where ε ≪ 1 : indeed, our regret shows that (for d = poly ( T ) ) it is sufficient to set ε ≥ T -1 / 4 for matching the optimal non-private regret √ T log d , whereas previous results require a much larger ε ≥ T -1 / 6 to get privacy for free. This is also important in practice, when multiple applications of DP-OPE are necessary: using advanced composition, our result shows that we can solve K ≈ √ T instances of DP-OPE with ε = 1 and still obtain the non-private regret of order √ T ; in contrast, prior work only allows to solve K ≈ T 1 / 3 instances while still attaining the non-private regret.

Faster rates for DP-OCO (Section 3.2). As another application, we use our transformation for DP-OCO with the regularized multiplicative weights algorithm of [AKST23b]. We obtain a new algorithm for DP-OCO that has regret √ T + T 1 / 3 √ d/ε 2 / 3 , improving over the best existing results that established regret √ T + T 1 / 3 √ d/ε + T 3 / 8 √ d/ε 3 / 4 [AKST23b] or d 1 / 4 √ T/ √ ε using DP-FTRL [KMS + 21].

Lower bounds for low-switching private algorithms (Section 4). To understand the limitations of low-switching private algorithms, we prove a lower bound for the natural family of private algorithms with limited switching, showing that the upper bounds obtained via our reduction are nearly tight for this family of algorithms up to logarithmic factors. This shows that new techniques, beyond limited switching, are required in order to improve upon our upper bounds.

Related work. Our transformation and algorithms build on a long line of work in online learning with limited switching [KV05, GVW10, AT18, CYLK20, SK21, AKST23b]. As is evident from prior work in private online learning, the problems of lazy online learning and private online learning are tightly connected [AFKT23b, AFKT23a, AKST23a, AKST23b]. In this problem, the algorithm wishes to minimize its regret while making at most S switches: the algorithm can update the model at most S times throughout the T rounds. Recent work has resolved the lazy OPE problem: [AT18] show a lower bound of √ T +( T/S ) log( d ) on the regret, which is achieved by several algorithms such as Follow-the-perturbed-leader [KV05] and the shrinking dartboard algorithm [GVW10]. For lazy OCO, however, optimal rates are yet to be known: [AKST23b] recently show that a lazy version of the regularized multiplicative weights algorithm obtains regret √ T +( T/S ) √ d , whereas the best lower bound is √ T + T/S [SK21].

2 Preliminaries

2.1 Problem setup

We consider an interactive T -round game between an algorithm ALG and an oblivious adversary Adv . Before the interaction, the adversary Adv chooses T loss functions ℓ 1 , . . . , ℓ T ∈ L = { ℓ | ℓ : X → R } . Then, at each round t ∈ [ T ] , the algorithm ALG , which observed ℓ 1 , · · · , ℓ t -1 chooses x t ∈ X , and then the loss function ℓ t chosen by Adv is revealed. The regret of the algorithm ALG is defined below:

We study online optimization under the constraint that the algorithm is differentially private. For an algorithm ALG and a sequence S = ( ℓ 1 , . . . , ℓ T ) chosen by an oblivious adversary Adv , we let ALG ( S ) := ( x 1 , . . . , x T ) denote the output of ALG over the loss sequence S . We have the following definition of privacy against oblivious adversaries. 3

Definition 2.1 (Differential privacy) . A randomized algorithm ALG is ( ε, δ ) -differentially private against oblivious adversaries ( ( ε, δ ) -DP) if, for all neighboring sequences S = ( ℓ 1 , . . . , ℓ T ) ∈ L T

and S ′ = ( ℓ ′ 1 , . . . , ℓ ′ T ) ∈ L T that differ in a single element, and for all events O in the output space of ALG , we have

We focus on two important instances of differentially private online optimization:

(i) DP Online Convex Optimization (DP-OCO). In this problem, the adversary picks loss functions ℓ ∈ L OCO := { ℓ | ℓ : X → R is convex and L -Lipschitz } where X ⊂ R d is a convex set with diameter D = diam ( X ) := sup x,y ∈X ∥ x -y ∥ , and the algorithm chooses x t ∈ X . The goal of the algorithm is to minimize regret while being ( ε, δ ) -differentially private.

(ii) DP Online Prediction from Experts (DP-OPE). In this problem, the adversary picks loss functions ℓ ∈ L OPE = { ℓ | ℓ : [ d ] → [0 , 1] } where X = [ d ] is the set of d experts, and the algorithm chooses x t ∈ [ d ] . The goal of the algorithm is to minimize regret while being ( ε, δ ) -differentially private.

2.2 Tools from differential privacy

Our analysis crucially relies on the following divergence between two distributions.

Definition 2.2 ( δ -Approximate Max Divergence) . For two distributions µ and ν , we define

We let D δ ∞ ( µ, ν ) := max { D δ ∞ ( µ ∥ ν ) , D δ ∞ ( ν ∥ µ ) } .

We also use the notion of indistinguishability between two distributions.

Definition 2.3. ( ( ε, δ ) -indistinguishability) Two distributions µ, ν are ( ε, δ ) -indistinguishable, denoted µ ≈ ( ε,δ ) ν , if D δ ∞ ( µ, ν ) ≤ ε .

Note that if an algorithm ALG has ALG ( S ) ≈ ( ε,δ ) ALG ( S ′ ) for all neighboring datasets S , S ′ then ALG is ( ε, δ ) -differentially private. We direct readers to Appendix A for additional background information and detailed preliminaries.

3 L2P: From Lazy to Private Algorithms for Online Learning

This section presents our L2P transformation, which turns lazy online learning algorithms into private ones. The transformation has an input algorithm A with measure µ t at round t and samples x t from the normalized measure µ t , which satisfies the following condition:

Assumption 3.1. The online algorithm A has at time t a measure µ t that is a function of ℓ 1 , . . . , ℓ t -1 (and density function µ t ) such that for some δ 0 ≤ 1 and 0 < η ≤ 1 / 10 that are data-independent, we have

D δ 0 ∞ ( µ t +1 , µ t ) ≤ η ,

µ t +1 ( x ) /µ t ( x ) = func ( ℓ t , x ) for all x ∈ X where func is a data-independent function.

While algorithms satisfying Assumption 3.1 need not be lazy, this assumption is satisfied by most existing lazy online learning algorithms such as the shrinking dartboard (Section 3.1) and lazy regularized multiplicative weights (Section 3.2). Moreover, any algorithm that satisfies this assumption can be made lazy via our reduction.

Technique Overview: Suppose the neighboring datasets differ from the s 0 -th loss function. The high-level intuition behind our framework is that our algorithm only loses the privacy budget when it makes a switch (draws a fresh sample) whenever t > s 0 . Hence, in the framework, we try to make the algorithm make as few switches as possible. This modification can lead to additional regret compared to lazy online learning algorithms, and we need to balance the privacy-regret trade-off. The family of

low-switching algorithms is ideal for privatization because its built-in low-switching property can achieve a better trade-off.

Our starting point is the ideas in [AFKT23b, AKST23b] to privatize low-switching algorithms, which use correlated sampling to argue that a sample from x t -1 ∼ µ t -1 is likely a good sample from µ t and therefore switching at round t is often not necessary. In particular, at round t , these algorithms sample a Bernoulli random variable S t ∼ Ber ( c · µ t ( x t -1 ) /µ t -1 ( x t -1 )) for some constant c and use the same model x t = x t -1 if S t = 1 , and otherwise sample new model x t ∼ µ t if S t = 0 (which happens with small probability). This guarantees that the marginal probability of the lazy iterates remains the same as the original iterates. Finally, to preserve the privacy of the switching decisions, existing algorithms add a fake switching probability p where the algorithm switches independently of the input. To summarize, existing low-switching private algorithms work roughly as follows:

This sketch is the starting point of our transformation, and we will introduce two new components to improve performance. The first component aims to avoid the accumulation of privacy cost for switching in the current approaches where each user can affect the switching probability for all subsequent rounds: this happens since µ t ( x t -1 ) /µ t -1 ( x t -1 ) is usually a function of the whole history ℓ 1 , . . . , ℓ t , and hence the existing low-switching private algorithms lose the privacy budget even it does not make real switches. To address this, we deploy a new correlated sampling strategy in L2P where the loss ℓ t at time t affects the switching probability only at time t , hence paying a privacy cost for switching only in a single round. To this end, we construct a parallel sequence of models { y t } t ∈ [ T ] (independent of x t ) that is used for normalizing the ratio µ t ( x t -1 ) /µ t -1 ( x t -1 ) to become independent of the history. In particular, at round t , we switch with probability proportional to

The main observation here is that µ t ( x t -1 ) µ t -1 ( x t -1 ) · µ t -1 ( y t -1 ) µ t ( y t -1 ) = µ t ( x t -1 ) µ t -1 ( x t -1 ) · µ t -1 ( y t -1 ) µ t ( y t -1 ) and this ratio is a function of ℓ t our input online learning algorithms which satisfies Assumption 3.1. This will, therefore, improve the privacy guarantee of the final algorithm.

The second main observation in L2P is that having a large batch size (batching rounds together) does not significantly affect the regret of lazy online algorithms compared to non-lazy algorithms but can further reduce the times to make switches and save the privacy budget. Our main novelty is a new analysis of the effect of batching on the regret of lazy algorithms (Proposition 3.3), which states that running a lazy online algorithm with a batch size of B would have an additive error of TB 2 η 2 to the regret where η is a measure of distance between µ t and µ t -1 . This significantly improves over existing analysis by [AFKT23b, Theorem 2] which shows that batching can add an additive term of B/η to the regret.

Having reviewed our main techniques, we proceed to present the full details of our L2P transformation in Algorithm 1, denoting ν s = µ ( s -1) B +1 where B is the batch size.

The regret of our transformation depends on the regret of its input algorithm. For the measure { µ t } T t =1 , we denote its regret

The following theorem summarizes the main guarantees of Algorithm 1.

Theorem 3.2. Let p ∈ (0 , 1) and B ∈ N . Assuming Assumption 3.1, Tp/B ≥ 1 , and for any δ 1 > 0 such that ηB log(1 /δ 1 ) /p ≤ 1 , our transformation L2P is ( ε, δ ) -DP with

and has regret

We begin by proving the utility guarantees of our transformation. It will follow directly from the following proposition, which bounds the regret of running L2P over a lazy online learning algorithm.

Proposition 3.3 (Regret of Batched Lazy Algorithm) . Let ALG be an online learning algorithm that satisfies Assumption 3.1. Let ηB log(1 /δ 1 ) /p ≤ 1 , and δ 1 , η < 1 / 2 . Then running L2P with the input algorithm ALG has regret

To prove Proposition 3.3, we first show that we can instead analyze the utility of a simpler algorithm that samples from ν s at each round. This is due to the following lemma, which shows that ∥ ̂ ν s -ν s ∥ TV is small where ̂ ν s is the marginal distribution of x s in Algorithm 1.

Lemma 3.4. Let ̂ ν s be the marginal distribution of x s in Algorithm 1. When ηB log(1 /δ 1 ) /p ≤ 1 , we have ∥ ̂ ν s -ν s ∥ TV ≤ 3( s -1)(2 e +log(1 /δ 1 ) /p ) Bδ 0 .

We also require the following lemma which allows to build a coupling over multiple variables, such that the variables are as close as possible. This will be used to construct a coupling between the lazy algorithm and the L2P algorithm that runs it.

̸

Lemma 3.5 ([AS19]) . Given a collection S of random variables, all absolutely continuous w.r.t. a common σ -finite measure. Then, there exists a coupling Γ , such that for any variables X,Y ∈ S , we have Pr[ X = Y ] ≤ 2 ∥ X -Y ∥ TV 1+ ∥ X -Y ∥ TV .

We are now ready to prove Proposition 3.3

Proof. Let Reg ′ T denote the regret when the marginal distribution of x t is ν t instead of ̂ ν t induced in the Algorithm. Since each loss function is bounded,

Thus, it now suffices to upper bound Reg ′ T .

Due to the preconditions that D δ 0 ∞ ( µ i +1 , µ i ) ≤ η and δ 0 ≤ η , we know ∥ µ i +1 -µ i ∥ TV ≤ 2 η . Recall that we assume x s ∼ ν s . Suppose z i is the action taken by the input lazy algorithm A for i ∈ [ T ] and the marginal distribution of z i is µ i . By Lemma 3.5, we can construct a coupling Γ s between x s and z := ( z ( s -1) B +1 , · · · , z sB ) , such that

̸

̸

Letting I s = 1 ( ∃ i ∈ [( s -1) B +1 , sB ] , z i = x s ) , we have

Hence we get Reg ′ T ≤ Reg T ( { µ t } T t =1 ) + T B · O ( B 3 η 2 ) , which completes the proof.

Now we turn to prove the privacy of L2P . We begin with the following lemma, which provides the privacy guarantees of sampling a new model x t from the distribution µ t . We defer the proof to Appendix B.

Lemma 3.6. Let { µ t } T t =0 satisfy Assumption 3.1 where η ≤ 1 / 10 . Then for any neighboring sequences S and S ′ with corresponding { µ t } T t =0 and { µ ′ t } T t =0 that differ one loss function, we have

We use correlated sampling in the algorithm rather than sampling from x t directly. To this end, we need the following lemma, which provides upper and lower bounds on the ratio used for correlated sampling.

Lemma 3.7. For any s ∈ [ T/B ] , if ηB log(1 /δ 1 ) /p ≤ 1 , then with probability at least 1 -(2 /η + log(1 /δ 1 ) /p ) · eBδ 0 -δ 1 ,

The privacy proof will build on the previous two lemmas to control the privacy cost of updating the model and the cost of the switching time. We defer the proof to Appendix B.

One remaining issue is we need to conditional on the high probability events in Lemma 3.7 for the privacy guarantee and can not directly apply Advanced Composition (Lemma A.3). Now, we modify the Advanced Composition for our usage. In the classic k -fold adaptive composition experiment, the adversary, after getting the first i -1 answers Y 1 , · · · , Y i -1 (denoted by Y [ i -1] for simplicity), can output two datasets D 0 i and D 1 i , a query q i , and receives the answer Y i ∼ M i ( D b i , q i ) for the secret bit b ∈ { 0 , 1 } . If each M i is ( ε i , δ i ) -DP, then the joint distributions over the answers Y [ k ] satisfy the advanced composition theorem.

In our case, however, we know there exists a subset G i -1 ( D b [ i -1] ) , such that with probability at least 1 -λ i , Y [ i -1] ∈ G i -1 ( D b [ i -1] ) . Conditional on Y [ i -1] ∈ ∩ b ∈{ 0 , 1 } G i -1 ( D b [ i -1] ) ,

Then we have the following lemma:

Lemma 3.8. Given the k mechanisms satisfying the Condition (1) , then the class of mechanisms satisfy (˜ ε ˜ δ , 1 -(1 -˜ δ )Π t ∈ [ k ] (1 -˜ δ t )) + 2 ∑ t ∈ [ k ] λ t -DP under k -fold adaptive composition, with ˜ ε ˜ δ defined in Equation (4) .

Proof. Without losing generality, suppose we know the adversary and how they generate the databases and queries. We can construct a series of mechanisms M ′ i , such that M ′ i draws Y i from M i ( D b i , q i ) , and outputs Y i if Y i ∈ ∩ b ∈{ 0 , 1 } G i -1 ( D b [ i -1] ) , and outputs 0 otherwise. Let ( Y ′ 1 ,b , · · · , Y ′ k,b ) be the outputs of M ′ i with secret bit b , and we know the TV distance between ( Y ′ 1 ,b , · · · , Y ′ k,b ) and ( Y 1 ,b , · · · , Y k,b ) is at most ∑ t ∈ [ k ] λ t for any b ∈ { 0 , 1 } . Moreover, we know

by the advanced composition. The basic composition finishes the proof.

3.1 Application to DP-OPE

This section discusses the first application of our transformation to differentially private online prediction from experts (DP-OPE). Towards this end, we apply our transformation over the multiplicative weights algorithms [AHK12], which can be made lazy as done in the shrinking dartboard algorithm [GVW10]. It has the following measure at round t

The following proposition shows that this measure satisfies the desired properties required by our transformation. We let µ mw t denote the density corresponding to µ mw t .

Lemma 3.9. Assume ℓ 1 , . . . , ℓ T where ℓ t : [ d ] → [0 , 1] . Then we have that

Proof. The first item follows from the guarantees of the exponential mechanism as ℓ t ( x ) ∈ [0 , 1] for all x ∈ [ d ] . The second item follows immediately from the definition of µ mw .

Having proved our desired properties, our transformation now gives the following theorem.

Theorem 3.10 (DP-OPE) . Let ℓ 1 , . . . , ℓ T where ℓ t : [ d ] → [0 , 1] . Setting B = 1 /ε and η = min( ε 0 , ε ) 2 / 3 /T 1 / 3 where ε 0 = T -1 / 4 log 3 / 4 d , the L2P transformation (Algorithm 1) applied with the measure { µ mw t } T t =1 is ( ε, δ ) -DP and has regret

Proof. First, based on theorem 3.2, note that the setting of B = 1 /ε and η ≤ min( ε 0 , ε ) 2 / 3 /T 1 / 3 where ε 0 = T -1 / 4 log 3 / 4 d guarantee the algorithm is ( ε, δ ) -DP.

To upper bound the regret, we use existing guarantees of the multiplicative weights algorithm [AHK12], combined with Theorem 3.2 to get that the regret is

where the second inequality follows by setting B = 1 /ε , and the third inequality follows by setting η ≤ min( ε 0 , ε ) 2 / 3 /T 1 / 3 , and the last inequality follows since ε 0 = T -1 / 4 log 3 / 4 d .

3.2 Application to DP-OCO

In this section, we use our transformation for differentially private online convex optimization (DPOCO) using the regularized multiplicative weights algorithm [AKST23b], which has the following measure

Letting µ rmw denote the corresponding density function, we have the following properties.

Lemma 3.11. Assume ℓ 1 , . . . , ℓ T : X → R be convex and L -Lipschitz functions. Then we have that

Proof. The first item follows from Lemma 3.5 in [GLL22, AKST23b]. The second item follows immediately from the definition of µ rmw t .

Combining these properties with our transformation, we get the following result.

Theorem 3.12 (DP-OCO) . Let ℓ 1 , . . . , ℓ T : X → R be convex and L -Lipschitz functions. Setting B = 1 2 ε log(1 /δ ) , λ = L D max { √ T, √ d log T η } , β = η 2 λ/ 20 L 2 , η = ε 2 / 3 T 1 / 3 log( T/δ ) and p = η/ε , the L 2 P transformation (Algorithm 1) applied with the measure { µ rmw t } T t =1 is ( ε, δ ) -DP and has regret

Proof. First, based on Theorem 3.2, note that there are three constraints to make the algorithm private:

Setting of B = 1 2 ε log(1 /δ ) , λ = L D max { √ T, √ d log T η } , β = η 2 λ/ 20 L 2 , η = ε 2 / 3 T 1 / 3 log( T/δ ) and p = η/ε guarantees the algorithm is ( ε, δ ) -DP.

For utility, we use theorem 3.2 with the existing regret bounds for the regularized multiplicative weights algorithm (Theorem 4.1 in [AKST23b]) to get that the algorithm has regret

4 Lower bound for low-switching private algorithms

In this section, we prove a lower bound for DP-OPE for a natural family of private low-switching algorithms that contains most of the existing low-switching private algorithms such as our algorithms and the ones in [AFKT23b, AKST23b]. Our lower bound matches our upper bounds for DP-OPE and suggests that new techniques beyond limited switching are required in order to obtain faster rates.

For our lower bounds, we will assume that the algorithm satisfies the following condition:

Condition 4.1. (Limited switching algorithms) The online algorithm ALG works as follows: at each round t , ALG is allowed to either set x t +1 = x t or sample x t +1 ∼ µ t +1 where µ t +1 is a function of ℓ 1 , . . . , ℓ t and is supported over X . The algorithm releases the resampling rounds { t 1 , . . . , t S } and models { x t 1 , . . . , x t S } .

Our lower bound will hold for algorithms that satisfy concentrated differential privacy. We use this notion as it allows to get tight characterization of the composition of private algorithms and in most settings have similar rates to approximate differential privacy. We can also prove a tight lower bound for pure differential privacy using the same techniques. We have the following lower bound for concentrated DP. We defer the proof to Appendix C.

Theorem 4.2. Let T ≥ 1 and ε ≥ 100 log 3 / 2 ( dT ) /T . If an algorithm ALG satisfies Condition 4.1 and is ε 2 -CDP, then there exists an oblivious adversary that chooses ℓ 1 , . . . , ℓ T : [ d ] → [0 , 1] such that the regret is lower bounded by

Finally, we note that this lower bound only holds for switching-based algorithms: indeed, the binary-tree-based algorithm of [AS17] obtains regret √ d log( d ) /ε which is better in the lowdimensional regime. This motivates the search for new strategies beyond limited switching for the high-dimensional regime.

5 Conclusion

In this paper, we proposed a new transformation that allows the conversion of lazy online learning algorithms into private algorithms and demonstrates two applications (DP-OPE and DP-OCO) where this transformation offers significant improvements over prior work. Moreover, for DP-OPE, we show a lower bound for natural low-switching-based private algorithms, which shows that new techniques are required for low-switching algorithms to improve our transformation's regret. This begs the question of whether the same lower bound holds for all algorithms or whether a different strategy that breaks the low-switching lower bound exists. As for DP-OCO, it is interesting to see whether better upper or lower bounds can be obtained. The current normalized regret, omitting logarithmic terms, is proportional to √ d/ ( εT ) 2 / 3 . This is different than most applications in private optimization where the normalized error is usually a function of √ d/ ( εT ) . Hence, it is natural to conjecture that the normalized regret can be improved to d 1 / 3 / ( εT ) 2 / 3 .

References

[AFKT23a] Hilal Asi, Vitaly Feldman, Tomer Koren, and Kunal Talwar. Near-optimal algorithms for private online optimization in the realizable regime. Proceedings of the 40th International Conference on Machine Learning , 2023.

[AFKT23b] Hilal Asi, Vitaly Feldman, Tomer Koren, and Kunal Talwar. Private online prediction from experts: Separations and faster rates. Proceedings of the Thirty Sixth Annual Conference on Computational Learning Theory , 2023.

[AHK12] Sanjeev Arora, Elad Hazan, and Satyen Kale. The multiplicative weights update method: a meta algorithm and applications. Theory of Computing , 8(1):121-164, 2012.

[AKST23a] Naman Agarwal, Satyen Kale, Karan Singh, and Abhradeep Thakurta. Differentially private and lazy online convex optimization. In The Thirty Sixth Annual Conference on Learning Theory , pages 4599-4632. PMLR, 2023.

[AKST23b] Naman Agarwal, Satyen Kale, Karan Singh, and Abhradeep Thakurta. Improved differentially private and lazy online convex optimization. arXiv:2312.11534 [cs.CR] , 2023.

[AS17] Naman Agarwal and Karan Singh. The price of differential privacy for online learning. In Proceedings of the 34th International Conference on Machine Learning , pages 32-40, 2017.

[AS19] Omer Angel and Yinon Spinka. Pairwise optimal coupling of multiple random variables. arXiv preprint arXiv:1903.00632 , 2019.

[AT18] Jason Altschuler and Kunal Talwar. Online learning over a finite action set with limited switching. In Proceedings of the Thirty First Annual Conference on Computational Learning Theory , 2018.

[BS16] Mark Bun and Thomas Steinke. Concentrated differential privacy: Simplifications, extensions, and lower bounds. In Theory of Cryptography - 14th International Conference, TCC 2016-B, Proceedings, Part I , volume 9985 of Lecture Notes in Computer Science , pages 635-658, 2016.

[CYLK20] Lin Chen, Qian Yu, Hannah Lawrence, and Amin Karbasi. Minimax regret of switchingconstrained online convex optimization: No phase transition. Advances in Neural Information Processing Systems , 33:3477-3486, 2020.

[GLL22] Sivakanth Gopi, Yin Tat Lee, and Daogao Liu. Private convex optimization via exponential mechanism. In Conference on Learning Theory , pages 1948-1989. PMLR, 2022.

[GVW10] Sascha Geulen, Berthold Vöcking, and Melanie Winkler. Regret minimization for online buffering problems using the weighted majority algorithm. In Proceedings of the Twenty Third Annual Conference on Computational Learning Theory , 2010.

[JKT12] Prateek Jain, Pravesh Kothari, and Abhradeep Thakurta. Differentially private online learning. In Proceedings of the Twenty Fifth Annual Conference on Computational Learning Theory , 2012.

[JT14] Prateek Jain and Abhradeep Thakurta. (Near) dimension independent risk bounds for differentially private learning. In Proceedings of the 31st International Conference on Machine Learning , pages 476-484, 2014.

[KMS + 21] Peter Kairouz, Brendan McMahan, Shuang Song, Om Thakkar, Abhradeep Thakurta, and Zheng Xu. Practical and private (deep) learning without sampling or shuffling. arXiv:2103.00039 [cs.CR] , 2021.

[KOV15] Peter Kairouz, Sewoong Oh, and Pramod Viswanath. The composition theorem for differential privacy. In International conference on machine learning , pages 1376-1385. PMLR, 2015.

[KV05] A. Kalai and S. Vempala. Efficient algorithms for online decision problems. Journal of Computer and System Sciences , 71(3):291-307, 2005.

[SK21] Uri Sherman and Tomer Koren. Lazy oco: Online convex optimization on a switching budget. In Proceedings of the Thirty Fourth Annual Conference on Computational Learning Theory , 2021.

[ST13] Adam Smith and Abhradeep Thakurta. (Nearly) optimal algorithms for private online learning in full-information and bandit settings. In Advances in Neural Information Processing Systems 26 , 2013.

A Missing details for preliminaries

In this section, we provide additional preliminaries and provide missing details for some of the results in the preliminaries section.

For our lower bounds, we require the notion of concentrated differential privacy. To this end, we first define the α -Renyi divergence ( α > 1 ) between two probability measures:

Concentrated DP is defined below:

Definition A.1 (concentrated DP) . Let ρ ≥ 0 . We say an algorithm ALG satisfies ρ -concentrated differential privacy ( ρ -CDP) against oblivious adversaries if for any neighboring sequences S = ( ℓ 1 , . . . , ℓ T ) ∈ L T and S ′ = ( ℓ ′ 1 , . . . , ℓ ′ T ) ∈ L T that differ in a single element, and any α ≥ 1 , D α ( ALG ( D ) ∥ ALG ( D ′ )) ≤ αρ .

Now we list a few standard results from the privacy literature that we use in the paper, namely group privacy and privacy composition.

Lemma A.2. (Group Privacy) Let ALG be an ( ε, δ ) -DP algorithm and let S , S ′ L T be two datasets that differ in k elements. Then for any measurable set S in the output space of ALG

Lemma A.3 (Advanced Composition,[KOV15]) . For any ε t > 0 , δ t ∈ (0 , 1) for t ∈ [ k ] , and ˜ δ ∈ (0 , 1) , the class of ( ε t , δ t ) -DP mechanisms satisfy (˜ ε ˜ δ , 1 -(1 -˜ δ )Π t ∈ [ k ] (1 -˜ δ t )) -DP under k -fold adaptive composition, for

Moreover, we use the fact that distributions with bounded D δ ∞ satisfy the following property.

Lemma A.4. Let ε ≤ 1 / 10 . If D δ ∞ ( µ, ν ) ≤ ε/ 2 then we have

Finally, we have the following standard conversion from ρ -concentrated DP to ( ε, δ ) -DP.

Lemma A.5 ([BS16]) . If ALG is ρ -CDP with ρ ≤ 1 , then it is (3 √ ρ log(1 /δ ) , δ ) -DP for all δ ∈ (0 , 1 / 4) .

A.1 Proof of Lemma A.4

Let S ′ = { X : µ ( X ) ν ( X ) ∈ [ e -ε , e ε ] } , and assume towards a contradiction that µ ( S ′ ) < 1 -6 δ/ε . Then there is a set S = S ′ , such that µ [ S ] > 6 δ/ε . We divide S by letting S 1 = { X ∈ S : µ ( X ) /ν ( X ) > e ε } and S 2 = S \ S 1 .

Case (1): µ ( S 1 ) ≥ µ ( S ) / 2 ≥ 3 δ/ε . Then we know

where we use that µ ( S 1 ) ≥ e ε ν ( S 1 ) and 1 -ε/ 3 ≥ e -ε/ 2 for ε < 1 / 10 . This is a contradiction.

Case (2): Suppose µ ( S 2 ) ≥ 3 δ/ε . We know S 2 = { X ∈ S : µ ( X ) /ν ( X ) < e -ε } = { X ∈ S : ν ( X ) /µ ( X ) > e ε } . Then we know ν ( S 2 ) ≥ e ε µ ( S 2 ) ≥ 3 e ε δ/ε . Similarly, we have

This is a contradiction as well and proves the statement.

B Missing Proofs for Section 3

B.1 Proof of Theorem 3.2

The regret bound follows directly from Proposition 3.3. It suffices to prove the privacy guarantee.

Fix two arbitrary neighboring datasets S and S ′ , and suppose the sequences differ at t 0 -step, that is { ℓ ( t 0 -1) B +1 , · · · , ℓ t 0 B } differ one loss function from { ℓ ′ ( t 0 -1) B +1 , · · · , ℓ ′ t 0 B } .

Define ζ t as the indicator that at least one of A t +1 , S t +1 and S ′ t +1 is zero. Let { ( x t , y t , ζ t ) } t ∈ [ T ] and { ( x ′ t , y ′ t , ζ ′ t ) } t ∈ [ T ] be the random variables with neighboring datasets. Let Σ t = { ( x τ , y τ , ζ τ ) } τ ∈ [ t ] be the random variables for the first t -iterations. We will argue that { ( x t , y t , ζ t ) } t ∈ [ T ] and { ( x ′ t , y ′ t , ζ ′ t ) } t ∈ [ T ] are indistinguishable, and privacy will follow immediately.

Let E t be the event such that ν t +1 ( x t ) ν t ( x t ) · ν t ( y t ) ν t +1 ( y t ) ∈ [ e -2 Bη , e 2 Bη ] . Hence we know Pr[ E t ] ≥ 1 -(2 + log(1 /δ 1 ) /p ) eBδ 0 -δ 1 by Lemma 3.7 for any t . Define E ′ t in a similar way. Moreover, let E G be the event that ∑ T/B t =2 1 ( A t = 0 or S ′ t = 0) ≤ 2 Tp log(1 /δ 1 ) /B . By Chernoff bound, we know

We also let E t be the event such that ν t +1 ( x t ) ν t ( x t ) · ν t ( y t ) ν t +1 ( y t ) ∈ [ e -2 Bη , e 2 Bη ] . Lemma 3.7 implies that Pr[ E t ] ≥ 1 -(2 /η +log(1 /δ 1 ) /p ) eBδ 0 -δ 1 for any t . Define E ′ t in a similar way.

Then it suffices to show { ( x t , y t , ζ t ) } t ∈ [ T ] and { ( x ′ t , y ′ t , ζ ′ t ) } t ∈ [ T ] are ( ε, δ x ) -indistinguishable conditional on E := E G ∪ E ′ G ∪ t ∈ [ T/B ] ( E t ∪ E ′ t ) . Then this will imply that { ( x t , y t , ζ t ) } t ∈ [ T ] and { ( x ′ t , y ′ t , ζ ′ t ) } t ∈ [ T ] are ( ε, δ x +(2 T +2) δ 1 +2 T (2 /η +log(1 /δ 1 ) /p ) eBδ 0 ) -indistinguishable.

Now we show, conditional on E , any value Σ such that Σ t -1 = Σ ′ t -1 = Σ , ( x t , y t , ζ t ) and ( x ′ t , y ′ t , ζ ′ t ) are ( ε t , δ 0 ) -indistinguishable where

Case 1 ( t < t 0 ) : It is clear that the claim is correct for t ≤ t 0 as ( x t , y t , ζ t ) and ( x ′ t , y ′ t , ζ ′ t ) have the same distribution then.

Case 2 ( t = t 0 ) : Now consider the case where t = t 0 .

Note that ( x t 0 , y t 0 ) and ( x ′ t 0 , y ′ t 0 ) have identical distributions, and it suffices to consider the indistinguishability of ζ t 0 and ζ ′ t 0 .

We have

Similarly, we have

Case 3 ( t > t 0 ) : As for the case when t > t 0 , when ζ t = 0 ( A t +1 = S t +1 = S ′ t +1 = 1 ), the variables are 0 -indistinguishable since x t = x t -1 and y t = y t -1 in this case. Consider the remaining possibility. Given the assumption that µ t +1 /µ t is a function of ℓ t , for any possible Σ , we have

For any set S , by the assumption on µ t , we have

where the inequality comes from Lemma 3.6 by the divergence bound between µ t and µ ′ t . This completes the proof of Equation (5).

The final privacy guarantee follows from combining Equation (5) and Advanced composition (Lemma A.3).

B.2 Proof of Lemma 3.4

We prove this statement by induction. For t = 1 , the statement is obviously correct. We assume ∥ ̂ ν t -ν t ∥ TV ≤ 3( t -1)(2( e/η +log(1 /δ 1 ) /p ) Bδ 0 + δ 1 ) prove that ∥ ̂ ν t +1 -ν t +1 ∥ TV ≤ 3 t (2 e + log(1 /δ 0 ) /p ) Bδ 0 .

Let X good := { x : log ν t +1 ( x ) ν t ( x ) ∈ [ -Bη,Bη ] } and Y good := { y : log ν t ( y ) ν t +1 ( y ) ≤ [ -Bη,Bη ] } . Let ̂ φ t ( y ) be the distribution of y t . Note that the distribution of y t is independent of { x τ } τ ∈ [ T/B ] , while the distribution of x t +1 is independent of y t +1 but depends on y t . By the assumption and group privacy, we know D Be Bη δ 0 ∞ ( ν t +1 , ν t ) ≤ Bη , and hence we have

Let t 0 ≤ t be largest integer such that A t 0 = 1 , that is, y t is sampled from ν t 0 for some random t 0 ≤ t . We have

With probability at least 1 -δ 1 , we know | t -t 0 | ≤ log(1 /δ 1 ) /p . Hence we get

We know

Denote the good set

Let ˜ φ t be the distribution of y t conditional on y t ∈ Y good . Let ̂ Γ t be the marginal distribution over ( x t , y t ) , that is x t ∼ ̂ ν t and y t ∼ ̂ φ t . Let Γ t be the distribution over ( x t , y t ) where x t ∼ ν t , y t ∼ ˜ φ t , and Γ t be the distribution of Γ t conditional on ( x t , y t ) ∈ S good .

We know ∥ ̂ Γ t -Γ t ∥ TV ≤ (2 e/η +log(1 /δ 1 ) /p ) Bδ 0 (3 t -2) . Let q t +1 be the distribution of x t +1 if ( x t , y t ) is sampled from Γ t instead of ̂ Γ t . By the property that post-processing does not increase the TV distance, we know

Now it suffices to bound the TV distance between q t +1 and ν t +1 . For any set E , we have

Thus we have

Note that for any ( x, y ) ∈ S good , we have

Fixing any y , we know the above term is bounded by

where the last inequality follows from ν t +1 ( X good ) ≥ 1 -Bδ 0 . Hence, we prove that

This suggests that

B.3 Proof of Lemma 3.6

Let S = ( ℓ 1 , . . . , ℓ T ) and S ′ = ( ℓ ′ 1 , . . . , ℓ ′ T ) differ in a single round t 0 . We fix t and prove the claim is correct. If t ≤ t 0 , then the claim clearly holds as µ t = µ ′ t . For t = t 0 + 1 , note that Assumption 3.1 implies that D δ 0 ∞ ( µ t 0 +1 , µ t 0 ) ≤ η and D δ 0 ∞ ( µ ′ t 0 +1 , µ t 0 ) ≤ η , hence by group privacy we get that D ( e η +1) δ 0 ∞ ( µ t , µ ′ t ) ≤ 2 η . Finally, for t > t 0 +1 , note that Assumption 3.1 implies that µ t = µ 0 · func ( ℓ 1 ) · func ( ℓ 2 ) · · · func ( ℓ t -1 ) and µ ′ t = µ 0 · func ( ℓ ′ 1 ) · func ( ℓ ′ 2 ) · · · func ( ℓ ′ t -1 ) . Thus, swapping the losses at rounds t -1 and t 0 results in the same distributions µ t and µ ′ t , therefore privacy follows from the same arguments as the case when t = t 0 +1 . The final claim follows as e η +1 ≤ 4 .

B.4 Proof of Lemma 3.7

To prove lemma 3.7, we first prove the same result under a simpler setting where x t ∼ ν t and y t ∼ ν t . Lemma B.1. For any 0 ≤ t ≤ T/B -1 , if Bη ≤ 1 / 20 , x t ∼ ν t and y t ∼ ν t independently, then with probability at least 1 -6 e Bη δ 0 /η ,

Proof. Let Z t = ∫ ν t ( x )d x . We know ν t = ν t /Z t by our notation. Then we have that

The statement follows from the Assumption 3.1 and the group privacy

Then the statement follows from Lemma A.4, the independence between x t , y t and Union bound.

We are now ready to prove Lemma 3.7.

Proof. Fix any t . Let t 0 ≤ t be largest integer such that A t 0 = 1 , that is, y t is sampled from ν t 0 for some random t 0 ≤ t . By the group privacy, we know D Be Bη ( t -t 0 ) δ 0 ( t -t 0 ) ∞ ( ν t , ν t 0 ) ≤ Bη ( t -t 0 ) .

Define the bad set

Therefore, we have that

By the CDF of the geometric distribution, we know with probability at least 1 -δ 1 , we get | t 0 -t | ≤ log(1 /δ 1 ) /p . Let E be the event that | t 0 -t | ≤ log(1 /δ 1 ) /p . Hence we know

C Missing proofs for Section 4

We prove a sequence of lemmas that are needed for the proof. The first lemma shows that the algorithm has to split the privacy budget across all resampling rounds. To this end, let S be a random variable that corresponds to the number of resampling steps in the algorithm, let T i be the random variable corresponding to the round of the i 'th resampling (where we let T i = T +1 if i > S ), and let Z i be the random variable corresponding to the model sampled at time T i (letting Z i = 1 if i > S ).

Lemma C.1. (Composition) Let S, T i , Z i and S ′ , T ′ i , Z ′ i denote the random variables for two neighboring datasets. Under the assumptions of Theorem 4.2, if ALG is ε 2 -CDP, then for all α ≥ 1

Proof. As ALG is ε 2 -concentrated DP and outputs T 1 , . . . , T S and Z 1 , . . . , Z S , we have that

where the second inequality follows as the random variables T i , Z i and T ′ j , Z ′ j are constant for i > S and j > S ′ , and the last inequality follows as Z i is independent of ( T 1 , . . . , T i ) and ( Z 1 , . . . , Z i -1 ) given T i .

We defer the proof of the following Lemma to the appendix.

Lemma C.2. Let T ≥ 1 , ε ≤ 1 /T and δ ≤ 1 / 2 . Assume ℓ : [ d ] →{ 0 , 1 } where ℓ [ x ] ∼ Ber (1 / 2) for each x ∈ [ d ] . Let D = ( ℓ, . . . , ℓ ) and let ALG be an ( ε, δ ) -DP algorithm that outputs ( z 1 , . . . , z T ) = ALG ( D ) . Then

We are now ready to prove our main lower bound.

Proof. (of Theorem 4.2) We consider the following construction for the lower bound: the adversary sets S adv = ( Tε ) 2 / 3 , the sequence of losses will have E = S 2 adv epochs, each of size B = T/E = T/ ( Tε ) 4 / 3 = 1 ( Tε ) 1 / 3 ε . Inside each epoch, the adversary samples ℓ ∼ Ber (1 / 2) d and plays the same loss function for the whole epoch.

Let S be random variable denoting the number of switches in the algorithm. In this case, we argue that each switch must have a small privacy budget (Lemma C.1), and thus, the price inside each epoch has to be large (Lemma C.2). Let T 1 , . . . , T S be the rounds where the algorithm resamples ( T i = T +1 for i > S ) and let Z 1 , Z 2 , . . . , Z S be the resampled models ( Z i = 1 for i > S ). Lemma C.1 implies that

Now note that inside an epoch e , if the algorithm does not switch, then it will suffer loss B/ 2 in that epoch. Otherwise, if it switches, assume without loss of generality there is at most one switch inside each epoch (see Lemma C.2). Let j e ∈ [ T ] denote the index such that Z j e was sampled in epoch e . Note that the algorithm in this epoch has D α ( Z j e || Z ′ j e | T j e ) = αε 2 e , hence it is ε 2 e -CDP. Standard conversion from concentrated DP to approximate DP (Lemma A.5) implies that it is (3 ε e √ log(1 /δ ) , δ ) -DP where δ ≤ 1 /T 3 d . Hence Lemma C.2 implies the error for this epoch is B · ( 1 2 -3 Bε e √ log(1 /δ ) 2 ) -1 /T . Letting E switch ⊂ [ E ] denote the epochs where there is a switch, we have that the loss of the algorithm is

where the last inequality follows since ∑ e ∈ E switch ε e ≤ √ E ∑ E e =1 ε 2 e ≤ √ Eε . Note also that the loss of the best expert is

Overall we get that the regret of the algorithm is

where ( i ) follows since E ≤ ( Tε ) 4 / 3 , and ( ii ) holds since 3 √ log(1 /δ ) 2 ε ≤ T 1 / 3 2 ε 2 / 3 for ε ≥ 100 log 3 / 2 ( dT ) /T ≥ 27 log 3 / 2 (1 /δ ) /T . The claim follows.

C.1 Proof of Lemma C.2

Proof. For this lower bound, we assume that the algorithm has full access to D to release z 1 , . . . , z T . First, note that if the algorithms picks z = z i with probability 1 /T and releases ( z, . . . , z ) , then it has the same error since

Therefore, we assume that the algorithm releases a single z = ALG ( D ) that is ( ε, δ ) -DP. Denote D ℓ = ( ℓ, . . . , ℓ ) . Note that as we sample ℓ ∼ Ber (1 / 2) d , the probability p := Pr( ℓ = ℓ 0 ) = Pr( ℓ = ℓ 1 ) for all ℓ 0 , ℓ 1 ∈ { 0 , 1 } d . Letting ℓ = 1 -ℓ , we have that

Now note that for any ℓ 0 we have

where the first inequality follows since ALG is ( ε, δ ) -DP and group privacy. The claim follows

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We provide new algorithms with improved guarantees for heavy-tailed private SCO in several settings, which is what we claim in the abstract and introduction.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: The paper discusses places where the main result is lossy, providing an appendix section dedicated towards improving it to be optimal. It also compares its results to another similar bound in the literature, discussing regimes where our bound is weaker.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We provide a full set of verifiable details for all of our theoretical results.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [NA]

Justification: The paper does not include experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [NA]

Justification: The paper does not include experiments requiring code.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [NA]

Justification: The paper does not include experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [NA]

Justification: The paper does not include experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [NA]

Justification: The paper does not include experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We read the ethics guidelines and believe we meet them.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: We do not believe our paper poses a significant negative societal impact, as it is about making existing learning algorithms differentially private under less stringent distributional assumptions, which we do not foresee being used in any significant malicious cases. We do believe that our algorithms can have positive societal impacts, but do not wish to overclaim to this effect because our results are primarily theoretical.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper does not release data or models.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [NA]

Justification: The paper does not use existing assets.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
comments: SUMMARY:
This paper studies private online prediction from experts (OPE) and online convex optimization (OCO) problems and proposes a general transformation that converts lazy (non-private) algorithms into private algorithms. By applying it to existing lazy algorithms, they obtain improved regret bounds for both problems. A lower bound for lazy algorithms is also provided, suggesting that new techniques other than lazy algorithms are needed if one wants a better regret.

STRENGTHS:
1. The proposed method is a general transformation, namely, it applies to any algorithms that satisfy certain properties.
2. The results obtained outperform previous bounds, especially in the high privacy regime ($\varepsilon \ll 1$).
3. A matching lower bound is provided.

WEAKNESSES:
1. The improvement made in this paper may not be very significant. It only improves the dependency on $\varepsilon$, while in many scenarios $\varepsilon$ is treated as a constant.
2. The lower bound may not be essentially matched -- Theorem 4.2 requires the algorithm to be $\varepsilon^2$-CDP instead of $(\varepsilon,\delta)$-DP. It does not mean that every $(\varepsilon,\delta)$-DP algorithm should have that lower bound.

QUESTIONS:
1. The paper assumes an oblivious adversary. Can the results be extended to the adaptive adversarial setting?
2. It looks like Condition 4.1 is not limited to algorithms with a small number of switches. It only requires that once the algorithm resamples, the resampling distribution depends only on past loss functions and not on internal randomness. Is my understanding correct?

LIMITATIONS:
Irrelevant.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates private online learning, focusing on online prediction from experts (OPE) and online convex optimization (OCO). The authors propose a new transformation that translates lazy, low-switching online learning algorithms into private algorithms. Based on this transformation, their resulting algorithms attain regret bounds that significantly improve over prior art in the high privacy regime. Furthermore, they complement their results with a lower bound for DP-OPE, showing that these rates are optimal for a natural family of low-switching private algorithms.\n\nSTRENGTHS:\n* The improvements in results for DP-OPE and DP-OCO are highly valuable. \n* The research motivation of this work is clear, and the proposed transformation (L2P) is a very solid contribution to DP online learning.\n\nWEAKNESSES:\n* The transformation in this work is based on low-switching online algorithms, and is only applicable to oblivious adversaries. In contrast, previous DP-OCO algorithms [ST13, AS17, KMS+21]  can adapt to non-oblivious setting. \n* The theoretical innovation in this paper is limited, as it largely follows previous work [AFKT23b].\n\nQUESTIONS:\n* In footnote 2, the authors state that their algorithms will satisfy a stronger notion of differential privacy against adaptive adversaries. However, I believe that for non-oblivious adversaries, the proposed DP online algorithms in this work cannot provide regret guarantees. Is this correct? If so, I would like to understand why the low-switching private algorithms cannot handle non-oblivious adversaries, whereas previous work using binary tree techniques [ST13, AS17, KMS+21]  can.\n* From Theorem 3.2, it is evident that $\\epsilon$ of L2P increases over time, meaning that its privacy weakens progressively. Could this be a concern for the proposed DP algorithm? \n* The authors established a lower bound for low-switching private online algorithms based on **CDP** analysis, whereas the upper bound provided in this paper is based on **DP** analysis, i.e, Theorem 3.2 and 3.9. Could you clarify the implications of this difference?\n\nLIMITATIONS:\nSee above", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}

comments: SUMMARY:
The paper studies the differentially private (DP) variants of the classical online prediction form experts problem (OPE) and online convex optimization (OCO) problem. The main contribution is a (black-box) approach to transform lazy (i.e. slow-varying) online learning algorithms into private algorithms. The paper shows that the transformation only incurs a small additional regret as compared to the regret for the classical OPE/OCO regret. In particular, for the  DP-OPE and DP-OCO, the proposed algorithms improve the O(1/eps) regret in previous works to O(1/eps^{2/3}). In addition, a matching regret lower bound is provided for the family of low-switching private algorithms for the DP-OPE problem.

STRENGTHS:
The paper is well-written with all claims supported by proofs. Compared to previous works, the L2P algorithm proposed uses a new correlated sampling strategy to avoid accumulation of privacy cost, and the paper improves the regret of lazy algorithms due to batching. In addition, the paper provides a lower bound for the DP-OPE problem, showing that limited switching is not enough to obtain faster rate, suggesting the need for other techniques.

WEAKNESSES:
The lower bound is only valid for the family of slow-varying algorithms, thus the algorithms proposed might not be optimal.

QUESTIONS:
The paper is relatively dense, with many definitions related to differential privacy (section 2), which might be unfamiliar for people who have little knowledge about differential privacy before. I’m wondering if all those definitions are necessary for the main part of the paper (maybe some can be moved to the appendix)?

LIMITATIONS:
Yes the paper has discussed the limitations and provides improvement in the appendix.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-12

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a new mechanism that converts lazy online learning algorithms into private algorithms. Unlike previous private online algorithms that use individualized privatized methods, the paper's new mechanism is a black box private algorithm that can be applied to many popular non-private methods such as the lazy shrinking dartboard algorithm or the regularized multiplicative weights algorithms.\n\nSTRENGTHS:\n- The problem is well-motivated. Even though previous private algorithms have good theoretical guarantees, their privatized mechanism is usually tailor-made to solve a specific problem, which limits their applicability. The new black box algorithm makes converting a non-private to a private one simple and seamless.\n\n-  The private term has improved dependence in the $\\epsilon$ term which could be valuable in the high privacy regime (where $\\epsilon \\ll 1$). \n\n- The authors also provide lower bounds that match their upper bound.\n\n- The paper is well-written overall.\n\nWEAKNESSES:\n- I'm not sure if I agree with the claim that the new bound is an improvement over the previous bound. Obviously, the high privacy regime is very important, and as the authors have mentioned, this would allow for the composition of more instances. However, as far as I know, a lot of practical application of DP algorithm uses $\\epsilon \\approx O(1)$ so the new bound could be worse than previous results.\n\n- It would be nice if there are some proof of concept experiments to compare the new mechanism with the algorithms in previous works.\n\nQUESTIONS:\n- Can the author provide some intuition on why we would want to use concentrated DP to prove the lower bound? How does concentrated DP make it easier for us to construct the lower bound?\n\n- What are some non-lazy algorithms that satisfy Assumption 3.1? What would the regret bound look like for these non-lazy algorithms.\n\n- I'm a bit confused about the setting of $\\epsilon$ in Theorem 3.2. To get the optimal regret bound, we need to set $\\epsilon$ using the provided formula which I think is roughly $O(1)$. Doesn't that defeat the point of the proposed framework where we only see improvement when $\\epsilon <<1$.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-22"}