Title: Littlestone Classes are Privately Online Learnable
Authors: Noah Golowich
Roi Livni
Venue: NeurIPS 2021
Keywords: differential privacy
online classification
online learning

Abstract:     We consider the problem of online classification under a privacy constraint. In this setting a learner observes sequentially a stream of labelled examples $(x_t, y_t)$, for $1 \leq t \leq T$, and returns at each iteration $t$ a hypothesis $h_t$ which is used to predict the label of each new example $x_t$. The learner's performance is measured by her regret against a known hypothesis class $\mathcal{H}$. We require that the algorithm satisfies the following privacy constraint: the sequence $h_1, \ldots, h_T$ of hypotheses output by the algorithm needs to be an $(\epsilon, \delta)$-differentially private function of the whole input sequence $(x_1, y_1), \ldots, (x_T, y_T)$.

We provide the first non-trivial regret bound for the realizable setting. Specifically, we show that if the class $\mathcal{H}$ has constant Littlestone dimension then, given an oblivious sequence of labelled examples, there is a private learner that makes in expectation at most $O(\log T)$ mistakes -- comparable to the optimal mistake bound in the non-private case, up to a logarithmic factor. Moreover, for general values of the Littlestone dimension $d$, the same mistake bound holds but with a doubly-exponential in $d$ factor. 

    A recent line of work has demonstrated a strong connection between classes that are online learnable and those that are differentially-private learnable. Our results strengthen this connection and show that an online learning algorithm can in fact be directly privatized (in the realizable setting).

We also discuss an adaptive setting and provide a sublinear regret bound of $O(\sqrt{T})$.

TL;DR: We provide a differentially private online learning algorithm for Littlestone classes with finite mistake bound.

Review discussion:
The authors gave a clear introduction of the setting of interest, background on online learning and differential privacy, and the results are clearly stated.
On the other hand, I found the technical sections (5 and 6) harder to follow as the proofs are quite involved.
That said, I think the authors in general did a good job in presenting the algorithm, sketching the analyses, and explaining why the adaptive setting becomes more challenging.
This work adds on to a series of exciting recent results on the private learnability of binary hypothesis classes.
While being built on some ideas/tools from previous work, both the algorithm and its analysis are nontrivial.
Although the paper leaves a few important questions open (e.g., improving the dependence on both $d$ and $T$, and extending to the more challenging agnostic case), I found the current results quite solid and should be published.
Therefore, I recommend acceptance.
Minor comments:
- Line 99: "(private )"
- Line 206: Should all $(x_v, y_v)$ on this line be $(x_{\overline{v}}, y_{\overline{v}})$ instead?
===Added After Author Response===
I have read the authors' response as well as the other reviews. My overall evaluation of the paper remains the same.
No
Thanks for the positive feedback. We will make serious attempt to improve sections 5 and 6.
Regarding line 206: Yes, thanks, all $(x_v, y_v)$ on that line should be $(x_{\bar v}, y_{\bar v})$.
Strength: the work strengthens recent results connecting differential privacy and on line learnability by showing scenarios where on line learning can be directly endowed with differential privacy property.
Weakness: the regret grown doubly exponentially with the Littlestone dimension of the hypotheses class.
Originality: the main proof tool bears similarity with recent analogous results for the stochastic setting, however the authors have to overcome some specific new problems to adapt it to the on line setting.
Overall, I think the results are solid and consistute a significant contribution.
Clarity and presentation: The main algorithmic elements and proof techniques are well explained and the paper is easy to read.
However, there are several syntactic errors (words and parts of sentences are swapped, prepositions are missing, verbs are not properly inflected, etc.) and the paper would benefit from a thorough proofreading
No
Thank you for your positive feedback and comments.
Indeed, improving the double-exponential rate will strengthen our result, and we consider it as a future work.
The paper will go a thorough proofreading, and syntactic errors will be taken care of.
I think this is a solid contribution to the area of differentially-private learning.
While quantitatively the bounds are not great, e.g. the doubly-exponential dependence on the Littlestone dimension, I think formulating the problem and proving the first non-trivial upper bounds is substantial.
The paper is very well-written and easy to follow.
Technically, the paper relies on the property of global stability introduced in prior work. The authors show how to construct an online version of a globally stable algorithm to obtain the log(T) factor in the oblivious case with a novel application of the sparse vector technique. The adaptive case is achieved via a reduction to the oblivious case and using T-fold composition, but this incurs a sqrt{T} factor.
It might be helpful for readers of Section 6 if the authors could separate how an online version of a globally stable learner implies the online private learner, and how they construct this globally stable learner from SOA.
I would appreciate it if the authors could clarify the following. The authors considered requiring the sequence of hypotheses h_1,...,h_T to be differentially-private, but what about requiring only the sequence of predictions h_1(x_1),...,h_T(x_T) to be differentially-private? Is this notion of privacy different? is it useful?
In Theorem 4.1 and Theorem 4.2, the upper bounds on the mistake bound of the proposed algorithms incur some dependence on T (log(T) and sqrt{T}). We know that in standard online classification, in the realizable setting, no such dependence is incurred. Can the authors comment on whether such dependence on T is necessary in private online classification?
Can the authors clarify if their algorithm can be made to work with any black-box online learner? Is knowledge of the class H required or is having a black-box online learner sufficient?
Similar to online-to-batch conversions in standard classification, can private online learners be leveraged to construct private learners in the stochastic setting? This could be further motivation for studying the problem of private online learning.
Minor comments.
- Line 128: event E \subseteq W?
No
Thank you for the positive feedback!
> I would appreciate it if the authors could clarify the following. The authors considered requiring the sequence of hypotheses h_1,...,h_T to be differentially-private, but what about requiring only the sequence of predictions h_1(x_1),...,h_T(x_T) to be differentially-private? Is this notion of privacy different? is it useful?
This is a very good point, and we should add a discussion and compare the models. The models are indeed different and the guarantee in terms of privacy is weaker, but, potentially, might allow improved rates.
For instance, in the iid setting, it was studied in [1] below, where sample complexity linear in the VC dimension was shown to be achievable.
[1] Amos Beimel, Kobbi Nissim, Uri Stemmer. "Private Learning and Sanitization: Pure vs. Approximate Differential Privacy", 2014.
> Can the authors comment on whether such dependence on T is necessary in private online classification?
We are unaware of any lower bounds for the dependence on T, this is a good question and we will add a discussion.
> Can the authors clarify if their algorithm can be made to work with any black-box online learner? Is knowledge of the class H required or is having a black-box online learner sufficient?
The black-box online learner A used in DP-SOA can be any online learning algorithm which achieves the optimal mistake bound (in particular, it must change its internal state after making a mistake). Explicit knowledge of H is not required, though explicit knowledge of the hypotheses the algorithm A produces is required (e.g., to choose an x_{\bar v} in the 2nd line of the While loop of DP-SOA).
> Can private online learners be leveraged to construct private learners in the stochastic setting?
Yes, one can establish online-to-batch conversions for the setting of private learning (but currently this will not yield improved rates and will not even compare to the sample-efficient known rates for the stochastic setting).
>Line 128:
Yes, thanks, it should be $E \subset W$.
The paper is a strong submission and makes progress on an important question. Private learnability in the offline setting has been well-studied in recent years, with the results of Alon et al, Bun et al., and Ghazi et al. that have established the connection between finite Littlestone dimension and private learnability. How such a result extends to private *online* learnability is a natural question, and some works have studied regret bounds for private online convex optimization. This submission considers arbitrary hypothesis classes of finite Littlestone dimension and shows that a hypothesis class with Littlestone dimension d can be learned privately with a mistake bound that scales as 2^(O(2^d)).
The paper is fairly easy to follow, although some more inuition could improve the readability of Algorithm 1. There are some minor typos (e.g., the word "adversarial" was misspelled numerous times on page 3).
In terms of novelty: The techniques are based on ideas that have been explored in other works. The DP-SOA algorithm, for instances, relies on using copies of a non-private SOA algorithm on different subsequences.
Moreover, the mistake bounds proceed via construction of an online version of a globally stable algorithm, which draws upon ideas from Bun et al.
However, the way in which these ideas are tied together is nontrivial, which merits originality.
The paper appears to be technically sound, although I have not verified all the details of the proofs (e.g., material in the supplementary material).
No
We thank the reviewer for the comments and suggestions. We will fix typos as well as try to clarify algorithm 1.

Meta-review:
role: PC
data: comment: A very nice paper that does what it title says: gives a differentially-private online algorithm for classes of finite Littlestone dimension.  The paper is clearly written and gives a good background of the problem, and while the paper mostly makes use of known techniques, it still gives a nontrivial and interesting result.

Decision: Accept (Spotlight)