Littlestone Classes are Privately Online Learnable

√ 𝑇 ) . 1 Introduction . Privacy-preserving machine learning has attracted considerable attention in recent years , motivated by the fact that individuals ’ data is often collected to train statistical models , and such models can leak sensitive data about those individuals [ 13 , 32 ] . The notion of differential privacy has emerged as a central tool which can be used to formally reason about the privacy-accuracy tradeoffs one must make in the process of analyzing and learning from data . A considerable body of literature on differentially private machine learning has resulted , ranging from empirical works which train deep neural networks with a differentially private form of stochastic gradient descent [ 1 ] , to a recent line of theoretical works which aim to characterize the optimal sample complexity of privately learning an arbitrary hypothesis class [ 3 , 11 , 20 ] . Nearly all of these prior works on differentially private learning , however , are limited to the statistical learning setting ( also known as the offline setting ) : this is the setting where the labeled data , ( 𝑥𝑡 , 𝑦𝑡 ) , are assumed to be drawn i.i.d . from some unknown population distribution . This setting , while very well-understod and readily amenable to analysis , is unlikely to hold in practice . Indeed , the data ( 𝑥𝑡 , 𝑦𝑡 ) fed as input into the learning algorithm may shift over time ( e.g. , as a consequence of demographic changes in a population ) , or may be subject to more drastic changes which are adaptive to the algorithm ’ s prior predictions ( e.g. , drivers ’ reactions to the recommendations of route-planning apps may affect traffic patterns , which influence the input data to those apps ) . For this reason , it is desirable to develop provable algorithms which make fewer assumptions on the data . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . In this work , we do so by studying the setting of ( private ) online learning , in which the sequence of data ( 𝑥𝑡 , 𝑦𝑡 ) is allowed to be arbitrary , and we also discuss a certain notion of privacy in a setting where it is even allowed to adapt to the algorithm ’ s predictions in prior rounds . We additionally restrict our attention to the problem of classification , namely where the labels 𝑦𝑡 ∈ { 0 , 1 } ; thus we introduce the problem of differentially private online classification , and prove the following results ( see Section 3 for the exact setup ) : • In the realizable setting with an oblivious adversary , we introduce a private learning algorithm which , for hypothesis classes of Littlestone dimension 𝑑 ( see Section 2.1 ) and time horizon 𝑇 , achieves a mistake bound of ? ̃ ? ( 2𝑂 ( 2𝑑 ) · log𝑇 ) , ignoring the dependence on privacy parameters ( Theorem 4.1 ) . • In the realizable setting with an adaptive adversary , we show that a slight modification of the above algorithm achieves a mistake bound of ? ̃ ? ( 2𝑂 ( 2𝑑 ) · √ 𝑇 ) ( Theorem 4.2 ) . We remark that no algorithm ( even without privacy , allowing randomization , and in the oblivious adversary setting ) can achieve a mistake bound of smaller than Ω ( 𝑑 ) for classes of Littlestone dimension 𝑑 [ 30 , 33 ] . Therefore , a class of infinite Littlestone dimension can not have any finite mistake bound , and the regret for any algorithm , for any time horizon 𝑇 , is Ω ( 𝑇 ) . Thus , our results listed above , which show a mistake-bound ( which is also the regret in the realizable setting ) of ? ̃ ? 𝑑 ( √ 𝑇 ) for classes of Littlestone dimension 𝑑 , establish that in the realizable setting , finiteness of the Littlestone dimension is necessary and sufficient for online learnability ( [ 31 ] ) with differential privacy . Recently it was shown by Alon et al . [ 3 ] and Bun et al . [ 11 ] ( later to be improved by Ghazi et al . [ 20 ] ) that finiteness of the Littlestone dimension is necessary and sufficient for private learnability in the offline setting , namely with i.i.d . data ( and both in the realizable and agnostic settings ) . Since , as remarked above , the Littlestone dimension characterizes online learnability ( even without privacy ) , this means that a binary hypothesis class is privately ( offline ) learnable if and only if it is online learnable . Our result thus strengthens this connection , showing that the equivalence also includes private online learnability ( in the realizable setting ) . 1.1 Related work . A series of papers [ 15 , 25 , 21 , 17 , 2 ] has studied the problem of diferentially private online convex optimization , which includes specific cases such as private prediction from expert advice and , when one assumes imperfect feedback , private non-stochastic multi-armed bandits [ 35 , 36 , 18 , 24 ] . These results show that in many regimes privacy is free for such problems : for instance , for the problem of prediction from the expert advice ( with 𝑁 experts ) , Agarwal and Singh [ 2 ] shows that an ²-differentially private algorithm ( based on follow-the-regularized-leader ) achieves regret of 𝑂 ( √ 𝑇 + 𝑁 log 2 𝑇 ² ) , which matches the non-private regret bound of 𝑂 ( √︁ 𝑇 log 𝑁 ) when 𝑇 ≥ Ω̃ ( ( 𝑁/² ) 2 ) . Our results can be seen as extending such “ privacy is ( nearly ) free ” results to the nonparametric setting where we instead optimize over an arbitrary class of finite Littlestone dimension . Our techniques are different from those of the above papers . In addition to [ 11 , 20 ] which establish private learning algorithms for classes with finite Littlestone dimension in the i.i.d . ( offline ) setting , there has been an extensive line of work on private learning algorithms in the offline setting : [ 29 , 7 , 5 , 19 ] study the complexity of private learning with pure differential privacy , [ 26 , 9 , 10 , 4 ] study the sample complexity of privately learning thresholds , and [ 27 , 28 , 6 ] study the sample complexity of privately learning halfspaces . 2 Preliminaries . In this section we introduce some background concepts used in the paper . 2.1 Online Learning . We begin by revisiting the standard setting of online-learning : We consider a sequential game between a learner and an adversary . Both learner and adversary know the sets X and H. The game proceeds for 𝑇 rounds ( again 𝑇 is known ) and at each round 𝑡 ≤ 𝑇 , the adversary chooses a pair ( 𝑥𝑡 , 𝑦𝑡 ) and presents the learner with the example 𝑥𝑡 . The learner then must present the adversary with a hypothesis ( perhaps randomly ) ℎ𝑡 : X → { 0 , 1 } . ℎ𝑡 is not required to lie in H1 . Finally the adversary presents the learner with 𝑦𝑡 , which the learner uses to update its internal state . The performance of the learner is measured by its regret which is its number of mistake vs. the optimal decision in hindsight : 𝔼 [ 𝑇∑︁ 𝑡=1 1 [ ℎ𝑡 ( 𝑥𝑡 ) ≠ 𝑦𝑡 ] − min ℎ★∈H 𝑇∑︁ 𝑡=1 1 [ ℎ★ ( 𝑥𝑡 ) ≠ 𝑦𝑡 ] ] . ( 1 ) The adversary is said to be realizable if it presents the learner with a sequence of examples ( 𝑥𝑡 , 𝑦𝑡 ) so that there is some ℎ★ ∈ H so that for each 𝑡 ∈ [ 𝑇 ] , ℎ★ ( 𝑥𝑡 ) = 𝑦𝑡 . In the realizable setting , the regret simply counts the number of mistakes the learner makes . And we measure the performance by its mistake bound , namely the maximum , over all possible realizable adversaries , of 𝔼 [ 𝑇∑︁ 𝑡=1 1 [ ℎ𝑡 ( 𝑥𝑡 ) ≠ 𝑦𝑡 ] ] . In the setting with an agnostic adversary , we do not require such ℎ★ to exist ; and we measure the learner by its ( worst-case ) regret , as in Eq . ( 1 ) . In this paper we focus on the realizable setting ; the ( private ) agnostic setting is left as an interesting direction for future work . Additionally , we normally make a distinction between two types of adversaries : An oblivious adversary chooses its sequence in advance and at each iteration ( 𝑥𝑡 , 𝑦𝑡 ) is revealed to the learner . In the adversarial setting , the adversary may choose ( 𝑥𝑡 , 𝑦𝑡 ) as a function of the learner ’ s previous choices : i.e . ℎ1 , . . . , ℎ𝑡−1 . This definition follows the standard setup of online learning ( see [ 12 ] for example ) . We note though , that in the non-private setting of online binary classification , one can obtain results against an adversary that even gets to observe the learner ’ s prediction at time-step 𝑡 . However , we will simplify here by considering the more standard setting . It is interesting to find out if we can compete against such a strong adversary in the private setup . Littlestone dimension We next turn to introduce the Littlestone dimension which is a combinatorial measure that turns out to characterize learnability in the above setting . Let H be a class of hypotheses ℎ : X → { 0 , 1 } . To define the Littlestone dimension of H , we first introduce mistake trees : a mistake tree of depth 𝑑 is a complete binary tree , each of whose non-leaf nodes 𝑣 is labeled by a point 𝑥𝑣 ∈ X , and so that the two out-edges of 𝑣 are labeled by 0 and 1 . We associate each root-to-leaf path in a mistake tree with a sequence ( 𝑥1 , 𝑦1 ) , . . . , ( 𝑥𝑑 , 𝑦𝑑 ) , where for each 𝑖 ∈ [ 𝑑 ] , the 𝑖th node in the path is labeled 𝑥𝑖 and the path takes the out-edge from that node labeled 𝑦𝑖 . A mistake tree is said to be shattered by H if for any root-to-leaf path whose corresponding sequence is ( 𝑥1 , 𝑦1 ) , . . . , ( 𝑥𝑑 , 𝑦𝑑 ) , there is some ℎ ∈ H so that ℎ ( 𝑥𝑖 ) = 𝑦𝑖 for all 𝑖 ∈ [ 𝑑 ] . The Littlestone dimension of H , denoted Ldim ( H ) , is the depth of the largest mistake tree that is shattered by H. The Standard Optimal Algorithm ( SOA ) Suppose H is a binary hypothesis class with Littlestone dimension 𝑑. Littlestone [ 30 ] showed that there is an algorithm , called the Standard Optimal Algorithm ( SOA ) , which , against an adaptive and realizable adversary , has a mistake bound of 𝑑 ; moreover , this is the best possible mistake bound . We will access the SOA as a black box . The underlying assumption we make is that given a realizable sequence ( 𝑥1 , 𝑦1 ) , . . . , ( 𝑥𝑇 , 𝑦𝑇 ) , the 𝑆𝑂𝐴 makes at most Ldim ( H ) mistakes . We will also assume that whenever the algorithm 𝑆𝑂𝐴 makes a mistake then it changes it state : namely , if the algorithm makes mistake on example 𝑡 then ℎ𝑡+1 ≠ ℎ𝑡 , this is in fact true for the SOA algorithm , but it can be seen that any algorithm with mistake bound can be modified to make sure this holds ( simply by reiterating the mistake until the algorithm does change state ) . We refer the reader to [ 30 , 33 ] for the specifics of it .

Summary: The paper introduces the problem of online classification under a differential privacy constraint. There is a learner and an adversary. Specifically, it is required that the sequence of hypotheses outputted by the learner is (epsilon,delta)-differentially private in the sequence of examples presented by the adversary.  The paper focuses on the realizable setting, where the sequence of examples chosen by the adversary is consistent with some unknown hypothesis that lies in a hypothesis class known to both the learner and the adversary.  Two cases are studied. The adversary can be oblivious or adaptive. For oblivious adversaries, the paper proposes an algorithm with regret that is doubly exponential in Littlestone dimension and log(T) factor. For adaptive adversaries, an upper bound doubly exponential in Littlestone dimension times sqrt{T} regret is achieved.