Title: Private Online Learning against an Adaptive Adversary: Realizable and Agnostic Settings
Venue: NeurIPS 2025
Accepted: yes
Keywords: differential privacy

online learning
Abstract: We revisit the problem of private online learning, in which a learner receives a sequence of $T$ data points and has to respond at each time-step a hypothesis. It is required that the entire stream of output hypotheses should satisfy differential privacy. Prior work of Golowich and Livni [2021] established that every concept class $\mathcal{H}$ with finite Littlestone dimension $d$ is privately online learnable in the realizable setting. In particular, they proposed an algorithm that achieves an $O_{d}(\log T)$ mistake bound against an oblivious adversary. However, their approach yields a suboptimal $\tilde{O}\_{d}(\sqrt{T})$ bound against an adaptive adversary. In this work, we present a new algorithm with a mistake bound of $O_{d}(\log T)$ against an adaptive adversary, closing this gap. We further investigate the problem in the agnostic setting, which is more general than the realizable setting as it does not impose any assumptions on the data. We give an algorithm that obtains a sublinear regret of $\tilde{O}_d(\sqrt{T})$ for generic Littlestone classes, demonstrating that they are also privately online learnable in the agnostic setting.


Peer reviews:
comments: SUMMARY:
The authors study the problem of online learning with differential privacy. They make the following contributions:
1. In the realizable setting, they propose a differentially private algorithm that achieves a mistake bound of $O_d(\log T)$ for an adaptive adversary, where $d$ is the Littlestone dimension of the hypothesis class and $T$ is the time horizon. This improves upon the previously known bound of $O_d(\sqrt{T})$.
2. In the agnostic setting, they propose a differentially private algorithm that achieves a regret bound of $\tilde{O}_d(\sqrt{T})$. However, the proposed algorithm is improper. They also propose a proper learner with a regret bound of $\tilde{O}_d(T^{3/4})$.

STRENGTHS AND WEAKNESSES:
The paper is well written, with ideas presented in a clear and concise manner. The authors address the important problem of ensuring differential privacy in online learning, considering both the realizable and agnostic settings. While the work builds on prior research—such as Golowich and Livni [2021] and Bun et al. [2020]—and employs established tools like SOA, AboveThreshold, and Sanitization, it introduces several clever ideas and novel arguments (e.g., Lazy Update and uniform convergence) to improve upon the existing results.

QUESTIONS:
In many online learning scenarios, the time horizon $T$ is not known in advance. I noticed that several parameters—such as the threshold in Algorithm 2 and the number of experts in Section 4.2—depend explicitly on $T$. Is it possible to remove this dependency?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 4
IMPACT: 4
ORIGINALITY: 3
DATE: 2025-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn online learning, a learner receives a sequence of $T$ data points from an adversary and has to respond at each time step with an hypothesis. A concept class $\\mathcal{H}$ is said to be privately online learnable if there is an online learner achieving sublinear regret, that is also differentially-private w.r.t. all the stream of returned hypotheses. Apart from the classical differences, agnostic VS realizable setting and proper VS improper learner, one can differentiate between an oblivious adversary, that chooses the entire sequence of data points at the beginning, and adaptive one that can determine the sequence online, after observing the learner’s outputs. The authors show (1) an algorithm improving the mistake bound presented by Golowich and Livni [2021] against an adaptive adversary (closing the gap between the oblivious/adaptive cases) in the realizable setting and (2) an algorithm achieving sublinear regret for classes with finite Littlestone dimension in the agnostic setting with an adaptive adversary.\n\nSTRENGTHS AND WEAKNESSES:\n**Positive points**:\nThe paper is generally well-written and easy to follow, even if quite dense. Studying differential privacy in online learning is an interesting topic and not generally an easy one. In Section 3, the realizable setting is analyzed: the authors first give an in-depth explanation of the reasons why the procedure by Golowich and Livni [2021] does not attain a regret that is logarithmic in $T$ and then propose an algorithm that efficiently addresses those limitations. I think that the analysis is interesting and sound and the algorithm makes use of known procedures as subroutines. In Section 4, the authors deal with the agnostic setting: in particular they show a simpler algorithm employing sanitization and a more complex one employing experts and sanitization and achieving a better regret bound with an improper learner.  It is interesting to see many techniques blended together to obtain those results. \n\n**Negative points**:\nAs mentioned, the paper is dense. Without a knowledge of many results in the literature, it becomes a bit hard to follow some passages, even if the authors do their best to sum everything up. Also, I noticed that the main result cited at the beginning is not referenced again in Section 3 and that can be confusing.\n\n**Minor comments**:\nTypo in algorithm 2: failuare -> failure. Typo in Theorem 2.6 Pirvate -> Private.\n\nAll the above comments motivate my score.\n\nQUESTIONS:\n1) Can you justify the necessity of including the condition over the dual Littlestone dimension in Theorem 4.2/ Corollary 4.3? This condition (if I am not mistaken) is required also for the results in \"Sample-efficient proper PAC learning with approximate differential privacy\" and I would like to have an intuitive explanation.\n2) Do you believe it is possible to eliminate the $2^{2^{O(d)}}$  factor in the Littlestone dimension? Just, your opinion about it.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses private online learning in the adaptive setting, where an adversary can choose inputs based on the learner's previous responses. The authors achieve two main results that improve upon existing mistake bounds.\n1) Realizable Setting: The authors provide an algorithm with an O_d(log(T)) mistake bound for learning any concept class with Littlestone dimension d, which closes the gap to the known lower bound. Their approach builds on Golowich and Livni (2021) but introduces a key innovation: instead of releasing updates at every timestep (which accumulates privacy costs), they use \"lazy updates\" that defer computation until necessary, followed by random permutation to maintain the privacy analysis structure.\n2) Agnostic Setting: The authors develop a new algorithm achieving an O_d(√(T)) mistake bound. Their approach first constructs a private proper learner, then applies sanitization techniques for generic Littlestone classes from Ghazi et al. (2021) to improve the regret bound, though this results in an improper learner.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \nThe results are foundational in our understanding of private online learning in the adaptive setting and improve upon previous works in this area by giving better upper bounds in the realizable setting and new results in the agnostic setting. \n\nWeaknesses: \nThe explanations in Sections 3 and 4 lack sufficient clarity and detail, which diminishes the correctness of the proposed techniques. The authors would benefit from reallocating space from the extensive Preliminaries section to provide a more detailed explanation of Algorithm 2 and its different components.\nWhile I understand the high-level approach of addressing the √(T) composition cost at each timestep through lazy updates, several technical concerns remain unaddressed. In the adaptive setting, don’t you have to re-initialize the SOA algorithm after every lazy update? Otherwise, the shared randomness across lazy updates could introduce correlations between historical predictions and current outputs, potentially creating exploitable patterns for adversaries.\nThis concern suggests that the privacy analysis extends beyond straightforward composition arguments and involves more nuanced considerations. However, Section 3 does not appear to address these subtleties, leaving important questions about their solution unresolved.\n\nQUESTIONS:\nCan you explain why in Algorithm 2, you don’t need to reinitialize SOA after a lazy update?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 1, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis submission studies differentially private online learning of Littlestone classes against an adaptive adversary. It builds on work of Golowich and Livni, who previously showed that classes with finite Littlestone dimension can be learned with mistake bound $O(\\log T)$ against an oblivious adversary, and with mistake bound $O(\\sqrt{T})$ against an adaptive adversary. This paper improves these results in two directions. First, it shows that in the realizable setting considered by Golowich and Livni, Littlestone classes can be learned with mistake bound $O(\\log T)$ against an adaptive adversary. Second, in the agnostic setting with an adaptive adversary, such classes can be learned with regret $O(\\sqrt{T})$.\n\nThe paper's realizable learner builds on the strategy of Golowich and Livni, but introduces two main refinements to address how their probabilistic analysis breaks in the presence of an adaptive adversary. The first idea is to apply lazy updates to ensure independence between examples revealed and the randomness used in their permutation argument. The second is to use uniform convergence to ensure accurate estimation for all functions in the target class, rather than just the ground truth labeling function.\n\nIn the agnostic setting, it provides two different algorithms that reduce private online learning to sanitization (a.k.a. private synthetic data generation). The first is a simple proper learner (albeit with suboptimal regret) that breaks the time horizon into batches, applies a sanitizer to each batch to produce synthetic examples, and runs a non-private online learner on the synthetic data. The second is an improper learner with improved regret that simulates the experts algorithm, evaluating experts using sanitization. These online learners can then be instantiated using the known private sanitizer of Ghazi et al. for Littlestone classes.\n\nSTRENGTHS AND WEAKNESSES:\n(+) Resolves the main open questions left by Golowich and Livni. Note that there was no reason to believe a priori that the best achievable mistake bound/regret of DP online learning against an adaptive adversary should coincide with that of an oblivious adversary; indeed, there are related settings in which there is a separation.\n\n(+) The algorithms and analyses in Section 4 introduce some nice conceptual ideas that could be useful more generally.\n\n(-) Handling the realizable case requires overcoming some technical challenges from prior work, but is otherwise relatively incremental. This isn't necessarily a strength or a weakness, but the paper's technical contributions don't have that much to do with understanding the combinatorial structure of Littlestone classes, and more to do with developing and analyzing online learners.\n\nQUESTIONS:\n* How close to proper can the algorithm handling the realizable case be made? E.g., by replacing SOA hypotheses with Hanneke-Livni-Moran's construction?\n\n* I understand you're space-constrained, but seeing as Section 4.2 reflects most of the paper's algorithmic novelty, I would have liked to have seen a more detailed description of that algorithm in the main body of the paper in place of the algorithm described in Section 3.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work studies mistake/regret bounds for private online learning against adaptive adversaries. They improve over existing $O(\\sqrt{T})$ mistake bounds for private realizable online learning of finite Littlestone classes against adaptive adversaries and obtain $\\tilde{O}(\\sqrt{T})$ regret bounds for private agnostic online learning of finite Littlestone classes against adaptive adversaries.\n\nSTRENGTHS AND WEAKNESSES:\nThis paper makes a significant contribution to understanding mistake/regret bounds for private online learning against adaptive adversaries. While the main algorithmic idea is structurally similar to that of Golowich and Livni, there are notable technical obstacles to improving their mistake bounds against adaptive adversaries, which this work overcomes. \n\nI unfortunately found this paper hard to read for a few reasons. To the author’s credit, they give a summary of of the online learning of Golowich and Livni and explain why it is insufficient for achieving the desired $O(\\log T)$ mistake bounds, but I don’t know how helpful this summary would be for those who are not already familiar with the algorithm. More importantly, the reader has to do some digging to find a formal theorem statement for the algorithm in Section 3, and then more digging to find the setting of parameter $N_0$, which I think is helpful for the reader to know while reading the pseudocode for Algorithm 2. It would be good to have a theorem statement in Section 3 with the parameters necessary for privacy and the stated mistake bounds specified. \n\nSome minor notes/typos: \n\nIn abstract - “each time step a hypothesis”\n\nTheorem 2.6 - “pirvate histogram”\n\n“incurring a mistake bound scales”\n\nAlgorithm 2 - “failuare” \n\nI think there’s a duplication issue with the citation for GGKM “Sample Efficient Proper PAC Learning with Approximate Differential Privacy”\n\nQUESTIONS:\nSee notes above.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-10"}