Title: Sequentially Auditing Differential Privacy
Venue: NeurIPS 2025
Accepted: yes
Keywords: Differential privacy

testing by betting
Abstract: We propose a practical sequential test for auditing differential privacy guarantees of black-box mechanisms. The test processes streams of mechanisms' outputs providing anytime-valid inference while controlling Type I error, overcoming the fixed sample size limitation of previous batch auditing methods. Experiments show this test detects violations with sample sizes that are orders of magnitude smaller than existing methods, reducing this number from 50K to a few hundred examples, across diverse realistic mechanisms. Notably, it identifies DP-SGD privacy violations in \textit{under} one training run, unlike prior methods needing full model training.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nRecently there has been a great deal of interest in \"auditing\" differential privacy guarantees of algorithms such as DP-SGD among others. The popular approach to auditing DP usually looks at sample size as static -- it runs the DP mechanism as a black-box n times with different randomness, and then provides an answer about its privacy properties. Usually, it provably accepts every DP mechanism, and empirically rejects non-private mechanisms with a certain amount of probability. \n\nThis work proposes a new framework and mechanism to audit DP mechanisms in a different, sequential way. If I understand correctly, here the number t of runs of the black-box is varying (and growing), and the algorithm either decides to terminate with a decision, or to keep asking for more samples. These tests are enabled through drawing a novel connection between differential privacy, and recent advances in sequential hypothesis testing. There is also a nice theorem on the relationship between the Hockey-stick divergence and kernel-MMD tests that is of independent interest.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n+ This is a very strong and novel paper. The result is strong and elegant and arises due to a novel connections between two disparate areas. I also really liked the last part about auditing DPSGD in less than one run, which is a novel advance.\n+ The paper is quite well-written, and even though the math is fairly dense it feels easy to read. Related works are well-cited and credit properly assigned. \n\nWeaknesses:\n\n- One weakness of the presentation is that this paper audits DP in a framework that is somewhat different from the standard auditing framework. It took me a while to understand this -- and I think the presentation could be even better if this were directly explained earlier on in the paper. \n- A minor point -- the first DPSGD paper was actually Song, S., Chaudhuri, K., & Sarwate, A. D. (2013, December). Stochastic gradient descent with differentially private updates. In 2013 IEEE global conference on signal and information processing (pp. 245-248). IEEE.\n\nQUESTIONS:\nNone.\n\nLIMITATIONS:\nn/a", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nGiven two neighbouring datasets $S$ and $S’$, a mechanism $\\mathcal{A}$, and i.i.d samples $X_1, X_2, \\dots \\sim \\mathcal{A}(S)$ and $Y_1, Y_2 \\dots \\sim \\mathcal{A}(S’)$, this paper studies sequential hypothesis tests for distinguishing between:\n- $H_0$: the Hockey-stick divergence of order $e^\\epsilon$ between $\\mathcal{A}(S)$ and $\\mathcal{A}(S’)$ is smaller than $\\delta$ \n\nV.S. \n\n- $H_1$: the Hockey-stick divergence is $> \\delta$. \n\nFirst, to solve this, the paper uses the Maximum Mean Discrepancy (MMD) as a proxy for the Hockey-stick divergence. While this proxy was explored in prior work (e.g., [23]), the authors tighten the connection in Theorem 3.1. Then, the paper adapts the “two-sided” sequential MMD Test from [40] into a “one-sided” test against a specific non-zero threshold.  A practical sequential test is developed (Algorithm 1) and analysed with guarantees on Type I error, expected stopping time, and exponential growth under the alternative in Theorem 3.3.  Finally, the test is implemented to audit first some classic mean estimation DP mechanisms, then to audit DP-SGD in the white-box setting.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n- The paper is well-written and the problem is well-motivated\n- The idea of applying sequential hypothesis testing techniques for DP auditing is novel and interesting\n- The paper provides a practical test, with both non-trivial theoretical guarantees and good experimental results\n\nWeakness:\n- As stated multiple times in the paper, the main limitation of the test proposed is the assumption that the two neighbouring datasets $S$ and $S’$ are given as input. In most practical DP auditing settings, generating such pairs, especially worst-case ones, is the most challenging part, and this part is not addressed in the paper.\n\n- The terms \"black-box\" and \"white-box\" are used inconsistently or imprecisely. For instance, in the first part, \"black-box\" meant that the auditor does not know the inner functioning of the mechanism $\\mathcal{A}$, while in the DP-SGD auditing, “white-box” means that the auditor has access to the intermediate gradients, but still does not know the inner functioning of the mechanism $\\mathcal{A}$. A clearer definition of these terms early in the paper would help.\n\n\n- While the adaptation of the test from [40] is useful, the technical novelty compared to [40] is still unclear/incremental. The paper could do more to explicitly highlight what is technically new beyond switching from two-sided to one-sided.\n\nQUESTIONS:\n- What exactly are the “non-trivial adaptations and subtle but key modifications of the original results\npresented in [ 40 ]” (Line 140)? Could you explicitly enumerate or summarise what changes were needed to adapt the two-sided test to your one-sided setting?\n\n\n- Improving further Theorem 3.1 via TV: In the proof of Theorem 3.1, it seems that you first upper bound the MMD by a Total Variation (TV). But it seems that Remark A.1.b) in the paper by Kairouz et al. (“The Composition Theorem for Differential Privacy”) provides tighter bounds on TV in terms of DP parameters. Can't you directly plug in those to yield an even tighter bound than your Theorem 3.1?\n\n- Isn’t auditing DP-SGD in the white-box setting equivalent to auditing the mean mechanism applied to gradients, and thus section 4.2 is the same as section 4.1? If not, what are the differences?\n\n- Is it possible to use your sequential test for auditing DP-SGD in the black-box model, i.e. you only have query access to the learned model and not access to gradients anymore?\n\nLIMITATIONS:\nThere is no specific \"limiations\" sections in the paper, but the authors discuss them throughout the paper", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a Maximum Mean Discrepancy (MMD)-based one-sided sequential test for auditing differential privacy. It enables anytime-valid inference with Type I error control, overcoming the inefficiency of prior batch tests. The authors also derive a new bound linking MMD to the Hockey-Stick divergence, which improves test power. Experiments show the method detects DP violations with far fewer samples—e.g., within a single DP-SGD training run, whereas the previous method requires a complete training round—thus making it practical and efficient for real-world use.\n\nSTRENGTHS AND WEAKNESSES:\n**Strength:**  \nThe paper presents a novel MMD-based one-sided sequential testing framework for auditing differential privacy, introducing testing-by-betting techniques that are new to this area. A notable strength is the ability to detect violations without requiring a full training run.\n\n\n**Weakness:**  \n* The method assumes access to a fixed worst-case pair of neighboring datasets $(S, S')$. Identifying such pairs can be difficult in practice, limiting the method's applicability in fully black-box settings.\n* The method relies on estimating the function $f_t$ via online optimization (e.g., Online Newton Step), which introduces additional computational overhead.\n* It would also be helpful to include experiments comparing the accuracy of the proposed method with that of existing one-run auditing methods, to contextualize potential computation-accuracy tradeoffs.\n\n\n\n\n\n\n\n\n\n\n* It would also be helpful to include experiment comparising the accuracy against one-run auditing methods\n\nQUESTIONS:\nIt would be helpful if the authors could elaborate on how Algorithm 1 is applied to audit DP-SGD in practice. Since DP-SGD is an iterative algorithm where each update depends on previous model parameters, the gradients at different iterations are not i.i.d. samples from a fixed distribution pair. This seems to violate the assumption in Line 5 of Algorithm 1, where samples $ X_t \\sim A(S) $ and $ Y_t \\sim A(S')$ are assumed to be drawn independently from two fixed distributions.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes new methods for auditing differentially private\nalgorithms. Specifically, the paper focuses on approximate\ndifferential privacy and proposes to use a methodology inspired by\nrecent work on sequential hypothesis testing based on the theory of\ne-values, which are roughly nonnegative random variables\ncharacterizing the evidence for rejecting a null hypothesis, which\ncompose well adaptively. The proposed tests are based on maximum mean\ndiscrepancy and uses Online Newton Step and Online Gradient Ascent for\nlearning the needed parameters. The paper presents an experimental\nevaluation on multiple mechanisms and benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n- The paper uses a recently developed approach to sequential hypothesis\n  testing to address the problem of auditing differentially private\n  mechanisms\n\n- the tests presented in the paper improves considerably in terms of\n  performance with respect to previous approaches\n\nWeaknesses\n\n- The experimental evaluation does not provide any direct comparison with\n  previous methods,\n\n- similar to some previous work, the paper assumes that worst-case\n  adjacent databases are known.\n\nSeveral work have studied auditing methods for differentially private\nmechanism, especially in the context of differentially private\noptimization and learning. This paper brings new methods to this area\nby introducing an approach based on sequential hypothesis testing\nusing e-values. The paper combines this ideas in a test using off the\nshelf learning algorithms to learn some of the parameters needed for\ndeciding the test. The proposed tests are more adaptive than the ones\nproposed in previous work and are able to converge faster. This makes\nthem more efficient for auditing purposes than previous\napproaches. The efficiency is evaluated experimentally in the paper\nwith encouraging results. Although, to be fair, there is not concrete\nexperimental comparison of the proposed method with previous works.\nOne limitation, acknowledged by the paper, is that the paper assumes\nthe knowledge of worst-case adjacent databases. This makes the\nproposed approach more aligned with tools for auditing privacy\nparameters rather than with tools for finding privacy violations.\n\nQUESTIONS:\nYour test uses OGA and ONS as black boxes. Does it means that you could obtain better tests by using similar algorithms with improved guarantees?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-23"}