Title: Differential Privacy on Fully Dynamic Streams
Venue: NeurIPS 2025
Accepted: yes
Keywords: Differential Privacy

Linear Queries

Fully Dynamic Streams
Abstract: A fundamental problem in differential privacy is to release privatized answers to a class of linear queries with small error. This problem has been well studied in the static case. In this paper, we consider the fully dynamic setting where items may be inserted into or deleted from the dataset over time, and we need to continually release query answers at every time instance. We present efficient black-box constructions of such dynamic differentially private mechanisms from static ones with only a polylogarithmic degradation in the utility.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper is about answering linear queries on a fully dynamic stream while preserving differential privacy. Previous work has only worked on insertion only streams where an element is never deleted. Moreover, the insertion only algorithms can be extended to fully dynamic but with a rather high error: their error is a function of $|D_t|$, the size of the dataset at time t. In a fully dynamic stream, if the number of elements ever inserted until time t is $N_t$, naively using insertion only algorithms gives an error that is a function of $N_t$ and not $|D_t|$, while $|D_t|$ can be much smaller than $N_t$.\n\nThis work shows how to use an insertion only algorithm in a blackbox way and get same errors in terms of the size of the dataset at time t. Their approach consists of many interesting components. First they introduce a dynamic interval tree that can cope with deletions and any query can be answered by only inspecting log t nodes. If $D^+_t$ is the set of nodes ever inserted up to time t and $D^-_t$ is the set of nodes deleted on or before t, they compute a linear query by computing it on $D^+_t$ minus its value on $D^-_t$. Their second component is \"restarts\", where they compute a query from scratch on $D_t$ using a static algorithm instead of computing it on  $D^+_t$ and  $D^-_t$. They do these restarts when almost half of the elements in a tree node are deleted, and this way they can control the error to be in a function of $|D_t|$. Their third component is a clever distribution of privacy budget which is not novel on its own but is nicely used.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The paper is very well written. They provide really good examples to explain their ideas, and as explained above I like the components of their algorithm. their contribution is strong.\n\nWeaknesses: \n- minor: the proof of Lemma 7 refers to Algorithm 1 which is stated after this lemma in the appendix. It would have be good to provide a summary of this algorithm or explain the parameters before referring to them.\n\nQUESTIONS:\nOn line 341 you allocate privacy budget of $1/\\ell^{1+\\eta}$ to nodes at level $\\ell$. However as time goes on the level of nodes can increase. Could you explain why this is fine?\n\nLIMITATIONS:\nI can't think of any limitations other than the ones stated.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-20"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies privately answering linear queries that are union-preserving (i.e., $f(D_1\\cup D_2)=f(D_1)+f(D_2)$) in the turnstile streaming model. Prior works mainly focus on insertion-only streams, as most of the DP tools can be naturally attuned in this model. Note that simply treats a turnstile stream as two insertion-only streams is not enough, as popular private mechanisms have square root dependencies on the size of the set, doing so would lead to undesirable bound on the error. Thus, to obtain reasonable error bound on turnstile streams, the authors use a novel online interval tree to keep track of the lifespan of elements. This is significantly different from classical interval trees, where all intervals are known in advance. Authors show that it is possible to maintain the intervals in an online fashion while not blowing up the complexity of the data structure significantly. Using this data structure, authors show that the problem is then reduced to a deletion-only problem, but running an insertion-only algorithm is not enough, so the proposed algorithm restarts the data structure when half of the elements are deleted. With some extra caveats being taken care of, the authors arrive at a private algorithm for linear queries over turnstile streams.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The result obtained in this paper is strong, it achieves nearly-optimal dependence (up to polylog factors) on the size of the stream, which is nontrivial. Although the core mechanism (private multiplicative weights and insertion-only algorithm) has been developed, designing the correct data structure to utilize these tools is an important contribution.\n\n2. The online interval tree is a novel contribution that might find applications other than DP in the turnstile streaming setting. In fact, adapting the online interval tree for turnstile streams could be abstracted as a framework for tasks that have good insertion-only and static algorithms that depend on the size of the stream.\n\n3. The paper overall is well-written, the algorithm is explained clearly and the online interval tree data structure, facilitated with several figures, is easy to understand. \n\nWeaknesses:\n\n1. The mechanism is developed for the general setting, it would be interesting to see more white-box application of the proposed algorithm for particular problems, such as half-space queries, with even more improved error bounds.\n\n2. The paper is technically dense, thus the discussions on future directions are limited.\n\nQUESTIONS:\n1. What do you think are some other applications of the online interval tree when used for turnstile streams?\n\n2. For non-union-preserving linear queries, is it possible to extend the developed techniques to handle them? If not, what are the major barriers?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper tackles continual release of differentially-private answers for fully dynamic (turnstile) data streams, where elements may be both inserted and deleted. While previous work handled insertion-only streams or the static setting, the authors design a black-box transformation that turns any static DP mechanism for a union-preserving family of queries (e.g. linear queries) into a mechanism that operates on fully dynamic streams with only a polylogarithmic loss in accuracy and running-time.\n\nThe core technical contribution is an online interval tree that stores each item in at most one node per tree level, permitting a decomposition of the data set at time $t$ into $O(\\log t)$ disjoint parts that can each be handled by an insertion-only sub-mechanism . \n\nA careful choice of parameters bounds the cumulative privacy loss while keeping the error blow-up within polylogarithmic factors.\nIn particular, plugging private multiplicative weights as the static algorithm, they achieve $\\tilde O(\\sqrt{|D_t|})$ error for approximate DP and $\\tilde O(|D_t|^{3/4})$ for pure-DP. Here $|D_t|$ is the size of the current dataset $D_t$. Notably, there is no dependence on the number of queries for linear queries, whereas straightforward extensions of prior insertion-only works would result in polynomial dependence on the number of queries.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n1. To the best of my knowledge, this is the first black-box method that matches static-query accuracy up to polylog factors for fully dynamic streams, closing the gap with insertion-only results.\n1. The online interval tree data structure seems to be novel and may be of independent interest even beyond privacy.\n\n---\n\n### Weaknesses\n1. The results only hold for union-preserving query families. While this includes the class of linear queries, it limits applicability.\n\nQUESTIONS:\n1. Are there any interesting classes of union-preserving query families beyond linear queries?\n\n---\n\n### Suggestions\n1. I would suggest (informally) stating the main results within the introduction of the paper and quantifying the improvements from prior works.\n2. There are some references for continual release which may be relevant.\n\n---\n\n- Song, Little, Mehta, Vinterbo, Chaudhuri. \"Differentially private continual release of graph statistics.\" arXiv 2018.\n- Fichtenberger, Henzinger, Ost. \"Differentially Private Algorithms for Graphs Under Continual Observation.\" ESA 2021.\n- Epasto, Liu, Mukherjee, Zhou. \"Sublinear Space Graph Algorithms in the Continual Release Model.\" arXiv 2024.\n- Jain, Smith, Wagaman. \"Time-aware projections: Truly node-private graph statistics under continual observation.\" SP 2024.\n- Raskhodnikova, Steiner. \"Fully Dynamic Algorithms for Graph Databases with Edge Differential Privacy.\" PACMMoD 2025\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nWhile differential privacy (DP) oftentimes imposes a challenging trade-off between utility and privacy in the static setting, dynamic algorithms add another factor tearing this trade off apart, which is time (or number of updates). The authors of the submission present a construction for a dynamic, $(\\epsilon, \\delta)$-DP algorithm for any union-preserving query (e.g., linear queries), given a static, $(\\epsilon', \\delta')$-DP algorithm. A query $f$ is union preserving on multisets $D_1, D_2$ if $f(D_1 \\cup D_2) = f(D_1) + f(D_2)$. For linear queries, using the DP multiplicative weights mechanism, their fully dynamic algorithm achieves an error of $\\tilde{O}(n_t^{2/3})$ for pure DP ($\\delta=0$), and $\\tilde{O}(\\sqrt{n_t})$ for approximate DP with a multiplicative $O(\\log^2 t)$ blow-up in time, where $n_t$ is the size of the dynamic dataset at time $t$. This improves over existing constructions with similar error bounds but which are only partially dynamic (insertions or deletions), and fully dynamic constructions with error bounds that that are linear in $t$. For union-preserving queries, the bound is a bit more complex: Given a static DP mechanism with an of error $\\alpha^{(k)}$ when transformed into an insert-only mechanism via appropriate standard techniques, roughly speaking, the resulting dynamic error bound is $\\alpha^{(\\log^2 t)}$, and dependencies on $\\epsilon, \\delta$, size of the dataset are substituted by themselves multiplied by a $O(\\log^3 t)$ or $O(1/\\log^3 t)$ factor. The running time is claimed as the running time of the static black-box algorithm on $n_t$ times $O(\\log^2 t)$.\n\nSTRENGTHS AND WEAKNESSES:\nThe algorithm is based on the established, high-level concept of maintaining the dynamic updates and privacy budget of a DP datastructure in a tree of logarithmic depth, but generalizes it by keeping track of insertion/deletion pairs represented as intervals. The authors propose an online version of an interval tree, where the index of a node in an in-order traversal is $t$. An insertion at time $t$ is tracked on the path from the root to the node $v_t$, and it is removed from it at the time of deletion. At each node, the algorithm uses three black-box mechanisms: a static DP mechanism for $f$ to provide an answer for the dataset at time $t' \\\\leq t$, an insertion-only DP mechanism to track deletions since $t'$, and a insertion-only DP mechanism to count deletions. The latter mechanism is required because when a node's privacy budget is exceeded, it needs to restart its mechanism (and sets $t' = t$).\n\nThe main technical contribution and novelty of this paper is a black-box construction for dynamic data streams and union-preserving queries. Except for a few special cases like counting (where $f \\\\equiv 1$), it reduces the best known multiplicative error bound for linear queries in the number of updates $t$ from linear to $log^2$. Arguably, counting is an important special case, and the improvement for this case is $\\\\log^2.5$ to $\\\\log^2$. The paper is well written except for a few minor improvements to notations that could be made, and appears to be sound on the level of arguments (without checking numbers).\n\nQUESTIONS:\nThe running time as stated is a bit confusing to me. It seems to me that it should be the time to run the static mechanism for every time step, plus the time for running the incremental mechanisms for the respective restart intervals. However, $time(|D_t|)$ is the time complexity to process $|D_t|$ incremental updates to me. Would'n you need to sum up the time complexities for each restart interval / lifetime of an incremental mechanism separately instead of bounding them by $|D_t|$, which may be even 0?\n\nThe notation $\\\\alpha^{(k)}$, in particular the definition of $(k)$, is a bit informal. The extensive list of arguments is very helpful, but the meaning of the stream-related exponent $k$ for a static algorithm is a bit ambiguous or vague.\n\nThe definition of union-preserving might deserve to be more prominent than a footnote.\n\nLIMITATIONS:\nYes, all limitations addressed.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a new black-box mechanism that integrates any given algorithm for the private linear query problem with the binary tree mechanism to construct an algorithm for private query release under continual observation. This method achieves lower additive error for the streaming turnstile model than that indicated by prior work. For context, the private linear query problem simply asks the data curator to return the result of an arbitrary set of linear (counting) queries under the constraints of $\\varepsilon$-differential privacy - prior work shows that this is possible with additive error $\\sqrt{n}/\\varepsilon$, where $n$ is the size of the private data set. The continual observation setting requires the data curator to accept modifications to the data set (insertions, deletions, or no change) in a stream, and respond to the set of queries after every update. A general-purpose technique in the privacy literature to deal with queries with an additive structure (union-preserving, as mentioned by the authors) is to use the binary mechanism of Chan et. al., that essentially allows one to use batch algorithms for the query release problem and suffer additive error that is inflated by a factor that is only polylogarithmic in the length of the stream, despite making a release after each new stream element (naive privacy accounting would lead to a $\\sqrt{T}$ additive error when the stream is of length $T$ instead which is much worse).\n\nIt is observed by the authors that for the turnstile setting, this general purpose method suffers the drawback that the net error is not $\\sqrt{n}\\log^c T$ for some $c$ but instead on the order of $(\\sqrt{n_+} + \\sqrt{n_{-}}) \\log^c T$, where $n_+$ s the number of insertions and $n_-$ is the number of deletions. For long streams, clearly this is much (potentially arbitrarily) worse than the former. The main contribution of this paper is a new algorithm that achieves additive error with only a $\\sqrt{n}$ term.\n\nMethods: The algorithm suggested is essentially a modification of the binary tree mechanism (maintaining a copy of the batch algorithm at each internal node of a binary tree over the stream), where one backtracks to tag nodes with 'liveness' interval endpoints when deletions are made. Just as in the binary tree mechanism, multiple copies of an element are maintained at each level, so are they here, but instead of each element only being added to the nodes corresponding to the dyadic intervals in which it lies, it is added to all nodes whose interval happens to lie within the liveness interval of that element. This leads to the possibility of double counting when a query is made, to deal with this one considers only the responses from the batch algorithm at `left' nodes.\n\nTo achieve the main improvement, the algorithm refreshes the privacy budget of each internal node when half the elements it carried previously are deleted, which ensures that the additive error remains on the order of $\\sqrt{n}$. The restart points themselves can be privatized with the Sparse Vector Technique (SVT). Refreshing the budget in this manner is not too wasteful because this can happen only logarithmically many times for any node over the course of the stream.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The problem is very general, of interest to the privacy community and impactful.\n2. The algorithm is simple and straightforward to understand, intuition is given well.\n\nWeaknesses:\n1. I think the proofs need to be written more rigorously, possibly in the appendix - at a high-level this approach makes sense, but for instance not having a formal definition of a left node and a right node made the reading a bit harder.\n2. Although intuition is given well, the writing could be polished a lot more - this is a very minor and subjective issue but subheadings like 'Putting it Together' are maybe a bit informal and not very helpful to a reader trying to find the right subsection quickly, for instance.\n\nQUESTIONS:\n1. What happens if an element is deleted more often than it is inserted? It looks like it doesn't make much of a difference since the algorithm would just check the internal nodes and find nothing to add a deletion tag to, but this might be good to discuss explicitly.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-30"}