SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We obtain our stronger item-level lower bounds (for pure and approximate differential privacy) by embedding multiple instances of a 1-way marginal query. We then apply lower bounds of Hardt and Talwar [38] and Bun et al. [12] for releasing all 1-way marginals in the batch model in conjunction with our reduction. The 1-way marginals of a dataset  $y \in \{0,1\}^{n \times d}$ , consisting of  $n$  records and  $d$  attributes, are the averages of all  $d$  attributes of  $y$ . Deletions in the stream are once again crucially used to embed a marginal query for one attribute and then clean the slate for the next attribute. Changing one record/row in the dataset  $y$  translates to  $d$  changes of an item in the constructed stream, and thus this reduction is particularly tailored to item-level lower bounds.
The study of differential privacy under continual release was initiated by two concurrent works [25, 13]. They proposed the binary-tree mechanism for computing sums of binary bits. The binary-tree mechanism has found numerous applications in the continual release setting and elsewhere, demonstrating the versatility of this mechanism. Under continual release, it has been extended to work for sums of real values [52], weighted sums [9], graph statistics [55, 31], and most relevantly, counting distinct elements [9, 29, 35]. It has also been employed for private online learning [45, 57, 1] and for answering range queries [25, 26, 28].
Prior to our work, the CountDistinct problem with continual release was studied exclusively in the insertions-only model. Bolot et al. [9] were the first to study this problem and showed a  $O(\log^{1.5}T)$ -accurate item-level-DP mechanism. Ghazi et al. [35] considered the more challenging sliding-window model and showed nearly-matching upper and lower bounds for this setting, parameterized by the window size, for item-level and event-level differential privacy. Epatso et al. [29] studied the more general  $\ell_p$ -frequency estimation problem with a focus on space efficiency. For distinct elements, i.e.,  $p = 0$ , their mechanism provides an estimate with  $1 + \eta$  multiplicative error and  $O(\log^2 T)$  additive error, using space poly  $(\log T / \eta)$ . They also extended their results to the sliding-window model. Two of the works [9, 29] reduced the CountDistinct problem to the bit summation primitive, which allowed them to use the binary-tree mechanism. Since the streams are restricted to be insertion-only, the bit summation primitives they considered have low constant sensitivity. The same primitives have sensitivity  $\Omega (T)$  for turnstile streams, and thus this approach cannot be directly extended to our setting. Ghazi et al. [35] observed that for fixed and sliding windows, the distinct elements problem can be reduced to range queries. For the special case when the window is the entire stream, their reduction is to the summation problem.
In concurrent work, Henzinger et al. [40] studied CountDistinct with insertions and deletions in a different version of the continual release model (which we call the 'likes' model), where an item can be deleted at a time step only if it is already present in the stream at that time step, and inserted only if it is absent from the stream at that time step. Our model is more general, since multiple insertions and deletions of the same item can happen consecutively. Our upper bound and our item-level privacy lower bound can be extended to the 'likes' model. On the other hand, our event-level private lower bound provably does not apply to that model: in the 'likes' model, for event-level privacy, there is a simple reduction to the bit summation problem in the continual release model such that the resulting algorithm incurs only a polylogarithmic in  $T$  error, whereas we show that in our model, any event-level private algorithm incurs a polynomial in  $T$  error.
Henzinger et al. [40] showed error bounds for item-level privacy in the likes model that are parameterized by the total number of updates  $K$  in the stream. The parameter  $K$  is related to our concept of flippancy: in the likes model,  $K$  equals the sum of all items' flippancies and, in general, is at least that sum. Henzinger et al. [40] give an  $(\varepsilon, 0)$ -DP algorithm with error  $\tilde{O}(\sqrt{K} \log T)$  and show a nearly matching lower bound on the error for  $(\varepsilon, 0)$ -DP algorithms using a packing argument. This lower bound applies to our model as well. It is incomparable to our lower bounds, since it scales differently and depends on a different parameter. In our model, their algorithm can be analyzed to give
error bounds in terms of the sum  $K'$  of the flippancies of the items and incurs error  $\tilde{O}(\sqrt{K'}\log T)$ ; however, it is unclear if their algorithm can be analyzed in our model to give bounds in terms of the (potentially smaller) maximum flippancy.
Another line of work investigated private sketches for distinct elements, motivated by the popularity of sketching algorithms for the streaming setting. Mergeable sketches for counting distinct elements have received particular attention [56, 15, 51, 39], since they allow multiple parties to estimate the joint count of distinct elements by merging their private sketches. While these sketches can be combined with the binary-tree mechanism to obtain private mechanisms for CountDistinct, the utility deteriorates when many  $(\log T)$  sketches are merged. In fact, Desfontaines et al. [19] showed that achieving both privacy and high accuracy is impossible when many sketches for counting distinct elements are merged. Other private sketches have been studied [54, 20, 58] for the streaming batch setting (without continual release). The distinct elements problem has also been studied in a distributed setting [14, 34] and under pan-privacy [50]. In particular, our lower bound for event-level privacy uses ideas from the lower bound of Mir et al. [50], as described in Section 1.2
The CountDistinct problem has been extensively studied in the non-private streaming setting, where the goal is to achieve low space complexity [32, 3, 16, 36, 37, 6, 5, 22, 43, 59, 30, 7, 33, 10, 46]. Blocki et al. [8] showed a black-box transformation for every streaming algorithm with tunable accuracy guarantees into a DP algorithm with similar accuracy, for low sensitivity functions. Their transformation does not obviously extend to the continual release setting. Moreover CountDistinct has high sensitivity for turnstile streams.
The first lower bound in the continual release model of differential privacy was an  $\Omega (\log T)$  bound on the accuracy of mechanisms for bit summation, shown by Dwork et al. [25]. Jain et al. [44] gave the first polynomial separation in terms of error between the continual release model and the batch model under differential privacy. Our lower bounds also show such a separation. The lower bounds of Jain et al. [44] were for the problems of outputting the value and index of the attribute with the highest sum, amongst  $d$  attributes of a dataset. Our lower bounds are inspired by their sequential embedding technique to reduce multiple instances of a batch problem to a problem in the continual release model. Similar to them, we also reduce from the 1-way marginals problem to obtain our item-level lower bound. However, our event-level lower bound involves reducing from a different problem, and our reductions use the specific structure of CountDistinct for turnstile streams.
1.4 Broader impact, limitations, and open questions
We study the achievable error of DP mechanisms for CountDistinct under continual observation in streams with insertions and deletions. We show that it is characterized by the maximum flippancy of the stream. Our work is motivated by societal concerns, but focused on fundamental theoretical limits. It contributes to the broader agenda of obtaining privacy-preserving algorithms for data analysis. We discuss natural directions for future research and some limitations of our work.
Tight bounds: We found the best achievable error in some settings, but our upper and lower bounds do not match in some parameter regimes. What is the right error bound for event-level privacy for streams  $x$  with maximum flippancy  $w_{x}$  between  $\sqrt{T}$  and  $T^{2/3}$ ? Our results yield a lower bound of  $T^{1/4}$  and an upper bound of roughly  $\sqrt{w_{x}}$ .
Bounded memory: We did not consider any memory restrictions. Prior to our work, no other work addressed CountDistinct with deletions under continual release—with or without space constraints. We consider only the privacy constraint since it is more fundamental—it cannot be avoided by buying more memory—and the best algorithms with unbounded memory provide a benchmark by which to evaluate space-constrained approaches.
Space complexity is certainly a natural topic for future work. While it is not clear how to apply the sketching techniques of Epasto et al. [29] to the turnstile setting, it would be interesting to come up with accurate, private, and low-memory mechanisms for counting distinct elements in turnstile streams. Such algorithms would necessarily mix multiplicative and additive error guarantees (due to space and privacy constraints, respectively).
1.5 Organization
In Section 2, we present a mechanism for privately approximating CountDistinct on turnstile streams. Preliminaries on differential privacy, proofs omitted from Section 2 as well as proofs of Theorems 1.6 and 1.7 all appear in the supplementary material.
2 Item-level private mechanisms for CountDistinct
In this section, we present our item-level-DP mechanism for CountDistinct for turnstile streams. Its guarantees are stated in Theorem 2.1 with zero Concentrated Differential Privacy (zCDP). Using this notion of privacy, one can show tight bounds for the Gaussian mechanism and cleaner and tighter bounds for composition. Theorem 2.1 is the key ingredient in our proof of Theorem 1.5. This section is dedicated to our CountDistinct mechanism and the main ideas behind it. All proofs are deferred to the supplementary material.
Theorem 2.1 (Upper bound). For all  $\rho \in (0,1]$  and sufficiently large  $T\in \mathbb{N}$ , there exists a  $\rho$ -item-level- $z$ CDP mechanism for CountDistinct that is  $\alpha$ -accurate for all turnstile streams  $x$  of length  $T$ , where
$$
\alpha = O \Big (\frac {\sqrt {w _ {x}} \log T + \log^ {3} T}{\sqrt {\rho}} \Big),
$$
In Section 2.1, we describe a modification to the binary-tree mechanism which, when analyzed carefully, provides the desired error guarantees—but only if the maximum flippancy of the stream is known upfront. In Section 2.2, we use this mechanism, in conjunction with a method for adaptively estimating the flippancy bound, to obtain our item-level-DP mechanism for CountDistinct.
2.1 Enforcing a given flippancy bound  $w$
When a flippancy upper bound  $w$  is given upfront, we leverage the structure of the binary-tree mechanism to privately output the number of distinct elements at each time  $t \in [T]$ , where  $T$  is the stream length. The mechanism and its error guarantees are presented in Algorithm 1 and Theorem 2.2, respectively. To provide intuition, we first describe the mechanism when it is run on streams with maximum flippancy at most  $w$ . We then discuss a modification that ensures privacy of the mechanism for all streams regardless of maximum flippancy.
Algorithm 1 stores vectors  $\tilde{f}_u \in \{0,1\}^T$  for all elements  $u \in \mathcal{U}$  that appear in the stream. For streams with maximum flippancy at most  $w$ , the vector  $\tilde{f}_u$  is equal to the existence vector  $f_u$ . In this case, by Definition 1.1, the number of distinct elements at timestep  $t \in [T]$  equals  $\sum_{u \in \mathcal{U}} \tilde{f}_u[t]$ . The mechanism outputs values  $\sum_{u \in \mathcal{U}} \tilde{f}_u[t]$  with Gaussian noise added according to the binary-tree mechanism, with privacy parameter  $\approx \rho / w$  (see Definition 2.4)—that is, with noise scaled up by a factor of  $\approx \sqrt{w / \rho}$ . The accuracy of this mechanism follows from that of the binary-tree mechanism.
However, if the mechanism computed  $f_{u}$  instead of  $\tilde{f}_{u}$ , it would not be private for streams with maximum flippancy greater than  $w$ , since it adds noise that scales according to  $w$ . That is because for every stream  $x \in \mathcal{U}_{\pm}^{T}$  with maximum flippancy  $w_{x} > w$  there exists a neighboring stream  $x'$  such that the vectors CountDistinct(x) and CountDistinct(x') differ in as many as  $\Theta(w_{x})$  indices. To provide privacy for such streams, the mechanism simply "truncates" the vector  $f_{u} \in \{0,1\}^{T}$  to obtain  $\tilde{f}_{u}[t] = 0$  for all  $t \geq t^{*}$  if the flippancy of  $u$  in  $x[1:t^{*}]$  exceeds  $w$ . This corresponds to running the naive version of the mechanism (that uses  $f_{u}$  instead of  $\tilde{f}_{u}$ ) on the "truncated" version of the stream  $x$ , where elements in  $x$  are ignored after their flippancy exceeds  $w$ . (Note that the computation of  $\tilde{f}_{u}$  can be done online since  $\tilde{f}_{u}[t]$  depends only on  $x[1:t]$ .) With a careful analysis of the value stored in each node of the binary tree, we are able to show that this mechanism is  $\rho$ -item-level-zCDP for all streams, however, it loses accuracy for streams with many high flippancy elements. In Section 2.2, we leverage this mechanism to provide estimates of CountDistinct that are both accurate and private for all streams.
Theorem 2.2 (Mechanism for a given flippancy bound  $w$ ). Fix  $\rho \in (0,1]$ , sufficiently large  $T \in \mathbb{N}$  and  $w \leq T$ . Algorithm 1 is a mechanism for CountDistinct for turnstile streams that is  $\rho$ -item-
level-zCDP for all input streams of length  $T$ , and  $\alpha$ -accurate for streams of length  $T$  with maximum flippancy at most  $w$ , where  $\alpha = O\left(\frac{\sqrt{w}\log T + \log^3T}{\sqrt{\rho}}\right)$ .
Algorithm 1 Mechanism  $\mathcal{M}$  for CountDistinct with given flippancy bound
Input: Time horizon  $T\in \mathbb{N}$  , privacy parameter  $\rho >0$  , flippancy bound  $w > 0$  stream  $x\in \mathcal{U}_{\pm}^{T}$  Output: Vector  $s\in \mathbb{R}^T$  of distinct count estimates   
1: Sample a binary-tree random variable  $Z\in \mathbb{R}^T$  with parameter  $\rho^{\prime} = \frac{\rho}{4w(\log T + 1)}\triangleright$  Definition 2.4   
2: Initialize  $\mathcal{U}_x = \emptyset$    
3: for all  $t\in [T]$  do   
4: Obtain entry  $x[t]$  and skip to Step 10 if  $x[t] = \bot$    
5: Suppose  $x[t]$  is an insertion or deletion of a universe element  $u$    
6: if  $u\notin \mathcal{U}_x$  then insert  $u$  into  $\mathcal{U}_x$  ; initialize count  $u = 0$  and  $\tilde{f}_u = 0^T$ $\triangleright$  vector with  $T$  zeros   
7: if  $x[t] = +u$  then count  $u + = 1$  else count  $u - = 1$    
8: for all  $v\in \mathcal{U}_x$  do   
9: if flip  $(v,x[1:t])\leq w$  and count  $v > 0$  then set  $\tilde{f}_v[t] = 1$    
10: Return  $s[t] = (\sum_{u\in \mathcal{U}_x}\tilde{f}_u[t]) + Z[t]$
Definition 2.3 (Dyadic decomposition). For  $t \in \mathbb{N}$ , the dyadic decomposition of the interval  $(0, t]$  is a set of at most  $\log t + 1$  disjoint intervals whose union is  $(0, t]$ , obtained as follows. Consider the binary representation of  $t$  (which has at most  $\log t + 1$  bits), and express  $t$  as a sum of distinct powers of 2. Then, the first interval is  $(0, r]$ , where  $r$  is the largest power of 2 in the sum. The second interval starts at  $r + 1$  and its size is the second largest power of 2 in the sum. The remaining intervals are defined similarly for all remaining summands. For example, for  $t = 11 = 8 + 2 + 1$ , the dyadic decomposition of  $(0, 11]$  is the intervals  $(0, 8]$ ,  $(8, 10]$  and  $(10, 11]$ .
Definition 2.4 (Binary tree and binary-tree random variable). Let  $\rho > 0$  be a privacy parameter and  $T \in \mathbb{N}$  be a power of 2. Consider a complete binary tree with  $T$  leaves whose nodes are labeled as follows. The  $T$  leaves are labeled by the intervals  $(t - 1, t]$  for all  $t \in [T]$  and the internal nodes are labeled by intervals obtained from the union of their children's intervals. Specifically, the binary tree consists of  $\log T + 1$  levels. A level  $\ell \in [0, \log T]$  partitions the interval  $(0, T]$  into a set of  $\frac{T}{2^{\ell}}$  disjoint intervals, each of length  $2^{\ell}$ , of the form  $((i - 1) \cdot 2^{\ell}, i \cdot 2^{\ell}]$ . The nodes in level  $\ell$  are labelled by the intervals in this partition.
The binary-tree random variable  $Z \in \mathbb{R}^T$  with parameter  $\rho$  is defined as follows. For each node  $(t_1, t_2]$  in the binary tree with  $T$  leaves, let  $Z_{(t_1, t_2]} \sim \mathcal{N}(0, 1 / \rho)$ . For each  $t \in [T]$ , consider the dyadic decomposition of the interval  $(0, t]$  (Definition 2.3) and let  $Z[t]$  be the sum of the random variables corresponding to the intervals in this dyadic decomposition.
The proof of Theorem 2.2 can be found in Section B.
2.2 Adaptively estimating a good flippancy bound  $w$
In this section, we leverage the privacy and accuracy guarantees of Algorithm 1 to construct a new mechanism (Algorithm 3) for estimating CountDistinct. It achieves the privacy and accuracy guarantees of Theorem 2.1, when the maximum flippancy is not known upfront. Algorithm 3 instantiates  $\log T + 1$  different copies  $\mathcal{B}_0, \ldots, \mathcal{B}_{\log T}$  of Algorithm 1 with flippancy bounds  $2^0, \ldots, 2^{\log T}$ , respectively (the maximum flippancy of a stream is at most  $T$ ). To obtain an accurate estimate of the distinct elements count, at each time  $t \in [T]$ , we privately select  $i \in [0, \log T]$  such that the output of  $\mathcal{B}_i$  satisfies the desired accuracy guarantee for the stream entries  $x[1:t]$  received so far. Selecting such  $i$  amounts to selecting a good bound on the maximum flippancy of the stream  $x[1:t]$ . Next, we describe how to obtain this bound using the sparse vector technique (Algorithm 2).
The maximum flippancy has high sensitivity; changing one stream entry can change the maximum flippancy drastically. However, the number of items with flippancy greater than any particular threshold is a function of sensitivity one. Furthermore, since Algorithm 1 when run with flippancy bound  $w$  already has error about  $\sqrt{w / \rho}$ , its accuracy guarantee remains asymptotically the same even if it simply ignores that many elements with flippancy greater than  $w$ . Thus, Algorithm 3 uses the sparse vector technique to maintain an upper bound on the flippancy of  $x[1:t]$  such that not too
many elements in  $x[1:t]$  violate that bound. This bound, in combination with the error guarantee of Algorithm 1, suffices to provide the desired low error guarantee. Since the sparse vector algorithm remains differentially private even when its queries are chosen adaptively, the privacy guarantees of Algorithm 3 follow from the privacy of Algorithms 1 and 2.
Algorithm 2 SVT: Answering Threshold Queries with Sparse Vector Technique
Input: Stream  $x$  queries  $q_{1}, q_{2}, \ldots$  of sensitivity 1, cutoff  $c > 0$ , privacy parameter  $\rho$   
Output: Stream of Above or Below answers  
1: Let  $\varepsilon = \sqrt{2\rho}$  and set count  $= 0$   
2: Let  $Z \sim \mathrm{Lap}(2 / \varepsilon)$   
3: for each query  $q_{t}$  do  
4: Let  $Z_{t} \sim \mathrm{Lap}(4c / \varepsilon)$   
5: if  $q_{t}(x) + Z_{t} \geq Z$  and count  $< c$  then  
6: Return Above  
7: count = count + 1  
8: else  
9: Return Below
Algorithm 3 Mechanism  $\mathcal{M}$  for CountDistinct
Input: Time horizon  $T\in \mathbb{N}$  , privacy parameter  $\rho >0$  stream  $x\in \mathcal{U}_{\pm}^{T}$    
Output: Vector  $s$  of distinct count estimates   
1: Initialize vector  $w_{\mathrm{max}} = 1\circ 0^{T - 1}$    
2: for all  $i\in [0,\log T]$  do   
3: Initialize  $\mathcal{B}_i$  as Algorithm 1 with horizon  $T$  , privacy parameter  $\frac{\rho}{2(\log T + 1)}$  ,flippancy  $2^{i}$    
4: Initialize SVT with privacy parameter  $\rho /2$  and cutoff log  $T$  See Algorithm 2   
5: for all  $t\in [T]$  do   
6: Obtain entry  $x[t]$    
7: If  $t\geq 2$  , set  $w_{\mathrm{max}}[t] = w_{\mathrm{max}}[t - 1]$    
8: for all  $i\in [\log T]$  do   
9: Send  $x[t]$  to mechanism  $\mathcal{B}_i$  and get output  $s_{i,t}$    
10: while True do   
11: Consider query  $q_{t} = |\{u\in \mathcal{U}:f l i p(u,x[1:t])\geq w_{\mathrm{max}}[t]\} | - \sqrt{\frac{w_{\mathrm{max}}[t]}{\rho}}$    
12: Send query  $q_{t}$  to SVT and if the output is "Below", break   
13: Update  $w_{\mathrm{max}}[t] = 2\cdot w_{\mathrm{max}}[t]$    
14: Return  $s_{j,t}$  for  $j = \log (w_{\mathrm{max}}[t])$  ▷ Note that  $j\in [0,\log T]$
The proof of Theorem 2.1 can be found in Section C.
We thank Teresa Steiner and an anonymous reviewer for useful suggestions on the initial version of this paper. S.S. was supported by NSF award CNS-2046425 and Cooperative Agreement CB20ADR0160001 with the Census Bureau. A.S. and P.J. were supported in part by NSF awards CCF-1763786 and CNS-2120667 as well as Faculty Awards from Google and Apple.
[1] Naman Agarwal and Karan Singh. The price of differential privacy for online learning. In Proceedings, International Conference on Machine Learning (ICML), pages 32-40, 2017.
[2] Aditya Akella, Ashwin R. Bharambe, Michael K. Reiter, and Srinivasan Seshan. Detecting DDoS attacks on ISP networks. In Workshop on Management and Processing of Data Streams, 2003.
[3] Noga Alon, Y. Matias, and Mario Szegedy. The space complexity of approximating the frequency moments. In Proceedings, ACM Symposium on Theory of Computing (STOC), 1996.
[57] Abhradeep Guha Thakurta and Adam Smith. (Nearly) optimal algorithms for private online learning in full-information and bandit settings. In Advances in Neural Information Processing Systems (NeurIPS), volume 26, 2013.
[58] Lun Wang, Iosif Pinelis, and Dawn Song. Differentially private fractional frequency moments estimation with polylogarithmic space. In International Conference on Learning Representations (ICLR), 2022.
[59] David P. Woodruff. Optimal space lower bounds for all frequency moments. In Proceedings, ACM-SIAM Symposium on Discrete Algorithms (SODA), pages 167-175. SIAM, 2004.