SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\operatorname {D C} \left(D; \ell_ {A} ^ {*}\right) - \frac {2 \ell_ {A} ^ {*}}{\varepsilon} \log \left(\frac {1}{2 \beta}\right) - \frac {8 \ell_ {A} ^ {*}}{\varepsilon} \log \left(\frac {\ell_ {\max }}{\beta}\right) \leq \operatorname {D C} \left(D; \ell_ {A} ^ {*}\right) + (\hat {\ell} - \ell_ {A} ^ {*}) \frac {A}{\varepsilon} - \frac {2 \hat {\ell}}{\varepsilon} \log \left(\frac {1}{2 \beta}\right), \tag {15}
$$
$$
\hat {\ell} \cdot \left(2 \log \left(\frac {1}{2 \beta}\right) - A\right) \leq \ell_ {A} ^ {*} \cdot \left(2 \log \left(\frac {1}{2 \beta}\right) + 8 \log \left(\frac {\ell_ {\max }}{\beta}\right) - A\right). \tag {16}
$$
Now we set  $A = \log \left(\frac{1}{2\beta}\right)$  to obtain
$$
\hat {\ell} \cdot \log \left(\frac {1}{2 \beta}\right) \leq \ell_ {A} ^ {*} \cdot \left(\log \left(\frac {1}{2 \beta}\right) + 8 \log \left(\frac {\ell_ {\max }}{\beta}\right)\right). \tag {17}
$$
Substituting Equation (17) into Equation (13) gives
$$
\underset {(\hat {\ell}, \hat {\nu}) \leftarrow \mathcal {M} (D)} {\mathbb {P}} \left[ \hat {\nu} \geq \max  _ {\ell \in [ \ell_ {\max } ]} \mathrm {D C} (D; \ell) - \frac {2 \ell + 2 \ell_ {A} ^ {*}}{\varepsilon} \log \left(\frac {1}{2 \beta}\right) - \frac {8 \ell + 1 6 \ell_ {A} ^ {*}}{\varepsilon} \log \left(\frac {\ell_ {\max }}{\beta}\right) \right] \geq 1 - 2 \beta . \tag {18}
$$
We simplify Equation (18) using  $\log \left(\frac{1}{2\beta}\right) \leq \log \left(\frac{\ell_{\max}}{\beta}\right)$  to obtain Equation (7).
Computational efficiency: Finally, the runtime of DPDISTINCTCOUNT  $(D)$  is dominated by  $\ell_{\mathrm{max}}$  calls to the SENSITIVEDISTINCTCOUNT  $(D)$  subroutine, which computes the maximum size of a bipartite matching on a graph with  $|E| = \sum_{i\in [n]}|u_i|\cdot \min \{\ell ,|u_i|\} \leq |D|\cdot \ell_{\mathrm{max}}$  edges and  $|V| + |U| = \mathrm{DC}(D) + \sum_{i\in [n]}\min \{\ell ,|u_i|\} \leq 2|D|$  vertices. The Hopcroft-Karp-Karzanov algorithm runs in time  $O(|E|\cdot \sqrt{|V| + |U|})\leq O(|D|^{1.5}\cdot \ell_{\mathrm{max}})$  time.
Proof of Theorem 1.2. Privacy: For convenience we analyze Algorithm 4, which produces the same result as Algorithm 2. (The difference is that Algorithm 4 is written to be efficient, while Algorithm 2 is written in a redundant manner to make the subroutine we are analyzing clear.) The privacy of the algorithm follows by composing the privacy guarantees of the generalized exponential mechanism and Laplace noise addition. The only missing part is to prove that SENSITIVEAPPROXDISTINCTCOUNT(·, ℓ) has sensitivity ℓ.
$$
\begin{array}{l} \text {S E N S I T I V E A P P R O X D I S T I N C T C O u n t} \left(\left(u _ {1}, \dots , u _ {n}\right), \ell\right) \\ = \text {S E N S I T I V E A P P R O X D I S T I N C T C O u n t} (\underbrace {u _ {1} , \cdots , u _ {n} , \cdots , u _ {1} , \cdots , u _ {n}} _ {\ell \text {t i m e s}}, 1); \\ \end{array}
$$
therefore, it is enough to prove that SENSITIVEAPPROXDISTINCTCOUNT(.,1) has sensitivity 1.
Assume  $D = (u_{1},\dots ,u_{n})$  and  $D^{\prime} = (u_{1},\dots ,u_{j - 1},u_{j + 1},\dots ,u_{n})$ . Let  $S_{1},\dots ,S_{n}$  and  $v_{1},\dots ,v_{n}$  be the states of  $S$  and  $v$  ( $v_{i} = \bot$  if  $u_{i}\setminus \bar{S}_{i - 1} = \emptyset$ ), respectively, when
Algorithm 4 Approximate Distinct Count Algorithm
1: procedure SENSITIVEAPPROXDISTINCTCOUNT  $(D = (u_{1},\dots ,u_{n})\in (\Omega^{*})^{n};\ell \in \mathbb{N})$    
2:  $S\gets \emptyset$    
3: for  $\ell^{\prime}\in [\ell ]$  do   
4: for  $i\in [n]$  with  $u_{i}\setminus S\neq \emptyset$  do   
5: Choose lexicographically first  $v\in u_i\setminus S$  .  $\triangleright$  Match  $(i,\ell^{\prime})$  to  $v$    
6: Update  $S\leftarrow S\cup \{v\}$    
7: end for   
8: end for   
9: return  $|S|$    
10: end procedure   
11: procedure DPAPPROXDISTINCTCOUNT  $(D = (u_{1},\dots ,u_{n})\in (\Omega^{*})^{n};\ell_{\max}\in \mathbb{N},\varepsilon >0,\beta \in$ $(0,\frac{1}{2}))$    
12: for  $\ell \in [\ell_{\mathrm{max}}]$  do   
13: Define  $q_{\ell}(D)\coloneqq$  SENSITIVEAPPROXDISTINCTCOUNT  $(D;\ell) - \frac{2\ell}{\varepsilon}\cdot \log \left(\frac{1}{2\beta}\right)$    
14: end for   
15:  $\hat{\ell}\gets \mathrm{GEM}(D;\{q_\ell \}_{\ell \in [\ell_{\mathrm{max}}]},\{\ell \}_{\ell \in [\ell_{\mathrm{max}}]},\varepsilon /2,\beta).$  ▷ Algorithm 3   
16:  $\hat{\nu}\gets q_{\hat{\ell}}(D)+\mathrm{Lap}\left(2\hat{\ell}/\varepsilon\right)$    
17: return  $(\hat{\ell},\hat{\nu})\in [\ell_{\mathrm{max}}]\times \mathbb{R}$    
18: end procedure
we run APPROXSENSITIVEDISTINCTCOUNT  $(D,1)$ . Similarly, let  $S_1',\dots,S_{j-1}',S_{j+1}',\dots,S_n', v_1',\dots,v_{j-1}',v_{j+1}',\dots,v_n'$  be the states of  $S$  and  $v$  ( $v_i' = \bot$  if  $u_i \setminus S_{i-1}' = \emptyset$ ), respectively, when we run APPROXSENSITIVEDISTINCTCOUNT  $(D',1)$ . Our goal is to show that  $||S_n| - |S_n'|| \leq 1$ . Define  $S_j' = S_{j-1}'$  and  $v_j' = \bot$ . Clearly  $S_i' = S_i$  and  $v_i' = v_i$  for all  $i < j$ . For  $i \geq j$ , we claim that  $S_i' \subseteq S_i$  and  $|S_i| \leq |S_i'| + 1$ . This is true for  $i = j$ , since  $S_j' = S_{j-1}' = S_{j-1}$  and  $S_j = S_{j-1} \cup \{v_j\}$ . We prove the claim for  $i > j$  by induction. I.e., assume  $S_{i-1} = S_{i-1}' \cup \{v_{i-1}^*\}$  for some  $v_{i-1}^*$  (possibly  $v_{i-1}^* = \bot$ , whence  $S_{i-1} = S_{i-1}'$ ). Now  $v_i$  is the lexicographically first element of  $u_i \setminus S_{i-1}$  and  $v_i'$  is the lexicographically first element of  $u_i \setminus S_{i-1}'$  (or  $\bot$  if these sets are empty). By the induction assumption,  $u_i \setminus S_{i-1} \subseteq u_i \setminus S_{i-1}' \subseteq (u_i \setminus S_{i-1}) \cup \{v_{i-1}^*\}$ . Thus either  $v_i' = v_i$  or  $v_i' = v_{i-1}^*$ . If  $v_i' = v_i$ , then  $S_i' = S_{i-1}' \cup \{v_i'\} \subset S_{i-1} \cup \{v_i\} = S_i$  and  $S_i' \cup \{v_{i-1}^*\} = S_{i-1}' \cup \{v_i', v_{i-1}^*\} \supset S_{i-1} \cup \{v_i\} = S_i$ . If  $v_i' = v_{i-1}^*$ , then  $S_i' = S_{i-1}' \cup \{v_{i-1}^*\} \subset S_{i-1} \cup \{v_{i-1}^*\} = S_{i-1} \subset S_i$  and  $S_i' \cup \{v_i\} = S_{i-1}' \cup \{v_i, v_{i-1}^*\} \supset S_{i-1} \cup \{v_i\} = S_i$ , so the claim holds with  $v_i^* = v_i$ .
The sensitivity bound implies that  $q_{\ell}(D) = |S| - \frac{2\ell}{\varepsilon}\log (1 / 2\beta)$  has sensitivity  $\ell$ . Since the generalized exponential mechanism is  $\varepsilon /2$ -DP and adding Laplace noise is also  $\varepsilon /2$ -DP, the overall algorithm is  $\varepsilon$ -DP by composition.
Lower bound: Let us denote by  $\widehat{\mathrm{DC}}(D; \ell) = \text{SENSITIVEAPPROXDISTINCTCOUNT}(\cdot, \ell)$  the value of  $|S|$  we obtain on Line 8 in Algorithm 2. Note that  $\widehat{\mathrm{DC}}(D; \ell)$  is the size of a maximal matching in  $G_{\ell}$ , where  $G_{\ell}$  is the bipartite graph corresponding to the input  $D$  with  $\ell$  copies of each person (see Algorithm 1 for a formal description of the graph). Since a maximal matching is a 2-approximation to a maximum matching [God], we have
$$
\frac {1}{2} \mathrm {D C} (D; \ell) \leq \widehat {\mathrm {D C}} (D; \ell) \leq \mathrm {D C} (D; \ell) \leq \mathrm {D C} (D). \tag {19}
$$
Since  $\hat{\nu}\gets q_{\hat{\ell}}(D) + \mathrm{Lap}\left(2\hat{\ell} /\varepsilon\right)$ , we have
$$
\mathbb {P} _ {\hat {\nu}} \left[ \hat {\nu} \leq q _ {\hat {\ell}} (D) + \frac {2 \hat {\ell}}{\varepsilon} \log \left(\frac {1}{2 \beta}\right) \right] = \mathbb {P} _ {\hat {\nu}} \left[ \hat {\nu} \geq q _ {\hat {\ell}} (D) - \frac {2 \hat {\ell}}{\varepsilon} \log \left(\frac {1}{2 \beta}\right) \right] = 1 - \beta . \tag {20}
$$
For notational convenience, we define  $\{\bot\} = \emptyset$ .
Substituting  $q_{\ell}(D) = \widehat{\mathrm{DC}}(D; \ell) - \frac{2\ell}{\varepsilon} \log(1/2\beta)$  into Equations (19) and (20) gives Equation (9)
$$
\mathbb {P} _ {\hat {\nu}} [ \hat {\nu} \leq \mathrm {D C} (D) ] \geq \mathbb {P} _ {\hat {\nu}} \left[ \hat {\nu} \leq \widehat {\mathrm {D C}} (D; \hat {\ell}) \right] = 1 - \beta .
$$
$$
\mathbb {P} _ {\hat {\ell}} \left[ q _ {\hat {\ell}} (D) \geq \max  _ {\ell \in [ \ell_ {\max } ]} q _ {\ell} (D) - \ell \cdot \frac {4}{\varepsilon / 2} \log \left(\ell_ {\max } / \beta\right) \right] \geq 1 - \beta . \tag {21}
$$
Combining Equations (20) and (21) and the definition of  $q_{\ell}(D)$  with a union bound yields
$$
\underset {(\hat {\ell}, \hat {\nu}) \leftarrow \mathcal {M} (D)} {\mathbb {P}} \left[ \hat {\nu} \geq \max  _ {\ell \in [ \ell_ {\max } ]} \widehat {\mathrm {D C}} (D; \ell) - \frac {2 \ell + 2 \hat {\ell}}{\varepsilon} \log \left(\frac {1}{2 \beta}\right) - \frac {8 \ell}{\varepsilon} \log \left(\frac {\ell_ {\max }}{\beta}\right) \right] \geq 1 - 2 \beta . \tag {22}
$$
Now we assume  $\max_{i\in [n]}|u_i|\leq \ell_*\leq \ell_{\mathrm{max}}$  for some  $\ell_{*}$ . This implies  $\mathrm{DC}(D) = \mathrm{DC}(D;\ell)$ . We have
$$
\max  _ {\ell \in [ \ell_ {\max } ]} \widehat {\mathrm {D C}} (D; \ell) - \frac {2 \ell + 2 \hat {\ell}}{\varepsilon} \log \left(\frac {1}{2 \beta}\right) - \frac {8 \ell}{\varepsilon} \log \left(\frac {\ell_ {\max }}{\beta}\right) \tag {23}
$$
$$
\geq \widehat {\mathrm {D C}} (D; \ell_ {*}) - \frac {2 \ell_ {*} + 2 \hat {\ell}}{\varepsilon} \log \left(\frac {1}{2 \beta}\right) - \frac {8 \ell_ {*}}{\varepsilon} \log \left(\frac {\ell_ {\operatorname* {m a x}}}{\beta}\right) \tag {24}
$$
$$
\geq \frac {1}{2} \mathrm {D C} (D) - \frac {2 \ell_ {*} + 2 \hat {\ell}}{\varepsilon} \log \left(\frac {1}{2 \beta}\right) - \frac {8 \ell_ {*}}{\varepsilon} \log \left(\frac {\ell_ {\operatorname* {m a x}}}{\beta}\right). \tag {25}
$$
As in the proof of Theorem 1.1, if the event in Equation (21) happens, we can show that
$$
\hat {\ell} \cdot \log \left(\frac {1}{2 \beta}\right) \leq \ell_ {A} ^ {*} \cdot \left(\log \left(\frac {1}{2 \beta}\right) + 8 \log \left(\frac {\ell_ {\operatorname* {m a x}}}{\beta}\right)\right), \tag {26}
$$
$$
\ell_ {A} ^ {*} := \underset {\ell \in \left[ \ell_ {\max } \right]} {\arg \max } \widehat {\mathrm {D C}} (D; \ell) - \frac {\ell}{\varepsilon} \log \left(\frac {1}{2 \beta}\right) \tag {27}
$$
Note that  $\ell_A^* \leq \ell_*$ , since  $\widehat{\mathrm{DC}}(D; \ell) \leq \widehat{\mathrm{DC}}(D; \ell_*)$  for all  $\ell \in [\ell_{\max}]$ . (That is simply to say that the size of the maximal matching cannot be increased by making more copies of a vertex once there is one copy for each neighbor.) Combining bounds yields
$$
\underset {(\hat {\ell}, \hat {\nu}) \leftarrow \mathcal {M} (D)} {\mathbb {P}} \left[ \hat {\nu} \geq \frac {1}{2} \mathrm {D C} (D) - \frac {4 \ell_ {*}}{\varepsilon} \log \left(\frac {1}{2 \beta}\right) - \frac {2 4 \ell_ {*}}{\varepsilon} \log \left(\frac {\ell_ {\operatorname* {m a x}}}{\beta}\right) \right] \geq 1 - 2 \beta . \tag {28}
$$
Since  $\log \left(\frac{1}{2\beta}\right) \leq \log \left(\frac{\ell_{\max}}{\beta}\right)$ , this implies Equation (10).
Computational efficiency: It only remains to verify that Algorithm 2 can be implemented in  $O(|D| + \ell_{\max} \log \ell_{\max})$  time. We can implement  $S$  using a hash table to ensure that we can add an element or query membership of an element in constant time. (We can easily maintain a counter for the size of  $S$ .) We assume  $D$  is presented as a linked list of linked lists representing each  $u_i$  and furthermore that the linked lists  $u_i$  are sorted in lexicographic order. The outer loop proceeds through the linked list for  $D = (u_1, \dots, u_n)$ . For each  $u_i$ , we simply pop elements from the linked list and check if they are in  $S$  until either we find  $v \in u_i \setminus S$  (and add  $v$  to  $S$ ) or  $u_i$  becomes empty (in which case we remove it from the linked list for  $D$ .) Since each iteration decrements  $|D|$ , the runtime of the main loop is  $O(|D|)$ . Running the generalized exponential mechanism (Algorithm 3) takes  $O(\ell_{\max} \log \ell_{\max})$  time.
B Implementing the Generalized Exponential Mechanism
We conclude with some remarks about implementing the generalized exponential mechanism of Raskhodnikova and Smith [RS15] given in Algorithm 3. There are two parts to this; first we must compute the normalized scores and then we must run the standard exponential mechanism.
Implementing the standard exponential mechanism is well-studied [Ilv20] and can be performed in linear time (with some caveats about randomness and precision). We remark that, instead of the exponential mechanism, we can use report-noisy-max or permute-and-flip [MS20; DKSWXZ+21; DDKPWWXZ23]. These variants may be easier to implement and may provide better utility too.
The normalized scores are given by
$$
\forall i \in [ m ] \quad s _ {i} = \min  _ {j \in [ m ]} \frac {\left(q _ {i} (D) - t \Delta_ {i}\right) - \left(q _ {j} (D) - t \Delta_ {j}\right)}{\Delta_ {i} + \Delta_ {j}}, \tag {29}
$$
where  $q_{i}(D)$  is the value of the query on the dataset,  $\Delta_{i} > 0$  is the sensitivity of  $q_{i}$ , and  $t$  is a constant. Naively it would take  $\Theta(m^{2})$  time to compute all the normalized scores. However, a more complex algorithm can compute the scores in  $O(m \log m)$  time.
Observe that, for each  $i\in [m]$  , we have
$$
\begin{array}{l} s _ {i} = \min _ {j \in [ m ]} \frac {(q _ {i} (D) - t \Delta_ {i}) - (q _ {j} (D) - t \Delta_ {j})}{\Delta_ {i} + \Delta_ {j}} \iff \min _ {j \in [ m ]} \frac {(q _ {i} (D) - t \Delta_ {i}) - (q _ {j} (D) - t \Delta_ {j}) - s _ {i} (\Delta_ {i} + \Delta_ {j})}{\Delta_ {i} + \Delta_ {j}} = 0 \\ \iff \min  _ {j \in [ m ]} \left(q _ {i} (D) - t \Delta_ {i}\right) - \left(q _ {j} (D) - t \Delta_ {j}\right) - s _ {i} (\Delta_ {i} + \Delta_ {j}) = 0 \\ \iff q _ {i} (D) - (s _ {i} + t) \Delta_ {i} = \underbrace {\max  _ {j \in [ m ]} q _ {j} (D) + (s _ {i} - t) \Delta_ {j}} _ {f (s _ {i} - t)}. \\ \end{array}
$$
That is, we can compute  $s_i$  by solving the equation  $q_i(D) - (s_i + t)\Delta_i = f(s_i - t)$ .
Since  $f(x) \coloneqq \max_{j \in [m]} q_j(D) + x\Delta_j$  is the maximum of increasing linear functions, we have that  $f$  is a convex, increasing, piecewise-linear function, with at most  $m$  pieces. We can represent  $f$  by a sorted list of the points where the linear pieces connect, along with the linear function on each of the pieces. We can compute this representation of  $f$  as a pre-processing step in  $O(m\log m)$  time; we sort the lines  $y(x) = q_j(D) + x\Delta_j$  by their slope  $\Delta_j$  and then compute the intersection points between consecutive lines (we delete lines that never realize the max).
Given the above representation of  $f$  and the values  $q_{i}(D), \Delta_{i}, t$ , we can compute  $s_{i}$  in  $O(\log m)$  time. We must solve  $q_{i}(D) - (s_{i} + t)\Delta_{i} = f(s_{i} - t)$  for  $s_{i}$ . We can perform binary search on the pieces of  $f$  to identify  $j$  such that  $f(s_{i} - t) = q_{j}(D) + (s_{i} - t)\Delta_{j}$ . Once we have this we can directly compute  $s_{i} = \frac{(q_{i}(D) - t\Delta_{i}) - (q_{j}(D) - t\Delta_{j})}{\Delta_{i} + \Delta_{j}}$ . The binary search takes  $O(\log m)$  time and we must compute  $m$  scores. Thus the overall runtime (including the pre-processing) is  $O(m\log m)$ .