SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Using a slightly weaker result, [70, 25] was able to extend it to an algorithm for generating a private semi-coreset for  $k$ -means or  $k$ -median. Given Theorem A.9, the algorithm simply computes  $C = \{c_1, \ldots, c_k\}$ , and gives each  $c_i$  a weight which is the number of points in  $X$  closest to  $c_i$ , plus  $\mathrm{Lap}(1 / \varepsilon)$  noise. By combining Theorem A.9 and the conversion of [70, 25] (e.g., see [25, Lemma C.1], which uses a slightly weaker bound), the following is immediate.
Lemma A.10. For some  $\kappa = O(1)$ , there exists a polynomial-time  $(\varepsilon, \delta)$ -DP algorithm that, given a set  $X = \{x_{1}, \ldots, x_{n}\}$  in a fixed ball of radius  $R$  in  $\mathbb{R}^d$ , computes a  $(\kappa, U, \rho)$ -semi-coreset for  $k$ -means, where  $U = O\left(\frac{k \log^2 n \log(1 / \delta) + k \sqrt{d \log(1 / \delta)}}{\varepsilon}\right)$ .
B Crude Approximation
In this section, we devise a crude bicriteria approximation that will serve as a starting point in developing our more refined algorithm. To recall the setup of the bicriteria problem (see Definition
Algorithm 2 Approximate dist-DP bicriteria algorithm
1: Input: Parameters  $n, d, k, \varepsilon, \delta, \Lambda, \rho$ , dataset  $X = \{x_{1}, \ldots, x_{n}\} \subset \mathbb{R}^{d}$ .
2: Output: Crude bicriteria approximation  $F = \{f_{1},\ldots ,f_{\alpha \cdot k}\} \subset \mathbb{R}^{d}$ : will be  $(O(\varepsilon),O(\delta),\rho)$ -dist-DP.
3: Initialize  $A = O(\varepsilon^{-1}\sqrt{\log\delta^{-1}}\cdot d\sqrt{d + \log n})$ $B = n$ $REP = O(\log n)$
4: Initialize  $\varepsilon' = \Theta \left( \frac{\varepsilon}{\sqrt{\log n \log(A \cdot B) \log(1 / \delta)}} \right)$  and  $\delta' = \Theta \left( \frac{\delta}{\log n \log(A \cdot B))} \right)$ .
5: for  $i = 1$  to  $n$  do
6:  $\tilde{x}_i\coloneqq x_i + \frac{\rho\cdot\sqrt{2\log(1.25 / \delta)}}{\varepsilon}\cdot \mathcal{N}(0,I).$
7: for rep = 1 to REP $\mathbf{\tilde{d}}$ o
8: Create a randomly shifted Quadtree with largest level  $\ell = 0$  with side length  $\Lambda$  and smallest level with side length  $\rho / B$ .
9: for  $\ell = 0$  to  $L_{1} \coloneqq \log_{2}(\Lambda / (A\rho))$  do
0: for each cell  $g$  at level  $\ell$  containing some  $\tilde{x}_i$  do
1: count  $(g) = \# \{\tilde{x}_i$  in cell  $g\}$
2: Let  $g_1, \ldots, g_{4k}$  be the  $4k$  cells at level  $\ell$  with maximum count  $(g)$ .
3: Add the centers of  $g_{1}, \ldots, g_{4k}$  to  $F$ .
4:  $\mathbf{for}\ell = \log_2\left(\frac{\Lambda}{A\rho}\right) + 1\mathbf{to}L_2\coloneqq \log_2\left(\frac{B\Lambda}{\rho}\right)\mathbf{do}$
5: for each cell  $g$  at level  $\ell$  that contains some  $x_{i}\in X$  do
6: count  $(g) = \# \{x_{i}$  in cell  $g\} +\mathrm{TLap}(1 / \varepsilon^{\prime},1 / \delta^{\prime})$
7: Let  $g_1, \ldots, g_{4k}$  be the  $4k$  cells at level  $\ell$  with maximum count  $(g)$ .
8: Add each center  $g_i$  to  $F$ , if count  $(g_i) \geq \frac{K}{\varepsilon'} \log \frac{1}{\delta'}$  for some constant  $K$ .
9: Return  $F$ .
A.5), we are given a dataset  $X = \{x_{1},\ldots ,x_{n}\}$ , contained in a given ball of radius  $\Lambda$  in  $\mathbb{R}^d$ . We wish to compute an  $(\alpha ,\beta ,V)$ -approximation that satisfies  $(\varepsilon ,\delta ,\rho)$ -dist-DP. By this, we must output a set of  $\alpha \cdot k$  centers  $F = \{f_1,\dots ,f_{\alpha \cdot k}\}$  such that  $\mathrm{cost}(X;F)\leq \beta \cdot \mathrm{OPT}_k(X) + V\cdot \rho^p$  for some parameters  $\alpha ,\beta ,V$ , where  $p = 1$  for  $k$ -median and  $p = 2$  for  $k$ -means. In addition,  $F$  should be  $(\varepsilon ,\delta ,\rho)$ -dist-DP with respect to  $X$ .
Our desire for  $\alpha, \beta, V$  is that they are polynomial in  $d, k, \log n, \varepsilon^{-1}, \log \delta^{-1}$ , and  $\log \frac{\Lambda}{\rho}$ . We do not wish for any polynomial dependencies on  $n$ , either in the approximation ratio or in the additive error.
We recall the algorithm description from Section 4. We also include the pseudocode here, as Algorithm 2.
We now focus on analyzing the privacy and accuracy of the algorithm. Formally, in this section we prove the following.
Theorem B.1. For any  $0 < \varepsilon, \delta < \frac{1}{2}$  and  $\rho \leq \frac{\Lambda}{2}$ , there exists an  $(\varepsilon, \delta, \rho)$ -dist- $DP$ $(\alpha, \beta, V)$ -bicriteria approximation for  $k$ -median, with  $\alpha = O\left(\log n \cdot (\log n + \log \frac{\Lambda}{\rho})\right)$ ,  $\beta = O(d^{3/2})$ , and  $V = kd^2/\varepsilon^2 \cdot \text{poly} \log (n, d, \varepsilon^{-1}, \delta^{-1})$ .
Likewise, there exists an  $(\varepsilon, \delta, \rho)$ -dist-  $DP$ $(\alpha, \beta, V)$ -bicriteria approximation for  $k$ -means, with  $\alpha = O\left(\log n \cdot (\log n + \log \frac{\Lambda}{\rho})\right)$ ,  $\beta = O(d^3)$ , and  $V = kd^4 / \varepsilon^3 \cdot \text{poly} \log (n, d, \varepsilon^{-1}, \delta^{-1})$ .
Analysis of Privacy: The  $\tilde{x}_i$  points will be  $(\varepsilon, \delta, \rho)$ -dist-DP, by the Gaussian Mechanism (Proposition A.1). The levels above  $\rho \cdot A$  are strictly determined by  $\tilde{x}_i$ . For each level of grid length between  $\rho \cdot A$  and  $\rho / B$ , changing one data point changes at most 2 grid cells each by 1, which implies  $(2\varepsilon', 2\delta')$ -DP. In addition, this happens over  $O(\log(A \cdot B))$  levels and  $O(\log n)$  repetitions for each. By applying the advanced composition theorem (Theorem A.3, we have that as long as  $\varepsilon, \delta < 1$ , for our choices of  $\varepsilon', \delta'$ , the composition is  $(O(\varepsilon), O(\delta))$ -DP.
In total, the algorithm is  $(O(\varepsilon), O(\delta), \rho)$ -dist-DP.
Analysis of Accuracy: Let  $X = \{x_{1},\ldots ,x_{n}\}$  be our original set of points, and let  $C = \{c_1,\dots ,c_k\}$  be the optimal set of  $k$  centers. For any radius  $r$ , let  $n_r$  be the number of points  $x\in X$  such that  $d(x,C)\geq r$ . Then, it is well known that the  $k$ -means cost and  $k$ -median cost, up to an  $O(1)$ -
multiplicative factor, equal
$$
\sum_ {t \in \mathbb {Z}} 2 ^ {2 t} \cdot n _ {2 ^ {t}} \quad \text {a n d} \quad \sum_ {t \in \mathbb {Z}} 2 ^ {t} \cdot n _ {2 ^ {t}},
$$
respectively. For the set of centers  $F$  generated, we similarly define  $\hat{n}_r$  to be the number of points  $x\in X$  such that  $d(x,C)\geq r$
It is well-known that the magnitude of a  $d$ -dimensional Gaussian  $\mathcal{N}(0,I)$  is bounded by  $O(\sqrt{d + \log 1 / \beta})$  with failure probability  $\beta$ . Hence, we set  $A = O(\varepsilon^{-1}\sqrt{\log\delta^{-1}}\cdot d\sqrt{d + \log n})$ , so that with high probability,  $\| \tilde{x}_i - x_i\|_2 \leq \rho \cdot A / (40d)$  for all  $i$ . Now, for any  $r \geq \rho \cdot A / (40d)$ , if there exist  $k$  balls of radius  $r$  that contain all but  $n_r$  of the points in  $X$ , then there exist  $k$  balls of radius  $2r$  that contain all but  $n_r$  of the points in  $\tilde{X}$ . In addition, given a randomly shifted grid of dimension  $40r \cdot d$ , a ball of radius  $2r$ , in expectation, is split into at most 2 pieces, by Proposition A.8. Therefore, by Markov's inequality, the  $k$  balls of radius  $2r$  are split into at most  $4k$  cells with at least  $50\%$  probability, which means that the top  $4k$  cells at grid level  $40rd$  contain all but at most  $n_r$  points. Hence, because the center of the  $40r \cdot d$ -side length grid has radius  $20rd^{3/2}$ , this means  $\hat{n}_{20rd^{3/2}} \leq n_r$  for all  $r \geq \rho \cdot A / (40d)$  with at least  $50\%$  probability: repeating this  $O(\log n)$  times, this holds with at least  $1 - n^{-5}$  probability, even across all levels.
Next, suppose that  $r \leq \rho \cdot A / (40d)$ . In this case, if we didn't add noise we would have  $\hat{n}_{20rd^{3/2}} \leq n_r$  as in the previous case. This time, however, we add noise to the count of each cell rather than the number of points. In addition, we may not include a cell if its noisy count is at most  $\frac{K}{\varepsilon'} \cdot \log \frac{1}{\delta'}$ , but note that this means its true count is at most  $\frac{2K}{\varepsilon'} \cdot \log \frac{1}{\delta'}$ . Therefore, since the count of each cell is altered by  $O\left(\frac{1}{\varepsilon'} \log \frac{1}{\delta'}\right)$ , we have that  $\hat{n}_{20rd^{3/2}} \leq n_r + O\left(\frac{k}{\varepsilon'} \cdot \log \frac{1}{\delta'}\right)$  for all  $r \leq \rho \cdot \frac{A}{40d}$ .
In summary, we have that  $\hat{n}_{20rd^{3/2}} \leq n_r$  for  $r \geq \rho \cdot O(\varepsilon^{-1}\sqrt{\log\delta^{-1}} \cdot \sqrt{d + \log n})$ . In addition, for  $r \leq \rho \cdot O(\varepsilon^{-1}\sqrt{\log\delta^{-1}} \cdot \sqrt{d + \log n})$ , we have that  $\hat{n}_{20rd^{3/2}} \leq n_r + O(k / \varepsilon' \cdot \log 1 / \delta')$ . If we set  $B = n$ , then below  $r = \rho \cdot \sqrt{d} / n$  we have  $\hat{n}_r \leq n$  by default and above  $r = \rho \cdot \sqrt{d} / n$  the above bounds hold. This implies that
$$
\begin{array}{l} \sum_ {t \in \mathbb {Z}} 2 ^ {t} \hat {n} _ {2 ^ {t}} \leq O (d ^ {3 / 2}) \cdot \left(\sum_ {t \in \mathbb {Z}} 2 ^ {t} n _ {2 ^ {t}}\right) + O (\rho) \cdot d ^ {3 / 2} \cdot \sum_ {t \in \mathbb {Z}: 2 ^ {t} \leq \varepsilon^ {- 1} \sqrt {\log \delta^ {- 1}} \cdot \sqrt {d + \log n}} 2 ^ {t} \cdot O \left(\frac {k}{\varepsilon^ {\prime}} \cdot \log \frac {1}{\delta^ {\prime}}\right) + O \left(\rho \cdot \frac {\sqrt {d}}{n}\right) \cdot n \\ = O (d ^ {3 / 2}) \cdot \mathrm {O P T} _ {k} (X) + O \left(\frac {k d ^ {2}}{\varepsilon^ {2}}\right) \cdot \mathrm {p o l y} \log (n, d, \varepsilon^ {- 1}, \delta^ {- 1}) \cdot \rho . \\ \end{array}
$$
Hence, we obtain a bicriteria with multiplicative approximation  $\beta = O(d^{3/2})$  and additive error  $k\sqrt{d} \cdot \mathrm{poly}(\varepsilon^{-1}, \log \delta^{-1}, \log n) \cdot \rho$ , for  $k$ -median. The same calculation for  $k$ -means will give us a multiplicative approximation  $\beta = O(d^3)$  and additive error  $kd^4/\varepsilon^3 \cdot \mathrm{poly}(\log (n, d, \varepsilon^{-1}, \delta^{-1}) \cdot \rho^2$ .
Finally, the number of centers we output is simple to compute. We have  $O(\log n)$  repetitions, and each repetition has  $O\left(\log \frac{\Lambda}{\rho / n}\right)$  levels, each of which we select at most  $4k$  cell centers from. Hence, we select  $O\left(k \cdot \log n \cdot (\log n + \log \frac{\Lambda}{\rho})\right)$  points, meaning that  $\alpha = O\left(\log n \cdot (\log n + \log \frac{\Lambda}{\rho})\right)$ .
C From Crude to Accurate
In this section, we devise an improved approximation that only uses  $k$  centers and achieves a constant approximation ratio. We will subsequently prove Theorem 1.1.
Our approach utilizes both the crude approximation from Section 4/Section B and previously known constant-approximation differentially private (but not dist-DP) algorithms from the literature. We show how to combine these to create a dist-DP semi-coreset. This idea is partially inspired by the work of [22] for (non-private) coreset constructions and more recently [25] for fast private (semi-)coreset constructions. More accurately, given a set of  $n$  points  $X = \{x_{1},\ldots ,x_{n}\} \in \mathbb{R}^{d}$ , we will compute a (weighted) set of points  $Y$  that is  $(\varepsilon ,\delta ,\rho)$ -dist-DP with respect to  $X$ , such that for any set of  $k$  centers  $C = \{c_1,\dots ,c_k\}$ ,  $\mathrm{cost}(Y;C) = \Theta (\mathrm{cost}(X;C))\pm O(\mathrm{OPT}_k(X))\pm W\cdot \rho^p$ , where  $W$  will be polynomial in  $d,k,\varepsilon^{-1},\log \delta^{-1},\log n$ , and  $\log \frac{\Lambda}{\rho}$ .
If we can achieve this, then we just have to compute an  $O(1)$ -approximate  $k$ -means (or  $k$ -median) solution to  $Y$ , which does not have to be private since  $Y$  already is. Indeed, if we do so, then the centers  $C$  that we find for  $Y$  satisfy
$$
\operatorname {c o s t} (Y; C) \leq O (1) \cdot \operatorname {c o s t} (Y; C ^ {*}) \leq O (1) \cdot \operatorname {c o s t} (X; C ^ {*}) + O (\operatorname {O P T} _ {k} (X)) + O (W) \cdot \rho^ {p}
$$
for any set of  $k$  centers  $C^*$ . Hence, this implies that
$$
\operatorname {c o s t} (Y; C) \leq O (1) \cdot \operatorname {O P T} _ {k} (X) + O (W) \cdot \rho^ {p}.
$$
Finally, we also have that  $\mathrm{cost}(Y;C)\geq \Omega (1)\cdot \mathrm{cost}(X;C) - O(\mathrm{OPT}_k(X)) - W\cdot \rho^p$  , which means  $\mathrm{cost}(X;C)\leq O(1)\cdot \mathrm{cost}(Y;C) + O(\mathrm{OPT}_k(X)) + O(W)\cdot \rho^p$  , so as desired, we have
$$
\operatorname {c o s t} (X; C) \leq O (1) \cdot \operatorname {O P T} _ {k} (X) + O (W) \cdot \rho^ {p}.
$$
Hence, it suffices to prove the following theorem.
Theorem C.1. For any  $0 < \varepsilon, \delta < \frac{1}{2}$  and  $\rho \leq \frac{\Lambda}{2}$ , there exists an  $(\varepsilon, \delta, \rho)$ -dist-  $DP$  ( $O(1), W, \rho$ )-semicoreset for  $k$ -means (resp.,  $k$ -median), with  $W = O\left(\frac{k^2d^{4.5}}{\varepsilon^3}\right) \cdot \text{poly} \log \left(n, d, \frac{1}{\varepsilon}, \frac{1}{\delta}, \frac{\Lambda}{\rho}\right)$  for  $k$ -means and  $W = O\left(\frac{k^2d^{2.5}}{\varepsilon^2}\right) \cdot \text{poly} \log \left(n, d, \frac{1}{\varepsilon}, \frac{1}{\delta}, \frac{\Lambda}{\rho}\right)$  for  $k$ -median.
We assume we have a private  $(\alpha, \beta, V)$ -bicriteria approximation  $F$ . Recall this means we have an  $(\varepsilon, \delta, \rho)$ -dist-DP set of (at most)  $\alpha \cdot k$  centers  $F$  such that  $\mathrm{cost}(X; F) \leq \beta \cdot \mathrm{OPT}_k(X) + V \cdot \rho^p$ .
We recall the algorithm description and pseudocode (Algorithm 1) from Section 5. Hence, for the remainder of this section we focus on proving Theorem C.1. In addition, after proving Theorem C.1, we briefly discuss the runtime and how to make the runtime close to linear, and parallelizable.
Analysis of Privacy: We will think of the algorithm as having 3 adaptive components. First, we must create  $F$ , which is  $(\varepsilon, \delta, \rho)$ -dist-DP, by Theorem B.1. In addition, we create  $\{\tilde{x}_i\}$ , which as a set is  $(\varepsilon, \delta, \rho)$ -dist-DP, by Proposition A.1. Note that  $\tilde{X}_0$ , the sets  $I_j$ , and the sizes  $\hat{n}_j$  are also only dependent on  $F$  and  $\{\tilde{x}_i\}$ . So, for each  $j$  with  $\hat{n}_j < T$ , the algorithm's creation of  $\tilde{X}_j$  is only depends on  $F$  and  $\{\tilde{x}_i\}$ . Finally, we must compute  $\tilde{X}_j$  for each  $j$  such that  $\hat{n}_j \geq T$ . However, each coreset is  $(\varepsilon, \delta)$ -DP which also implies  $(\varepsilon, \delta, \rho)$ -dist-DP, and we are computing the coresets on disjoint subsets of indices, which are fixed. So overall, computing all of the  $\tilde{X}_j$  is  $(\varepsilon, \delta, \rho)$ -dist-DP if we fix  $F$  and each  $\tilde{x}_i$ .
By basic adaptive composition (Theorem A.2), the overall procedure is  $(3\varepsilon, 3\delta, \rho)$ -dist-DP.
Analysis of Accuracy: We focus on accuracy for  $k$ -means; the proof for  $k$ -median is extremely similar.
First, note that  $d(x_{i},\tilde{x}_{i}) \leq O\left(\frac{\rho\sqrt{\log(1 / \delta)}}{\varepsilon} \cdot \sqrt{d + \log n}\right)$  for all  $i$ . Also, note that for general positive reals  $A,B$ ,  $(A\pm B)^2 = A^2\pm 2AB + B^2$ , and  $2AB \leq \gamma A^{2} + \frac{1}{\gamma} B^{2}$  for any positive  $\gamma$ . This means that for any  $0 < \gamma < 1$ ,  $(A + B)^{2} = (1\pm \gamma)A^{2}\pm O\left(\frac{1}{\gamma}\right)B^{2}$ , and  $(A - B)^{2} = (1\pm \gamma)A^{2}\pm O\left(\frac{1}{\gamma}\right)B^{2}$ . Hence,
$$
\begin{array}{l} d (\tilde {x} _ {i}, C) ^ {2} = \left(d (x _ {i}, C) \pm O \left(\frac {\sqrt {\log (1 / \delta)}}{\varepsilon} \cdot \sqrt {d + \log n}\right) \cdot \rho\right) ^ {2} \\ = (1 \pm \gamma) \cdot d \left(x _ {i}, C\right) ^ {2} \pm O \left(\left(d + \log n\right) \cdot \frac {\log (1 / \delta)}{\varepsilon^ {2}} \cdot \frac {1}{\gamma}\right) \cdot \rho^ {2}. \tag {1} \\ \end{array}
$$
Therefore, for any set  $C$  of size at most  $k$ ,
$$
\sum_ {i \in I _ {0}} d (\tilde {x} _ {i}, C) ^ {2} = (1 \pm \gamma) \cdot \sum_ {i \in I _ {0}} d (x _ {i}, C) ^ {2} \pm O \left((d + \log n) \cdot \frac {\log (1 / \delta)}{\varepsilon^ {2}} \cdot \frac {1}{\gamma}\right) \cdot \rho^ {2} \cdot | I _ {0} |,
$$
where we recall that  $i \in I_0$  if and only if  $d(\tilde{x}_i, F) > S \cdot \rho$ . For  $S \geq \Omega\left(\frac{\sqrt{\log(1 / \delta)}}{\varepsilon} \cdot \sqrt{d + \log n}\right)$ , this implies that  $d(x_i, F) \geq \frac{S}{2} \cdot \rho$ . However, if  $d(\tilde{x}_i, F) > S \cdot \rho$ , then  $d(x_i, F) \geq \frac{S}{2} \cdot \rho$ , and the
number of such  $i$  with  $d(x_{i},F)\geq \frac{S}{2}\cdot \rho$  is at most  $\frac{\mathrm{cost}(X;F)}{(S / 2)^2\rho^2}\leq \frac{4\beta\cdot\mathrm{OPT}_k(X) + 4V\cdot\rho^2}{S^2\cdot\rho^2}$ . Hence, apart from the  $1\pm \gamma$  multiplicative error, we also incur an additional additive error of
$$
O \left((d + \log n) \cdot \frac {\log (1 / \delta)}{\varepsilon^ {2}} \cdot \frac {1}{\gamma}\right) \cdot \frac {\beta}{S ^ {2}} \cdot \mathrm {O P T} _ {k} (X) + O \left((d + \log n) \cdot \frac {\log (1 / \delta)}{\varepsilon^ {2}} \cdot \frac {1}{\gamma}\right) \cdot \frac {V}{S ^ {2}} \cdot \rho^ {2}.
$$
So, by setting  $S = O\left(\frac{1}{\varepsilon \cdot \gamma} \cdot \sqrt{(d + \log n) \cdot \log(1 / \delta) \cdot \beta}\right)$ , we obtain an additional  $1 \pm O(\gamma)$  multiplicative error and an additive error of  $O\left(\frac{\gamma}{\beta} \cdot V \cdot \rho^2\right)$ . This deals with the error from points sent to  $\tilde{X}_0$ . To summarize, we have
$$
\sum_ {i \in I _ {0}} d \left(\tilde {x} _ {i}, C\right) ^ {2} = (1 \pm O (\gamma)) \cdot \sum_ {i \in I _ {0}} d \left(x _ {i}, C\right) ^ {2} \pm O \left(\frac {\gamma}{\beta} \cdot V \cdot \rho^ {2}\right) \tag {2}
$$
for all sets  $C$  of size at most  $k$ .
Next, we deal with points in  $\hat{X}_j$  with  $\hat{n}_j < T$ . In this case, we still have that (1) holds, which means for any such  $j$  and any subset  $C$  of  $k$  points,
$$
\sum_ {i \in I _ {j}} d (\tilde {x} _ {i}, C) ^ {2} = (1 \pm \gamma) \cdot \sum_ {i \in I _ {j}} d (x _ {i}, C) ^ {2} \pm O \left((d + \log n) \cdot \frac {\log (1 / \delta)}{\varepsilon^ {2}} \cdot \frac {1}{\gamma} \cdot T\right) \cdot \rho^ {2}, \tag {3}
$$
since  $|I_j| = \hat{n}_j < T$ .
Finally, we deal with the rest of the points, for which we use a regular differentially private semicoreset algorithm on each  $\hat{X}_j$ . Recall that we choose  $S$  so that  $\| \tilde{x}_i - x_i\|_2 \leq S \cdot \rho$  for all  $i$ , which means every point  $x_i$  for  $i \in I_j$  is in  $B(f_j, 2S\rho)$ , i.e., the ball of radius  $2S\rho$  of  $f_j$ . We apply the private semi-coreset algorithm from Lemma A.10 with respect to the larger ball  $B(f_j, \rho \cdot S / \gamma)$ . This means that for any subset  $C$  of size at most  $k$  in  $\mathbb{R}^d$ ,
$$
\cos \left(\tilde {X} _ {j}; C\right) = \Theta (1) \cdot \cos \left(\hat {X} _ {j}; C\right) \pm O (1) \cdot \operatorname {O P T} _ {k} \left(\hat {X} _ {j}\right) \pm U \cdot \left(\frac {2 S \rho}{\gamma}\right) ^ {2}, \tag {4}
$$
where  $U = O\left(\frac{k\log^2n\log(1 / \delta) + k\sqrt{d\log(1 / \delta)}}{\varepsilon}\right)$ . We emphasize that Lemma A.10 holds even with respect to a center set  $C$  that is not contained in the ball  $B(f_j,\rho \cdot S / \gamma)$ .
We now combine Equations (2), (3), and (4), setting  $\gamma$  to be a fixed small constant. Since  $\tilde{X}$  is the aggregation of all  $\tilde{X}_j$ 's for  $j = 0,1,\ldots ,\alpha \cdot k$ , and recalling that  $\hat{X}_j = \{x_i:i\in I_j\}$ , we have that
$$
\begin{array}{l} \operatorname {c o s t} (\tilde {X}; C) \\ = \sum_ {j = 0} ^ {\alpha \cdot k} \operatorname {c o s t} \left(\tilde {X} _ {j}; C\right) \\ = \Theta (1) \cdot \sum_ {j = 0} ^ {\alpha \cdot k} \mathrm {c o s t} (\hat {X} _ {j}; C) \pm O (1) \cdot \sum_ {j = 1} ^ {\alpha \cdot k} \mathrm {O P T} _ {k} (\hat {X} _ {j}) \pm O \left(\frac {V}{\beta} + \alpha k \cdot (d + \log n) \cdot \frac {\log (1 / \delta)}{\varepsilon^ {2}} \cdot T + \alpha k \cdot U \cdot S ^ {2}\right) \cdot \rho^ {2} \\ = \Theta (1) \cdot \operatorname {c o s t} (X; C) \pm O (1) \cdot \mathrm {O P T} _ {k} (X) \pm O \left(\frac {k ^ {2} d ^ {4 . 5}}{\varepsilon^ {3}}\right) \cdot \operatorname {p o l y} \log \left(n, d, \frac {1}{\varepsilon}, \frac {1}{\delta}, \frac {\Lambda}{\rho}\right) \cdot \rho^ {2}. \\ \end{array}
$$
The second line is true by combining the equations and setting  $\gamma$  to be a small constant. The third line is true since  $\hat{X}_j$  forms a partition of  $X$ , and by our parameter settings of  $\alpha, \beta, S, T, U, V$ .
This completes the proof of Theorem C.1, in the  $k$ -means case. The  $k$ -median case follows the same analysis (though we will set  $S = O\left(\frac{1}{\varepsilon \cdot \gamma} \cdot \sqrt{(d + \log n) \cdot \log(1 / \delta)} \cdot \beta\right)$  in this case), and will result in the additive term of  $O\left(\frac{V}{\beta} + \alpha k \cdot \sqrt{(d + \log n) \cdot \frac{\log(1 / \delta)}{\varepsilon^2}} \cdot T + \alpha k \cdot U \cdot S\right) \cdot \rho = O\left(\frac{k^2 d^{2.5}}{\varepsilon^2}\right) \cdot \text{poly} \log \left(n, d, \frac{1}{\varepsilon}, \frac{1}{\delta}, \frac{\Lambda}{\rho}\right) \cdot \rho$ . By combining this with our discussion at the beginning of this section, we have also proven Theorem 1.1.
Runtime: Finally, we note that this algorithm can be implemented efficiently. Indeed, in Algorithm 2, creating the points  $\tilde{x}_i$ , creating each quadtree data structure, computing the counts (only for the  $\tilde{x}_i$  and  $x_i$  points), and adding laplace noise all takes  $\tilde{O}(nd)$  time, where we also hide logarithmic factors in  $\frac{\Lambda}{\rho}$ . Finally, picking the  $k$  heaviest cells in each grid also takes  $\tilde{O}(n)$  time.
In Algorithm 1, there are two potential bottlenecks. The first is mapping each point  $\tilde{x}_i$  to its closest center  $f_j$ , which takes  $O(nd \cdot |F|) = \tilde{O}(ndk)$  time, hiding logarithmic factors in  $\frac{\Lambda}{\rho}$ . We additionally have to compute a private semi-coreset for the points in each  $\alpha \cdot k$  sets of points  $\hat{X}_j$ . However, using the private algorithm of [25, Theorem C.2], computing an  $O(1)$ -approximate private semicoreset can be done in time  $\tilde{O}(\hat{n}_jd) + \mathrm{poly}(k) \cdot d$ . Note that  $\hat{n}_j = |\hat{X}_j|$  and  $\sum \hat{n}_j = n$ . Hence, because  $\alpha = O\left(\log \frac{\Lambda}{\rho}\right)$ , the overall algorithm, apart from the assignment of each  $x_i$  to  $\hat{X}_j$ , takes  $\tilde{O}(nd) + \mathrm{poly}(k) \cdot d$  time, hiding logarithmic factors in  $\frac{\Lambda}{\rho}$ .
To improve the  $nkd$  to  $nd$ , we may use a  $K = O(\log n)$ -approximate nearest neighbor data structure to map each  $\tilde{x}_i$  to its  $K$ -approximate nearest neighbor  $f_j \in F$ . By using the locality-sensitive hashing algorithm of [4], we can compute every  $K$ -approximate nearest neighbor of each  $\tilde{x}_i$  in  $\tilde{O}(nd)$  time instead. We remark that the privacy analysis will be unchanged, and the accuracy analysis will be similar, up to getting a slightly worse additive approximation. Namely, if the points  $\hat{X}_j$  were previously within  $O(S)$  of the center  $f_j$ , they may now have distance  $O(S \cdot K)$ . Hence, the semi-coreset computation will have to be done with respect to a ball of radius  $O(\rho \cdot S \cdot K / \gamma)$ , but for  $K = O(\log n)$  and  $\gamma$  a constant, this doesn't affect the additive error by more than an  $O(\log^2 n)$  factor.
Hence, we can compute a private semi-coreset in  $\tilde{O}(nd) + \mathrm{poly}(k) \cdot d$  time. Finally, we need to compute an offline (non-private)  $k$ -means (or  $k$ -median) approximation. As this is not related to private clustering, we simply sketch how this can be done.