SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Let  $G = \{g^{(1)},\ldots ,g^{(k)}\}$  be a  $\beta$ -approximate solution to the statistical Euclidean  $(k,p)$ -clustering problem on  $(\pi (\mathcal{X}),\mathbb{P}_{\pi})$ . We would like to argue that the partition of  $\mathcal{X}$  induced by  $G$  is a  $(1 + \varepsilon)\beta$ -approximate partition with probability at least  $1 - \delta$ . To do so, we first approximate  $\mathcal{X}$  with a finite set  $\tilde{\mathcal{X}}$  in a replicable fashion while preserving partition costs. This allows us to apply Proposition E.5.
Recall we write  $\Lambda$  to denote a replicable estimate of OPT such that  $\Lambda \in [\mathrm{OPT} / \beta (1 + \varepsilon),(1 + \varepsilon)$  OPT] (cf. Theorem D.23). Consider a grid of length  $\varepsilon \Lambda /(4p\Delta)$ . For a point  $x\in \mathcal{X}$ , we assign it to the center of the cell that it belongs to, say  $\tilde{x}$ . Let  $\tilde{\mathcal{X}}\coloneqq \{\tilde{x}:x\in \mathcal{X}\}$  denote this finite set and  $\tilde{\mathbb{P}}$  the distribution induced by the discretization. Remark that  $\kappa (x,\tilde{x})\leq \varepsilon \Lambda /(4p)$  for every  $x\in \mathcal{X}$ .
Note that a partition of  $\mathcal{X}$  induces a partition of  $\tilde{\mathcal{X}}$  and vice versa. We proceed without making any distinction between the two.
Proposition E.7. For any point  $u\in \mathcal{B}_d$
$$
| \kappa (x, u) ^ {p} - \kappa (\tilde {x}, u) ^ {p} | \leq p | \kappa (x, u) - \kappa (\tilde {x}, u) |.
$$
Proof. By the mean value theorem, the function  $g:[0,1]\to \mathbb{R}$  given by  $z\mapsto z^p$  is  $p$ -Lipschitz:
$$
\begin{array}{l} | g (y) - g (z) | \leq \sup  _ {\xi \in [ 0, 1 ]} g ^ {\prime} (\xi) | y - z | \\ \leq p | y - z |. \\ \end{array}
$$
We write  $\widetilde{\mathrm{cost}} (\mathcal{C})$  to denote the cost of the partition  $\mathcal{C}$  of  $\tilde{\mathcal{X}}$ .
Proposition E.8. For any partition  $\mathcal{C}$  of  $\mathcal{X}$
$$
\frac {1}{1 + \varepsilon} \operatorname {c o s t} (\mathcal {C}) \leq \widetilde {\operatorname {c o s t}} (\mathcal {C}) \leq (1 + \varepsilon) \operatorname {c o s t} (\mathcal {C}).
$$
$$
\begin{array}{l} \left| \mathbb {E} _ {x} \left[ \sum_ {j = 1} ^ {k} \mathbb {1} \{x \in C ^ {(j)} \} \cdot \kappa (x, u ^ {(j)}) ^ {p} \right] - \mathbb {E} _ {x} \left[ \sum_ {j = 1} ^ {k} \mathbb {1} \{x \in C ^ {(j)} \} \cdot \kappa (\tilde {x}, u ^ {(j)}) ^ {p} \right] \right| \\ \leq \mathbb {E} _ {x} \left[ \sum_ {j = 1} ^ {k} \mathbb {1} \{x \in C ^ {(j)} \} \cdot | \kappa (x, u ^ {(j)}) ^ {p} - \kappa (\tilde {x}, u ^ {(j)}) ^ {p} | \right] \\ \leq \mathbb {E} _ {x} \left[ \sum_ {j = 1} ^ {k} \mathbb {1} \{x \in C ^ {(j)} \} \cdot p | \kappa (x, u ^ {(j)}) - \kappa (\tilde {x}, u ^ {(j)}) | \right] \\ \leq p \cdot \frac {\varepsilon \Lambda}{4 p} \\ \leq \frac {\varepsilon}{2} \mathrm {O P T}. \\ \end{array}
$$
Corollary E.9. Fix  $\varepsilon \in (0,1/4), \delta \in (0,1)$  and suppose  $\pi: \mathbb{R}^d \to \mathbb{R}^m$  is a standard random dimension-reduction map with
$$
d = O \left(\frac {p ^ {4}}{\varepsilon^ {2}} \log \frac {k}{\varepsilon \delta}\right).
$$
$$
\frac {1}{(1 + \varepsilon)} \mathrm {O P T} \leq \widetilde {\mathrm {O P T}} _ {\pi} \leq (1 + \varepsilon) \mathrm {O P T},
$$
with probability at least  $1 - \delta$ , where the probability is taken w.r.t. the choice of the random map.
Proof. The optimal clustering cost coincides with the cost of the partition induced by the optimal centers. An application of Proposition E.8 yields the desired result.  $\square$
Theorem E.10. Fix  $\varepsilon \in (0,1/4)$ ,  $\delta \in (0,1)$  and suppose  $\pi \in \pi_{d,m}$  is a standard random dimension-reduction map with
Let  $G = \{g^{(1)},\ldots ,g^{(k)}\}$  be a  $\beta$ -approximate solution to the statistical Euclidean  $(k,p)$ -clustering problem on  $(\pi (\tilde{\mathcal{X}}),\tilde{\mathbb{P}}_{\pi})$ . Then the partition of  $\mathcal{X}$  induced by  $G$  is a  $(1 + \varepsilon)\beta$ -approximate partition with probability at least  $1 - \delta$ .
That is, for  $\mathcal{C} = \{C^{(1)},\dots ,C^{(k)}\}$  given by
$$
C ^ {(j)} := \left\{x \in \mathcal {X}: g ^ {(j)} = \operatorname {a r g m i n} _ {g \in G} \kappa (\pi (\tilde {x}), g ^ {(j)}) ^ {p} \right\},
$$
$$
\operatorname {c o s t} (\mathcal {C}) \leq (1 + \varepsilon) \beta \operatorname {O P T}.
$$
$$
\begin{array}{l} \operatorname {c o s t} (\mathcal {C}) \leq (1 + \varepsilon) \widetilde {\operatorname {c o s t}} (\mathcal {C}) \quad \text {P r o p o s i t i o n E . 8} \\ \leq (1 + \varepsilon) ^ {2} \widetilde {\operatorname {c o s t}} (\pi (\mathcal {C})) \quad \text {P r o p o s i t i o n E . 5} \\ \leq (1 + \varepsilon) ^ {2} \widetilde {\operatorname {c o s t}} (G) \quad G \text {i n d u c e s} \pi (\mathcal {C}) \\ \leq (1 + \varepsilon) ^ {2} \widehat {\beta \mathrm {O P T}} _ {\pi} \quad G \text {i s} \beta \text {- a p p r o x i m a t e} \\ \leq (1 + \varepsilon) ^ {3} \beta \mathrm {O P T}. \quad \text {C o r o l l a r y E . 9} \\ \end{array}
$$
E.4 Summary of Dimensionality Reduction
Before proving Theorem 5.1, let us summarize the stages of our pipeline.
1) Scale the input data by  $\frac{1}{\sqrt{a}}$ .
2) Estimate OPT of the scaled data within a multiplicative error.
3) Discretize the scaled data with a fine grid.
4) Project the scaled data with a random map  $\pi \in \pi_{d,m}$ .
5) Compute a coreset on the projected data.
6) Estimate the probability mass at each point of the coreset.
7) Call the  $\beta$ -approximation oracle to produce  $k$  centers  $g^{(1)}, \ldots, g^{(k)}$ .
8) Output the clustering function induced by the centers.
The clustering function proceeds as follows. Given a point  $x\in \mathcal{X}$
1) Scale  $x$  by  $\sqrt[1]{a}$ .
2) Discretize the scaled point with the same grid used in the algorithm.
3) Project the discretized point using the same  $\pi$  used in the algorithm.
4) Output the label of the closest center  $g^{(j)}$  in the lower dimension for  $j \in [k]$ .
We are now ready to prove Theorem 5.1, which we restate below for convenience.
Proof (Theorem 5.1). We randomly draw some  $\pi \in \pi_{d,m}$  for
Then, we discretize a sufficiently large sample after scaling and push it through  $\pi$ . We then run the corresponding algorithm (cf. Theorem 4.2, Theorem 4.3) to produce some centers  $G$  for the compressed data. This allows us to partition  $\mathcal{X}$  according to the closest member of  $G$  in the lower dimension. An application of Theorem E.10 yields the performance guarantees of this algorithm.
The sample complexity guarantee from Theorem 4.2, Theorem 4.3 is for the scaled distribution and the scaled optimal cost satisfies  $\mathrm{OPT}_{\pi}^{\prime} = d^{-p / 2}\mathrm{OPT}_{\pi}$ . Hence the sample complexity resolves to
$$
\tilde {O} \left(\mathrm {p o l y} \left(\frac {k d}{\rho \mathrm {O P T}}\right) \left(\frac {2 \sqrt {m}}{\varepsilon}\right) ^ {O (m)} \log \frac {1}{\delta}\right)
$$
We conclude the proof by remarking that for the Euclidean metric,  $\Delta = \sqrt{d}$  and that  $O(\log n)^{O(\log n)} = n^{O(\log \log n)}$ .
F  $k$ -Centers
In this section, we will explain our approach to solving the statistical  $k$ -centers problem (cf. Problem 2.3). As we explained before, since this problem has a different flavor compared to  $(k,p)$ -clustering, we introduce some extra assumptions in order to be able to solve it from samples. To the best of our knowledge, this is the first work that considers a model of statistical flavor for the  $k$ -centers problem.
F.1 Assumptions
First, we describe an assumption that is necessary to obtain good clustering solutions even if we do not insist on using replicable algorithms.
Assumption F.1 (Clusterable). For some absolute constants  $\beta, B > 0$ , an i.i.d. sample  $S = x_1, \ldots, x_n$  of size  $n$  from  $\mathbb{P}$  has the following property. With probability at least  $q$ , there exists some  $(\beta, B)$ -approximate solution  $F_*$  to Problem 2.3 such that, for every  $f \in F_*$  there is some  $x_f \in S$  with  $F_*(x_f) = f$ . Here  $q \in [0,1]$ .
Assumption F.1 is necessary to obtain good clustering solutions even in the non-replicable case. Our distribution  $\mathbb{P}$  must be sufficiently well-behaved so that solving the sample  $k$ -centers problem translates to a good solution in the population case. Essentially, this assumption states that with high probability, the sample will contain a point from every single cluster of some  $(\beta, B)$ -approximate solution. In order to design replicable algorithms, we need to make a stronger assumption.
Assumption F.2 (Replicable). For some absolute constants  $\beta, B$ , there exists a  $(\beta, B)$ -approximate solution  $F_{*}$  to Problem 2.3 with the following property. With probability at least  $q$ , in an i.i.d. sample  $S = x_{1}, \ldots, x_{n}$  of size  $n$  from  $\mathbb{P}$ , for every  $f \in F_{*}$ , there is some  $x_{f} \in S$  with  $F_{*}(x_{f}) = f$ . Here  $q \in [0,1]$ .
Note that Assumption F.1 guarantees that, with high probability, every sample will help us identify some good solution. However, these good solutions could be vastly different when we observe different samples, thus this assumption is not sufficient to design a replicable algorithm with high utility. Assumption F.2 is necessary to attain replicable solutions. Indeed, this assumption essentially states that there is a fixed solution for which we will observe samples from every cluster with high probability. If this does not hold and we observe points from clusters of solutions that vary significantly, we cannot hope to replicably recover a fixed approximate solution. Note that Assumption F.2 is stronger than Assumption F.1, since we have flipped the order of the existential quantifiers. One way to satisfy this assumption is to require that there is some  $(\beta, B)$ -approximate solution so that the total mass that is assigned to every cluster of it is at least  $1/n$  for some  $n \in \mathbb{N}$ . Then, by observing  $\widetilde{O}(n)$  points, we will receive at least one from every cluster with high probability. This is made formal in Proposition F.3.
Proposition F.3 ([Janson, 2018]). Let  $\delta \in (0,1)$ . Under Assumption F.2, a sample of
$$
O \left(\frac {n m}{q} \log \frac {1}{\delta}\right)
$$
points contains at least  $m$  points from each cluster of  $F_{*}$  with probability at least  $1 - \delta$ .
Proof (Proposition F.3). Let  $Y_{i}$  be a geometric variable with success rate  $q$ , which denotes the number of batches of  $n$  samples we draw until a point from each cluster of  $F_{*}$  is observed. Then  $Y := \sum_{i=1}^{m} Y_{i}$  upper bounds the number of batches until we obtain  $m$  points from each cluster.
As shown in Janson [2018],
$$
\mathbb {P} \{Y \geq \lambda \mathbb {E} Y \} \leq \exp (1 - \lambda)
$$
$$
\mathbb {E} Y = \frac {m}{q}.
$$
The result follows by picking  $\lambda = 1 + \log (1 / \delta)$
F.2 High-Level Overview of the Approach
We are now ready to provide a high-level overview of our approach to derive Theorem 3.3. We first take a fixed grid of side  $c$  in order to cover the unit ball. Then, we sample sufficiently many points from  $\mathbb{P}$ . Subsequently, we round all the points of the sample to the centers of the cells of the grid that they fall into. Using these points, we empirically estimate the density of the distribution on every cell of the grid. Next, in order to ensure that our solution is replicable, we take a random threshold from a predefined interval and discard the points from all the cells whose density falls below the threshold. Finally, we call the approximation oracle that we have access to using the points that have survived after the previous step. We remark that unlike the  $(k,p)$ -clustering problem (cf. Problem 2.1), to the best of our knowledge, there does not exist any dimensionality reduction techniques that apply to the  $k$ -centers problem. In the following subsections we explain our approach in more detail.
F.3 Oracle on Sample
As we explained before, we require black-box access to an oracle  $\mathcal{O}$  for the combinatorial  $k$ -centers problem on a sample which outputs  $(\widehat{\beta},\widehat{B})$ -approximate solutions  $\widehat{F}$  for Problem 2.4. Thus, we need to show that, with high probability, the output of this solution is a good approximation to Problem 2.3. This is shown in Proposition F.4.
Proposition F.4. Suppose there is a  $(\beta, B)$ -approximate solution  $F_*$  for Problem 2.3 and that we are provided with a sample of size  $O(n \log(1/\delta)/q)$  containing an observation from each cluster of  $F_*$  with probability at least  $1 - \delta$ . Then, the output  $\widehat{F}$  of the oracle  $\mathcal{O}$  on the sample satisfies
$$
\kappa (x, \widehat {F}) \leq (2 \beta + \widehat {\beta}) \mathrm {O P T} + 2 B + \widehat {B}.
$$
for all  $x\in \mathcal{X}$ , with probability at least  $1 - \delta$
In the previous result  $(\beta, B), (\widehat{\beta}, \widehat{B})$  are the approximation parameters that we inherit due to the fact that we receive samples from  $\mathbb{P}$  and that we solve the combinatorial problem approximately using  $\mathcal{O}$ , respectively.
Proof (Proposition F.4). Let  $F_{*}$  denote the  $(\beta, B)$ -approximate solution from Assumption F.2. We condition on the event that such a solution exists in the sample, which occurs with probability at least  $1 - \delta$ .
Fix  $x \in \mathcal{X}$ , which is not necessarily observed in the sample. We pay a cost of  $\beta \mathrm{OPT} + B$  to travel to the nearest center of  $F_{*}$ . Next, we pay another  $\beta \mathrm{OPT} + B$  to travel to the observed sample from this center. Finally, we pay a cost of
$$
\begin{array}{l} \widehat {\operatorname {c o s t}} (\widehat {F}) \leq \widehat {\beta \mathrm {O P T}} + \widehat {B} \\ \leq \widehat {\beta} \mathrm {O P T} + \widehat {B}. \\ \end{array}
$$
to arrive at the closest center in  $\widehat{F}$ .
F.4 Oracle with Grid
The issue with using directly the output of  $\mathcal{O}$  on a given sample is that the result will not be replicable, since we have no control over its behavior. In a similar way as with the  $(k,p)$ -clustering problem, the main tool that we have in our disposal to "stabilize" the output of  $\mathcal{O}$  is to use a grid. Unlike the other problems, we cannot use the hierarchical grid approach in this one. The reason is that, due to the min-max nature of k-centers, we need to cover the whole space and not just the points where the distribution is dense. Recall that so far we have established that the oracle  $\mathcal{O}$  outputs  $(2\beta + \widehat{\beta}, 2B + 2\widehat{B})$ -approximate solutions  $F$  for Problem 2.3, given that Assumption F.2 holds. Algorithm F.1 "discretizes" the points of the sample using a grid with side-length  $c$  before applying the oracle. Proposition F.5 shows that by doing that, we have to pay an extra additive term that is of the order  $O(c\Delta)$ .
Algorithm F.1 Oracle with Grid
1: Oracle with Grid(oracle  $\mathcal{O}$ , sample  $x_{1},\ldots ,x_{n}$ , grid length  $c$ ):
2: for  $i\gets 1,\ldots ,n$  do
3: Decompose  $x_{i} = z_{i}\cdot c + x_{i}^{\prime}$  for some  $z_{i}\in \mathbb{Z}^{d},x_{i}^{\prime}\in [0,c)^{d}$
4:  $\tilde{x}_i\gets z_i\cdot c + (c / 2,c / 2,\ldots ,c / 2)\left\{\tilde{x}_i\right.$  is the center of the cell that contains  $x_{i}$
5: end for
6: Return  $\mathcal{O}(\tilde{x}_1, \ldots, \tilde{x}_n)$
Proposition F.5. Suppose there is a  $(\beta, B)$ -approximate solution  $F_*$  for Problem 2.3 and that we are provided with a sample of size  $n$  containing an observation from each cluster of  $F_*$ . Let  $\tilde{G}$  denote the output of Algorithm F.1 using a  $(\hat{\beta}, \hat{B})$ -approximation oracle  $\mathcal{O}$  for Problem 2.4. Then, for any  $x \in \mathcal{X}$ ,
$$
\kappa (x, \tilde {G}) \leq (2 \beta + \widehat {\beta}) \mathrm {O P T} + 2 B + \widehat {B} + (4 \beta + 2 \widehat {\beta} + 1) c \Delta .
$$
Proof (Proposition F.5). For the sake of analysis, imagine we used the grid to discretize all points of  $\mathcal{X} \mapsto \widetilde{\mathcal{X}}$ . Then  $F_{*}$  is a solution with cost  $\beta \widetilde{\mathrm{OPT}} + B + c\Delta$  and we observe a point from each of its clusters in our discretized sample. Here  $\widetilde{\mathrm{OPT}}$  is the cost of a solution to Problem 2.3 on  $\widetilde{\mathcal{X}}$ .
Proposition F.4 shows that the oracle returns a solution  $\widetilde{G}$  with cost at most
$$
(2 \beta + \widehat {\beta}) \widetilde {\mathrm {O P T}} + (2 B + c \Delta) + \widehat {B}
$$
on the discretized points.
Then for any  $x\in \mathcal{X}$
$$
\begin{array}{l} \widehat {\mathrm {O P T}} = \kappa (x, \tilde {G} _ {\mathrm {O P T}}) \\ \leq \kappa (x, \tilde {x}) + \kappa (\tilde {x}, \tilde {G} _ {\mathrm {O P T}}) \\ \leq \kappa (x, \tilde {x}) + \kappa (\tilde {x}, F _ {\mathrm {O P T}}) \\ \leq 2 \kappa (x, \tilde {x}) + \kappa (x, F _ {\mathrm {O P T}}) \\ \leq 2 c \Delta + \mathrm {O P T}. \\ \end{array}
$$
The conclusion follows by combining the two inequalities.
F.5 Replicable Active Cells
At a first glance, we might be tempted to work towards replicability by arguing that, with high probability, every grid cell is non-empty in one execution of the algorithm if and only if it is nonempty in another execution. However, since we have no control over  $\mathbb{P}$ , it is not clear how one can prove such a statement. Our approach is to take more samples and ignore cells which do not have a sufficient number of points using some random threshold. The idea is similar to the heavy-hitters algorithm (cf. Algorithm D.2) originally conceived by Impagliazzo et al. [2022] and is presented in Algorithm F.2. Then, we move all the points of active cells to the center of the cell. Notice that in the  $k$ -center objective, unlike the  $(k,p)$ -clustering objective, we do not care about how much mass is placed in a cell, just whether it is positive or not.
We first derive a bound on the number of cells that the distribution puts mass on. It is not hard to see that since we are working in a bounded domain, the grid contains at most  $\left( \frac{1}{c} \right)^d$  cells in total. Thus there are at most  $M \leq \left( \frac{1}{c} \right)^d$  cells that have non-zero mass. Proposition F.6 shows how we can obtain such a bound in the unbounded domain setting.
Proposition F.6. Let  $F_{*}$  be a  $(\beta, B)$ -approximate solution for Problem 2.3. Let  $M$  denote the maximum number of cells which intersect some cluster of  $F_{*}$  and  $\tilde{\mathbb{P}}$  the discretized distribution that is supported on centers of the cells of the grid of size  $c$ . Then, the support of  $\tilde{\mathbb{P}}$  is at most  $kM$  where
$$
M \leq \left(\frac {c + 2 c \Delta + 2 (\beta \mathrm {O P T} + B)}{c}\right) ^ {d}.
$$
Proof (Proposition F.6). Fix a center  $f$  and consider the cell  $Z$  containing the center. The farthest point in the farthest cell in the cluster of  $f$  is of distance at most  $\beta \mathrm{OPT} + B + c\Delta$  and thus the cells we are concerned with sit in an  $\ell_{\infty}$  ball of that radius. Not including  $Z$ , we walk past at most
$$
\frac {\beta \mathrm {O P T} + B + c \Delta}{c}
$$
cells along the canonical basis of  $\mathbb{R}^d$ , hence there are at most
$$
\left[ 1 + 2 c \Delta + \frac {2}{c} (\beta \mathrm {O P T} + B) \right] ^ {d}
$$
Notice that the previous discussion and Proposition F.5 reveal a natural trade-off in the choice of the parameter  $c$ : by reducing  $c$ , we decrease the additive error of our approximation algorithm but we increase the number of samples at a rate  $(1 / c)^d$ . Since we have bounded the number of cells from which we can observe a point, the next step is to use a sufficiently large number of samples so that we can estimate replicably whether a cell is active or not. This is demonstrated in Algorithm F.2.
Algorithm F.2 Replicable Active Cells
1: rActiveCellsgrid length  $c$
2:  $m\gets \tilde{O}\left(\lambda knM^2 /\rho^2\right)$
3:  $N \leftarrow \lambda n m M$
4: Sample  $x_{1},\ldots ,x_{N}\sim \mathbb{P},$
5: Lazy initialize counter  $Z = 0$  for every cell of the grid with length  $c$
6: for  $i\gets 1,\dots ,N$  do
7:  $G_{i}\gets$  cell that samples  $x_{i}$  falls into
8: Increment  $Z(G_{i}) \gets Z(G_{i}) + 1$
9: end for
10: Choose  $v \in [0, m / N]$  uniformly randomly
11: Output all  $G_{i}$  such that  $Z(G_{i}) / N \geq v$
Proposition F.7. Suppose Assumption F.2 holds and let  $F_{*}$  be the  $(\beta, B)$ -approximate solution. Let  $\delta, \rho \in (0,1)$ . Then Algorithm F.2 is  $\rho$ -replicable and returns cells of the grid such that at least one sample from each cluster of  $F_{*}$  falls into these cells with probability at least  $1 - \delta$ . Moreover, it has sample complexity
$$
\tilde {O} \left(\frac {n ^ {2} k M ^ {3} \log (1 / \delta)}{q ^ {2} \rho^ {2}}\right).
$$
Proof (Proposition F.7). Enumerate the  $kM$  cells within the support of  $\mathbb{P}$  and let the counter  $Z^{(j)}$  denote the number of samples at the  $j$ -th cell. Then  $Z = (Z^{(1)}, \ldots, Z^{(kM)})$  follows a multinomial distribution with parameters  $(p^{(1)}, \ldots, p^{(kM)})$  and  $N$ , where  $p$  is unknown to us. Nevertheless, by Proposition B.1, for any  $\varepsilon \in (0, 1)$ , sampling
$$
N \geq \frac {\ln^ {5 / \delta} + k M \ln 2}{2 \varepsilon^ {2}}
$$
points implies that  $\sum_{j=1}^{a} |\widehat{p}^{(j)} - p^{(j)}| < 2\varepsilon$  with probability at least  $1 - \delta/5$ . From the definition of  $N$ , this is equivalent to
$$
m \geq \frac {\ln^ {5 / \delta} + k M \ln 2}{n M \varepsilon^ {2}}.
$$
By Proposition F.3, we observe at least  $mM$  points from each cluster of  $F_{*}$  with probability at least  $1 - \delta$ .
Let  $\widehat{p}_1^{(j)},\widehat{p}_2^{(j)}$  be the empirical mean of  $Z^{(j)}$  across two different executions of Algorithm F.2. Then by the triangle inequality,  $\sum_{j = 1}^{kM}|\widehat{p}_1^{(j)} - \widehat{p}_2^{(j)}|\leq 4\varepsilon$
Since we observe at least  $Mm$  points from each cluster of  $F_{*}$ , there is at least one point from each cluster with at least  $m$  observations. Thus outputting the points  $j$  such that  $\widehat{p}^{(j)}\geq v$  for some  $v\in [0,m / N]$  means we always output one point from each cluster.
Now, the outputs between two executions are not identical only if the point  $v$  "splits" some pair  $\widehat{p_1^{(j)}}, \widehat{p_2^{(j)}}$ . This occurs with probability at most  $4\varepsilon / (m/N) = 4N\varepsilon / m$ , To bound this value by  $\rho/5$ , set
$$
\begin{array}{l} \varepsilon \leq \frac {\rho m}{2 0 N} \\ = \frac {\rho}{2 0 \lambda n M}. \\ \end{array}
$$
Plugging this back yields
$$
m \geq \frac {4 0 0 \lambda n M \ln \frac {5}{\delta} + 4 0 0 \lambda n k M ^ {2} \ln 2}{\rho^ {2}}.
$$
All in all, Algorithm F.2 outputs the desired result with probability at least  $1 - \delta$  and is  $\rho$ -replicable given
$$
\tilde {O} \left(\frac {n ^ {2} k M ^ {3} \log (1 / \delta)}{q ^ {2} \rho^ {2}}\right)
$$
samples from  $\mathbb{P}$
F.6 Putting it Together
We now have all the ingredients in place to prove Theorem 3.3. The proof follows directly by combining Proposition F.5, and Proposition F.7. Essentially, we replicably estimate the active cells of our grid and then move all the points of each active cell to its center.
For completeness, we also state the formal version of Theorem 3.3.
Theorem F.8 (Theorem 3.3; Formal). Let  $\delta, \rho \in (0,1)$ . Suppose Assumption F.2 holds and let  $F_*$  be the  $(\beta, B)$ -approximate solution. Then, given black-box access to a  $(\hat{\beta}, \hat{B})$ -approximation oracle for Problem 2.4, Algorithm F.2 is  $\rho$ -replicable and returns a solution  $\hat{F}$  such that
$$
\kappa (x, \hat {F}) \leq (2 \beta + \widehat {\beta}) \mathrm {O P T} + 2 B + \widehat {B} + (4 \beta + 2 \hat {\beta} + 1) c \Delta
$$
with probability at least  $1 - \delta$ . Moreover, the algorithm requires at most
$$
\tilde {O} \left(\frac {n ^ {2} k (1 / c) ^ {3 d} \log (1 / \delta)}{q ^ {2} \rho^ {2}}\right)
$$