SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\sum_ {(I n, O u t) \in \mathcal {S}} w (I n, O u t) \cdot \operatorname {R e a s s i g n} (I n, O u t) \leq
$$
$$
\sum_ {(I n, O u t) \in \mathcal {S}} w (I n, O u t) \cdot \sum_ {p \in C _ {O u t}} \operatorname {c o s t} (p, \mathcal {C} [ \mathcal {O} ^ {*} [ p ] ]) - \operatorname {c o s t} (p, \mathcal {C} [ p ]) \leq
$$
$$
(1 + 1 / p) \cdot \sum_ {c _ {j} \in \mathcal {C}} \sum_ {p \in C _ {j}} \operatorname {c o s t} (p, \mathcal {C} [ \mathcal {O} ^ {*} [ p ] ]) - \operatorname {c o s t} (p, \mathcal {C} [ p ]) \leq
$$
$$
(1 + 1 / p) \cdot \left(\sum_ {p \in P} \operatorname {c o s t} (p, \mathcal {C} [ \mathcal {O} ^ {*} [ p ] ]) - \mathrm {A L G}\right).
$$
The second inequality uses (iii) from Observation 7. Applying Lemma 8 completes the proof.
Recall the notions of radius and core of an optimal cluster introduced in Section 2. We say that a swap  $(In, Out)$  is strongly improving if  $\mathsf{cost}(P, (\mathcal{C} \cup In) \setminus Out) \leq (1 - \delta / k) \cdot \mathsf{cost}(P, \mathcal{C})$ . Let  $In = \{o_1 \ldots o_s\} \subseteq \widetilde{\mathcal{O}}$  and  $Out = \{c_1 \ldots c_s\} \subseteq \widetilde{\mathcal{C}}$  we say that an ideal swap  $(In, Out)$  is good if for every  $q_1 \in \mathsf{core}(o_1) \ldots q_s \in \mathsf{core}(o_s)$  the swap  $(\mathcal{Q}, Out)$  is strongly improving, where  $\mathcal{Q} = \{q_1 \ldots q_s\}$ . We call an ideal swap bad otherwise. We say that an optimal center  $o_i \in \widetilde{\mathcal{O}}$  is good if that's the case for at least one of the ideal swaps it belongs to, otherwise we say that it is bad. Notice that each optimal center in  $\widetilde{\mathcal{O}}$  is assigned to a swap in  $\mathcal{S}$ , so it is either good or bad. Denote with  $G$  the union of cores of good optimal centers in  $\widetilde{\mathcal{O}}$ .
Lemma 10. If an ideal swap  $(In, Out)$  is bad, then we have
$$
\operatorname {c o s t} \left(O _ {I n} ^ {*}, \mathcal {C}\right) \leq (2 + \delta) \operatorname {c o s t} \left(O _ {I n} ^ {*}, \mathcal {O} ^ {*}\right) + \operatorname {R e a s s i g n} (\operatorname {I n}, \operatorname {O u t}) + \delta A L G / k. \tag {2}
$$
Proof. Let  $In = \{o_1 \ldots o_s\}$ ,  $\mathcal{Q} = \{q_1 \ldots q_s\}$  such that  $q_1 \in \mathrm{core}(o_1) \ldots q_s \in \mathrm{core}(o_s)$ . Then, by Lemma 1  $\mathrm{cost}(O_{In}^*, \mathcal{Q}) \leq (2 + \delta) \mathrm{cost}(O_{In}^*, \mathcal{O}^*)$ . Moreover,  $\mathrm{Reassign}(In, Out) = \mathrm{cost}(P \setminus O_{In}^*, \mathcal{C} \setminus Out) - \mathrm{cost}(P \setminus O_{In}^*, \mathcal{C})$  because points in  $P \setminus C_{Out}$  are not affected by the swap. Therefore,  $\mathrm{cost}(P, (\mathcal{C} \cup \mathcal{Q}) \setminus Out) \leq (2 + \delta) \mathrm{cost}(O_{In}^*, \mathcal{O}^*) + \mathrm{Reassign}(In, Out) + \mathrm{cost}(P \setminus O_{In}^*, \mathcal{C})$ . Suppose by contradiction that Equation (2) does not hold, then
$$
\operatorname {c o s t} (P, \mathcal {C}) - \operatorname {c o s t} (P, (\mathcal {C} \cup \mathcal {Q}) \backslash O u t) =
$$
$$
\operatorname {c o s t} \left(P \setminus O _ {I n} ^ {*}, \mathcal {C}\right) + \operatorname {c o s t} \left(O _ {I n} ^ {*}, \mathcal {C}\right) - \operatorname {c o s t} \left(P, (\mathcal {C} \cup \mathcal {Q}) \setminus O u t\right) \geq \delta \mathrm {A L G} / k.
$$
Hence,  $(\mathcal{Q}, Out)$  is strongly improving and this holds for any choice of  $\mathcal{Q}$ , contradiction.
Lemma 11. If  $ALG / OPT > \eta^2 + \delta$  then  $cost(G, \mathcal{C}) = \Omega_{\delta}(cost(P, \mathcal{C}))$ . Thus, if we  $D^2$ -sample  $q$  we have  $P[q \in G] = \Omega_{\delta}(1)$ .
Proof. First, we observe that the combined current cost of all optimal clusters in  $\mathcal{O}^*\setminus \widetilde{\mathcal{O}}$  is at most  $k\cdot \delta \mathrm{ALG} / k = \delta \mathrm{ALG}$ . Now, we prove that the combined current cost of all  $O_{i}^{*}$  such that  $o_i$  is bad is  $\leq (1 - 2\delta)\mathrm{ALG}$ . Suppose, by contradiction, that it is not the case, then we have:
$$
(1 - 2 \delta) \mathrm {A L G} <   \sum_ {\text {B a d} o _ {i} \in \widetilde {\mathcal {O}}} \operatorname {c o s t} \left(O _ {i} ^ {*}, \mathcal {C}\right) \leq \sum_ {\text {B a d} (I n, O u t) \in \mathcal {S}} w (I n, O u t) \cdot \operatorname {c o s t} \left(O _ {I n} ^ {*}, \mathcal {C}\right) \leq
$$
$$
\sum_ {\text {B a d} (I n, O u t)} w (I n, O u t) \cdot ((2 + \delta) \operatorname {c o s t} \left(O _ {I n} ^ {*}, \mathcal {O} ^ {*}\right) + \operatorname {R e a s s i g n} (I n, O u t) + \delta \mathrm {A L G} / k) \leq
$$
$$
(2 + \delta) \mathrm {O P T} + (2 + 2 / p) \mathrm {O P T} + (2 + 2 / p) \sqrt {\mathrm {A L G}} \sqrt {\mathrm {O P T}} + \delta \mathrm {A L G}.
$$
The second and last inequalities make use of Observation 7. The third inequality uses Lemma 10.
Setting  $\eta^2 = \mathrm{ALG / OPT}$  we obtain the inequality  $\eta^2 -(2 + 2 / p\pm O(\delta))\eta -(4 + 2 / p\pm O(\delta))\leq 0$  Hence, we obtain a contradiction in the previous argument as long as  $\eta^2 -(2 + 2 / p\pm O(\delta))\eta -(4 + 2 / p\pm O(\delta)) > 0$  . A contradiction there implies that at least an  $\delta$  -fraction of the current cost is due to points in  $\bigcup_{\mathrm{Good} o_i\in \widetilde{\mathcal{O}}}O_i^*$  . We combine this with Lemma 2 and conclude that the total current cost of  $G = \bigcup_{\mathrm{Good} o_i\in \widetilde{\mathcal{O}}}\operatorname {core}(O_i^*)$  is  $\Omega_{\delta}(\operatorname {cost}(P,\mathcal{C}))$
Finally, we prove Lemma 6. Whenever  $q_{1} \in G$  we have that  $q_{1} \in \mathrm{core}(o_{1})$  for some good  $o_{1}$ . Then, for some  $s \leq p$  we can complete  $o_{1}$  with  $o_{2} \ldots o_{s}$  such that  $In = \{o_{1} \ldots o_{s}\}$  belongs to a good swap. Concretely, there exists  $Out \subseteq \mathcal{C}$  such that  $(In, Out)$  is a good swap. Since  $In \subset \widetilde{\mathcal{O}}$  we have  $\mathrm{cost}(O_{i}^{*}, \mathcal{C}) > \delta \mathrm{OPT} / k$  for all  $o_{i} \in In$ , which combined with Lemma 2 gives that for  $i = 2 \ldots s$ $P[q_{i} \in \mathrm{core}(o_{i})] \geq \Omega_{\delta}(1 / k)$ . Hence, we have  $P[q_{i} \in \mathrm{core}(o_{i})$  for  $i = 1 \ldots s] \geq \Omega_{\delta, p}(1 / k^{p - 1})$ . Whenever we sample  $q_{1} \ldots q_{s}$  from  $\mathrm{core}(o_{1}) \ldots \mathrm{core}(o_{s})$ , we have that  $(\mathcal{Q}, Out)$  is strongly improving. Notice, however, that  $(\mathcal{Q}, Out)$  is a  $s$ -swap and we may have  $s < p$ . Nevertheless, whenever we sample  $q_{1} \ldots q_{s}$  followed by any sequence  $q_{s + 1} \ldots q_{p}$  it is enough to choose  $Out' = Out \cup \{q_{s + 1} \ldots q_{p}\}$  to obtain that  $(\{q_{1} \ldots q_{p}\}, Out')$  is an improving  $p$ -swap.
4 A Faster  $(9 + \varepsilon)$ -Approximation Local Search Algorithm
The MSLS algorithm from Section 3 achieves an approximation ratio of  $\eta^2 + \varepsilon$ , where  $\eta^2 - (2 + 2/p)\eta - (4 + 2/p) = 0$  and  $\varepsilon > 0$  is an arbitrary small constant. For large  $p$  we have  $\eta \approx 10.48$ . On the other hand, employing  $p$  simultaneous swaps, Kanungo et al. [2004] achieve an approximation factor of  $\xi^2 + \varepsilon$  where  $\xi^2 - (2 + 2/p)\xi - (3 + 2/p) = 0$ . If we set  $p \approx 1/\varepsilon$  this yields a  $(9 + O(\varepsilon))$ -approximation. In the same paper, they prove that 9-approximation is indeed the best possible for  $p$ -swap local search, if  $p$  is constant (see Theorem 3.1 in Kanungo et al. [2004]). They showed that 9 is the right locality gap for local search, but they matched it with a very slow algorithm. To achieve a  $(9 + \varepsilon)$ -approximation, they discretize the space reducing to  $O(n\varepsilon^{-d})$  candidate centers and perform an exhaustive search over all size- $(1/\varepsilon)$  subsets of candidates at every step. As we saw in the related work section, it is possible to combine techniques from coreset and dimensionality reduction to reduce the number of points to  $n' = k \cdot poly(\varepsilon^{-1})$  and the number of dimensions to  $d' = \log k \cdot \varepsilon^{-2}$ . This reduces the complexity of Kanungo et al. [2004] to  $k^{O(\varepsilon^{-3} \log \varepsilon^{-1})}$ .
In this section, we leverage techniques from Cohen-Addad et al. [2021] to achieve a  $(9 + \varepsilon)$ -approximation faster. In particular, we obtain the following.
Theorem 12. Given a set of  $n$  points in  $\mathbb{R}^d$  with aspect ratio  $\Delta$ , there exists an algorithm that computes a  $9 + \varepsilon$ -approximation to  $k$ -means in time  $ndk^{O(\varepsilon^{-2})}\log^{O(\varepsilon^{-1})}(\Delta)\cdot 2^{-poly(\varepsilon^{-1})}$ .
Notice that, besides being asymptotically slower, the pipeline obtained combining known techniques is highly impractical and thus it did not make for an experimental test-bed. Moreover, it is not obvious how to simplify such an ensemble of complex techniques to obtain a practical algorithm.
Limitations of MSLS. The barrier we need to overcome in order to match the bound in Kanungo et al. [2004] is that, while we only consider points in  $P$  as candidate centers, the discretization they employ considers also points in  $\mathbb{R}^d \setminus P$ . In the analysis of MSLS we show that we sample each point  $q_i$  from  $\mathrm{core}(O_i^*)$  or equivalently that  $q_i \in B(o_i, (1 + \epsilon)\rho_i)$ , where  $\rho_i$  is such that  $O_i^*$  would have the same cost w.r.t.  $o_i$  if all its points were moved on a sphere of radius  $\rho_i$  centered in  $o_i$ . This allows us to use a Markov's inequality kind of argument and conclude that there must be  $\Omega_{\epsilon}(|O_i^*|)$  points in  $O_i^* \cap B(o_i, (1 + \epsilon)\rho_i)$ . However, we have no guarantee that there is any point at all in  $O_i^* \cap B(o_i, (1 - \varepsilon)\rho_i)$ . Indeed, all points in  $O_i^*$  might lie on  $\partial B(o_i, \rho_i)$ . The fact that potentially all our candidate centers  $q$  are at distance at least  $\rho_i$  from  $o_i$  yields (by Lemma 1)  $\mathrm{cost}(O_i^*, q) \geq 2\mathrm{cost}(O_i^*, o_i)$ , which causes the zero-degree term in  $\xi^2 - (2 + 2/p)\xi - (3 + 2/p) = 0$  from Kanungo et al. [2004] to become a 4 in our analysis.
Improving MSLS by taking averages. First, we notice that, in order to achieve  $(9 + \varepsilon)$ -approximation we need to set  $p = \Theta(1 / \varepsilon)$ . The main hurdle to achieve a  $(9 + \varepsilon)$ -approximation is that we need to replace the  $q_{i}$  in MSLS with a better approximation of  $o_{i}$ . We design a subroutine that computes, with constant probability, an  $\varepsilon$ -approximation  $\hat{o}_{i}$  of  $o_{i}$  (namely,  $\mathrm{cost}(O_{i}^{*}, \hat{o}_{i}) \leq (1 + \varepsilon) \mathrm{cost}(O_{i}^{*}, o_{i})$ ). The key idea is that, if sample uniformly  $O(1 / \varepsilon)$  points from  $O_{i}^{*}$  and define  $\hat{o}_{i}$  to be the average of our samples then  $\mathrm{cost}(O_{i}^{*}, \hat{o}_{i}) \leq (1 + \varepsilon) \mathrm{cost}(O_{i}^{*}, o_{i})$
Though, we do not know  $O_{i}^{*}$ , so sampling uniformly from it is non-trivial. To achieve that, for each  $q_{i}$  we identify a set  $N$  of nice candidate points in  $P$  such that a  $poly(\varepsilon)/k$  fraction of them are from  $O_{i}^{*}$ . We sample  $O(1/\varepsilon)$  points uniformly from  $N$  and thus with probability  $(\varepsilon/k)^{O(1/\varepsilon)}$  we sample only points from  $O_{i}^{*}$ . Thus far, we sampled  $O(1/\varepsilon)$  points uniformly from  $N \cap O_{i}^{*}$ . What about
4The complexity in Theorem 12 can be improved by applying the same preprocessing techniques using coresets and dimensionality reduction, similar to what can be used to speed up the approach of Kanungo et al. [2004]. Our algorithm hence becomes asymptotically faster.
the points in  $O_{i}^{*} \setminus N$ ? We can define  $N$  so that all points in  $O_{i}^{*} \setminus N$  are either very close to some of the  $(q_{j})_{j}$  or they are very far from  $q_{i}$ . The points that are very close to points  $(q_{j})_{j}$  are easy to treat. Indeed, we can approximately locate them and we just need to guess their mass, which is matters only when  $\geq poly(\varepsilon)\mathrm{ALG}$ , and so we pay only a  $\log^{O(1 / \varepsilon)}(1 / \varepsilon)$  multiplicative overhead to guess the mass close to  $q_{j}$  for  $j = 1\ldots p = \Theta (1 / \varepsilon)$ . As for a point  $f$  that is very far from  $q_{i}$  (say,  $||f - q_i||\gg \rho_i$ ) we notice that, although  $f$ 's contribution to  $\mathrm{cost}(O_i^*,o_i)$  may be large, we have  $\mathrm{cost}(f,o)\approx \mathrm{cost}(f,o_i)$  for each  $o\in B(q_i,\rho_i)\subseteq B(o_i,(2 + \varepsilon)\rho_i)$  assuming  $q_{i}\in \mathrm{core}(o_{i})$ .
In this section, we show that our new algorithm using multi-swap local search can be employed to design an efficient seeding algorithm for Lloyd's which outperforms both the classical  $k$ -means++ seeding and the single-swap local search from Lattanzi and Sohler [2019].
Algorithms. The multi-swap local search algorithm that we analysed above performs very well in terms of solution quality. This empirically verifies the improved approximation factor of our algorithm, compared to the single-swap local search of Lattanzi and Sohler [2019].
Motivated by practical considerations, we heuristically adapt our algorithm to make it very competitive with SSLS in terms of running time and still remain very close, in terms of solution quality, to the theoretically superior algorithm that we analyzed. The adaptation of our algorithm replaces the phase where it selects the  $p$  centers to swap-out by performing an exhaustive search over  $\binom{k+p}{p}$  subsets of centers. Instead, we use an efficient heuristic procedure for selecting the  $p$  centers to swap-out, by greedily selecting one by one the centers to swap-out. Specifically, we select the first center to be the cheapest one to remove (namely, the one that increases the cost by the least amount once the points in its cluster are reassigned to the remaining centers). Then, we update all costs and select the next center iteratively. After  $p$  repetitions we are done. We perform an experimental evaluation of the "greedy" variant of our algorithm compared to the theoretically-sound algorithm from Section 3 and show that employing the greedy heuristic does not measurably impact performance.
The four algorithms that we evaluate are the following: 1)  $\mathbf{KM} + +$ : The  $k$ -means++ from Arthur and Vassilvitskii [2007], 2) SSLS: The Single-swap local search method from Lattanzi and Sohler [2019], 3) MSLS: The multi-swap local search from Section 3, and 4) MSLS-G: The greedy variant of multi-swap local search as described above.
We use MSLS-G-  $p = x$  and MSLS-  $p = x$ , to denote MSLS-G and MSLS with  $p = x$ , respectively. Notice that MSLS-G-  $p = 1$  is exactly SSLS. Our experimental evaluation explores the effect of  $p$ -swap LS, for  $p > 1$ , in terms of solution cost and running time.
Datasets. We consider the three datasets used in Lattanzi and Sohler [2019] to evaluate the performance of SSLS: 1) KDD-PHY - 100,000 points with 78 features representing a quantum physic task kdd [2004], 2) RNA - 488,565 points with 8 features representing RNA input sequence pairs Uzilov et al. [2006], and 3) KDD-BIO - 145,751 points with 74 features measuring the match between a protein and a native sequence kdd [2004]. We discuss the results for two or our datasets, namely KDD-BIO and RNA. We defer the results on KDD-PHY to the appendix and note that the results are very similar to the results on RNA.
We performed a preprocessing step to clean-up the datasets. We observed that the standard deviation of some features was disproportionately high. This causes all costs being concentrated in few dimensions making the problem, in some sense, lower-dimensional. Thus, we apply min-max scaling to all datasets and observed that this causes all our features' standard deviations to be comparable.
Experimental setting. All our code is written in Python. The code will be made available upon publication of this work. We did not make use of parallelization techniques. To run our experiments, we used a personal computer with 8 cores, a 1.8 Ghz processor, and 15.9 GiB of main memory. We run all experiments 5 times and report the mean and standard deviation in our plots. All our plots report the progression of the cost either w.r.t local search steps, or Lloyd's iterations. We run experiments on all our datasets for  $k = 10, 25, 50$ . The main body of the paper reports the results for  $k = 25$ , while the rest can be found in the appendix. We note that the conclusions of the experiments for  $k = 10, 50$  are similar to those of  $k = 25$ .
Removing centers greedily. We first we compare MSLS-G with MSLS. To perform our experiment, we initialize  $k = 25$  centers using  $k$ -means++ and then run 50 iterations of local search for both
Figure 1: Comparison between MSLS and MSLS-G, for  $p = 3$ , for  $k = 25$ , on the datasets KDD-BIO and RNA. The  $y$  axis shows the solution cost divided by the means solution cost of KM++.
algorithms, for  $p = 3$  swaps. Due to the higher running of the MSLS we perform this experiments on  $1\%$  uniform sample of each of our datasets. We find out that the performance of the two algorithms is comparable on all our instances, while they both perform roughly  $15\% -27\%$  at convergence. Figure 1 shows the aggregate results, over 5 repetitions of our experiment.
It may happen that MSLS, which considers all possible swaps of size  $p$  at each LS iteration, performs worse than MSLS-G as a sub-optimal swap at intermediate iterations may still lead to a better local optimum by coincidence. Given that MSLS-G performs very comparably to MSLS, while it is much faster in practice, we use MSLS-G for the rest of our experiments where we compare to baselines. This allows us to consider higher values of  $p$ , without compromising much the running time.
Results: Evaluating the quality and performance of the algorithms. In our first experiment we run  $\mathrm{KM}++$  followed by 50 iterations of MSLS-G with  $p = 1,4,7,10$  and plot the relative cost w.r.t.  $\mathrm{KM}++$  at each iteration, for  $k = 25$ . The first row of Figure 2 plots the results. Our experiment shows that, after 50 iterations MSLS-G for  $p = 4,7,10$  achieves improvements of roughly  $10\%$  compared to MSLS-G- $p = 1$  and of the order of  $20\% - 30\%$  compared to  $\mathrm{KM}++$ . We also report the time per iteration that each algorithm takes. For comparison, we report the running time of a single iteration of Lloyd's next to the dataset's name. It is important to notice that, although MSLS-G- $p = 1$  is faster, running more iterations MSLS-G- $p = 1$  is not sufficient to compete with MSLS-G when  $p > 1$ .
Results: Evaluating the quality after postprocessing using Lloyd. In our second experiment, we use  $\mathrm{KM}++$  and MSLS-G as a seeding algorithm for Lloyd's and measure how much of the performance improvement measured in the first experiment is retained after running Lloyd's. First, we initialize our centers using  $\mathrm{KM}++$  and the run 15 iterations of MSLS-G for  $p = 1,4,7$ . We measure the cost achieved by running 10 iterations of Lloyd's starting from the solutions found by MSLS-G as well as  $\mathrm{KM}++$ . In Figure 2 (second row) we plot the results. Notice that, according to the running times from the first experiment, 15 iterations iterations of MSLS-G take less than 10 iterations of Lloyd's for  $p = 4,7$  (and also for  $p = 10$ , except on RNA). We observe that MSLS-G for  $p > 1$  performs at least as good as SSLS from Lattanzi and Sohler [2019] and in some cases maintains non-trivial improvements.
Results: Evaluating the quality and performance of the algorithms against a fixed deadline. In this experiment we run  $\mathrm{KM}++$  followed by MSLS-G with  $p = 1,4,7,10$ , for a set of fixed amounts of time. This setting allows the versions of MSLS-G with smaller swap size to perform more iterations compared to the versions of the algorithm with a larger swap size, as smaller swap size leads to lower running time per iteration. Let  $\tau$  be the average time that Lloyd's algorithm requires to complete a simple iteration on a specific instance. We plot the cost of the solution produced by each algorithm after running  $\lambda \times \tau$  for each  $\lambda \in \{1,\dots ,20\}$  in Figure 3. Our experiment shows that MSLS-G for  $p = 4,7,10$  achieves improvements of more than  $5\%$  compared to MSLS-G- $p = 1$  even when compared against a fixed running time, and of the order of  $20\% -30\%$  compared to  $\mathrm{KM}++$ .
Conclusion and Future Directions
We present a new algorithm for the  $k$ -means problem and we show that it outperforms theoretically and experimentally state-of-the-art practical algorithms with provable guarantees in terms of solution
Figure 2: The first row compares the cost of MSLS-G, for  $p \in \{1,4,7,10\}$ , divided by the mean cost of KM++ at each LS step, for  $k = 25$ . The legend reports also the running time of MSLS-G per LS step (in seconds). The second row compares the cost after each of the 10 iterations of Lloyd with seeding from MSLS-G, for  $p \in \{1,4,7,10\}$  and 15 local search steps and KM++, for  $k = 25$ .
Figure 3: Comparison of the cost produced by MSLS-G, for  $p \in \{1, 4, 7, 10\}$  and  $k = 25$  on the datasets KDD-BIO and KDD-PHU, divided by the mean cost of KM++ after running for fixed amount of time in terms of multiplicative factors to the average time for an iteration of Lloyd's algorithm (i.e., for deadlines that are  $1 \times \ldots$ ,  $20 \times$  the average time of an iteration of Lloyd).
quality. A very interesting open question is to improve our local search procedure by avoiding the exhaustive search over all possible size- $p$  subsets of centers to swap out, concretely an algorithm with running time  $\tilde{O}(2^{poly(1/\varepsilon)}ndk)$ .
Acknowledgements. This work was partially done when Lorenzo Beretta was a Research Student at Google Research. Moreover, Lorenzo Beretta receives funding from the European Union's Horizon 2020 research and innovation program under the Marie Skłodowska-Curie grant agreement No. 801199.
Proofs from Section 3
Lemma 8.  $\sum_{p\in P}\cos t(p,\mathcal{C}[\mathcal{O}^*[p]])\leq 2OPT + ALG + 2\sqrt{ALG}\sqrt{OPT}.$
$$
\sum_ {p \in P} \mathsf {c o s t} (p, \mathcal {C} [ \mathcal {O} ^ {*} [ p ] ]) =
$$
$$
\sum_ {o _ {i} \in \mathcal {O} ^ {*}} \sum_ {p \in O _ {i} ^ {*}} \operatorname {c o s t} (p, \mathcal {C} [ o _ {i} ]) =
$$
$$
\sum_ {o _ {i} \in \mathcal {O} ^ {*}} | O _ {i} ^ {*} | \cdot \operatorname {c o s t} (o _ {i}, \mathcal {C} [ o _ {i} ]) + \operatorname {c o s t} (O _ {i} ^ {*}, o _ {i}) =
$$
$$
\operatorname {O P T} + \sum_ {p \in P} \operatorname {c o s t} \left(\mathcal {O} ^ {*} [ p ], \mathcal {C} \left[ \mathcal {O} ^ {*} [ p ] \right]\right) \leq
$$
$$
\mathrm {O P T} + \sum_ {p \in P} \operatorname {c o s t} \left(\mathcal {O} ^ {*} [ p ], \mathcal {C} [ p ]\right) \leq
$$
$$
\mathrm {O P T} + \sum_ {p \in P} \left(\left| \left| \mathcal {O} ^ {*} [ p ] - p \right| \right| + \left| \left| p - \mathcal {C} [ p ] \right| \right|\right) ^ {2} =
$$
$$
2 \mathrm {O P T} + \mathrm {A L G} + 2 \sum_ {p \in P} | | \mathcal {O} ^ {*} [ p ], p | | \cdot | | p, \mathcal {C} [ p ] | | \leq 2 \mathrm {O P T} + \mathrm {A L G} + 2 \sqrt {\mathrm {A L G}} \sqrt {\mathrm {O P T}}.
$$
The second equality is due to Lemma 1 and the last inequality is due to Cauchy-Schwarz.
Proofs from Section 4
In this section, we prove the following.
We start with a key lemma showing that a sample of size  $O(1 / \varepsilon)$  is enough to approximate 1-mean.
Lemma 13 (Form Inaba et al. [1994]). Given an instance  $P \subseteq \mathbb{R}^d$ , sample  $m = 1 / (\varepsilon \delta)$  points uniformly at random from  $P$  and denote the set of samples with  $S$ . Then  $\text{cost}(P, \mu(S)) \leq (1 + \varepsilon) \text{cost}(P, \mu(P))$  with probability at least  $1 - \delta$ .