SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Multi-Swap  $k$ -Means++
Lorenzo Beretta*
lorenzo2beretta@gmail.com
Vincent Cohen-Addad
cohenaddad@google.com
silviol@google.com
nikosp@google.com
The  $k$ -means++ algorithm of Arthur and Vassilvitskii (SODA 2007) is often the practitioners' choice algorithm for optimizing the popular  $k$ -means clustering objective and is known to give an  $O(\log k)$ -approximation in expectation. To obtain higher quality solutions, Lattanzi and Sohler (ICML 2019) proposed augmenting  $k$ -means++ with  $O(k \log \log k)$  local search steps obtained through the  $k$ -means++ sampling distribution to yield a  $c$ -approximation to the  $k$ -means clustering problem, where  $c$  is a large absolute constant. Here we generalize and extend their local search algorithm by considering larger and more sophisticated local search neighborhoods hence allowing to swap multiple centers at the same time. Our algorithm achieves a  $9 + \varepsilon$  approximation ratio, which is the best possible for local search. Importantly we show that our approach yields substantial practical improvements, we show significant quality improvements over the approach of Lattanzi and Sohler (ICML 2019) on several datasets.
Clustering is a central problem in unsupervised learning. In clustering one is interested in grouping together "similar" object and separate "dissimilar" one. Thanks to its popularity many notions of clustering have been proposed overtime. In this paper, we focus on metric clustering and on one of the most studied problem in the area: the Euclidean  $k$ -means problem.
In the Euclidean  $k$ -means problem one is given in input a set of points  $P$  in  $\mathbb{R}^d$ . The goal of the problem is to find a set of  $k$  centers so that the sum of the square distances to the centers is minimized. More formally, we are interested in finding a set  $C$  of  $k$  points in  $\mathbb{R}^d$  such that  $\sum_{p \in P} \min_{c \in C} ||p - c||^2$ , where with  $||p - c||$  we denote the Euclidean distance between  $p$  and  $c$ .
The  $k$ -means problem has a long history, in statistics and operations research. For Euclidean  $k$ -means with running time polynomial in both  $n$ ,  $k$  and  $d$ , a 5.912-approximation was recently shown in Cohen-Addad et al. [2022a], improving upon Kanungo et al. [2004], Ahmadian et al. [2019], Grandoni et al. [2022] by leveraging the properties of the Euclidean metric. In terms of lower bounds, the first to show that the high-dimensional  $k$ -means problems were APX-hard were Guruswami and Indyk [2003], and later Awasthi et al. [2015] showed that the APX-hardness holds even if the centers can be placed arbitrarily in  $\mathbb{R}^d$ . The inapproximability bound was later slightly improved by Lee et al. [2017] until the recent best known bounds of Cohen-Addad and Karthik C. S. [2019], Cohen-Addad et al. [2022d, 2021] that showed that it is NP-hard to achieve a better than 1.06-approximation and hard to approximate it better than 1.36 assuming a stronger conjecture. From a more practical point of view, Arthur and Vassilitskii [2009] showed that the widely-used popular heuristic of Lloyd Lloyd
*Authors are ordered in alphabetical order.
[1957] can lead to solutions with arbitrarily bad approximation guarantees, but can be improved by a simple seeding strategy, called  $k$ -means++, so as to guarantee that the output is within an  $O(\log k)$  factor of the optimum Arthur and Vassilvitskii [2007].
Thanks to its simplicity  $k$ -means++ is widely adopted in practice. In an effort to improve its performances Lattanzi and Sohler [2019], Choo et al. [2020] combine  $k$ -means++ and local search to efficiently obtain a constant approximation algorithm with good practical performance. These two studies show that one can use the  $k$ -means++ distribution in combination with a local search algorithm to get the best of both worlds: a practical algorithm with constant approximation guarantees.
However, the constant obtained in Lattanzi and Sohler [2019], Choo et al. [2020] is very large (several thousands in theory) and the question as whether one could obtain a practical algorithm that would efficiently match the  $9 + \varepsilon$ -approximation obtained by the  $n^{O(d / \epsilon)}$  algorithm of Kanungo et al. [2004] has remained open. Bridging the gap between the theoretical approach of Kanungo et al. [2004] and  $k$ -means++ has thus been a long standing goal.
Our Contributions. We make significant progress on the above line of work.
- We adapt techniques from the analysis of Kanungo et al. [2004] to obtain a tighter analysis of the algorithm in Lattanzi and Sohler [2019]. In particular in Corollary 4, we show that their algorithm achieves an approximation of ratio of  $\approx 26.64$ .
- We extend this approach to multi-swaps, where we allow swapping more than one center at each iteration of local search, improving significantly the approximation to  $\approx 10.48$  in time  $O(nd\cdot poly(k))$ .
- Leveraging ideas from Cohen-Addad et al. [2021], we design a better local search swap that improves the approximation further to  $9 + \varepsilon$  (see Theorem 12). This new algorithm matches the  $9 + \varepsilon$ -approximation achieved by the local search algorithm in Kanungo et al. [2004], but it is significantly more efficient. Notice that 9 is the best approximation achievable through local search algorithms, as proved in Kanungo et al. [2004].
- We provide experiments where we compare against  $k$ -means++ and Lattanzi and Sohler [2019]. We study a variant of our algorithm that performs very competitively with our theoretically sound algorithm. The variant is very efficient and still outperforms previous work in terms of solution quality, even after the standard postprocessing using Lloyd.
Additional Related Work. We start by reviewing the approach of Kanungo et al. [2004] and a possible adaptation to our setting. The bound of  $9 + \varepsilon$  on the approximation guarantee shown by Kanungo et al. [2004] is for the following algorithm: Given a set  $S$  of  $k$  centers, if there is a set  $S^{+}$  of at most  $2 / \varepsilon$  points in  $\mathbb{R}^d$  together with a set  $S^{-}$  of  $|S^{+}|$  points in  $S$  such that  $S\setminus S^{-}\cup S^{+}$  achieves a better  $k$ -means cost than  $S$ , then set  $S\coloneqq S\setminus S^{-}\cup S^{+}$  and repeat until convergence. The main drawback of the algorithm is that it asks whether there exists a set  $S^{+}$  of points in  $\mathbb{R}^d$  that could be swapped with elements of  $S$  to improve the cost. Identifying such a set, even of constant size, is already non-trivial. The best way of doing so is through the following path: First compute a coreset using the state-of-the-art coreset construction of Cohen-Addad et al. [2022b] and apply the dimensionality reduction of Becchetti et al. [2019], Makarychev et al. [2019], hence obtaining a set of  $\tilde{O}(k/\varepsilon^4)$  points in dimension  $O(\log k/\varepsilon^2)$ . Then, compute grids using the discretization framework of Matousek [2000] to identify a set of  $\varepsilon^{-O(d)}\sim k^{O(\varepsilon^{-2}\log(1/\varepsilon))}$  grid points that contains nearly-optimum centers. Now, run the local search algorithm where the sets  $S^{+}$  are chosen from the grid points by brute-force enumeration over all possible subsets of grid points of size at most, say  $s$ . The running time of the whole algorithm with swaps of magnitude  $s$ , i.e.:  $|S^{+}|\leq s$ , hence becomes  $k^{O(s\cdot \varepsilon^{-2}\log(1 / \varepsilon))}$  for an approximation of  $(1 + \varepsilon)(9 + 2 / s)$ , meaning a dependency in  $k$  of  $k^{O(\varepsilon^{-3}\log(1 / \varepsilon))}$  to achieve a  $9 + \varepsilon$ -approximation. Our results improve upon this approach in two ways: (1) it improves over the above theoretical bound and (2) does so through an efficient and implementable, i.e.: practical, algorithm.
Recently, Grunau et al. [2023] looked at how much applying a greedy rule on top of the  $k$ -means++ heuristic improves its performance. The heuristic is that at each step, the algorithm samples  $\ell$  centers and only keeps the one that gives the best improvement in cost. Interestingly the authors prove that from a theoretical standpoint this heuristic does not improve the quality of the output. Local search algorithms for  $k$ -median and  $k$ -means have also been studied by Gupta and Tangwongsan [2008] who drastically simplified the analysis of Arya et al. [2004]. Cohen-Addad and Schwiegelshohn [2017]
demonstrated the power of local search for stable instances. Friggstad et al. [2019], Cohen-Addad et al. [2019] showed that local search yields a PTAS for Euclidean inputs of bounded dimension (and doubling metrics) and minor-free metrics. Cohen-Addad [2018] showed how to speed up the local search algorithm using  $kd$ -trees (i.e.: for low dimensional inputs).
For fixed  $k$ , there are several known approximation schemes, typically using small coresets Becchetti et al. [2019], Feldman and Langberg [2011], Kumar et al. [2010]. The state-of-the-art approaches are due to Bhattacharya et al. [2020], Jaiswal et al. [2014]. The best known coreset construction remains Cohen-Addad et al. [2022c,b].
If the constraint on the number of output centers is relaxed, then we talk about bicriteria approximations and  $k$ -means has been largely studied Bandyapadhyay and Varadarajan [2016], Charikar and Guha [2005], Cohen-Addad and Mathieu [2015], Korupolu et al. [2000], Makarychev et al. [2016].
Notation. We denote with  $P \subseteq \mathbb{R}^d$  the set of input points and let  $n = |P|$ . Given a point set  $Q \subseteq P$  we use  $\mu(Q)$  to denote the mean of points in  $Q$ . Given a point  $p \in P$  and a set of centers  $A$  we denote with  $A[p]$  the closest center in  $A$  to  $p$  (ties are broken arbitrarily). We denote with  $\mathcal{C}$  the set of centers currently found by our algorithm and with  $\mathcal{O}^*$  an optimal set of centers. Therefore, given  $p \in P$ , we denote with  $\mathcal{C}[p]$  and  $\mathcal{O}^*[p]$  its closest ALG-center and OPT-center respectively. We denote by  $\mathrm{cost}(Q, A)$  the cost of points in  $Q \subseteq P$  w.r.t. the centers in  $A$ , namely
$$
\operatorname {c o s t} (Q, A) = \sum_ {q \in Q} \min  _ {c \in A} | | q - c | | ^ {2}.
$$
We use ALG and OPT as a shorthand for  $\mathrm{cost}(P,\mathcal{C})$  and  $\mathrm{cost}(P,\mathcal{O}^*)$  respectively. When we sample points proportionally to their current cost (namely, sample  $q$  with probability  $\mathrm{cost}(q,\mathcal{C}) / \mathrm{cost}(P,\mathcal{C})$ ) we call this the  $D^{2}$  distribution. When using  $O_{\varepsilon}(\cdot)$  and  $\Omega_{\varepsilon}(\cdot)$  we mean that  $\varepsilon$  is considered constant. We use  $\widetilde{O}(f)$  to hide polylogarithmic factors in  $f$ . The following lemma is folklore.
Lemma 1. Given a point set  $Q \subseteq P$  and a point  $p \in P$  we have
$$
\operatorname {c o s t} (Q, p) = \operatorname {c o s t} (Q, \mu (Q)) + | Q | \cdot \| p - \mu (Q) \| ^ {2}.
$$
Let  $O_{i}^{*}$  be an optimal cluster, we define the radius of  $O_{i}^{*}$  as  $\rho_{i}$  such that  $\rho_{i}^{2} \cdot |O_{i}^{*}| = \mathrm{cost}(O_{i}^{*}, o_{i})$ , where  $o_{i} = \mu(O_{i}^{*})$ . We define the  $\delta$ -core of the optimal cluster  $O_{i}^{*}$  as the set of points  $p \in O_{i}^{*}$  that lie in a ball of radius  $(1 + \delta)\rho_{i}$  centered in  $o_{i}$ . In symbols,  $\mathrm{core}(O_{i}^{*}) = P \cap B(o_{i}, (1 + \delta)\rho_{i})$ . Throughout the paper,  $\delta$  is always a small constant fixed upfront, hence we omit it.
Lemma 2. Let  $O_{i}^{*}$  be an optimal cluster and sample  $q \in O_{i}^{*}$  according to the  $D^{2}$ -distribution restricted to  $O_{i}^{*}$ . If  $\text{cost}(O_{i}^{*}, \mathcal{C}) > (2 + 3\delta) \cdot \text{cost}(O_{i}^{*}, o_{i})$  then  $\operatorname{Pr}[q \in \text{core}(O_{i}^{*})] = \Omega_{\delta}(1)$ .
Proof. Define  $\alpha := \mathrm{cost}(O_i^*, \mathcal{C}) / \mathrm{cost}(O_i^*, o_i) > 2 + 3\delta$ . Thanks to Lemma 1, for each  $c \in \mathcal{C}$  we have  $\|c - o_i\|^2 \geq (\alpha - 1)\rho_i^2$ . Therefore, for each  $y \in \mathrm{core}(O_i^*)$  and every  $c \in \mathcal{C}$  we have
$$
\mathsf {c o s t} (y, c) = | | y - c | | ^ {2} \geq \left(\sqrt {\alpha - 1} - (1 + \delta)\right) ^ {2} \cdot \rho_ {i} ^ {2} = \Omega_ {\delta} (\alpha \rho_ {i} ^ {2}).
$$
Moreover, by a Markov's inequality argument we have  $|O_i^* \setminus \operatorname{core}(O_i^*)| \leq \frac{1}{1 + \delta} \cdot |O_i^*|$  and thus  $|\operatorname{core}(O_i^*)| \geq \Omega_\delta(|O_i^*|)$ . Combining everything we get
$$
\operatorname {cost}(\operatorname {core}(O_{i}^{*}),\mathcal{C})\geq |\operatorname {core}(O_{i}^{*})|\cdot \min_{\substack{c\in \mathcal{C}\\ y\in \operatorname {core}(O_{i}^{*})}}\operatorname {cost}(y,c) = \Omega_{\delta}(|O_{i}^{*}|)\cdot \Omega_{\delta}(\alpha \rho_{i}^{2})
$$
and  $|O_i^*| \cdot \alpha \rho_i^2 = \mathrm{cost}(O_i^*, \mathcal{C})$ , hence  $\mathrm{cost}(\mathrm{core}(O_i^*), \mathcal{C}) = \Omega_\delta(\mathrm{cost}(O_i^*, \mathcal{C}))$ .
3 Multi-Swap  $k$ -Means++
The single-swap local search (SSLS)  $k$ -means++ algorithm in Lattanzi and Sohler [2019] works as follows. First,  $k$  centers are sampled using  $k$ -means++ (namely, they are sampled one by one according to the  $D^2$  distribution, updated for every new center). Then,  $O(k\log \log k)$  steps of local search follow. In each local search step a point  $q \in P$  is  $D^2$ -sampled, then let  $c$  be the center among
the current centers  $\mathcal{C}$  such that  $\mathrm{cost}(P, (\mathcal{C} \setminus \{c\}) \cup \{q\})$  is minimum. If  $\mathrm{cost}(P, (\mathcal{C} \setminus \{c\}) \cup \{q\}) < \mathrm{cost}(P, \mathcal{C})$  then we swap  $c$  and  $q$ , or more formally we set  $\mathcal{C} \gets (\mathcal{C} \setminus \{c\}) \cup \{q\}$ .
We extend the SSLS so that we allow to swap multiple centers simultaneously and call this algorithm multi-swap local search (MSLS)  $k$ -means++. Swapping multiple centers at the same time achieves a lower approximation ratio, in exchange for a higher time complexity. In this section, we present and analyse the  $p$ -swap local search (LS) algorithm for a generic number of  $p$  centers swapped at each step. For any constant  $\delta > 0$ , we obtain an approximation ratio  $\mathrm{ALG} / \mathrm{OPT} = \eta^2 + \delta$  where
$$
\eta^ {2} - (2 + 2 / p) \eta - (4 + 2 / p) = 0. \tag {1}
$$
The Algorithm. First, we initialize our set of centers using  $k$ -means++. Then, we run  $O(ndk^{p-1})$  local search steps, where a local search step works as follows. We  $D^2$ -sample a set  $In = \{q_1 \ldots q_p\}$  of points from  $P$  (without updating costs). Then, we iterate over all possible sets  $Out = \{c_1 \ldots c_p\}$  of  $p$  distinct elements in  $\mathcal{C} \cup In$  and select the set  $Out$  such that performing the swap  $(In, Out)$  maximally improves the cost<sup>2</sup>. If this choice of  $Out$  improves the cost, then we perform the swap  $(In, Out)$ , else we do not perform any swap for this step.
Theorem 3. For any  $\delta >0$ , the  $p$ -swap local search algorithm above runs in  $\widetilde{O} (ndk^{2p})$  time and, with constant probability, finds an  $(\eta^2 +\delta)$ -approximation of  $k$ -means, where  $\eta$  satisfies Equation (1).
Notice that the SSLS algorithm of Lattanzi and Sohler [2019] is exactly the  $p$ -swap LS algorithm above for  $p = 1$ .
Corollary 4. The single-swap local search in Lattanzi and Sohler [2019], Choo et al. [2020] achieves an approximation ratio  $< 26.64$ .
Corollary 5. For  $p = O(1)$  large enough, multi-swap local search achieves an approximation ratio  $< 10.48$  in time  $O(nd \cdot poly(k))$ .
3.1 Analysis of Multi-Swap  $k$ -means++
In this section we prove Theorem 3. Our main stepping stone is the following lemma.
Lemma 6. Let ALG denote the cost at some point in the execution of MSLS. As long as ALG/OPT >  $\eta^2 +\delta$  , a local search step improves the cost by a factor  $1 - \Omega (1 / k)$  with probability  $\Omega (1 / k^{p - 1})$
Proof of Theorem 3. First, we show that  $O(k^p \log \log k)$  local steps suffice to obtain the desired approximation ratio, with constant probability. Notice that a local search step can only improve the cost function, so it is sufficient to show that the approximation ratio is achieved at some point in time. We initialize our centers using  $k$ -means++, which gives a  $O(\log k)$ -approximation in expectation. Thus, using Markov's inequality the approximation guarantee  $O(\log k)$  holds with arbitrary high constant probability. We say that a local-search step is successful if it improves the cost by a factor of at least  $1 - \Omega(1/k)$ . Thanks to Lemma 6, we know that unless the algorithm has already achieved the desired approximation ratio then a local-search step is successful with probability  $\Omega(1/k^{p-1})$ . To go from  $O(\log k)$  to  $\eta^2 + \delta$  we need  $O(k \log \log k)$  successful local search steps. Standard concentration bounds on the value of a Negative Binomial random variable show that, with high probability, the number of trials to obtain  $O(k \log \log k)$  successful local-search steps is  $O(k^p \log \log k)$ . Therefore, after  $O(k^p \log \log k)$  local-search steps we obtain an approximation ratio of  $\eta^2 + \delta$ .
To prove the running time bound it is sufficient to show that a local search step can be performed in time  $\widetilde{O}(ndk^{p-1})$ . This is possible if we maintain, for each point  $x \in P$ , a dynamic sorted dictionary<sup>3</sup> storing the pairs  $(\mathrm{cost}(x, c_i), c_i)$  for each  $c_i \in \mathcal{C}$ . Then we can combine the exhaustive search over all possible size- $p$  subsets of  $\mathcal{C} \cup In$  and the computation of the new cost function using time  $O(ndk^{p-1} \log k)$ . To do so, we iterate over all possible size- $(p-1)$  subsets  $Z$  of  $\mathcal{C} \cup In$  and update all costs as if these centers were removed, then for each point  $x \in P$  we compute how much its cost increases if we remove its closest center  $c_x$  in  $(\mathcal{C} \cup In) \setminus Z$  and charge that amount to  $c_x$ . In the end, we consider  $Out = Z \cup \{c\}$  where  $c$  is the cheapest-to-remove center found in this way.
The rest of this section is devoted to proving Lemma 6. For convenience, we prove that Lemma 6 holds whenever  $\mathrm{ALG / OPT} > \eta^2 +O(\delta)$ , which is wlog by rescaling  $\delta$ . Recall that we now focus on
2If  $In\cap Out\neq \emptyset$  then we are actually performing the swap  $(In\backslash Out,Out\backslash In)$  of size  $<  p$
3Also known as dynamic predecessor search data structure.
a given step of the algorithm, and when we say current cost, current centers and current clusters we refer to the state of these objects at the end of the last local-search step before the current one. Let  $O_1^* \ldots O_k^*$  be an optimal clustering of  $P$  and let  $\mathcal{O}^* = \{o_i = \mu(O_i^*) \mid i = 1 \ldots k\}$  be the set of optimal centers of these clusters. We denote with  $C_1 \ldots C_k$  the current set of clusters at that stage of the local search and with  $\mathcal{C} = \{c_1 \ldots c_k\}$  the set of their respective current centers.
We say that  $c_{i}$  captures  $o_{j}$  if  $c_{i}$  is the closest current center to  $o_{j}$ , namely  $c_{i} = \mathcal{C}[o_{j}]$ . We say that  $c_{i}$  is busy if it captures more than  $p$  optimal centers, and we say it is lonely if it captures no optimal center. Let  $\widetilde{\mathcal{O}} = \{o_{i} \mid \mathrm{cost}(O_{i}^{*},\mathcal{C}) > \delta \cdot \mathrm{ALG} / k\}$  and  $\widetilde{\mathcal{C}} = \mathcal{C} \setminus \{\mathcal{C}[o_{i}] \mid o_{i} \in \mathcal{O}^{*} \setminus \widetilde{\mathcal{O}}\}$ . For ease of notation, we simply assume that  $\widetilde{\mathcal{O}} = \{o_{1}\ldots o_{h}\}$  and  $\widetilde{\mathcal{C}} = \{c_{1}\ldots c_{h^{\prime}}\}$ . Notice that  $h^\prime > h$ .
Weighted ideal multi-swaps. Given  $In \subseteq P$  and  $Out \subseteq \widetilde{\mathcal{C}}$  of the same size we say that the swap  $(In, Out)$  is an ideal swap if  $In \subseteq \widetilde{\mathcal{O}}$ . We now build a set of weighted ideal multi-swaps  $S$ . First, suppose wlog that  $\{c_1 \ldots c_t\}$  is the set of current centers in  $\widetilde{\mathcal{C}}$  that are neither lonely nor busy. Let  $\mathcal{L}$  be the set of lonely centers in  $\widetilde{\mathcal{C}}$ . For each  $i = 1 \ldots t$ , we do the following. Let  $In$  be the set of optimal centers in  $\widetilde{\mathcal{O}}$  captured by  $c_i$ . Choose a set  $\mathcal{L}_i$  of  $|In| - 1$  centers from  $\mathcal{L}$ , set  $\mathcal{L} \gets \mathcal{L} \setminus \mathcal{L}_i$  and define  $Out = \mathcal{L}_i \cup \{c_i\}$ . Assign weight 1 to  $(In, Out)$  and add it to  $S$ . For each busy center  $c_i \in \{c_{t+1} \ldots c_{h'}\}$  let  $A$  be the set of optimal centers in  $\widetilde{\mathcal{O}}$  captured by  $c_i$ , pick a set  $\mathcal{L}_i$  of  $|A| - 1$  lonely current centers from  $\mathcal{L}$  (a counting argument shows that this is always possible). Set  $\mathcal{L} \gets \mathcal{L} \setminus \mathcal{L}_i$ . For each  $o_j \in A$  and  $c_\ell \in \mathcal{L}_i$  assign weight  $1 / (|A| - 1)$  to  $(o_j, c_\ell)$  and add it to  $S$ . Suppose we are left with  $\ell$  centers  $o_1' \ldots o_\ell' \in \widetilde{\mathcal{O}}$  such that  $C[o_i'] \notin \widetilde{\mathcal{C}}$ . Apparently, we have not included any  $o_i'$  in any swap yet. However, since  $|\widetilde{\mathcal{C}}| \geq |\widetilde{\mathcal{O}}|$ , we are left with at least  $\ell' \geq \ell$  lonely centers  $c_1' \ldots c_{\ell'}' \in \widetilde{\mathcal{C}}$ . For each  $i = 1 \ldots \ell$  we assign weight 1 to  $(o_i', c_i')$  and add it to  $S$ .
Observation 7. The process above generates a set of weighted ideal multi-swaps such that: (i) Every swap has size at most  $p$ ; (ii) The combined weights of swaps involving an optimal center  $o_i \in \widetilde{\mathcal{O}}$  is 1; (iii) The combined weights of swaps involving a current center  $c_i$  is at most  $1 + 1/p$ .
Consider an ideal swap  $(In, Out)$ . Let  $O_{In}^{*} = \bigcup_{o_{i} \in In} O_{i}^{*}$  and  $C_{Out} = \bigcup_{c_{j} \in Out} C_{j}$ . Define the reassignment cost  $\text{Reassign}(In, Out)$  as the increase in cost of reassigning points in  $C_{Out} \setminus O_{In}^{*}$  to centers in  $\mathcal{C} \setminus Out$ . Namely,
$$
\operatorname {R e a s s i g n} (I n, O u t) = \operatorname {c o s t} \left(C _ {O u t} \backslash O _ {I n} ^ {*}, \mathcal {C} \backslash O u t\right) - \operatorname {c o s t} \left(C _ {O u t} \backslash O _ {I n} ^ {*}, \mathcal {C}\right).
$$
We take the increase in cost of the following reassignment as an upper bound to the reassignment cost. For each  $p \in C_{Out} \setminus O_{In}^{*}$  we consider its closest optimal center  $\mathcal{O}^{*}[p]$  and reassign  $p$  to the current center that is closest to  $\mathcal{O}^{*}[p]$ , namely  $\mathcal{C}[\mathcal{O}^{*}[p]]$ . In formulas, we have
$$
\begin{array}{l} \operatorname {R e a s s i g n} (I n, O u t) \leq \sum_ {p \in C _ {O u t} \backslash O _ {I n} ^ {*}} \operatorname {c o s t} (p, \mathcal {C} [ \mathcal {O} ^ {*} [ p ] ]) - \operatorname {c o s t} (p, \mathcal {C} [ p ]) \\ \leq \sum_ {p \in C _ {O u t}} \operatorname {c o s t} (p, \mathcal {C} [ \mathcal {O} ^ {*} [ p ] ]) - \operatorname {c o s t} (p, \mathcal {C} [ p ]). \\ \end{array}
$$
Indeed, by the way we defined our ideal swaps we have  $\mathcal{C}[\mathcal{O}^*[p]] \notin Out$  for each  $p \notin O_{In}^*$  and this reassignment is valid. Notice that the right hand side in the equation above does not depend on  $In$ .
$$
\text {L e m m a} 8. \sum_ {p \in P} \operatorname {c o s t} (p, \mathcal {C} [ \mathcal {O} ^ {*} [ p ] ]) \leq 2 O P T + A L G + 2 \sqrt {A L G} \sqrt {O P T}.
$$
Proof. Deferred to the supplementary material.
Lemma 9. The combined weighted reassignment costs of all ideal multi-swaps in  $S$  is at most  $(2 + 2 / p) \cdot (OPT + \sqrt{ALG} \sqrt{OPT})$ .
Proof. Denote by  $w(In, Out)$  the weight associated with the swap  $(In, Out)$ .