SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Input: A clustering function  $f$ , a set of points  $x_{1},\ldots ,x_{n}\subseteq \mathbb{R}^{d}$ , accuracy parameter  $\varepsilon >0$ , failure probability  $\delta \in (0,1)$ , and window size  $W > 0$
Output: An approximation of  $f$  on the  $W$  most recent points
1: Let  $\mathrm{CORESET}(X, n, d, k, \varepsilon, \delta)$  be an online coreset construction with  $S(n, d, k, \varepsilon, \delta)$  points on a set  $X \subseteq \mathbb{R}^d$
2:  $m \gets O\left(S\left(n, d, k, \frac{\varepsilon}{\log n}, \frac{\delta}{n}\right) \log n\right)$
3: Initialize blocks  $B_0, B_1, \ldots, B_{\log n} \gets \emptyset$
4: for each point  $x_{t}$  with  $t \in [n]$  do
5: if  $B_0$  does not contain  $m$  points then
6: Prepend  $x_{t}$  to  $B_{0}$ , i.e.,  $B_{0} \gets \{x_{t}\} \cup B_{0}$
7:
8: Let  $i$  be the smallest index such that  $B_{i} = \emptyset$
9:  $B_{i}\gets \mathrm{CORESET}\left(Y,n,d,k,\frac{\varepsilon}{\log n},\frac{\delta}{n^{2}}\right)$  for  $Y = B_0\cup \ldots \cup B_{i - 1}$ $\triangleright Y$  is an ordered set of weighted points
10: for  $j = 0$  to  $j = i - 1$  do
11:  $B_{j}\gets \emptyset$
12:  $B_0 \gets \{x_t\}$
13: return the ordered set  $B_{\log n} \cup \ldots \cup B_0$
Proof. Consider Algorithm 2. Let  $\mathrm{CORESET}(X, n, d, k, \varepsilon, \delta)$  be a randomized algorithm that, with probability at least  $1 - \delta$ , computes an online coreset for a  $k$ -clustering problem  $f$  with  $S(n, d, k, \varepsilon, \delta)$  points.
We first claim that for each  $B_i$  is a  $\left(1 + \frac{\varepsilon}{\log n}\right)^i$  online coreset for  $2^{i-1}m$  points. To that end, observe that  $B_i$  can only be non-empty if at some time,  $B_0$  contains  $m$  points and  $B_1, \ldots, B_{i-1}$  are all non-empty. By the correctness of the subroutine CORESET,  $B_i$  is a  $\left(1 + \frac{\varepsilon}{\log n}\right)$  online coreset for the points in  $B_0 \cup \ldots \cup B_{i-1}$  at some point during the stream. Hence by induction,  $B_i$  is a  $\left(1 + \frac{\varepsilon}{\log n}\right)\left(1 + \frac{\varepsilon}{\log n}\right)^{i-1} = \left(1 + \frac{\varepsilon}{\log n}\right)^i$  coreset for  $m + \sum_{j=1}^{i-1} 2^{j-1}m = 2^{i-1}m$  points.
Now, because Algorithm 2 inserts the newest points at the beginning of  $B_0$ , then the stream is fed in reverse to the merge-and-reduce procedure. Thus, for any  $W \in [2^{i-1}, 2^i)$ ,  $B_0 \cup \ldots \cup B_i$  provides an online coreset  $k$ -clustering for the  $W$  most recent points in the stream.
To analyze the probability of failure, we remark that there are at most  $n$  points in the stream. For each point, there are at most  $n$  coresets constructed by the subroutine CORESET (in fact, the number of coreset constructions is upper bounded by  $O(\log n)$ ). Since each subroutine is called with failure probability  $\frac{\delta}{n^2}$ , then by a union bound, the total failure probability is at most  $\delta$ .
To analyze the space complexity, note that there are at most  $O(\log n)$  coreset constructions  $B_0, \ldots, B_{\log n}$  maintained by the algorithm. Each coreset construction samples  $S\left(n, d, k, \frac{\varepsilon}{\log n}, \frac{\delta}{n^2}\right)$  points. Hence, the total number of sampled points is  $O\left(S\left(n, d, k, \frac{\varepsilon}{\log n}, \frac{\delta}{n^2}\right) \log n\right)$ .
Using an offline algorithm for  $(k,z)$ -clustering for post-processing after the data stream, we have:
In this section, we show that any  $(1 + \varepsilon)$ -online coreset for  $(k,z)$ -clustering requires  $\Omega\left(\frac{k}{\varepsilon^2}\log n\right)$  points. The intuition is somewhat straightforward and in a black-box manner. [25] showed the existence of a set  $X$  of  $\Omega\left(\frac{k}{\varepsilon^2}\right)$  unit vectors such that any sublinear space data structure would not be able to accurately determine  $\operatorname{Cost}(C,X)$  for a set of  $k$  unit vectors  $C$ . They thus showed that any offline  $(1 + \varepsilon)$ -coreset construction for  $(k,z)$ -clustering required  $\Omega\left(\frac{k}{\varepsilon^2}\right)$  points.
Because an online  $(1 + \varepsilon)$ -coreset must answer queries on all prefixes of the stream, our goal is to essentially embed  $\Omega (\log n)$  instances of the hard instance of [25] into the stream, which would require  $\Omega \left(\frac{k}{\varepsilon^2}\log n\right)$  points. To enforce the data structure to sample  $\Omega \left(\frac{k}{\varepsilon^2}\right)$  points for each of the hard instance, we give each of the instances increasingly exponential weight. That is, we give the points in the  $i$ -th instance  $\tau^i$  weight for some constant  $\tau > 1$ , by inserting  $\tau^i$  copies of each of the points. Because the weight of the  $i$ -th instance is substantially greater than the sum of the weights of the previous instances, any  $(1 + \varepsilon)$ -online coreset must essentially be a  $(1 + \varepsilon)$ -coreset for the  $i$ -th instance, thus requiring  $\Omega \left(\frac{k}{\varepsilon^2}\right)$  points for the  $i$ -th instance. This reasoning extends to all of the  $\Omega (\log n)$  instances, thereby giving a lower bound of  $\Omega \left(\frac{k}{\varepsilon^2}\log n\right)$  points.
We first recall the following offline coreset lower bound by [25].
Theorem D.1. [25] For  $d = \Theta\left(\frac{k}{\varepsilon^2}\right)$ , let  $X = e_1, \ldots, e_d \in \mathbb{R}^{2d}$  be the set of elementary vectors. Let  $z$  be a constant and let  $a_1, \ldots, a_m \in \mathbb{R}^{2d}$  with corresponding weights  $w_1, \ldots, w_m \in \mathbb{R}$  be a weighted set  $P$  of points. Then there exists a set of  $k$  unit vectors  $C = c_1, \ldots, c_k \in \mathbb{R}^{2d}$  such that for  $m = o\left(\frac{k}{\varepsilon^2}\right)$ ,
(1)  $\operatorname{Cost}(C, X) = \sum_{i=1}^{d} \min_{j \in [k]} \| e_i - c_j \|^2 \geq 2^{z/2} d - 2^{z/2} \cdot \max(1, z/2) \cdot \sqrt{dk}$ .
(2)  $\operatorname{Cost}(C, P) = \sum_{i=1}^{m} w_i \min_{j \in [k]} \|a_i - c_j\|_2^2 < (1 - \varepsilon)(2^{z/2}d - 2^{z/2} \cdot \max(1, z/2) \cdot \sqrt{dk})$ .
We remark that the first property is due to Lemma 31 pf [25] and the second property is due to Lemma 33 and Lemma 34 of [25].
Let  $\gamma = \Theta\left(\log \frac{n}{d'}\right)$ . Let  $d' = \Theta\left(\frac{k}{\varepsilon^2}\right)$  be the dimension of the hard instance in Theorem D.1 and set  $d = \gamma d'$ , so that we can partition the space  $\mathbb{R}^{2d}$  into  $\gamma$  groups of  $2d'$  coordinates.
We define a stream by creating  $\gamma$  weighted instances of the hard instance defined in Theorem D.1. Each of the  $\gamma$  hard instances will be embedded into a separate partition of  $2d^{\prime}$  coordinates of  $\mathbb{R}^{2d}$ . Namely, the first instance consists of the vectors  $e_1,\ldots ,e_{d'}$  being inserted into the stream. By Theorem D.1, any  $(1 + \varepsilon)$ -coreset must contain  $\Omega\left(\frac{k}{\varepsilon^2}\right)$  points. The next instance consists of the
vectors  $e_{1 + 2d'}$ , ...,  $e_{3d'}$  each being inserted  $\tau = 100$  times into the stream. That is, after the vector  $e_{d'}$  arrives in the stream from the first hard instance, then  $t$  copies of  $e_{1 + 2d'}$  arrive in the stream, followed by then  $t$  copies of  $e_{2 + 2d'}$ , and so forth. Due to the weights of these vectors, any  $(1 + \varepsilon)$ -coreset must essentially be a  $(1 + \varepsilon)$ -coreset for the second hard instance and thus contain  $\Omega\left(\frac{k}{\varepsilon^2}\right)$  points with support in the second group of  $2d'$  coordinates.
More generally, for each  $i \in [\gamma]$ , the stream inserts  $t^{i-1}$  copies of  $e_{1+2(i-1)d'}$ , followed by  $\tau^{i-1}$  copies of  $e_{2+2(i-1)d'}$ , and so on. The main intuition is that due to the weights of the  $i$ -th group of  $d'$  elementary vectors, an  $(1+\varepsilon)$ -online coreset must contain a  $(1+\varepsilon)$ -coreset for the  $i$ -th hard instance. Moreover, since the  $(1+\varepsilon)$ -online coreset must be a coreset for any prefix of the stream, then it needs to be a  $(1+\varepsilon)$ -coreset for each of the hard instances. Hence, the online coreset must contain  $\gamma \cdot \Omega\left(\frac{k}{\varepsilon^2}\right) = \Omega\left(\frac{k}{\varepsilon^2} \cdot \frac{\log n}{\log \frac{1}{\varepsilon}}\right)$  points.
Lemma D.2. Let  $\tau = 100$ . For each integer  $i > 0$ , let  $S_{i}$  be the stream that consists of  $\tau^{i - 1}$  consecutive copies of  $e_{1 + 2(i - 1)d'}$ , followed by  $\tau^{i - 1}$  copies of  $e_{2 + 2(i - 1)d'}$ , and so on. Let  $S$  be the stream that consists of  $S_{1} \circ S_{2} \circ \ldots$ . Then for each  $i$ , any  $(1 + \varepsilon)$ -online coreset after the arrival of  $S_{i}$  must consist of  $i \cdot \Omega\left(\frac{k}{\varepsilon^2}\right)$  points.
Proof. We prove the claim by induction on  $i$ . The base case of  $i = 1$  follows from Theorem D.1.
Now suppose the claim holds for a fixed  $i - 1$ . Let  $X_{i}$  be the set of points that have arrived after  $S_{i}$ , i.e.,  $X_{i} = S_{1} \circ \ldots \circ S_{i}$ . Let  $C_{i-1}$  be any  $(1 + \varepsilon)$ -online coreset for  $S$  after the arrival of  $S_{i-1}$ . Let  $P_{i}$  be a set of weighted points sampled during stream  $S_{i}$ , so that  $C_{i} = C_{i-1} \cup P_{i}$ . Since each point in  $S_{i}$  has weight  $\tau^{i}$ , then by scaling the first property of Theorem D.1, we have that there exists a set of  $k$  unit vectors  $U_{i} = c_{1}, \ldots, c_{k} \in \mathbb{R}^{2d}$  such that
$$
\begin{array}{l} \operatorname {C o s t} (U, X _ {i}) = \sum_ {a = 1} ^ {i} \sum_ {b = 1} ^ {d ^ {\prime}} \tau^ {a} \min  _ {j \in [ k ]} \| e _ {b + 2 (a - 1) d ^ {\prime}} - c _ {j} \| _ {2} ^ {z} \\ \geq \sum_ {b = 1} ^ {d ^ {\prime}} \tau^ {i} \min  _ {j \in [ k ]} \| e _ {b + 2 (i - 1) d ^ {\prime}} - c _ {j} \| _ {2} ^ {z} \\ \geq \left(\tau^ {i}\right) \left(2 ^ {z / 2} d - 2 ^ {z / 2} \cdot \max  (1, z / 2) \cdot \sqrt {d k}\right). \tag {1} \\ \end{array}
$$
In particular, the unit vectors  $U_{i} = c_{1},\ldots ,c_{k}$  have support entirely in the  $i$ -th group of  $2d^{\prime}$  coordinates in  $\mathbb{R}^{2d}$ . By the same argument, there exists a set  $U_{i - 1}$  with the same properties in the  $(i - 1)$ -th group of  $2d^{\prime}$  coordinates in  $\mathbb{R}^{2d}$ .
By the correctness of the online coreset, we have
$$
\operatorname {C o s t} \left(U _ {i - 1}, C _ {i - 1}\right) \leq (1 + \varepsilon) \operatorname {C o s t} \left(U _ {i - 1}, X _ {i - 1}\right) = (1 + \varepsilon) \sum_ {a = 1} ^ {i - 1} \operatorname {C o s t} \left(U _ {i - 1}, S _ {a}\right).
$$
Since  $U_{i-1}$  consists of unit vectors and each substream  $S_a$  consists of unit vectors, then we have
$$
\operatorname {C o s t} \left(U _ {i - 1}, S _ {a}\right) \leq 2 d ^ {\prime} \tau^ {a}.
$$
Thus for  $\varepsilon \in (0,1)$
$$
\operatorname {C o s t} \left(U _ {i - 1}, C _ {i - 1}\right) \leq 2 \sum_ {a = 1} ^ {i - 1} \left(2 d ^ {\prime} \tau^ {a}\right) \leq 8 d ^ {\prime} \tau^ {i - 1} <   \frac {1}{1 0} d ^ {\prime} \tau^ {i},
$$
since  $\tau = 100$ . On the other hand, since  $U_{i-1}$  has support entirely in the  $(i-1)$ -th group of  $2d'$  coordinates and  $S_i$  has support entirely in the  $i$ -th group of  $2d'$  coordinates in  $\mathbb{R}^{2d}$ , then
$$
\operatorname {C o s t} \left(U _ {i - 1}, X _ {i}\right) \geq \operatorname {C o s t} \left(U _ {i - 1}, S _ {i}\right) \geq 2 d ^ {\prime} \tau^ {i}.
$$
Thus for  $C_i$  to be a  $(1 + \varepsilon)$ -online coreset for  $\varepsilon \in (0, 1)$ ,  $C_i$  must sample additional points from  $X_i$  on top of  $C_{i-1}$ . Hence,  $P_i \neq \emptyset$ .
In particular, let  $P_{i}$  consist of vectors  $y_{1},\ldots ,y_{m}$  with weights  $w_{1},\ldots ,w_{m}$ . Since  $P_{i}\neq \emptyset$ , then
$$
\operatorname {C o s t} \left(U _ {i}, C _ {i}\right) = \operatorname {C o s t} \left(U, C _ {i - 1} \cup P _ {i}\right) \leq \operatorname {C o s t} \left(U, P _ {i}\right).
$$
If  $|P_i| = o\left(\frac{k}{\varepsilon^2}\right)$ , then by the second property of Theorem D.1, we have
$$
\mathrm {C o s t} (U _ {i}, P _ {i}) = \sum_ {b = 1} ^ {m} \min _ {j \in [ k ]} w _ {b} \| y _ {b} - c _ {j} \| _ {2} ^ {2} <   \tau^ {i} (1 - \varepsilon) (2 ^ {z / 2} d - 2 ^ {z / 2} \cdot \max (1, z / 2) \cdot \sqrt {d k}),
$$
which together with Equation 1 contradicts the fact that  $C_i$  is an  $(1 + \varepsilon)$ -online coreset for  $X_i$ .
Therefore, we have  $|P_i| = \Omega \left( \frac{k}{\varepsilon^2} \right)$ . Moreover, since  $P_i$  has disjoint support from  $C_{i-1}$ , then by induction,
$$
| C _ {i} | = | C _ {i - 1} \cup P _ {i} | = | C _ {i - 1} | + | P _ {i} | = i \cdot \Omega \left(\frac {k}{\varepsilon^ {2}}\right).
$$
Proof. Let  $\gamma = \Theta\left(\log \frac{n}{d^{\prime}}\right)$ . For each  $i \in [\gamma]$ , construct the stream  $S_{i}$  as in the statement of Lemma D.2. Observe that  $|S_{i}| = d^{\prime} \cdot t^{i}$  for  $t = 100$  and so under the settings of the parameter  $\gamma$  with the appropriate constant, the total length of the stream  $S = S_{1} \circ \ldots \circ S_{\gamma}$  is precisely  $n$ . Moreover, by Lemma D.2, any  $(1 + \varepsilon)$ -online coreset must store  $\gamma \cdot \Omega\left(\frac{k}{\varepsilon^{2}}\right) = \Omega\left(\frac{k}{\varepsilon^{2}} \log n\right)$  points for  $n = \mathrm{poly}(d)$ .
E On the Proof of Theorem B.2
We remark that Theorem 1 of [27] is stated for sampling a fixed number of points with replacement from each group, rather than sampling each point independently without replacement. By contrast, Theorem B.2 is stated for sampling each point independently without replacement. In this section, we briefly outline the proof of Theorem 1 of [27] and how the analysis translates to the statement of Theorem B.2.
At a high level, the coreset construction of [27] first collects rings of an approximate solution  $\mathcal{A}$  of  $k$  points into groups, using a similar approach to that described in Appendix B with  $\beta = 1$ . The algorithm then computes a coreset for each group first using a procedure GROUPSAMPLE and then using a procedure SENSITIVITYSAMPLE for some of the points not considered by the first procedure. We briefly describe both procedures, as well as how to adapt them to the setting where each point is sampled independently and without replacement.
E.1 Adaptation of Group Sampling
The GROUPSAMPLE procedure of [27] samples a fixed  $\Lambda_{1}$  number of points from each group  $G$  with probability proportional to the contribution of each corresponding cluster of the point to the group. That is, given clusters  $\widetilde{C_1},\ldots ,\widetilde{C_k}$  induced by  $\mathcal{A}$  on  $G$ , GROUPSAMPLE then performs  $\Lambda_{1}$  rounds of sampling. Each round samples a single point, where a point  $p\in \widetilde{C}_i$  is sampled proportional to  $\frac{\mathrm{Cost}(\widetilde{C_i},\mathcal{A})}{|\widetilde{C_i}|\cdot\mathrm{Cost}(G,\mathcal{A})}$  and rescaled appropriately. Then GROUPSAMPLE offers the following guarantees:
Lemma E.1 (Lemma 2 of [27]). Let  $(X,\mathrm{dist})$  be a metric space,  $k,z$  be positive integers,  $G$  be a group of clients and  $\mathcal{A}$  be an  $\alpha$ -approximate solution to  $(k,z)$ -clustering on  $G$  so that:
- For every cluster  $\widetilde{C}$  induced by  $\mathcal{A}$  on  $G$ , all points of  $\widetilde{C}$  contribute the same cost in  $\mathcal{A}$  up to a factor of 2.
- For all clusters  $\widetilde{C}$  induced by  $\mathcal{A}$  on  $G$ , we have that  $\frac{\operatorname{Cost}(G, \mathcal{A})}{2k} \leq \operatorname{Cost}(\widetilde{C}, \mathcal{A})$ .
Let  $\mathbb{C}$  be an  $\mathcal{A}$ -approximate centroid set for  $(k,z)$ -clustering on  $G$ .
Then there exists a procedure GROUPSAMPLE that constructs a set  $\Omega$  of size
$$
\Lambda_ {1} = O \left(\frac {\operatorname* {m a x} \left(\alpha^ {2} , \alpha^ {z}\right) \log^ {2} \frac {1}{\varepsilon}}{2 ^ {O (z \log z)} \operatorname* {m i n} \left(\varepsilon^ {2} , \varepsilon^ {z}\right)} \left(k \log | \mathbb {C} | + \log \log \frac {1}{\varepsilon} + \log n\right)\right),
$$
such that with high probability, it simultaneously holds for all sets  $S$  of  $k$  centers that
$$
| \operatorname {C o s t} (G, S) - \operatorname {C o s t} (\Omega , S) | \leq O \left(\frac {\varepsilon}{\alpha}\right) (\operatorname {C o s t} (G, S) + \operatorname {C o s t} (G, \mathcal {A}).
$$
We outline the high-level approach of the proof of Lemma E.1 and how can it can be adjusted for an  $(\alpha, \beta)$ -approximate solution  $\mathcal{A}$ , as well as a process that samples each point independently without replacement, rather than using  $\Lambda_1$  rounds as GROUPSAMPLE.
The proof of Lemma E.1 involves further partitioning the points of  $G$  into three subsets, based on the cost induced by the point. Namely, given a set  $S$  of  $k$  centers, a point  $p$  in group  $G$  is categorized as tiny, interesting, or huge, depending on  $\operatorname{Cost}(p, S)$  (though the interesting and huge points actually have a small overlap to allow slack in the proof). [27] applies standard Chernoff bounds to show that the number of sampled points is well-concentrated around its expectation and then applies Bernstein's inequality to show that the clustering costs of the tiny points, the interesting points are well-concentrated around their expectations. In particular, they show that the expected number of sampled points from each cluster  $\widetilde{C_i}$  is
$$
\frac {\Lambda_ {1} \operatorname {C o s t} (\widetilde {C _ {i}} , \mathcal {A})}{\operatorname {C o s t} (G , \mathcal {A})} \geq \frac {\Lambda_ {1}}{2 k},
$$
due to the assumption that for all clusters  $\widetilde{C}$  induced by  $\mathcal{A}$  on  $G$ , we have that  $\frac{\operatorname{Cost}(G, \mathcal{A})}{2k} \leq \operatorname{Cost}(\widetilde{C}, \mathcal{A})$ .
We first remark that if  $\mathcal{A}$  is an  $(\alpha, \beta)$ -approximate solution rather than an  $\alpha$ -approximate solution, i.e., if  $\mathcal{A}$  has  $\beta k$  centers rather than  $k$  centers, then the definition of the rings and groups would instead insist that for all clusters  $\widetilde{C}$  induced by  $\mathcal{A}$  on  $G$ , we have that  $\frac{\operatorname{Cost}(G, \mathcal{A})}{2\beta k} \leq \operatorname{Cost}(\widetilde{C}, \mathcal{A})$ . Then by oversampling  $\Lambda_1$  by a factor of  $\beta$ , i.e., sampling  $\beta \Lambda_1$  points would ensure that the expected number of sampled points from each cluster  $\widetilde{C}_i$  would be
$$
\frac {\beta \Lambda_ {1} \operatorname {C o s t} (\widetilde {C _ {i}} , \mathcal {A})}{\operatorname {C o s t} (G , \mathcal {A})} \geq \frac {\beta \Lambda_ {1}}{2 \beta k} = \frac {\Lambda_ {1}}{k}.
$$
It then remains to argue the correctness of sampling each point independently without replacement rather than a fixed  $\beta \Lambda_{1}$  number of points, which simply holds by adjusting the applications of the Chernoff bounds and Bernstein's inequality so that there is a separate random variable for each point in the input rather than for each of the  $\Lambda_{1}$  rounds.
E.2 Adaptation of Sensitivity Sampling
The SENSITIVITYSAMPLE procedure of [27] samples a fixed  $\Lambda_{2}$  number of points from each group  $G$  with probability proportional to the contribution of the point. Specifically, SENSITIVITYSAMPLE then performs  $\Lambda_{2}$  rounds of sampling, where each round samples a point  $p$  in the group  $G$  with probability proportional to  $\frac{\mathrm{Cost}(p,\mathcal{A})}{\mathrm{Cost}(G,\mathcal{A})}$  and rescales the sampled point appropriately. Then SENSITIVITYSAMPLE offers the following guarantee:
Lemma E.2 (Lemma 3 of [27]). Let  $(X,\mathrm{dist})$  be a metric space,  $k$ ,  $z$  be positive integers, and  $\mathcal{A}$  be an  $\alpha$ -approximate solution to  $(k,z)$ -clustering on  $G$ . Let  $\mathbb{C}$  be an  $\mathcal{A}$ -approximate centroid set for  $(k,z)$ -clustering on  $G$ . Let  $G$  be either a group  $G_b^O$  or  $G_{\max}^O$ . Then there exists a procedure SENSITIVITYSAMPLE that constructs a set  $\Omega$  of size
$$
\Lambda_ {2} = O \left(\frac {2 ^ {O (z \log z)} \alpha^ {2} \log^ {2} \frac {1}{\varepsilon}}{\varepsilon^ {2}} \left(k \log | \mathbb {C} | + \log \log \frac {1}{\varepsilon} + \log n\right)\right),
$$
$$
| \operatorname {C o s t} (G, S) - \operatorname {C o s t} (\Omega , S) | \leq O \left(\frac {\varepsilon}{\alpha z \log \frac {z}{\varepsilon}}\right) (\operatorname {C o s t} (G, S) + \operatorname {C o s t} (G, \mathcal {A}).
$$
We outline the high-level approach of the proof of Lemma E.2 and how can it can be adjusted for an  $(\alpha, \beta)$ -approximate solution  $\mathcal{A}$ , as well as a process that samples each point independently without replacement, rather than using  $\Lambda_{2}$  rounds as SENSITIVITYSAMPLE.
The proof of Lemma E.2 partitions the points of  $G$  into two categories, based on the cost induced by the point. Given a set  $S$  of  $k$  centers, the close points are the points  $p$  in  $G$  that have  $\operatorname{Cost}(p, S) \leq 4^z \cdot \operatorname{Cost}(p, \mathcal{A})$ . The far points are the remaining points in  $G$ , i.e., the points  $p$  in  $G$  with  $\operatorname{Cost}(p, S) > 4^z \cdot \operatorname{Cost}(p, \mathcal{A})$ .
[27] applies Bernstein's inequality to show that the clustering cost of the close points is well-concentrated around their expectations. We can again adjust the application of Bernstein's inequality so that there is a separate random variable for each point in the input rather than for each of the  $\Lambda_{2}$  samples.
To handle the far points, [27] again uses Bernstein's inequality to show that with high probability, the clustering points of these points with respect to  $S$  can be replaced with the distance to the closest center  $c \in \mathcal{A}$  plus the distance from  $c$  to the closest center in  $S$ . Conditioned on this event, the latter distance can then be charged to the remaining points of the cluster from the original dataset, i.e., the remaining points of the cluster not necessarily restricted to group  $G$ , which are significantly more numerous and already paying a similar value in  $S$ . In particular, Bernstein's inequality utilizes the fact that the second moment of the estimated cost of a cluster  $C$  is at most
$$
\frac {\operatorname {C o s t} (G , \mathcal {A})}{\Lambda_ {2} ^ {2}} \operatorname {C o s t} (C \cap G, \mathcal {A}) \leq \frac {2 k}{\Lambda_ {2} ^ {2}} (\operatorname {C o s t} (C \cap G, \mathcal {A})) ^ {2},
$$
for  $\beta = 1$ . Thus for general  $\beta$ , we recover the same guarantee by oversampling  $\Lambda_{2}$  by a factor of  $\beta$ , i.e., sampling  $\beta \Lambda_{2}$  points would ensure that the second moment would be at most  $\frac{2k}{\Lambda_2^2} \mathrm{Cost}^2 (C \cap G, \mathcal{A})$ . It then remains to argue the correctness of sampling each point independently without replacement rather than a fixed  $\beta \Lambda_{2}$  number of points, which again holds by adjusting the application of Bernstein's inequality so that there is a separate random variable for each point in the input rather than for each of the  $\Lambda_{2}$  rounds.
F Additional Experiments on Synthetic Data
We first describe the methodology and experimental setup of our empirical evaluation on a synthetic dataset before detailing the experimental results. To emphasize the benefits of our algorithm against worst-case input, we generate a synthetic dataset that would fully capture the failure cases of previous baselines.