SlowGuess/NeurIPS_2023_OCR huggingface dataset:
- The average cost of cluster  $C_i$  is denoted by  $\kappa_{C_i} := \frac{\operatorname{Cost}(C_i, \mathcal{A})}{|C_i|}$ .
- For any  $i,j$ , the ring  $R_{i,j}$  is the set of points  $x \in C_i$  such that  $2^j \kappa_{C_i} \leq \operatorname{Cost}(x, \mathcal{A}) < 2^{j+1} \kappa_{C_i}$ . For any  $j$ ,  $R_j = \cup R_{i,j}$ .
- The inner ring  $R_{I}(C_{i}) = \cup_{j\leq 2z\log \frac{\varepsilon}{2}}R_{i,j}$  is the set of points of  $C_i$  with cost at most  $\left(\frac{\varepsilon}{z}\right)^{2z}\kappa_{C_i}$ . More generally for a solution  $S$ , let  $R_I^S$  denote the union of the inner rings induced by  $S$ .
- The outer ring  $R_O(C_i) = \cup_{j \geq 2z \log \frac{z}{\varepsilon}} R_{i,j}$  is the set of points of  $C_i$  with cost at least  $\left(\frac{z}{\varepsilon}\right)^{2z} \kappa_{C_i}$ . More generally for a solution  $S$ , let  $R_O^S$  denote the union of the outer rings induced by  $S$ .
- The main ring  $R_{M}(C_{i})$  is the set of points of  $C_i$  that are not in the inner or outer rings, i.e.,  $R_{M}(C_{i}) = C_{i} \setminus (R_{I}(C_{i}) \cup R_{O}(C_{i})$ .
- For any  $j$ , the group  $G_{j,b}$  consists of the  $(2^{b - 1} + 1)$ -th to  $(2^b)$ -th points of each ring  $R_{i,j}$  that arrive in  $S$ .
- For any  $j$ , we use  $G_{j,\min}$  to denote the union of the groups with the smallest costs, i.e.,
$$
G _ {j, \min } = \left\{x \mid \exists i, x \in R _ {i, j}, \operatorname {C o s t} \left(R _ {i, j}, \mathcal {A}\right) <   2 \left(\frac {\varepsilon}{4 z}\right) ^ {z} \frac {\operatorname {C o s t} \left(R _ {j} , \mathcal {A}\right)}{\beta k} \right\}.
$$
- The outer groups  $G_{b}^{O}$  partition the outer rings  $R_{O}^{A}$  so that
$$
G _ {b} ^ {O} = \left\{x \mid \exists i, x \in C _ {i}, \left(\frac {\varepsilon}{4 z}\right) ^ {z} \frac {\operatorname {C o s t} \left(R _ {O} ^ {A} , \mathcal {A}\right)}{\beta k} \cdot 2 ^ {b} \leq \operatorname {C o s t} \left(R _ {O} \left(C _ {i}\right), \mathcal {A}\right) <   \left(\frac {\varepsilon}{4 z}\right) ^ {z} \frac {\operatorname {C o s t} \left(R _ {O} ^ {A} , \mathcal {A}\right)}{\beta k} \cdot 2 ^ {b + 1} \right\}.
$$
We define  $G_{\mathrm{min}}^{O} = \cup_{b\leq 0}G_{b}^{O}$  and  $G_{\mathrm{max}}^{O} = \cup_{b\geq z\log \frac{4z}{\varepsilon}}G_{b}^{O}$ .
Algorithm 1 RINGSAMPLE
Input: Points  $x_{1},\ldots ,x_{n}\in [\Delta ]^{d}$
Output: A set  $W$  of weighted points and timestamps
1: Initiate an instance of  $(\alpha, \beta)$ -bicriteria algorithm MULTMEYERSON
2:  $\gamma \gets \frac{C\max(\alpha^2,\alpha^z)\beta}{\min(\varepsilon^2,\varepsilon^z)}\log^2\frac{1}{\varepsilon}\left(k\log |\mathbb{C}| + \log \log \frac{1}{\varepsilon} +\log n\right)\log^2\frac{1}{\varepsilon}$
3:  $W\gets \emptyset$
4: for each point  $x_{t}$ ,  $t \in [n]$  do
5: Let  $c_{i}$  be the center assigned for  $x_{t}$  by MULTMEYERSON
6: Let  $2^{j} \leq \| x_{t} - c_{i}\|_{2}^{z} < 2^{j + 1}$  for  $j \in \mathbb{Z}$ .
7: Let  $b \in \mathbb{Z}$  so that the number of points in  $R_{i,j}$  is between  $2^{b-1} + 1$  and  $2^b$ .
8: Let  $r_t$  be the number of points in  $G_{j,b}$  at time  $t$
9:  $p_x \gets \min \left(\frac{4}{r_t} \cdot \gamma \log n, 1\right)$
10: With probability  $p_x$ , add  $x$  to  $W$  with timestamp  $t$  and weight  $\frac{1}{p_x}$
11: return W
We then adapt the offline coreset construction of [27] to an online setting at the cost of logarithmic overheads, which suffice for our purpose. The algorithm (Algorithm 1) has the following guarantees:
Lemma 2.2. Let  $\mathbb{C}$  be an  $\mathcal{A}$ -approximate centroid set for a fixed group  $G$ . There exists an algorithm RINGSAMPLE that samples
$$
O \left(\frac {\max (\alpha^ {2} , \alpha^ {z}) \beta}{\min (\varepsilon^ {2} , \varepsilon^ {z})} \log^ {2} \frac {1}{\varepsilon} \left(k \log | \mathbb {C} | + \log \log \frac {1}{\varepsilon} + \log n\right) \log^ {2} n \log^ {2} \Delta \log^ {2} \frac {1}{\varepsilon}\right)
$$
points and with high probability, outputs  $a(1 + \varepsilon)$ -online coreset for the  $k$ -means clustering problem.
Informally, an approximate centroid set is a set of possible points so that taking the centers from this set generates an approximately accurate solution (see Appendix B for a formal definition). To bound  $\log |\mathbb{C}|$ , we construct and apply a terminal embedding to project each point to a lower dimension and then appeal to known bounds for approximate centroid sets in low-dimensional Euclidean, thereby giving our online coreset algorithm with the guarantees of Theorem 1.3.
Sliding window model. We first recall a standard approach for using offline coreset constructions for insertion-only streaming algorithms. Suppose there exists a randomized algorithm that produces an online coreset algorithm that uses  $S(n, \varepsilon, \delta)$  points for an input stream of length  $n$ , accuracy  $\varepsilon$ , and failure probability  $\delta$ , where for the ease of discussion, we omit additional dependencies. A standard approach for using coresets on insertion-only streams is the merge-and-reduce approach, which partitions the stream into blocks of size  $S\left(n, \frac{\varepsilon}{2\log n}, \frac{\delta}{\mathrm{poly}(n)}\right)$  and builds a coreset for each block. Each coreset is then viewed as the leaves of a binary tree with height at most  $\log n$ , since the binary tree has at most  $n$  leaves. Then at each level of the binary tree, for each node in the level, a coreset of size  $S\left(n, \frac{\varepsilon}{2\log n}, \frac{\delta}{\mathrm{poly}(n)}\right)$  is built from the coresets representing the two children of the node. Due to the mergeability property of coresets, the coreset at the root of the tree will be a coreset for the entire stream with accuracy  $\left(1 + \frac{\varepsilon}{2\log n}\right)^{\log n} \leq (1 + \varepsilon)$  and failure probability  $\delta$ .
This approach fails for sliding window algorithms because the elements at the beginning of the data stream can expire, and so coresets corresponding to earlier blocks of the stream may no longer accurate, which would result in the coreset at the root of the tree also no longer being accurate. On the other hand, suppose we partition the stream into blocks consisting of  $S\left(n,\frac{\varepsilon}{2\log n},\frac{\delta}{\mathrm{poly}(n)}\right)$  elements as before, but instead of creating an offline coreset for each block, we can create an online coreset for the elements in reverse. That is, since the elements in each block are explicitly stored, we can create offline an artificial stream consisting of the elements in the block in reverse and then give the artificial stream as input to the online coreset construction. Note that if we also first consider the "latter" coreset when merging two coresets, then this effectively reverses the stream. Moreover, by the correctness of the online coreset, our data structure provides correctness over any prefix of the reversed stream, or equivalently, any suffix of the stream and specifically, correctness over the sliding window. We thus further adapt the merge-and-reduce framework to show that randomized online coresets for problems in clustering can also be used to achieve randomized algorithms for the corresponding problems in the sliding window model. We formalize this approach in Algorithm 2.
Theorem 2.3. Let  $x_{1}, \ldots, x_{n}$  be a stream of points in  $[\Delta]^{d}$ ,  $\varepsilon > 0$ , and let  $X = \{x_{n - W + 1}, \ldots, x_{n}\}$  be the  $W$  most recent points. Suppose there exists a randomized algorithm that with probability at least  $1 - \delta$ , outputs an online coreset algorithm for a  $k$ -clustering problem with  $S(n, d, k, \varepsilon, \delta)$  points. Then there exists a randomized algorithm that with probability at least  $1 - \delta$ , outputs a coreset for the  $k$ -clustering problem in the sliding window model with  $O\left(S\left(n, d, k, \frac{\varepsilon}{\log n}, \frac{\delta}{n^2}\right) \log n\right)$  points.
By Theorem 1.3 and Theorem 2.3, we have:
Theorem 2.4. There exists an algorithm that samples  $\frac{k}{\min(\varepsilon^4, \varepsilon^{2 + z})}$  polylog  $\frac{n\Delta}{\varepsilon}$  points and with high probability, outputs a  $(1 + \varepsilon)$ -coreset to  $(k, z)$ -clustering in the sliding window model.
Using an offline algorithm for  $(k,z)$ -clustering for post-processing after the data stream, we have Theorem 1.1.
In this section, we conduct simple empirical demonstrations as proof-of-concepts to illustrate the benefits of our algorithm. Our empirical evaluations were conducted using Python 3.10 using a 64-bit
Algorithm 2 Merge-and-reduce framework for randomized algorithms in the sliding window model, using randomized constructions of online coresets
Input: A clustering function  $f$ , a set of points  $x_{1}, \ldots, x_{n} \subseteq \mathbb{R}^{d}$ , accuracy parameter  $\varepsilon > 0$ , failure probability  $\delta \in (0,1)$ , and window size  $W > 0$   
Output: An approximation of  $f$  on the  $W$  most recent points  
1: Let CORESET  $(X, n, d, k, \varepsilon, \delta)$  be an online coreset construction with  $S(n, d, k, \varepsilon, \delta)$  points on a set  $X \subseteq \mathbb{R}^{d}$   
2:  $m \gets O\left(S\left(n, d, k, \frac{\varepsilon}{\log n}, \frac{\delta}{n}\right) \log n\right)$   
3: Initialize blocks  $B_{0}, B_{1}, \ldots, B_{\log n} \gets \emptyset$   
4: for each point  $x_{t}$  with  $t \in [n]$  do  
5: if  $B_{0}$  does not contain  $m$  points then  
6: Prepend  $x_{t}$  to  $B_{0}$ , i.e.,  $B_{0} \gets \{x_{t}\} \cup B_{0}$   
7: else  
8: Let  $i$  be the smallest index such that  $B_{i} = \emptyset$   
9:  $B_{i} \gets \text{CORESET}(Y, n, d, k, \frac{\varepsilon}{\log n}, \frac{\delta}{n^{2}})$  for  $Y = B_{0} \cup \ldots \cup B_{i-1} \triangleright Y$  is an ordered set of weighted points  
10: for  $j = 0$  to  $j = i - 1$  do  
11:  $B_{j} \gets \emptyset$   
12:  $B_{0} \gets \{x_{t}\}$   
13: return the ordered set  $B_{\log n} \cup \ldots \cup B_{0}$
operating system on an AMD Ryzen 7 5700U CPU, with 8GB RAM and 8 cores with base clock 1.80 GHz. The general approach to our experiments is to produce a data stream  $S$  that defines dataset  $X$ , whose generation we describe below, as well as in Appendix F. We then compare the performance of a simplified version of our algorithm with various state-of-the-art baselines.
Baselines. Our first baseline (denoted off for offline) is the simple Lloyd's algorithm on the entire dataset  $X$ , with multiple iterations using the k-means++ initialization. This is a standard approach for finding a good approximation to the optimal clustering cost, because finding the true optimal centers requires exponential time. Because this offline Lloyd's algorithm has access to the entire dataset, the expected behavior is that this algorithm will have the best objective, i.e., smallest clustering cost. However, we emphasize that this algorithm requires storing the entire dataset  $X$  in memory and thus its input size is significantly larger than the sublinear space algorithms.
To compare with the offline Lloyd's algorithm, we run a number of sublinear space algorithms. These algorithms generally perform some sort of processing on the datastream  $X$  to create a coreset  $C$ . We normalize the space requirement of these algorithms by permitting each algorithm to store  $m$  points across specific ranges of  $m$ . We then run Lloyd's algorithm on the coreset  $C$ , with the same number of iterations using the k-means++ initialization.
Our first sublinear space algorithm is uniform sampling on the dataset  $X$ . That is, we form  $C$  by uniformly sampling  $m$  points from  $X$ , before running Lloyd's algorithm. We use uni to denote this algorithm whose first step is based on uniformly sampling. Our second sublinear space algorithm is the importance sampling approach used by histogram-based algorithms, e.g., [15, 11, 8]. These algorithms perform importance sampling, i.e., sample points into the coreset  $C$  with probability proportional to their distances from existing samples and delete points once the clustering cost of  $C$  is much higher than the clustering cost of the dataset  $X$ . We use hist(ogram) to denote this algorithm that is based on the histogram frameworks for sliding windows.
Our final algorithm is a simplification of our algorithm. As with the histogram-based algorithm, we perform importance sampling on the stream  $S$  to create the coreset  $C$  of size  $m$ . Thus we do not implement the ring and group sampling subroutines in our full algorithm. However, the crucial difference compared to the histogram-based approach is that we forcefully discard points of  $C$  that have expired. We use imp to denote this algorithm whose first step is based on importance sampling.
Dataset. We first describe the methodology and experimental setup of our empirical evaluation on a real-world dataset with an amount of synthetic noise before detailing the experimental results. The
first component of our dataset consists of the points of the SKIN (Skin Segmentation) dataset  $X^{\prime}$  from the publicly available UCI repository [6], which was also used in the experiments of [8]. The dataset  $X^{\prime}$  consists of 245,057 points with four features, where each point refers to a separate image, such that the first three features are constructed over BGR space, and the fourth feature is the label for whether or not the image refers to a skin sample. We subsequently pre-process each dataset to have zero mean and unit standard deviation in each dimension.
We then form our dataset  $X$  by augmenting  $X'$  with 201 points in four-dimensional space, where 100 of these points were drawn from a spherical Gaussian with unit standard deviation in each direction and centered at  $(-10, 10, 0, 0)$  and 100 of these points were drawn from a spherical Gaussian with unit standard deviation in each direction and centered at  $(10, -10, 0, 0)$ . The final point of  $X$  was drawn from a spherical Gaussian with unit standard deviation centered at  $(500, 500, 0, 0)$ . Thus our dataset  $X$  has dimensions  $n = 245, 258$  and  $d = 4$ . We then create the data stream  $S$  by prepending two additional points drawn from spherical Gaussians with standard deviation 2.75 centered at  $(-10, 10, 0, 0)$  and  $(-10, -10, 0, 0)$  respectively, so that the stream has length 245, 260. We set the window length to be 245, 258 in accordance with the "true" data set, so that the first two points of the stream will be expired by the data stream.
**Experimental setup.** For each of the instances of Lloyd's algorithm, either on the entire dataset  $X$  or the sampled coreset  $C$ , we use 10 iterations using the k-means++ initialization. While the offline Lloyd's algorithm stores the entire dataset  $X$  of 245, 258 points in memory, we only allow each of the sublinear-space algorithms to store a fixed  $m$  points. We compare the algorithms across  $m \in \{5, 10, 15, 20, 25, 30\}$  and  $k \in \{2, 3, 4, 5, 6, 7, 8, 9, 10\}$ . Note that in the original dataset, each of the points has a label for either skin or non-skin, which would be reasonable for  $k = 2$ . However, due to the artificial structure possibly induced by the synthetic noise, it also makes sense to other values of  $k$ . In particular, preliminary experiments from uniform sampling by the elbow method indicated that  $k = 3$  would be a reasonable setting. Thus we fix  $k = 3$  while varying  $m \in \{5, 10, 15, 20, 25, 30\}$  and we arbitrarily fix  $m = 25$  while varying  $k \in \{2, 3, 4, 5, 6, 7, 8, 9, 10\}$ .
Experimental results. For each choice of  $m$  and  $k$ , we ran each algorithm 30 times and tracked the resulting clustering cost. Our algorithm demonstrated superior performance than the other sublinear-space algorithms across all values of  $m \in \{5,10,15,20,25,30\}$  and  $k \in \{2,3,4,5,6,7,8,9,10\}$ , and was even quite competitive with the offline Lloyd's algorithm, even though our algorithm only used memory size  $m \leq 30$ , while the offline algorithm used memory 245, 258.
Uniform sampling performed well for  $k = 2$ , which in some case captures the structure imposed on the data through the skin vs. non-skin label, but for larger  $k$ , the optimal solutions start placing centers to handle the synthetic noise, which may not be sampled by uniform sampling. Thus uniform sampling performed relatively poorly albeit quite stably for larger  $k$ . In contrast, the histogram-based algorithm performed poorly for small  $k$  across all our ranges of  $m$ , due to sampling the extra points in  $S \setminus X$ , so that the resulting Lloyd's algorithm on  $C$  moved the centers far away from the optimal centers of  $X$ . On the other hand, the histogram-based algorithm performed well for larger  $k$ , likely due to additional centers that could be afforded to handle the points in  $S \setminus X$ . We plot our results in Figure 1 and defer additional experiments to Appendix F.
In this paper, we give an algorithm that outputs a  $(1 + \varepsilon)$ -approximation to  $(k,z)$ -clustering in the sliding window model, while using  $\frac{k}{\min(\varepsilon^4,\varepsilon^{2 + z})}$  polylog  $\frac{n\Delta}{\varepsilon}$  words of space when the points are from  $[\Delta]^d$ . Our algorithm not only improves on a line of work [5, 16, 34, 8, 35], but also nearly matches the space used by the offline coreset constructions of [27], which is known to be near-optimal in light of a  $\Omega\left(\frac{k}{\varepsilon^{2 + z}}\right)$  lower bound for the size of an offline coreset [45].
We also give a lower bound that shows a logarithmic overhead in the number of points is needed to maintain a  $(1 + \varepsilon)$ -online coreset compared to a  $(1 + \varepsilon)$ -coreset. That is, we gave in Theorem 1.4 a set  $X \subset [\Delta]^d$  of  $n$  points  $x_1, \ldots, x_n$  such that any  $(1 + \varepsilon)$ -online coreset for  $k$ -means clustering on  $X$  requires  $\Omega\left(\frac{k}{\varepsilon^2} \log n\right)$  points. However, this does not rule out whether the  $\log n$  overhead is necessary for  $(k, z)$ -clustering in the sliding window model, since a sliding window algorithm does
(a) Comparisons for varying  $k$
(b) Comparisons for varying  $m$
Fig. 1: Comparison of average clustering costs made by uniform sampling, histogram-based algorithm, and our coreset-based algorithm across various settings of space allocated to the algorithm, given a synthetic dataset. For comparison, we also include the offline k-means++ algorithm as a baseline, though it is inefficient because it stores the entire dataset.
not necessarily need to maintain an online coreset. We leave this question as a possible direction for future work.
David P. Woodruff and Samson Zhou were partially supported by a Simons Investigator Award and by the National Science Foundation under Grant No. CCF-1815840. This work was done in part while Samson Zhou was at Carnegie Mellon University, UC Berkeley, and Rice University.
[1] Marcel R. Ackermann, Marcus Martens, Christoph Raupach, Kamil Swierkot, Christiane Lammersen, and Christian Sohler. Streamkm++: A clustering algorithm for data streams. ACM J. Exp. Algorithmics, 17(1), 2012. 2
[2] Miklós Ajtai, Vladimir Braverman, T. S. Jayram, Sandeep Silwal, Alec Sun, David P. Woodruff, and Samson Zhou. The white-box adversarial data stream model. In PODS '22: International Conference on Management of Data, 2022, pages 15–27, 2022. 2
[3] Apple. https://images.apple.com/privacy/docs/Differential_Privacy_Overview.pdf. 2
[4] Brian Babcock, Shivnath Babu, Mayur Datar, Rajeev Motwani, and Jennifer Widom. Models and issues in data stream systems. In Proceedings of the Twenty-first ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems, pages 1-16, 2002. 2
[5] Brian Babcock, Mayur Datar, Rajeev Motwani, and Liadan O'Callaghan. Maintaining variance and k-midians over data stream windows. In Proceedings of the Twenty-Second ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems, pages 234–243, 2003. 2, 3, 9
[6] Rajen Bhatt and Abhinav Dhall. Skin segmentation dataset. UCI Learning Repository. 9
[7] Jeremiah Blocki, Seunghoon Lee, Tamalika Mukherjee, and Samson Zhou. Differentially private  $l_{2}$ -heavy hitters in the sliding window model. In The Eleventh International Conference on Learning Representations, ICLR, 2023. 2
[8] Michele Borassi, Alessandro Epasto, Silvio Lattanzi, Sergei Vassilvitskii, and Morteza Zadimoghaddam. Sliding window algorithms for k-clustering problems. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems, NeurIPS, 2020. 2, 5, 8, 9, 16
[62] Zhewei Wei, Xuancheng Liu, Feifei Li, Shuo Shang, Xiaoyong Du, and Ji-Rong Wen. Matrix sketching over sliding windows. In Proceedings of the 2016 International Conference on Management of Data, SIGMOD Conference, pages 1465-1480. ACM, 2016. 2
[63] David P. Woodruff and Samson Zhou. Tight bounds for adversarially robust streams and sliding windows via difference estimators. In 62nd IEEE Annual Symposium on Foundations of Computer Science, FOCS, pages 1183-1196, 2021. 2
For a positive integer  $n$ , we use the notation  $[n]$  to denote the set  $\{1, \dots, n\}$ . Similarly, we use  $[\Delta]^d$  to denote  $\{1, \dots, \Delta\}^d$ . We use  $\mathrm{poly}(n)$  to denote a fixed polynomial in  $n$  with degree determined as necessary by setting the appropriate constants in corresponding variables. Similarly, we use  $\mathrm{polylog}(n)$  to denote  $\mathrm{poly}(\log n)$ . We suppress polylogarithmic dependencies by writing  $\tilde{O}\left(f(\cdot)\right) = O\left(f(\cdot)\right)$  polylog  $f(\cdot)$ .
For  $(k,z)$ -clustering on a set  $X = \{x_{1},\ldots ,x_{n}\} \subset \mathbb{R}^{d}$  using a set  $C$  of  $k$  centers and a distance function  $\mathrm{dist}(\cdot ,\cdot)$ , we define the notation  $\operatorname {Cost}(X,C) = \sum_{i = 1}^{n}\min_{c\in C}\mathrm{dist}(x_i,c)^z$ . We also define the notation  $\operatorname {Cost}_{|S|\leq k}(X,S)\coloneqq \min_{S:|S|\leq k}\operatorname {Cost}(X,S)$ , so that  $\operatorname {Cost}_{|S|\leq k}$  is the cost of an optimal  $(k,z)$ -clustering.
Definition A.1  $((\alpha, \beta)$ -approximation). We say a set of centers  $C$  provides an  $(\alpha, \beta)$ -approximation to the optimal  $k$ -means clustering on a set  $X$  if  $|C| \leq \beta k$  and
$$
\operatorname {C o s t} (X, C) \leq \alpha \text {O P T}.
$$
Definition A.2 (Coreset). A coreset for  $(k,z)$ -clustering on an approximation parameter  $\varepsilon > 0$  and a set  $X$  of points  $x_1, \ldots, x_n \in \mathbb{R}^d$  with distance function  $\text{dist}$  is a subset  $S$  of weighted points of  $X$  with weight function  $w$  such that for any set  $C$  of  $k$  points, we have
$$
(1 - \varepsilon) \sum_ {i = 1} ^ {n} \operatorname {d i s t} (x _ {i}, C) ^ {z} \leq \sum_ {q \in S} w (q) \operatorname {d i s t} (q, S) ^ {z} \leq (1 + \varepsilon) \sum_ {i = 1} ^ {n} \operatorname {d i s t} (x _ {i}, C) ^ {z}.
$$
Definition A.3 (Online Coreset). An online coreset for  $(k,z)$ -clustering on an approximation parameter  $\varepsilon > 0$  and a set  $X$  of points  $x_{1},\ldots ,x_{n}\in \mathbb{R}^{d}$  with distance function dist is a subset  $S$  of weighted points of  $X$  with weight function  $w$  such that for any set  $C$  of  $k$  points and for any  $t\in [n]$ , we have
$$
(1 - \varepsilon) \sum_ {i = 1} ^ {t} \operatorname {d i s t} (x _ {i}, C) ^ {z} \leq \sum_ {q \in S _ {t}} w (q) \operatorname {d i s t} (q, S _ {t}) ^ {z} \leq (1 + \varepsilon) \sum_ {i = 1} ^ {t} \operatorname {d i s t} (x _ {i}, C) ^ {z},
$$
where  $S_{t} = S \cap \{X_{1},\ldots ,X_{t}\}$ , i.e., the subset of  $S$  that has arrived at time  $t$ .
Theorem A.4 (Bernstein's inequality). Let  $X_{1}, \ldots, X_{n}$  be independent random variables such that  $\mathbb{E}[X_i^2] < \infty$  and  $X_{i} \geq 0$  for all  $i \in [n]$ . Let  $X = \sum_{i} X_{i}$  and  $\gamma > 0$ . Then
$$
\Pr \left[ X \leq \mathbb {E} [ X ] - \gamma \right] \leq \exp \left(\frac {- \gamma^ {2}}{2 \sum_ {i} \mathbb {E} \left[ X _ {i} ^ {2} \right]}\right).
$$