SlowGuess/NeurIPS_2023_OCR huggingface dataset:
If  $X_{i} - \mathbb{E}[X_{i}]\leq \Delta$  for all  $i$ , then for  $\sigma_i^2 = \mathbb{E}[X_i^2 ] - \mathbb{E}[X_i]^2$
$$
\mathbf {P r} \left[ X \geq \mathbb {E} [ X ] + \gamma \right] \leq \exp \left(\frac {- \gamma^ {2}}{2 \sum_ {i} \sigma_ {i} ^ {2} + 2 \gamma \Delta / 3}\right).
$$
Meyerson sketch. We briefly review the Meyerson sketch [54] and the relevant properties that we need from the Meyerson sketch. The Meyerson sketch provides an  $(\alpha, \beta)$ -approximation to  $(k, z)$ -clustering on a data stream of points  $x_1, \ldots, x_n \in [\Delta]^d$  with  $\alpha = 2^{z+7}$  and  $\beta = O(2^{2z} \log n \log \Delta)$ . Moreover, for our purposes, it provides the crucial property that on the arrival of each point  $x_i$ , the algorithm irrevocably assigns  $x_i$  to one of the  $\beta k$  centers. Specifically, the clustering cost at the end of the stream is computed with respect to the center that  $x_i$  is assigned at time  $i$ , which may not be the closest center to  $x_i$  because the closer center can be opened at a later time.
For the ease of discussion, we describe the Meyerson sketch for  $z = 1$ ; the intuition generalizes naturally to other values of  $z$ . The Meyerson sketch performs via a guess-and-double approach, where it first attempts to guess the cost of the optimal clustering cost. Using the guess of the cost, it then turns each point into a center with probability proportional to the distance of that point from the existing centers. This subroutine is illustrated in Algorithm 3. If too many centers have been opened, then the Meyerson sketch determines that the guess for the optimal clustering cost must have been too low and increases the guess. The overall algorithm is given in Algorithm 4.
We require the following properties from the Meyerson sketch.
Algorithm 3 High probability MEYERSON  $(X, \widehat{OPT}, \alpha, \delta, \Delta, z, k)$  sketch
Input: Points  $X \coloneqq x_{1},\ldots ,x_{n}\in \mathbb{R}^{d}$  with aspect ratio  $\Delta$ , estimate  $\widetilde{\mathrm{OPT}}\geq 0$  such that  $\alpha \mathrm{OPT}\leq \widetilde{\mathrm{OPT}}\leq \mathrm{OPT}$  for some  $\alpha \in (0,1)$ , failure probability  $\delta \in (0,1)$
Output: A coreset for  $k$ -clustering on  $X$
1:  $\gamma \gets 2\log \frac{1}{\delta}$
2: for  $i\in [\gamma ]$  do
3: for  $\bar{t} \in [n]$  do
4: if  $t = 1$  then
5:  $M_{i}\gets x_{1},C_{\mu_{i}}\gets 0,w_{i}(x_{1}) = 1$
6: else
7: if  $|M_i| \leq 4k(1 + \log \Delta) \left( \frac{2^{z+3}}{\alpha^z} + 1 \right)$  then
8: With probability  $\min \left(\frac{k(1 + \log\Delta)\mathrm{dist}(x_t,M_i)^z}{\widetilde{\mathrm{OPT}}},1\right)$ , add  $x_{t}$  to  $M_{i}$  with weight 1, i.e.,  $w_{i}(x_{t}) = 1$
9: Otherwise, let  $z = \operatorname{argmin}_{y \in M_i} \mathrm{dist}(x_t, y)$ , increment the weight of  $z$ , i.e.,  $w_i(z) \gets w_i(z) + 1$ , and increase  $C_{\mu_i} \gets C \mu_i \mathrm{dist}(x_t, z)^p$
10: Let  $j = \operatorname*{argmin}_{i:|M_i|\leq 4k(1 + \log \Delta)\left(\frac{2^z + 3}{\alpha^z} + 1\right)} C_{\mu_i}$  be the index of the minimal cost sketch with at most  $4k(1 + \log \Delta)\left(\frac{2^{z + 3}}{\alpha^z} + 1\right)$  samples
11: return  $\cup_{i\in [\gamma ]}M_i,w_j$  , and  $C_{\mu_j}$
Algorithm 4 High probability MULTMEYERSON sketch
Output: A coreset for  $k$ -means clustering on  $X$  if  $\widehat{\mathrm{OPT}}$  upper bounds the cost of the optimal clustering
1:  $\gamma \gets \log nd(\Delta^z)$
2: for  $i\in [\gamma ]$  do
3: Run MEYERSON  $(X,2^{i},\alpha = \frac{1}{2},\delta ,\Delta ,z,k)$  in parallel
4: Let  $j$  be the minimal index in  $[\gamma]$  such that MEYERSON with input  $2^j$  has size smaller than  $8k\log \frac{1}{\delta} (1 + \log \Delta)(2^{2z + 3} + 1)$  and cost smaller than  $2^{z + 6 + j}$
5: return the output for MEYERSON  $(X,2^{j},\alpha = \frac{1}{2},\delta ,\Delta ,z,k)$
(1) on the arrival of each point  $x_{i}$ , assigns  $x_{i}$  to a center in  $C$  through a mapping  $\pi : X \to C$ , where  $C$  contains at most  $O\left(2^{2z}k\log n\log \Delta\right)$  centers
(2)  $\sum_{x\in X}\| x_i - \pi (x_i)\| _2^2\leq 2^{z + 7}\operatorname{Cost}_{|S|\leq k}(X,S)$
(3) MULTMEYERSON uses  $O\left(2^z k \log^3 (nd\Delta)\right)$  words of space
B Online  $(1 + \varepsilon)$ -Coreset
In this section, we describe how to construct an online  $(1 + \varepsilon)$  coreset for  $(k,z)$ -clustering under general discrete metrics. We first describe the offline coreset construction of [27] and then argue that the construction can be adapted to an online setting at the cost of logarithmic overheads, which suffice for our purpose.
Let  $\mathcal{A}$  be an  $(\alpha, \beta)$ -approximation for a  $(k, z)$ -clustering on an input set  $X \subseteq [\Delta]^d$  and let  $C_1, \ldots, C_{\beta k}$  be the clusters of  $X$  induced by  $\mathcal{A}$ . Suppose the points of  $X$  arrive in a data stream  $S$ . For a fixed  $\varepsilon > 0$ , [27] define the following notions of rings and groups:
- The average cost of cluster  $C_i$  is denoted by  $\kappa_{C_i} := \frac{\operatorname{Cost}(C_i, A)}{|C_i|}$ .
- For any  $i,j$ , the ring  $R_{i,j}$  is the set of points  $x \in C_i$  such that
$$
2 ^ {j} \kappa_ {C _ {i}} \leq \operatorname {C o s t} (x, \mathcal {A}) <   2 ^ {j + 1} \kappa_ {C _ {i}}.
$$
For any  $j$ ,  $R_{j} = \cup R_{i,j}$ .
- The inner ring  $R_{I}(C_{i}) = \cup_{j\leq 2z\log \frac{\varepsilon}{z}}R_{i,j}$  is the set of points of  $C_i$  with cost at most  $\left(\frac{\varepsilon}{z}\right)^{2z}\kappa_{C_i}$ . More generally for a solution  $S$ , let  $R_{I}^{S}$  denote the union of the inner rings induced by  $S$ .
- The outer ring  $R_O(C_i) = \cup_{j \geq 2\log \frac{z}{\varepsilon}} R_{i,j}$  is the set of points of  $C_i$  with cost at least  $\left(\frac{z}{\varepsilon}\right)^{2z} \kappa_{C_i}$ . More generally for a solution  $S$ , let  $R_O^S$  denote the union of the outer rings induced by  $S$ .
- The main ring  $R_M(C_i)$  is the set of points of  $C_i$  that are not in the inner or outer rings, i.e.,  $R_M(C_i) = \bar{C}_i \setminus (R_I(C_i) \cup R_O(C_i))$ .
- For any  $j$ , the group  $G_{j,b}$  consists of the  $(2^{b - 1} + 1)$ -th to  $(2^b)$ -th points of each ring  $R_{i,j}$  that arrive in  $S$ .
- For any  $j$ , we use  $G_{j,\min}$  to denote the union of the groups with the smallest costs, i.e.,
We remark that unlike the definition of [27],  $G_{j,\min}$  is a subset of the groups  $G_{j,b}$  with  $b \geq 1$ , but we shall nevertheless show that our sampling procedure preserves the cost contributed by each group. We also require the following slight variation of the definition of  $\mathcal{A}$ -approximate centroid set from [53] due to [27].
Definition B.1 ( $\mathcal{A}$ -approximate centroid set). Let  $X \subseteq \mathbb{R}^d$  be a set of points, let  $k, z$  be two positive integers, and let  $\varepsilon > 0$  be an accuracy parameter. Given a set  $\mathcal{A}$  of centers, we say a set  $\mathbb{C}$  is an  $\mathcal{A}$ -approximate centroid set for  $(k, z)$ -clustering on  $X$  if for every set of  $k$  centers  $\mathcal{S} \subseteq \mathbb{R}^d$ , there exists  $\widetilde{\mathcal{S}} \subseteq \mathbb{R}^d$  of  $k$  points such that for all  $x \in X$  with  $\mathrm{Cost}(x, \mathcal{S}) \leq \left(\frac{8z}{\varepsilon}\right)^z \mathrm{Cost}(x, \mathcal{A})$  or  $\mathrm{Cost}(x, \widetilde{\mathcal{S}}) \leq \left(\frac{8z}{\varepsilon}\right)^z \mathrm{Cost}(x, \mathcal{A})$ ,
$$
| \operatorname {C o s t} (x, \mathcal {S}) - \operatorname {C o s t} (x, \widetilde {\mathcal {S}}) | \leq \frac {\varepsilon}{z \log (z / \varepsilon)} (\operatorname {C o s t} (x, \mathcal {S}) - \operatorname {C o s t} (x, \mathcal {A}).
$$
The following statement is implied by the proof of Theorem 1 in [27].
Theorem B.2. [27, 24] Let  $z > 0$  be a constant. Let  $x \in G$  for a group induced by an  $(\alpha, \beta)$ -bicriteria assignment  $\mathcal{A}$ . For each cluster  $C_i$  with  $i \in [\beta k]$ , let  $D_i = C_i \cap G$ . Let  $\mathbb{C}$  be an  $\mathcal{A}$ -approximate centroid set for  $G$  and let
$$
\gamma = \frac {C \max  \left(\alpha^ {2} , \alpha^ {z}\right) \beta}{\min  \left(\varepsilon^ {2} , \varepsilon^ {z}\right)} \log^ {2} \frac {1}{\varepsilon} \left(k \log | \mathbb {C} | + \log \log \frac {1}{\varepsilon} + \log n\right) \log^ {2} \frac {1}{\varepsilon},
$$
for some sufficiently large constant  $C > 0$ . Let
$$
\zeta_ {x} = \frac {\operatorname {C o s t} (D _ {i} , \mathcal {A})}{| D _ {i} | \operatorname {C o s t} (G , \mathcal {A})} \cdot \gamma \log n, \qquad \eta_ {x} = \frac {\operatorname {C o s t} (x , \mathcal {A})}{\operatorname {C o s t} (G , \mathcal {A})} \cdot \gamma \log n.
$$
Suppose each point  $x \in X$  is sampled and reweighted independently into a set  $\Omega_0$  with probability  $p_x$ , where
$$
p _ {x} \geq \min  (\zeta_ {x} + \eta_ {x}, 1).
$$
Let  $\Omega_1 = \Omega_0 \setminus (R_I(C_i) \cup (C_i \cap \cup_j G_{j,\min}) \cup (R_O(C_i) \cap G_{\min}^O)$ .
Suppose  $\Omega_2$  is the set of centers in  $\mathcal{A}$ , where each center  $c_i$  with  $i \in [\beta k]$  has weight  $w_i$ , where  $w_i$  is a  $(1 + \varepsilon)$ -approximation to  $|R_I(C_i)| + |C_i \cap \cup_j G_{j,\min}| + |R_O(C_i) \cap G_{\min}^O|$ . Then  $(\Omega_1 \setminus \Omega_2) \cup \Omega_2$  is  $(1 + \varepsilon)$ -coreset for the  $(k, z)$ -clustering problem with probability  $1 - \frac{1}{\mathrm{poly}(n)}$ .
We first show that the sampling probabilities for each point in the stream by RINGSAMPLE in Algorithm 1 satisfies the conditions of Theorem B.2.
Lemma B.3. Let  $x \in G$  for a group induced by an  $(\alpha, \beta)$ -bicriteria assignment  $\mathcal{A}$  at a time  $t$ , with  $t \in [n]$ . For each cluster  $C_i$  with  $i \in [\beta k]$ , let  $D_i = C_i \cap G$ . Let  $\mathbb{C}$  be an  $\mathcal{A}$ -approximate centroid set for  $G$  and let
$$
\gamma = \frac {C \max (\alpha^ {2} , \alpha^ {z}) \beta}{\min (\varepsilon^ {2} , \varepsilon^ {z})} \log^ {2} \frac {1}{\varepsilon} \left(k \log | \mathbb {C} | + \log \log \frac {1}{\varepsilon} + \log n\right) \log^ {2} \frac {1}{\varepsilon},
$$
for some sufficiently large constant  $C > 0$  Let
$$
\zeta_ {x} = \frac {\operatorname {C o s t} (D _ {i} , \mathcal {A})}{| D _ {i} | \operatorname {C o s t} (G , \mathcal {A})} \cdot \gamma \log n, \quad \eta_ {x} = \frac {\operatorname {C o s t} (x , \mathcal {A})}{\operatorname {C o s t} (G , \mathcal {A})} \cdot \gamma \log n.
$$
Then the probability  $p_x$  that RINGSAMPLE (Algorithm 1) samples each point  $x$  satisfies
$$
p _ {x} \geq \min  \left(\zeta_ {x} + \eta_ {x}, 1\right).
$$
Proof. Suppose that  $x \in R_{i,j}$  and  $x \in G_{j,b}$  at time  $t$ , for some  $i \in [\beta k]$  in an assignment by  $\mathcal{A}$  from MULTMEYERSON. Let  $u$  be the time that  $x$  arrived in the stream. By the properties of the Meyerson sketch, i.e., MULTMEYERSON in Theorem 2.1,  $x$  is irrevocably assigned to a cluster  $C_i$  with  $i \in [\beta k]$  at time  $u$ . Hence,  $x$  must also be assigned to ring  $R_{i,j}$  at time  $u$ . Moreover, since the stream is insertion-only, then the number of points in all rings  $R_{i,j}$  for a fixed  $j$  across all  $i \in [\beta k]$  is monotonically non-decreasing. Thus  $x$  must also be assigned to group  $G_{j,b}$  at time  $u$ .
Let  $p_x$  be the sampling probability of  $x$  by RINGSAMPLE in Algorithm 1 at time  $u$ . We have that
$$
p _ {x} = \min \left(\frac {4}{r _ {u}} \cdot \gamma \log n, 1\right),
$$
where  $r_u$  is the number of points in  $G_{j,b}$  at time  $u$ . Let  $G_{j,b}^{(u)}$  be the subset of  $G_{j,b}$  that have arrived at time  $u$  and let  $G_{j,b}^{(t)}$  be the subset of  $G_{j,b}$  that have arrived at time  $t$ . Let  $c_i$  be the center assigned to point  $x$ , so that  $\mathrm{Cost}(x, c_i) = \mathrm{Cost}(x, \mathcal{A})$  and let  $C_i^{(u)}$  be the points assigned to  $c_i$  at time  $u$ . Similarly, let  $D_i^{(u)} = C_i^{(u)} \cap G_{j,b}^{(u)}$ . By the definition of  $R_{i,j}$  and  $G_{j,b}$ ,
$$
\frac {\| x - c _ {i} \| _ {2} ^ {\tilde {z}}}{\operatorname {C o s t} (G _ {j , b} ^ {(u)} , \mathcal {A})} \leq \frac {2 ^ {j + 1}}{\operatorname {C o s t} (G _ {j , b} ^ {(u)} , \mathcal {A})} \leq \frac {2 ^ {j + 1}}{r _ {u} \cdot 2 ^ {j}} = \frac {2}{r _ {u}}.
$$
Since both the cost of group  $G_{j,b}$  and the number of points in  $D_{i}$  is monotonically non-decreasing over time, then at time  $t$ , we have
$$
\frac {\zeta_ {x}}{\gamma \log n} = \frac {\operatorname {C o s t} (D _ {i} , \mathcal {A})}{| D _ {i} | \operatorname {C o s t} (G _ {j , b} , \mathcal {A})} \leq \frac {2 | D _ {i} | \| x - c _ {i} \| _ {2} ^ {z}}{| D _ {i} | \operatorname {C o s t} (G _ {j , b} ^ {(t)} , \mathcal {A})} \leq \frac {2 \| x - c _ {i} \| _ {2} ^ {2}}{\operatorname {C o s t} (G _ {j , b} ^ {(u)} , \mathcal {A})} \leq \frac {4}{r _ {u}}.
$$
Similarly, we have that due to the monotonicity of the cost of group  $G_{j,b}$  over time,
$$
\frac {\eta_ {x}}{\gamma \log n} = \frac {\| x - c _ {i} \| _ {2} ^ {z}}{\operatorname {C o s t} (G _ {j , b} ^ {(t)} , \mathcal {A})} \leq \frac {\| x - c _ {i} \| _ {2} ^ {z}}{\operatorname {C o s t} (G _ {j , b} ^ {(u)} , \mathcal {A})} \leq \frac {2}{r _ {u}}.
$$
Thus for sufficiently large constant  $C$  in the definition of  $\gamma$  in RINGSAMPLE, we have that
$$
p _ {x} \geq \min  \left(\zeta_ {x} + \eta_ {x}, 1\right),
$$
since  $p_x = \min \left(\frac{4}{r_u} \cdot \gamma \log n, 1\right)$ .
We next justify the space complexity of Algorithm 1, i.e., showing that with high probability, an upper bound of the number of samples can be determined.
Lemma B.4. RINGSAMPLE (Algorithm 1) samples
$$
O \left(\frac {\max  \left(\alpha^ {2} , \alpha^ {z}\right) \beta}{\min  \left(\varepsilon^ {2} , \varepsilon^ {z}\right)} \log^ {2} \frac {1}{\varepsilon} \left(k \log | \mathbb {C} | + \log \log \frac {1}{\varepsilon} + \log n\right) \log^ {2} n \log^ {2} \Delta \log^ {2} \frac {1}{\varepsilon}\right)
$$
points with high probability.
Proof. Recall that by definition, the groups  $G_{j,b}$  partition the points  $X = x_{1},\ldots ,x_{n}\subseteq [\Delta ]^{d}$ . For a fixed  $j$  and  $b$ , let  $Y_{i}$  be an indicator random variable for whether the  $i$ -th point of  $G_{j,b}$  is sampled by RINGSAMPLE. Then we have  $\mathbb{E}\left[Y_i\right]\leq \frac{4}{i}\cdot \gamma \log n$  and similarly  $\mathbb{E}\left[Y_i^2\right]\leq \frac{4}{i}\cdot \gamma \log n$ . By Bernstein's inequality, Theorem A.4, we have that
$$
\mathbf {P r} \left[ \sum Y _ {i} \geq 8 0 \gamma \log^ {2} n \right] \leq \frac {1}{n ^ {4}}
$$
and more generally, we have that  $\sum Y_{i} = O\left(\gamma \log^{2}n\right)$  with high probability. Thus by a union bound over all  $j$  and  $b$ , we have that the number of sampled points is at most
$$
O \left(\gamma \log^ {2} n \log^ {2} \Delta\right) = O \left(\frac {1}{\min  \left(\varepsilon^ {2} , \varepsilon^ {z}\right)} \log^ {2} \frac {1}{\varepsilon} \left(k \log | \mathbb {C} | + \log \log \frac {1}{\varepsilon} + \log n\right) \log^ {2} n \log^ {2} \Delta \log^ {2} \frac {1}{\varepsilon}\right)
$$
$$
\mathrm {f o r} \gamma = \frac {C \max (\alpha^ {2} , \alpha^ {z}) \beta}{\min (\varepsilon^ {2} , \varepsilon^ {z})} \log^ {2} \frac {1}{\varepsilon} \left(k \log | \mathbb {C} | + \log \log \frac {1}{\varepsilon} + \log n\right) \log^ {2} \frac {1}{\varepsilon}.
$$
Moreover, note that we can for all  $t \in [n]$ , we can explicitly track both  $|G_{j,b}^{(t)}|$  and  $\mathrm{Cost}(G_{j,b}^{(t)},\mathcal{A})$  as the stream is updated, because once the bicriteria algorithm assigns a point to a center in  $\mathcal{A}$ , the assignment will remain the same for the rest of the stream. Thus, we have the following:
Lemma B.5. For each  $j$  and  $b$ , there exists an algorithm that maintains both  $|G_{j,b}^{(t)}|$  and  $\mathrm{Cost}(G_{j,b}^{(t)},\mathcal{A})$  for all  $t\in [n]$  using  $O$  ( $\log (nd\Delta)$ ) space.
Putting things together, we give the full guarantees of RINGSAMPLE in Algorithm 1.
points and with high probability, outputs a  $(1 + \varepsilon)$ -online coreset for the  $k$ -means clustering problem.
Proof. Consider RINGSAMPLE. Before claiming the algorithm gives an  $(1 + \varepsilon)$ -online coreset, we first consider a fixed time  $t \in [n]$ . Then correctness at time  $t$  follows from applying Theorem B.2, given Lemma B.3 and Lemma B.5. We then observe that once a center is formed by RINGSAMPLE, i.e., once a point is sampled, then it irrevocably remains a center in the data structure. Therefore, conditioned on the correctness at time  $t$ , then the data structure will always correctly give an  $(1 + \varepsilon)$ -coreset to the prefix of  $t$  points in the stream at any later point  $t'$  in the stream,  $t' \in [n]$  with  $t' > t$ . It thus suffices to argue correctness over all  $t \in [n]$ , which requires a simple union bound. The space complexity follows from Lemma B.4 and Lemma B.5.
To apply Lemma 2.2, we require upper bounding the term  $\log |\mathbb{C}|$ . To that end, we first require the following definition of doubling dimension.
Definition B.6 (Doubling dimension). The doubling dimension of a metric space  $X$  with metric  $d$  is the smallest integer  $\ell$  such that for any  $x \in X$ , it is possible to cover the ball of radius  $2r$  around  $x$  with  $2^{\ell}$  balls of radius  $r$ .
Observe that general discrete metric spaces with  $n$  points have doubling dimension  $O(\log n)$  since all points can be covered by  $2^{\log n}$  balls.
We then recall the following result that upper bounds the size log  $|\mathbb{C}|$  for metric spaces with doubling dimension  $d$ .
Lemma B.7. [27] Given a subset  $X$  from a metric space with doubling dimension  $d$ ,  $\varepsilon > 0$ , and an  $\alpha$ -approximate solution  $\mathcal{A}$  with at most  $k$  polylog  $(n)$  centers, there exists an  $\mathcal{A}$ -approximate centroid set for  $X$  of size  $|X| \cdot \left(\frac{\alpha}{\varepsilon}\right)^{O(d)}$ .
It is known that the Euclidean space has doubling dimension  $\Theta(d)$ , which would give a  $d$  dependency on our coreset size. However, [38] showed that the  $d$  dependency can be replaced with  $\frac{k}{\varepsilon^2}$ , which was subsequently improved by a line of works, e.g., [58, 40, 46], ultimately down to a dependency of  $\frac{1}{\varepsilon^2} \log \frac{k}{\varepsilon}$  using the following notion of terminal embeddings:
Fig. 2: Merge and reduce framework on a stream of length  $n$ . The coresets at level 1 are the entire blocks. The coresets at level  $i$  for  $i > 1$  are each  $\left(1 + O\left(\frac{\varepsilon}{2\log n}\right)\right)$ -coresets of the coresets at their children nodes in level  $i - 1$ .
Definition B.8 (Terminal embedding). Let  $\varepsilon \in (0,1)$  and  $X \subseteq \mathbb{R}^d$  be a set of  $n$  points. Then a mapping  $f: \mathbb{R}^d \to \mathbb{R}^m$  is a terminal embedding if for all  $x \in X$  and all  $y \in \mathbb{R}^d$ ,
$$
(1 - \varepsilon) \| x - y \| _ {2} \leq \| f (x) - f (y) \| _ {2} \leq (1 + \varepsilon) \| x - y \| _ {2}.
$$
[55] gave a construction of a terminal embedding with  $m = O\left(\frac{1}{\varepsilon^2}\log n\right)$  that can be applied in linear space through exhaustive search when polynomial runtime is not required. Thus Lemma 2.2 nows give the following:
For the purpose of clarity, we emphasize that the algorithm does not use sublinear space, even though the sample complexity is sublinear. Namely, for each stream update, we construct and apply a terminal embedding to project each point to a lower dimension. We then compute the appropriate sampling probability of the projected point, but then sample the original point with the computed sampling probability.
C Sliding Window Model
In this section, we describe how our online coreset construction for  $(k,z)$ -clustering on general discrete metric spaces can be used to achieve near-optimal space algorithms for  $(k,z)$ -clustering in the sliding window model.
We first recall a standard approach for using offline coreset constructions for insertion-only streaming algorithms. Suppose there exists a randomized algorithm that produces an online coreset algorithm that uses  $S(n, \varepsilon, \delta)$  points for an input stream of length  $n$ , accuracy  $\varepsilon$ , and failure probability  $\delta$ , where for the ease of discussion, we omit additional dependencies, such as on the dimension  $d$ , the clustering constraint  $k$ , the parameter  $z$ , or additional parameters for whatever problem the coreset construction may approximate. A standard approach for using coresets on insertion-only streams is the merge-and-reduce approach, which partitions the stream into blocks of size  $S\left(n, \frac{\varepsilon}{2\log n}, \frac{\delta}{\mathrm{poly}(n)}\right)$  and builds a coreset for each block. Each coreset is then viewed as the leaves of a binary tree with height at most  $\log n$ , since the binary tree has at most  $n$  leaves. Then at each level of the binary tree, for each node in the level, a coreset of size  $S\left(n, \frac{\varepsilon}{2\log n}, \frac{\delta}{\mathrm{poly}(n)}\right)$  is built from the coresets representing the two children of the node. Due to the mergeability property of coresets, the coreset at the root of the tree will be a coreset for the entire stream with accuracy  $\left(1 + \frac{\varepsilon}{2\log n}\right)^{\log n} \leq (1 + \varepsilon)$  and failure probability  $\delta$ . We give an illustration of this approach in Figure 2.
This approach fails for sliding window algorithms because the elements at the beginning of the data stream can expire, and so coresets corresponding to earlier blocks of the stream may no longer accurate, which would result in the coreset at the root of the tree also no longer being accurate. On the other hand, suppose we partition the stream into blocks consisting of  $S\left(n,\frac{\varepsilon}{2\log n},\frac{\delta}{\mathrm{poly}(n)}\right)$  elements as before, but instead of creating an offline coreset for each block, we can create an online coreset
for the elements in reverse. That is, since the elements in each block are explicitly stored, we can create offline an artificial stream consisting of the elements in the block in reverse and then give the artificial stream as input to the online coreset construction. Note that if we also first consider the "latter" coreset when merging two coresets, then this effectively reverses the stream. Moreover, by the correctness of the online coreset, our data structure provides correctness over any prefix of the reversed stream, or equivalently, any suffix of the stream and specifically, correctness over the sliding window.
Indeed, [9] showed that deterministic online coresets for problems in randomized numerical linear algebra can be used to achieve deterministic algorithms for the corresponding problems in the sliding window model. We thus further adapt the merge-and-reduce framework to show that randomized online coresets for problems in clustering can also be used to achieve randomized algorithms for the corresponding problems in the sliding window model. We formalize this approach in Algorithm 2, duplicated below:
Algorithm 5 Merge-and-reduce framework for randomized algorithms in the sliding window model, using randomized constructions of online coresets