SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\frac {n _ {i + 1} + \ell}{n _ {i} - \ell^ {\prime}} = f \left(\frac {\ell}{\ell + \ell^ {\prime}}\right) \leq f (1) = \frac {n _ {i + 1} + \ell + \ell^ {\prime}}{n _ {i}} \leq \frac {n _ {i + 1} + \tau n _ {i + 1}}{n _ {i}}.
$$
Since  $|U_1| = |U|$ ,  $|U_{t - 1}| > (1 - \tau)\alpha k' = \Omega (k)$ , and  $\beta^{*}$  is a constant, it follows from Lemma B.1 that  $t = O\left(\log (|U| / k)\right) = \tilde{O}(1)$ .
Bounding the Radii  $\nu_{i}$  (Lemma B.5). Let  $U_{i}^{\mathrm{OLD}}$  denote the state of the  $i$ th layer the last time it was reconstructed for  $i\in [t]$ . We now use the following crucial lemma which is analogous to Lemma 4.3.3 in [26].
Lemma B.3. Given any positive  $\xi$ , there exists a sufficiently large choice of  $\alpha$  such that  $\nu_{i} \leq 2\mu_{\gamma}(U_{i}^{\mathrm{OLD}})$  for each  $i \in [t - 1]$  with probability at least  $1 - e^{-\xi k'}$ .
Henceforth, we fix some positive  $\xi$  and sufficiently large  $\alpha$  such that Lemma B.3 holds.
Lemma B.4. Given metric subspaces  $U_{1}$  and  $U_{2}$  of  $U$  such that  $|U_{1} \oplus U_{2}| \leq \epsilon \gamma |U_{1}|$ , we have that  $\mu_{\gamma}(U_1) \leq 2\mu_{\gamma^*}(U_2)^5$
$^{5}\oplus$  denotes symmetric difference, i.e.  $U_{1}\oplus U_{2} = (U_{1}\setminus U_{2})\cup (U_{2}\setminus U_{1})$
Proof. Let  $X$  be a subset of  $U_{2}$  of size  $k$  such that  $\nu_{\gamma(1+2\epsilon)}(X, U_{2}) = \mu_{\gamma(1+2\epsilon)}(U_{2})$ ,  $\rho = \mu_{\gamma(1+2\epsilon)}(U_{2})$ , and  $A = B_{U_{1}}(X, \rho)$ . Now note that
$$
\begin{array}{l} | A | = \left| B _ {U _ {1} \cup U _ {2}} (X, \rho) \backslash B _ {U _ {2} \backslash U _ {1}} (X, \rho) \right| \\ \geq \left| B _ {U _ {2}} (X, \rho) \right| - \left| B _ {U _ {2} \backslash U _ {1}} (X, \rho) \right| \\ \geq \gamma (1 + 2 \epsilon) | U _ {2} | - | U _ {2} \backslash U _ {1} | \\ \geq \gamma (1 + 2 \epsilon) | U _ {2} | - \epsilon \gamma | U _ {1} | \\ \geq \gamma (1 + 2 \epsilon) \left(| U _ {1} | - \epsilon \gamma | U _ {1} |\right) - \epsilon \gamma | U _ {1} | \\ = \gamma | U _ {1} | + \epsilon \gamma (1 - \gamma (1 + 2 \epsilon)) | U _ {1} | \\ \geq \gamma | U _ {1} |. \\ \end{array}
$$
Since there also exists a subset  $Y \subseteq A$  of size  $k$  such that  $A \subseteq B_{U_1}(Y, 2\rho)$ , it follows that  $\nu_{\gamma}(Y, U_1) \leq 2\rho$ . Hence,  $\mu_{\gamma}(U_1) \leq \nu_{\gamma}(Y, U_1) \leq 2\mu_{\gamma(1 + 2\epsilon)}(U_2)$ .
Lemma B.5.  $\nu_{i}\leq 4\mu_{\gamma^{*}}(U_{i})$  for each  $i\in [t - 1]$  with probability at least  $1 - e^{-\xi k^{\prime}}$
Proof. For each  $i \in [t - 1]$ ,  $|U_i \oplus U_i^{\mathrm{OLD}}| \leq \tau |U_i^{\mathrm{OLD}}|$  since, by Invariant 3.1, at most  $\tau |U_i^{\mathrm{OLD}}|$  points have been inserted or deleted from  $U_i$  since it was last reconstructed. Noticing that  $\tau \leq \epsilon \gamma$ , we can see that
$$
\left| U _ {i} \oplus U _ {i} ^ {\mathrm {O L D}} \right| \leq \epsilon \gamma \left| U _ {i} ^ {\mathrm {O L D}} \right|.
$$
By now applying Lemma B.4 it follows that  $\mu_{\gamma}(U_i^{\mathrm{OLD}}) \leq 2\mu_{\gamma^*}(U_i)$ . The lemma follows by combining this result with Lemma B.3.
Upper Bounding cost  $(\sigma)$  (Lemma B.6).
Lemma B.6.
$$
\operatorname {c o s t} (\sigma) \leq \sum_ {i = 1} ^ {t} 2 \nu_ {i} | C _ {i} |.
$$
Proof. We first note that for all  $i \in [t - 1]$ ,  $\mathrm{cost}(\sigma, C_i) \leq 2\nu_i |C_i|$ . This follows directly from the fact that each point  $x$  in  $C_i$  is assigned to some point  $y \in C_i$  such that  $d(x,y) \leq 2\nu_i$ . Since the  $C_i$  partition  $U$  and  $\mathrm{cost}(\sigma, C_t) = 0$ , we get:
$$
\operatorname {c o s t} (\sigma) = \sum_ {i = 1} ^ {t} \operatorname {c o s t} (\sigma , C _ {i}) \leq \sum_ {i = 1} ^ {t} 2 \nu_ {i} | C _ {i} |.
$$
Lower Bounding opt(U) (Lemma B.10). Let  $r$  denote  $\lceil \log_{1 - \beta^{*}}\frac{1 - \gamma^{*}}{3}\rceil$  and for each  $i\in [t]$  let  $\mu_{i}$  denote  $\mu_{\gamma^{*}}(U_i)$
For the rest of this subsection we fix an arbitrary  $S \subseteq U$  of size  $k$ . For each  $i \in [t]$ , let  $F_i$  denote the set  $\{x \in U_i \mid d(x, S) \geq \mu_i\}$ , and for any integer  $m > 0$ , let  $F_i^m$  denote  $F_i \setminus (\cup_{j > 0} F_{i + jm})$  and  $G_{i,m}$  denote the set of all integers  $j \in [t]$  and  $j \equiv i \pmod{m}$ .
Lemma B.7. Given some  $i \in [t]$  and a subset  $X \subseteq F_i$ , we have that  $|F_i| \geq (1 - \gamma^*) |U_i|$  and  $\text{cost}(S, X) \geq \mu_i |X|$ .
Proof. It follows directly from the definition of  $\mu_{i}$  that we have that  $|F_i| \geq (1 - \gamma^*)|U_i|$ . By the definition of  $F_{i}$ , we have that  $\mathsf{cost}(S,X) = \sum_{x\in X}d(x,S) \geq \mu_i|X|$ .
The following lemma is proven in [26].
Lemma B.8 ([26], Lemma 4.3.8). Given integers  $\ell \in [t]$  and  $m > 0$ , we have that
$$
\operatorname {c o s t} \left(S, \cup_ {i \in G _ {\ell , m}} F _ {i} ^ {m}\right) \geq \sum_ {i \in G _ {\ell , m}} \mu_ {i} | F _ {i} ^ {m} |.
$$
Lemma B.9. For all  $i \in [t - 1]$ , we have that  $|F_i^r| \geq \frac{1}{2} |F_i|$ .
Proof. We first note that for all  $i \in [t - r]$ , we have that  $|F_{i + r}| \leq \frac{1}{3} |F_i|$ . This follows from the fact that
$$
\left| F _ {i + r} \right| \leq \left| U _ {i + r} \right| \leq (1 - \beta^ {*}) ^ {r} \left| U _ {i} \right| \leq \frac {(1 - \beta^ {*}) ^ {r}}{1 - \gamma^ {*}} \left| F _ {i} \right| \leq \frac {1}{3} \left| F _ {i} \right|,
$$
where the first inequality follows from the fact that  $F_{i + r} \subseteq U_{i + r}$ , the second inequality follows from Lemma B.1, the third inequality follows from Lemma B.7, and the fourth inequality follows from the definition of  $r$ . We now get that
$$
\left| F _ {i} ^ {r} \right| = \left| F _ {i} \backslash \cup_ {j > 0} F _ {i + j r} \right| \geq \left| F _ {i} \right| - \sum_ {j > 0} \frac {1}{3 ^ {j}} \left| F _ {i} \right| \geq \frac {1}{2} \left| F _ {i} \right|.
$$
Lemma B.10.
$$
\operatorname {c o s t} (S) \geq \frac {1 - \gamma^ {*}}{2 r} \sum_ {i = 1} ^ {t} \mu_ {i} | C _ {i} |.
$$
Proof. Let  $\ell = \arg \max_{0\leq \ell < r}\{\sum_{i\in G_{\ell ,r}}\mu_i|F_i^r |\}$ . Then we have that
$$
\begin{array}{l} \operatorname {c o s t} (S) \geq \operatorname {c o s t} \left(S, \cup_ {i \in G _ {\ell , r}} F _ {i} ^ {r}\right) \geq \sum_ {i \in G _ {\ell , r}} \mu_ {i} \left| F _ {i} ^ {r} \right| \geq \frac {1}{r} \sum_ {i = 1} ^ {t} \mu_ {i} \left| F _ {i} ^ {r} \right| \geq \frac {1}{2 r} \sum_ {i = 1} ^ {t} \mu_ {i} \left| F _ {i} \right| \\ \geq \frac {1 - \gamma^ {*}}{2 r} \sum_ {i = 1} ^ {t} \mu_ {i} | U _ {i} | \geq \frac {1 - \gamma^ {*}}{2 r} \sum_ {i = 1} ^ {t} \mu_ {i} | C _ {i} |. \\ \end{array}
$$
The second inequality follows from Lemma B.8, the third inequality from averaging and the choice of  $\ell$ , the fourth inequality from Lemma B.9, and the fifth inequality from Lemma B.7.
Proof of Lemma 3.3. It follows that with probability at least  $1 - e^{-\xi k'}$  we have that
$$
\operatorname {c o s t} (\sigma) \leq \sum_ {i = 1} ^ {t} 2 \nu_ {i} | C _ {i} | \leq \sum_ {i = 1} ^ {t} 8 \mu_ {i} | C _ {i} | \leq \frac {1 6 r}{1 - \gamma^ {*}} \operatorname {c o s t} (S)
$$
for any set  $S \subseteq U$  of size  $k$ . Hence, we have that
$$
\operatorname {c o s t} (\sigma) \leq \frac {1 6 r}{1 - \gamma^ {*}} \operatorname {o p t} (U).
$$
C Proof of Corollary 3.4
In order to prove this corollary, we apply the extraction technique presented in [28] (with full details appearing in [26]) which is a slight generalization of the techniques from [19]. In particular, we use the following theorem which follows as an immediate corollary of Theorem 6 in [26]. For completeness, we provide a proof of this theorem.
Theorem C.1. Given a  $\phi$ -approximate  $m$ -assignment  $\pi : U \to U$ , any  $\psi$ -approximate solution to the weighted  $k$ -median instance  $(\pi(U), d, w)$ , where each point  $x \in \pi(U)$  receives weight  $w(x) := |\pi^{-1}(x)|$ , is also a  $(\phi + 2(1 + \phi)\psi)$ -approximate solution to the  $k$ -median instance  $(U, d)$ .
Proof. Let  $S^*$  be a solution to the weighted  $k$ -median instance  $(\pi(U), d, w)$  and let  $S$  be an optimal solution to the  $k$ -median instance  $(U, d)$ . Let  $\phi$  and  $\psi$  be constants such that  $\mathrm{cost}(\pi, U) \leq \phi \cdot \mathrm{opt}(U)$  and  $\mathrm{cost}_w(S^*, \pi(U)) \leq \psi \cdot \mathrm{opt}_w(\pi(U))$ . We now show that  $\mathrm{cost}(S^*, U) = O(1) \cdot \mathrm{opt}(U)$ . We first note that
$$
\begin{array}{l} \operatorname {c o s t} \left(S ^ {*}, U\right) = \sum_ {x \in U} d (x, S ^ {*}) \\ \leq \sum_ {x \in U} d (x, \pi (x)) + \sum_ {y \in \pi (U)} w (y) \cdot d (y, S ^ {*}) \\ = \operatorname {c o s t} (\pi , U) + \operatorname {c o s t} _ {w} \left(S ^ {*}, \pi (U)\right) \\ \leq \phi \cdot \operatorname {o p t} (U) + \operatorname {c o s t} _ {w} \left(S ^ {*}, \pi (U)\right). \\ \end{array}
$$
Now note that, for any  $X \subseteq U$  of size at most  $k$ , there exists some  $Y \subseteq \pi(U)$  of size at most  $k$  such that  $\mathrm{cost}_w(Y, \pi(U)) \leq 2 \cdot \mathrm{cost}_w(X, \pi(U))$ . Since  $\mathrm{cost}_w(S^*, \pi(U)) \leq \psi \cdot \mathrm{cost}_w(Y, \pi(U))$  for all  $Y \subseteq \pi(Y)$  of size at most  $k$ , we get the following.
$$
\begin{array}{l} \operatorname {c o s t} _ {w} \left(S ^ {*}, \pi (U)\right) \leq 2 \psi \cdot \operatorname {c o s t} _ {w} \left(S, \pi (U)\right) \\ = 2 \psi \cdot \sum_ {y \in \pi (U)} w (y) \cdot d (y, S) \\ = 2 \psi \cdot \sum_ {x \in U} d (\pi (x), S) \\ \leq 2 \psi \cdot \sum_ {x \in U} d (x, \pi (x)) + 2 \psi \cdot \sum_ {x \in U} d (x, S) \\ = 2 \psi \cdot \operatorname {c o s t} (\pi , U) + 2 \psi \cdot \operatorname {o p t} (U) \\ \leq 2 (1 + \phi) \psi \cdot \operatorname {o p t} (U). \\ \end{array}
$$
By combining these two chains of inequalities, we get that
$$
\operatorname {c o s t} \left(S ^ {*}, U\right) \leq \phi \cdot \operatorname {o p t} (U) + \operatorname {c o s t} _ {w} \left(S ^ {*}, \pi (U)\right) \leq (\phi + 2 (1 + \phi) \psi) \cdot \operatorname {o p t} (U).
$$
It immediately follows that we can get a  $O(1)$ -approximate solution to the instance  $(U,d)$  by running a static weighted  $k$ -median algorithm on the instance  $(\sigma(U),d,w)$ .
D Lower Bounds on Update and Query Time
In the static (i.e. non-dynamic) setting, the  $k$ -median problem is defined as follows: given a metric space  $U$ , return a set  $S$  of at most  $k$  points from  $U$  which minimizes the value of  $\sum_{x \in S} d(x, S)$ . The following lower bound for the static  $k$ -median problem is proven by Mettu in [26].
Theorem D.1. Any  $O(1)$ -approximate randomized (static) algorithm for the  $k$ -median problem, which succeeds with even negligible probability, runs in time  $\Omega(nk)$ .
Informally, the proof of this lower bound is obtained by constructing, for each  $\delta > 0$ , an input distribution of metric spaces (with polynomially bounded aspect ratio) on which no deterministic algorithm for the  $k$ -median problem succeeds with probability more than  $\delta$ . Theorem D.1 then follows by an application of Yao's minmax principle.
We can use this lower bound from the static setting in order to get a lower bound for the dynamic setting. First note that any incremental algorithm for  $k$ -median with amortized update time  $u(n, k)$  and query time  $q(n, k)$  can be used to construct a static algorithm for the  $k$ -median problem with running time  $n \cdot u(n, k) + q(n, k)$  by inserting each point in the input metric space  $U$  followed by a solution query. Hence, by Theorem D.1, we must have that  $n \cdot u(n, k) + q(n, k) = \Omega(nk)$ . Now assume that some incremental algorithm for  $k$ -median has query time  $\tilde{O}(\text{poly}(k))$ . If this algorithm also has an amortized update time of  $\tilde{o}(k)$ , then for the range of values of  $k$  where  $q(n, k) = \tilde{o}(nk)$ , it follows that  $\tilde{o}(nk)$  is  $\Omega(nk)$ , giving a contradiction. Hence, the amortized update time must be  $\tilde{\Omega}(k)$  and Theorem D.2 follows.
Theorem D.2. Any  $O(1)$ -approximate incremental algorithm for the  $k$ -median problem with  $\tilde{O}(\mathrm{poly}(k))$  query time must have  $\tilde{\Omega}(k)$  amortized update time.
It follows that the update time of our algorithm is optimal up to polylogarithmic factors.
E Omitted Experimental Results
E.1 Update Time Evaluation
Figure 2: The cumulative update time for the different algorithms, on the Song dataset for  $k = 10$  (top left),  $k = 50$  (top right),  $k = 100$  (bottom).
Figure 3: The cumulative update time for the different algorithms, on the Census dataset for  $k = 10$  (top left),  $k = 50$  (top right),  $k = 100$  (bottom).
Total Update Time (sec) (kddcup, k = 100)
Figure 4: The cumulative update time for the different algorithms, on the KDD-Cup dataset for  $k = 10$  (top left),  $k = 50$  (top right),  $k = 100$  (bottom).
Total Update Time (sec) (drift, k = 100)
Figure 5: The cumulative update time for the different algorithms, on the Drift dataset for  $k = 10$  (top left),  $k = 50$  (top right),  $k = 100$  (bottom).
Total Update Time (sec) (sift10M, k = 100)
Figure 6: The cumulative update time for the different algorithms, on the SIFT10M dataset for  $k = 10$  (top left),  $k = 50$  (top right),  $k = 100$  (bottom).
E.2 Solution Cost Evaluation
Cost of Solution (song,  $k = 100$ )
Figure 7: The solution cost by the different algorithms, on Song for  $k = 10$  (top left),  $k = 50$  (top right),  $k = 100$  (bottom).
Figure 8: The solution cost by the different algorithms, on Census for  $k = 10$  (top left),  $k = 50$  (top right),  $k = 100$  (bottom).
Figure 9: The solution cost by the different algorithms, on KDD-Cup for  $k = 10$  (top left),  $k = 50$  (top right),  $k = 100$  (bottom).
Figure 10: The solution cost by the different algorithms, on Drift for  $k = 10$  (top left),  $k = 50$  (top right),  $k = 100$  (bottom).
Figure 11: The solution cost by the different algorithms, on SIFT10M for  $k = 10$  (top left),  $k = 50$  (top right),  $k = 100$  (bottom).
E.3 Query Time Evaluation
Table 4: The average query times for the algorithm OURLG  $(\phi = 500)$  and HK  $(\psi = 1000)$  (we omit the parameter value from the table to simplify the presentation), on the different datasets that we consider and for  $k \in \{10, 50, 100\}$ .
Song	Census	KDD-Cup	Drift	SIFT10M
OURALG	HK	OURALG	HK	OURALG	HK	OURALG	HK	OURALG	HK
k=10	0.569	0.327	0.478	0.280	0.069	0.176	0.729	0.421	0.732	0.419
k=50	0.610	0.347	0.511	0.295	0.075	0.141	0.784	0.447	0.795	0.448
k=100	0.665	0.373	0.552	0.317	0.085	0.131	0.857	0.483	0.866	0.483
E.4 Parameter Tuning
E.4.1 Update Time
Figure 12: The cumulative update time for different parameters of OURALG and HK, for  $k = 50$ , on datasets Song (top left), Census (top right), and KDD-Cup (bottom).
E.4.2 Solution Cost
Cost of Solution (kddcup,  $k = 50$ )
Figure 13: The solution cost for different parameters of OURALG and HK, for  $k = 50$ , on datasets Song (top left), Census (top right), and KDD-Cup (bottom).
E.4.3 Query Time
Table 5: The average query times for the algorithm OURALG and HK with different parameters, on the different datasets for  $k = 50$ .
Song	Census	KDD-Cup
HK(ψ = 250)	0.026	0.021	0.012
HK(ψ = 500)	0.087	0.073	0.043
HK(ψ = 1000)	0.293	0.249	0.156
OURALG(φ = 250)	0.223	0.187	0.054
OURALG(φ = 500)	0.439	0.364	0.086
OURALG(φ = 1000)	0.719	0.605	0.146
E.5 Randomized Order of Updates
E.5.1 Update Time
Total Update Time (sec) (kddcup,  $k = 50$ )
Figure 14: The cumulative update time for different parameters of OURALG and HK, for  $k = 50$ , over a sequence of updates given by a randomized order of the points in the dataset, on the datasets Song (top left), Census (top right), and KDD-Cup (bottom).
E.5.2 Solution Cost
Figure 15: The solution cost for different parameters of OURALG and HK, for  $k = 50$ , over a sequence of updates given by a randomized order of the points in the dataset, on the datasets Song (top left), Census (top right), and KDD-Cup (bottom).
E.5.3 Query Time
Table 6: The average query times for the algorithm OURALG and HK with different parameters, for  $k = 50$ , over a sequence of updates given by a randomized order of the points in each of the datasets that we consider.
Song	Census	KDD-Cup
HK(ψ = 250)	0.025	0.021	0.014
HK(ψ = 500)	0.086	0.073	0.050
HK(ψ = 1000)	0.292	0.247	0.173
OURALG(φ = 250)	0.225	0.185	0.062
OURALG(φ = 500)	0.440	0.364	0.100
OURALG(φ = 1000)	0.723	0.605	0.165
E.6 Larger Experiment
Figure 16: The total update time for OURALG  $(\phi = 500)$  and HK  $(\psi = 1000)$ , on the larger instance derived from KDD-Cup, for  $k = 50$ .
Figure 17: The solution cost produced by OURALG  $(\phi = 500)$  and HK  $(\psi = 1000)$  two algorithms, on the larger instance derived from KDD-Cup, for  $k = 50$
The average query times for OURALG  $(\phi = 500)$  and HK  $(\psi = 1000)$  while handling this longer sequence of updates were 0.416 and 0.225 respectively.