SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proof. We first describe how our dynamic algorithm maintains the assignment  $\sigma$  implicitly. In each layer  $i\in [t]$ , the assignment  $\sigma_{i}$  partitions the set  $C_i$  into  $|S_i|$  clusters  $\{\sigma_i^{-1}(x)\}_{x\in S_i}$ . For each such cluster  $Z = \sigma_i^{-1}(x)$ , we maintain: (1) a unique id, given by  $\operatorname{id}(Z)$ , (2) its center, given by  $\operatorname{center}(\operatorname{id}(Z)) := x$ , and (3) its size, given by  $\operatorname{size}(\operatorname{id}(Z)) := |Z|$ . For each point  $y\in Z$ , we also maintain the id of the cluster it belongs to, given by  $\operatorname{cluster}(y) := \operatorname{id}(Z)$ . Using these data structures, for any  $y\in C_i$  we can report in  $O(1)$  time the center  $\sigma_i(y)$ , and we can also report the size of a cluster in  $O(1)$  time.
Recall that  $t = \tilde{O}(1)$  as per Lemma 3.2. Thus, Algorithm 5 takes  $\tilde{O}(1)$  time, excluding the call to Rebuild in Line 6. For Algorithm 6, the key thing to note is that using the data structures described above, Lines 8 - 10 can be implemented in  $O(1)$  time, by setting  $y$  as the new center of the cluster  $\sigma_i^{-1}(x)$  and by decreasing the size of the cluster by one. It follows that excluding the call to Rebuild in Line 16, Algorithm 6 also runs in  $\tilde{O}(1)$  time.
Claim 3.8. A call to ConstructFromLayer(i), as described in Algorithm 4, takes  $\tilde{O}(k \cdot |U_i|)$  time.
Proof (Sketch). By Lemma 3.2, the while loop in Algorithm 4 runs for  $\tilde{O}(1)$  iterations. Hence, within a  $\tilde{O}(1)$  factor, the runtime of Algorithm 4 is dominated by the call to AlmostCover  $(U_j)$  in Line 4. Accordingly, we now consider Algorithm 2. As the ball  $B_{U'}(X,r)$  can be computed in  $O(|X| \cdot |U'|)$  time, using a binary search the value  $\nu'$  (see Line 2) can be found in  $\tilde{O}(k \cdot |U'|)$  time. The rest of the steps in Algorithm 2 also take  $\tilde{O}(k \cdot |U'|)$  time. Thus, it takes  $\tilde{O}(k \cdot |U_j|)$  time to implement Line 4 of Algorithm 4. Hence, the total runtime of Algorithm 4 is  $\sum_{j \in [i,t]} \tilde{O}(k \cdot |U_j|) = \sum_{j \in [i,t]} \tilde{O}(k \cdot |U_i|) = \tilde{O}(k \cdot |U_i|)$ .
Define the potential  $\Phi := \sum_{i \in [t]} n_i^*$ . For each update in  $U$ , the potential  $\Phi$  increases by at most  $t$ , excluding the calls to Rebuilt (see Algorithm 5 and Algorithm 6). Thus, by Lemma 3.2, each update increases the value of  $\Phi$  by at most  $\tilde{O}(1)$ . Now, consider any call to ReconstructFromLayer(i). Algorithm 7 and Invariant 3.1 ensure that just before this call, we had  $n_i^* \geq \tau n_i = \Omega(|U_i|)$ , since  $\tau$  is a constant. Hence, because of Line 3 in Algorithm 4, during this call the value of  $\Phi$  decreases by at least  $\Omega(|U_i|)$ . Claim 3.8, in contrast, implies that the time taken to implement this call is  $\tilde{O}(k \cdot |U_i|)$ .
To summarize, each update in  $U$  creates  $\tilde{O}(1)$  units of potential, and the total time spent on the calls to Rebuild is at most  $\tilde{O}(k)$  times the decrease in the potential. Since the potential  $\Phi$  is always nonnegative, it follows that the amortized time spent on the calls to Rebuild is  $\tilde{O}(k)$ . This observation, along with Claim 3.7, implies Lemma 3.6.
Datasets. We conduct the empirical evaluation of our algorithm on five datasets from the UCI repository [17]: (1) Census [23] with 2,458,285 points of dimension 68, (2) KDD-Cup [32] containing 311,029 points of dimension 74, (3) Song [5] with 515,345 points of dimension 90, (4) Drift [33, 30] with 13,910 points of dimension 129, and (5) SIFT10M [17] with 11,164,866 points of dimension 128.
We generate the dynamic instances that we use in our study as follows. We keep the first 10,000 points of each dataset, in the order that they appear in their file. This choice allows us to test against competing algorithms that are not as efficient, and at the same time captures the different practical aspects of the algorithms that we consider; as a sanity check, we perform an experiment on an instance that is an order of magnitude larger to confirm the scalability of our algorithm, and that the relative behavior in terms of the cost of the solutions remains the same among the tested algorithms.
<sup>3</sup>We are not aware of any dynamic dataset containing the sequence of arrivals and that is publicly available. We generated sequences of updates following a typical setting used in studies of dynamic algorithms (e.g., [16]).
We use the  $L_{2}$  distance of the embeddings of two points to compute their distance. To avoid zero distance without altering the structural properties of the instances, we add  $1 / |U|$  to all distances, where  $U$  is the set of points of the instance.
Order of Updates. We obtain a dynamic sequence of updates following the sliding window approach. In this typical approach, one defines a parameter  $\kappa$  indicating the size of the window, and then "slides" the window over the static sequence of points in steps creating at each step  $i$  a point insertion of the  $i$ -th point (if there are more than  $i$  points) and a point deletion of the  $(i - \kappa)$ -th point (if  $\kappa > i$ ), until each point of the static instance is inserted once and deleted once. We set  $\kappa = 2,000$ , which results in our dynamic instances having at most 2,000 points at any point in time.
Algorithms. We compare our algorithm from Section 3 against the state-of-the-art algorithm for maintaining a solution to the  $k$ -median problem, which is obtained by dynamically maintaining a coreset for the same problem [20]. In particular, [20] presented an algorithm that can dynamically maintain an  $\epsilon$ -coreset of size  $O(\epsilon^{-2}k\mathrm{poly}(\log n,\log (1 / \epsilon)))$  for the  $k$ -median and the  $k$ -means problems. Their algorithm has a worst-case update time of  $O(\epsilon^{-2}k^{2}\mathrm{poly}(\log n,\log (1 / \epsilon)))$ . To the best of our knowledge, this is the first implementation of the algorithm in [20]. For brevity, we refer to our algorithm as OURALG, and the algorithm from [20] as HK.
Since the exact constants and thresholds required to adhere to the theoretical guarantees of [20] are impractically large, there is no obvious way to use the thresholds provided by the algorithm in practice without making significant modifications. In order to give a fair comparison of the two algorithms, we implemented each of them to have a single parameter controlling a trade-off between update time, query time, and the cost of the solution. OURALG has a parameter  $\phi$  which we set to be the number of points sampled during the construction of a layer in Algorithm 2. In other words, we sample  $\phi$  points in Line 1 of Algorithm 2 instead of  $\alpha k'$ . We fix the constants  $\epsilon$  and  $\beta$  used by our algorithm (see Section 3) to 0.2 and 0.5 respectively. HK has a parameter  $\psi$  which we set to be the number of points sampled during the static coreset construction from [9] which is used as a black box, replacing the large threshold required to adhere to the theoretical guarantees. Since this threshold is replaced by a single parameter, it also replaces the other parameters used by HK, which are only used in this thresholding process. For the bicriteria algorithm required by this static coreset construction, we give the same implementation as the empirical evaluations in [9] and use kmeans++ with 2 iterations of Lloyd's.
For each of  $\phi$  and  $\psi$ , we experimented with the values 250, 500, 1000. As the values of  $\phi$  and  $\psi$  are increased, the asymptotic update times and query times of the corresponding algorithms increase, while the cost of the solution decreases.
Setup. All of our code is written in Java and is available online.4 We did not use parallelization. We used a machine with 8 cores, a 2.9Ghz processor, and 16 GiB of main memory.
Results. We compare OURALG  $(\phi = 500)$  against HK  $(\psi = 1000)$ . We selected these parameters such that they obtain a good solution quality, without these parameters being unreasonably high w.r.t. the size of our datasets. Our experiments suggest that the solution quality produced by OURALG is robust against different values of  $\phi$  (typically these versions of the algorithm differ by less than  $1\%$ ), while that of HK is more sensitive (in several instances the differences are in the range of  $3 - 9\%$ , while for KDD-Cup HK  $(\psi = 250)$  produces much worse solutions). The update time among the different versions of each algorithm do not differ significantly. Lastly, the query time of OURALG is less sensitive compared to the query time of HK. Specifically, the average query time of OURALG  $(\phi = 1000)$  is roughly 3 times slower than OURALG  $(\phi = 250)$ , while the average query time of HK  $(\psi = 1000)$  is 11 times slower compared to HK  $(\psi = 250)$ . For OURALG we selected the middle value  $\phi = 500$ . Since the solution quality of HK drops significantly when using smaller values of  $\psi$ , we selected  $\psi = 1000$  to prioritize for the quality of the solution produced by the algorithm. Since we did not observe significant difference in the behavior across the datasets, we tuned these parameters considering three or our datasets. We provide the relevant plots in Appendix E.4.
Update Time Evaluation. In Figure 1 (left) we plot the total update time over the sequence of updates for the dataset Song, and only for  $k = 50$ . OURALG runs up to more than 2 orders of
<sup>4</sup>https://github.com/martin-costa/NeurIPS23-dynamic-k-clustering
magnitude faster compared to HK, independently of their parameter setup (see Appendix E for more details). The relative performance of the algorithms is similar for all datasets and choices of  $k$ ; we summarize the results in Table 1, while the corresponding plots can be found in Appendix E.
Figure 1: The total update time in seconds (left) and the solution cost (right) for OURALG  $(\phi = 500)$  and HK  $(\psi = 1000)$  on Song, for  $k = 50$
Table 1: The ratio of the total update time incurred by  $\mathrm{HK}(\psi = 1000)$  divided by the time for OURALG  $(\phi = 500)$ , for the five datasets that we considered and for  $k \in \{10, 50, 100\}$ .
Song	Census	KDD-Cup	Drift	SIFT10M
k = 10	31.774	32.172	62.776	32.408	32.857
k = 50	134.850	131.333	201.147	140.826	142.989
k = 100	262.427	255.927	377.350	276.848	280.925
Solution Cost Evaluation. Figure 1 (right) shows the cost of the solution obtained by OURALG  $(\phi = 500)$  and HK  $(\psi = 1000)$ , on dataset Song for  $k = 50$ . In general, the two algorithms have similar performance with OURALG  $(\phi = 500)$  producing slightly better  $(1.3\% - 6.5\%)$  solutions on most instances; the exceptions being on the KDD-Cup dataset, where for  $k = 10$ $\mathrm{HK}(\psi = 1000)$  performs  $1.2\%$  better, and for  $k = 100$  where OURALG  $(\phi = 500)$  performs  $35\%$  better. We summarize their relevant performance on the different datasets in Table 2, and provide the complete set of plots in the Appendix.
Table 2: The mean ratio of the solution cost for  $\mathrm{HK}(\psi = 1000)$  divided by that of OURALG  $(\phi = 500)$  averaged over the updates, for the five datasets that we considered and for  $k \in \{10, 50, 100\}$ .
Song	Census	KDD-Cup	Drift	SIFT10M
k = 10	1.013	1.018	0.988	1.020	1.014
k = 50	1.036	1.056	1.019	1.037	1.032
k = 100	1.049	1.065	1.538	1.059	1.038
Query Time Evaluation. Finally, we compare OURALG  $(\phi = 500)$  to HK  $(\psi = 1000)$  in terms of their query time. To measure query time, we evenly distribute 100 queries across the update sequence and measure the average query time. In Table 3 we report the average ratio of the query times obtained by OURALG  $(\phi = 500)$  divided by that of HK  $(\psi = 1000)$  over the queries executed on each of the datasets and parameters of  $k$  that we considered. The query time obtained by the two algorithms is typically within a factor 2 of each other. OURALG  $(\phi = 500)$  performs better on KDD-Cup, while HK  $(\psi = 1000)$  performs better on rest of the datasets. This inconsistency is due to differences in the structure of the datasets (we also tried randomizing the order of the updates, and it didn't change the relative picture w.r.t. query time).
Longer Sequence of Updates. While the above experiments capture well the different aspects of the two algorithms, we further conducted an experiment on the KDD-Cup dataset where we consider
Table 3: The mean ratio of query times for  $\mathrm{HK}(\psi = 1000)$  divided by that of OURALG  $(\phi = 500)$ , for the five datasets that we considered and for  $k \in \{10, 50, 100\}$ .
Song	Census	KDD-Cup	Drift	SIFT10M
k = 10	0.575	0.586	2.629	0.577	0.572
k = 50	0.568	0.577	1.888	0.570	0.564
k = 100	0.560	0.573	1.543	0.564	0.558
a window size of  $\kappa = 5,000$  and applied it to the first 100,000 points. As expected the relative performance in terms of solution cost and query time remains the same, while the gap in terms of update time grows significantly (HK  $(\psi = 1000)$  performs more than 3 orders of magnitude worse than OURALG  $(\phi = 500)$ ). The corresponding plots and numbers are provided in the Appendix.
Conclusion of the Experiments. Our experimental evaluation shows the practicality of our algorithms on a set of five standard datasets used for the evaluation of dynamic algorithms in metric spaces. In particular, our experiments verify the theoretically superior update time of our algorithm w.r.t. to the state-of-the-art algorithm for the fully-dynamic  $k$ -median problem [20], where our algorithm performs orders of magnitude faster than HK. On top of that, the solution quality produced by our algorithm is better than HK in most settings, and the query times of the two algorithms are comparable. We believe that our algorithm is the best choice when it comes to efficiently maintaining a dynamic solution for the  $k$ -median problem in practice.
[1] Sara Ahmadian, Ashkan Norouzi-Fard, Ola Svensson, and Justin Ward. Better guarantees for k-means and euclidean k-median by primal-dual algorithms. SIAM Journal on Computing, 49(4):FOCS17-97, 2019.
[2] Nir Ailon, Ragesh Jaiswal, and Claire Monteleoni. Streaming k-means approximation. Advances in neural information processing systems, 22, 2009.
[3] MohammadHossein Bateni, Hossein Esfandiari, Hendrik Fichtenberger, Monika Henzinger, Rajesh Jayaram, Vahab Mirrokni, and Andreas Wiese. Optimal fully dynamic  $k$ -center clustering for adaptive and oblivious adversaries. SODA, 2023.
[4] Jon Louis Bentley and James B Saxe. Decomposable searching problems i. static-to-dynamic transformation. Journal of Algorithms, 1(4):301-358, 1980.
[5] Thierry Bertin-Mahieux, Daniel PW Ellis, Brian Whitman, and Paul Lamere. The million song dataset. 2011.
[6] Sayan Bhattacharya, Silvio Lattanzi, and Nikos Parotsidis. Efficient and stable fully dynamic facility location. Advances in neural information processing systems, 2022.
[7] Michele Borassi, Alessandro Epasto, Silvio Lattanzi, Sergei Vassilvitskii, and Morteza Zadimoghaddam. Sliding window algorithms for k-clustering problems. Advances in Neural Information Processing Systems, 33:8716-8727, 2020.
[8] Vladimir Braverman, Vincent Cohen-Addad, Shaofeng H.-C. Jiang, Robert Krauthgamer, Chris Schwiegelshohn, Mads Bech Toftrup, and Xuan Wu. The power of uniform sampling for coresets. In 63rd IEEE Annual Symposium on Foundations of Computer Science, FOCS 2022, Denver, CO, USA, October 31 - November 3, 2022, pages 462-473. IEEE, 2022.
[9] Vladimir Braverman, Dan Feldman, and Harry Lang. New frameworks for offline and streaming coreset constructions. CoRR, abs/1612.00889, 2016.
[10] Vladimir Braverman, Gereon Frahling, Harry Lang, Christian Sohler, and Lin F Yang. Clustering high dimensional dynamic data streams. In International Conference on Machine Learning, pages 576-585. PMLR, 2017.
[28] Ramgopal R. Mettu and C. Greg Plaxton. Optimal time bounds for approximate clustering. In UAI '02, Proceedings of the 18th Conference in Uncertainty in Artificial Intelligence, pages 344-351. Morgan Kaufmann, 2002.
[29] Krzysztof Nowicki and Krzysztof Onak. Dynamic graph algorithms with batch updates in the massively parallel computation model. In Proceedings of the 2021 ACM-SIAM Symposium on Discrete Algorithms (SODA), pages 2939-2958. SIAM, 2021.
[30] Irene Rodriguez-Lujan, Jordi Fonollosa, Alexander Vergara, Margie Homer, and Ramon Huerta. On the calibration of sensor arrays for pattern recognition using the minimal number of experiments. Chemometrics and Intelligent Laboratory Systems, 130:123-134, 2014.
[31] Michael Shindler, Alex Wong, and Adam Meyerson. Fast and accurate k-means for large datasets. Advances in neural information processing systems, 24, 2011.
[32] S.J. Stolfo, Wei Fan, Wenke Lee, A. Prodromidis, and P.K. Chan. Cost-based modeling for fraud and intrusion detection: results from the jam project. In Proceedings DARPA Information Survivability Conference and Exposition. DISCEX'00, volume 2, pages 130-144 vol.2, 2000.
[33] Alexander Vergara, Shankar Vembu, Tuba Ayhan, Margaret A Ryan, Margie L Homer, and Ramón Huerta. Chemical gas sensor drift compensation using classifier ensembles. Sensors and Actuators B: Chemical, 166:320-329, 2012.
A Extension to  $k$ -Means and  $(k,p)$ -Clustering
As stated in [21, 8], while [28] only discusses the  $k$ -median problem, their construction can easily be modified to work for  $k$ -means clustering and further generalized to work for  $(k,p)$ -clustering, where the  $(k,p)$ -clustering problem is defined in the same way as  $k$ -median problem except that we want to minimize  $\sum_{x\in U}d(x,S)^p$  for some  $S\subseteq U$  of size at most  $k$ . Note that  $(k,1)$ -clustering and  $(k,2)$ -clustering correspond to  $k$ -median and  $k$ -means respectively.
We define a  $\rho$ -metric space  $(U,d)$  in the same way as a metric space except for relaxing the condition that  $d$  must satisfy the triangle inequality to the condition that  $d(x,y) \leq \rho(d(x,z) + d(z,y))$  for all  $x,y,z \in U$ . Given a metric space  $(U,d)$  and some  $p \geq 1$ , the results in Section 6 of [9] can easily be used to show that  $(U,d^{p})$  is a  $2^{p-1}$ -metric space, where  $d^{p}(x,y)$  is defined to be  $d(x,y)^{p}$ .
We now show that the assignment  $\sigma$  maintained by our algorithm is  $O(\rho^3)$ -approximate when  $U$  is a  $\rho$ -metric space (i.e. that  $\mathrm{cost}(\sigma) = O(\rho^3) \cdot \mathrm{opt}(U)$ ) and that the extraction technique of [19] can be generalized to  $\rho$ -metric spaces.
Lemma A.1. When the underlying space  $U$  is a  $\rho$ -metric space, the assignment  $\sigma$  maintained by our algorithm is  $O(\rho^3)$ -approximate.
Proof. By making the appropriate changes to the proofs of Lemma B.3 and Lemma B.4, we get generalizations of these lemmas to  $\rho$ -metric spaces, where the lemma statements are the same except for an extra  $\rho$  factor in the inequalities.
Lemma A.2. Given any positive  $\xi$ , there exists a sufficiently large choice of  $\alpha$  such that  $\nu_{i} \leq 2\rho \cdot \mu_{\gamma}(U_{i}^{\mathrm{OLD}})$  for each  $i \in [t - 1]$  with probability at least  $1 - e^{-\xi k'}$ .
Lemma A.3. Given metric subspaces  $U_{1}$  and  $U_{2}$  of  $U$  such that  $|U_{1} \oplus U_{2}| \leq \epsilon \gamma |U_{1}|$ , we have that  $\mu_{\gamma}(U_1) \leq 2\rho \cdot \mu_{\gamma^*}(U_2)$ .
These two lemmas immediately imply the following generalization of Lemma B.5.
Lemma A.4.  $\nu_{i}\leq 4\rho^{2}\cdot \mu_{\gamma^{*}}(U_{i})$  for each  $i\in [t - 1]$  with probability at least  $1 - e^{-\xi k^{\prime}}$
The upper bound on  $\mathrm{cost}(\sigma)$  given in Lemma B.6 can be generalized by noticing that  $\mathrm{cost}(\sigma, C_i) \leq 2\rho\nu_i|C_i|$  for all  $i \in [t-1]$ , which us that
$$
\operatorname {c o s t} (\sigma) \leq \sum_ {i = 1} ^ {t} 2 \rho \nu_ {i} | C _ {i} |.
$$
The lower bound on  $\operatorname{opt}(U)$  given in Lemma B.10 holds for  $\rho$ -metric spaces with no modifications. Hence, we get that with probability at least  $1 - e^{-\xi k'}$  we have that
$$
\operatorname {c o s t} (\sigma) \leq \sum_ {i = 1} ^ {t} 2 \rho \nu_ {i} | C _ {i} | \leq \sum_ {i = 1} ^ {t} 8 \rho^ {3} \mu_ {i} | C _ {i} | \leq \frac {1 6 \rho^ {3} r}{1 - \gamma^ {*}} \operatorname {c o s t} (S).
$$
By making the appropriate modifications to the proof of Theorem C.1, we can extend this theorem to work for  $\rho$ -metric spaces. In particular, we can obtain a proof of Theorem A.5 by taking the proof of Theorem C.1 and adding extra  $\rho$  factors whenever the triangle inequality is applied.
Theorem A.5. Given a  $\phi$ -approximate  $m$ -assignment  $\pi: U \to U$ , any  $\psi$ -approximate solution to the weighted  $k$ -median instance  $(\pi(U), d, w)$ , where each point  $x \in \pi(U)$  receives weight  $w(x) := |\pi^{-1}(x)|$ , is also a  $(\phi \rho + 2(1 + \phi) \psi \rho^3)$ -approximate solution to the  $k$ -median instance  $(U, d)$  where  $U$  is a  $\rho$ -metric space.
Since the algorithm in [27] is  $O(1)$ -approximate on  $O(1)$ -metric spaces, it immediately follows by applying Theorem A.5 and Lemma A.1 that our algorithm maintains a  $O(1)$ -approximate solution to the  $k$ -median problem on  $(U, d^p)$  for  $p = O(1)$ . Since the  $k$ -median problem on  $(U, d^p)$  is exactly the  $(k, p)$ -clustering problem on  $(U, d)$ , it follows that our algorithm generalizes to solve instances of  $(k, p)$ -clustering in metric spaces.
B Proofs of Lemma 3.2 and Lemma 3.3
Throughout this section, we fix  $\gamma$  to be any real such that  $\beta < \gamma < 1$  and  $\epsilon$  to be any real such that  $0 < \epsilon < \min\{\frac{1 - \gamma}{2\gamma}, 1\}$ . Let  $\beta^{*}$  and  $\gamma^{*}$  denote  $\beta(1 - \epsilon)$  and  $\gamma(1 + 2\epsilon)$  respectively.
We first prove Lemma B.1, which shows that the sizes of the sets  $U_{i}$  decrease exponentially with  $i$ .
Lemma B.1. For all  $i\in [t - 1]$ $|U_{i + 1}|\leq (1 - \beta^{*})|U_{i}|$
Proof. Consider the ratio  $|U_{i+1}| / |U_i|$ . Since  $U_{i+1} \subseteq U_i$  and  $U_{i+1}$  is reconstructed every time  $U_i$  is reconstructed, it follows that  $|U_{i+1}| / |U_i|$  is at most  $(n_{i+1} + \ell) / (n_i + \ell - \ell')$ , where  $n_j$  is the size of  $U_j$  at the time it was last reconstructed and  $\ell$  and  $\ell'$  are the number of insertions and deletions that have occurred since the last time  $U_{i+1}$  was reconstructed respectively. By Lemma B.2, we get that this expression is upper bounded by  $(n_{i+1} + \tau n_{i+1}) / n_i$ . Now we can observe that
$$
\frac {| U _ {i + 1} |}{| U _ {i} |} \leq \frac {n _ {i + 1} + \ell}{n _ {i} + \ell - \ell^ {\prime}} \leq \frac {n _ {i + 1} + \tau n _ {i + 1}}{n _ {i}} \leq \frac {n _ {i + 1}}{n _ {i}} + \tau \leq (1 - \beta) + \epsilon \beta = 1 - \beta^ {*},
$$
where we use the facts that  $n_{i + 1}\leq (1 - \beta)n_i$  and  $\tau \leq \epsilon \beta$  in the final inequality.
Lemma B.2. Given some integer  $i \in [t - 1]$ , let  $\ell$  and  $\ell'$  be the number of insertions and deletions that have occurred since the last time  $U_{i + 1}$  was reconstructed respectively. Then we have that
$$
\frac {n _ {i + 1} + \ell}{n _ {i} + \ell - \ell^ {\prime}} \leq \frac {n _ {i + 1} + \tau n _ {i + 1}}{n _ {i}}.
$$
Proof. First, note that  $(n_{i+1} + \ell) / (n_i + \ell - \ell') \leq (n_{i+1} + \ell) / (n_i - \ell')$ . Now, given some reals  $A \geq a \geq 0$  and  $0 \leq N \leq A - a$ , we define a function  $f: [0,1] \to \mathbb{R}$  by  $f(x) = (a + xN) / (A - (1 - x)N)$ . The derivative of  $f$  is  $-N(a - A + N) / ((x - 1)N + A)^2$  and is non-negative for all  $x \in [0,1]$ . Hence,  $f(x) \leq f(1)$  for all  $x \in [0,1]$ .
By setting  $A = n_i$ ,  $a = n_{i+1}$ ,  $N = \ell + \ell'$  and noting that  $\ell + \ell' \leq \tau n_{i+1}$  by Invariant 3.1 and  $n_{i+1} \leq (1 - \beta)n_i$ , we get that
$$
\ell + \ell^ {\prime} \leq \tau n _ {i + 1} \leq \beta n _ {i + 1} = (1 + \beta) n _ {i + 1} - n _ {i + 1} \leq (1 + \beta) (1 - \beta) n _ {i} - n _ {i + 1} \leq n _ {i} - n _ {i + 1},
$$
and hence it follows that