Title: Fully Dynamic $k$-Clustering in $\tilde O(k)$ Update Time
Venue: NeurIPS 2023
Accepted: yes
Keywords: clustering

k-median

k-means

dynamic algorithms

amortized analysis
Abstract: We present a $O(1)$-approximate fully dynamic algorithm for the $k$-median and $k$-means problems on metric spaces with amortized update time $\tilde O(k)$ and worst-case query time $\tilde O(k^2)$. We complement our theoretical analysis with the first in-depth experimental study for the dynamic $k$-median problem on general metrics, focusing on comparing our dynamic algorithm to the current state-of-the-art by Henzinger and Kale [ESA'20]. Finally, we also provide a lower bound for dynamic $k$-median which shows that any $O(1)$-approximate algorithm with $\tilde O(\text{poly}(k))$ query time must have $\tilde \Omega(k)$ amortized update time, even in the incremental setting.

Fully Dynamic k -Clustering in ˜ O ( k ) Update Time

Abstract

We present a O (1) -approximate fully dynamic algorithm for the k -median and k -means problems on metric spaces with amortized update time ˜ O ( k ) and worst-case query time ˜ O ( k 2 ) . We complement our theoretical analysis with the first in-depth experimental study for the dynamic k -median problem on general metrics, focusing on comparing our dynamic algorithm to the current state-of-the-art by Henzinger and Kale [20]. Finally, we also provide a lower bound for dynamic k -median which shows that any O (1) -approximate algorithm with ˜ O (poly( k )) query time must have ˜ Ω( k ) amortized update time, even in the incremental setting.

1 Introduction

Clustering is a fundamental problem in unsupervised learning with several practical applications. In clustering, one is interested in partitioning elements into different groups (i.e. clusters ), so that elements in the same group are more similar to each other than to elements in other groups. One of the most studied formulations of clustering is the metric clustering formulation. In this setting, elements are represented by points in a metric space, and the distances between points represent how similar the corresponding elements are (the closer elements are, the more similar they are). More formally, the input to our problem consists of a set of points U in a metric space with distance function d : U × U → R ≥ 0 , a real number p ≥ 1 , and an integer k ≥ 1 . The goal is to compute a subset S ⊆ U of size | S | ≤ k , so as to minimize cost ( S ) := ∑ x ∈ U d ( x, S ) p , where d ( x, S ) := min y ∈ S d ( x, y ) . We refer to the points in S as centers . We note that this captures well-known problems such as k -median clustering (when p = 1 ) or k -means clustering (when p = 2 ).

Due to this simple and elegant formulation, metric clustering has been extensively studied throughout the years, across a range of computational models [14, 22, 1, 12, 15, 2, 31, 11, 7]. In this paper, we focus on the dynamic setting , where the goal is to efficiently maintain a good clustering when the input data keeps changing over time. Because of its immediate real-world applications, the dynamic clustering problem has received a lot of attention from the machine learning community in recent years [16, 6, 10, 24, 18]. Below, we formally describe the model [13, 20, 3] considered in this paper.

At preprocessing , the algorithm receives the initial input U . Subsequently, the input keeps changing by means of a sequence of update operations, where each update inserts/deletes a point in U . Throughout this sequence of updates, the algorithm needs to implicitly maintain a solution S ∗ ⊆ U to the current input ( U, d ) . The algorithm has an approximation ratio of ρ ≥ 1 if and only if we always have cost ( S ∗ ) ≤ ρ · opt ( U ) , where opt ( U ) := min S ⊆ U, | S |≤ k cost ( S ) denotes the optimal objective. Finally, whenever one queries the algorithm, it has to return the list of centers in S ∗ . The performance of a dynamic algorithm is captured by its approximation ratio, its update time, and its query time. Let

Martín Costa

U INIT be the state of the input U at preprocessing. A dynamic algorithm has (amortized) update time O ( λ ) if for any sequence of t ≥ 0 updates, the total time it spends on preprocessing and handling the updates is O (( t + | U INIT | ) · λ ) . The time it takes to answer a query is called its query time .

Our Contributions. Our primary contribution is to design an algorithm for this problem with a near-optimal update time of ˜ O ( k ) , without significantly compromising on the approximation ratio and query time. 1 Interestingly, our algorithm can be easily extended to dynamic k -clustering for any constant p (see Appendix A). In the theorem below, we summarize our main result for dynamic k -median and k -means.

Theorem 1.1. There exists a dynamic algorithm that, with high probability, maintains a O (1) -approximate solution to the k -median and k -means problems for general metric spaces under point insertions and deletions with ˜ O ( k ) amortized update time and ˜ O ( k 2 ) worst-case query time.

It is important to note that in practice an algorithm often receives the updates in batches of variable sizes [25, 29], so it is common for there to be a significant number of updates between cluster requests. As a consequence, optimizing the update time is of primary importance in practical applications. For example, if the size of each batch of updates is Ω( k ) , then our amortized query time becomes ˜ O ( k 2 /k ) = ˜ O ( k ) , because the algorithm has to answer a query after processing at least one batch. Thus, our dynamic algorithm has near-optimal update and query times when the updates arrive in moderately large batches.

In addition, it is possible to observe that any dynamic k -clustering algorithm must have Ω( k ) query time, since the solution it needs to return while answering a query can consist of k centers. We also show a similar lower bound on the update time of any constant approximate dynamic algorithm for this problem. This lower bound holds even in an incremental setting, where we only allow for point-insertions in U . We defer the proof of Theorem 1.2 to Appendix D.

Theorem 1.2. Any O (1) -approximate incremental algorithm for the k -median problem with ˜ O (poly( k )) query time must have ˜ Ω( k ) amortized update time.

Theorem 1.2 implies that the ultimate goal in this line of research is to get a O (1) -approximate dynamic k -clustering algorithm with ˜ O ( k ) update time and ˜ O ( k ) query time. Prior to this work, however, the state-of-the-art result for dynamic k -median (and k -means) was due to Henzinger and Kale [20], who obtained O (1) -approximation ratio, ˜ O ( k 2 ) update time and ˜ O ( k 2 ) query time. In this context, our result is a meaningful step toward obtaining an asymptotically optimal algorithm for the problem.

We supplement the above theorem with experiments that compare our algorithm with that of [20]. To the best of our knowledge, this is the first work in the dynamic clustering literature with a detailed experimental evaluation of dynamic k -median algorithms for general metrics. Interestingly, we observe that experimentally our algorithm is significantly more efficient than previous work without impacting the quality of the solution.

Our Techniques. We first summarize the approach used in the previous state-of-the-art result. In [20], the authors used a static algorithm for computing a coreset of size ˜ O ( k ) as a black box to get a fully dynamic algorithm for maintaining a coreset of size ˜ O ( k ) for general metric spaces in worst-case update time ˜ O ( k 2 ) . Their algorithm works by maintaining a balanced binary tree of depth O (log n ) , where each leaf in the tree corresponds to a point in the metric space [4]. Each internal node of the tree then takes the union of the (weighted) sets of points maintained by its two children and computes its coreset, which is then fed to its parent. They maintain this tree dynamically by taking all the leaves affected by an update and recomputing all the coresets at nodes contained in their leaf-to-root paths. Using state-of-the-art static coreset constructions, this update procedure takes time ˜ O ( k 2 ) . Unfortunately, if one wants to ensure that the final output is a valid corset of the metric space after each update, then there is no natural way to bypass having to recompute the leaf-to-root path after the deletion of a point that is contained in the final coreset. Hence, it is not at all clear how to modify this algorithm in order to reduce the update time to ˜ O ( k ) .

We circumvent this bottleneck by taking a completely different approach compared to [20]. Our algorithm instead follows from the dynamization of a ˜ O ( nk ) time static algorithm for k -median by Mettu and Plaxton [28], where n = | U | . We refer to this static algorithm as the MP algorithm. Informally, the MP algorithm works by constructing a set of O (log( n/k )) layers by iteratively sampling random points at each layer, defining a clustering of the points in the layer that are 'close' to these samples, and removing these clusters from the layer. The algorithm then obtains a solution to the k -clustering problem by running a static algorithm for weighted k -median (defined in Section 2), on an instance of size O ( k log( n/k )) defined by the clusterings at each layer. In order to dynamize this algorithm, we design a data structure that maintains O (log( n/k )) layers that are analogous to the layers in the MP algorithm. By allowing for some 'slack' in the way that these layers are defined, we ensure that they can be periodically reconstructed in a way that leads to good amortized update time, while only incurring a small loss in the approximation ratio. The clustering at each layer is obtained by random sampling every time it is reconstructed and is maintained by arbitrarily reassigning a point in a cluster as the new center when the current center is deleted. We obtain a solution to the k -clustering problem from this data structure in the same way as the (static) MP algorithm-by running a static algorithm for weighted k -median on an instance defined by the clusterings maintained at each layer.

2 Preliminaries

In our computational model, the algorithm has access to the distance d ( x, y ) for any x, y ∈ U in O (1) time 2 . Given any two sets X,S ⊆ U , define the cost of X w.r.t. S as cost ( S, X ) := ∑ x ∈ X min s ∈ S d ( x, s ) . In addition, let cost ( S ) = cost ( S, U ) . Next, define an assignment to be a function π : U → U , and say that π assigns x ∈ U to π ( x ) . We refer to an assignment π with | π ( U ) | ≤ m as an m -assignment . The cost of X ⊆ U w.r.t. assignment π is cost ( π, X ) := ∑ x ∈ X d ( x, π ( x )) . We denote cost ( π, U ) by cost ( π ) . For any ρ ≥ 1 , say that π is ρ -approximate if cost ( π ) ≤ ρ · opt ( U ) , where opt ( U ) is the cost of the optimal solution.

Given any subset U ′ ⊆ U and x ∈ U ′ , we denote by B U ′ ( x, r ) the set { y ∈ U ′ | d ( x, y ) ≤ r } , i.e. the closed ball in U ′ of radius r ≥ 0 centered at x . When it is clear from the context that U ′ = U , we omit the subscript U ′ and simply write B ( x, r ) . For X ⊆ U ′ ⊆ U , we define B U ′ ( X,r ) to be the union of the balls B U ′ ( x, r ) for x ∈ X .

Definition 2.1. Given 0 < ρ < 1 and subsets X ⊆ U ′ ⊆ U , we define the following real numbers:

In words, the real number ν ρ ( X,U ′ ) is the smallest radius r such that the closed ball of radius r around X captures at least a ρ -proportion of the points in U ′ . The real number µ ρ ( U ′ ) is then defined to be the smallest such radius ν ρ ( X,U ′ ) over all subsets X ⊆ U ′ of size k . Note that ν ρ ( X,U ′ ) and µ ρ ( U ′ ) are both increasing as functions of ρ .

Finally, we will sometimes refer to the weighted k -median problem . An instance of this problem is given by a triple ( U, d, w ) , where w : U → R ≥ 0 assigns a nonnegative weight to every point x ∈ U , in a metric space with distance function d . Here, the goal is to compute a subset of at most k centers S ⊆ U , so as to minimize cost w ( S ) := ∑ x ∈ U w ( u ) · d ( x, S ) . We let opt w ( U ) := min S ⊆ U : | S |≤ k cost w ( S ) denote the optimal objective value of this weighted k -median instance. We will analogously use the symbol cost w ( S, X ) := ∑ x ∈ X w ( x ) · d ( x, S ) to denote the cost of a set of points X w.r.t. S and the weight function w .

3 Our Algorithm

Throughout this section, we fix the following parameters: α , β , ϵ , τ and k ′ ; where α ≥ 1 is a sufficiently large constant, β is any constant in the interval (0 , 1) , ϵ > 0 is a sufficiently small constant, τ := ϵβ , and k ′ := max { k, log( | U | ) } .

3.1 The Static Algorithm of [28]

Our starting point is the static algorithm of [28] for computing a ˜ O ( k ) -assignment σ : U → U , with S = σ ( U ) , such that cost ( σ ) ≤ O (1) · opt ( U ) . The relevant pseudocode appears in Algorithm 1 and Algorithm 2.

Algorithm 1 StaticAlgo ( U ) 1: i ← 1 and U 1 ← U 2: while | U i | > αk ′ do 3: ( S i , C i , σ i , ν i ) ← AlmostCover ( U i ) 4: U i +1 ← U i \ C i 5: i ← i +1 6: end while 7: t ← i 8: S t ← U t , C t ← S t and ν t ← 0 9: Assign each x ∈ C t to itself (i.e., σ t ( x ) := x ∈ S t ) 10: S ←∪ j ∈ [ t ] S j 11: Let σ : U → S be the assignment such that for all j ∈ [ t ] and x ∈ C j , we have σ ( x ) = σ j ( x ) 12: return ( S, σ, t )

Algorithm 2 AlmostCover ( U ′ )

1: Construct a set S ′ , by sampling αk ′ points from U ′ independently and u.a.r. with replacement

2: ν ′ ← ν β ( S ′ , U ′ ) and C ′ ← B ( S ′ , ν ′ )

3: Assign each x ∈ C ′ to some σ ′ ( x ) ∈ S ′ such that d ( x, σ ′ ( x )) ≤ ν ′

4: return ( S ′ , C ′ , σ ′ , ν ′ )

The algorithm runs for t iterations . Let U i ⊆ U denote the set of unassigned points at the start of iteration i ∈ [ t -1] (initially, we have U 1 = U ). During iteration i , the algorithm samples a set of αk ′ points S i as centers, uniformly at random from U i . It then identifies the smallest radius ν i such that the balls of radius ν i around S i cover at least β -fraction of the points in U i . Let C i ⊆ U i denote the set of points captured by these balls (note that C i ⊇ S i and | C i | ≥ β · | U i | ). The algorithm assigns each x ∈ C i to some center σ i ( x ) ∈ S i within a distance of ν i . It then sets U i +1 ← U i \ C i , and proceeds to the next iteration. In the very last iteration t , we have | U t | ≤ αk ′ , and the algorithm sets S t := U t , C t := U t , and σ t ( x ) := x for each x ∈ U t .

The algorithm returns the assignment σ with set of centers σ ( U ) = S , where σ is simply the union of σ i for all i ∈ [ t ] .

Fix any i ∈ [ t -1] . Since | C i | ≥ β · | U i | , we have | U i +1 | ≤ (1 -β ) · | U i | . As β is a constant, it follows that the algorithm runs for t = ˜ O (1) iterations. Since each iteration picks O ( k ) new centers, we get: | S | = ˜ O ( k ) , and hence σ is a ˜ O ( k ) -assignment. Furthermore, [28] showed that cost ( σ ) = O (1) · opt ( U ) .

3.2 Our Dynamic Algorithm

The main idea behind our dynamic algorithm is that it maintains the output S of the static algorithm from Section 3.1 in a lazy manner, by allowing for some small slack at appropriate places. Thus, it maintains t layers , where each layer i ∈ [ t ] corresponds to iteration i of the static algorithm. In the dynamic setting, the value of t changes over time. Specifically, layer i ∈ [ t ] consists of the tuple ( U i , S i , C i , σ i , ν i ) , as in Algorithm 1. Whenever a large fraction of points gets inserted or deleted from some layer j ∈ [ t ] , the dynamic algorithm rebuilds all the layers i ∈ [ j, t ] from scratch.

The dynamic algorithm maintains the sets U i , S i and C i explicitly, and the assignment σ i implicitly, in a manner which ensures that for all x ∈ C i it can return σ i ( x ) ∈ S i in O (1) time. Furthermore, for each layer i ∈ [ t ] , it explicitly maintains the value n i (resp. n ∗ i ), which denotes the size of the set U i at (resp. the number of updates to U i since) the time it was last rebuilt. We explain this in more detail below. The value ν i is needed only for the sake of analysis.

Algorithm 3 Preprocess ( U )

1: U 1 ← U

2: ConstructFromLayer (1)

Algorithm 4 ConstructFromLayer ( i )

1: j ← i

2: while | U j | > αk ′ do

3: n j ←| U j | and n ∗ j ← 0

4: ( S j , C j , σ j , ν j ) ← AlmostCover ( U j )

5: U j +1 ← U j \ C j

6: j ← j +1

7: end while

8: t ← j

9: S t ← U t , C t ← S t and ν t ← 0

10: Assign each x ∈ C t to itself (i.e., σ t ( x ) := x ∈ S t )

11: S ←∪ j ∈ [ t ] S i

12: Let σ : U → S be the assignment such that for all j ∈ [ t ] and x ∈ C j , we have σ ( x ) = σ j ( x )

Preprocessing: At preprocessing, we essentially run the static algorithm from Section 3.1 to set the value of t and construct the layers i ∈ [ t ] . See Algorithm 3 and Algorithm 4. Note that at this stage n ∗ j = 0 for all layers j ∈ [ t ] .

Algorithm 5 Insert ( x )

5: Add x to C and S , and set σ ( x ) ← x

1: for i = 1 , ..., t do 2: Add x to U i 3: n ∗ i ← n ∗ i +1 4: end for t t t 6: Rebuild

Handling the insertion of a point x in U : We simply add the point x to U i , for each layer i ∈ [ t ] . Next, in the last layer t , we create a new center at point x and assign the point x to itself. Finally, we call the Rebuild subroutine (to be described below). See Algorithm 5.

Handling the deletion of a point x from U : Let j ∈ [ t ] be the last layer (with the largest index) containing the point x . We remove x from each layer i ∈ [ j ] . Next, if x was a center at layer j , then we pick any arbitrary point x ′ ∈ σ -1 j ( x ) \ { x } (if such a point exists), make x ′ a center, and assign every point y ∈ σ -1 j ( x ) \ { x } to the newly created center x ′ . Finally, we call the Rebuild subroutine (to be described below). See Algorithm 6.

The rebuild subroutine: We say that a layer i ∈ [ t ] is rebuilt whenever our dynamic algorithm calls ConstructFromLayer ( j ) for some j ≤ i , and that there is an update in layer i whenever we add/remove a point in U i . It is easy to see that n ∗ i denotes the number of updates in layer i ∈ [ t ] since the last time it was rebuilt (see Line 3 in Algorithm 4, Algorithm 5 and Algorithm 6). Next, we observe that Line 6 in Algorithm 5 and Line 16 in Algorithm 6, along with the pseudocode of Algorithm 7, imply the following invariant.

Invariant 3.1. n ∗ i ≤ τn i for all i ∈ [ t ] . Here, n i is the size of U i just after the last rebuild of layer i , and n ∗ i is the number of updates in layer i since that last rebuild.

Intuitively, the above invariant ensures that the layers maintained by our dynamic algorithm remain close to the layers of the static algorithm in Section 3.1. This is crucially exploited in the proofs of Lemma 3.2 (which helps us bound the update and query times) and Lemma 3.3 (which leads to the desired bound on the approximation ratio). We defer the proofs of these two lemmas to Appendix B.

Lemma 3.2. We always have t = ˜ O (1) , where t denotes the number of layers maintained by our dynamic algorithm.

Algorithm 6 Delete ( x )

Algorithm 7 Rebuild

Lemma 3.3. The assignment σ maintained by our dynamic algorithm always satisfies cost ( σ ) = O (1) · opt ( U ) .

Answering a query: Upon receiving a query, we consider a weighted k -median instance ( S, d, w ) , where each point x ∈ S receives a weight w ( x ) := | σ -1 ( x ) | . Next, we compute a O (1) -approximate solution S ∗ ⊆ S , with | S ∗ | ≤ k , to this weighted instance, so that cost w ( S ∗ , S ) ≤ O (1) · opt w ( S ) . We then return the centers in S ∗ .

Corollary 3.4 implies that our dynamic algorithm has O (1) approximation ratio. It holds because of Lemma 3.3, and its proof immediately follows from [19]. We delegate this proof to Appendix C.

Corollary 3.5. Our algorithm has ˜ O ( k 2 ) query time.

Proof (Sketch) . By Lemma 3.2, we have t = ˜ O (1) . Since the dynamic algorithm maintains at most ˜ O ( k ) centers S i in each layer i (follows from Invariant 3.1), we have | S | = ∑ i ∈ [ t ] | S i | = ˜ O ( k ) . Using appropriate data structures (see the proof of Claim 3.7), in O (1) time we can find the number of points assigned to any center x ∈ S under σ (given by | σ -1 ( x ) | = w ( x ) ). Thus, the weighted k -median instance ( S, d, w ) is of size ˜ O ( k ) , and upon receiving a query we can construct the instance in ˜ O ( k ) time. We now run a static O (1) -approximation algorithm [27] on ( S, d, w ) , which returns the set S ∗ in ˜ O ( k 2 ) time.

Lemma 3.6. Our algorithm has ˜ O ( k ) update time.

Corollaries 3.4, 3.5 and Lemma 3.6 imply Theorem 1.1. It now remains to prove Lemma 3.6.

3.3 Proof of Lemma 3.6

We first bound the time taken to handle an update, excluding the time spent on rebuilding the layers.

Claim 3.7. Excluding the calls to Rebuild , Algorithm 5 and Algorithm 6 both run in ˜ O (1) time.

Proof. We first describe how our dynamic algorithm maintains the assignment σ implicitly. In each layer i ∈ [ t ] , the assignment σ i partitions the set C i into | S i | clusters { σ -1 i ( x ) } x ∈ S i . For each such cluster Z = σ -1 i ( x ) , we maintain: (1) a unique id, given by id ( Z ) , (2) its center, given by center ( id ( Z )) := x , and (3) its size, given by size ( id ( Z )) := | Z | . For each point y ∈ Z , we also maintain the id of the cluster it belongs to, given by cluster ( y ) := id ( Z ) . Using these data structures, for any y ∈ C i we can report in O (1) time the center σ i ( y ) , and we can also report the size of a cluster in O (1) time.

Recall that t = ˜ O (1) as per Lemma 3.2. Thus, Algorithm 5 takes ˜ O (1) time, excluding the call to Rebuild in Line 6. For Algorithm 6, the key thing to note is that using the data structures described above, Lines 8 - 10 can be implemented in O (1) time, by setting y as the new center of the cluster σ -1 i ( x ) and by decreasing the size of the cluster by one. It follows that excluding the call to Rebuild in Line 16, Algorithm 6 also runs in ˜ O (1) time.

Claim 3.8. A call to ConstructFromLayer ( i ) , as described in Algorithm 4, takes ˜ O ( k · | U i | ) time.

Proof (Sketch) . By Lemma 3.2, the while loop in Algorithm 4 runs for ˜ O (1) iterations. Hence, within a ˜ O (1) factor, the runtime of Algorithm 4 is dominated by the call to AlmostCover ( U j ) in Line 4. Accordingly, we now consider Algorithm 2. As the ball B U ′ ( X,r ) can be computed in O ( | X | · | U ′ | ) time, using a binary search the value ν ′ (see Line 2) can be found in ˜ O ( k · | U ′ | ) time. The rest of the steps in Algorithm 2 also take ˜ O ( k · | U ′ | ) time. Thus, it takes ˜ O ( k · | U j | ) time to implement Line 4 of Algorithm 4. Hence, the total runtime of Algorithm 4 is ∑ j ∈ [ i,t ] ˜ O ( k · | U j | ) = ∑ j ∈ [ i,t ] ˜ O ( k · | U i | ) = ˜ O ( k · | U i | ) .

Define the potential Φ := ∑ i ∈ [ t ] n ∗ i . For each update in U , the potential Φ increases by at most t , excluding the calls to Rebuild (see Algorithm 5 and Algorithm 6). Thus, by Lemma 3.2, each update increases the value of Φ by at most ˜ O (1) . Now, consider any call to ReconstructFromLayer ( i ) . Algorithm 7 and Invariant 3.1 ensure that just before this call, we had n ∗ i ≥ τn i = Ω( | U i | ) , since τ is a constant. Hence, because of Line 3 in Algorithm 4, during this call the value of Φ decreases by at least Ω( | U i | ) . Claim 3.8, in contrast, implies that the time taken to implement this call is ˜ O ( k · | U i | ) .

To summarize, each update in U creates ˜ O (1) units of potential, and the total time spent on the calls to Rebuild is at most ˜ O ( k ) times the decrease in the potential. Since the potential Φ is always nonegative, it follows that the amortized time spent on the calls to Rebuild is ˜ O ( k ) . This observation, along with Claim 3.7, implies Lemma 3.6.

4 Experimental Evaluation

Datasets. We conduct the empirical evaluation of our algorithm on five datasets from the UCI repository [17]: (1) Census [23] with 2 , 458 , 285 points of dimension 68 , (2) KDD-Cup [32] containing 311 , 029 points of dimension 74 , (3) Song [5] with 515 , 345 points of dimension 90 , (4) Drift [33, 30] with 13 , 910 points of dimension 129 , and (5) SIFT10M [17] with 11 , 164 , 866 points of dimension 128 . 3

We generate the dynamic instances that we use in our study as follows. We keep the first 10 , 000 points of each dataset, in the order that they appear in their file. This choice allows us to test against competing algorithms that are not as efficient, and at the same time captures the different practical aspects of the algorithms that we consider; as a sanity check, we perform an experiment on an instance that is an order of magnitude larger to confirm the scalability of our algorithm, and that the relative behavior in terms of the cost of the solutions remains the same among the tested algorithms.

We use the L 2 distance of the embeddings of two points to compute their distance. To avoid zero distance without altering the structural properties of the instances, we add 1 / | U | to all distances, where U is the set of points of the instance.

Order of Updates. We obtain a dynamic sequence of updates following the sliding window approach. In this typical approach, one defines a parameter κ indicating the size of the window, and then 'slides' the window over the static sequence of points in steps creating at each step i a point insertion of the i -th point (if there are more than i points) and a point deletion of the ( i -κ ) -th point (if κ > i ), until each point of the static instance is inserted once and deleted once. We set κ = 2 , 000 , which results in our dynamic instances having at most 2 , 000 points at any point in time.

Algorithms. We compare our algorithm from Section 3 against the state-of-the-art algorithm for maintaining a solution to the k -median problem, which is obtained by dynamically maintaining a coreset for the same problem [20]. In particular, [20] presented an algorithm that can dynamically maintain an ϵ -coreset of size O ( ϵ -2 k poly(log n, log(1 /ϵ ))) for the k -median and the k -means problems. Their algorithm has a worst-case update time of O ( ϵ -2 k 2 poly(log n, log(1 /ϵ ))) . To the best of our knowledge, this is the first implementation of the algorithm in [20]. For brevity, we refer to our algorithm as OURALG, and the algorithm from [20] as HK.

Since the exact constants and thresholds required to adhere to the theoretical guarantees of [20] are impractically large, there is no obvious way to use the thresholds provided by the algorithm in practice without making significant modifications. In order to give a fair comparison of the two algorithms, we implemented each of them to have a single parameter controlling a trade-off between update time, query time, and the cost of the solution. OURALG has a parameter ϕ which we set to be the number of points sampled during the construction of a layer in Algorithm 2. In other words, we sample ϕ points in Line 1 of Algorithm 2 instead of αk ′ . We fix the constants ϵ and β used by our algorithm (see Section 3) to 0 . 2 and 0 . 5 respectively. HK has a parameter ψ which we set to be the number of points sampled during the static coreset construction from [9] which is used as a black box, replacing the large threshold required to adhere to the theoretical guarantees. Since this threshold is replaced by a single parameter, it also replaces the other parameters used by HK, which are only used in this thresholding process. For the bicriteria algorithm required by this static coreset construction, we give the same implementation as the empirical evaluations in [9] and use kmeans++ with 2 iterations of Lloyd's.

For each of ϕ and ψ , we experimented with the values 250 , 500 , 1000 . As the values of ϕ and ψ are increased, the asymptotic update times and query times of the corresponding algorithms increase, while the cost of the solution decreases.

Setup. All of our code is written in Java and is available online. 4 We did not use parallelization. We used a machine with 8 cores, a 2.9Ghz processor, and 16 GiB of main memory.

Results. We compare OURALG ( ϕ = 500) against HK ( ψ = 1000) . We selected these parameters such that they obtain a good solution quality, without these parameters being unreasonably high w.r.t. the size of our datasets. Our experiments suggest that the solution quality produced by OURALG is robust against different values of ϕ (typically these versions of the algorithm differ by less than 1% ), while that of HK is more sensitive (in several instances the differences are in the range of 3 -9% , while for KDD-Cup HK ( ψ = 250) produces much worse solutions). The update time among the different versions of each algorithm do not differ significantly. Lastly, the query time of OURALG is less sensitive compared to the query time of HK. Specifically, the average query time of OURALG ( ϕ = 1000) is roughly 3 times slower than OURALG ( ϕ = 250) , while the average query time of HK ( ψ = 1000) is 11 times slower compared to HK ( ψ = 250) . For OURALG we selected the middle value ϕ = 500 . Since the solution quality of HK drops significantly when using smaller values of ψ , we selected ψ = 1000 to prioritize for the quality of the solution produced by the algorithm. Since we did not observe significant difference in the behavior across the datasets, we tuned these parameters considering three or our datasets. We provide the relevant plots in Appendix E.4.

Update Time Evaluation. In Figure 1 (left) we plot the total update time over the sequence of updates for the dataset Song , and only for k = 50 . OURALG runs up to more than 2 orders of

magnitude faster compared to HK, independently of their parameter setup (see Appendix E for more details). The relative performance of the algorithms is similar for all datasets and choices of k ; we summarize the results in Table 1, while the corresponding plots can be found in Appendix E.

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

10

5

10

3

10

1

10 1

10 3

OurAlgo(

=500)

HK(

=1000)

Total Update Time (sec) (song, k = 50)

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

1e6

OurAlgo(

=500)

HK(

=1000)

Cost of Solution (song, k = 50)

Solution Cost Evaluation. Figure 1 (right) shows the cost of the solution obtained by OURALG ( ϕ = 500) and HK ( ψ = 1000) , on dataset Song for k = 50 . In general, the two algorithms have similar performance with OURALG ( ϕ = 500) producing slightly better ( 1 . 3% -6 . 5% ) solutions on most instances; the exceptions being on the KDD-Cup dataset, where for k = 10 HK ( ψ = 1000) performs 1 . 2% better, and for k = 100 where OURALG ( ϕ = 500) performs 35% better. We summarize their relevant performance on the different datasets in Table 2, and provide the complete set of plots in the Appendix.

Query Time Evaluation. Finally, we compare OURALG ( ϕ = 500) to HK ( ψ = 1000) in terms of their query time. To measure query time, we evenly distribute 100 queries across the update sequence and measure the average query time. In Table 3 we report the average ratio of the query times obtained by OURALG ( ϕ = 500) divided by that of HK ( ψ = 1000) over the queries executed on each of the datasets and parameters of k that we considered. The query time obtained by the two algorithms is typically within a factor 2 of each other. OURALG ( ϕ = 500) performs better on KDD-Cup , while HK ( ψ = 1000) performs better on rest of the datasets. This inconsistency is due to differences in the structure of the datasets (we also tried randomizing the order of the updates, and it didn't change the relative picture w.r.t. query time).

Longer Sequence of Updates. While the above experiments capture well the different aspects of the two algorithms, we further conducted an experiment on the KDD-Cup dataset where we consider

a window size of κ = 5 , 000 and applied it to the first 100 , 000 points. As expected the relative performance in terms of solution cost and query time remains the same, while the gap in terms of update time grows significantly (HK ( ψ = 1000) performs more than 3 orders of magnitude worse than OURALG ( ϕ = 500) ). The corresponding plots and numbers are provided in the Appendix.

Conclusion of the Experiments. Our experimental evaluation shows the practicality of our algorithms on a set of five standard datasets used for the evaluation of dynamic algorithms in metric spaces. In particular, our experiments verify the theoretically superior update time of our algorithm w.r.t. to the state-of-the-art algorithm for the fully-dynamic k -median problem [20], where our algorithm performs orders of magnitude faster than HK. On top of that, the solution quality produced by our algorithm is better than HK in most settings, and the query times of the two algorithms are comparable. We believe that our algorithm is the best choice when it comes to efficiently maintaining a dynamic solution for the k -median problem in practice.

References

Sara Ahmadian, Ashkan Norouzi-Fard, Ola Svensson, and Justin Ward. Better guarantees for k-means and euclidean k-median by primal-dual algorithms. SIAM Journal on Computing , 49(4):FOCS17-97, 2019.

Nir Ailon, Ragesh Jaiswal, and Claire Monteleoni. Streaming k-means approximation. Advances in neural information processing systems , 22, 2009.

MohammadHossein Bateni, Hossein Esfandiari, Hendrik Fichtenberger, Monika Henzinger, Rajesh Jayaram, Vahab Mirrokni, and Andreas Wiese. Optimal fully dynamic k -center clustering for adaptive and oblivious adversaries. SODA , 2023.

Jon Louis Bentley and James B Saxe. Decomposable searching problems i. static-to-dynamic transformation. Journal of Algorithms , 1(4):301-358, 1980.

Thierry Bertin-Mahieux, Daniel PW Ellis, Brian Whitman, and Paul Lamere. The million song dataset. 2011.

Sayan Bhattacharya, Silvio Lattanzi, and Nikos Parotsidis. Efficient and stable fully dynamic facility location. Advances in neural information processing systems , 2022.

Michele Borassi, Alessandro Epasto, Silvio Lattanzi, Sergei Vassilvitskii, and Morteza Zadimoghaddam. Sliding window algorithms for k-clustering problems. Advances in Neural Information Processing Systems , 33:8716-8727, 2020.

Vladimir Braverman, Vincent Cohen-Addad, Shaofeng H.-C. Jiang, Robert Krauthgamer, Chris Schwiegelshohn, Mads Bech Toftrup, and Xuan Wu. The power of uniform sampling for coresets. In 63rd IEEE Annual Symposium on Foundations of Computer Science, FOCS 2022, Denver, CO, USA, October 31 - November 3, 2022 , pages 462-473. IEEE, 2022.

Vladimir Braverman, Dan Feldman, and Harry Lang. New frameworks for offline and streaming coreset constructions. CoRR , abs/1612.00889, 2016.

Vladimir Braverman, Gereon Frahling, Harry Lang, Christian Sohler, and Lin F Yang. Clustering high dimensional dynamic data streams. In International Conference on Machine Learning , pages 576-585. PMLR, 2017.

Vladimir Braverman, Harry Lang, Keith Levin, and Morteza Monemizadeh. Clustering problems on sliding windows. In Proceedings of the twenty-seventh annual ACM-SIAM symposium on Discrete algorithms , pages 1374-1390. SIAM, 2016.

Jarosław Byrka, Thomas Pensyl, Bartosz Rybicki, Aravind Srinivasan, and Khoa Trinh. An improved approximation for k-median and positive correlation in budgeted optimization. ACM Transactions on Algorithms (TALG) , 13(2):1-31, 2017.

TH Hubert Chan, Arnaud Guerqin, and Mauro Sozio. Fully dynamic k-center clustering. In Proceedings of the 2018 World Wide Web Conference , pages 579-587, 2018.

Moses Charikar, Sudipto Guha, Éva Tardos, and David B Shmoys. A constant-factor approximation algorithm for the k-median problem. In Proceedings of the thirty-first annual ACM symposium on Theory of computing , pages 1-10, 1999.

Moses Charikar, Liadan O'Callaghan, and Rina Panigrahy. Better streaming algorithms for clustering problems. In Proceedings of the thirty-fifth annual ACM symposium on Theory of computing , pages 30-39, 2003.

Vincent Cohen-Addad, Niklas Oskar D Hjuler, Nikos Parotsidis, David Saulpic, and Chris Schwiegelshohn. Fully dynamic consistent facility location. Advances in Neural Information Processing Systems , 32, 2019.

Dheeru Dua and Casey Graff. UCI machine learning repository, 2017.

Gramoz Goranci, Monika Henzinger, Dariusz Leniowski, Christian Schulz, and Alexander Svozil. Fully dynamic k -center clustering in low dimensional metrics. In Proceedings of the Symposium on Algorithm Engineering and Experiments, ALENEX 2021, Virtual Conference, January 10-11, 2021 , pages 143-153. SIAM, 2021.

Sudipto Guha, Nina Mishra, Rajeev Motwani, and Liadan O'Callaghan. Clustering data streams. In 41st Annual Symposium on Foundations of Computer Science , pages 359-366. IEEE Computer Society, 2000.

Monika Henzinger and Sagar Kale. Fully-dynamic coresets. In 28th Annual European Symposium on Algorithms, ESA 2020, September 7-9, 2020, Pisa, Italy (Virtual Conference) , volume 173 of LIPIcs , pages 57:1-57:21, 2020.

Lingxiao Huang and Nisheeth K. Vishnoi. Coresets for clustering in euclidean spaces: importance sampling is nearly optimal. In Proccedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing, STOC 2020, Chicago, IL, USA, June 22-26, 2020 , pages 1416-1429. ACM, 2020.

Kamal Jain and Vijay V Vazirani. Approximation algorithms for metric facility location and k-median problems using the primal-dual schema and lagrangian relaxation. Journal of the ACM (JACM) , 48(2):274-296, 2001.

Ron Kohavi et al. Scaling up the accuracy of naive-bayes classifiers: A decision-tree hybrid. In Kdd , volume 96, pages 202-207, 1996.

Silvio Lattanzi and Sergei Vassilvitskii. Consistent k-clustering. In International Conference on Machine Learning , pages 1975-1984. PMLR, 2017.

Quanquan C. Liu, Jessica Shi, Shangdi Yu, Laxman Dhulipala, and Julian Shun. Parallel batch-dynamic algorithms for k-core decomposition and related graph problems. In 34th ACM Symposium on Parallelism in Algorithms and Architectures (SPAA) , pages 191-204. ACM, 2022.

Ramgopal R. Mettu. Approximation Algorithms for NP-Hard Clustering Problems . PhD thesis, Department of Computer Science, University of Texas at Austin, 2002.

Ramgopal R. Mettu and C. Greg Plaxton. The online median problem. In 41st Annual Symposium on Foundations of Computer Science, FOCS 2000, 12-14 November 2000, Redondo Beach, California, USA , pages 339-348. IEEE Computer Society, 2000.

Ramgopal R. Mettu and C. Greg Plaxton. Optimal time bounds for approximate clustering. In UAI '02, Proceedings of the 18th Conference in Uncertainty in Artificial Intelligence , pages 344-351. Morgan Kaufmann, 2002.

Krzysztof Nowicki and Krzysztof Onak. Dynamic graph algorithms with batch updates in the massively parallel computation model. In Proceedings of the 2021 ACM-SIAM Symposium on Discrete Algorithms (SODA) , pages 2939-2958. SIAM, 2021.

Irene Rodriguez-Lujan, Jordi Fonollosa, Alexander Vergara, Margie Homer, and Ramon Huerta. On the calibration of sensor arrays for pattern recognition using the minimal number of experiments. Chemometrics and Intelligent Laboratory Systems , 130:123-134, 2014.

Michael Shindler, Alex Wong, and Adam Meyerson. Fast and accurate k-means for large datasets. Advances in neural information processing systems , 24, 2011.

S.J. Stolfo, Wei Fan, Wenke Lee, A. Prodromidis, and P.K. Chan. Cost-based modeling for fraud and intrusion detection: results from the jam project. In Proceedings DARPA Information Survivability Conference and Exposition. DISCEX'00 , volume 2, pages 130-144 vol.2, 2000.

Alexander Vergara, Shankar Vembu, Tuba Ayhan, Margaret A Ryan, Margie L Homer, and Ramón Huerta. Chemical gas sensor drift compensation using classifier ensembles. Sensors and Actuators B: Chemical , 166:320-329, 2012.

A Extension to k -Means and ( k, p ) -Clustering

As stated in [21, 8], while [28] only discusses the k -median problem, their construction can easily be modified to work for k -means clustering and further generalized to work for ( k, p ) -clustering, where the ( k, p ) -clustering problem is defined in the same way as k -median problem except that we want to minimize ∑ x ∈ U d ( x, S ) p for some S ⊆ U of size at most k . Note that ( k, 1) -clustering and ( k, 2) -clustering correspond to k -median and k -means respectively.

We define a ρ -metric space ( U, d ) in the same way as a metric space except for relaxing the condition that d must satisfy the triangle inequality to the condition that d ( x, y ) ≤ ρ ( d ( x, z ) + d ( z, y )) for all x, y, z ∈ U . Given a metric space ( U, d ) and some p ≥ 1 , the results in Section 6 of [9] can easily be used to show that ( U, d p ) is a 2 p -1 -metric space, where d p ( x, y ) is defined to be d ( x, y ) p .

We now show that the assignment σ maintained by our algorithm is O ( ρ 3 ) -approximate when U is a ρ -metric space (i.e. that cost ( σ ) = O ( ρ 3 ) · opt ( U ) ) and that the extraction technique of [19] can be generalized to ρ -metric spaces.

Lemma A.1. When the underlying space U is a ρ -metric space, the assignment σ maintained by our algorithm is O ( ρ 3 ) -approximate.

Proof. By making the appropriate changes to the proofs of Lemma B.3 and Lemma B.4, we get generalizations of these lemmas to ρ -metric spaces, where the lemma statements are the same except for an extra ρ factor in the inequalities.

Lemma A.2. Given any positive ξ , there exists a sufficiently large choice of α such that ν i ≤ 2 ρ · µ γ ( U OLD i ) for each i ∈ [ t -1] with probability at least 1 -e -ξk ′ .

Lemma A.3. Given metric subspaces U 1 and U 2 of U such that | U 1 ⊕ U 2 | ≤ ϵγ | U 1 | , we have that µ γ ( U 1 ) ≤ 2 ρ · µ γ ∗ ( U 2 ) .

These two lemmas immediately imply the following generalization of Lemma B.5.

Lemma A.4. ν i ≤ 4 ρ 2 · µ γ ∗ ( U i ) for each i ∈ [ t -1] with probability at least 1 -e -ξk ′ .

The upper bound on cost ( σ ) given in Lemma B.6 can be generalized by noticing that cost ( σ, C i ) ≤ 2 ρν i | C i | for all i ∈ [ t -1] , which us that

The lower bound on opt ( U ) given in Lemma B.10 holds for ρ -metric spaces with no modifications. Hence, we get that with probability at least 1 -e -ξk ′ we have that

By making the appropriate modifications to the proof of Theorem C.1, we can extend this theorem to work for ρ -metric spaces. In particular, we can obtain a proof of Theorem A.5 by taking the proof of Theorem C.1 and adding extra ρ factors whenever the triangle inequality is applied.

Theorem A.5. Given a ϕ -approximate m -assignment π : U → U , any ψ -approximate solution to the weighted k -median instance ( π ( U ) , d, w ) , where each point x ∈ π ( U ) receives weight w ( x ) := | π -1 ( x ) | , is also a ( ϕρ +2(1 + ϕ ) ψρ 3 ) -approximate solution to the k -median instance ( U, d ) where U is a ρ -metric space.

Since the algorithm in [27] is O (1) -approximate on O (1) -metric spaces, it immediately follows by applying Theorem A.5 and Lemma A.1 that our algorithm maintains a O (1) -approximate solution to the k -median problem on ( U, d p ) for p = O (1) . Since the k -median problem on ( U, d p ) is exactly the ( k, p ) -clustering problem on ( U, d ) , it follows that our algorithm generalizes to solve instances of ( k, p ) -clustering in metric spaces.

B Proofs of Lemma 3.2 and Lemma 3.3

Throughout this section, we fix γ to be any real such that β < γ < 1 and ϵ to be any real such that 0 < ϵ < min { 1 -γ 2 γ , 1 } . Let β ∗ and γ ∗ denote β (1 -ϵ ) and γ (1 + 2 ϵ ) respectively.

B.1 Proof of Lemma 3.2

We first prove Lemma B.1, which shows that the sizes of the sets U i decrease exponentially with i . Lemma B.1. For all i ∈ [ t -1] , | U i +1 | ≤ (1 -β ∗ ) | U i | .

Proof. Consider the ratio | U i +1 | / | U i | . Since U i +1 ⊆ U i and U i +1 is reconstructed every time U i is reconstructed, it follows that | U i +1 | / | U i | is at most ( n i +1 + ℓ ) / ( n i + ℓ -ℓ ′ ) , where n j is the size of U j at the time it was last reconstructed and ℓ and ℓ ′ are the number of insertions and deletions that have occurred since the last time U i +1 was reconstructed respectively. By Lemma B.2, we get that this expression is upper bounded by ( n i +1 + τn i +1 ) /n i . Now we can observe that

where we use the facts that n i +1 ≤ (1 -β ) n i and τ ≤ ϵβ in the final inequality.

Lemma B.2. Given some integer i ∈ [ t -1] , let ℓ and ℓ ′ be the number of insertions and deletions that have occurred since the last time U i +1 was reconstructed respectively. Then we have that

Proof. First, note that ( n i +1 + ℓ ) / ( n i + ℓ -ℓ ′ ) ≤ ( n i +1 + ℓ ) / ( n i -ℓ ′ ) . Now, given some reals A ≥ a ≥ 0 and 0 ≤ N ≤ A -a , we define a function f : [0 , 1] → R by f ( x ) = ( a + xN ) / ( A -(1 -x ) N ) . The derivative of f is -N ( a -A + N ) / (( x -1) N + A ) 2 and is non-negative for all x ∈ [0 , 1] . Hence, f ( x ) ≤ f (1) for all x ∈ [0 , 1] .

By setting A = n i , a = n i +1 , N = ℓ + ℓ ′ and noting that ℓ + ℓ ′ ≤ τn i +1 by Invariant 3.1 and n i +1 ≤ (1 -β ) n i , we get that

and hence it follows that

Since | U 1 | = | U | , | U t -1 | > (1 -τ ) αk ′ = Ω( k ) , and β ∗ is a constant, it follows from Lemma B.1 that t = O (log( | U | /k )) = ˜ O (1) .

B.2 Proof of Lemma 3.3

Bounding the Radii ν i (Lemma B.5). Let U OLD i denote the state of the i th layer the last time it was reconstructed for i ∈ [ t ] . We now use the following crucial lemma which is analogous to Lemma 4.3.3 in [26].

Lemma B.3. Given any positive ξ , there exists a sufficiently large choice of α such that ν i ≤ 2 µ γ ( U OLD i ) for each i ∈ [ t -1] with probability at least 1 -e -ξk ′ .

Henceforth, we fix some positive ξ and sufficiently large α such that Lemma B.3 holds.

Lemma B.4. Given metric subspaces U 1 and U 2 of U such that | U 1 ⊕ U 2 | ≤ ϵγ | U 1 | , we have that µ γ ( U 1 ) ≤ 2 µ γ ∗ ( U 2 ) . 5

Proof. Let X be a subset of U 2 of size k such that ν γ (1+2 ϵ ) ( X,U 2 ) = µ γ (1+2 ϵ ) ( U 2 ) , ρ = µ γ (1+2 ϵ ) ( U 2 ) , and A = B U 1 ( X,ρ ) . Now note that

Since there also exists a subset Y ⊆ A of size k such that A ⊆ B U 1 ( Y, 2 ρ ) , it follows that ν γ ( Y, U 1 ) ≤ 2 ρ . Hence, µ γ ( U 1 ) ≤ ν γ ( Y, U 1 ) ≤ 2 µ γ (1+2 ϵ ) ( U 2 ) .

Lemma B.5. ν i ≤ 4 µ γ ∗ ( U i ) for each i ∈ [ t -1] with probability at least 1 -e -ξk ′ .

Proof. For each i ∈ [ t -1] , | U i ⊕ U OLD i | ≤ τ | U OLD i | since, by Invariant 3.1, at most τ | U OLD i | points have been inserted or deleted from U i since it was last reconstructed. Noticing that τ ≤ ϵγ , we can see that

By now applying Lemma B.4 it follows that µ γ ( U OLD i ) ≤ 2 µ γ ∗ ( U i ) . The lemma follows by combining this result with Lemma B.3.

Upper Bounding cost ( σ ) (Lemma B.6).

Lemma B.6.

Proof. We first note that for all i ∈ [ t -1] , cost ( σ, C i ) ≤ 2 ν i | C i | . This follows directly from the fact that each point x in C i is assigned to some point y ∈ C i such that d ( x, y ) ≤ 2 ν i . Since the C i partition U and cost ( σ, C t ) = 0 , we get:

Lower Bounding opt ( U ) (Lemma B.10). Let r denote ⌈ log 1 -β ∗ 1 -γ ∗ 3 ⌉ and for each i ∈ [ t ] let µ i denote µ γ ∗ ( U i ) .

For the rest of this subsection we fix an arbitrary S ⊆ U of size k . For each i ∈ [ t ] , let F i denote the set { x ∈ U i | d ( x, S ) ≥ µ i } , and for any integer m> 0 , let F i m denote F i \ ( ∪ j> 0 F i + jm ) and G i,m denote the set of all integers j ∈ [ t ] and j ≡ i (mod m ) .

Lemma B.7. Given some i ∈ [ t ] and a subset X ⊆ F i , we have that | F i | ≥ (1 -γ ∗ ) | U i | and cost ( S, X ) ≥ µ i | X | .

Proof. It follows directly from the definition of µ i that we have that | F i | ≥ (1 -γ ∗ ) | U i | . By the definition of F i , we have that cost ( S, X ) = ∑ x ∈ X d ( x, S ) ≥ µ i | X | .

The following lemma is proven in [26].

Lemma B.8 ([26], Lemma 4.3.8) . Given integers ℓ ∈ [ t ] and m> 0 , we have that

Lemma B.9. For all i ∈ [ t -1] , we have that | F r i | ≥ 1 2 | F i | .

Proof. We first note that for all i ∈ [ t -r ] , we have that | F i + r | ≤ 1 3 | F i | . This follows from the fact that

where the first inequality follows from the fact that F i + r ⊆ U i + r , the second inequality follows from Lemma B.1, the third inequality follows from Lemma B.7, and the fourth inequality follows from the definition of r . We now get that

Lemma B.10.

Proof. Let ℓ = arg max 0 ≤ ℓ<r { ∑ i ∈ G ℓ,r µ i | F r i |} . Then we have that

The second inequality follows from Lemma B.8, the third inequality from averaging and the choice of ℓ , the fourth inequality from Lemma B.9, and the fifth inequality from Lemma B.7.

Proof of Lemma 3.3. It follows that with probability at least 1 -e -ξk ′ we have that

for any set S ⊆ U of size k . Hence, we have that

C Proof of Corollary 3.4

In order to prove this corollary, we apply the extraction technique presented in [28] (with full details appearing in [26]) which is a slight generalization of the techniques from [19]. In particular, we use the following theorem which follows as an immediate corollary of Theorem 6 in [26]. For completeness, we provide a proof of this theorem.

Theorem C.1. Given a ϕ -approximate m -assignment π : U → U , any ψ -approximate solution to the weighted k -median instance ( π ( U ) , d, w ) , where each point x ∈ π ( U ) receives weight w ( x ) := | π -1 ( x ) | , is also a ( ϕ +2(1 + ϕ ) ψ ) -approximate solution to the k -median instance ( U, d ) .

Proof. Let S ∗ be a solution to the weighted k -median instance ( π ( U ) , d, w ) and let S be an optimal solution to the k -median instance ( U, d ) . Let ϕ and ψ be constants such that cost ( π, U ) ≤ ϕ · opt ( U ) and cost w ( S ∗ , π ( U )) ≤ ψ · opt w ( π ( U )) . We now show that cost ( S ∗ , U ) = O (1) · opt ( U ) . We first note that

Now note that, for any X ⊆ U of size at most k , there exists some Y ⊆ π ( U ) of size at most k such that cost w ( Y, π ( U )) ≤ 2 · cost w ( X,π ( U )) . Since cost w ( S ∗ , π ( U )) ≤ ψ · cost w ( Y, π ( U )) for all Y ⊆ π ( Y ) of size at most k , we get the following.

By combining these two chains of inequalities, we get that

It immediately follows that we can get a O (1) -approximate solution to the instance ( U, d ) by running a static weighted k -median algorithm on the instance ( σ ( U ) , d, w ) .

D Lower Bounds on Update and Query Time

In the static (i.e. non-dynamic) setting, the k -median problem is defined as follows: given a metric space U , return a set S of at most k points from U which minimizes the value of ∑ x ∈ S d ( x, S ) . The following lower bound for the static k -median problem is proven by Mettu in [26].

Theorem D.1. Any O (1) -approximate randomized (static) algorithm for the k -median problem, which succeeds with even negligible probability, runs in time Ω( nk ) .

Informally, the proof of this lower bound is obtained by constructing, for each δ > 0 , an input distribution of metric spaces (with polynomially bounded aspect ratio) on which no deterministic algorithm for the k -median problem succeeds with probability more than δ . Theorem D.1 then follows by an application of Yao's minmax principle.

We can use this lower bound from the static setting in order to get a lower bound for the dynamic setting. First note that any incremental algorithm for k -median with amortized update time u ( n, k ) and query time q ( n, k ) can be used to construct a static algorithm for the k -median problem with running time n · u ( n, k ) + q ( n, k ) by inserting each point in the input metric space U followed by a solution query. Hence, by Theorem D.1, we must have that n · u ( n, k ) + q ( n, k ) = Ω( nk ) . Now assume that some incremental algorithm for k -median has query time ˜ O (poly( k )) . If this algorithm also has an amortized update time of ˜ o ( k ) , then for the range of values of k where q ( n, k ) = ˜ o ( nk ) , it follows that ˜ o ( nk ) is Ω( nk ) , giving a contradiction. Hence, the amortized update time must be ˜ Ω( k ) and Theorem D.2 follows.

Theorem D.2. Any O (1) -approximate incremental algorithm for the k -median problem with ˜ O (poly( k )) query time must have ˜ Ω( k ) amortized update time.

It follows that the update time of our algorithm is optimal up to polylogarithmic factors.

E Omitted Experimental Results

E.1 Update Time Evaluation

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

10

5

10

3

10

1

10 1

10 3

OurAlgo(

=500)

HK(

=1000)

Total Update Time (sec) (song, k = 10)

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

10

5

10

3

10

1

10 1

10 3

OurAlgo(

=500)

HK(

=1000)

Total Update Time (sec) (song, k = 50)

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

10

5

10

3

10

1

10 1

10 3

OurAlgo(

=500)

HK(

=1000)

Total Update Time (sec) (song, k = 100)

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

10

5

10

3

10

1

10 1

10 3

OurAlgo(

=500)

HK(

=1000)

Total Update Time (sec) (census, k = 10)

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

10

5

10

3

10

1

10 1

10 3

OurAlgo(

=500)

HK(

=1000)

Total Update Time (sec) (census, k = 50)

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

10

5

10

3

10

1

10 1

10 3

OurAlgo(

=500)

HK(

=1000)

Total Update Time (sec) (census, k = 100)

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

10

5

10

3

10

1

10 1

10 3

OurAlgo(

=500)

HK(

=1000)

Total Update Time (sec) (kddcup, k = 10)

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

10

5

10

3

10

1

10 1

10 3

OurAlgo(

=500)

HK(

=1000)

Total Update Time (sec) (kddcup, k = 50)

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

10

5

10

3

10

1

10 1

10 3

OurAlgo(

=500)

HK(

=1000)

Total Update Time (sec) (kddcup, k = 100)

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

10

4

10

3

10

2

10

1

10 0

10 1

10 2

10 3

OurAlgo(

=500)

HK(

=1000)

Total Update Time (sec) (drift, k = 10)

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

10

4

10

2

10 0

10 2

10 4

OurAlgo(

=500)

HK(

=1000)

Total Update Time (sec) (drift, k = 50)

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

10

5

10

3

10

1

10 1

10 3

OurAlgo(

=500)

HK(

=1000)

Total Update Time (sec) (drift, k = 100)

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

10

4

10

3

10

2

10

1

10 0

10 1

10 2

10 3

OurAlgo(

=500)

HK(

=1000)

Total Update Time (sec) (sift10M, k = 10)

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

10

5

10

3

10

1

10 1

10 3

OurAlgo(

=500)

HK(

=1000)

Total Update Time (sec) (sift10M, k = 50)

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

10

5

10

3

10

1

10 1

10 3

OurAlgo(

=500)

HK(

=1000)

Total Update Time (sec) (sift10M, k = 100)

E.2 Solution Cost Evaluation

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

0

1

2

3

4

1e6

OurAlgo(

=500)

HK(

=1000)

Cost of Solution (song, k = 10)

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

1e6

OurAlgo(

=500)

HK(

=1000)

Cost of Solution (song, k = 50)

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

0.0

0.5

1.0

1.5

2.0

2.5

3.0

1e6

OurAlgo(

=500)

HK(

=1000)

Cost of Solution (song, k = 100)

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

0

5000

10000

15000

20000

OurAlgo(

=500)

HK(

=1000)

Cost of Solution (census, k = 10)

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

0

2500

5000

7500

10000

12500

15000

OurAlgo(

=500)

HK(

=1000)

Cost of Solution (census, k = 50)

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

0

2000

4000

6000

8000

10000

12000

14000

OurAlgo(

=500)

HK(

=1000)

Cost of Solution (census, k = 100)

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

0

5000

10000

15000

20000

25000

OurAlgo(

=500)

HK(

=1000)

Cost of Solution (kddcup, k = 10)

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

0

1000

2000

3000

4000

5000

6000

7000

OurAlgo(

=500)

HK(

=1000)

Cost of Solution (kddcup, k = 50)

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

0

1000

2000

3000

4000

5000

OurAlgo(

=500)

HK(

=1000)

Cost of Solution (kddcup, k = 100)

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

0

1

2

3

4

1e8

OurAlgo(

=500)

HK(

=1000)

Cost of Solution (drift, k = 10)

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

1e8

OurAlgo(

=500)

HK(

=1000)

Cost of Solution (drift, k = 50)

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

0.0

0.5

1.0

1.5

2.0

2.5

1e8

OurAlgo(

=500)

HK(

=1000)

Cost of Solution (drift, k = 100)

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

0

200000

400000

600000

800000

OurAlgo(

=500)

HK(

=1000)

Cost of Solution (sift10M, k = 10)

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

0

100000

200000

300000

400000

500000

600000

700000

OurAlgo(

=500)

HK(

=1000)

Cost of Solution (sift10M, k = 50)

0

2500

5000

7500

10000

12500

15000

17500

20000

Updates

0

100000

200000

300000

400000

500000

600000

700000

OurAlgo(

=500)

HK(

=1000)

Cost of Solution (sift10M, k = 100)

E.3 Query Time Evaluation

E.4 Parameter Tuning

E.4.1 Update Time

0

1000

2000

3000

4000

5000

6000

Updates

10

6

10

4

10

2

10 0

10 2

OurAlgo(

=250)

OurAlgo(

=500)

OurAlgo(

=1000)

HK(

=250)

HK(

=500)

HK(

=1000)

Total Update Time (sec) (song, k = 50)

0

1000

2000

3000

4000

5000

6000

Updates

10

6

10

4

10

2

10 0

10 2

OurAlgo(

=250)

OurAlgo(

=500)

OurAlgo(

=1000)

HK(

=250)

HK(

=500)

HK(

=1000)

Total Update Time (sec) (census, k = 50)

0

1000

2000

3000

4000

5000

6000

Updates

10

5

10

3

10

1

10 1

10 3

OurAlgo(

=250)

OurAlgo(

=500)

OurAlgo(

=1000)

HK(

=250)

HK(

=500)

HK(

=1000)

Total Update Time (sec) (kddcup, k = 50)

E.4.2 Solution Cost

0

1000

2000

3000

4000

5000

6000

Updates

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

1e6

OurAlgo(

=250)

OurAlgo(

=500)

OurAlgo(

=1000)

HK(

=250)

HK(

=500)

HK(

=1000)

Cost of Solution (song, k = 50)

0

1000

2000

3000

4000

5000

6000

Updates

0

2500

5000

7500

10000

12500

15000

17500

OurAlgo(

=250)

OurAlgo(

=500)

OurAlgo(

=1000)

HK(

=250)

HK(

=500)

HK(

=1000)

Cost of Solution (census, k = 50)

0

1000

2000

3000

4000

5000

6000

Updates

0

2000

4000

6000

8000

10000

OurAlgo(

=250)

OurAlgo(

=500)

OurAlgo(

=1000)

HK(

=250)

HK(

=500)

HK(

=1000)

Cost of Solution (kddcup, k = 50)

E.4.3 Query Time

E.5 Randomized Order of Updates

E.5.1 Update Time

0

1000

2000

3000

4000

5000

6000

Updates

10

6

10

4

10

2

10 0

10 2

OurAlgo(

=250)

OurAlgo(

=500)

OurAlgo(

=1000)

HK(

=250)

HK(

=500)

HK(

=1000)

Total Update Time (sec) (song, k = 50)

0

1000

2000

3000

4000

5000

6000

Updates

10

6

10

4

10

2

10 0

10 2

OurAlgo(

=250)

OurAlgo(

=500)

OurAlgo(

=1000)

HK(

=250)

HK(

=500)

HK(

=1000)

Total Update Time (sec) (census, k = 50)

0

1000

2000

3000

4000

5000

6000

Updates

10

6

10

4

10

2

10 0

10 2

OurAlgo(

=250)

OurAlgo(

=500)

OurAlgo(

=1000)

HK(

=250)

HK(

=500)

HK(

=1000)

Total Update Time (sec) (kddcup, k = 50)

E.5.2 Solution Cost

0

1000

2000

3000

4000

5000

6000

Updates

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

1e6

OurAlgo(

=250)

OurAlgo(

=500)

OurAlgo(

=1000)

HK(

=250)

HK(

=500)

HK(

=1000)

Cost of Solution (song, k = 50)

0

1000

2000

3000

4000

5000

6000

Updates

0

2500

5000

7500

10000

12500

15000

17500

OurAlgo(

=250)

OurAlgo(

=500)

OurAlgo(

=1000)

HK(

=250)

HK(

=500)

HK(

=1000)

Cost of Solution (census, k = 50)

0

1000

2000

3000

4000

5000

6000

Updates

0

2000

4000

6000

8000

10000

OurAlgo(

=250)

OurAlgo(

=500)

OurAlgo(

=1000)

HK(

=250)

HK(

=500)

HK(

=1000)

Cost of Solution (kddcup, k = 50)

E.5.3 Query Time

E.6 Larger Experiment

Total Update Time (sec) (kddcup, k = 50)

0

25000

50000

75000

100000 125000 150000 175000 200000

Updates

10

3

10

1

10 1

10 3

10 5

OurAlgo(

=500)

HK(

=1000)

0

25000

50000

75000

100000 125000 150000 175000 200000

Updates

0

2500

5000

7500

10000

12500

15000

17500

OurAlgo(

=500)

HK(

=1000)

Cost of Solution (kddcup, k = 50)

The average query times for OURALG ( ϕ = 500) and HK ( ψ = 1000) while handling this longer sequence of updates were 0 . 416 and 0 . 225 respectively.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies dynamic algorithms for k-median (and k-means) in general metrics. The main result is an \\tilde{O}(k) update time, poly(k) query time algorithm that achieves O(1)-approximation. Here, the update model is point insertion/deletion, and the algorithm has access to the distance oracle. A query operation asks to return a list of k points that is O(1)-approximate.\n\nThe result improves over the k^2 update time achieved in HK20. This is achieved using a somewhat different but more direct approach. In particular, in HK20, a general reduction to coreset via merge-and-reduce framework was employed, but this work employs a direct dynamization of MP04.\n\nComprehensive experiments are also provided. Compared with HK20 as a baseline, the new algorithm runs faster and achieves an overall better accuracy.\n\nSTRENGTHS:\n- The paper achieves near-linear in k update time, which is a nice improvement over the previous k^2\n- The experiments are convincing, and justify the theoretical improvement\n- A nearly-matching lower bound is provided\n\nWEAKNESSES:\n- This work only yields O(1)-approx, while HK20 uses coreset approach and hence can build an eps-coreset even for general metrics (even though it does not imply an efficient algorithm for finding a solution). This also means this work cannot be easily generalized to the Euclidean case, where near-linear time PTAS was known.\n- The query time is k^2, which may be improved\n\nQUESTIONS:\n- The first paragraph of \"our techniques\" tried to explain why HK20 has k^2 dependence, but it's not very clear. Here, the black-box coreset is linear in k, and the depth of the binary tree is O(log n). Then how come we have another k factor?\n- Followup of the above question: is it true that HK20 relies on a bi-criteria solution? Is that part constitute the bottleneck? Also, does HK20 give bi-criteria solution whose update time is near-linear in k? If so, then maybe it's also an interesting angle to compare with their result, since yours may be viewed as a (strict) improvement of it.\n- In the end of Sec 1, it is mentioned that an algorithm for weighted k-median is presented in Sec 2. But I didn't find it\n- In Sec 3.1, it seems the algorithm only returns a bi-criteria solution. What's the procedure for making it a real feasible solution? How does that procedure affects the final time complexity?\n\nLIMITATIONS:\nI didn't find these explicitly discussed. A discussion of limitations, and possibly mentioning future directions, could be helpful. This paper is a theory paper and I don't see a potential negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the $k$-median/means problems in fully dynamic settings. Clustering in fully dynamic setting is a recent hot topic, where fully dynamic $k$-center has been well studied in literature. However, little was known for fully dynamic $k$-median/means problems. Inspired by the static framework proposed by Mettu and Plaxton, where a minimum radius ball coverage strategy is used to obtain $t=O(logn)$ layers of representations for good approximation of the given $k$-median/means instance. This paper modifies the classic framework by Mettu and Plaxton and presents an $O(1)$-approximation algorithm for the fully dynamic $k$-median/means problems with $\\tilde{O}(k)$ amortized update time and $\\tilde{O}(k^2)$ worst case query time, which improves the previous coreset-based method with $\\tilde{O}(k^2)$ worst case update time.\n\nSTRENGTHS:\n1. This paper proposes simple but efficient approximation algorithm for the fully dynamic $k$-median/means problems with $\\tilde{O}(k)$ amortized update time and $\\tilde{O}(k^2)$ worst case query time, which improves the previous result with $\\tilde{O}(k^2)$ update time and $\\tilde{O}(k^2)$ query time. The authors also show that the update time of our algorithm is optimal up to polylogarithmic factors.\n\n2. This paper gives detailed experimental evaluation of fully dynamic k-median algorithms for general metrics and shows that the proposed framework is more efficient than previous ones.\n\nWEAKNESSES:\n1. The techniques used in this paper seem to rely heavily on the minimum ball coverage method by Mettu and Plaxton. \n\n2. The challenges for obtaining good update time and query time is not well discussed.\n\n3. The theoretical analysis is not an easy read in a limited time. The intuition behind the analysis and algorithm before going into the details of lemmas and proofs should be given before the proofs.\n\nQUESTIONS:\n1. In each insertion, why should the data point to be inserted added to each layer $i \\in [t]$ instead of one of the $t$ layers?\n\n2. Can the authors discuss the challenges for applying the Mettu's and Plaxton's method for solving the fully dynamic $k$-clustering problems?\n\nLIMITATIONS:\nSince this is a theoretical paper, I don't think there is potential negative societal impact of this paper.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nFully dynamic k-clustering in O(k) update time\n\nThis paper studies fully dynamic k-clustering. It gives a fully dynamic algorithm that maintains O(1)-approximate solutions to k-median and k-means with \\tilde O(k) amortized update time and \\tilde O(k^2) worst-case query time. On the negative side, the authors showed that the Omega(k) amortized update time is required if one needs to achieve O(1)-approximation ratio and poly(k) query time. So they gave the optimal update time, and the time improves the prior best-known update time of O(k^2).  The authors also did experiments to complement the theoretical analysis. \n\nThe dynamic algorithm is built on the static algorithm of [27]. It runs for many iterations. In each iteration i, the algorithm samples a set S_i of points and creates a set of smallest-radius balls around the samples so that the balls cover at least beta fraction of the remaining points, for a constant beta. The algorithm then removes the covered points and repeats the process.  This gives a partition of the points into \\tilde O(k) balls. They construct an assignment that maps every point to its ball center. Then a k-median or k-means solution can be constructed using the centers only. \n\nThe dynamic algorithm maintains the tree-structure constructed in the static algorithm in a lazy manner by allowing some slack at many places. It needs to rebuild the tree from some layer if the cost of the assignment at that layer or above becomes bad. On average, it needs many updates for the algorithm to rebuild a tree from some layer, giving a good amortized update time.\n\nSTRENGTHS:\nOverall, the paper gives a tight update time of \\tilde O(k) for the fully dynamic k-clustering problem, using elegant techniques. This is a solid accept.\n\nWEAKNESSES:\nThe hidden approximation ratio is a little big.\n\nQUESTIONS:\n1. Can you give a rough bound on the approximation ratio of the algorithm?\n\n2. What happens if each update contains a batch of p points? Can you achieve an update time of O(p + k), instead of O(pk)?\n\nLIMITATIONS:\nNo limitations.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-06"}

comments: SUMMARY:
This paper considers the dynamic version of the $k$-median and the $k$-means clustering problem in an arbitrary metric space. The authors provide an $O(k)$ amortized time (which is near optimal based on a lower bound that the authors provide) for insertions and deletions of points and a query time of $O(k^2)$. This is an improvement of the recent result by Henzinger and Kale, ESA 2020 which provided a dynamic algorithm with a worst case $O(k^2)$ update time. The algorithm in this paper is based on making the algorithm of Mettu and Plaxton dynamic. The authors also provide implementations of their algorithm as well as prior works (including a coreset construction algorithm) and then provide an empirical performance analysis of these algorithms.

STRENGTHS:
This paper is providing a near-optimal bound for amortized update time for dynamic k-means and k-median clustering problems.

 The authors provide implementations of their algorithm as well as prior works which is very useful.

WEAKNESSES:
In terms of techniques, the methods used are adaptation of an existing algorithm of Mettu and Plaxton and perhaps somewhat incremental in nature.

QUESTIONS:
My questions were answered in a previous review cycle

LIMITATIONS:
NA
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-07