A Constant Approximation Algorithm for Sequential Random-Order No-Substitution k-Median Clustering

1 Introduction . Clustering is a fundamental unsupervised learning task used for various applications , such as anomaly detection [ Leung and Leckie , 2005 ] , recommender systems [ Shepitsen et al. , 2008 ] and cancer diagnosis [ Zheng et al. , 2014 ] . In recent years , the problem of sequential clustering has been actively studied , motivated by applications in which data arrives sequentially , such as online recommender systems [ Nasraoui et al. , 2007 ] and online community detection [ Aggarwal , 2003 ] . In this work , we study k-median clustering in the sequential no-substitution setting , a term first introduced in Hess and Sabato [ 2020 ] . In this setting , a stream of data points is sequentially observed , and some of these points are selected by the algorithm as cluster centers . However , a point can be selected as a center only immediately after it is observed , before observing the next point . In addition , a selected center can not be substituted later . This setting is motivated by applications in which center selection is mapped to a real-world irreversible action . For instance , consider a stream of website users , where the goal is to instantaneously identify users that serve as social cluster centers and provide them with a promotional gift while they are still on the website . As another example , consider recruiting representative participants for a clinical trial out of a stream of incoming patients . ∗This work was done while the author was at the University of California San Diego . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . The goal in the no-substitution k-median setting is to obtain a near-optimal k-median cost value , while selecting a number of centers that is as close as possible to k. For an adversarially ordered stream , it has been shown [ Moshkovitz , 2021 ] that an algorithm that selects a number of centers that is sublinear in the stream length can not obtain a constant approximation guarantee , unless some structural assumptions are imposed on the input data . In this work , we show that in contrast , when the stream order is random , a constant approximation guarantee can be obtained without imposing structural assumptions on the data . Previous works on the random-order setting provide either an algorithm with an almost-constant approximation assuming a bounded metric space [ Hess and Sabato , 2020 ] or an algorithm with an approximation factor that is exponential in k [ Moshkovitz , 2021 ] . Thus , our guarantees are stronger and more general than the previous guarantees . Moreover , the number of centers selected by our algorithm is only quasi-linear in k and does not depend on the stream length . Main result . We propose a new algorithm , MUNSC ( Multiscale No-Substitution Clustering ) , that obtains a constant approximation factor with probability 1− δ over the random order of the stream , while selecting only O ( k log2 ( k/δ ) ) centers . MUNSC operates by executing several instances of a new center-selection procedure , called SelectProc , where each instance is applied to a stream prefix of a different scale . SelectProc decides which centers to select using a novel technique , in which a small stream prefix is used to estimate a truncated version of the optimal cost of the entire stream . This truncated version ignores outliers , and so can be estimated reliably . The value of the estimate is used to determine which centers to select from the rest of the stream . The multiscale use of SelectProc by MUNSC allows MUNSC to select only a quasi-linear number of centers . As part of our analysis , we propose a new concept of a linear bin division , which allows a highprobability association of the costs of stream subsets , while selecting a number of centers that is independent of the stream length . This improves a previous construction of Meyerson et al . [ 2004 ] . The guarantees for MUNSC are provided in our main theorem , Theorem 4.1 , stated in Section 4 . Table 1 below compares our guarantees to previous works ; See Section 2 for additional details . We note that our algorithm and guarantees are easily extendable to k-means clustering , as well as to any other clustering objective that satisfies the weak triangle inequality . MUNSC uses as a black box an offline k-median approximation algorithm with a constant approximation factor . Whenever the offline algorithm is efficient [ e.g. , Charikar et al. , 2002 , Li and Svensson , 2016 ] , then so is MUNSC . Limitations . The proposed algorithm is the first constant-approximation no-substitution clustering algorithm that makes no structural assumptions on the input data set . The only limitation is the assumption that the input order is random . This assumption is useful in cases where there is no adversary and any order is as likely , as in the example applications mentioned above . Moreover , a random order is a standard assumption in many streaming settings , satisfied also whenever the input stream is drawn i.i.d . from a distribution . In some cases , the randomness assumption might hold only approximately . In these cases , we expect a graceful degradation of the guarantees . We leave for future work a comprehensive treatment of input orders that are neither random nor adversarial . We expect the techniques of the current work to serve as a cornerstone in such a treatment . Paper structure . We discuss related work in Section 2 . The formal setting and notations are defined in Section 3 . The algorithm and its guarantees are given in Section 4 . In Section 5 , we outline the proof of the main result . Some parts of the proof are deferred to appendices , which can be found in the supplementary material . We conclude with a discussion in Section 6 . 2 Related Work . Several works have studied settings related to the no-substitution k-median clustering setting . Table 1 summarizes the upper bounds mentioned below . First , some works studied related settings under an adversarial arrival order . Liberty et al . [ 2016 ] studied online k-means clustering , in which centers are sequentially selected , and each observed point is allocated to one of the previously selected centers . The proposed algorithm can be applied to the no-substitution setting , yielding an approximation factor of O ( log ( n ) ) , where n is the stream length . For this algorithm to select a sublinear number of centers , the aspect ratio of the input data must be bounded . Bhaskara and Rwanpathirana [ 2020 ] studied the same setting as Liberty et al . [ 2016 ] , improving the approximation factor to a constant , under the same assumptions . Bhattacharjee and Moshkovitz [ 2021 ] explicitly studied the no-substitution setting , and provided an approximation factor of O ( k3 ) , under a different assumption on the input data set . The setting of a random arrival order has been studied in several recent works . Hess and Sabato [ 2020 ] proposed an algorithm that selects exactly k centers . They obtained a constant approximation factor , with an additional additive term that vanishes for large streams , under the assumption that the metric space has a bounded diameter . Their guarantee is with high probability . Moshkovitz [ 2021 ] obtained an approximation factor that is exponential in k , with a number of centers that is linear in k and logarithmic in the stream length n. Assuming a known stream length , the same work also obtained an approximation factor which is exponential in k while selecting O ( k5 ) centers . Both guarantees hold only with a constant probability . As can be seen in Table 1 , to date , the only known constant approximation algorithms for no-substitution k-median clustering impose structural assumptions on the input data . A related sequential clustering setting is the streaming setting [ e.g. , Guha et al. , 2000 , Ailon et al. , 2009 , Chen , 2009 , Ackermann et al. , 2012 , Braverman et al. , 2016 ] , in which the main restriction is the amount of memory available to the algorithm . This setting allows substituting selected centers , but algorithms in this setting can be used in the no-substitution setting , by collecting all the centers ever selected . However , we are not aware of any algorithm in this setting with a competitive bound on the total number of selected centers . 3 Setting and Notation . For an integer i , denote [ i ] : = { 1 , . . . , i } . Let ( X , ρ ) be a finite metric space , where X is a set of size n and ρ : X × X → R+ is a metric . For a point x ∈ X and a set T ⊆ X , we denote ρ ( x , T ) : = miny∈T ρ ( x , y ) . For an integer k ≥ 2 , a k-clustering of X is a set of ( at most ) k points from X which represent cluster centers . Throughout this work , whenever an item is selected based on minimizing ρ , we assume that ties are broken based on a fixed arbitrary ordering . Given a set S ⊆ X , the k-median cost of T on S is cost ( S , T ) : = ∑ x∈S ρ ( x , T ) . The k-median clustering problem aims to select a k-clustering T of X with a minimal overall cost cost ( X , T ) . We denote by OPT an optimal solution to this problem : OPT ∈ argminT⊆X , |T |≤k cost ( X , T ) . In the sequential no-substitution k-median setting that we study , X is not known a priori . The points from X are presented to the algorithm sequentially , in a random order . We assume that the stream length , n , is provided as input to the algorithm ; see Section 6 for a discussion on supporting an unknown stream length . The algorithm may select an observed point as a center only before observing the next point . Any selected point can not later be removed or substituted . The goal of the algorithm is to select a small number of centers , such that with a high probability , the overall cost of the selected set on the entire X approximates the optimal k-median cost , cost ( X , OPT ) . In other words , we seek an algorithm with a low competitive ratio , in the sense of competitive analysis [ Borodin and El-Yaniv , 2005 ] . An offline k-median algorithm A is an algorithm that takes as input a finite set Algorithm 1 The sequential clustering algorithm : MUNSC input δ ∈ ( 0 , 1 ) , k ∈ N , n ∈ N ( stream length ) , A ( an offline k-median algorithm ) , sequential access to the input stream . 1 : α1 ← δ/ ( 4k ) ; For i ∈ N , αi ← α1 · 2i−1 . 2 : I ← blog2 ( 1/ ( 6α1 ) ) c , δ′ ← δ/ ( I + 1 ) . 4 I is set so that αI+1 ∈ ( 1/12 , 1/6 ] . 3 : Prepare I + 1 copies of SelectProc , indexed by i ∈ [ I + 1 ] , as follows : In all copies , use inputs k , n , A , τ ← φαI+1 , and set the confidence parameter to δ′ . In copy i ∈ [ I + 1 ] , set α← αi . In all but the last copy , set M ← 1 and γ ← 2αi . In the last copy ( index I + 1 ) , set M ← log ( 8k+/δ′ ) , and γ ← 1− 2αI+1 . 4 : Read each point from the input stream and feed it to each of the copies of SelectProc . If any of the copies selected the point , then select it as a center . of points S and the parameter k , and outputs a k-clustering of S , denoted A ( S , k ) . For β ≥ 1 , we say that A is a β-approximation offline k-median algorithm on ( X , ρ ) , if for all input sets S ⊆ X , cost ( S , A ( S , k ) ) ≤ β · cost ( S , OPTS ) , where OPTS is an optimal solution with centers from S. Formally , OPTS ∈ argminT⊆S , |T |≤k cost ( S , T ) .

Summary: The paper studies $k$-median clustering in the random-order streaming model. In this particular setting, the input is randomly permuted and fed to the algorithm element by element. The input itself is a unknown finite set of points that together with a distance function forms a discrete metric space. The goal of the algorithm is to construct a clustering, by selecting a only small number of the input points as cluster centers, such that the cost of the clustering approximates the cost of an optimal $k$-median clustering of the whole space. Imposed restrictions are that the algorithm can only pick an input element as cluster center before it sees the next element and centers that have been chosen can not be substituted.  The paper contributes an algorithm that returns with high probability of success a clustering with $O(k \log^2(k/\delta))$ centers that approximates the cost of an optimal $k$-median clustering by a constant factor.