Title: Constant Approximation for Individual Preference Stable Clustering
Venue: NeurIPS 2023
Accepted: yes
Keywords: clustering

fairness

approximation algorithms
Abstract: Individual preference (IP) stability, introduced by Ahmadi et al. (ICML 2022), is a natural clustering objective inspired by stability and fairness constraints. A clustering is $\alpha$-IP stable if the average distance of every data point to its own cluster is at most $\alpha$ times the average distance to any other cluster. Unfortunately, determining if a dataset admits a $1$-IP stable clustering is NP-Hard. Moreover, before this work, it was unknown if an $o(n)$-IP stable clustering always exists, as the prior state of the art only guaranteed an $O(n)$-IP stable clustering. We close this gap in understanding and show that an $O(1)$-IP stable clustering always exists for general metrics, and we give an efficient algorithm which outputs such a clustering.  We also introduce generalizations of IP stability beyond average distance and give efficient near optimal algorithms in the cases where we consider the maximum and minimum distances within and between clusters.

Constant Approximation for Individual Preference Stable Clustering

Abstract

Individual preference (IP) stability, introduced by Ahmadi et al. (ICML 2022), is a natural clustering objective inspired by stability and fairness constraints. A clustering is α -IP stable if the average distance of every data point to its own cluster is at most α times the average distance to any other cluster. Unfortunately, determining if a dataset admits a 1 -IP stable clustering is NP-Hard. Moreover, before this work, it was unknown if an o ( n ) -IP stable clustering always exists , as the prior state of the art only guaranteed an O ( n ) -IP stable clustering. We close this gap in understanding and show that an O (1) -IP stable clustering always exists for general metrics, and we give an efficient algorithm which outputs such a clustering. We also introduce generalizations of IP stability beyond average distance and give efficient near optimal algorithms in the cases where we consider the maximum and minimum distances within and between clusters.

1 Introduction

In applications involving and affecting people, socioeconomic concepts such as game theory, stability, and fairness are important considerations in algorithm design. Within this context, Ahmadi et al. [1] (ICML 2022) introduced the notion of individual preference stability (IP stability) for clustering. At a high-level, a clustering of an input dataset is called 1 -IP stable if, for each individual point, its average distance to any other cluster is larger than the average distance to its own cluster. Intuitively, each individual prefers its own cluster to any other, and so the clustering is stable.

There are plenty of applications of clustering in which the utility of each individual in any cluster is determined according to the other individuals who belong to the same cluster. For example, in designing personalized medicine , the more similar the individuals in each cluster are, the more effective medical decisions, interventions, and treatments can be made for each group of patients. Similarly, stability guarantees are desired in designing personalized learning environments or marketing campaigns to ensure that no individual wants to deviate from their assigned cluster. Furthermore the focus on individual utility in IP stability (a clustering is only stable if every individual is 'happy') enforces a notion of individual fairness in clustering.

In addition to its natural connections to cluster stability, algorithmic fairness, and Nash equilibria, IP stability is also algorithmically interesting in its own right. While clustering is well-studied with respect to global objective functions (e.g. the objectives of centroid-based clustering such as k -means

Sandeep Silwal MIT

or correlation/hierarchical clustering), less is known when the goal is to partition the dataset such that every point in the dataset is individually satisfied with the solution. Thus, IP stability also serves as a natural and motivated model of individual preferences in clustering.

1.1 Problem Statement and Preliminaries

The main objective of our clustering algorithms is to achieve IP stability given a set P of n points lying in a metric space ( M,d ) and k , the number of clusters.

̸

Definition 1.1 (Individual Preference (IP) Stability [1]) . The goal is to find a disjoint k -clustering C = ( C 1 , · · · , C k ) of P such that every point, on average , is closer to the points of its own cluster than to the points in any other cluster. Formally, for all v ∈ P , let C ( v ) denote the cluster that contains v . We say that v ∈ P is IP stable with respect to C if either C ( v ) = { v } or for every C ′ ∈ C with C ′ = C ,

The clustering C is 1-IP stable (or simply IP stable) if and only if every v ∈ P is stable with respect to C .

Ahmadi et al. [1] showed that an arbitrary dataset may not admit an IP stable clustering. This can be the case even when n = 4 . Furthermore, they proved that it is NP-hard to decide whether a given a set of points have an IP stable k -clustering, even for k = 2 . This naturally motivates the study of the relaxations of IP stability.

̸

Definition 1.2 (Approximate IP Stability) . A k -clustering C = ( C 1 , · · · , C k ) of P is α -approximate IP stable, or simply α -IP stable, if for every point v ∈ P , the following holds: either C ( v ) = { v } or for every C ′ ∈ C and C ′ = C ,

The work of [1] proposed algorithms to outputting IP stable clusterings on the one-dimensional line for any value of k and on tree metrics for k = 2 . The first result implies an O ( n ) -IP stable clustering for general metrics, by applying a standard O ( n ) -distortion embedding to one-dimensional Euclidean space. In addition, they give a bicriteria approximation that discards an ε -fraction of the input points and outputs a O ( log 2 n ε ) -IP stable clustering for the remaining points.

Given the prior results, it is natural to ask if the O ( n ) factor for IP stable clustering given in [1] can be improved.

1.2 Our Results

NewApproximations. Improving on the O ( n ) -IP stable algorithm in [1], we present a deterministic algorithm which for general metrics obtains an O (1) -IP stable k -clustering, for any value of k . Note that given the existence of instances without 1 -IP stable clusterings, our approximation factor is optimal up to a constant factor.

Theorem 1.3. (Informal; see Theorem 3.1) Given a set P of n points in a metric space ( M,d ) and a number of desired clusters k ≤ n , there exists an algorithm that computes an O (1) -IP stable k -clustering of P in polynomial time.

Our algorithm outputs a clustering with an even stronger guarantee that we call uniform (approximate) IP stability. Specifically, for some global parameter r and for every point v ∈ P , the average distance from v to points in its own cluster is upper bounded by O ( r ) and the average distance from v to points in any other cluster is lower bounded by Ω( r ) . Note that the general condition of O (1) -IP stability would allow for a different value of r for each v .

We again emphasize that Theorem 1.3 implies that an O (1) -IP stable clustering always exists, where prior to this work, only the O ( n ) bound from [1] was known for general metrics.

Additional k -center clustering guarantee. The clustering outputted by our algorithm satisfies additional desirable properties beyond O (1) -IP stability. In the k -center problem, we are given n points in a metric space, and our goal is to pick k centers as to minimize the maximal distance of any point to the nearest center. The clustering outputted by our algorithm from Theorem 1.3 has the added benefit of being a constant factor approximation to the k -center problem in the sense that if the optimal k -center solution has value r 0 , then the diameter of each cluster outputted by the algorithm is O ( r 0 ) . In fact, we argue that IP stability is more meaningful when we also seek a solution that optimizes some clustering objective. If we only ask for IP stability, there are instances where it is easy to obtain O (1) -IP stable clusterings, but where such clusterings do not provide insightful information in a typical clustering application. Indeed, as we will show in Appendix B, randomly k -coloring the nodes of an unweighted, undirected graph (where the distance between two nodes is the number of edges on the shortest path between them), gives an O (1) -IP stable clustering when k ≤ O ( √ n log n ) . Our result on trees demonstrates the idiosyncrasies of individual objectives thus our work raises further interesting questions about studying standard global clustering objectives under the restriction that the solutions are also (approximately) IP stable.

̸

Max and Min-IP Stability. Lastly, we introduce a notion of f -IP stability, generalizing IP stability. Definition 1.4 ( f -IP Stability) . Let ( M,d ) be a metric space, P a set of n points of M , and k the desired number of partitions. Let f : P × 2 P → R ≥ 0 be a function which takes in a point v ∈ P , a subset C of P , and outputs a non-negative real number. we say that a k -clustering C = ( C 1 , · · · , C k ) of P is f -IP stable if for every point v ∈ P , the following holds: either C ( v ) = { v } or for every C ′ ∈ C and C ′ = C ,

Note that the standard setting of IP stability given in Definition 1.1 corresponds to the case where f ( v, C ) = (1 / | C | ) × ∑ v ′ ∈ C d ( v, v ′ ) . The formulation of f -IP stability, therefore, extends IP stability beyond average distances and allows for alternative objectives that may be more desirable in certain settings. For instance, in hierarchical clustering, average, minimum, and maximum distance measures are well-studied.

In particular, we focus on max-distance and min-distance in the definition of f -IP stable clustering in addition to average distance (which is just Definition 1.1), where f ( v, C ) = max v ′ ∈ C d ( v, v ′ ) and f ( v, C ) = min v ′ ∈ C d ( v, v ′ ) . We show that in both the max and min distance formulations, we can solve the corresponding f -IP stable clustering (nearly) optimally in polynomial time. We provide the following result:

Theorem 1.5 (Informal; see Theorem 4.1 and Theorem 4.2) . In any metric space, Min-IP stable clustering can be solved optimally and Max-IP stable clustering can be solved approximately within a factor of 3 , in polynomial time.

We show that the standard greedy algorithm of k -center, a.k.a, the Gonzalez's algorithm [15], yields a 3 -approximate Max-IP stable clustering. Moreover, we present a conceptually clean algorithm which is motivated by considering the minimum spanning tree (MST) to output a Min-IP stable clustering. This implies that unlike the average distance formulation of IP stable clustering, a Min-IP stable clustering always exists. Both algorithms work in general metrics.

Empirical Evaluations. We experimentally evaluate our O (1) -IP stable clustering algorithm against k -means++, which is the empirically best-known algorithm in [1]. We also compare k -means++ with our optimal algorithm for Min-IP stability. We run experiments on the Adult data set 2

used by [1]. For IP stability, we also use four more datasets from UCI ML repositoriy [11] and a synthetic data set designed to be a hard instance for k -means++. On the Adult data set, our algorithm performs slightly worse than k -means++ for IP stability. This is consistent with the empirical results of [1]. On the hard instance 3 , our algorithm performs better than k -means++, demonstrating that the algorithm proposed in this paper is more robust than k -means++. Furthermore for Min-IP stability, we empirically demonstrate that k -means++ can have an approximation factors which are up to a factor of 5x worse than our algorithm. We refer to Section 5 and Appendix C for more details.

1.3 Technical Overview

The main contribution is our O (1) -approximation algorithm for IP stable clustering for general metrics. We discuss the proof technique used to obtain this result. Our algorithm comprises two steps. We first show that for any radius r , we can find a clustering C = ( C 1 , . . . , C t ) such that (a) each cluster has diameter O ( r ) , and (b) the average distance from a point in a cluster to the points of any other cluster is Ω( r ) .

Conditions (a) and (b) are achieved through a ball carving technique, where we iteratively pick centers q i of distance > 6 r to previous centers such that the radius r ball B ( q i , r ) centered at q i contains a maximal number of points, say s i . For each of these balls, we initialize a cluster D i containing the s i points of B ( q i , r ) . We next consider the annulus B ( q i , 3 r ) \ B ( q i , 2 r ) . If this annulus contains less than s i points, we include all points from B ( q i , 3 r ) in D i . Otherwise, we include any s i points in D i from the annulus. We assign each unassigned point to the first center picked by our algorithm and is within distance O ( r ) to the point. This is a subtle but crucial component of the algorithm as the more natural 'assign to the closest center' approach fails to obtain O (1) -IP stability.

One issue remains. With this approach, we have no guarantee on the number of clusters. We solve this by merging some of these clusters while still maintaining that the final clusters have radius O ( r ) . This may not be possible for any choice of r . Thus the second step is to find the right choice of r . We first run the greedy algorithm of k -center and let r 0 be the minimal distance between centers we can run the ball carving algorithm r = cr 0 for a sufficiently small constant c . Then if we assign each cluster of C to its nearest k -center, we do indeed maintain the property that all clusters have diameter O ( r ) , and since c is a small enough constant, all the clusters will be non-empty. The final number of clusters will therefore be k . As an added benefit of using the greedy algorithm for k -center as a subroutine, we obtain that the diameter of each cluster is also O ( r 0 ) , namely the output clustering is a constant factor approximation to k -center.

1.4 Related Work

Fair Clustering. One of the main motivations of IP stable clustering is its interpretation as a notion of individual fairness for clustering [1]. Individual fairness was first introduced by [12] for the classification task, where, at high-level, the authors aim for a classifier that gives 'similar predictions' for 'similar' data points. Recently, other formulations of individual fairness have been studied for clustering [17, 2, 7, 8], too. [17] proposed a notion of fairness for centroid-based clustering: given a set of n points P and the number of clusters k , for each point, a center must be picked among its ( n/k ) -th closest neighbors. The optimization variant of it was later studied by [19, 20, 24].[7] studied a pairwise notion of fairness in which data points represent people who gain some benefit from being clustered together. In a subsequent work, [6] introduced a stochastic variant of this notion. [2] studied the setting in which the output is a distribution over centers and 'similar' points are required to have 'similar' centers distributions.

Stability in Clustering. Designing efficient clustering algorithms under notions of stability is a well-studied problem 4 . Among the various notion of stability, average stability is the most relevant to our model [4]. In particular, they showed that if there is a ground-truth clustering satisfying the requirement of Equation (1) with an additive gap of γ > 0 , then it is possible to recover the solution in the list model where the list size is exponential in 1 /γ . Similar types of guarantees are shown in the work by [9]. While this line of research mainly focuses on presenting faster algorithms utilizing the

strong stability conditions, the focus of IP stable clustering is whether we can recover such stability properties in general instances, either exactly or approximately.

Hedonic Games. Another game-theoretic study of clustering is hedonic games [10, 5, 13]. In a hedonic game, players choose to form coalitions (i.e., clusters) based on their utility. Our work differs from theirs, since we do not model the data points as selfish players. In a related work, [23] proposes another utility measure for hedonic clustering games on graphs. In particular, they define a closeness utility, where the utility of node i in cluster C is the ratio between the number of nodes in C adjacent to i and the sum of distances from i to other nodes in C . This measure is incomparable to IP stability. In addition, their work focuses only on clustering in graphs while we consider general metrics.

2 Preliminaries and Notations

We let ( M,d ) denote a metric space, where d is the underlying distance function. We let P denote a fixed set of points of M . Here P may contain multiple copies of the same point. For a given point x ∈ P and radius r ≥ 0 , we denote by B ( x, r ) = { y ∈ P | d ( x, y ) ≤ r } , the ball of radius r centered at x . For two subsets X,Y ⊆ P , we denote by d ( X,Y ) = inf x ∈ X,y ∈ Y d ( x, y ) . Throughout the paper, X and Y will always be finite and then the infimum can be replaced by a minimum. For x ∈ P and Y ⊆ P , we simply write d ( x, Y ) for d ( { x } , Y ) . Finally, for X ⊆ P , we denote by diam( X ) = sup x,y ∈ X d ( x, y ) , the diameter of the set X . Again, X will always be finite, so the supremum can be replaced by a maximum.

3 Constant-Factor IP Stable Clustering

In this section, we prove our main result: For a set P = { x 1 , . . . , x n } of n points with a metric d and every k ≤ n , there exists a k -clustering C = ( C 1 , . . . , C k ) of P which is O (1) -approximate IP stable. Moreover, such a clustering can be found in time ˜ O ( n 2 T ) , where T is an upper bound on the time it takes to compute the distance between two points of P .

Algorithm Our algorithm uses a subroutine, Algorithm 1, which takes as input P and a radius r ∈ R and returns a t -clustering D = ( D 1 , . . . , D t ) of P with the properties that (1) for any 1 ≤ i ≤ t , the maximum distance between any two points of D i is O ( r ) , and (2) for any x ∈ P and any i such that x / ∈ D i , the average distance from x to points of D i is Ω( r ) . These two properties ensure that D is O (1) -approximate IP stable. However, we have no control on the number of clusters t that the algorithm produces. To remedy this, we first run a greedy k -center algorithm on P to obtain a set of centers { c 1 , . . . , c k } and let r 0 denote the maximum distance from a point of P to the nearest center. We then run Algorithm 1 with input radius r = cr 0 for some small constant c . This gives a clustering D = ( D 1 , . . . , D t ) where t ≥ k . Moreover, we show that if we assign each cluster of D to the nearest center in { c 1 , . . . , c k } (in terms of the minimum distance from a point of the cluster to the center), we obtain a k -clustering C = ( C 1 , . . . , C k ) which is O (1) -approximate IP stable. The combined algorithm is Algorithm 2.

We now describe the details of Algorithm 1. The algorithm takes as input n points x 1 , . . . , x n of a metric space ( M,d ) and a radius r . It first initializes a set Q = ∅ and then iteratively adds points x from P to Q that are of distance greater than 6 r from points already in Q such that | B ( x, r ) | , the number of points of P within radius r of x , is maximized. This is line 5-6 of the algorithm. Whenever a point q i is added to Q , we define the annulus A i := B ( q i , 3 r ) \ B ( q i , 2 r ) . We further let s i = | B ( q i , r ) | . At this point the algorithm splits into two cases. If | A i | ≥ s i , we initialize a cluster D i which consists of the s i points in B ( x, r ) and any arbitrarily chosen s i points in A i . This is line 8-9 of the algorithm. If on the other hand | A i | < s , we define D i := B ( q i , 3 r ) , namely D i contains all points of P within distance 3 r from q i . This is line 10 of the algorithm. After iteratively picking the points q i and initializing the clusters D i , we assign the remaining points as follows. For any point x ∈ P \ ⋃ i D i , we find the minimum i such that d ( x, q i ) ≤ 7 r and assign x to D i . This is line 13-16 of the algorithm. We finally return the clustering D = ( D 1 , . . . , D t ) .

We next describe the details of Algorithm 2. The algorithm iteratively pick k centers c 1 , . . . , c k from P for each center maximizing the minimum distance to previously chosen centers. For each center c i , it initializes a cluster, starting with C i = { c i } . This is line 4-7 of the algorithm. Letting r 0 be

Algorithm 1 BALL-CARVING 1: Input : A set P = { x 1 , . . . , x n } of n points with a metric d and a radius r > 0 . 2: Output : Clustering D = ( D 1 , . . . , D t ) of P . 3: Q ←∅ , i ← 1 4: while there exists x ∈ P with d ( x, Q ) > 6 r do 5: q i ← arg max x ∈ P : d ( x,Q ) > 6 r | B ( x, r ) | 6: Q ← Q ∪ { q i } , s i ←| B ( q i , r ) | , A i ← B ( q i , 3 r ) \ B ( q i , 2 r ) 7: if | A i | ≥ s i 8: S i ← any set of s i points from A i 9: D i ← B ( q i , r ) ∪ S i 10: else D i ← B ( q i , 3 r i ) 11: i ← i +1 12: end while 13: for x ∈ P assigned to no D i do 14: j ← min { i | d ( x, q i ) ≤ 7 r } 15: D j ← D j ∪ { x } 16: end for 17: t ←| Q | 18: return D = ( D 1 , . . . , D t ) Algorithm 2 IP-CLUSTERING 1: Input : Set P = { x 1 , . . . , x n } of n points with a metric d and integer k with 2 ≤ k ≤ n . 2: Output : k -clustering C = ( C 1 , . . . , C k ) of P . 3: S ←∅ 4: for i = 1 , . . . , k do 5: c i ← arg max x ∈ P { d ( x, S ) } 6: S ← S ∪ { c i } , C i ←{ c i } 7: end for 8: r 0 ← min { d ( c i , c j ) | 1 ≤ i < j ≤ k } 9: D ← BALL-CARVING ( P, r 0 / 15) 10: for D ∈ D do 11: j ← arg min i { d ( c i , D ) } 12: C j ← C j ∪ D 13: end for 14: return C = ( C 1 , . . . , C k )

the minimum distance between pairs of distinct centers, the algorithm runs Algorithm 1 on P with input radius r = r 0 / 15 (line 8-9). This produces a clustering D . In the final step, we iterate over the clusters D of D , assigning D to the C i for which d ( c i , D ) is minimized (line 11-13). We finally return the clustering ( C 1 , . . . , C k ) .

Analysis We now analyze our algorithm and provide its main guarantees.

Theorem 3.1. Algorithm 2 returns an O (1) -approximate IP stable k clustering in time O ( n 2 T + n 2 log n ) . Furthermore, the solution is also a constant factor approximation to the k -center problem.

In order to prove this theorem, we require the following lemma on Algorithm 1.

̸

Lemma 3.2. Let ( D 1 , . . . , D t ) be the clustering output by Algorithm 1. For each i ∈ [ t ] , the diameter of D i is at most 14 r . Further, for x ∈ D i and j = i , the average distance from x to points of D j is at least r 4 .

Given Lemma 3.2, we can prove the the main result.

Proof of Theorem 3.1. We first argue correctness. As each c i was chosen to maximize the minimal distance to points c j already in S , for any x ∈ P , it holds that min { d ( x, c i ) | i ∈ [ k ] } ≤ r 0 . By Lemma 3.2, in the clustering D output by BALL-CARVING ( P, r 0 / 15) each cluster has diameter at most 14 15 r 0 < r 0 , and thus, for each i ∈ [ k ] , the cluster D ∈ D which contains c i will be included in

̸

̸

C i in the final clustering. Indeed, in line 11 of Algorithm 2, d ( c i , D ) = 0 whereas d ( c j , D ) ≥ 1 15 r 0 for all j = i . Thus, each cluster in ( C 1 , . . . , C k ) is non-empty. Secondly, the diameter of each cluster is at most 4 r 0 , namely, for each two points x, x ′ ∈ C i , they are both within distance r 0 + 14 15 r 0 < 2 r 0 of c i . Finally, by Lemma 3.2, for x ∈ D i and j = i , the average distance from x to points of D j is at least r 0 60 . Since, C is a coarsening of D , i.e., each cluster of C is the disjoint union of some of the clusters in D , it is straightforward to check that the same property holds for the clustering C . Thus C is O (1) -approximate IP stable.

We now analyze the running time. We claim that Algorithm 2 can be implemented to run in O ( n 2 T + n 2 log n ) time, where T is the time to compute the distance between any two points in the metric space. First, we can query all pairs to form the n × n distance matrix A . Then we sort A along every row to form the matrix A ′ . Given A and A ′ , we easily implement our algorithms as follows.

First, we argue about the greedy k -center steps of Algorithm 2 , namely, the for loop on line 4. The most straightforward implementation computes the distance from every point to new chosen centers. At the end, we have computed at most nk distances from points to centers which can be looked up in A in time O ( nk ) = O ( n 2 ) as k ≤ n . In line 8, we only look at every entry of A at most once so the total time is also O ( n 2 ) . The same reasoning also holds for the for loop on line 10. It remains to analyze the runtime.

Given r , Alg. 1 can be implemented as follows. First, we calculate the size of | B ( x, r ) | for every point x in our dataset. This can easily be done by binary searching on the value of r along each of the (sorted) rows of A ′ , which takes O ( n log n ) time in total. We can similarly calculate the sizes of | B ( x, 2 r ) | and | B ( x, 3 r ) | , and thus the number of points in the annulus | B ( x, 3 r ) \ B ( x, 2 r ) | in the same time to initialize the clusters D i . Similar to the k -center reasoning above, we can also pick the centers in Algorithm 1 which are > 6 r apart iteratively by just calculating the distances from points to the chosen centers so far. This costs at most O ( n 2 ) time, since there are at most n centers. After initializing the clusters D i , we finally need to assign the remaining unassigned points (line 13-16). This can easily be done in time O ( n ) per point, namely for each unassigned point x , we calculate its distance to each q i assigning it to D i where i is minimal such that d ( x, q i ) ≤ 7 r . The total time for this is then O ( n 2 ) . The k -center guarantees follow from our choice of r 0 and Lemma 3.2.

Remark 3.3 . We note that the runtime can possibly be improved if we assume special structure about the metric space (e.g., Euclidean metric). See Appendix A for a discussion.

̸

Proof of Lemma 3.2. The upper bound on the diameter of each cluster follows from the fact that for any cluster D i in the final clustering D = { D 1 , . . . , D t } , and any x ∈ D i , it holds that d ( x, q i ) ≤ 7 r . The main challenge is to prove the lower bound on the average distance from x ∈ D i to D j where j = i .

̸

Suppose for contradiction that, there exists i, j with i = j and x ∈ D i such that the average distance from x to D j is smaller than r/ 4 , i.e., 1 | D j | ∑ y ∈ D j d ( x, y ) < r/ 4 . Then, it in particular holds that | B ( x, r/ 2) ∩ D j | > | D j | / 2 , namely the ball of radius r/ 2 centered at x contains more than half the points of D j . We split the analysis into two cases corresponding to the if-else statements in line 7-10 of the algorithm.

Case 1: | A j | ≥ s j : In this case, cluster D j consists of at least 2 s j points, namely the s j points in B ( q j , r ) and the set S j of s j points in A j assigned to D j in line 8-9 of the algorithm. It follows from the preceding paragraph that, | B ( x, r/ 2) ∩ D j | > s j . Now, when q j was added to Q , it was chosen as to maximize the number of points in B ( q j , r ) under the constraint that q j had distance greater than 6 r to previously chosen points of Q . Since | B ( x, r ) | > | B ( x, r/ 2) | > | B ( q j , r ) | , at the point where q j was chosen, Q already contained some point q j 0 (with j 0 < j ) of distance at most 6 r to x and thus of distance at most 7 r to any point of B ( x, r/ 2) . It follows that B ( x, r/ 2) ∩ D j contains no point assigned during line 13- 16 of the algorithm. Indeed, by the assignment rule, such a point y would have been assigned to either D j 0 or potentially an even earlier initialized cluster of distance at most 7 r to y . Thus, B ( x, r/ 2) ∩ D j is contained in the set B ( q j , r ) ∪ S j . However, | B ( q j , r ) | = | S j | = s j and moreover, for ( y 1 , y 2 ) ∈ B ( q j , r ) × S j , it holds that d ( y 1 , y 2 ) > r . In particular, no ball of radius r/ 2 can contain more than s j points of B ( q j , r ) ∪ S j . As | B ( x, r/ 2) ∩ D j | > s j , this is a contradiction.

Case 2: | A j | < s j : In this case, D j includes all points in B ( q j , 3 r ) . As x / ∈ D j , we must have that x / ∈ B ( q j , 3 r ) and in particular, the ball B ( x, r/ 2) does not intersect B ( q j , r ) . Thus,

so | D j | > 2 s j , and finally, | B ( x, r/ 2) ∩ D j | > | D j | / 2 > s j . Similarly to case 1, B ( x, r/ 2) ∩ D j contains no points assigned during line 13- 16 of the algorithm. Moreover, B ( x, r/ 2) ∩ B ( q j , 3 r ) ⊆ A j . In particular, B ( x, r/ 2) ∩ D j ⊆ S j , a contradiction as | S j | = s j but | B ( x, r/ 2) ∩ D j | > s j .

4 Min and Max-IP Stable Clustering

The Min-IP stable clustering aims to ensure that for any point x , the minimum distance to a point in the cluster of x is at most the minimum distance to a point in any other cluster. We show that a Min-IP stable k -clustering always exists for any value of k ∈ [ n ] and moreover, can be found by a simple algorithm (Algorithm 3).

Algorithm 3 MIN-IP-CLUSTERING

1: Input : Pointset P = { x , . . . , x } from a metric space ( M,d ) and integer k with 2 ≤ k ≤ n .

1 n 2: Output : k -clustering C = ( C 1 , . . . , C k ) of P . 3: L ←{ ( x i , x j ) } 1 ≤ i<j ≤ n sorted according to d ( x i , x j ) 4: E ←∅ 5: while G = ( P, E ) has > k connected components do 6: e ← an edge e = ( x, y ) in L with d ( x, y ) minimal. 7: L ← L \ { e } 8: if e connects different connected components of G then E ← E ∪ { e } 9: end while 10: return the connected components ( C 1 , . . . , C k ) of G .

The algorithm is identical to Kruskal's algorithm for finding a minimum spanning tree except that it stops as soon as it has constructed a forest with k connected components. First, it initializes a graph G = ( V, E ) with V = P and E = ∅ . Next, it computes all distances d ( x i , x j ) between pairs of points ( x i , x j ) of P and sorts the pairs ( x i , x j ) according to these distances. Finally, it goes through this sorted list adding each edge ( x i , x j ) to E if it connects different connected components of G . After computing the distances, it is well known that this algorithm can be made to run in time O ( n 2 log n ) , so the total running time is O ( n 2 ( T +log n )) where T is the time to compute the distance between a single pair of points.

̸

Proof. Let C be the clustering output by the algorithm. Conditions (1) and (2) in the definition of a min-stable clustering are trivially satisfied. To prove that (3) holds, let C ∈ C with | C | ≥ 2 and x ∈ C . Let y 0 = x be a point in C such that ( x, y 0 ) ∈ E (such an edge exists because C is the connected component of G containing x ) and let y 1 be the closest point to x in P \ C . When the algorithm added ( x, y 0 ) to E , ( x, y 1 ) was also a candidate choice of an edge between connected components of G . Since the algorithm chose the edge of minimal length with this property, d ( x, y 0 ) ≤ d ( x, y 1 ) . Thus, we get the desired bound:

Theorem 4.2. The solution output by the greedy algorithm of k -center is a 3 -approximate Max-IP stable clustering.

̸

Proof. To recall, the greedy algorithm of k -center (aka Gonzalez algorithm [15]) starts with an arbitrary point as the first center and then goes through k -1 iterations. In each iteration, it picks a new point as a center which is furthest from all previously picked centers. Let c 1 , · · · , c k denote the selected centers and let r := max v ∈ P d ( v, { c 1 , · · · , c k } ) . Then, each point is assigned to the cluster of its closest center. We denote the constructed clusters as C 1 , · · · , C k . Now, for every i = j ∈ [ k ] and each point v ∈ C i , we consider two cases:

d ( v, c i ) ≤ r/ 2 .

d ( v, c i ) > r/ 2 .

In both cases, max u i ∈ C i d ( v, u i ) ≤ 3 max u j ∈ C j d ( v, u j )

5 Experiments

While the goal and the main contributions of our paper are mainly theoretical, we also implement our optimal Min-IP clustering algorithm as well as extend the experimental results for IP stable clustering given in [1]. Our experiments demonstrate that our optimal Min-IP stable clustering algorithm is superior to k -means++, the strongest baseline in [1], and show that our IP clustering algorithm for average distances is practical on real world datasets and is competitive to k -means++ (which fails to find good stable clusterings in the worst case [1]). We give our experimental results for Min-IP stability and defer the rest of the empirical evaluations to Section C. All experiments were performed in Python 3. The results shown below are an average of 10 runs for k -means++.

̸

Metrics We measure the quality of a clustering using the same metrics used in [1] for standardization. Considering the question of f -IP stability (Definition 1.4), let the violation of a point x be defined as Vi ( x ) = max C i = C ( x ) f ( x,C ( x ) \{ x } ) .

For example, setting f ( x, C ) = ∑ y ∈ C d ( x, y ) / | C | corresponds to the standard IP stability objective and f ( x, C ) = min y ∈ C d ( x, y ) is the Min-IP formulation. Note point x is stable iff Vi ( x ) ≤ 1 .

We measure the extent to which a k -clustering C = ( C 1 , . . . , C k ) of P is (un-)stable by computing MaxVi = max x ∈ P Vi ( x ) (i.e. maximum violation) and MeanVi = ∑ x ∈ P Vi ( x ) / | P | (mean violation).

Results For Min-IP stability, we have an optimal algorithm; it always return a stable clustering for all k . We see in Figures 1 that for the max and mean violation metrics, our algorithm outperforms k -means++ by up to a factor of 5x , consistently across various values of k . k -means ++ can return a much worse clustering under Min-IP stability on real data, motivating the use of our theoreticallyoptimal algorithm in practice.

20

40

60

80

100

k

0

1

2

3

4

5

6

MaxVi

Adult data set

Our Alg

k-means++

20

40

60

80

100

k

0.1

0.2

0.3

0.4

0.5

MeanVi

Adult data set

Our Alg

k-means++

6 Conclusion

We presented a deterministic polynomial time algorithm which provides an O (1) -approximate IP stable clustering of n points in a general metric space, improving on prior works which only guaranteed an O ( n ) -approximate IP stable clustering. We also generalized IP stability to f -stability and provided an algorithm which finds an exact Min-IP stable clustering and a 3-approximation for Max-IP stability, both of which hold for all k and in general metric spaces.

References

Saba Ahmadi, Pranjal Awasthi, Samir Khuller, Matthäus Kleindessner, Jamie Morgenstern, Pattara Sukprasert, and Ali Vakilian. Individual preference stability for clustering. In International Conference on Machine Learning (ICML) , 2022.

Nihesh Anderson, Suman K Bera, Syamantak Das, and Yang Liu. Distributional individual fairness in clustering. arXiv preprint arXiv:2006.12589 , 2020.

P. Awasthi and M.-F. Balcan. Center based clustering: A foundational perspective. In Handbook of Cluster Analysis . CRC Press, 2014.

Maria-Florina Balcan, Avrim Blum, and Santosh Vempala. A discriminative framework for clustering via similarity functions. In Symposium on Theory of computing (STOC) , pages 671-680, 2008.

Anna Bogomolnaia and Matthew O Jackson. The stability of hedonic coalition structures. Games and Economic Behavior , 38(2):201-230, 2002.

B Brubach, D Chakrabarti, J Dickerson, A Srinivasan, and L Tsepenekas. Fairness, semisupervised learning, and more: A general framework for clustering with stochastic pairwise constraints. In AAAI Conference on Artificial Intelligence (AAAI) , 2021.

Brian Brubach, Darshan Chakrabarti, John Dickerson, Samir Khuller, Aravind Srinivasan, and Leonidas Tsepenekas. A pairwise fair and community-preserving approach to k-center clustering. In International Conference on Machine Learning (ICML) , pages 1178-1189, 2020.

Darshan Chakrabarti, John P Dickerson, Seyed A Esmaeili, Aravind Srinivasan, and Leonidas Tsepenekas. A new notion of individually fair clustering: α -equitable k -center. In International Conference on Artificial Intelligence and Statistics , pages 6387-6408. PMLR, 2022.

Amit Daniely, Nati Linial, and Michael Saks. Clustering is difficult only when it does not matter. arXiv preprint arXiv:1205.4891 , 2012.

Jacques H Dreze and Joseph Greenberg. Hedonic coalitions: Optimality and stability. Econometrica: Journal of the Econometric Society , pages 987-1003, 1980.

D. Dua and C. Graff. UCI machine learning repository, 2019. Adult data set available on https://archive.ics.uci.edu/ml/datasets/adult . Breast Cancer Wisconsin data set available on https://archive.ics.uci.edu/ml/datasets/breast+cancer+ wisconsin+(diagnostic) . Car Evaluation data set available on https://archive. ics.uci.edu/ml/datasets/Car+Evaluation . Drug Consumption data set available on https://archive.ics.uci.edu/ml/datasets/Drug+consumption+(quantified) . Indian Liver Patient data set available on https://archive.ics.uci.edu/ml/datasets/ ILPD+(Indian+Liver+Patient+Dataset) .

Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Innovations in Theoretical Computer Science (ITCS) , pages 214-226, 2012.

Edith Elkind, Angelo Fanelli, and Michele Flammini. Price of pareto optimality in hedonic games. Artificial Intelligence , 288:103357, 2020.

E. Fehrman, A. K. Muhammad, E. M. Mirkes, V. Egan, and A. N. Gorban. The five factor model of personality and evaluation of drug consumption risk, 2017.

Teofilo F Gonzalez. Clustering to minimize the maximum intercluster distance. Theoretical computer science , 38:293-306, 1985.

Piotr Indyk and Sandeep Silwal. Faster linear algebra for distance matrices. In Neural Information Processing Systems (NeurIPS) , 2022.

C. Jung, S. Kannan, and N. Lutz. A center in your neighborhood: Fairness in facility location. In Symposium on Foundations of Responsible Computing (FORC) , 2020.

Ron Kohavi et al. Scaling up the accuracy of naive-bayes classifiers: A decision-tree hybrid. In Kdd , volume 96, pages 202-207, 1996.

S. Mahabadi and A. Vakilian. Individual fairness for k -clustering. In International Conference on Machine Learning (ICML) , 2020.

Maryam Negahbani and Deeparnab Chakrabarty. Better algorithms for individually fair k -clustering. Advances in Neural Information Processing Systems (NeurIPS) , 34:13340-13351, 2021.

F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research , 12:2825-2830, 2011.

Jack Snoeyink. Point location. In Jacob E. Goodman and Joseph O'Rourke, editors, Handbook of Discrete and Computational Geometry, Second Edition , pages 767-785. Chapman and Hall/CRC, 2004.

Ana-Andreea Stoica and Christos Papadimitriou. Strategic clustering. 2018.

A. Vakilian and M. Yalciner. Improved approximation algorithms for individually fair clustering. In International Conference on Artificial Intelligence and Statistics (AISTATS) , 2022.

A Discussion on the Runtime of Algorithm 2

We remark that the runtime of our O (1) -approximate IP-stable clustering algorithm can potentially be improved if we assume special structure about the metric space, such as a tree or Euclidean metric. In special cases, we can improve the running time by appealing to particular properties of the metric which allow us to either calculate distances or implement our subroutines faster. For example for tree metrics, all distances can be calculated in O ( n 2 ) time, even though T = O ( n ) . Likewise for the Euclidean case, we can utilize specialized algorithms for computing the all pairs distance matrix, which obtain speedups over the naive methods [16], or use geometric point location data structures to quickly compute quantities such as | B ( x, r ) | [22]. Our presentation is optimized for simplicity and generality so detailed discussions of specific metric spaces are beyond the scope of the work.

B Random Clustering in Unweighted Graphs

In this appendix, we show that for unweigthed, undirected, graphs (where the distance d ( u, v ) between two vertices u and v is the length of the shortest path between them), randomly k -coloring the nodes gives an O (1) -approximate IP-stable clustering whenever k = O ( n 1 / 2 / log n ) .

We start with the following lemma.

Lemma B.1. Let γ = O (1) be a constant. There exists a constant c > 0 (depending on γ ) such that the following holds: Let T = ( V, E ) be an unweighted tree on n nodes rooted at vertex r . Suppose that we randomly k -color the nodes of T . Let V i ⊆ V be the nodes of color i , let X i = ∑ v ∈ V i d ( r, v ) , and let X = ∑ v ∈ V d ( r, v ) . If k ≤ c √ n log n , then with probability 1 -O ( n -γ ) , it holds that X/ 2 ≤ kX i ≤ 2 X for all i ∈ [ k ] .

Proof. Wewill fix i , and prove that the bound X/ 2 ≤ X i ≤ 2 X holds with probability 1 -O ( n -γ -1 ) . Union bounding over all i then gives the desired result. Let ∆ = max v ∈ V d ( r, v ) be the maximum distance from the root to any vertex of the tree. We may assume that ∆ ≥ 5 as otherwise the result follows directly from a simple Chernoff bound. Since the tree is unweighted and there exists a node v of distance ∆ to r , there must also exist nodes of distances 1 , 2 , . . . , ∆ -1 to r , namely the nodes on the path from r to v . For the remaining nodes, we know that the distance is at least 1 . Therefore,

and so µ i = E [ X i ] ≥ n +∆ 2 / 3 k . Since the variables ( d ( r, v )[ v ∈ V i ]) v ∈ V sum to X i and are independent and bounded by ∆ , it follows by a Chernoff bound that for any 0 ≤ δ ≤ 1 ,

By the AM-GM inequality,

Putting δ = 1 / 2 , the bound above thus becomes

where the last bound uses the assumption on the magnitude of k in the lemma. Choosing c = 1 11( γ +1) , the desired result follows.

Next, we state our result on the O (1) -approximate IP-stability for randomly colored graphs.

Theorem B.2. Let γ = O (1) and k ≤ c √ n log n for a sufficiently small constant c . Let G = ( V, E ) be an unweighted, undirected graph on n nodes, and suppose that we k -color the vertices of G randomly. Let V i denote the nodes of color i . With probability at least 1 -n -γ , ( V 1 , . . . , V k ) forms an O (1) -approximate IP-clustering.

Proof. Consider a node u and let X i = ∑ v ∈ V i \{ u } d ( u, v ) . Node that the distances d ( u, v ) are exactly the distances in a breath first search tree rooted at v . Thus, by Lemma B.1, the X i 's are all within a constant factor of each other with probability 1 -O ( n -γ -1 ) . Moreover, a simple Chernoff bound shows that with the same high probability, | V i | = n k + O (√ n log n k ) = Θ ( n k ) for all i ∈ [ k ] .

In particular, the values Y i = X i | V i \{ u }| for i ∈ [ k ] also all lie within a constant factor of each other which implies that u is O (1) -stable in the clustering ( V 1 , . . . , V k ) . Union bounding over all nodes u , we find that with probability 1 -O ( n -γ ) , ( V 1 , . . . , V k ) is an O (1) -approximate IP-clustering.

Remark B.3 . The assumed upper bound on k in Theorem B.2 is necessary (even in terms of log n ). Indeed, consider a tree T which is a star S on n -k log k vertices along with a path P of length k log k having one endpoint at the center v of the star. With probability Ω(1) , some color does not appear on P . We refer to this color as color 1 . Now consider the color of the star center. With probability at least 9 / 10 , say, this color is different from 1 and appears Ω(log k ) times on P with average distance Ω( k log k ) to the star center v . Let the star center have color 2 . With high probability, each color appears Θ( n/k ) times in S . Combining these bounds, we find that with constant probability, the average distance from v to vertices of color 1 is O (1) , whereas the average distance from v to vertices of color 2 is Ω ( 1 + k 2 (log k ) 2 n ) . In particular for the algorithm to give an O (1) -approximate IP-stable clustering, we need to assume that k = O ( √ n log n ) .

C Additional Empirical Evaluations

We implement our O (1) -approximation algorithm for IP-clustering. These experiments extend those of [1] and confirm their experimental findings: k -means++ is a strong baseline for IP-stable clustering. Nevertheless, our algorithm is competitive with it while guaranteeing robustness against worst-case datasets, a property which k -means++ does not posses.

Our datasets are the following. There are three datasets from [11] used in [1], namely, Adult , Drug [14], and IndianLiver . We also add two additional datasets from UCI Machine Learning Repository [11], namely, BreastCancer and Car . For IP-clustering, we also consider a synthetic dataset which is the hard instance for k -means++ given in [1].

Our goal is to show that our IP-clustering algorithm is practical and in real world datasets, is competitive with respect to k -means++, which was the best algorithm in the experiments in [1]. Furthermore, our algorithm is robust and outperform k -means++ for worst case datasets.

As before, all experiments were performed in Python 3. We use the k -means++ implementation of Scikit-learn package [21]. We note that in the default implementation in Scikit-learn, k -means++ is initiated many times with different centroid seeds. The output is the best of 10 runs by default. As we want to have control of this behavior, we set the parameter n_init=1 and then compute the average of many different runs.

̸

Additionally to the metrics used in the main experimental section, we also compute the number of unstable points, defined as the size of the set U = { x ∈ M : x is not stable } . In terms of clustering qualities, we additionally measure three quantities. First, we measure 'cost', which is the average within-cluster distances. Formally, Cost = ∑ k i =1 1 ( | Ci | 2 ) ∑ x,y ∈ C i ,x = y d ( x, y ) . We

̸

then measure k -center costs, defined as the maximum distances from any point to its center. Here, centers are given naturally from k-means++ and our algorithm. Finally, k -means costs, defined as k-means-cost = ∑ k i =1 1 | C i | ∑ x,y ∈ C i ,x = y d ( x, y ) 2 .

C.1 Hard Instance for k -means++ for IP-Stability

We briefly describe the hard instance for k -means++ for the standard IP-stability formulation given in [1]; see their paper for full details. The hard instance consists of a gadget of size 4 . In the seedfinding phase of k -means++, if it incorrectly picks two centers in the gadget, then the final clustering is not β -approximate IP-stable, where β is a configurable parameter. The instance for k -clustering is produced by concatenating these gadgets together. In such an instance, with a constant probability, the clustering returned by k -means++ is not β -approximate IP-stable and in particular. We remark that the proof of Theorem 2 in [1] easily implies that k -means++ cannot have an approximation factor better than n c for some absolute constant c > 0 , i.e., we can insure β = Ω( n c ) . Here, we test both our algorithm and k -means++ in an instance with 8,000 points (for k = 2 , 000 clusters).

IP-Stability results We first discuss five real dataset. We tested the algorithms for the range of k up to 25 . The result in Figures 2 and 3 is consistent with the experiments in the previous paper as we see that k -means++ is a very competitive algorithm for these datasets. For small number of clusters, our algorithm sometimes outperforms k -means++. We hypothesize that on these datasets, especially for large k , clusters which have low k -means cost separate the points well and therefore are good clusters for IP-stability.

Next we discuss the k -means++ hard instance. The instance used in Figure 3 was constructed with β = 50 . We vary k but omit the results for higher k values since the outputs from both algorithms are stable. We remark that the empirical results with different β gave qualitatively similar results. For maximum and mean violation, our algorithm outperforms k -means++ (Figure 3).

D Future Directions

There are multiple natural open questions following our work.

Note that in some cases, α -IP stable clusterings for α < 1 may exist. On the other hand, in the hard example on n = 4 from [1], we know that there some constant C > 1 such that no C -IP stable clutering exists. For a given input, let α ∗ be the minimum value such that a α ∗ -IP stable clustering exists. Is there an efficient algorithm which returns an O ( α ∗ ) -IP stable clustering? Note that our algorithm satisfies this for α = Ω(1) . An even stronger result would be to find a PTAS which returns a (1 + ε ) α ∗ -IP stable clustering.

For what specific metrics (other than the line or tree metrics with k = 2 ) can we get 1 -IP stable clusterings efficiently?

In addition to stability, it is desirable that a clustering algorithm also achieves strong global welfare guarantee. Our algorithm gives constant approximation for k -center. What about other metrics, such as k -means?

-★-（

Our Alg

-★- k-means++

Adult

Drug

IndianLiver

200

★★★

★★★★

500

★★

#Unf

200

★

100

+*★

*+*********★

★★**

★★★

★★***★

本*******

0

0

0

0

5

10

15

20

25

0

5

10

15

20

25

0

5

10

15

20

25

3.0

2.5

2.5

MaxVi

2.5

2.0

2.0

2.0

★

1.5

★★★**★

★★★★★★★

★

1.5

★

★★★★★★★★★★★

★★★★★★

1.5

1.0

★***********************

1.0

★

1.0

0.5

0.5

0.5

0

5

10

15

20

25

0

5

10

15

20

25

0

5

10

15

20

25

1.6

1.6

1.6

MeanVi

1.2

1.2

1.2

0.8

★★★★★★★★★*★★★*

0.8

0.8

★★★★

+****

0.4

0.4

0.4

0

5

10

15

20

25

0

5

10

15

20

25

0

5

10

15

20

25

40

60

75

50

40

真

20

25

20

0

5

10

15

20

25

0

5

10

15

20

25

0

5

10

15

20

25

k-means Cost

10000

40000

★

★

10000

★

★★

5000

★★

★★★

★★★★有★★决★★有★

30000

★★★★★★★*★★★★★★

5000

20000

0

5

10

15

20

25

0

5

10

15

20

25

0

5

10

15

20

25

k-center Cost

★★★★

★★

+★★

15

10

★★★★★★

10

+★*★★***★**★*

★★

10

★★

+***

5

★

5

5

0

5

10

15

20

25

0

5

10

15

20

25

0

5

10

15

20

25

k

k

k

★－

Our Alg

-★- k-means++

BreastCancer

Car

Hard

400

★★*★★★★★★★

#Unf

100

*★★***★*

★★★

#*★★

200

★

★*★*★长

材

+*****

750

500

★

★★

250

★

****++

0

0

0

5

10

15

20

25

0

5

10

15

20

25

0

1000

2000

3000

MaxVi

1.6

1.3

2.0

17

★

1.4

1.2

★★★

★★★★★★★

★★★★

★★

1.1

★

1.5

!

好

1.2

★

+★*

0

5

10

15

20

25

0

5

10

15

20

25

0

1000

2000

3000

★★★

0.95

★★★★★★★★★★★★★★★★★★

0.60

MeanVi

0.8

0.55

0.6

★★

0.90

★+*★

0.50

0

5

10

15

20

25

0

5

10

15

20

25

0

1000

2000

3000

★★

100

100

Cost

4.0

*******★****

50

50

*

3.5

0

5

10

15

20

25

0

5

10

15

20

25

0

1000

2000

3000

k-means Cost

30000

★

80000

★

10

20000

60000

10000

0

0

5

10

15

20

25

0

5

10

15

20

25

0

1000

2000

3000

k-center Cost

★

★

15

+★+★

8

★★

0.05

★★★

★★

★★★★

★★

★★★

★★★★★★★★★*★

10

6

★*★★★*★★*★★*★★★

★★★★

0.00

0

5

10

15

20

25

0

5

10

15

20

25

0

1000

2000

3000

k

k

k

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper continues the study of individual preference stability (IP stability) initiated by Ahmadi et. al. (2022). Given a set of points in a metric space, a k-clustering is said to be 1-IP stable if for every point in the set, its average distance to the points in its cluster is at most its average distance to the points in a different cluster; so, in this sense, a point prefers its own cluster to other clusters. There are datasets for which a 1-IP stable clustering does not exist, and further, it is NP-hard to decide whether a dataset admits a 1-IP stable k-clustering. In light of this, it is natural to broaden the definition to approximate IP stability: a k-clustering is \\alpha-IP stable if the average distance of a point to another cluster is at most \\alpha times its average distance to its own cluster. The work of Ahmadi et. al. shows that an O(n)-IP stable clustering always exists and gives an algorithm for computing one. The present work closes the gap between the known lower and upper bounds: the authors propose an algorithm that always outputs an O(1)-stable clustering (so they also prove that one always exists, a fact not already established). Interestingly, the output clustering has the additional property that it is a constant factor approximation for k-center. Indeed, the greedy k-center algorithm of Gonzalez is a phase in their algorithm, and the \"ball-carving\" phase of their algorithm is similar in spirit to another approximation algorithm for k-center (although there are important nuances in the algorithm in the present work that differentiate it). The fact that the algorithm doubles as an approximation algorithm for k-center, the authors note, ensures that the O(1)-IP stable clustering is in some sense meaningful; for, they demonstrate that a random k-coloring a graph will produce an O(1)-IP stable clustering (for certain k), but of course such a clustering is not meaningful in general. \n\nThe authors also introduce two variants of IP stability. For Min-IP stability, in which \"average distance\" is replaced with \"min distance\" in the definition of IP stability, they show that the Single Link algorithm produces an exactly stable (optimal) clustering. For Max-IP stability (defined analogously), they show the greedy k-center algorithm of Gonzalez gives a 3-IP stable clustering. \n\nFinally, the authors show experiments comparing their algorithms to k-means ++ as a baseline. While their algorithm performs slightly worse than k-means ++ for IP stability in general, they claim its robustness by showing a hard instance for which it outperforms k-means ++. They also show that in practice k-means ++ can be up to five times worse than their (optimal) algorithm for Min-IP stability.\n\nSTRENGTHS:\n- This work closes the (previously large) gap between upper and lower bounds for IP-stability. It demonstrates the existence of constant-stable clusterings and how to find them efficiently. It also provides a convincing argument that a clustering produced by the algorithm is meaningful by showing that it also gives a constant approximation for k-center, which I think is a key contribution given that the authors show, on the flip side, that a clustering can satisfy O(1)-stability but otherwise not uncover meaningful structure. \n\n- The algorithm takes inspiration from approximation algorithms for k-center but there are important subtleties in the adaptation of these algorithms. The Ball-Carving algorithm (Algorithm 1) is a more nuanced version of an approximation algorithm for k-center (in which one guesses the optimal radius, and then repeatedly cuts out balls of that radius until no points remain). In Algorithm 1, more care is taken as to how to choose the center of the ball at each iteration (instead of being chosen arbitrarily as in the approximation algorithm)  as well as how to assign points to centers (instead of just carving out a ball of a certain radius). Finally, the centers from running the algorithm of Gonzalez are used to prune the number of clusters produced in the carving algorithm. \n\n- The authors repurpose existing algorithms (Single Link clustering and the greedy algorithm of Gonzalez) to give guarantees for Min- and Max- IP stability. This is interesting in its own right because it shows that the notion of stability is intimately related to other clustering objectives.\n\nWEAKNESSES:\n- While Algorithms 1 and 2 are described very clearly, little motivation or intuition is given. For instance, it would be useful to provide examples that show the necessity of the nuances in the Ball-Carving algorithm. \n\n- A discussion of how Algorithms 1/2 differ from or build on the algorithms of Ahmadi et. al. would help highlight the novelty of the contributions. \n\n- A more specific comment: The paragraph on lines 66-70 seems inconsistent with later claims (unless I have misinterpreted something). It seems that if the parameter r is chosen to be anything, then the properties claimed in lines 66-70 hold, but potentially with more than k clusters. Only when r is chosen as a function of r_0  and then the pruning step in Algorithm 2 is performed do the properties seem to hold with k clusters. If this is the case, then I think the current wording in lines 66-70 is easily misinterpreted. Still, the stronger guarantee of uniform IP stability with r chosen as a function of r_0 is noteworthy. \n\n- It seems that Algorithm 3 is simply the well-known algorithm of Single Link clustering. This should be labelled as such and cited.\n\nQUESTIONS:\n- A variant of Single-Link clustering is used in the work of Awasthi, Blum, and Sheffet (2012) to find optimal clusterings on perturbation stable instances. Given the algorithmic similarities, do you see any connections between these two notions of stability? \n\n- Do you have any intuition as to why k-center approximation algorithms in particular are useful for IP-stability algorithms, given that IP-stability (in its original form) is based on average distances? While k-median and k-center are in general different notions, it is perhaps not clear a priori which objective could help inspire an algorithm for IP-stability. Have you considered the question of finding an algorithm that produces a clustering that is O(1)-IP stable and also constant approximate for other objectives?\n\nLIMITATIONS:\nNot applicable.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper considers the problem of finding stable clusterings under a specific notion of stability - individual-preference stability which roughly requires the clustering produced to have the property where the average distance of any datapoint to points within its own cluster to be smaller than the average distance to points within any other cluster. The clustering is said to be $\\alpha$-stable if the average in-cluster distance to be no more than a multiplicative $\\alpha>1$ factor of the average distance to points assigned to any other cluster. This is a fairly natural problem, and was recently proposed by Ahmadi et. al. (ICML 22), who gave some preliminary results for it, including NP-Hardness of deciding whether the input dataset admits a $1$-stable clustering.\n\nThis paper makes three key contributions - \n\n(a) It shows that given any dataset with $n$ points in a general metric space, and any desired number of clusters $k$, there always exists a  $k$-clustering that is $O(1)$-stable. \n\n(b) This $O(1)$-stable $k$-clustering can be found by a computationally efficient algorithm. Moreover, the resulting clustering produced is a constant factor approximation to the optimal $k$-center objective.\n\n(c) For min and max stability (i.e. where the average is replaced with min and max, respectively), they show that for any dataset in a general metric space, and for any choice of $k$, (i) a min-stable $k$-clustering always exists, and is achieved by the usual early-termination of Kruskals minimum spanning tree algorithm, and (ii) a 3 approximate max-stable $k$-clustering always exists and is achieved by the greedy $k$-center algorithm.\n\nSTRENGTHS:\nThis paper considers a very natural question of obtaining a stable clustering, and substantially expands our understanding of this problem. Overall the paper is quite well-written and easy to read, and would be a well-suited for a venue such as NeurIPS. \n\n(a) They show that given a set of any $n$ points in an arbitrary metric space, a $O(1)$-stable $k$-clustering always exists for any choice of $k$. Moreover, the clustering includes all $n$ points. The only known results prior to this work was the existence of a $1$-stable clustering for any set of $n$ points in a $1$-D Euclidean space or metrics induced by a weighted tree, or a bicriteria $O(\\log^2 n/\\epsilon)$ stable clustering achieved by discarding an $\\epsilon$-fraction of the points from the input set. For the stricter requirement of clustering all $n$ points, the only known result was a trivial $O(n)$-stable clustering.\n\n(b) The algorithm that finds this $O(1)$-stable $k$-clustering is simple and quite computationally efficient. The earlier bicriteria approximation result was achieved by applying the HST hammer naively. The new algorithm has the additional desirable property that the resulting clustering produced achieves a constant factor approximation for the $k$-center objective.\n\nWEAKNESSES:\nThe paper has two main weaknesses, which in my opinion are not deal breakers given its other strengths.\n\n(a) The extension to $f$-stable clusterings is obvious, and the results for $f=$ min and max stable clusterings are also very elementary. It would have been far more interesting had the authors characterized properties of a generic $f$ required such that any instance always admits an $O(1)$-$f$-stable $k$-clustering.\n\n(b) The fact that the algorithm achieves a $O(1)$ approximation to the $k$-center objective seems like an afterthought. It doesnt seem like the algorithm was explicitly designed to additionally achieve good approximation guarantees for this other objective, and more like it happened to be that way.\n\nQUESTIONS:\nI understand that these questions are out of scope of this paper, but im curious about the following thing -\n\nGiven how you say that its only interesting to look at stability when the algorithm additionally gives good approximation guarantees for other natural clustering objective functions, I really want to know what happens when you simultaneously want to optimize for other objective functions such as say $k$-means or $k$-medians. Is it possible to get a clustering that achieves a good approximation for these standard clustering objectives, while simultaneously being approximately stable? Or are there cases where these two objectives of clustering \"quality\" and \"stability\" are at odds of each other - achieving one objective must necessarily incur a large penalty for the other objective?\n\nMy other question is related to your extension of the notion of stability, and one I have raised before in the weaknesses - can you characterize properties of a generic $f$ required such that any instance always admits an $O(1)$-$f$-stable $k$-clustering?\n\nLIMITATIONS:\nI dont see any obvious limitations of this work.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper concerns alpha-individual preference (IP) stable clusterings, meaning clusterings such that its average distance to points in its cluster is at most alpha times greater than to that in any other cluster. Previously, only O(n)-IP solutions were known, 1-IP solutions were known to not always exist, and we did not know if there existed O(1)-IP solutions. This paper answers affirmatively on metric spaces.\n\nThey provide an O(1)-IP algorithm on metric spaces which additionally is a constant-factor approximation to the k-center problem. The algorithm follows the popular greedy ball technique on metrics, which selects radius-bounded balls (bounded by O(1) times the optimal k-center solution) that cover many points and are distant from each other, and then fixes unassigned points (according to the greedy cluster order) and merges some clusters according to their closest k-center centers.\n\nAdditionally, they define variants Max-IP and Min-IP (where you instead look at the maximum/minimum distances, as opposed to the averages). They show Min-IP can be solved optimally by running Kruskal’s algorithm until there are k trees in the forest. Additionally, greedy k-center 3-approximates Max-IP.\n\nFinally, the experimentally validate their IP algorithm on real and synthetic (adersarily-designed) datasets. On both, their algorithm outperforms k-means++, showing its robustness to hard instances.\n\nSTRENGTHS:\nThis is a nice result that bridges a significant gap posed by previous works (i.e., the gap between O(n)-IP and O(1)-IP). In addition, it is complemented by the lower bound of 1-IP, and it implies a few nice future directions for this work, including finding the minimum alpha such that an alpha-IP solution exists and characterizing when 1-IP solutions exist. The extensions to Max-IP and Min-IP are also strong. For the most part, the writing was clear (though I did find some typos here and there). The methods seemed new in many ways, though they were heavily founded based off existing methods (i.e., clustering by balls on metric graphs).\n\nWEAKNESSES:\nThe biggest weakness is that, while the improvement in approximation is significant, I think the problem is somewhat narrow in scope. I noticed the authors only cited one past work on this area (presumably, the one that proposed it), and the applications were not cited. While this does certainly seem like an interesting new problem, I am not sure of what its place is within ML research literature.\n\nQUESTIONS:\n1. Can you motivate this problem a bit more? Has anyone other than [1] tried attacking this problem?\n2. Do you have any citations for uses?\n3. I understand that IP was the main focus of the paper, but it would be nice to have an argument for the motivations of Min-IP and Max-IP. Do you know any applications they might work for?\n\nLIMITATIONS:\nI did not see a limitations section. It would be preferable but is not necessary for this work.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn $\\alpha$-individual preference (IP) stable clustering, the average distance between every data point to other points in its cluster must be at most $\\alpha$ times the average distance between it to points in any other cluster. \n\nThis paper gives the first polynomial time $O(1)$-IP stable clustering algorithm in general metrics, which improves [Ahmadi et.al, ICML 2022] 's $O(n)$-IP stable result. The algorithm in this paper also has more interesting features that I appreciate. Firstly, it satisfies an even stronger uniform IP stability guarantee. Precisely, there exists a global parameter $r$, such that for every data point, its average distance to points in its cluster is bounded by $O(r)$ while its average distance to point in any other cluster is at least $O(r)$. Secondly, the k-clustering it finds is also a constant factor approximation to the classical k-center clustering problem, which makes the solution much more meaningful in applications. I think this result shows that k-center clustering can admit IP stability without paying too much. Exciting news!\n\nMoreover, the algorithm presented is clean and easy to follow with its analysis building on certain clever observations. Authors also do experiments against [Ahmadi et.al, ICML 2022]  to show the practical value of their algorithm. At last, the authors also study other definitions of IP-stable and obtain similar results.\n\nOverall, I recommend \"accept\".\n\nSTRENGTHS:\n1. The first efficient algorithm to compute O(1)-IP stable approximate clustering in general metrics. Additionally, the algorithm has more important features beyond IP stability. The result significantly improves the previous works.\n\n2. The algorithm and analysis are interesting and have some non-trivial insights.\n\n3. The experiments also outperform the baseline.\n\nWEAKNESSES:\nBoth the constants for IP approximation and k-center approximation may be too large. While IP stability is motivated by practical applications, I would expect that the guarantee is not a quite large number such as 2 or 3. The paper's experiment already shows that the worst case may be avoided in practice and I would like to see more discussion in this direction. Anyway, as a theoretical submission, I think this is totally fine and does not lower my rating.\n\nQUESTIONS:\n1. It is argued that the running time can be improved in Euclidean space. What is the exact running time that you can achieve? What about $\\tilde{O}(nkd)$? (a typical running time for Euclidean $k$-clustering.)\n\n2. Have you formally proved that your algorithm returns an $O(1)$-approximation for $k$-center in the main text? I know it can be seen from Algorithm 2 but it is better to provide formal proof and write the exact approximation ratio.\n\nLIMITATIONS:\nI do not see the potential negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-11"}