SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Anders Aamand
aamand@mit.edu
Justin Y. Chen
justc@mit.edu
Allen Li
cliu568@mit.edu
Sandeep Silwal
silwal@mit.edu
Pattara Sukprasert*
Databricks
pat.sukprasert@databricks.com
Ali Vakilian
TTIC
vakilian@ttic.edu
Fred Zhang
z0@berkeley.edu
Individual preference (IP) stability, introduced by Ahmadi et al. (ICML 2022), is a natural clustering objective inspired by stability and fairness constraints. A clustering is  $\alpha$ -IP stable if the average distance of every data point to its own cluster is at most  $\alpha$  times the average distance to any other cluster. Unfortunately, determining if a dataset admits a 1-IP stable clustering is NP-Hard. Moreover, before this work, it was unknown if an  $o(n)$ -IP stable clustering always exists, as the prior state of the art only guaranteed an  $O(n)$ -IP stable clustering. We close this gap in understanding and show that an  $O(1)$ -IP stable clustering always exists for general metrics, and we give an efficient algorithm which outputs such a clustering. We also introduce generalizations of IP stability beyond average distance and give efficient near optimal algorithms in the cases where we consider the maximum and minimum distances within and between clusters.
In applications involving and affecting people, socioeconomic concepts such as game theory, stability, and fairness are important considerations in algorithm design. Within this context, Ahmadi et al. [1] (ICML 2022) introduced the notion of individual preference stability (IP stability) for clustering. At a high-level, a clustering of an input dataset is called 1-IP stable if, for each individual point, its average distance to any other cluster is larger than the average distance to its own cluster. Intuitively, each individual prefers its own cluster to any other, and so the clustering is stable.
There are plenty of applications of clustering in which the utility of each individual in any cluster is determined according to the other individuals who belong to the same cluster. For example, in designing personalized medicine, the more similar the individuals in each cluster are, the more effective medical decisions, interventions, and treatments can be made for each group of patients. Similarly, stability guarantees are desired in designing personalized learning environments or marketing campaigns to ensure that no individual wants to deviate from their assigned cluster. Furthermore the focus on individual utility in IP stability (a clustering is only stable if every individual is "happy") enforces a notion of individual fairness in clustering.
In addition to its natural connections to cluster stability, algorithmic fairness, and Nash equilibria, IP stability is also algorithmically interesting in its own right. While clustering is well-studied with respect to global objective functions (e.g. the objectives of centroid-based clustering such as  $k$ -means
* Significant part of the work was done while P.S. was a Ph.D. candidate at Northwestern University.
or correlation/hierarchical clustering), less is known when the goal is to partition the dataset such that every point in the dataset is individually satisfied with the solution. Thus, IP stability also serves as a natural and motivated model of individual preferences in clustering.
1.1 Problem Statement and Preliminaries
The main objective of our clustering algorithms is to achieve IP stability given a set  $P$  of  $n$  points lying in a metric space  $(M, d)$  and  $k$ , the number of clusters.
Definition 1.1 (Individual Preference (IP) Stability [1]). The goal is to find a disjoint  $k$ -clustering  $\mathcal{C} = (C_1, \dots, C_k)$  of  $P$  such that every point, on average, is closer to the points of its own cluster than to the points in any other cluster. Formally, for all  $v \in P$ , let  $C(v)$  denote the cluster that contains  $v$ . We say that  $v \in P$  is IP stable with respect to  $\mathcal{C}$  if either  $C(v) = \{v\}$  or for every  $C' \in \mathcal{C}$  with  $C' \neq C$ ,
$$
\frac {1}{| C (v) | - 1} \sum_ {u \in C (v)} d (v, u) \leq \frac {1}{| C ^ {\prime} |} \sum_ {u \in C ^ {\prime}} d (v, u). \tag {1}
$$
The clustering  $\mathcal{C}$  is 1-IP stable (or simply IP stable) if and only if every  $v\in P$  is stable with respect to  $\mathcal{C}$ .
Ahmadi et al. [1] showed that an arbitrary dataset may not admit an IP stable clustering. This can be the case even when  $n = 4$ . Furthermore, they proved that it is NP-hard to decide whether a given a set of points have an IP stable  $k$ -clustering, even for  $k = 2$ . This naturally motivates the study of the relaxations of IP stability.
Definition 1.2 (Approximate IP Stability). A  $k$ -clustering  $\mathcal{C} = (C_1, \dots, C_k)$  of  $P$  is  $\alpha$ -approximate IP stable, or simply  $\alpha$ -IP stable, if for every point  $v \in P$ , the following holds: either  $C(v) = \{v\}$  or for every  $C' \in \mathcal{C}$  and  $C' \neq C$ ,
$$
\frac {1}{| C (v) | - 1} \sum_ {u \in C (v)} d (v, u) \leq \frac {\alpha}{| C ^ {\prime} |} \sum_ {u \in C ^ {\prime}} d (v, u). \tag {2}
$$
The work of [1] proposed algorithms to outputting IP stable clusterings on the one-dimensional line for any value of  $k$  and on tree metrics for  $k = 2$ . The first result implies an  $O(n)$ -IP stable clustering for general metrics, by applying a standard  $O(n)$ -distortion embedding to one-dimensional Euclidean space. In addition, they give a bicriteria approximation that discards an  $\varepsilon$ -fraction of the input points and outputs a  $O\left(\frac{\log^2 n}{\varepsilon}\right)$ -IP stable clustering for the remaining points.
Given the prior results, it is natural to ask if the  $O(n)$  factor for IP stable clustering given in [1] can be improved.
1.2 Our Results
New Approximations. Improving on the  $O(n)$ -IP stable algorithm in [1], we present a deterministic algorithm which for general metrics obtains an  $O(1)$ -IP stable  $k$ -clustering, for any value of  $k$ . Note that given the existence of instances without 1-IP stable clusterings, our approximation factor is optimal up to a constant factor.
Theorem 1.3. (Informal; see Theorem 3.1) Given a set  $P$  of  $n$  points in a metric space  $(M, d)$  and a number of desired clusters  $k \leq n$ , there exists an algorithm that computes an  $O(1)$ -IP stable  $k$ -clustering of  $P$  in polynomial time.
Our algorithm outputs a clustering with an even stronger guarantee that we call uniform (approximate) IP stability. Specifically, for some global parameter  $r$  and for every point  $v \in P$ , the average distance from  $v$  to points in its own cluster is upper bounded by  $O(r)$  and the average distance from  $v$  to points in any other cluster is lower bounded by  $\Omega(r)$ . Note that the general condition of  $O(1)$ -IP stability would allow for a different value of  $r$  for each  $v$ .
We again emphasize that Theorem 1.3 implies that an  $O(1)$ -IP stable clustering always exists, where prior to this work, only the  $O(n)$  bound from [1] was known for general metrics.
Additional  $k$ -center clustering guarantee. The clustering outputted by our algorithm satisfies additional desirable properties beyond  $O(1)$ -IP stability. In the  $k$ -center problem, we are given  $n$  points in a metric space, and our goal is to pick  $k$  centers as to minimize the maximal distance of any point to the nearest center. The clustering outputted by our algorithm from Theorem 1.3 has the added benefit of being a constant factor approximation to the  $k$ -center problem in the sense that if the optimal  $k$ -center solution has value  $r_0$ , then the diameter of each cluster outputted by the algorithm is  $O(r_0)$ . In fact, we argue that IP stability is more meaningful when we also seek a solution that optimizes some clustering objective. If we only ask for IP stability, there are instances where it is easy to obtain  $O(1)$ -IP stable clusterings, but where such clusterings do not provide insightful information in a typical clustering application. Indeed, as we will show in Appendix B, randomly  $k$ -coloring the nodes of an unweighted, undirected graph (where the distance between two nodes is the number of edges on the shortest path between them), gives an  $O(1)$ -IP stable clustering when  $k \leq O\left(\frac{\sqrt{n}}{\log n}\right)$ . Our result on trees demonstrates the idiosyncrasies of individual objectives thus our work raises further interesting questions about studying standard global clustering objectives under the restriction that the solutions are also (approximately) IP stable.
Max and Min-IP Stability. Lastly, we introduce a notion of  $f$ -IP stability, generalizing IP stability.
Definition 1.4 ( $f$ -IP Stability). Let  $(M, d)$  be a metric space,  $P$  a set of  $n$  points of  $M$ , and  $k$  the desired number of partitions. Let  $f: P \times 2^P \to \mathbb{R}^{\geq 0}$  be a function which takes in a point  $v \in P$ , a subset  $C$  of  $P$ , and outputs a non-negative real number. We say that a  $k$ -clustering  $\mathcal{C} = (C_1, \dots, C_k)$  of  $P$  is  $f$ -IP stable if for every point  $v \in P$ , the following holds: either  $C(v) = \{v\}$  or for every  $C' \in \mathcal{C}$  and  $C' \neq C$ ,
$$
f (v, C (v) \backslash \{v \}) \leq f (v, C ^ {\prime}). \tag {3}
$$
Note that the standard setting of IP stability given in Definition 1.1 corresponds to the case where  $f(v, C) = (1 / |C|) \times \sum_{v' \in C} d(v, v')$ . The formulation of  $f$ -IP stability, therefore, extends IP stability beyond average distances and allows for alternative objectives that may be more desirable in certain settings. For instance, in hierarchical clustering, average, minimum, and maximum distance measures are well-studied.
In particular, we focus on max-distance and min-distance in the definition of  $f$ -IP stable clustering in addition to average distance (which is just Definition 1.1), where  $f(v, C) = \max_{v' \in C} d(v, v')$  and  $f(v, C) = \min_{v' \in C} d(v, v')$ . We show that in both the max and min distance formulations, we can solve the corresponding  $f$ -IP stable clustering (nearly) optimally in polynomial time. We provide the following result:
Theorem 1.5 (Informal; see Theorem 4.1 and Theorem 4.2). In any metric space, Min-IP stable clustering can be solved optimally and Max-IP stable clustering can be solved approximately within a factor of 3, in polynomial time.
We show that the standard greedy algorithm of  $k$ -center, a.k.a, the Gonzalez's algorithm [15], yields a 3-approximate Max-IP stable clustering. Moreover, we present a conceptually clean algorithm which is motivated by considering the minimum spanning tree (MST) to output a Min-IP stable clustering. This implies that unlike the average distance formulation of IP stable clustering, a Min-IP stable clustering always exists. Both algorithms work in general metrics.
Metric	Approximation Factor	Reference	Remark
1D Line metric	1	[1]
Weighted tree	1	[1]	Only for k=2
General metric	O(n)	[1]
General metric	O(1)	This work
Table 1: Our results on IP stable  $k$  -clustering of  $n$  points. All algorithms run in polynomial time.
Empirical Evaluations. We experimentally evaluate our  $O(1)$ -IP stable clustering algorithm against  $k$ -means++, which is the empirically best-known algorithm in [1]. We also compare  $k$ -means++ with our optimal algorithm for Min-IP stability. We run experiments on the Adult data set<sup>2</sup>
$^{2}$ https://archive.ics.uci.edu/ml/datasets/adult; see [18].
used by [1]. For IP stability, we also use four more datasets from UCI ML repository [11] and a synthetic data set designed to be a hard instance for  $k$ -means++. On the Adult data set, our algorithm performs slightly worse than  $k$ -means++ for IP stability. This is consistent with the empirical results of [1]. On the hard instance<sup>3</sup>, our algorithm performs better than  $k$ -means++, demonstrating that the algorithm proposed in this paper is more robust than  $k$ -means++. Furthermore for Min-IP stability, we empirically demonstrate that  $k$ -means++ can have an approximation factors which are up to a factor of 5x worse than our algorithm. We refer to Section 5 and Appendix C for more details.
1.3 Technical Overview
The main contribution is our  $O(1)$ -approximation algorithm for IP stable clustering for general metrics. We discuss the proof technique used to obtain this result. Our algorithm comprises two steps. We first show that for any radius  $r$ , we can find a clustering  $\mathcal{C} = (C_1, \ldots, C_t)$  such that (a) each cluster has diameter  $O(r)$ , and (b) the average distance from a point in a cluster to the points of any other cluster is  $\Omega(r)$ .
Conditions (a) and (b) are achieved through a ball carving technique, where we iteratively pick centers  $q_{i}$  of distance  $> 6r$  to previous centers such that the radius  $r$  ball  $B(q_{i}, r)$  centered at  $q_{i}$  contains a maximal number of points, say  $s_{i}$ . For each of these balls, we initialize a cluster  $D_{i}$  containing the  $s_{i}$  points of  $B(q_{i}, r)$ . We next consider the annulus  $B(q_{i}, 3r) \setminus B(q_{i}, 2r)$ . If this annulus contains less than  $s_{i}$  points, we include all points from  $B(q_{i}, 3r)$  in  $D_{i}$ . Otherwise, we include any  $s_{i}$  points in  $D_{i}$  from the annulus. We assign each unassigned point to the first center picked by our algorithm and is within distance  $O(r)$  to the point. This is a subtle but crucial component of the algorithm as the more natural "assign to the closest center" approach fails to obtain  $O(1)$ -IP stability.
One issue remains. With this approach, we have no guarantee on the number of clusters. We solve this by merging some of these clusters while still maintaining that the final clusters have radius  $O(r)$ . This may not be possible for any choice of  $r$ . Thus the second step is to find the right choice of  $r$ . We first run the greedy algorithm of  $k$ -center and let  $r_0$  be the minimal distance between centers we can run the ball carving algorithm  $r = cr_0$  for a sufficiently small constant  $c$ . Then if we assign each cluster of  $\mathcal{C}$  to its nearest  $k$ -center, we do indeed maintain the property that all clusters have diameter  $O(r)$ , and since  $c$  is a small enough constant, all the clusters will be non-empty. The final number of clusters will therefore be  $k$ . As an added benefit of using the greedy algorithm for  $k$ -center as a subroutine, we obtain that the diameter of each cluster is also  $O(r_0)$ , namely the output clustering is a constant factor approximation to  $k$ -center.
1.4 Related Work
Fair Clustering. One of the main motivations of IP stable clustering is its interpretation as a notion of individual fairness for clustering [1]. Individual fairness was first introduced by [12] for the classification task, where, at high-level, the authors aim for a classifier that gives "similar predictions" for "similar" data points. Recently, other formulations of individual fairness have been studied for clustering [17, 2, 7, 8], too. [17] proposed a notion of fairness for centroid-based clustering: given a set of  $n$  points  $P$  and the number of clusters  $k$ , for each point, a center must be picked among its  $(n / k)$ -th closest neighbors. The optimization variant of it was later studied by [19, 20, 24].[7] studied a pairwise notion of fairness in which data points represent people who gain some benefit from being clustered together. In a subsequent work, [6] introduced a stochastic variant of this notion. [2] studied the setting in which the output is a distribution over centers and "similar" points are required to have "similar" centers distributions.
Stability in Clustering. Designing efficient clustering algorithms under notions of stability is a well-studied problem<sup>4</sup>. Among the various notions of stability, average stability is the most relevant to our model [4]. In particular, they showed that if there is a ground-truth clustering satisfying the requirement of Equation (1) with an additive gap of  $\gamma > 0$ , then it is possible to recover the solution in the list model where the list size is exponential in  $1 / \gamma$ . Similar types of guarantees are shown in the work by [9]. While this line of research mainly focuses on presenting faster algorithms utilizing the
3The construction of this hard instance is available in the appendix of [1].
4For a comprehensive survey on this topic, refer to [3].
strong stability conditions, the focus of IP stable clustering is whether we can recover such stability properties in general instances, either exactly or approximately.
Hedonic Games. Another game-theoretic study of clustering is hedonic games [10, 5, 13]. In a hedonic game, players choose to form coalitions (i.e., clusters) based on their utility. Our work differs from theirs, since we do not model the data points as selfish players. In a related work, [23] proposes another utility measure for hedonic clustering games on graphs. In particular, they define a closeness utility, where the utility of node  $i$  in cluster  $C$  is the ratio between the number of nodes in  $C$  adjacent to  $i$  and the sum of distances from  $i$  to other nodes in  $C$ . This measure is incomparable to IP stability. In addition, their work focuses only on clustering in graphs while we consider general metrics.
We let  $(M,d)$  denote a metric space, where  $d$  is the underlying distance function. We let  $P$  denote a fixed set of points of  $M$ . Here  $P$  may contain multiple copies of the same point. For a given point  $x \in P$  and radius  $r \geq 0$ , we denote by  $B(x,r) = \{y \in P \mid d(x,y) \leq r\}$ , the ball of radius  $r$  centered at  $x$ . For two subsets  $X,Y \subseteq P$ , we denote by  $d(X,Y) = \inf_{x \in X,y \in Y} d(x,y)$ . Throughout the paper,  $X$  and  $Y$  will always be finite and then the infimum can be replaced by a minimum. For  $x \in P$  and  $Y \subseteq P$ , we simply write  $d(x,Y)$  for  $d(\{x\},Y)$ . Finally, for  $X \subseteq P$ , we denote by  $\operatorname{diam}(X) = \sup_{x,y \in X} d(x,y)$ , the diameter of the set  $X$ . Again,  $X$  will always be finite, so the supremum can be replaced by a maximum.
3 Constant-Factor IP Stable Clustering
In this section, we prove our main result: For a set  $P = \{x_{1},\ldots ,x_{n}\}$  of  $n$  points with a metric  $d$  and every  $k\leq n$ , there exists a  $k$ -clustering  $\mathcal{C} = (C_1,\dots ,C_k)$  of  $P$  which is  $O(1)$ -approximate IP stable. Moreover, such a clustering can be found in time  $\widetilde{O} (n^{2}T)$ , where  $T$  is an upper bound on the time it takes to compute the distance between two points of  $P$ .
Algorithm Our algorithm uses a subroutine, Algorithm 1, which takes as input  $P$  and a radius  $r \in \mathbb{R}$  and returns a  $t$ -clustering  $\mathcal{D} = (D_1, \ldots, D_t)$  of  $P$  with the properties that (1) for any  $1 \leq i \leq t$ , the maximum distance between any two points of  $D_i$  is  $O(r)$ , and (2) for any  $x \in P$  and any  $i$  such that  $x \notin D_i$ , the average distance from  $x$  to points of  $D_i$  is  $\Omega(r)$ . These two properties ensure that  $\mathcal{D}$  is  $O(1)$ -approximate IP stable. However, we have no control on the number of clusters  $t$  that the algorithm produces. To remedy this, we first run a greedy  $k$ -center algorithm on  $P$  to obtain a set of centers  $\{c_1, \ldots, c_k\}$  and let  $r_0$  denote the maximum distance from a point of  $P$  to the nearest center. We then run Algorithm 1 with input radius  $r = cr_0$  for some small constant  $c$ . This gives a clustering  $\mathcal{D} = (D_1, \ldots, D_t)$  where  $t \geq k$ . Moreover, we show that if we assign each cluster of  $\mathcal{D}$  to the nearest center in  $\{c_1, \ldots, c_k\}$  (in terms of the minimum distance from a point of the cluster to the center), we obtain a  $k$ -clustering  $\mathcal{C} = (C_1, \ldots, C_k)$  which is  $O(1)$ -approximate IP stable. The combined algorithm is Algorithm 2.
We now describe the details of Algorithm 1. The algorithm takes as input  $n$  points  $x_{1},\ldots ,x_{n}$  of a metric space  $(M,d)$  and a radius  $r$ . It first initializes a set  $Q = \emptyset$  and then iteratively adds points  $x$  from  $P$  to  $Q$  that are of distance greater than  $6r$  from points already in  $Q$  such that  $|B(x,r)|$ , the number of points of  $P$  within radius  $r$  of  $x$ , is maximized. This is line 5-6 of the algorithm. Whenever a point  $q_{i}$  is added to  $Q$ , we define the annulus  $A_{i}\coloneqq B(q_{i},3r)\setminus B(q_{i},2r)$ . We further let  $s_i = |B(q_i,r)|$ . At this point the algorithm splits into two cases. If  $|A_{i}| \geq s_{i}$ , we initialize a cluster  $D_{i}$  which consists of the  $s_i$  points in  $B(x,r)$  and any arbitrarily chosen  $s_i$  points in  $A_{i}$ . This is line 8-9 of the algorithm. If on the other hand  $|A_{i}| < s$ , we define  $D_{i}\coloneqq B(q_{i},3r)$ , namely  $D_{i}$  contains all points of  $P$  within distance  $3r$  from  $q_{i}$ . This is line 10 of the algorithm. After iteratively picking the points  $q_{i}$  and initializing the clusters  $D_{i}$ , we assign the remaining points as follows. For any point  $x\in P\backslash \bigcup_{i}D_{i}$ , we find the minimum  $i$  such that  $d(x,q_i)\leq 7r$  and assign  $x$  to  $D_{i}$ . This is line 13-16 of the algorithm. We finally return the clustering  $\mathcal{D} = (D_1,\dots ,D_t)$ .
We next describe the details of Algorithm 2. The algorithm iteratively pick  $k$  centers  $c_{1},\ldots ,c_{k}$  from  $P$  for each center maximizing the minimum distance to previously chosen centers. For each center  $c_{i}$ , it initializes a cluster, starting with  $C_i = \{c_i\}$ . This is line 4-7 of the algorithm. Letting  $r_0$  be
Algorithm 1 BALL-CARVING
1: Input: A set  $P = \{x_{1},\ldots ,x_{n}\}$  of  $n$  points with a metric  $d$  and a radius  $r > 0$ .  
2: Output: Clustering  $\mathcal{D} = (D_1,\dots ,D_t)$  of  $P$ .  
3:  $Q \gets \emptyset, i \gets 1$   
4: while there exists  $x \in P$  with  $d(x,Q) > 6r$  do  
5:  $q_{i} \gets \arg \max_{x \in P:d(x,Q) > 6r} |B(x,r)|$   
6:  $Q \gets Q \cup \{q_i\}$ ,  $s_i \gets |B(q_i,r)|$ ,  $A_i \gets B(q_i,3r) \setminus B(q_i,2r)$   
7: if  $|A_i| \geq s_i$   
8:  $S_i \gets \text{any set of } s_i$  points from  $A_i$   
9:  $D_i \gets B(q_i,r) \cup S_i$   
10: else  $D_i \gets B(q_i,3r_i)$   
11:  $i \gets i + 1$   
12: end while  
13: for  $x \in P$  assigned to no  $D_i$  do  
14:  $j \gets \min \{i \mid d(x,q_i) \leq 7r\}$   
15:  $D_j \gets D_j \cup \{x\}$   
16: end for  
17:  $t \gets |Q|$   
18: return  $\mathcal{D} = (D_1,\dots ,D_t)$
Algorithm 2 IP-CLUSTERING
1: Input: Set  $P = \{x_{1},\ldots ,x_{n}\}$  of  $n$  points with a metric  $d$  and integer  $k$  with  $2\leq k\leq n$ .  
2: Output:  $k$ -clustering  $\mathcal{C} = (C_1,\dots,C_k)$  of  $P$ .  
3:  $S\gets \emptyset$   
4: for  $i = 1,\dots ,k$  do  
5:  $c_{i}\gets \arg \max_{x\in P}\{d(x,S)\}$   
6:  $S\gets S\cup \{c_i\} ,C_i\gets \{c_i\}$   
7: end for  
8:  $r_0\gets \min \{d(c_i,c_j)\mid 1\leq i < j\leq k\}$   
9:  $\mathcal{D}\gets \mathrm{BALL - CARVING}(P,r_0 / 15)$   
10: for  $D\in \mathcal{D}$  do  
11:  $j\gets \arg \min_{i}\{d(c_{i},D)\}$   
12:  $C_j\gets C_j\cup D$   
13: end for  
14: return  $\mathcal{C} = (C_1,\dots,C_k)$
the minimum distance between pairs of distinct centers, the algorithm runs Algorithm 1 on  $P$  with input radius  $r = r_0 / 15$  (line 8-9). This produces a clustering  $\mathcal{D}$ . In the final step, we iterate over the clusters  $D$  of  $\mathcal{D}$ , assigning  $D$  to the  $C_i$  for which  $d(c_i, D)$  is minimized (line 11-13). We finally return the clustering  $(C_1, \ldots, C_k)$ .
Analysis We now analyze our algorithm and provide its main guarantees.
Theorem 3.1. Algorithm 2 returns an  $O(1)$ -approximate  $IP$  stable  $k$  clustering in time  $O(n^{2}T + n^{2}\log n)$ . Furthermore, the solution is also a constant factor approximation to the  $k$ -center problem.
In order to prove this theorem, we require the following lemma on Algorithm 1.