Fair Clustering Under a Bounded Cost

1 Introduction . Machine learning algorithms are increasingly being applied to settings that directly influence human lives . This has spurred a growing fair machine learning community [ 9 ] , which develops machine learning algorithms that are made to satisfy certain fairness criteria . Choosing an appropriate definition of fairness—and even deciding if explicitly defining fairness is appropriate to begin with— is a morally-laden and application-specific decision [ 28 , 43 ] . We make no normative statements here ; rather , we focus on a commonly-used and often legally-backed family of fairness definitions—group fairness—in the context of clustering , arguably the most fundamental unsupervised learning problem . A recent group-membership fairness definition , called fair clustering in the literature , has received significant interest [ 20 , 10 , 11 , 2 , 7 , 31 , 22 , 8 , 3 , 33 , 1 ] . In fair clustering , each point has a color that designates its group membership , and a clustering objective such as k-median or k-means is given . The goal is to find a clustering that minimizes the objective subject to the constraint that each cluster has each color represented within some pre-specified proportions . For example , there may be two colors , red and blue , and the constraint could require per-color representation between 40 % and 60 % . An acknowledged fact in fair clustering—and , indeed , in many allocation and matching settings—is that the fairness ( e.g. , proportion ) constraint could cause degradation in the clustering objective [ 12 , 19 ] . A point may be assigned to a further away center ( cluster ) to satisfy the proportion constraint [ 20 ] . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . The degradation in the objective due to the imposed fairness constraint is called the price of fairness ( PoF ) , mathematically defined as PoF = ( cost of fair solution ) / ( cost of agnostic solution ) . Unlike some examples in the literature [ 12 , 21 ] , the price of fairness in the case of fair clustering is unbounded , as seen in Figure 1 . By enforcing a form of group fairness requiring an even split across colors in each cluster , a fair clustering algorithm would perform arbitrarily poorly as the two groups of points separate in space , while a “ colorblind ” algorithm would remain unchanged ( bottom-left and bottom-right of Figure 1 , respectively ) . The possibly unbounded increase in the clustering cost ( unbounded price of fairness ) indicates that fair clustering can yield clusters consisting of points that are far apart in the metric space instead of combining nearby points—often the main motivation behind clustering in machine learning and data analysis . Furthermore , the legal notion of disparate impact does not force an organization to output a fair clustering if it can justify an unfair one due to “ business necessity , ” i.e. , potential loss in quality [ 45 , 44 ] . This possible conflict between the clustering objective and the fairness constraint indicates the need for fair clustering algorithms that operate in a setting where the clustering cost can not exceed a pre-set upper bound . Our Contributions . In this paper , we address fair clustering under an exogenous threshold on the clustering objective . We formulate the problem mathematically in a general setting that captures all of the traditional k-clustering objectives , i.e . k-center , k-median , and k-means . Throughout , we focus on two general formulations of group fairness : GROUP-UTILITARIAN and GROUP-EGALITARIAN , along with the GROUP-LEXIMIN objective which generalizes the traditional GROUP-EGALITARIAN definition . We show that these objectives lead to problems that are NP-hard in general . Further , assuming P 6= NP we derive lower bounds on the additive approximation of any polynomial time algorithm for the GROUP-UTILITARIAN and GROUP-EGALITARIAN objectives . We provide bi-criteria approximation algorithms for the GROUP-UTILITARIAN and GROUP-EGALITARIAN objectives in which the constraint has a bounded violation and the objective is bounded from the optimal value by an additive error . For the GROUP-LEXIMIN objective we provide an effective heuristic . Further , we consider other possibly more “ flexible ” fairness objectives , but demonstrate inapproximability results for them . Finally , we test the performance of our algorithms on a collection of datasets and see that we obtain good solutions with low “ fairness violations. ” We note that due to the page limit , all proofs are placed in Appendix A . 2 Related Work . The metric clustering problems k-center , k-median , and k-means are fundamental in unsupervised learning and operations research . All are NP-hard with a long line of research on approximation algorithms . For k-center , two distinct algorithms achieve a 2-approximation which is tight assuming P 6= NP [ 26 , 23 , 27 ] . The current best approximation for k-median is a ( 2.675 + ) -approximation in nO ( ( 1/ ) log ( 1/ ) ) time [ 18 ] , and for k-means , there is a 6.357-approximation [ 4 ] . The fair clustering problem with group fairness constraints was proposed by [ 20 ] . They studied k-center and k-median in a setting with only two colors . Followup work by [ 11 , 10 , 7 , 31 ] gave extensions to the k-means objective , more than two colors , multi-color points ( i.e. , intersecting demographic groups ) , and scalability . Other works by [ 38 , 15 , 16 ] look at non-group-fairness definitions ; the former investigates individual fairness , while the latter two address probabilistic fairness guarantees for pairs or communities of points . The aforementioned works optimize the clustering objective subject to fairness constraints ; however , satisfying the fairness constraints may come at the expense of a significant increase in the clustering objective . Accordingly [ 48 ] and very recently [ 37 ] explored the cost/fairness tradeoff , but using a multi-objective approach . Unlike our work they do not establish approximation guarantees . Further , the fairness objectives used are different from ours , i.e . in [ 48 ] the fairness objective is penalized for proportions that are not precisely equal to the population level while in [ 37 ] it is penalized for color ratios that are not equal to 1 . Moreover , in [ 48 ] the cost/fairness tradeoff is a non-monotone function of a parameter which the user must adjust , while [ 37 ] only focuses on the k-means objective and provides convergence guarantees only for a smoothed version of the original problem . We focus on the formal tradeoff between fairness and the clustering objective . We note that the clustering objective can be replaced by the price of fairness ( PoF ) . However , in our setting , the results are clearer if we refer to the clustering objective instead of the PoF . Ultimately a higher PoF corresponds to a weakly higher fair clustering objective and vice versa . In fact , they are multiples of one another : PoF = ( cost of fair solution ) / ( cost of agnostic solution ) . This tradeoff between fairness and efficiency manifested the PoF concept , in operations research by [ 12 ] and simultaneously in computer science by [ 19 ] , showing general approaches to defining and measuring it . Similar to our work , others have adapted PoF as a hard constraint in emergency response [ 30 ] , organ allocation [ 39 ] , and rideshare [ 36 , 42 ] , and a partial constraint in scarce resource allocation for kidney dialysis [ 29 ] and organ exchange [ 40 ] . We propose a framework for balancing PoF for a traditional “ efficient ” objective in clustering , which finds application in areas such as advertising , network analysis , and data summarization . 3 Preliminaries . In a clustering problem , we are given a set of points C in a metric space . A distance function d ( i , j ) specifies the distance between each pair of points i , j ∈ C. Furthermore , d is symmetric , non-negative , and satisfies the triangle inequality . Common clustering cost ( loss ) functions ( e.g. , k-means or k-median ) can be written as min S : |S|≤k , φ Lkp ( C ) = min S : |S|≤k , φ ( ∑ j∈C d p ( j , φ ( j ) ) ) 1/p where k is the number of clusters , S is the set of cluster centers chosen from a candidate set of centers S , and φ : C → S is an assignment function that assigns points to cluster centers . The value p determines the type of clustering , i.e. , p =∞ , 1 , and 2 for k-center , k-median , and k-means , respectively . In fair clustering , each point has a color associated with it to indicate its group membership . Specifically , we have a function χ : C → H whereH is the set of possible colors . We denote the set of all points of color h by Ch . The fair clustering problem ( FC ) [ 20 , 10 , 11 , 2 , 7 , 31 , 8 ] is to minimize the clustering objective while satisfying additional fairness constraints : min S : |S|≤k , φ ( ∑ j∈C dp ( j , φ ( j ) ) ) 1/p ( 1a ) s.t . ∀i ∈ S , ∀h ∈ H : βh| Ci | ≤ | Chi | ≤ αh| Ci | ( 1b ) where Ci is the set of points in cluster i and Chi ⊆ Ci is the subset of points in cluster i with color h. βh and αh are pre-specified lower and upper proportion bounds for color h , respectively . Clearly , 0 < βh ≤ αh < 1 . In “ unfair ” clustering problems , the assignment function φ maps points to the nearest center in S , i.e. , φ ( j ) = argmini∈S d ( i , j ) since this minimizes the objective . However , satisfying the added constraints in fair clustering may cause points to be assigned to clusters that are farther away . This motivates the fair assignment problem ( FA ) , in which the set of centers S is given and the objective is to minimize the clustering cost subject to fairness constraints : min φ ( ∑ j∈C dp ( j , φ ( j ) ) ) 1/p ( 2a ) s.t . ∀i ∈ S , ∀h ∈ H : βh| Ci | ≤ | Chi | ≤ αh| Ci | ( 2b ) The only difference between the fair assignment ( 2 ) and fair clustering ( 1 ) problems is that S is not an optimization variable in the fair assignment problem . 4 Fair Clustering Under a Bounded Cost ( FCBC ) The fundamental idea of fair clustering under a bounded cost ( FCBC ) is to minimize a measure of unfairness subject to an upper bound on the clustering cost : min Unfairness ( 3a ) s.t . Clustering Cost ≤ Given upper bound ( 3b ) Next , we transform ( 3a ) and ( 3b ) above into a clear mathematical optimization problem . The Constraint ( 3b ) : The clustering cost is ( ∑ j∈C d p ( j , φ ( j ) ) ) 1/p . Let U denote the exogeneous upper bound on clustering cost . Then , ( 3b ) becomes ( ∑ j∈C d p ( j , φ ( j ) ) ) 1/p ≤ U . Note that for the case of the k-center where p = ∞ , the constraint reduces to a simpler form , specifically ∀j ∈ C , d ( j , φ ( j ) ) ≤ U . The Objective ( 3a ) : In prior work , a given clustering is considered fair if for each cluster , the proportions of each color lie within pre-specified lower and upper bounds , i.e . : ∀i ∈ S , ∀h ∈ H : βh| Ci | ≤ | Chi | ≤ αh| Ci | . However , bounding the clustering cost may make it impossible to have a fair feasible solution . Therefore , we instead set a measure of unfairness for each color and try to minimize this measure . Let ∆h denote the worst proportional violation across the clusters for a color h. Specifically , for a given clustering , ∆h ∈ [ 0 , 1 ] is the minimum non-negative value such that : ∀i ∈ S : ( βh −∆h ) | Ci | ≤ | Chi | ≤ ( αh + ∆h ) | Ci | . ( 4 ) Clearly , if ∆h = 0 , then color h is within the desired proportion in every cluster . Having set ∆h to be a measure of the unfair treatment that group h receives , we are faced with the question of setting the fairness objective , for which there are many reasonable options . We consider two prominent and intuitive fairness objectives [ 14 ] : GROUP-UTILITARIAN = min ∑ h∈H ∆h , GROUP-EGALITARIAN = min max h∈H ∆h The GROUP-UTILITARIAN objective minimizes the sum of proportional violations for all of the colors , treating all points of a specific color as a single player in a game . The GROUP-EGALITARIAN objective minimizes the maximum proportional violation across the colors . We also consider a more generalized version of the GROUP-EGALITARIAN objective , the GROUP-LEXIMIN objective . Like GROUP-EGALITARIAN , the GROUP-LEXIMIN objective minimizes the maximum ( worst ) violation , but it goes further to minimizes the second-worst violation , then the third-worst violation , and so on until no further improvement can be made . We now state the fair clustering under a bounded cost problem ( FCBC ) : min S : |S|≤k , φ UNFAIRNESS-OBJECTIVE ( 5a ) s.t . ( ∑ j∈C dp ( j , φ ( j ) ) ) 1/p ≤ U ( 5b ) where the UNFAIRNESS-OBJECTIVE could equal GROUP-UTILITARIAN , GROUP-EGALITARIAN , or GROUP-LEXIMIN . Similar to the fair assignment FA problem ( 2 ) , we may define the fair assignment under a bounded cost ( FABC ) problem as : min φ UNFAIRNESS-OBJECTIVE ( 6a ) s.t . ( ∑ j∈C dp ( j , φ ( j ) ) ) 1/p ≤ U ( 6b ) where similarly the optimization is over the assignment function φ while the set of centers S is fixed . 5 Hardness of FCBC & FABC First , we formally state the hardness of the fair clustering FC and the fair assignment FA problems . Theorem 5.1 . The fair clustering FC ( 1 ) and fair assignment FA ( 2 ) problems are NP-hard . We now establish the hardness of fair clustering under a bounded cost FCBC and fair assignment under a bounded cost FABC . We note that these hardness results follow for all objectives ( GROUPUTILITARIAN , GROUP-EGALITARIAN , and GROUP-LEXIMIN ) . Theorem 5.2 . Fair clustering under a bounded cost FCBC and fair assignment under a bounded cost FABC are NP-hard . Although we have shown that both the fair clustering and fair assignment problems under a bounded cost are NP-hard , the reductions rely on setting the upper bound U to a small enough value , precisely that of the optimal fair clustering cost . It seems reasonable to expect both problems to transition into being polynomial time solvable if the upper bound becomes sufficiently large . We show in Section 8 that such a result is not easy to establish and would lead to a true approximation for fair clustering which is yet to be produced in the fair clustering literature for arbitrary metric spaces and arbitrary lower and upper color proportion bounds . For a given clustering cost U , there are many clusterings ( solutions ) of cost not exceeding U . Let SU be the set of those solutions , i.e . if ( St , φt ) ∈ SU , then ( St , φt ) is a clustering with a cost that does not exceed U . Further , let Lt be the size of the smallest non-empty cluster1 in the clustering ( St , φt ) , 1An empty cluster is a cluster with no points assigned to it . This could happen if for example the assignment function φ does not map any point to a a given center including the center itself . then we define L ( U ) to be the size of the smallest cluster across all clusterings of cost not exceeding U , i.e . L ( U ) = min ( St , φt ) ∈SU Lt. Clearly , for U1 and U2 such that U2 ≥ U1 , then L ( U2 ) ≤ L ( U1 ) since SU1 ⊆ SU2 . We can conclude the following fact from the definition of L ( U ) : Fact 5.1 . For a given upper bound U , no clustering with cost less than or equal to U can have less than L ( U ) many points in a non-empty cluster . We show that the quantity L ( U ) plays a fundamental role . In fact , lower bounds on the additive approximation2 for the proportional violations and fairness objectives are related to L ( U ) as shown in the following theorem : Theorem 5.3 . For a given instance of the FCBC or FABC problem with an arbitrary upper bound U , unless P = NP no polynomial time algorithm can produce a solution with a cost not exceeding U that satisfies any of the following conditions : ( a ) The proportional violation of any color h ∈ H is ∆h < 1 8L ( U ) . ( b ) The additive approximation for the GROUP-UTILITARIAN objective is less than |H | 8L ( U ) . ( c ) The additive approximation for the GROUP-EGALITARIAN objective is less than 1 8L ( U ) . 6 Algorithms for FCBC Our main result for the FCBC problem is the following theorem which follows as a direct consequence of the guarantees of Theorems 6.2 , 6.4 , 6.5 , 6.6 , and 6.7 : Theorem 6.1 . For any clustering objective , given a boundU on the clustering cost , Algorithm 1 solves the fair clustering under a bounded cost FCBC problem at a cost of at most U ′ = ( 2 + α ) U where α is the approximation ratio of the color-blind clustering algorithm . The additive approximation is |H | ( + 2L ( U ′ ) ) for the GROUP-UTILITARIAN objective and + 2 L ( U ′ ) for the GROUP-EGALITARIAN objective . From the theorem above , it is clear that the additive approximation guarantees we have improve when the cost does not permit small clusters . Indeed , in the absence of outlier points and for reasonable values of k , small clusters are unlikely to exist . Further , empirically we verify the smallest cluster size and find that the smallest cluster size is 159 points ( see Section 9.3 ) . See Appendix C for more discussion . We now provide our general algorithm for fair clustering under a bounded cost FCBC which we denote by ALG-FCBC ( U , UNFAIRNESS-OBJECTIVE ) where we have made explicit reference to the dependence of ALG-FCBC on the given cost upper bound U and the desired UNFAIRNESSOBJECTIVE which could either be the GROUP-UTILITARIAN , GROUP-EGALITARIAN , or GROUPLEXIMIN objective . ALG-FCBC ( U , UNFAIRNESS-OBJECTIVE ) ( see Algorithm 1 ) involves two steps , in step ( 1 ) : we use a color-blind approximation algorithm to find the cluster centers S , in step ( 2 ) : we call the algorithm ALG-FABC ( S , U ′ , UNFAIRNESS-OBJECTIVE ) for the FABC problem . It should be noted that we have fed ALG-FABC the set of centers S from step ( 1 ) , further the cost upper bound for ALG-FABC is set to U ′ = ( 2 + α ) U while the UNFAIRNESS-OBJECTIVE remains unchanged . We further note that ALG-FABC will have the same clustering objective as ALG-FCBC , e.g . if ALG-FCBC is given the k-median objective so well ALG-FABC . Clearly , from algorithm ALG-FCBC the FCBC problem is closely related to the FABC problem . In fact , we establish the following general theorem for all clustering objectives : k-center , k-median , and k-means that shows that an algorithm which solves the FABC problem with provable guarantees can be used to solve the FCBC problem with provable guarantees : Theorem 6.2 . For any clustering objective and both the GROUP-UTILITARIAN and GROUPEGALITARIAN objectives , given an algorithm that solves fair assignment under a bounded cost FABC with additive approximation µ , the fair clustering under a bounded cost FCBC problem can be solved with an additive approximation of µ and at a cost of at most ( 2 + α ) U , where α is the approximation ratio of the color-blind clustering algorithm . 2An algorithm for a minimization problem with additive approximation µ > 0 , returns a value for the objective that is at most OPT +µ where OPT is the optimal value . Algorithm 1 : ALG-FCBC ( U , UNFAIRNESS-OBJECTIVE ) 1 : Choose a set of centers S by running a color-blind clustering algorithm of approximation ratio α . 2 : Set U ′ = ( 2 + α ) U and call ALG-FABC ( S , U ′ , UNFAIRNESS-OBJECTIVE )

Summary: EDIT: I increased my score following the discussions and rebuttal. The main question in the paper is how can we cluster data so that we optimize for some fairness desiderata. In other words, if we have a target for the amount of clustering cost we are aiming for, how can we find the most fair solution, subject to that cost constraints? Put differently, the authors address the problem of minimizing the amount of "unfairness" subject to the clustering cost being below a certain threshold. The two desiderata put forth by the authors in terms of fairness have to do with "group utilitarian" objective and with "group egalitarian" (they also further study a generalization called "group leximin" objective. The authors give both positive results and some impossibility results. Finally, they run some experiments to validate their findings.  These three objectives are desiderata that have been studied in the past. Briefly, the group utilitarian objective minimizes the sum of all fairness violations, whereas the group egalitarian objective minimizes the maximum fairness violation. Going a step further, the leximin (LM) objective minimizes the maximum fairness violation first, then minimizes the second worst etc.   One of the hardness results they prove is that for all three objectives, both fairly clustering and fairly assigning points to fixed centers is NP-hard.  On the positive side, they give an approximation algorithm that yields a (2+a)U-approximation where U is an exogenous parameter that is the upper bound on the clustering cost and a is the approximation ratio of a color-blind clustering algorithm. The algorithms presented for the different objectives depend on some assumptions about the sizes of the clusters. The idea behind the algorithms is to first run a color blind clustering algorithm to select centers, followed by an LP to get a fair assignment and a flow computation.  The authors perform some computational experiments on the adult and the census data sets to evaluate how their fairness guarantees look like.