Consistency of Constrained Spectral Clustering under Graph Induced Fair Planted Partitions
Anonymous Author(s)
AfﬁliationAddress email
Abstract
Spectral clustering is popular among practitioners and theoreticians alike.
Whileperformance guarantees for spectral clustering are well understood, recent studieshave focused on enforcing “fairness” in clusters, requiring them to be “balanced”with respect to a categorical sensitive node attribute (e.g. the race distributionin clusters must match the race distribution in the population).
In this paper, weconsider a setting where sensitive attributes indirectly manifest in an auxiliaryrepresentation graph rather than being directly observed.
This graph speciﬁesnode pairs that can represent each other with respect to sensitive attributes and isobserved in addition to the usual similarity graph .
Our goal is to ﬁnd clusters in thesimilarity graph while respecting a new individual-level fairness constraint encodedby the representation graph.
We develop variants of unnormalized and normalizedspectral clustering for this task and analyze their performance under a fair plantedpartition model induced by the representation graph.
This model uses both thecluster membership of the nodes and the structure of the representation graph togenerate random similarity graphs.
To the best of our knowledge, these are the ﬁrstconsistency results for constrained spectral clustering under an individual-levelfairness constraint.
Numerical results corroborate our theoretical ﬁndings.
Introduction
Consider a recommendation service that groups news articles by ﬁnding clusters in a graph whichconnects these articles via cross-references.
Unfortunately, as cross-references between articles withdifferent political viewpoints are uncommon, this service risks forming ideological ﬁlter bubbles.
To counter polarization, it must ensure that clusters on topics like ﬁnance and healthcare includea diverse range of opinions.
This is an example of a constrained clustering problem.
The popularspectral clustering algorithm [35, 40] has been adapted over the years to include constraints such asmust-link and cannot-link constraints [26, 45], size-balanced clusters [6] and statistical fairness [27].
These constraints can be broadly divided into two categories: (i) Population level constraints thatmust be satisﬁed by the clusters as a whole (e g. size-balanced clusters and statistical fairness); and(ii)
Individual level constraints that must be satisﬁed at the level of individual nodes (e.g. must/cannotlink constraints).
To the best of our knowledge, the only known statistical consistency guarantees forconstrained spectral clustering were studied in Kleindessner et al.
[27] in the context of a populationlevel fairness constraint where the goal is to ﬁnd clusters that are balanced with respect to a categoricalsensitive node attribute.
In this paper, we establish consistency guarantees for constrained spectralclustering under a new and more general individual level fairness constraint.
Informal problem description: We assume the availability of two graphs: a similarity graph G inwhich the clusters are to be found and a representation graph R , deﬁned on the same set of nodes as
G , which encodes the “is representative of” relationship.
Our goal is to ﬁnd clusters in G such that every node has a sufﬁcient number of its representatives from R in all clusters.
For example, G maybe a graph of consumers based on the similarity of their purchasing habits and R may be a graphbased on the similarity of their sensitive attributes such as gender, race, and sexual orientation.
This,for instance, would then be a step towards reducing discrimination in online marketplaces [20].
Contributions and results: First , in Section 3.1, we formalize our new individual level fairnessconstraint for clustering, called the representation constraint .
It is different from most existingfairness notions which either apply at the population level [13, 37, 8, 7] or are hard to integratewith spectral clustering [12, 34, 4].
Unlike these notions, our constraint can be used with multiplesensitive attributes of different types (categorical, numerical etc.) and only requires observing anabstract representation graph based on these attributes rather than requiring their actual values, therebydiscouraging individual proﬁling.
Appendix A discusses the utility of individual fairness notions.
Second , in Section 3.2, we develop the representation-aware variant of unnormalized spectralclustering to ﬁnd clusters that approximately satisfy the proposed constraint.
An analogous variantfor normalized spectral clustering is presented in Appendix B.2.
Third , in Section 4.1, we introduce R -PP, a new representation-aware (or fair) planted partition model.
This model generates random similarity graphs G conditioned on both the cluster membership ofnodes and a given representation graph R .
Intuitively, R -PP plants the properties of R in G .
Weshow that this model generates “hard” problem instances and establish the weak consistency 1 of ouralgorithms under this model for a class of d -regular representation graphs (Theorems 4.1 and 4.2).
Tothe best of our knowledge, these are the ﬁrst consistency results for constrained spectral clusteringunder an individual-level constraint.
In fact, we show that our results imply the only other similarconsistency result (but for a population-level constraint) in Kleindessner et al.
[27] as a special case(Appendix A).
Finally, fourth , we present empirical results on both real and simulated data to corroborate ourtheoretical ﬁndings (Section 5).
In particular, our experiments show that our algorithms perform wellin practice, even when the d -regularity assumption on R is violated.
Related work: Spectral clustering has been modiﬁed to satisfy individual level must-link andcannot-link constraints by pre-processing the similarity graph [26], post-processing the eigenvectorsof the graph Laplacian [31], and modifying its optimization problem [48, 49, 45, 46, 14].
It hasalso been extended to accommodate various population level constraints [6, 47].
We are unaware oftheoretical performance guarantees for any of these algorithms.
Of particular interest to us are the fairness constraints for clustering.
One popular population levelconstraint requires sensitive attributes to be proportionally represented in clusters [13, 37, 8, 7, 18, 19].
For example, if 50% of the population is female then the same proportion should be respected in allclusters.
Several efﬁcient algorithms for discovering such clusters have been proposed [38, 3, 24],though they almost exclusively focus on variants of k -means while we are interested in spectralclustering.
Kleindessner et al.
[27] deserve a special mention as they develop a spectral clusteringalgorithm for this fairness notion.
However, we recover all the results presented in Kleindessneret al.
[27] as a special case of our analysis as our proposed constraint interpolates between populationlevel and individual level fairness based on the structure of R .
While individual fairness notionsfor clustering have also been explored [12, 34, 4, 11], none of them have previously been used withspectral clustering.
See Caton and Haas [10] for a broader discussion on fairness.
A ﬁnal line of relevant work concerns consistency results for variants of unconstrained spectralclustering.
von Luxburg et al.
[41] established the weak consistency of spectral clustering assumingthat the similarity graph G encodes cosine similarity between examples using feature vectors drawnfrom a particular probability distribution.
Rohe et al.
[36] and Lei and Rinaldo [29] assume that G issampled from variants of the Stochastic Block Model (SBM) [25].
Zhang et al.
[50] allow clusters tooverlap.
Binkiewicz et al.
[9] consider auxiliary node attributes, though, unlike us, their aim is toﬁnd clusters that are well aligned with these attributes.
A faster variant of spectral clustering wasanalyzed by Tremblay et al.
Spectral clustering has also been studied on other types of graphssuch as hypergraphs [22, 23] and strong consistency guarantees are also known [21, 30, 42], albeit under stronger assumptions.
Notation: Deﬁne [ n ] := { 1 , 2 , . . . , n } for any integer n .
Let G =
( V , E ) denote a similarity graph,where V = { v 1 , v 2 , . .
. , v N } is the set of N nodes and E
⊆
V × V is the set of edges.
Clusteringaims to partition the nodes in G into K ≥ 2 non-overlapping clusters C 1 , . .
, C K ⊆ V .
We assumethe availability of another graph, called a representation graph R =
( V , ˆ E ) , which is deﬁned on thesame set of vertices as G but with different edges ˆ E .
The discovered clusters C 1 , . .
, C K are requiredto satisfy a fairness constraint encoded by R , as described in Section 3.1.
A , R ∈ { 0 , 1 } N × N denotethe adjacency matrices of graphs G and R , respectively.
We assume that G and R are undirected andthat G has no self-loops.
Unnormalized spectral clustering
We begin with a brief review of unnormalized spectral clustering which will be useful in describingour algorithm in Section 3.2.
The normalized variants of traditional spectral clustering and ouralgorithm have been deferred to Appendix B. Given a similarity graph G , unnormalized spectralclustering ﬁnds clusters by approximately minimizing the following metric known as ratio-cut [
[Equation]
[Equation]
Here, V\C i is the set difference between V and C i .
For any two subsets X , Y ⊆ V , Cut( X , Y ) =
12 (cid:80) v i ∈X ,v j ∈Y A ij counts the number of edges that have one endpoint in X and another in Y .
Let
D ∈ R N × N be a diagonal degree matrix where D ii =
(cid:80) Nj =
1 A ij for all i ∈ [ N ] .
It is easy to verifythat ratio-cut can be expressed in terms of the graph Laplacian L := D − A and a cluster membershipmatrix H ∈ R N × K as RCut( C 1 , . . .
, C K )
= trace { H (cid:124) LH } , where
[Equation]
Thus, to ﬁnd good clusters, one can minimize trace { H (cid:124) LH } over all H that have the form givenin (1).
However, the combinatorial nature of this constraint makes this problem NP-hard [44].
Unnormalized spectral clustering instead solves the following relaxed problem:
[Equation]
Note that H in (1) satisﬁes H (cid:124)
H
=
I .
The above relaxation is often referred to as the spectralrelaxation.
By Rayleigh-Ritz theorem [33, Section 5.2.2], the optimal matrix H ∗ is such that ithas u 1 , u 2 , . .
, u K ∈ R N as its columns, where u i is the eigenvector corresponding to the i thsmallest eigenvalue of L for all i ∈
[ K ] .
The algorithm clusters the rows of H ∗ into K clusters usingk -means clustering [32] to
ˆ
Representation constraint and representation-aware spectral clustering
we ﬁrst our individual level fairness constraint Section Unnormalized Representation- Spectral
Section
3.1 Representation constraint
A representation graph R connects nodes that represent each other based on sensitive attributes (e.g.political opinions).
Let N R ( i )
= { v j : R ij = 1 } be the set of neighbors of node v i in R .
The sizeof N R ( i ) ∩ C k speciﬁes node v i ’s representation in cluster C k .
To motivate our constraint, considerthe following notion of balance
ρ i of clusters deﬁned from the perspective of a particular node v i :
[Equation]
It is easy to see that 0 ≤ ρ i ≤ 1 and higher values of ρ i indicate that node v
i has an adequaterepresentation in all clusters.
Thus, one objective could be to ﬁnd clusters C 1 , . . .
, C K that solve thefollowing optimization problem.
[Equation]
where f ( · ) is inversely proportional to the quality of clusters (such as RCut ) and α ∈ [0 , 1] is auser speciﬁed threshold.
However, it is not clear how this approach can be combined with spectralclustering to develop a consistent algorithm.
We take a different approach described below.
First, note
that min i ∈ [ N ] ρ i ≤ min k,(cid:96) ∈
[ K ] |C k | |C (cid:96) | .
Therefore, the balance ρ i of the least balanced nodev i is maximized when its representatives N R
( i ) are split across clusters C 1 , . .
, C K in proportion totheir sizes.
Representation constraint requires this condition to be satisﬁed for each node in the graph.
Deﬁnition 3.1 (Representation constraint) .
Given a representation graph R , clusters C 1 , . .
, C K in Gsatisfy the representation constraint if |C k ∩ N R ( i ) | ∝ |C k | for all i ∈
[ N ] and k ∈
[ K ] , i.e.,
[Equation]
In other words, the representation constraint requires the representatives of any given node to have aproportional membership in all clusters.
For example, if v i is connected to 30% of all nodes in R ,then it must have 30% representation in all clusters discovered in G .
It is important to note that thisconstraint applies at the level of individual nodes unlike population level constraints [13].
While (4) can always be solved for a small enough value of α (with the convention that 0 /
= 1 ), theconstraint in Deﬁnition 3.1 may not always be feasible.
For example, (5) can never be satisﬁed ifa node has only two representatives (i.e., |N R ( i ) | = 2 ) and there are K > 2 clusters.
However, asexactly satisfying constraints in clustering problems is often NP-hard [15], most approaches look forapproximate solutions.
In the same spirit, our algorithms use spectral relaxation to approximatelysatisfy (5), ensuring their wide applicability even when exact satisfaction is impossible.
In practice, R can be obtained by computing similarity between nodes based on one or more sensitiveattributes (say by taking k -nearest neighbors).
These attributes can have different types as opposedto existing notions that expect categorical attributes (Appendix A).
Moreover, once R has beencalculated, the values of sensitive attributes need not be exposed to the algorithm, thus adding privacy.
Appendix A presents a toy example to demonstrate the utility of individual level fairness and showsthat (5) recovers the population level constraint from Chierichetti et al.
[13] and Kleindessner et al.[27] for particular conﬁgurations of R , thus recovering all results from Kleindessner et al.
[27] as aspecial case of our analysis.
Finally, while individual fairness notions have conventionally required similar individuals to betreated similarly [17], our constraint requires similar individuals (neighbors in R ) to be spreadacross different clusters (Deﬁnition 3.1).
This new type of individual fairness constraint may be ofindependent interest to the community.
Next, we describe one of the proposed algorithms.
3.2 Unnormalized representation-aware spectral clustering (UR EP SC)
The lemma below identiﬁes a sufﬁcient condition that implies the representation constraint and canbe added to the optimization problem (2) solved by spectral clustering.
See Appendix E for the proof.
Lemma 3.1.
Let H ∈ R N × K have the form speciﬁed in (1) .
The condition
[Equation]
implies that the corresponding clusters C 1 , . .
, C K satisfy the constraint in (5) .
Here, I is the N × Nidentity matrix and 1 is a N -dimensional all-ones vector.
With the unnormalized graph Laplacian L deﬁned in Section 2, we add the condition from Lemma 3.1to
the optimization problem after spectral relaxation in (2) and solve
[Equation]
[Equation] Clearly, the columns of any feasible H must belong to the null space of R ( I − 11 (cid:124) /N ) .
Thus, anyfeasible H can be expressed as H =
YZ for some matrix Z ∈ R N − r × K , where Y ∈ R N × N − ris an orthonormal matrix containing the basis vectors for the null space of R ( I − 11 (cid:124) /N ) as itscolumns.
Here, r is the rank of R ( I − 11 (cid:124) /N ) .
Because Y (cid:124) Y =
I , H (cid:124) H
=
Z (cid:124) Y (cid:124)
YZ =
Z (cid:124) Z .
Thus, H (cid:124) H =
I ⇔ Z (cid:124)
Z
= I .
The following problem is equivalent to (7) by setting H =
YZ .
[Equation]
As in standard spectral clustering, the solution to (8) is given by the K leading eigenvectors of
Y (cid:124) LY .
Of course, for K eigenvectors to exist, N − r must be at least K as Y (cid:124) LY has dimensions
N − r × N − r .
The clusters can then be recovered by using k -means clustering to cluster the rows of
H =
YZ , as in Algorithm 1.
Algorithm 2 summarizes this procedure.
We refer to this algorithm asunnormalized representation-aware spectral clustering (UR EP SC).
We make three important remarksbefore proceeding with the theoretical analysis.
Remark 1 (Spectral relaxation) .
As R ( I − 11 (cid:124) /N )
H = 0 implies the satisfaction of the representationconstraint only when H has the form given in (1), a feasible solution to (7) may not necessarilyresult in representation-aware clusters.
In fact, even in the unconstrained case, there are no generalguarantees that bound the difference between the optimal solution of (2) and the original NP-hardratio-cut problem [27].
Thus, the representation-aware nature of the clusters discovered by solving(8) cannot be guaranteed in general (as is the case with [27]).
Nonetheless, we show in Section 4 thatthe discovered clusters indeed satisfy the constraint under certain additional assumptions.
Remark 2 (Computational complexity) .
Algorithm 2 has a time complexity of O ( N 3 ) and spacecomplexity of O ( N 2 ) .
Finding the null space of R ( I − 11 (cid:124) /N ) to calculate Y and computingthe eigenvectors of appropriate matrices are the computationally dominant steps.
This matchesthe worst-case complexity of Algorithm 1.
For small K , several approximations can reduce thiscomplexity, but most such techniques require K = 2 [49, 47].
Remark 3 (Approximate UR EP SC) .
Algorithm 2 requires rank { R } ≤ N − K to ensure the existenceof K orthonormal eigenvectors of Y (cid:124) LY .
When a graph R violates this assumption, we instead usethe best rank R approximation of its adjacency matrix R ( R ≤ N − K ) and refer to this algorithm as
UR EP SC ( APPROX .).
This approximation of R need not have binary elements, but it works wellin practice (Section 5).
Appendix C provides more intuition behind this low rank approximation,contrasts this strategy with clustering R to recover latent sensitive groups that can be reused withexisting population level notions, and highlights the challenges associated with ﬁnding theoreticalguarantees for UR EP SC ( APPROX .), which is an interesting direction for future work.
Analysis
This section shows that Algorithms 2 and 4 (see Appendix B) recover ground truth clusters withhigh probability under certain assumptions on the representation graph.
We begin by introducing therepresentation-aware planted partition model in Section 4.1.
4.1 R -PP model
The well known Planted Partition random graph model independently connects two nodes in V withprobability p if they belong to the same cluster and q otherwise, where the ground truth clustermemberships are speciﬁed by a function π : V → [ K ] .
Below, we deﬁne a variant of this modelwith respect to a representation graph R and refer to it as the Representation-Aware (or Fair) Planted
Partition model or R -PP.
Deﬁnition 4.1 ( R -PP) .
A R -PP is deﬁned by the tuple ( π, R , p, q, r, s ) , where π : V → [ K ] mapsnodes in V to clusters, R is a representation graph, and 1 ≥ p ≥ q ≥ r ≥ s ≥ 0 are probabilitiesused for sampling edges.
Under this model, for all i > j ,
[Equation]
Similarity graphs G sampled from R -PP have two interesting properties: (i) Everything else beingequal, nodes have a higher tendency of connecting with other nodes in the same cluster ( p ≥ q andr ≥ s ); and (ii) Nodes connected in R have a higher probability of connecting in G ( p ≥ r andq ≥ s ).
Thus, R -PP plants both the clusters in π and the properties of R into the sampled graph G .
Remark 4 ( R -PP and “hard” problem instances) .
Clusters satisfying (5) must proportionally distributethe nodes connected in R amongst themselves.
However, R -PP makes nodes connected in R morelikely to connect in G , even if they belong to different clusters ( q ≥ r ).
In this sense, graphs sampledfrom R -PP are “hard” instances for our algorithms.
When R itself has latent groups, there are two natural ways to cluster the nodes: (i) Based on theclusters speciﬁed by π ; and (ii)
Based on the clusters in R .
The clusters based on option (ii) are likelyto not satisfy (5) as tightly connected nodes in R will be assigned to the same cluster.
We show inthe next section that, under certain assumptions, π can be deﬁned so that the clusters encoded by itsatisfy (5) by construction.
Recovering these ground truth clusters (instead of other natural choiceslike option (ii) ) then amounts to recovering representation-aware clusters.
4.2 Consistency results
As noted in Section 3.1, some representation graphs lead to constraints that cannot be satisﬁed.
Forour theoretical analysis, we restrict our focus to a case where the constraint in (5) is feasible.
Towardsthis end, an additional assumption on R is required.
Assumption 4.1.
R is a d -regular graph for K ≤ d ≤ N .
Moreover, R ii = 1 for all
i ∈
[ N ] andeach node in R is connected to d/K nodes from cluster C j for all j ∈ [ K ] (including the self-loop).
Assumption 4.1 ensures the existence of a π for which the ground-truth clusters satisfy (5).
Namely,assuming equal-sized clusters, set π ( v i )
=
k if ( k − 1) NK ≤ i ≤ k NK for all i ∈
[ N ] and k ∈
[ K ] .
Before presenting our main results, we need additional notation.
Let Θ ∈ { 0 , 1 } N × K indicate theground-truth cluster memberships encoded by π (i.e., Θ ij = 1 ⇔ v i ∈ C j ) and ˆ Θ ∈ { 0 , 1 } N × Kindicate the clusters returned by the algorithm ( ˆΘ ij = 1 ⇔ v
i ∈
ˆ C j ).
With J as the set ofall K × K permutation matrices, the fraction of misclustered nodes is deﬁned as M ( Θ , ˆ Θ ) =
min J ∈J 1 N || Θ − ˆ ΘJ || 0 [29].
Theorems 4.1 and 4.2 use the eigenvalues of the Laplacian matrixin the expected case, deﬁned as L = D − A , where A = E
[ A ] is the expected adjacency matrixof a graph sampled from R -PP and D ∈ R N × N is its corresponding degree matrix.
The next tworesults establish high-probability upper bounds on the fraction of misclustered nodes for UR EP SCand NR EP SC (see Appendix B) for similarity graphs G sampled from R -PP.
Theorem 4.1 (Error bound for UR EP SC) .
Let rank { R } ≤ N − K and assume that all clusters have equal sizes.
Let µ 1 ≤ µ 2 ≤ · · · ≤ µ N − r denote the eigenvalues of Y (cid:124)
L Y , where Y was deﬁned in Section 3.2.
Deﬁne γ = µ K +1 − µ K .
Under Assumption 4.1, there exists a universal constant const( C, α ) , such that if γ satisﬁes γ 2 ≥ const( C, α )(2 + (cid:15) ) pNK ln N and p ≥ C ln N/N for some C > 0 , then
[Equation]
for every (cid:15) > 0 with probability at least 1 − 2 N − α when a (1 + (cid:15) )
-approximate algorithm fork -means clustering is used in Step 5 of Algorithm 2.
Theorem 4.2 (Error bound for NR EP SC) .
Let rank { R } ≤ N − K and assume that all clustershave equal sizes.
Let µ 1 ≤ µ 2 ≤ · · · ≤ µ N − r denote the eigenvalues of Q − 1 Y (cid:124)
L Y Q − 1 , where
Q = √ Y (cid:124)
D Y and Y was deﬁned in Section 3.2.
Deﬁne γ = µ K +1 − µ K and λ 1 = qd + s ( N −d )
+ ( p − q ) dK + ( r − s ) N − dK .
Under Assumption 4.1, there are universal constants const 1 ( C, α ) ,const 4 ( C, α ) , and const 5 ( C, α ) such that if:
[Equation]
[Equation]
and
[Equation]
and p ≥ C ln N/N for some C > 0 , then,
[Equation]
for every (cid:15) > 0 with probability at least 1 − 2 N − α when a (1 + (cid:15) )
-approximate algorithm fork -means clustering is used in Step 6 of Algorithm 4.
All proofs have been deferred to Appendix D. Brieﬂy, we show that the top K eigenvectors of L(i) recover ground-truth clusters in the expected case (Lemmas D.1 to D.3) and (ii) lie in the nullspace of R ( I − 11 (cid:124) /N )
and hence are also the top K eigenvectors of Y (cid:124)
L Y (Lemma D.4).
Matrixperturbation arguments then establish a high probability mistake bound in the general case when thegraph G is sampled from a R -PP (Lemmas D.5–D.8).
Next, we discuss our assumptions and use theerror bounds above to establish the weak consistency of our algorithms.
4.3 Discussion
Note that I − 11 (cid:124) /N is a projection matrix and 1 is its eigenvector with eigenvalue 0 .
Any vectororthogonal to 1 is an eigenvector with eigenvalue 1 .
Thus, rank { I − 11 (cid:124) /N } =
N − 1 .
Becauserank { R ( I − 11 (cid:124) /N ) } ≤ min(rank { R } , rank { I − 11 (cid:124) /N } ) , requiring rank { R } ≤ N − K
ensuresthat rank { R ( I − 11 (cid:124) /N ) } ≤ N − K , which is necessary for (8) to have a solution.
The assumption on the size of the clusters and the d -regularity of R allows us to compute the smallest K eigenvalues of the Laplacian matrix in the expected case.
This is a crucial step in our proof.
Theadditional assumptions in Theorem 4.2 are easy to satisfy as λ 1 scales linearly with N for appropriatevalues of p, q, r, and s .
Similar assumptions also appear in Kleindessner et al.
Remark 5 .
In practice, Algorithms 2 and 4 only require the rank assumption on R to ensure thefeasibility of the corresponding optimization problems.
The assumptions on the size of clusters andd -regularity of R are only needed for our theoretical analysis.
The next two corollaries establish the weak consistency of our algorithms as a direct consequence of
Theorems 4.1 and 4.2.
Corollary 4.1 (Weak consistency of UR EP SC) .
Under the same setup as Theorem 4.1, for UR EP SC ,
M ( Θ , ˆ Θ ) =
o (1) with probability 1 − o (1) if γ = ω ( √ pNK ln N ) .
Corollary 4.2 (Weak consistency of NR EP SC) .
Under the same setup as Theorem 4.2, for NR EP SC ,
M ( Θ , ˆ Θ ) =
o (1) with probability 1 − o (1) if γ = ω ( √ pNK ln N/ ( λ 1 − p )) .
The conditions on γ are satisﬁed in many interesting cases.
For example, when there are P protectedgroups as in Chierichetti et al.
[13], the equivalent representation graph has P cliques that are notconnected to each other (see Appendix A).
Kleindessner et al.
[27] show that γ = θ ( N/K ) in thiscase (for the unnormalized variant), which satisﬁes the criterion given above if K is not too large.
Finally, Theorems 4.1 and 4.2 require a (1 + (cid:15) ) -approximate solution to k -means clustering.
Severalefﬁcient algorithms have been proposed in the literature for this task [28, 5, 2].
Such algorithms are [Figure] also available in commonly used software packages like MATLAB and scikit-learn.
The assumptionthat p ≥ C ln N/N controls the sparsity of the graph and is required in the consistency proofs forstandard spectral clustering as well [29].
Numerical results
We experiment with three types of graphs: synthetically generated d -regular and non- d -regularrepresentation graphs and a real-world dataset.
See Appendix F for analogous results for NR EP SC,the normalized variant of our algorithm.
Before proceeding further, we make an important remark.
Remark 6 (Comparison with Kleindessner et al.
We refer to the unnormalized variant of thealgorithm in Kleindessner et al.
[27] as UF AIR SC.
It assumes that each node belongs to one of the Pprotected groups P 1 , . . .
, P P ⊆ V that are observed by the learner.
UR EP SC recovers UF AIR SC asa special case when R is block diagonal (Appendix A).
To demonstrate the generality of UR EP SC,we only experiment with R ’s that are not block diagonal.
As UF AIR SC is not directly applicable inthis setting, to compare with it, we approximate the protected groups by clustering the nodes in Rusing standard spectral clustering.
Each discovered cluster is then treated as a protected group (alsosee Appendix C).d -regular representation graphs: We sampled similarity graphs from R -PP using p = 0 . 4 ,q = 0 .
3 , r = 0 .
2 , and s = 0 .
1 for various values of d , N , and K , while ensuring that the underlying
R satisﬁes Assumption 4.1 and rank { R } ≤ N − k .
Figure 1 compares UR EP SC with unnormalizedspectral clustering (USC) (Algorithm 1) and UF AIR SC.
Figure 1a shows the effect of varying N fora ﬁxed d = 40 and K = 5 .
Figure 1b varies K and keeps N = 1200 and d = 40 ﬁxed.
Similarly,
Figure 1c keeps N = 1200 and K = 5 ﬁxed and varies d .
In all cases, we use R = P = N/ 10 .
Theﬁgures plot the accuracy on y -axis and report the mean and standard deviation across 10 independentexecutions.
As the ground truth clusters satisfy Deﬁnition 3.1 by construction, a high accuracyimplies that the algorithm returns representation-aware clusters.
In Figure 1a, it appears that even
USC will return representation-aware clusters for a large enough graph.
However, this is not true ifthe number of clusters increases with N (Figure 1b), as is common in practice.
Representation graphs sampled from planted partition model: We divide the nodes into P = 5protected groups and sample a representation graph R using a (traditional) planted partition modelwith within (resp.
across) protected group(s) connection probability given by p in = 0 .
8 (resp.p out = 0 . 2 ).
Conditioned on R , we then sample similarity graphs from R -PP using the sameparameters as before.
We only experiment with UR EP SC ( APPROX .)
as an R generated this way mayviolate the rank assumption.
Moreover, because such an R may not be d -regular, high accuracy nolonger implies representation awareness, and we instead report the ratio of average individual balance¯ ρ = 1 N (cid:80) Ni
=
ρ i (see (3)) to the value of RCut in Figure 2.
Higher values indicate high qualityclusters that are also balanced from the perspective of individuals.
Figure 2 shows a trade-off betweenaccuracy and representation awareness.
One can choose an appropriate value of R in UR EP SC( APPROX .)
to get good quality clusters with a high balance.
[Figure] A v e r a g e B a l a n c e / R a t i o - C u t
[Figure]
[Figure]
B a l a n c e /
R a t i o - C u t
[Figure]
A real-world network: The FAO trade network from United Nations is a multiplex network with214 nodes representing countries and 364 layers representing commodities like coffee and barley[16].
Edges corresponding to the volume of trade between countries.
We connect each node toits ﬁve nearest neighbors in each layer and make the edges undirected.
The ﬁrst 182 layers areaggregated to construct R (connecting nodes that are connected in at least one layer) and the next182 layers are used for constructing G .
Note that R constructed this way is not d -regular.
To motivatethis experiment further, note that clusters based only on G only consider the trade of commodities183 – 364 .
However, countries also have other trade relations in R , leading to shared economicinterests.
If the members of each cluster jointly formulate their economic policies, countries have anincentive to inﬂuence the economic policies of all clusters by having their representatives in them.
As before, we use the low-rank approximation for the representation graph in UR EP SC ( APPROX .).
Figure 3 compares UR EP SC ( APPROX .) with UF AIR SC and has the same semantics as Figure 2.
Different plots in Figure 3 correspond to different choices of K .
UR EP SC ( APPROX .)
achieves ahigher ratio of average balance to ratio-cut.
In practice, a user would choose R by assessing therelative importance of a quality metric like ratio-cut and representation metric like average balance.
Appendix F contains results for NR EP SC and a few additional experiments related to UR EP SC.
Conclusion
We studied the consistency of constrained spectral clustering under a new individual level fairnessconstraint, called the representation constraint, using a novel R -PP random graph model.
Our worknaturally generalizes existing population level constraints [13] and associated spectral algorithms [27].
Four important avenues for future work include (i) the relaxation of the d -regularity assumption in ouranalysis (needed to ensure representation awareness of ground-truth clusters), (ii) better theoreticalunderstanding of UR EP SC ( APPROX .)
, (iii) improvement of the computational complexity of ouralgorithms, and (iv) exploring relaxed variants of our constraint and other (possibly non-spectral)algorithms for ﬁnding representation-aware clusters under such a relaxed constraint.
Checklist
For all authors...
(a Do the main claims made in the abstract and introduction accurately reﬂect the paper’scontributions and scope?
Yes ] See Section 3.1 for the new fairness constraint, Section 3.2 and appendix B.2 for the algorithms, Section 4.1 for the new random graphmodel , Sections
2 and .3 for consistency results and Section and appendix F
for
numerical results.
(b) Did you describe the limitations of your work?
[Yes] See Remarks 1 to 3 where
we discuss issues with spectral relaxation, computational complexity, and lack
oftheoretical guarantees for the approximate variant of our algorithms
Also see Section 6.
(c) Did you discuss any potential negative societal impacts of your work?
[No] Our
algorithms are intended to learn more “fair” clusters.
While one can argue
about thedeﬁnition of fairness
itself, within
the context of this work, we do not see a negative
societal impact if the algorithms are used as intended
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Assump- tion 4.1 and Theorems 4.1 and 4.2.
(b) Did you include complete proofs of all theoretical results?
[Yes] See Appendix D
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] Code submitted
as supplemental material.
(b) Did you specify all the training details (e g., data splits, hyperparameters,
how they
were chosen)?
[Yes] See the description of experiments in Section 5.
(c) Did you report error bars (e.g., with respect to the random seed after running exper-
iments multiple times)?
[Yes] A careful look at the plots shows that error bars are
present, though the variation is very small.
(d) Did you include the total amount of compute and the type of resources used (e g., type of GPUs, internal cluster, or cloud provider)?
[No] The experiments were run on a standard desktop and did not require any specialized hardware.
If you are using existing assets e.g , code data models or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] We have added an
appropriate reference for the FAO Trade Network.
No code from outside sources was
used.
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
[Table]
using/curating?
[No] FAO Trade Network is a publicly available dataset.
[Table]
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable
[Table]
information or offensive content?
[N/A]
[Table]
If you used crowdsourcing or conducted research with human subjects...
[Table]
Did you include the full text of instructions given to participants and screenshots, if
[Table]
you describe any
participant , with to Institutional Review [Table]
Board approvals,
[Table]
Did you include the estimated hourly wage paid to participants and the total amount
[Table]
spent on participant compensation
[N/ [Table]