Optimal Transport of Classifiers to Fairness
Anonymous Author(s)
AffiliationAddress email
Abstract
In past work on fairness in machine learning, the focus has been on forcing theprediction of classifiers to have similar statistical properties for people of differentdemographics.
To reduce the violation of these properties, fairness methods usuallysimply rescale the classifier scores, ignoring similarities and dissimilarities betweenmembers of different groups.
Yet, we hypothesize that such information is relevantin quantifying the unfairness of a given classifier.
To validate this hypothesis,we introduce Optimal Transport to Fairness (OTF), a method that quantifies theviolation of fairness constraints as the smallest Optimal Transport cost betweena probabilistic classifier and any score function that satisfies these constraints.
For a flexible class of linear fairness constraints, we construct a practical wayto compute OTF as a differentiable fairness regularizer that can be added to anystandard classification setting.
Experiments show that OTF can be used to achievean improved trade-off between predictive power and fairness.
Introduction
Machine learning methods are increasingly being deployed for automated decision making due tothe potential benefits in terms of efficiency and effectiveness.
Yet, making automated decisionsabout people comes with substantial legal and ethical risks, as evidenced by more and more cases ofundesirable algorithmic discrimination [27, 13, 10] with respect to the protected traits of individuals.
The research field of fairness in machine learning [29] therefore studies ways in which a model’sdiscrimination with respect to a person’s sensitive features can be reduced or removed.
Many notionsof fairness have been proposed for binary classification tasks [36].
A popular example is demographicparity , which is expressed as a constraint that enforces equality between the rates at which positivedecisions are made for each protected group [6].
Motivation For a given fairness notion, we would like to tune a model that (to some extent) satisfiesthe constraint that expresses the notion.
To this end, it can be practical to quantify the violation ofthis constraint as a fairness regularization term that can be added to any probabilistic classifier [26].
Such an approach provides an incentive to the model to learn parameters that result in fair probabilityscores, e.g. by learning to ignore highly biased features.
A straightforward fairness regularizer is thenorm of the difference between the quantities that should be equal according to the fairness notion[39, 30].
However, such a quantification of unfairness may be limiting, as it only considers the statisticalproperties of the model’s output, while ignoring the input features for which those scores werecomputed are ignored.
We argue that, to quantify the unfairness of a model as a fairness regularizer, it can be beneficial to more strongly consider data points with similar features but significant unfairdiscrepancies in probability scores.
To this end, we employ Optimal Transport (OT) [32] as a way tomeasure these discrepancies while taking feature similarity into account.
Contributions
• We propose to quantify the unfairness of a probabilistic classifier as the Optimal Transportto Fairness (OTF) cost, which is defined as the smallest OT cost between the score functionof the classifier and any fair score function over the same data. • We make novel derivations to compute OTF as a differentiable fairness regularizer that canbe added to the training loss of any probabilistic classifier and is efficiently computed for
popular notions of fairness such as demographic parity and equalised odds .
Our approach iscapable of handling multiple sensitive variables, that can be categorical or continuous.
• In experiments, our method shows its benefit of increased flexibility over other OT methods.
In some cases, it also achieves a trade-off between predictive power and fairness that is
significantly more effective for equalized odds.
Background2.1 Fair Classification
Let Z ≜
X × S × { 0 , 1 } denote the sample space, from which we draw samples Z ≜ ( X, S, Y )and try to estimate the output label Y ∈ { 0 , 1 } from input features X ∈ X , without discriminatingwith respect to sensitive features S ∈ S .
It is assumed that X ⊂ R d X and S ⊂ R d S .We
samplebinary predictions ˆ Y ∈ { 0 , 1 } from a probabilistic classifier h : X → [0 , 1] that assigns a score h ( X )to the belief that a sample with features X belongs to the positive class.
Our general goal in fairclassification is to minimize a loss function L Y ( h ) , in our case the cross-entropy between h ( X ) and
Y , while also scoring well on a fairness measurement of h ( X ) .
2.1.1 Group Fairness
The fairness of classifiers is commonly enforced through some notion of independence from sensitiveinformation S ∈ S .
In particular, group fairness is concerned with constraints over statisticalmeasurements of this independence over all members of a protected group [29].
For now, assume that
S consists of a single, categorical feature with d S values.
We denote S ∈ S as the one-hot encoding 1values S k ∈ { 0 , 1 } .
This encoding is indexed by values from [ d S ]
= { 0 , ..., d S − 1 } .
Assuming that the predictions ˆ Y | X are randomly sampled from a probabilistic classifier h ( X ) , thenthe traditional fairness notion of demographic parity (DP) is equivalent to enforcing zero covariancebetween h ( X ) and each group in S , which we refer to as Probabilistic Demographic Parity (PDP):
[Equation]
Remark 1.
If ˆ Y is not sampled from h ( X ) but instead decided by a threshold (e.g. ˆ Y = 1 ifh ( X ) > 0 . 5 ), then Eq.
(1) is a relaxation of the actual DP notion and may therefore admit unfairmodels [28].
However, such a threshold introduces a discontinuity with respect to the input, making itdifficult to efficiently find fair classifiers in practice [35, 1].
Moreover, by considering the probabilitiesdirectly, we account for the uncertainty expressed by the probabilistic classifier.
The notation in Eq.
(1) is easily extended to fairness notions that condition on other variables.
Forexample, the equalized odds (EO) [21] fairness notion conditions DP on the actual label Y .
Weapply EO to probabilistic classifiers by conditioning the covariance constraint of PDP on the one-hotencoding of Y and retrieve the following expression for Probabilistic Equalized Odds (PEO):
[Equation]
[Table]
[Equation]
2.1.2 Linear Fairness
The PDP and PEO covariance constraints in Eq.
(1) and Eq.
(2) are practical, because they can bewritten as linear sums over the probabilities given by the classifier h .
As was done in past work [1],we are generally interested in all fairness notions that can be enforced through such linear constraints.
Definition 1 (Linear fairness notion) .
A notion of fairness is a linear fairness notion when the set Fof all score functions f : X → [0 , 1] that satisfy it is given by
[Equation]
with 0 d F a vector of d F zeros and the vector-valued function g ( Z ) : Z → R d F constant w.r.t.
f ( X ) .
PDP and PEO are clearly linear fairness notions, with their g function listed in Table 1.
Linearconstraints are practical because the set of fair score functions F is convex.
Moreover, multiple linearfairness notions can be combined into a new linear fairness notion by concatenating their constraintvectors.
Sensitive information that is inherently continuous, like age, can be considered as a singledimension in S , thereby allowing for a mix of categorical and continuous sensitive attributes.
However, under linear fairness, a non-linear dependence between a score function f ( X ) and continuous sensitive attributes S can remain undetected.
Also, some well-known fairness notions are notlinear.
For example, notions related to sufficiency involve conditioning on f ( X ) , meaning that f ( X )also shows up in the g function.
To this end, approaches have been proposed [9] that can approximateclassifiers with non-linear fairness through a reduction to problems with linear constraints.
2.2 Optimal Transport of Score Functions
Optimal Transport (OT) [31] theory considers the problem of moving a mass from one measure toanother at the smallest possible total cost.
Here, we let every score function f correspond with themeasure θ f defined over the input space
X endowed with the Borel σ -algebra: θ f ≜ (cid:80) x ∈D X f ( x ) δ x ,with δ x the Dirac measure and D X the input features of samples in a collection D .
Note that the inputspace measure θ f is not normalized, though this is not necessary to apply OT theory.
In what follows,we implicitly consider the score functions h and f as their corresponding input space measures θ hand θ f when used in the OT problem.
See the Appendix for further clarification.
For a collection D of n samples (e.g. a full dataset or only a batch from it), let h and f denote then -dimensional vectors of score function values for all data points,
i.e. h i
= h ( x i ) and f i =
f
( x i ) .
Furthermore, for a non-negative transport cost function c defined over X ×X , let C ∈ R n × n + representthe matrix of cost terms, i.e. C ij =
c ( x i , x j ) .
Similarly, with π ( x i , x j ) the coupling that reflects howmuch score mass was transported from x i to x j , define the matrix P ∈ R n × n + with P ij =
π ( x i , x j ) .
The OT cost is then simplified to
[Equation]
with
[Equation]
where 1 n is the n -dimensional vector of ones and ⟨ C , [Equation]
Optimal Transport to Fairness
Assume that a set F is available that denotes all score functions which satisfy the required notion offairness.
We would like to quantify the unfairness of a classifier’s score function h as the amount of ’work’ minimally required to make it a member of this set by measuring how far h is from itsfair projection onto F , i.e. the f ∈ F that is closest to h .
We measure this closeness as the OT costbetween h and f , as we then assign a higher unfairness cost to h if scores need to be transportedbetween highly dissimilar individuals in order to reach a fair function f ∈ F .
Thus, we propose to quantify unfairness as the Optimal Transport to Fairness (OTF) cost, i.e. the costof the OT-based projection of h onto F :
[Equation]
In what follows, we expand on the OTF method by constructing it in three steps.1.
In Sec. 3.1, we directly express the OTF ( h ) objective as a linear programming problem bymaking the assumption that F is defined by a linear fairness notion as in Def. 1.2.
In Sec. 3.2, we add entropic smoothing to OTF ( h ) , thereby making the resulting OTF ϵ ( h )cost a differentiable fairness regularizer that can be computed efficiently.3.
In Sec. 3.3, we address the fact that due to this smoothing, OTF ϵ ( h ) does not necessarilyequal zero for a fair h .
To this end, we subtract the adjustment term OTFR ϵ ( h ) , resulting inthe adjusted OTF
ϵ ( h ) cost.
Finally in Sec. 3.4, we show how OTF
ϵ ( h ) can be minimized w.r.t.
(the parameters of) h .
For all derivations and proofs, we refer to the Appendix.
3.1 Optimal Transport to Linear Fairness
To compute the minimization in Eq.
(5), first note that in the OT ( h, f ) cost in Eq.
(4), f only showsup in the constraint P T 1 n = f on the column marginals of coupling matrix P .
It therefore sufficesto weaken this constraint to P T 1 n ∈ F .
Next, recall that a linear fairness notion is enforcedthrough a vector of constraints on the expectation of g ( Z ) f
( X ) .
For a collection D of n samples,let G cj
=
g c ( z j ) , i.e. G ∈ R d F × n is the constraints matrix with a row for every constraint and acolumn for every data point.
It defines the linear fairness notion for D .
Definition 2 (OTF) .
For a linear fairness notion expressed through constraints matrix G for nsamples and with non-negative cost matrix C , the Optimal Transport to Fairness cost for scorefunction h :
X → [0 , 1] is computed as
[Equation]
with
[Equation]
The optimal coupling computed for OTF ( h ) implicitly transports the scores of h to the fair vectorf given the coupling’s column marginals (i.e. f = P T 1 n ).
Note that for any coupling matrix thatsatisfies the first constraint ( P1 n = h ), we have that h T 1 n = 1 Tn P T 1 n = f T 1 n , meaning that hand this implicitly found vector f sum to the same value.
Scores of h are therefore only transportedto a fair score function and no extra scores are created or destroyed.
For P ij ≥ 0 , the optimization problem in Eq.
(6) can be solved through linear programming.
However, its solution is not differentiable with respect to the score function h [12, 17].
It is thusimpractical as a fairness regularization term.
3.2 Entropic Smoothing
A well-known trick to make OT costs differentiable is to apply entropic regularization or smoothing[12, 32], which is done by maximizing the entropy of the coupling while optimizing the OT cost.
Weapply the same principle to make OTF differentiable.
Definition 3 (OTF ϵ ) .
For a linear fairness notion expressed through constraints matrix G for nsamples and with non-negative cost matrix C , the smooth OTF cost with smoothing strength ϵ > 0for score function h : X → (0 , 1] is computed as
[Equation]
) with H ( P )
= − (cid:80) ij P ij (log ( P ij ) − 1) the entropy of coupling P and ϵ > 0 a hyperparameter thatregulates the strength of entropic smoothing.
The use of an entropy term in Eq.
(7) requires some justification, since we do not assume that hrepresents a normalized probability distribution (i.e. sums to one).
Therefore, P also does notrepresent a normalized joint distribution.
However, the H ( P ) term still provides the practicaladvantage that it only admits couplings with P ij > 0 , which is why we require that h i > 0 .
Furthermore, the addition of H ( P ) makes the objective cost strongly convex.
3.2.1 Duality
The OTF problem for a linear fairness notion has n + d F constraints, which is usually far fewer innumber than the n 2 variables that make up the couplings in the primal problem.
Because OTF ϵ isalso strongly convex, we will derive and then maximize the dual function instead.
Proposition 1.
If G has a non-empty null-space (i.e. ∃ f ∈ R n : f ̸ = 0 n ∧ Gf = 0 d F ), then the
OTF ϵ ( h ) problem has a unique solution for any h : X → (0 , 1] .
Moreover, the OTF ϵ problem isstrongly convex and then enjoys strong duality.
Its dual function is
[Equation]
) with λ ∈ R n and µ ∈ R d F the dual variable vectors for the marginalization and fairness constraints.
3.2.2 Optimization
Though Eq.
(8) could be maximized directly, we follow standard OT approaches [32] and performour optimization with exact coordinate ascent.
This strategy is particularly useful here, because theλ i variable that maximizes L ( λ , µ ) while µ is fixed, is found independently of other variables in λ .
All variables in λ can therefore be updated at the same time:
[Equation]
9) where we can use the stabilized log-sum-exp operation.
Unfortunately, there is no closed form expression for the µ c that maximizes L ( λ , µ ) .
Instead, wepreprocess L ( λ , µ ) and numerically solve for each µ c :
[Equation]
with η j ( λ ) =
(cid:80) i exp (cid:0) 1 ϵ [ − C
ij
+ λ i ] (cid:1) .
3.2.3 Complexity
The dual problem involves two variable vectors: λ with length n the number of samples and µ withlength d F the number of linear fairness constraints.
It is commonly the case that d F ≪ n , becausethe number of distinguished protected groups is limited.
This is in contrast with the traditional OT ϵproblem, where the dual problem involves two dual variable vectors of length n .
In terms of computational complexity, the update of the full λ has complexity O ( n ( n + d F )) .
However, we stress that each λ i can be updated in parallel with complexity O ( n + d F ) .
When keeping the λ vector fixed, we can perform a O ( n 2 ) operation to precompute the η j ( λ ) valuesfor the µ updates.
Though these updates no closed form, the inner loop of the update for µ c onlyhas complexity complexity O ( n + d F ) for each inner loop, and should only be performed for d Fvariables.
By permitting a high tolerance on the convergence of λ , the necessary number of updatesis also limited.
In our experiments, we found that a single update is often enough.
We can achieve a memory complexity of O ( n + d F ) .
However, we can perform efficient matrixcomputations by storing the cost matrix C and the fairness constraints matrix G .
The memoryrequired for practical use is therefore O ( n ( n + d F )) .
3.3 Adjusted OTF ϵ
A cause for concern with the smooth OTF ϵ cost is that, as opposed to the standard OTF, it doesnot necessarily equal zero if h is already a fair score function.
Indeed, while it may be feasible toachieve ⟨ C , P ⟩ = 0 , it is generally the case that the entropy H ( P ) > 0 .
We would thus like to findan interesting lower bound on OTF ϵ that is tight when h ∈ F .
Definition 4 (OTFR ϵ ) .
The OTF ϵ cost in Def. 3 is relaxed to the OTFR ϵ cost as
[Equation]
11) with
[Equation]
Definition 5 (OTF 0 ϵ ) .
For a score function h : X → (0 , 1] , the adjusted OTF ϵ cost is computed as
[Equation]
Whereas the OTF ϵ ( h ) problem involved transport to fair score vectors ( GP T 1 n = 0 d F ), the
OTFR ϵ ( h ) problem relaxes this constraint by allowing unfairness up to the unfairness already presentin h .
It is thus easily seen that OTF ϵ ( h ) ≥ OTFR ϵ ( h ) .
Interestingly, because this relaxation isbounded by the unfairness of h , we have that OTFR ϵ ( h ) is a tight lower bound if h is already fair.
Proposition 2.
h ∈ F = ⇒ OTF
ϵ ( h ) = 0 .
In the Appendix, we provide empirical intuition for this theoretical result by using OTF
ϵ ( h ) as apostprocessing approach and illustrate that OTF ϵ ( h ) gradually converges to OTFR ϵ ( h ) as the fairnessof h improves.
The adjustment term OTFR ϵ can be computed by using strong duality in a similar way as OTF ϵ ( h ) ,though now with two dual variables ϕ ∈ R d F and ψ ∈ R d F that enforce the fairness inequality, inaddition to the dual variable vector, here denoted by κ ∈ R n , that enforces the row constraint on thecoupling.
The dual function of OTFR ϵ is
[Equation]
with ϕ c ≤ 0 , ψ c ≤ 0 and γ c = | (cid:80) j G cj h j | .
For a full discussion, we refer to the Appendix.
3.4 Minimising Unfairness
Given an efficient way to compute the parameters of the adjusted, smooth OTF
ϵ ( h ) cost, we can nowjointly minimize the weighted sum of the classifier’s loss and unfairness cost with respect to h .
Let L Y ( h ) denote the cross-entropy loss of h ( X ) for the output labels Y .
To add our OTF
ϵ ( h ) costto this objective, we can pose the following optimization problem:
[Equation]
(13) with 0 ≤ α ≤ 1 a hyperparameter that regulates the importance of the unfairness cost term OTF
ϵ ( h ) .
So far, we considered the scores vector of h as constant with respect to the OTF
ϵ ( h ) problem.
However, the solutions for OTF ϵ ( h ) and OTFR ϵ ( h ) clearly depend on h .
Let λ ∗ , µ ∗ , κ ∗ , ϕ ∗ and ψ ∗denote the fully converged dual variables.
Then
[Equation]
[Equation] Even if we did not update the dual variables until convergence, we can still use intermediate valuesto approximate the true gradient [18, 16].
We further approximate the true OTF
ϵ ( h ) cost by onlycomputing it for a subset of the dataset, e.g. when batching is already used to optimize L Y ( h ) .
The OTF
ϵ ( h ) can be optimized on a batch by only using the values of C and the columns of G that relateto these data points.
For large datasets (including those we use in experiments), it is then unnecessaryto precompute the C matrix between all pairs of data points, since most pairs will never actuallyco-occur in randomly sampled batches.
Related Work
Fairness in machine learning is commonly understood in two (extreme) ways [29].
First, the notionof individual fairness states that similar individuals should be treated similarly [15].
Second, groupfairness requires that as a group, people are treated equally.
The primary goal of our work is toachieve group fairness by measuring the distance of a score function to a set of group-fair functions.
Yet, to an extent we also approach individual fairness because our unfairness quantification takes thefeatures of each individual into account in its cost function c .
By appropriately choosing c , it may bepossible to further improve upon individual fairness in future work.
Methods that aim to improve fairness can be categorized into three types: those that performpreprocessing where the data is made more fair [25, 8], as opposed to those that do postprocessingwhere the model’s predictions are modified to fit a notion of fairness [21].
As a fairness regularizationterm, our work clearly fits in the class of inprocessing methods, i.e where the machine learning modelor its training is directly altered to reduce unfairness [7, 39, 38, 1, 9].
However, we highlight that the
OTF cost could also be used as a postprocessing approach by setting α = 1 .
Our work draws inspiration from two kinds of inprocessing methods.
First, with those methodswhere a (still unfair) model is explicitly projected unto the set of fair models.
A popular definitionof distance for this projection is the Kullback-Leibler divergence [2, 23, 5].
Second, our work uses
Optimal Transport (OT) [32] theory.
Several works have applied OT to fairness, though they almostall do so by finding barycenters [34]: measures that are ‘equally far’ from the measures belongingto each demographic.
This type of method has been applied to preprocessing [19], classification[23, 40] and regression [11, 20], with the overarching goal being to move each group’s measurecloser to the barycenter.
Such methods require fairness notions to be encoded in the format of explicitequality, between a limited amount of groups.
They do not allow for more complex constraints suchas those involving continuous or multiple sensitive attributes, and make strong assumption that thecost function is a distance metric.
Our proposed method instead combines the idea of projecting the model to a fairness-constrained setand the use of OT to measure the cost of this projection.
In this manner, our work is most similar to[33], which computes the OT projection onto such a fair set.
However, they only use the method as agroup fairness test.
Our method has the advantage that it computes a differentiable cost that can beadded to the loss of any probabilistic classifier.
Experiments
Our experiments were performed on two staple datasets from fair binary classification literature[14] with different kinds of sensitive information.
First, the UCI Adult Census Income dataset withtwo binary sensitive attributes SEX and RACE .
Second, the UCI Bank Marketing dataset, where wemeasure fairness with respect to the original continuous AGE values and the binary,
5.1 Methods
All experiments were performed using a logistic regression classifier.
To achieve fairness, we jointlyoptimize fairness regularizers with the cross-entropy loss as in Eq.
(13), and compute the gradient oftheir sum with respect to the parameters of the classifier.
The OTF cost is the adjusted version from Def. 5, with ϵ = 10 − 3 .
For the cost function c , we usethe Euclidean distance between non-protected features.
The fairness constraints for both sensitivefeatures in the Adult dataset were combined by concatenating their G matrices.
As a first baseline, we use the L 1 norm of the PDP and PEO fairness constraints and will refer to thismethod as the Norm cost.
This simple type of fairness regularizer is frequently proposed in literature[26, 39, 37, 30].
In our experiments, it thus represents the class of methods that can accommodate aflexible range of fairness constraints while only considering the output of a classifier and not the inputfeatures.
Our second baseline is an OT Barycenter 2 approach [24].
This method directly minimizesthe OT cost between the score distributions of two sensitive groups.
However, the implementationcan only minimize PDP, and does not admit a composition of linear fairness notions or continuousattributes.
We therefore only minimize unfairness with respect to SEX and AGE _ BINNED .
To show thedrawbacks of this limitation, we also report PDP unfairness with respect to RACE and the continuous
AGE .
The third baseline is Unfair , which was a classifier trained without a fairness cost term.
5.2 Evaluation
To measure the predictive power of the methods, the ROC AUC score was measured.
Violationof PDP is quantified as the absolute value of the maximal Pearson correlation between the scoresh ( X ) and each dimension of the sensitive attributes S k .
For PEO, the violation is computed as themaximal PDP for the data points of each one-hot encoded output label Y l .
These measures mirror therequirements in Eq.
(1) and Eq.
(2) that the covariances ought to equal zero, though we choose the
Pearson correlation instead such that the violation is normalized between 0 and 1.
Thus we avoid thata method may improve on linear fairness notions by squeezing the variance of its scores.
All methods were tested for various strengths of α (see Eq.
Each configuration of each method(i.e.
each α value and fairness notion) was tested for five train-test splits with proportions 80/20.
Wereport mean test set scores in Fig.
1 and show the confidence ellipse 3 of their estimator for the firststandard deviation.
The train set results are reported in the Appendix and show the same trends.
Note that for the OTF and Norm regularizers, we omit fairness scores for notions that were notoptimized in that configuration.
While SEX and RACE are minimized at the same time, the AGE and
AGE _ BINNED attributes were minimized in separate training runs.
All experiments were conducted using half the hyperthreads on an internal machine equipped with a12 Core Intel(R)
Xeon(R)
Gold processor and 256 GB of RAM.
In total, all experiments, includingpreliminary and failed experiments, used this machine for approximately 200 hours.
5.3 Results
The results in the Adult dataset in Fig.
1a show that our OTF method makes a similar trade-offbetween AUC and PDP fairness as the baselines for various α values.
However, OTF clearlyoutperforms Norm for both sensitive attributes when they are trained to minimize PEO, especially inthe lower violation range.
This advantage may be due to the fact that OTF has the incentive to achievefairness by exchanging classifier scores between similar individuals.
Conditioning on the output labelrewards such exchanges, as individuals with similar labels can be expected to have similar features.
Note that the Barycenter approach was not trained to improve fairness with respect to RACE , as theimplementation can not handle multiple sensitive attributes.
Our proposed OTF method thereforedisplays its advantage in both an effective AUC-fairness trade-off, and a flexible capacity for fairness.
We mean, which is smaller than that of individual scores
[Figure]
The test set results on the Bank dataset in Fig.
1b are clearly subject to a high amount of noise.
Still,we see that OTF and Norm can be used to effectively minimize either PDP or PEO with respect to thecontinuous AGE attribute.
Implementations of differentiable regularizers that can accommodate suchcontinuous attributes are scarcely available.
The Barycenter method can only minimize the OT costbetween a limited number of score distributions, meaning it is naturally ill-equipped for continuousattributes.
In separate runs, we therefore also trained OTF and Norm to minimize unfairness withrespect to AGE _ BINNED .
We then see similar results as for the Adult dataset: our method is on parwith the baselines for PDP, but clearly outperforms the Norm baseline for PEO.
5.4 Limitations and Impact
We nuance our results by pointing out that the OTF method has a higher computation complexity( O ( n ( n + d F )) ) than Norm, which has complexity O ( nd F ) with n the batch size and d F the numberof fairness constraints.
Furthermore, as discussed in Sec. 2.1, we only evaluate linear fairness notionsand leave an extension to non-linear notions of fairness [9] for future work.
The fairness of a system should always be judged based on a holistic consideration of the context ofthe system [3] and relevant ethical principles [4].
Without it, any method to improve fairness mayreinforce the underlying injustice that led to the risks of discrimination that we aim to solve [22].
Conclusion
In this paper, we constructed the Optimal Transport to Fairness (OTF) method as a differentiablefairness regularization term, which combines the advantages of flexible linear fairness constraints with
Optimal Transport theory, thereby taking non-protected similarities between individuals into account.
In experiments, we show that our final method achieves a similar AUC-fairness trade-off as othercost terms for notions of fairness inspired by demographic parity, and significantly better for notionsinspired by equalized odds.
At the same time, OTF empirically displays why its increased flexibility,i.e. its ability to improve all linear fairness notions with respect to multiple sensitive attributes thatmay be categorical or continuous, provides an advantage over previous fairness applications of OT.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes] In Sec. 3, we proposed the OTF cost term and deployedtricks to minimize it effectively.
In Sec. 5, we show that the results of using ourproposed method are close to the baselines or surpassing it.(b)
Did you describe the limitations of your work?
[Yes] We describe limitations of ourconsidered fairness notions in Sec.
2.1 and the complexity of our proposed OTF costterm in Sec.
We briefly discuss overall limitations of fairness methods in Sec. 5.4.(c)
Did you discuss any potential negative societal impacts of your work?
[Yes] We brieflydiscuss the danger that inconsiderately applied fairness methods can pose in Sec.
5.4(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]2.
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] Our method isconstructed in several steps.
When required, we discuss and add additional assumptions.(b)
Did you include complete proofs of all theoretical results?
[Yes] Proofs of all propositions and derivations of other results are included in the Appendix.3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main exper-
imental results (either in the supplemental material or as a URL)?
[Yes] There are references to the datasets we used in the Appendix.
Code and a pipeline to run it are
included in the supplemental material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[Yes] Initial training details are given in Sec.5.2.
More details of the
hyperparameters are included in the Appendix.
(c) Did you report error bars (e g., with respect to the random seed after running experi-
ments multiple times)?
[Yes] Since we show the trade-off between two goals (accuracy
and fairness)
we show the confidence ellipses.
(d) Did you include the total amount of compute and the type of resources used (e.g., type
of GPUs, internal cluster, or cloud provider)?
[Yes] This is mentioned at the end of
Sec.
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] For the Barycenter
code, a reference to the Github implementation is included.
For datasets, we refer to
the UCI repository.
(b) Did you mention the license of the assets?
[Yes] For the Barycenter code, we include
the license.
We refer to the data according to the requested citation policy.
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[No] The datasets we use are popular, public datasets in machine learning fairness literature.
Obtaining consent is infeasible.
However, the data is highly anonymized and its popularity promotes reproducibility in the community.
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[Yes] As briefly mentioned in the Appendix, the data
is anonymous and free from any personal identifiers.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]