On the Fairness ROAD: Robust Optimization for Adversarial Debiasing
Authors: Vincent Grari
Thibault Laugel
Tatsunori Hashimoto
sylvain lamprier
Marcin Detyniecki
Venue: ICLR 2024 poster (2024)
Keywords: Fairness
DRO
Adversarial Learning
Primary area: societal considerations including fairness, safety, privacy
TL;DR: Enforcing local fairness via robust optimization and adversarial debiasing.

Abstract: In the field of algorithmic fairness, significant attention has been put on group fairness criteria, such as Demographic Parity and Equalized Odds. Nevertheless, these objectives, measured as global averages,  have raised concerns about persistent local disparities between sensitive groups. In this work, we address the problem of local fairness, which ensures that the predictor is unbiased not only in terms of expectations over the whole population, but also within any subregion of the feature space, unknown at training time. To enforce this objective, we introduce ROAD, a novel approach that leverages the Distributionally Robust Optimization (DRO) framework  within a fair adversarial learning objective, where an adversary tries to infer the sensitive attribute from the predictions. Using an instance-level re-weighting strategy, ROAD is designed to prioritize inputs that are likely to be locally unfair, i.e. where the adversary faces the least difficulty in reconstructing the sensitive attribute. Numerical experiments demonstrate the effectiveness of our method: it achieves Pareto dominance with respect to local fairness and accuracy for a given global fairness level across three standard datasets, and also enhances fairness generalization under distribution shift.

Published as a conference paper at ICLR 2024
ON THE FAIRNESS ROAD: ROBUST OPTIMIZATION FOR ADVERSARIAL DEBIASING
Vincent Grari Thibault Laugel=:1.24 Tatsunori Hashimoto", Sylvain Lamprier', Marcin Detyniecki 1,4,5
1 AXA Group Operations
2 Stanford University
3 LERIA, Université d' Angers, France
4 TRAIL, Sorbonne Université, Paris, France
5 Polish Academy of Science IBS PAN, Warsaw, Poland
{grari, 1 edu
code: https: StEps://get /ttpt/gttttt.h.g..gggg www.coo//////.oe axa rev- rev-erree-eeearcc/ooooa fairness/
ABSTRACT
the field of algorithmic faimesss, significant attention has been put on group fair- ness criteria, such as Demographic Parity and Equalized Odds. Nevertheless, these objectives, measured as global averages, have raised concerns about persistent lo- cal disparities between sensitive groups. In this work, address the problem local fairness, which ensures that the predictor is unbiased not only in terms of expectations
the whole population, but also within any subregion of the feature space, unknown at training time. To enforce this objective, we introduce ROAD, novel approach that leverages the Distributionally Robust Optimization (DRO) framework within a fair adversarial learning objective, where an adversary tries predict the sensitive attribute from the predictions. Using an instance-level re-weighting strategy, ROAD is designed to prioritize inputs that are likely to be locally unfair, i.c. where the adversary faces the least difficulty in reconstructing the sensitive attribute. Numerical experiments demonstrate the effectiveness of our method: it achieves, for a given glohal fairness level, Pareto dominance with respect local fairness and accuracy across three standard datasets, as well enhances fairness generalization under distribution shift.
INTRODUCTION
The increasing adoption of machine learning models in various applications such as healthcare or criminal justice, has raised concerns about the fairness of algorithmic decision-making processes. As these models are often trained on historical data, they have been shown to unintentionally perpet- uate existing biases and discrimination against certain vulnerable group (Obermeyer et al. 2019). Addressing fairess in ML has thus become an essential aspect of developing ethical and equitable systems, with the overarching goal of ensuring that prediction models are not influenced by sensi- tive attributes. One of its most common concepts, group fairness, entails dividing the population into demographic-sensitee
groups (e.g., male and female) and ensuring that the outcomes of a decision model are equitable across these different groups, measured with criteria like Demographic Parity (DP) (Dwork et al., 2012) and Equal Opportunity (EO) (Hardt et al. 2016)..
However, focusing solely these group fairness criteria, along with predictive performance, has been increasingly questioned as an objective: besides being shown to poorly generalize to unseen, e.g. drifted, environments (Kamp al. 2021), it has been more generally criticized for being too simplistic (Selbst et al. 2019; Binns 2020). leading to arbitrariness in the bias mitigation pro- cess (Krco et al. (023) and the risk of having some people pay for others (Mittelstadt al., 2023). Recognizing these issues, some researchers have long focused on exploring more localized fair- ness behaviors, proposing to measure bias sectionally within predefined demographic categories, in which comparison between sensitive groups is deemed meaningful for the considered task. For instance, using Conditional Demographic Disparity (Zliobaite et al. 2011). fairness in predicted
"qqual contribution

---

Published as a conference paper at ICLR 2024
salaries between men and women shall be evaluated by comparing individuals within the same job category and seniority level, rather than making a global comparison across sensitive groups.
Nevertheless, predefining these comparable groups to optimize their local faimess is often diffi- cult: for instance, which jobs should be deemed legally comparable with one another? (Wachter al. 202T) In this paper, therefore propose address the difficult problem of enforcing fairness in local subgroups that are unknown at training time (Sec. 2 For this purpose, we leverage the Distributionally Robust Optimization (DRO) framework, initially proposed address worst-case subgroup accuracy (see e.g. Duchi & Namkoong (2021)) Our approach ROAD (Robust Optimiza- tion for Adversarial Debiasing, described SeaE) combines DRO with a fair adversarial learning framework, which aims to minimize the ability of an adversarial model to reconstruct the sensitive attribute. By boosting attention on feature regions where predictions are the most unfair in the sense of this sensitive reconstruction, ROAD is able to find the best compromise between local fairness, accuracy and global fairness. Such dynamic focus done by relying on weighting process that respects some locality smoothness in the input space, in order to mitigate bias in any implicit sub- group of the population without supervision. Experiments, described Section show the efficacy of the approach on various datasets.
PROBLEM STATEMENT
Throughout this document, address a conventional supervised classification problem, trained using examples (2,, s:) where each example is composed of feature vector I; E Rd containing d predictors, binary sensitive attribute S;, and a binary label yt- These examples are sampled from training distribution r (X, S) model f with parameters wy that minimizes the loss function (f(((x)) y) (e.g. log loss for binary classification), whilst adhering fairesss constraints based specific fairness definitions relying on the sensitive attribute S. In this section, we present the fairness notions and works that are necessary to ground our proposition.
Our goal is to construct predictive
2.1 GROUP FAIRNESS
One key aspect of algorithmic fairness is group fairness, which aims to ensure that the outcomes of a decision model are equitable across different demographic groups. In this paper, we focus on two of the most well-known group fairness criteria: Demographic Parity and Equalized Odds.
Demographic Parity: Demographic parity (DP) (Dwork al. 2012) is achieved when the proportion of positive outcomes is equal across all demographic groups. Using the notations above, the learning problem of a model f under demographic parity constraints can be expressed as follows:
arg min (x,,,))> Cy(fw, v) s.t. [ (x)ls 1]- 1p (z)ls O]l <E (I)
wa
Where of represents the output prediction after threshold (e.g., (x) (1)20.)). The pa-
rameter c represents the deviation permitted from perfect statistical parity, allowing for flexibility in balancing accuracy and fairness. In the following, this deviation is noted Disparate Impact (DI), representing the absolute difference in positive outcomes between the two demographic groups.
Although numerous methods exist solve the problem described in Equation [ focus in this work on the family of fair adversarial learning, which has been shown to be the most power- ful framework for settings where acting on the training process is an option (i.c., in-processing method) (Louppe et al., 2017 Wadsworth et al. 2018 Zhang et al. 20188 Grari 2022). One of the most well-known fair adversarial approaches by Zhang et (2018) framed as lollows:
min
E(z.g.))vp Ly(Jw, (x), v) mry
min ((....)))pLs( (Jmy (x)), >
(2)
Where Ls represents a loss for sensitive reconstruction (e.g. a log loss for binary sensitive at- tribute). In this adversarial formulation, the goal is to learn a model f that minimizes the tra- ditional loss of the predictor model, while simultancously ensuring that adversary 9 with pa- rameters wo cannot effectively distinguish between the two sensitive demographic groups based on the predictor's output Sw, (z). The faimess constraint is thus imposed here the adversary's
2

---

Published as a conference paper at ICLR 2024
Gldblly
0.303
0.30
0.25
indrvidusa
0.152
3023
0.22
400
0.05-
6.10-
6.02
0R00
0.30
(20. 20)
sol
qe
(30
ue
(50
(78
G20
(00
160
Ago
Age
Figure 1: Visualizing local fairnsss: DI values measured globally (dashed lines) and locally (full lines) for biased model (red) and one trained with DI objective (in blue) (Zhang et al. (2018) with A=3). The bar plots show the sensitive attribute distribution in each age group. Left: Adult dataset; Right: Compas.
ability to reconstruct the sensitive attribute, which should be limited, i.c. the value of the loss function (g(g,, (fmr (z)).. 8) should be above a minimum value e'. In practice, to achieve bal- ance between the predictor's and the adversary's performance, relaxed formulation of Fquation 2 used: mingy max., E(c.v.))~pp-y(f
(z). (..e..s)))ppss (fwy (z)), s). The coeff- cient E IR+ controls the trade-off between the predictor's 's performance on the task of predicting Y and the adversary's ' performance reconstructing the sensitive attribute. A larger value of À em- phasizes the importance of restricting the adversary's ss ability to reconstruct the sensitive attribute, while smaller value prioritizes the performance of the predictor the main task.
Equalized Odds: Equalized Odds (EO) (Hardt et al. 2016) is another group fairness criterion that requires the classifier have equal true positive rates (TPR) and false positive rates (FPR) across demographic groups This criterion is especially relevant when misclassification can have signifi- cant impacts on individuals from different groups. To achieve EO, Zhang et al. (2018) employs an adversarial learning approach by concatenating the true outcome Y to the input of the adversary.
2.2
THE LOCAL FAIRNESS PROBLEM
The global aspect of these group faimess criteria begs the question of the emergence of local un- desired behaviors: by enforeing constraints on global averages between sensitive groups, still expect that some local differences may persist (Krco et al., 2023). We illustrate this phenomenon through a simple experiment, shown in Fig. [ On two datasets, Adult and Compas (described in App. A.8.I. two models are trained: an unconstrained model solely optimizing for accuracy (called Biased, in red), and the adversarial model from Zhang et al. (2018) (in blue) optimizing for Demo- graphic Parity for the sensitive attributes gender (Adult) and race (Compas). For each model, two types of Disparate Impact (DI) values are shown: the global DI values, calculated over all the test set (dashed lines); and the local ones, calculated in subgroups of the population (full lines). The sub- groups are defined here as age categories: discretized bins of the continuous attribute age. Although local DI values are generally lower for the fair model, they vary lot across subgroups, sometimes remaining unexpectedly high. This is especially true for less populated segments (e.g., higher age values), and segments where the sensitive attribute distribution is extremely unbalanced: as the fair- ness constraint only concerns global averages, more attention is put on densely populated regions. On the other hand, less populated segments are more likely be ignored during the training.
These local differences echo the long-asserted claim that the blunt application of group fairness metries bears inherent inequalities through their failure to account for any additional context (Selbst et al. 2019 Binns 2020). Here, although reductive, the additional context we refer to is the infor- mation already available in the dataset X, in which comparable subgroups (Wachter et al.) 202T) can be drawn to evaluate fairness. This helps defining the notion of Local Fairness that is the focus of this paper: locally fair model thus guarantees minimal differences in expectations within these comparable subgroups of X. Contrary to works intersectional faimess (Kearns et al. 2018). the desired behavior in Fig. is thus not to treat age as sensitive attribute: predictions f(x) are expected to vary along age. However, in the Compas dataset for instance, equality between race groups is expected hold regardless of the age category considered. It is important note that the
3

---

Published as a conference paper at ICLR 2024
notion studied here is also different from the one of individual faimess, which aims treat similarly individuals who are close w.r.t some predefined similarity measure (see, e.g. Dwork et al. (2012) )
without any notion of sensitive data, rather than minimize DI among subgroups of individuals. In the
same vein of fairness without demographics, (Hashimoto et al., 2018 Duchi et al. )023) consider the case of unknown subgroups via the Distributionally Robust Optimization (DRO) framework.
While their goal is to train models that perform uniformly well across all partitions of the popula- tion, our goal is to train a model that uniformly fair (regarding a sentitive attribute) accross all subregions of the feature space, which is quite different.
Having knowledge of these subgroups at training time would mean that it could be included as an additional constraint in the learning objective, akin to the work of Zliobaite et al. (2011). The cri- terion they propose, Conditional Demographic Disparity, measures Demographic Disparity across user-defined subcategories. However, several issues make this difficult, if not impossible, in prac- tice. Besides that such expert knowledge is generally unavailable, or costly to acquire, the subgroups definitions might even be inconsistent across different testing environments (e.g. conflicting legal definitions of job categories or gender (Wachter et al.) (021)), making its optimization futile. Fur- thermore, exploring multiple categories is problematic in a combinatorial perspective. In this paper, we propose to optimize accuracy while adhering to a worst-case fairness constraint, an objective that was originally introduced to enhance fairness generalization capabilities in scenarios involving distribution drift or noisy labels (cf. Sec. [.3) We implicitly define the subpopulations of interest, for which we aim to optimize fairness, using distributions within an uncertainty set Q, and present the DRO framework for the Demographic Parity criterion follows:
B.g [ug (w)) = 1] fu, o]| <c (3)
min wy
T ((....))> fy(Swn (x)).) S.t. max 9E
The constraint ensures that the Disparate Impact remains less than predefined threshold under the worst-case distribution q E Q. Working with distribution q allows us to enforce local fairness by targeting subpopulations of interest, thus creating more focused and adaptable model that addresses fairness problems both globally and at a granular level.
2.3 RELATED WORK AND POSITIONING
Several works have proposed to address the objective in Eq 3 either to ensure better fairness gener-
alization capabilities in drift scenarios (Rezaei et al. 1021 Ferry et al. 2022 Wang al. 2023) or when facing noisy labels (Mandal et al. 2020 Wang et al. 2020 Roh et al. 2021). The uncertainty set Q then represents the perturbations that might affect the data at test time, and can therefore take several forms. While we expect Q contains the distribution of test data, leaving too much freedom to q may lead to trivial solutions that degenerate as uniform classifiers (Martinez et al.) 202T). To do so, the uncertainty set Q is commonly defined ball centered on p using distribution distances or similarities. Examples include maximal Total Variation distance (Wang et al. 2020), Wasserstein distance (Wang al., 1 or Jaccard index (Ferry et al. (022). From the fairness without demo- graphics literature (Duchi et al. 2023), it is known that the maximal allowed divergence is connected to the risk of the smallest component of the training distribution, seen mixture of distributions. This observation also holds for worst-case fairness using DRO, as defined in Eq
To the best of our knowledge, our work is the first to address the topic of local fairness with unknown subgroups. This different objective implies additional constraints on the set Q considered in Eq 3 Notably, under our local fairness objective, we also want that the discrepancies of q w.r.t. are smooth in the feature space, so that the fairness constraint does not increase mitigation on specific disconnected individuals, but rather on local areas of the space. This will guide the design of our approach in the next section.
Moreover, due to the discrete nature of the problem expressed in Eq 3 (the constraint is applied on f which is binary), most existing works restrict to linear models (Wang et al. 2020 Rezaci et al. 2020) Mandal et al.] 2020 Taskesen et al.) 2020), or rule-based systems (Ferry et al. 2022). This allows them to look for analytical solutions using linear programming. Although Rezaei et al. (202T) is an exception in this regard, they suffer from several drawbacks, namely requiring knowledge about the target distribution at train time and about the sensitive attribute at test time. Solving Equation using a wider class of models remains therefore, to the best of our knowledge, unexplored.
4

---

Published as a conference paper at ICLR 2024
3 ROAD: ROBUST OPTIMIZATION FOR ADVERSARIAL DEBIASING
3.1 FORMALIZATION
To overcome the limitations of previous works, we introduce our proposition to address the fairness generalization problem by combining adversarial optimization and the DRO framework. In order to learn predictor fw, that is fair both globally and for any subregion of the feature space, the idea is therefore to boost, at each optimization step, the importance of regions q for which the sensitive reconstruction is the easiest for an optimal adversary 9w,* given the current prediction outcomes. Rewriting the faireess constraint of Equation 3 with adversary gh,, 3 + S, we thus focus on the following problem for Demographic Parity
min "y
((x,0,)) Cy (y(fu(()),))
s.t.
min L [e.. (fw, (x)), 8) > é' gEQ
(4)
with Wig = arg min ( Ls (Sw, (r)),
Wig
A major challenge with this formulation is that exploring all possible distributions in Q is infeasible in the general sense. Worse, modeling distribution g directly over the whole feature space as support is very difficult, and usually highly inefficient, even for Q restricted to distributions close to p. This motivates an adversarial alternative, which relies on importance weighting of training samples from We therefore restrict Q to the set of distributions that are absolutely continuous with respect inspired by Michel et al. (2022) This allows us to write q = rp with X' S R+ function that acts as a weighting factor. Given a training set I sampled from p. we can thus reformulate the overall objective, by substituting g with rp and applying its Lagrangian relaxation, as optimization problem on r E R = (rlrp- E 2}:
to I
min max wy PE n
1 1 CF, (1:). >g n
Σ r(a; S:) (Sw, (s)
(5)
1 Σ Ls Cs w (fu, (z1)),
with w arg min 1
With Ay a regularization parameter controlling the trade-off between accuracy and faimess in the predictor model. In the following, we describe two constraints, inspired from the DRO literature, that we consider ensure g keeps the properties of a distribution and avoids pessimistic solutions.
Validity Constraint To ensure keeps the properties of distribution (i.e., E RR), previous works in DRO (e.g. Michel et al. (2022)) enforce the constraint ((,,,))pp"(..s)) optimization. In the context of local farmess using our adversarial formulation from 5g we argue that this constraint is not sufficient to ensure a safe behavior with regard to the fairness criterion, as it allows disturbances in the prior probabilities of the sensitive (i.e., g(s) * p(s)). As discussed more deeply in Appendix(AA22. this may lead to shift of the optimum of the problem, by inducing group.
1 during the
stronger mitigation emphasis on samples from the most populated demographic-sensicne To avoid this issue, propose to further constrain by considering restricted set R {r R. rp ②), with Q c Q such that: Vs, q(s) = p(s). To achieve this, we rely the following constraint: Vs, Ep(ars) r(zr s) = 1. Besides guaranteeing the desired property g(s) p(s) (proof in Sec. A.2.1J, we also note that ensuring these constraints still imply the former one: Ep(,..)r(x, =
1, which guarantees that q(a, s) integrates to on its support. We further discuss the benefits of this conditional constraint in Section A.2.3
Shape Constraint As discussed in Section 2.3
the definition of Q heavily impacts the desired be- havior of the solution. In particular, controlling the shape of the allowed distributions 9 especially
'Adapting work to EO is straightforward: described in Sec. 2.1 adapting the adversarial method of Zhang et al. (20T8) the EO task simply requires concatenate the the true outcome Y to the prediction f(z) as input of the adversarial classifier. The same process can be followed for ROAD.
In the situation where all distributions in Q are absolutely continuous with respect all measurable subset A X Y, all g E Q. ((A) > 0 only if p( () > 0)
5

---

Published as a conference paper at ICLR 2024
erucial in setting such ours, where the focus of the mitigation process is done dynamically. Without any constraint (as proposed by Mandal et al. 20200) the mitigation could indeed end up focusing on specific points of the dataset where the sensitive reconstruction from (X) is the cas- iest, using very sharp distributions & close to a Dirac. This may turn partieularly unstable and, more critically, could concentrate the majority of fairness efforts on relatively small subset of samples. To control the shape of the bias mitigation distribution 9, we therefore choose to consider Q as KL- divergence ball centered on the training distribution p. However, similarly Michel et al. (2022). we do not explicitly enforce the KL constraint (due to the difficulty of projecting onto the KL ball) and instead use a relaxed form. Using previous notations, the KL constraint takes the simple form KL(g Ip) (L(prllv) = Epr log Ph E, r logr.
(X)
The spread of Q can then be controlled with temperature weight in the overall optimization process, which can be seen as the weight of a Shannon entropy regularizer defined on discrepancies of y regarding p. Setting = 0 means that no constraint the distribution of r is enforced, thus encouraging r to put extreme attention to lower values Lg. On the other hand, higher values of favors distributions q that evenly spreads over the whole dataset, hence converging towards a classical globally fair model for highest values (cf. Section 2.T). Note that setting this hyper- parameter is strongly related to implicitly tuning the size of the smallest subgroup of the population for which we ensure fairness (ef. section 2.3
ROAD Formulation The overall optimization problem of our Robust Optimization for Adversar- ial Debiasing (ROAD) framework can thus finally be formulated as (full derivation given in A.I):
n
(fm 11 a
min max- PER T1
ran, 85) (Sw (x:)), 8i)
r(a, 8:) log(r(zr 8:))
constrriin
with "y = arg min We
Ls (gw, (Sw, (x:)), 81)
71
3.2 TWO IMPLEMENTATIONS FOR ROAD
3.2.1 BROAD: A NON- PARAMETRIC APPROACH
Let us first introduce a non-parametric approach, called Boltzmann Robust Optimization Adversarial Debiasing (BROAD), where each r r(z,, s;) value results from the inner maximization problem from [3] As described below, this inner optimization accepts analytical solution, whenever values respect the aforementioned conditional validity constraints (proof in Appendix AA3)
Lemma 3.1. (Optimal Non-paramen parametric Ratio) Given a classifier fw, and an adversary Iwg the optimal weight r raa,, 81) for any sample from the training set, is given by:
-s (er (1.)),s1)
r(I,, 81)
noy Dea Cslang (1))))))))))))
With ns; = E 1,s,.- This expression allows us to set optimal weights for any sample from the training dataset, at no additional computational cost compared to classical adversarial fairness approach such as Zhang et al. (2018). However, this may induce an unstable optimization process, since weights may vary abruptly for even very slight variations of the classifier outputs. Moreover, it implies individuals weights, only interlinked via the outputs from the classifier, hence at the risk of conflicting with our notion of local fairness. We therefore propose another parametric imple- mentation, described in the next section, that improves the process by introducing local smoothness in the fairness weights.

---

Published as a conference paper at ICLR 2024
3.2.2 PARAMETRIC APPROACH
To introduce more local smoothness in the fairness weights assigned to training samples, we pro- pose an implementation of the r function via a neural network architecture. Our goal is to ensure that groups of similar individuals, who might be neglected in the context of group fairness mitigation (e.g., due to their under- representation in the training population, cf. Fig. T receive a similar level of attention during the training process. However, solely relying on adversarial accuracy, as done in BROAD, may induce many irregularities in such groups. The lipschitzness of neural networks can add additional implicit locality smoothness assumptions in the imput space, thus helping define the distributions q as subregions of the feature space. Note that, in this approach, the network architec- ture therefore plays a crucial role in how local the behavior of Tw. will be: more complex networks will indeed tend to favor more local solutions, for same value of In particular, network of infinite capacity that completes training will have, in theory, the same behavior as BROAD.
To enforce the conditional validity constraint presented earlier, parametrization with two batch-level normalizations, one for each demographic group. For each sample (Ii, yı: 8:) in the mini-batch, we define the normalized ratio
employ an exponential
Vi, Tw, (I,, 8:) I
The L (assss)eTT. ch (15109)
with h : X' x 10;1} - R neural network with weights wy, To train ROAD, we use an itera- tive optimization process, alternating between updating the predictor model's parameters wy and updating the adversarial models' parameters Wi and by multiple steps of gradient descent. This leads to a far more stable learning process and prevents the predictor classifier from dominating the adversaries. More details are provided in the appendix (see Alg, I.
4 EXPERIMENTS
4.1 ASSESSING LOCAL FAIRNESS
In this first experiment, we assess how effective ROAD for generating predictions that are lo- cally fair for unknown subpopulations, while guaranteeing certain level of global accuracy and global fairness. For this purpose, we use 3 datasets often used in fair classification, described in Appendix A.8.T Compas (Angwin et al., 2016), Law (Wightman 1998) and German Credit (Hof-) mann 1994). Each dataset is split into training and test subsets, and the models described below are trained optimize accuracy while mitigating fairness with respect to a sensitive attribute
To assess fairness at local level, various subpopulations chosen among features of X, i.e. ex- cluding 5, are selected in the test set. As an example on the Compas dataset, in which S is Race: to create the subgroups, Age is discretized into buckets with 10-year range. These intervals are then combined with the Gender feature, identifying 12 distinct subgroups. As measuring DI in seg- ments of low population is highly volatile, we filter out subgroups with less than 50 individuals (see App.[A.8.3) These subgroups are unknown at training time, and chosen arbitrarily to reflect possible important demographic subgroups (see Sec. 4.3.2 for further discussion). Given these subgroups G, the local fairness is then assessed on the worst Disparate Impact value across these subgroups:
Worst-I-DI = maxgeg [((,,)ee( (w,,())))))))))) (z)[s 1) E(z,)) (r){s 0)). To evaluate our ap- proach, we compare our results with the globally fair adversarial models from Zhang al. (2018)
and Adel et al. (2019), and 3 approaches that address faimees generalization: FairLR (Rezaci et al. 2020), FarcoELS (Ferry et al.) 2022) and CUMA (Wang et al. 2023) (cf. (A..2)....)
As local fairness can only be measured against global accuracy and faimess, we evaluate the ap- proaches by plotting the tradeoffs between global accuracy and worst-1-DI subject to a global DI constraint (we choose DI < 0.05, following the fairness literature (Pannekoek & Spigler 2021 To ensure a thorough exploration of these tradeoffs, we sweep across hyperparameter values for each algorithm (hyperparameter grids in App. A.8.4). Fig. 2 shows the resulting Pareto curves for each method. Overall, ROAD mostly outperforms all other methods. This tends to show how our method efficiently maximizes local fairness, without sacrificing any other desirable criterion too much On the other hand. BROAD does not always perform as effectively as ROAD, illustrating the benefit from the local smoothness induced by the use of neural network. Interest-

---

Published as a conference paper at ICLR 2024
COMPAS <0.05)
(Glbbal DJ x0.05)
(Globa) x0.05)
0.915
0.700
0.70
0.63
0.99
0.70
0.80
-- EROID Rort hinhhaam
5015 (Oe0)
FROID Fars lom Fogee
0.91
RG enal
0.72
0.bbhh Intreen (Cheng
GRE 23 roce (21
utaL 23)
0.45
3.5 6.5 C.MA Hine 231
O.MA Hong etal
Fet wo..coo Pory 231
0.90
0LKCE
4.4
335
3.3
0.5
C23
9.00
E.0
625
6.20
635
3.55
6.66
535
3.4
RLL COE 3. 1M 6.0
Local Unto mess
Local Unfalmess (worst
Unfsimess (Wors:
Figure 2: Results for the experiment on Local Fairness. For all datasets, the X-axis is Worst-I-DI, Y-axis is Global accuracy. The curves represented are, for each method, the Pareto front for the results satisfying the imposed global fairness constraint (here, Global DI < 0.05 for all datasets).
ADULT (th distritutiinn
ADULT (2014)
ADULT (2015)
0.34
0.25
0.70
0.74
0.33
0.22
Accurryy
KORD (Oust
0.85
RBA 10and
OSR reigg at E-00
GU96 conugg et (.0
0.30
di EChing
Oitutt (Ctung al
Ditulty ta (hanng
025
0.20
835
0.12
E3E
ECE
635
0.M4
902
0.04
1.05
302
600
Gooal Palmess (DO)
Glocal Paimess (EO)
Global Falmess (D0)
Figure 3: Pareto front results on distribution drift using the Adult dataset. For all figures, the X-axis is Equalized Odds; the Y-axis is Accuracy. Left: in-distribution (i.e. Adult UCI 1994) test dataset; Center and Right: resp. 2014 and 2015 test datasets from Folktables (Ding et al. 202I).
ingly, despite not including any robustness component, globally fair methods of Zhang et al. (2018) and Adel et al. (2019) still manage to slightly reduce local bias through their global mechanisms.
4.2 EXPERIMENTS ON DISTRIBUTION DRIFT
As discussed in Section DRO-based techniques have been considered before to help with the generalization of fairness. In this section, we therefore aim to show how our approach also leads to a better generalization of fairness in the face of distribution shift in addition to better-protecting subpopulations. For this purpose, we replicate the experimental protocol of Wang al. (2023): after training classifiers on the training set of the classical Adult dataset (1994), we evaluate the tradeoff between accuracy and global fairness (measured with Equalized Odds (EO)) on the 2014 and 2015 Folktables datasets (Ding et al. 2021), containing US Census data from corresponding years, thus simulating real-world temporal drift. The same approaches in the previous section, adapted to optimize for EO (details in Appendix (A.8.2). are tested. Once again, the hyperparameters of every method are adjusted to maximize the two considered criteria, and the Pareto front is shown in Fig.
Results on the classical Adult test set (in-distribution, left figure) are somewhat similar for most methods, with CUMA (Wang et al.) 2023) slightly out-performing other methods. However, on drifted test sets (center and right figures), ROAD seems to achieve significantly better results than other methods. including other DRO-based fairness approaches. This suggests that the parametric implementation proposed in the paper is better suited to ensure robust behavior.
4.3
ABLATION STUDIES
4.3.1
BEHAVIOR OF r AND IMPACT OF
The behavior of ROAD depends on which controls the extent to which the distributions E Q are allowed to diverge from The impact of T can be observed in the left figure of Fig, for the Compas dataset. As values of increase, the variance of the distribution of decreases, going from having most weights close to 0 and very high importance on a few others, to having most weights r; lying around 1. Choosing the right value of thus helps control the emphasis put some subpopulations.
8

---

Published as a conference paper at ICLR 2024
033
060
3.07
8:11
0.15
Temperature parameter
820 S4E
Su8g axerager
Figure 4: Analysis of the behavior of ROAD Compas. Left: distribution of for several values of r at epoch 200 (truncated at > 5). Center: Relationship between Local DI and the average value of assigned instances belonging the corresponding subgroups. Each dot subgroup. Right: Worst-I-DI as a function of r for different values for X (quartiles between 0.0 and 10.0)
Slaned
Weree Local
Goaa
0.0
subgroup del
Sungroup del.
Sungroup def.
Subgroup def
Subgroup def
Subgroup del.
Sungroup delf
Subgroup def. ?
Subgroup def
Subgroup def
Subgroup def
Subgroup
Figure 5: Worst- 1-DI scores for subgroups of the Law dataset of various definitions, built by varying age bin width and splits along gender. Full description or the subgroups is available in Sec. A.8.3
critical assumption ROAD relies on is that the adversary r puts more attention on locally unfair regions. We test this assumption the Compas dataset (same subgroups in Sec. 4.1 and observe the results in the middle of Fig. 4 For each subgroup & E G (blue dots), we measure its local fairness (y-axis) and the average weight E(g,,)e 8)) associated instances of k. The graph reveals correlation between these two notions, suggesting that more emphasis is indeed put on more unfair regions. As consequence of these two results, setting r helps control local bias, as shown in the right of Fig. 4 for various values of Xg. The perfect local fairness score achieved when = 0 is due to a constant model fuy: with no shape constraint, concentrates all the fairness effort on each training sample successively, which finally leads to f(X) = E[Y] for any input. Choosing higher value of T helps regularizing the process by inducing a distribution q(r(s) closer to p(r(s).
4.3.2 How IMPORTANT IS THE DEFINITION OF SUBGROUPS?
The main motivation for ROAD is its ability to maximize local fairness when the definition of the local subgroups is unknown. To assess the veracity of this claim, conduct another experiment where we measure the local fairness of ROAD when the definition of these subgroups vary. Con- cretely, train once biased model, globally fair model (Zhang al.] (018) and ROAD (with resp. accuracy scores 0.72, 0.60, and 0.60), and measure the local fairness for these models in sub- groups of various definitions. These subgroups of 5, 10, 15 and 20, first across the whole population and then across subpopulations of other, non- sensitive, variables. Fig. shows the local fairness results for the Law dataset (sensitive attribute is Race, subgroup attributes are Age and Gender). As expected, although the worst local DI for ROAD varies when the subgroup definition changes, it is almost consistently below the values reached by the globally fair model (except Def. 3 corresponding to the largest subgroups). This suggests that its tuning not over-reliant on one subgroup definition, showcasing the flexibility approach.
defined successively
age bins with width
CONCLUSION
In this work, we introduced the problem of enforcing local fairness unknown subpopulations. By leveraging the strengths of adversarial learning and Distributionally Robust Optimization, our pro- posed framework ROAD provides powerful approach for this setting, addressing the shortcomings of previous DRO-based approaches. Future works include extending our work to settings where the sensitive attribute is not available, to other differentiable penalties (e.g., Mutual Information in Ragonesi al. (202T), and further exploring the optimization of 3-network adversarial approach.

---

Published as a conference paper at ICLR 2024
REFERENCES
Tameem Adel, Isabel Valera, Zoubin Ghahramani, and Adrian Weller. One-network adversarial fairness. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 2412- 2420, 2019.
Ulrich Aïvodji, Julien Ferry, Sébastien Gambs, Marie-José Huguet, and Mohamed Siala. Faircorels, an open-source library for learning fair rule lists. In Proceedings of the 30th ACM International Conference Information & Knowledge Management, pp. 4665- 066544669, 2021.
Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. Machine bias. ProPublica, May 23, 2016, 2016.
Ari Ball-Burack, Michelle Seng Ah Lee, Jennifer Cobbe, and Jatinder Singh. Differential tweet- ment: Mitigating racial dialect bias in harmful tweet detection. In Proceedings of the 2021 ACM Conference Fairness, Accountability, and Transparency, pp. 116-128, 2021.
Reuben Binns. On the apparent conflict between individual and group fairness. In Proceedings of the 2020 conference on fairness, accountability, and Iransparency, pp. 514-524, 2020.
Alexandra Chouldechova. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data, 153)1153 2017.
Frances Ding, Moritz Hardt, John Miller, and Ludwig Schmidt. Retiring adult: New datasets for fair machine learning. Advances in Neural Information Processing Systems 34 (NeurIPS 2021), 2021.
John Duchi, Tatsunori Hashimoto, and Hongseok Namkoong. Distributionally robust losses for latent covariate mixtures. Operations Research, 71(2):649 664, 2023.
John Duchi and Hongseok Namkoong. Learning models with uniform performance via distribu- tionally robust optimization. The Annals of Statistics, 49(3), 2021.
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings of the 3rd Innovations in Theoretical Computer Science Con- ference, pp. 214-226, 2012.
Julien Ferry, Ulrich Aivodji, Sébastien Gambs, Marie-José Huguet, and Mohamed Siala. Improving fairness generalization through sample-robust optimization method. Machine Learning, pp- 1-62, 2022.
Vincent Grari. Adversarial mitigation to reduce unwanted biases in machine learning. PhD thesis, Sorbonne University, Paris, France, 2022. ateltttee// Eel-038288 0E-20388400
Vincent Grari, Boris Ruf, Sylvain Lamprier, and Marcin Detyniecki. Fair adversarial gradient tree boosting. In 2019 IEEE International Conference on Data Mining (ICDM), pp. 1060-1065. IEEE, 2019.
Vincent Grari, Arthur Charpentier, and Marcin Detyniecki. A fair pricing model via adversarial learning, arXiv preprint arXiv:2202.12008 2022.
Moritz Hardt, Eric Price, and Nathan Srebro. Equality of opportunity in supervised learning. In Advances in Neural Information Processing Systems, pp. 3315-3323, 2016.
Tatsunori Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. Fairness without demographies in repeated loss minimization. In International Conference on Machine Learning, pp. 1929-1938. PMLR, 2018.
Hans Hofmann. Statlog (German Credit Data). UCI Machine Learning Repository, 1994.
Serafina Kamp, Andong Luis Li Zhao, and Sindhu Kutty. Robustness of faimess: An experimen- tal analysis. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 591-606, 2021.
10

---

Published as a conference paper at ICLR 2024
Michael Kearns, Seth Neel, Aaron Roth, and Zhiwei Steven Wu. Preventing faimess gerryman- dering: Auditing and learning for subgroup fairness. In Proceedings of the 35th International Conference on Machine Learning, volume 80, pp. 2564-2572, 2018
Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. Inherent trade-off' in the fair deter- mination risk scores. 8th Innovations in Theoretical Computer Science Conference (ITCS fuer Informatik, 2017.
2017). Schloss Cresttul-Leibniz-Zeene
Natasa Kreo, Thibault Laugel, Jean-Michel Loubes, and Marcin Detyniecki. When mitigating bias is unfair: A comprehensive study on the impact of bias mitigation algorithms. arXiv preprint arXiv:2302.07185. 2023.
Preethi Lahoti, Alex Beutel, Jilin Chen, Kang Lee, Flavien Prost, Nithum Thain, Xuezhi Wang, and Ed Chi. Fairness without demographics through adversarially reweighted learning. Advances in neural information processing systems, 33:728-7445 2020.
Gilles Louppe, Michael Kagan, and Kyle Cranmer. Learning to pivot with adversarial networks. Advances in neural information processing systems, 30, 2017.
Debmalya Mandal, Samuel Deng, Suman Jana, Jeannette Wing, and Daniel J Hsu. Ensuring fairness beyond the training data. Advances in neural information processing systems, 33: 18445-88466, 2020.
Natalia Martinez, Martin A Bertran, Afroditi Papadaki, Miguel Rodrigues, and Guillermo Sapiro. Blind pareto Fairness and subgroup robustness. In International Conference on Machine Learning, pp. 7492-7501. PMLR, 2021
Paul Michel, Tatsunori Hashimoto, and Graham Neubig. Distributionally robust models with para- metric likelihood ratios. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022, 2022.
B Mittelstadt, S Wachter, and C Russell. The unfairness of fair machine learning: levelling down and strict egalitarianism by default. Michigan Technology Law Review, 2023.
Ziad Obermeyer, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. Dissecting racial bias in algorithm used to manage the health of populations. Science, 366((464))447 2019.
Marlotte Pannekoek and Giacomo Spigler. Investigating trade-offs in utility, fairness and differential privacy in neural networks. arXiv preprint arXiv:2102.05975, 2021.
Ruggero Ragonesi, Riccardo Volpi, Jacopo Cavazza, and Vittorio Murino. Learning unbiased repre- sentations via mutual information backpropagation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2729-2738, 2021.
Ashkan Rezaei, Rizal Fathony, Omid Memarrast, and Brian Ziebart. Fairness for robust log loss classification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp- 5511-5518, 2020.
Ashkan Rezaci, Anqi Liu, Omid Memarrast, and Brian D Ziebart. Robust fairness under covariate shift. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 9419- 9427, 2021.
Yuji Roh, Kangwook Lee, Steven Whang, and Changho Suh. Sample selection for fair and robust training. Advances in Neural Information Processing Systems, 34:815-827, 2021.
Andrew D Selbst, Danah Boyd, Sorelle A Friedler, Suresh Venkatasubramanian, and Janet Vertesi. Fairness and abstraction in sociotechnical systems. In Proceedings of the conference on fairness, accountability, and transparency, pp. 59-68, 2019.
Bahar Taskesen, Viet Anh Nguyen, Daniel Kuhn, and Jose Blanchet. A distributionally robust approach fair classification. arXiv preprint M0Xiv:2007.09530, 2020.
Nenad Tomasev, Kevin R McKee, Jackie Kay, and Shakir Mohamed. Faimess for unobserved char- acteristics: Insights from technological impacts on queer communities. In Proceedings of the 202/ AAAI/ACM Conference AI, Ethics, and Society, pp. 254-265 2021.
11

---

Published as a conference paper at ICLR 2024
S Wachter, B Mittelstadt, and C Russell. Why faimess cannot be automated: bridging the gap between eu non non-discrimination law and Al. Computer Law and Security Review, 41, 2021.
Christina Wadsworth, Francesca Vera, and Chris Piech. Achieving faimess through adversarial learning: application recidivism prediction. arXiv preprint
2018.
Haotao Wang, Junyuan Hong, Jiayu Zhou, and Zhangyang Wang. How robust is your fairness? evaluating and sustaining fairness under unseen distribution shifts. Transactions machine learning research, 2023, 2023.
Serena Wang, Wenshuo Guo, Harikrishna Narasimhan, Andrew Cotter, Maya Gupta, and Michael Jordan. Robust optimization for fairness with noisy protected groups. Advances in neural infor- mation processing systems, 33:5190-5203, 2020.
Yijie Wang. Viet Anh Nguyen, and Grani A Hanasusanto. Wasserstein robust classification with fairness constraints. arXiv preprint arXiv:2103.06822 068288 2021.
Linda F. Wightman. Lsac national longitudinal bar passage study. LSAC Research Report Series., 1998. URL AtRLbbtp:://app.. semant httpo//lar...r//rroroot/tSS
Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P Gummadi. Fair- ness constraints: Mechanisms for fair classification. arXiv preprint arXiv:1507.05259, 2015.
Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. Mitigating unwanted biases with adver- sarial learning. In Proceedings of the 2018 AAAI/ACM Conference AI, Ethics, and Society, pp. 335-340, 2018.
Indre Zliobaite, Faisal Kamiran, and Toon Calders. Handling conditional discrimination. In 201 IEEE 11th international conference on data mining, pp. 992-1001, IEEE, 2011.
12

---

Published as a conference paper at ICLR 2024
A APPENDIX
A.1 DERIVATION OF ROAD (EQ.
13 WITH GLOBAL AND CONDITIONAL NORMALIZATION
We start from the initial formulation of the problem as stated in equation that we relax, and include the KL regularization to get the quantity Le optimize:
g(z, s) p
Lo - Ep(x,g.s) [y - Xn (Bute..) (Is (fwy (2)), s)] +r
=By(a,,)
log
where q Q, with Q the set of joint distributions over support of (X, that are absolutely contin- uous with respect to p(X,S):
Let us write rex, s) p(a,a))
glm.e) Thus, using the importance sampling trick, we get:
Le - Ep(nys)) [y((fu (x).
N9 [ppa.s) [r(x, SsLs,, (fa, (z)), s)] + TBp(..s) [r(x, s) log (xx, s)]]
Following this, considering training dataset r = (Ii4, Mer 8t) sampled from p(X, S)S, our optimization problem could given as (which we refer to as single normalization):
min max-
Σ 1=1
Cy (y (zi), 9i) N
r (x, 8; Cs (Fwy (x;)), ss)
FER TI
r(a s;) log(r(z;, st))
n
KL constraint
with w
arg min We
Ls Ls(Sw, (Sw (x:)), 8))
where (rlpr 2} is uncertainty set ensuring that g = pr distribution (i.e., respecting (2, s)(((,,) = 1).
As explained in section (with further analysis section A.2.2) this is not restrictive enough to guarantee a stable optimization for our fairness objectives, for settings implying an adversarial trained on a dataset with unbalanced proportions of sensitive values p(S) is not uniform).
To go further, we propose to restrict to Ô Q, such that any q @ respects marginal equalities ((8) = p(s) for any E S. This leads to:
Lg Ep(a.y.))
[y (x),
log g(x, s) p(x, s)
Ag E By(ls)) [[slaw, (fw, (z)), s)] + TEp(s) By(1)))
From the continuity of distributions in Q w.t.. p and the fact that q(s) = p(s) for any & E Ó p(z(s) whenever q(xls) > O. Thus, using the importance sampling trick and introducing r(xss) A g(als))
we get: p(z|s)
L = Ep(rys)) [ (C)(fr(z))))))
Xg (p(())))()))) (s( (fw, (z) + (lppss)Bp((t [r(x]s) logr(z]s)]]
Noting that for any q E é. r(x,)) r(xls)r(s) rxrss), this leads to the following optimization problem, with R [r|pr 2):
13

---

Published as a conference paper at ICLR 2024
min max wy rER.
Σ 1=1
Ly (f
Σ
r 8;) (Sw; (Sw, (xx)): 8;)
+T
r 8:) log(r(2,, st)]
KL constraint
1
with w arg min
Cs (gw (fwy(1))), (1))),
71
which is equivalent to eq 13 since R frr E RlVs, Ep(als) r(z, s) 1}.
A.2
WHY DOES USING TWO CONDITIONAL NORMALIZATION CONSTRAINTS INSTEAD OF SINGLE GLOBAL ONE HELP?
A.2.1 THFORETICAL IMPLICATION OF THE DOUBLE NORMALIZATION CONSTRAINT
This section aims at proving that using two conditional normalizations of the r values, one for each sensitive, leads guarantee p(s) gs)), as claimed in the main paper and advised in the next sections.
Assuming we have function r that respects the property: Ep(=)s) r(æ,s)
for both s,
first start by observing that the classical global constraint also holds: Ep(x,s) ((x,s)
Ep(s) r(z,s) s) = E, p(s) = 1. This allows us to consider r(z, s) s(a.s) as a ratio of valid joint distributions, with ((z s) which integrates to one over the support of p.
Then, let us analyze the induced marginal ((s) resulting from such implicit distribution:
((s) = f g(x,, a(2,,))de s)dx = p(z, g(z,s) s) dr = p(s) s)dx = p(s) p(z,s)
As result, considering the uncertainty set R fr R/Vs, Ep(ale) r(z, s) 1} actually implies that the induced prior q(s) equals the sensitive prior p(s) observed in the dataset.
For completeness, we also show the reciprocal, that states that setting r such that p(s) = (ss) implies the considered constraints on r, which indicates that these validity constraints include all distributions q E Q such that g(s) = -p(s). To do so, let us consider that Vs. p(s) = g(s). Thus, we g(a.s) have p(s) g(,,, s)dr= f p(xr (lz.5) da. It directly follows that Vs, p(((())))) p(alsip(s) dr [m(,, s)] 1.
p(aaa)
Considering our validity constraints on thus equivalent as setting ((s) = p(s) for every 8, dealing with explicit distribution q was tractable.
A.2.2 ANALYSIS OF NORMALIZED WEIGHTS FOR A FULLY FAIR MODEL
In the context of faimesss we argue that ensuring classical global normalization constraint Ept,,s)7(2,, s) = during optimization not sufficient to provide an accurate behavior with re- gards to the fair metric, as it may lead to concentrate the majority of the fairness effort to a specific sensitive subgroup.
To understand why, let go back the justification of using an adversarial for group fairness as introduced in Zhang et al. (2018), taking demographic parity as an illustrative example (while the same remains true for other objectives, such as equalized odds). The objective of Demographic Parity is to obtain a classifier that allocates equal chances to both subgroups of the population given the sensitive S. Formally, a classifier f that respects: P( = 1|$ = 1) = P(Ý = 1 = 0),
where we note y = f(x) I(((X) > 0.5). Equivalently in the binary case, we need to ensure EJLiS 1] = E 0), or again P(Ý 1|5 1) P(Ý). During optimization, it is
14

---

Published as a conference paper at ICLR 2024
difficult to build regularization terms on comparison of such quantities, both of them implying the classifier. Besides the -in-differentiability induced by the use of an indicator function the loss, it would require backpropagate through estimators from both populations, which may either be subject to high variance or intractability. Rather, adversarial fairness Zhang et al. (208)) proposes to adversary network, ! that attempts to reconstruct the sensitive S from f(x). )- The adversary is optimal when it outputs [()) accurately for any input, and the classifier is fully fair when ((X) P(S ())( P(S 1). In that case, indeed have P(S- 1, (X)) P(S = ())((X)), which means independence and finally P(((X))S) = P(((X))
Now, let us assume situation where the classifier is fully fair, which thus means that P(SIS(X)) P(S) for any x E X. Let us also assume that we use an optimal adversary g (f(X). that accurately outputs (X)) for each (x, given f. Let us look at the optimal values of r given f by using our non non-paraman paraameett formulation BROAD (see section B.2.Ij. While using a single global normalization constraint Ep(z,s)((z 8) 1 in such situation of fully fair classifier, we have:
e -L =f))/ clg((((=s)))// for any sample i of the dataset. Without loss of generality, let us consider that P(S 1) > P(S 0). that case, we thus have: (ox((P(==1)// /Z > including Z
(og(P((=0))// /Z, for any constant Z means that in that situation, any sample such that s; = obtains a weight r; that is greater than the one for any sample such that 8, 0. In other words, with a global normalization constraint, samples from the most populated demooraphic--seniiive group are more constrained than other ones. Improving accuracy will thus be easier for the least populated group, leading to model with unbalanced error between the two populations.
E(z.,)er e (P((((((=)))/ This
On the other hand, when considering the conditional validity constraints, get for that situation:
(oR(P(S=a)))/
Pazza 2ر.8 (loR(P(8=)))/=
which ensures uniform fairness effort for every sample point from the dataset. In that ideal situa- tion, this means that q(z, s) = p(x, s), coming back to classical adversarial fairness approach such as in Zhang al. (2018).
A.2.3 EMPIRICAL OBSERVATIONS UNDER PRACTICAL SETTINGS
Sindlr (Giobal valldity combtraint)
(Glbbal
0.60
Dual (Vaiddly constreint conditined
Duair (eallhyy cevers et ao
33
10
Lie
120
510 4350 5328 AID X CIR
Paimess
Figure 6: Effeec of the conditional constraint E(r|s=1) r(a, 5) = (x ==)) s( = 1. M Left: dis- tributions of r(x]s- = 0) and r<*|s 1). Center: Ly distributions. Right: Pareto fronts of the Accuracy-Local Fairness scores for the Law dataset.
In the previous section, we have shown that ROAD with a single global validity normalization over- constrains the most populated group regarding the sensitive attribute, in the case of fully fair clas- sifier. We claim that this analysis of an extreme setting highlights a general behavior of the method, which better balances fairness constraints when using conditional validity normalization.
In this section, we propose to verify this assumption empirically, by considering the Law dataset, where the sensitive distribution is heavily imbalanced (P(S 1) (1/4)P(S = 0)). In the left image of Fig. 6 is shown the distribution of values assigned to instances verifying S = 1 (orange) and S 0 (green) for various training iterations, using global normalization in the top-most graph, and the conditional normalization on the bottom-most one. Using the global normalization, observe that overlap ever happens between the values of these sensitive groups. As
15

---

Published as a conference paper at ICLR 2024
result, most part of the fairness effort is supported by individuals from the largest population = 0. This is not the case with our proposed conditional normalization, which results in better local fairness (right image of fig. o We also observe in the center figure that the mitigation effort is greatly better balanced when using our conditional validity constraint.
While conditional validity constraint that implies ((s) p(s) may look limitative in case of distributional drifts (notably implying a drift of p(s)), we claim this is not the case as it does not constrain distributions inside both sensitive groups. First, we want point out that distribution drift on () without change on the support (i.e., with drifted distribution Prest that respects the absolute continuity hypothesis of q w.n.t to Prest) does not have any impact on the fairness of a fully locally fair model. It may only impact accuracy, which could be dealt with a second reweighting scheme for the prediction loss, same manner as done in classical DRO works such as Michel et al. (2022) (this was not considered in our work to avoid over-complexity of presentation and highlight our innovation, but we acknowledge the potential interest and value of this complementary direction).
In case of models where subgroups do not ensure P(Y|S) = P(Y), distributional drift may exhibit these local mistreatments with more emphasis on them in the test distribution, which explains the effectiveness of our approach for such setting (drift may make some subgroups more represented in the global evaluation metrics). On the other hand, drifting p(s), which corresponds to global drift, however not supposed to do so if the fairness effort is well balanced over both sensitive groups (as it is shown to be in figure 6 when using our conditional validity constraint). This is verified in experiments corresponding to 3 for which a drift on p(s) is applied (from 66% male in train, for 53% in 2014 and 2015 test sets), without limiting the effectiveness of our approach.
A.3 THEORETICAL PROOF: BROAD IS A BOLTZMANN DISTRIBUTION
This section contains proof for Lemma 3.1 that we rewrite below: Lemma A.1. (Optimal Non-parametric Ratio) Consider the optimization problem (inner maximiza- tion problem, on Wr, of Equation 13y:
max E ( 8) (Cs( (fw, (z)), s)) + ((s.. logr(z,
(7)
Myy
with wy = arg min E (mnyss) Cs (gm, (f (z)),
Under the global validity constraint: (p ((z, 8) =1
The solution to this problem can be rewritten as an inverted Boltzmann distribution with multiply- ing factor n.
e CC( Lhw (1.)))
r(z;, 8:)
1 EE Co (fug (1)))
Proof. The proof results from a direct application of the Karush-Kuhn-Tun Kucker conditions. Since there is no ambiguity, we use the following lighter notation for the sake of simplicity: instead of r(z, s); Ls instead of Ls (9ke (fw, (z)), s)); and E instead of ((a,y..)~p- Using these notations and writing the above problem in its relaxed form w.r.t. the distribution constraint, we obtain the following formulation:
max E(rLs) + + E(rlogr))) s(1 - E(r))
Following the Karush- Tuccer conditions applied to the derivative of the Lagrangian func- tion L of this problem in ri, we obtain:
0 + L's 7(1 + (1g + 0 0L Ør:
(8)
With L's = L (Jwy ( 84) the i-th component of L.g. The KKT condition on the derivative OL
in gives: dR = 0 E(r) 1. Combining these two results, we thus obtain:
1 Σ
1
-x-c3 1
B(r)=
rj
n
n
2==1 e- 2/
j=1
16

---

Published as a conference paper at ICLR 2024
Which again gives, reinjecting this result in Eq.
This leads to the form of a Boltzmann distribution (ignoring multiplication factor n), which proves the result.
The same proof can be derived under the conditional validity constraint.
A.4 ADDITIONAL ABLATION STUDIES
In this section, we present several additional results, that were not included in the paper due to lack of space.
A.4.1 DOES THE COMPLEXITY OF THE TWO ADVERSARIAL NETWORKS HAVE AN IMPACT ON THE QUALITY OF THE RESULTS?
In this section, we investigate the impact of the complexity of the two adversarial networks and r on the quality of the results. Previous works, such Grari al. (2019), have studied the influence of the adversary g complexity on the context fair adversarial learning objectives and show that a complex architecture can help to achieve significantly better results than a simple one (i.e., logistic regression).
Figure 7 presents the results of this study on the LAW dataset. We compare three levels of com- plexity for each network: simple (linear predictor). medium (only one hidden layer with 32 neurons with ReLu activation function), and complex (three hidden layers: 64 neurons, ReLu, 32 neurons, Relu, 16 neurons, Relu). The results show that a more complex adversary y tends to yield better re- sults, which consistent with the observations made by Grari et al. (2019) (i.e., complex network achieves better performance than simple logistic regression). One possible reason is that a single logistic regression may not retain enough information to predict the sensitive attribute accurately.
In contrast, for network (please note that due to the exponential parametrization, an exponential activation is applied it), while more complex architecture results in lower performance. As explained in Section 3.2.2, the network architecture plays a crucial role in determining the local behavior of Tw. More complex networks tend favor more local solutions for given value of r. In particular, network of infinite capacity that completes the training will, in theory, exhibit the same behavior as BROAD, thereby yielding more pessimistic solutions.
simpler linear prediction such as logistic regression is quite efficient
This underscores the importance of carefully considering the complexities of the adversaries when designing fair adversarial learning framework. Our analysis suggests that more complex adver- sary is preferable, while simpler network r can be more efficient. Further research is needed to better understand the interplay between these complexities and develop strategies for selecting the optimal combination for given problem setting.
A.4.2 ASSESSING LOCAL FAIRNESS ON THE WORST-3-DI INSTRAD OF THE WORST-1-DI
In this experiment, we aim gain deeper understanding of our methods' behavior. Instead of assessing local fairness based on the worst disparate impact value for the globally defined subgroups, focus the worst 3. This approach allows us to observe the impact of our instance-level re- weighting strategy at an intermediate level.
The definition of our segment remains consistent as in Section 4.1 In Figure we show the result- ing Accuracy- Worst-3-DI Pareto curves for each method. We observe that the results are slightly similar those observed the Worst-1-DI in the main text. For all datasets, in the Worst-1-DI experiment ROAD mostly outperforms all other methods. On the other hand, BROAD sometimes reaches comparable performance, illustrating the overly pessimistic solutions mentioned earlier.
17

---

Published as a conference paper at ICLR 2024
300 Epochs
300 Epochs
0.80
0.75
Accuracy
Accuracy
0.70
0.65
0.65
0.60
small and ru, medium
94, medium and fu, small
9x, and ra
Ro, medium fu, medium
large and medium
medium and large
0.50
Local Fairness (worst 1 DI)
Locall Fairness (worst DI)
Figure 7: Impact of the complexity of adversarial networks g and r on the quality of results on the LAW dataset. The comparison carried on three complexity levels; simple (a simple regression), medium (only one hidden layer), and complex (three hidden layers). The curves represented are, for each method, the Pareto front for the results satisfying the imposed global fairness constraint (here, Global DI < 0.05 for all datasets).
COMPASS (Global DI 0.05)
LAW (Clotal Di 0.05)
GERMAN (Global DI <0.05)
0.70
0.80
0.70
Accurauyy
0.24
9030
09/200 LD Phanel
0.50
Dery 2:222
740 Miciete a
1060 Cenl
Lndd Unfaimess tworst
Unfaimes (wors)
Local (worst 3 DO
Figure 8: Results for the experiment on Local Fairness. For all datasets, the X-axis is Worst-3-DI, Y-axis is Global accuracy. The curves represented are, for each method, the Pareto front for the results satisfying the imposed global fairness constraint (here, Global DI < 0.05 for all datasets).
A.4.3 ADDITIONAL RESULTS FOR THE ABLATION STUDY SUBGROUP DEFINITION (SECTION 4.3.2)
Section 4.3.2 described the experimental results the adaptability of ROAD to various subgroup definitions. We complete these results with the following: in Figure A.4.3. the experiments are repeated five times, and the same conclusions reached by the globally fair model Zhang et al. (2018) In addition; in Figure (.443) the details of the same results presented. Instead of only showing the Worst-1-DI values, all local DI values, for every subgroup, are shown.
observed: It is consistently below the values
Garderoo
Gaddend
Ttaad
Figure 9: Local DI scores for subgroups of the Law dataset defined as age bins (top row) and age bins and gender attributes (middle and bottom rows). Each column corresponds to bins of different sizes. Subgroups with less than 10 individuals are ignored. The process has been iterated five times.
18

---

Published as a conference paper at ICLR 2024
population
All papulation
population
population
Roond
Leca
Lecal
Local
Age bin (witth=s)
Age (wl
Age Bin (Mithh151
Age bin (wilth-20)
Gender=0
Ganderw0
Loca
Local
Lecal
Age: bin (Mithh-5)
Age bin (wldth-10)
Age bin (ildhh-15)
Agg bin (wioth-20)
Gender=1
Genderw1
Gender=1
Gender=1
Loca D1
Local
Age bin (witth-5g (5)
Age bin (width-10 101
Age bin (wichh
Age bit (width-20)
Figure 10: Local DI scores for subgroups of the Law dataset defined age bins (top row) and age bins and gender attribute (middle and bottom rows). Each column corresponds to bins of different sizes. Subgroups with less than 10 individuals are ignored.
A.5 ADDITIONAL RESULTS ON THE DIFFERENCE WITH TRADITIONAL PARAMETRIC DRO
In this section, we provide additional results and discussion to further clarify the differences be- tween our approach and parametric Distributionally Robust Optimization (DRO) approach such as presented in Michel et al. (2022). The main distinctions can be summarized as follows:
1. Different problem formulation and objective: Our method focuses on optimizing fair- ness criteria, such as Demographic Parity, by considering a sensitive attribute S. In contrast, Michel et al. (2022) apply parametric DRO to the training objective Ly without consider- ing any fairesss criterion. This fundamental difference in problem formulation leads to distinct optimization objectives, with our approach.
2. Different optimization: Our approach maximizes the minimum (max- mouux-mi)ppossblee adver- sarial loss and assigns more weight samples that are easier for the adversary to recon- struct S. This is stark contrast with the traditional DRO pursued by Michell et al.) (2022), which minimizes the maximum (min-max) training loss and focuses on classifier accuracy without considering fairness. Our technique employs a multi-objective optimization ap- proach that balances fairness and accuracy, while Michel et al. (2022 prioritize optimizing accuracy.
3. Different neural network architectures: Our optimization process involves three neural networks, while Michel et al.) (2022) utilize two. This difference in architecture complexity is direct result of addressing distinct problem formulations and optimization goals in each work.
4. Different validity constraint: The validity constraints used in our work and Michel et al. (2022) differ due to the unique optimization objectives and problem formulations. Our approach focuses satisfying faimess constraint while optimizing performance, whereas Michel et al. (2022) primarily concentrate on optimizing accuracy.
In Figure we present the results for the experiment on Local Fairness for the German dataset. The X-axis represents the Worst- I-DI, while the Y-axis corresponds to Global accuracy. We compare the Pareto front of our method (DRO) with that of Michel et al. (2022). From the figure, our method consistently achieves better results in the Pareto front compared to Michel et al. (2022). Our approach demonstrates more favorable trade-off between Worst-1-DI and Global accuracy, indicating that it can achieve higher accuracy while maintaining a higher degree of fairness (lower Worst-1-DD values) than the method proposed by Michel et al. (2022).
observe that
This result further supports the novelty and effectiveness of our approach, as it not only addresses distinet problem formulation and employs unique optimization strategy, but also demonstrates
19

---

Published as a conference paper at ICLR 2024
improved performance in terms of both fairness and accuracy compared to existing methods, such as Michel et al. (2022).
GERMAN
3.00
3.20
9.70
3.0
15000 (MC001022 al 2012
Local Unfaimass (Morsi
Figure 11: Results for the experiment on Local Fairness, For all datasets, the X-axis is Worst-I-DI, Y-axis is Global accuracy. The curves represented are, for each method, the Pareto front for the results satisfying the imposed global fairness constraint (here, Global DI < 0.05 for all datasets).
A.6 ADDITIONAL. RESULTS ON THE LOCAL. EQUALLZED--DDS CRITERION
As mentioned in the section Equalized Odds is another group faimess criterion. A classifier is considered fair according to this criterion if the outcome fuy (x) has equal false positive rates and false negative rates for both demographics S = and S 1 Hardt et al. (2016). This criterion is especially relevant when misclassification can have significant impacts on individuals from different groups. A metric to assess this is the disparate mistreatment (DM) Zafar et al. (2015). which we report as the sum of the two following quantities:
AFPR: : [P(fe, (x) = 1ly = 0, s = 1) P(fm, (x) 1ly = 0, 0)1
APNR: PP(f (x) = Gly = 1, PP (x) = Oly 1, O)l
As in experiment to assess equalized odds at a local level, we have chosen various subpop- ulations among features of X, i.e. excluding S selected in the test set. As measuring fair- ness criteria in segments of low population is highly volatile, we filter out subgroups contain- ing less than 50 individuals (more details in Appendix [A.8.3). These subgroups are unknown at training time, and chosen arbitrarily to reflect possible important demographic subgroups (see Section 4.3.2 for further discussion). Given the set of these subgroups G, the local fairness is then assessed on the worst False Positive Rate among these subgroups: Worst- = (fw,
maxgeg [f (x)]s y = 0) tive Rate among these subgroups: Worst-l-AFPR = maxgeg (R (f;
0, y = 0)), the False Nega-
1, = 0) ((,,))e (fan) (fu, (x)|s 0,y 0) and the worst Disparate Mistreatment Rate among these subgroups:
Worst- Porss-I-ADaa DDDRR = max [E (E(e,)) (x)ls 1, 0) Ex (fu, (s = 0, y = 0) +
(e (f [s) 1, 0) E( (f (s) 0,y 0)
To evaluate our approach, we compare our results with adversarial models specifically adapted for equalized odds from Zhang et al. (2018) and Adel al. (2019), as well as existing works that aim to address fairness generalization: FaiLL (Rezaci et al. 2020), and CUMA (Wang et al.) 2023) (cf. Appendix 8.2.2 for more details). While keep the same name there are differ- ent than demographic parity, for example, to achieve EO, Zhang et al. (2018) employs an ad- versarial learning approach by concatenating the outcome prediction f(z) the true outcome Y to the input of the adversary. The resulting relaxed optimization problem can be expressed as mingy max Ep () v) -XE AE.,Ls(go, (Jw, ()),,)))) s). This methodology balances minimiz- ing the predictor model's loss and maximizing the adversarial model's loss while satisfying equal- ized odds.
As local fairness can only be measured against global accuracy and global faireess, we evaluate the approaches by plotting the tradeoffs between global accuracy, Worst-I-AFPa Worst-I-APy AAFNR and Worst- -- subject common threshold in the fairness literature (Pannekock & Spigler (02T)). As in the section 4 to ensure thorough exploration of these tradeoffs, we sweep across hyperparameter values for each
global FPR and FNR constraint (we choose threshold equal 0.05,
20

---

Published as a conference paper at ICLR 2024
algorithm (details of hyperparameter grids in App. A.8.4). Figure 2 shows the resulting Accuracy- Worst-I-ADMR. Accuracy- Worst-1-ApNR and the Accuracy- Worst-I-ApPR Pareto curves for each method respectively.
For all datasets, ROAD mostly outperforms all other methods. This tends to show how the proposed method efficiently maximizes local fairess,, without sacrificing any other desirable criterion too much. On the other hand, BROAD does not always perform as effectively as ROAD, illustrating the benefit resulting from the local smoothness induced by the use of neural network.
GERMAWEEpulleee
Mogative)
0.26
0.24
0.74
0.74
K-80 Dount
AOAD (hund
8332 (Dant
0.20
6 d000 Nerroce (2hagg in
6.04 BPT moce (21019
0.70
6.004 Birrocce s2hing
Far1 Uhiue 21
Fart (S-oe
0.55
Unfalm
Figure 12: Results for the experiment on Local Fairness. For all datasets, Y-axis is Global accuracy. The curves represented are, for each method, the Pareto front for the results satisfying the imposed global faimeess constraint (here, Global FNR and Global FPR 0.05 for all datasets).
A.7 ADDITIONAL RESULTS THE CONDITIONED VALIDITY CONSTRAINT
In this experiment, our objective is to empirically compare two validity constraints: one considering only the conditionality with S, as previously discussed, and another that incorporates, in addition, the true value of the outcome in the expectation of the ratio. This approach stems from the idea that the mitigation of the disparate mistreatment A Ah6lld be divided into the two different values of Y to account for both false positives and false negatives per sensitive groups.
The conditional validity constraint of interests takes the following form:
Vsg Vyt, r, 8,
For each sample (zt, yos s;) in the mini-batch, define the normalized ratio
c (m
Vi, Tw, (2t, S/s S1.
(E e (193)))12))
Here, as before x 0: R a neural network with weights wy-
As previously mentioned, we employ an exponential parametrization with two batch-level normal- izations, one each demographic group. Moreover, due the conditionality with Y, we incorpo- rate two additional normalizations, leading to total of four demographic groups of interest.
To test the efficacy of this approach, we replicate the experimental protocol of the drift experiment described in subsection 4.2 Results the classical validity constraint (validity constraint S) and with the addition Y (validity constraint under and 8) are presented Figure I3 Surprisingly, the results show that the validity constraint with only S leads better generalization of fairness in the face of distribution shift. One assumption for interpreting these results is that the decomposition in four groups, i.c. along both Y and S, may lead to very small ma-batch subsamples. i.e. non- significant expectation estimates.
21

---

Published as a conference paper at ICLR 2024
assoibution)
(2014)
(2015)
0.70
0.30
0.32
Accuaay
0.70
19: by otrri Lnder
04:4 cer cemamann unders
0.79
630
625
925
0:55
6.3
6.55
0.00
6.00
0.02
3:62
3:15
5,03
6O05
9006
1.45
Gde
363
0.0
3.12
6.0
Glocal
(E0)
Global Falmess (E0)
Figure 13: Pareto front results on distribution drift using the Adult dataset. For all figures, the X-axis is Equalized Odds; the Y-axis is Accuracy. Left: in-distribution (i.e. Adult UCI 1994) test dataset; Center and Right: resp. 2014 and 2015 test datasets from Folktables (Ding al. 2021
A.8 ADDITIONAL EXPERIMENTAL DETAILS
We provide additional details on the experimental evaluations.
A.8.1 DATASETS DESCRIPTION
Experiment 1: Assessing Local Fairness
Compas The COMPAS data set/Angwin al. (2016) contains 13 attributes of about 7,000 convicted criminals with class labels that state whether or not the individual recidivated within 2 years. Here, use race as sensitive attribute, encoded binary attribute, Caucasian or not-Caucasian.
Law The Law School Admission dataset Wightman (1998) contains 10 features on 1,823 law school applicants, including LSAT scores, undergraduate GPA, and class labels repre- senting their success in passing the bar exam. The sensitive attribute selected in this dataset is race, a binary attribute.
German The German UCI credit dataset Hofmann (1944) comprises credit-related data for 1,000 individuals, including 20 attributes such as credit history, purpose, and employment status, as well as class labels that represent their credit risk (good or bad). For the sensitive attribute, we consider the gender attribute with Males the privileged group, and Females as the unprivileged group.
Experiment 2: Distribution Drift The training set used for this experiment is the training set of the traditional Adult dataset.
The test sets are the following:
in-distribution: traditional test set of the Adult dataset
2014: Adult 2014 dataset from Folktables, see below
2015: Adult 2015 dataset from Folktables, below
Adult dataset contains US Census data collected before 1996. Its very frequent use in the fairness literature was discussed in Ding et al. (202T). in which the authors criticize its out-of-date aspect, es- pecially in the context of evaluating social aspects of algorithms such their fairness. Instead, they introduce the Folktables, up-to-date, aligned, versions of the Adult dataset. Using these datasets, the authors discussed the generalization of the faimess metrics over time. similar setup then considered by Wang et al. (2023) to evaluate how robust their method, CUMA, was to temporal drift. In our work, directly replicate their experimental setup.
A.8.2 COMPETITORS AND BASELINES
Besides ROAD and BROAD, proposed in this work, the following method are used as competitors and baselines for the experiments:
22

---

Published as a conference paper at ICLR 2024
Globally fair model (hang al., 2018):
described in Section 2.1 To obtain the Pareto
curves in the results, the À hyperparameter is set to various values, allowing the exploration of the Faimess--accureee can be adapted to optimize either Demographic Parity or Equalized Odds.
tradeoff. As described in the footnote in Section 3 this approach
FAD (Adel et al., 2019); similarily to Zhang et al. (018)) FAD relies on an adversarial network to reconstruct the sensitive attribute and thus estimate bias. However, instead of reconstructing the sensitive attribute from the predictions far), FAD reconstructs it from an intermediary latent space. The authors have shown that this helped obtaining better results. Similarly to Zhang et al. (2018), FAD relies single hyperparameter À to bal- ance between accuracy and fairness. Originally optimizing Demographic Parity, the same adaption the approach from Zhang al. (2018) can be made to optimize EO. Although we have this method, it must be noted that several other notable adversarial learning for faimess approaches exist (Louppe et al. 2017 Wadsworth et al. 2018 Grari et al. 2022).
Robust FairCORELS (Ferry et al., 2022): this method leverages the DRO framework for fairness and integrates in the Fair approach (Aivodji et al. 2021). It relies on preprocessing step discretizing the whole dataset into segments. The uncertainty set for DRO is then defined as a Jaccard Index-ball around p. RobustFaiCOREES then uses lin- ear programming and efficient pruning strategies to explore Q and solve its otpimization problem. RobustFairCOREL. relies mainly on one hyperparameter: e, the unfairness tol- erance allowed, from which is derived the size of Q (maximum Jaceard distance from p allowed). Besides, it relies FaiCCORELS hyperparameters, which set to their default values. RobustFainCORELS can be used to optimize several fairness criteria, including Demographic Parity and Equalized Odds.
fairLR (Rezaei et al. ) to achieve more robust fairness, fairLR also leverages the DRO framework for fairness. The generalization they propose to ensure is more restrictive, as they make the assumption that there is no drift between the train and test distributions. They therefore define the ambiguity set Q as the set of distributions matching some statistics of the training data, that they use to train logistic regressions. These constraints can be adapted either to optimize for Demographic Parity or Equalized Odds. fairLR relies only one hyperparameter, C, a regularization parameter for the logistic regression. Although has scemingly direct link with fairness, we still consider it in the experiments.
CUMA (Wang et al. 2023) y this method leverages adversarial learning mitigate fairness. Besides, it proposes to add a new component to its loss function, based on the curvature smoothness of its loss function. The idea is that to ensure better fairness generalization, the curvatures of the loss functions of each sensitive group should be similar. The objective function then approximated using a neural network. uses two hyperparameters, to bal- ance accuracy with fairness and smoothness: a (adversarial component) and (smoothness component). Similarly to the Globally Fair model and FAD, CUMA can be adapted in the same manner to either optimize Demographic Parity of Equalized Odds.
A.8.3 SUBGROUPS DESCRIPTION
In this section, we describe how subgroups were defined in the experiments.
Experiment
Compas Subpopulations are created by discretizing the feature Age in buckets with width of 10, intersected with feature Gender. Subgroups of size of at least 50 are kept.
Law Subpopulations are created in the same manner, by discretizing the feature Age in buckets with a width of 10, intersected with feature Gender. Subgroups of size of least 20 are kept.
German Subpopulations created by discretizing the feature Age in buckets with width of 10, intersected with feature Gender. Subgroups of size of at least 20 are kept.
Ablation study: how important is the definition of subgroups Table describes how subgroups were defined in the 12 scenarios shown in Figure 10
23

---

Published as a conference paper at ICLR 2024
Subgroup definition
Age bin width
Gender
Subgroup def. 0
All population
Subgroup def.
All population
Subgroup def. 2
15
All population
Subgroup def. 3
All population
Subgroup def. 4
Gender=0
Subgroup def. 5
10
Gender=0
Subgroup def. 6
15
Gender=0
Subgroup def. 7
20
Gender=0
Subgroup def. 8
Gender=
Subgroup def. 9
10
Gender=
Subgroup def. 10
15
Gender=1
Subgroup def. 11
20
Gender=1
Table I: Description of how subgroups were defined in the 12 scenarios shown in Figure 5
A.8.4 IMPLEMENTATION DETAILS FOR ROAD AND BROAD
In order to obtain the results shown in Figure ad] 3 for ROAD and BROAD:
we explore the following hyperparameter values
>g: grid of 20 values between 0 and 5 grid of 10 values between 0 and
The networks Sun Im, and Tw, have the following architecture:
fwy: FC:64 R, FC:32 R, FC:1 Sig
Iw, FC:64 R, FC:32 R, FC:16 R, FC:1 Sigm
hw, FC:64 R, FC:32 R, FC:1
A.9 DESCRIPTION THE ALGORITHMS
A.9.1 ROAD ALGORITHM
The ROAD algorithm, for the Demographic Parity objective, aims train locally fair model by iteratively updating three components: the sensitive adversarial model Jmy- the adversarial ratio model Tw,, and the predictor model fwy- The following problem of ROAD is defined follows (cf.
Eq. 4
min max-
Σ ( (z)), Au1 ==1
(or, (Ii, 8;) Ls(Ow; (Fuz (s)))
+
(Pw, (Ii, 8:) log(f (2i, 8t))
(9)
n
cunstraint
with wy arg min
Cslow, (fr
with representing the normalized ratio per demographic sub- sub-groups.
To obtain better results, commonly done in the literature, perform several training iterations of the adversarial networks for each prediction iteration. This approach results more robust ad- versarial algorithms, preventing the predictor classifier from dominating the adversaries.
The required inputs for ROAD include the training data containing X, Y, and S (please note that S is not required at the testing time), the loss functions (i.e., logloss function applied in exper- iment), batch size, number of epochs, neural networks architectures, learning rates, and the control parameters (r and X).
24

---

Published as a conference paper at ICLR 2024
For each epoch, the algorithm iterates through all the batches in the training data. In each batch, the algorithm performs the following steps:
Update the sensitive adversarial model Iw, The sensitive adversarial takes the current output predictions fu, (a:) input. The sensitive adversarial loss is calculated based on the current parameters of the predictor and sensitive adversarial models. The sensitive adversarial model is then updated using traditional gradient descent.
Update the ratio model Tw, The likelihood ratio is normalized across demographic sub- groups. The likelihood ratio cost function is calculated with respect to the average of the product between the weighting ratio and sensitive adversarial loss in addition the KL constraint. Then, the adversarial model is updated using gradient descent.
Update the predictor model: The loss function for the predictor model fæe is calculated, taking into account the traditional loss function of the predictor and the average of the product between the weighting ratio and sensitive adversarial. The predictor model is then updated using gradient descent.
The process repeated for the specified number of epochs. By iteratively updating these three mod- els, the algorithm aims to achieve balance between minimizing the loss function and maintaining fairness across demographic subgroups in the resulting predictor model.
Algorithm ROAD: Robust Optimization for Adversarial Debiasing
Require:
Training set I,
Individual loss functions ly and Is.
Batchsize b, Number of epochs Tee Neural Networks Tw,,, Juy and Duy Learning rates ('f, Oty and Fairness control Ag E R and Temperature control T E R
Number of training iterations fg and
1: for epoch E [H,... do
2: for all batch ((01, 81, yi),..., (IB, }B}} drawn from T do
3: for i E [1, [1. do
4:
Wg) + E 1s( (Fw, (z.)), , s,) {Calculate the sensitive adversarial loss]
aJ, (w) (Update the sensitive adversarial model 9w, by gradient de- Sws
5: Wg + Wg - ag
scent) end for
6:
7:
for j E [1.....]] do
bob cltor (asie))
8:
Vs E (0,1) Tw, (Ii, 8) + per demographic subgroups}
(Normalize the likelihood ratio
Eiz Lanr
9:
J, Jr(wr: wggg)
Ei Tw, (Iz, (g(g (fre (zz)), + Calculate the likelihood ratio cost function]
Tmn (Mti 8i) log(r (1i 8i)) Wr < Wr ar
10:
(w Owr
(Update the adversarial model rw,, by gradient descent)
11:
end for
Dis clunnttiiis) E (eciee
12:
Vs E 1} Tw, (xi, =8) +
(Normalize the likelihood ratio per
demographic subgroups}
Arw, (2,, 1s(9w, (fw, (zx)), st)) (Calculate the loss function of the predictor model) aJ w dwg
J [w wg))
E (y (zz) v1)
14:
W + Wy a
{Update the predictor model}
15:
end for
16:
end for
25

---

Published as a conference paper at ICLR 2024
A.9.2 BROAD ALGORITHM
The BROAD algorithm describes the non-parametric, implementation for the analytical solution of the problem considered in Eq [3 The main difference between the two algorithms is that BROAD does not include the update step for the ratio model Tw, and instead calculates the likelihood ratio directly in the predictor model update step. While it simplifies the optimization problem, it has been observed that the uncertainty sets generated by non non-parametric approaches are pessimistic (i.e., may encompass the considered distributions). and as result, lead to sub-optimal solutions.
This con-parametric algorithm, for the Demographic Parity objective, aims to train a locally fair model by iteratively updating two components: the sensitive adversarial model 9wi, and the predictor model fuy. The required inputs for this algorithm are the same as those for the previous one.
Update the sensitive adversarial model 9w,- This step the same as in ROAD. The sen- sitive adversarial loss is calculated based on the current parameters of the predictor and sensitive adversarial models. The sensitive adversarial model is then updated using tradi- tional gradient descent.
Calculate the likelihood ratio: Instead of updating the ratio model rw, the ROAD al- gorithm, the likelihood ratio is calculated directly using the exponential of the negative sensitive adversarial loss divided by the temperature parameter The likelihood ratio is normalized across sensitive groups.
Update the predictor model: The loss function for the predictor model fw, is calculated, taking into account the traditional loss function of the predictor and the average of the product between the weighting ratio and sensitive adversarial. The predictor model is then updated using gradient descent.
The process is repeated for the specified number of epochs. By iteratively updating these mod- els, the algorithm aims to achieve a balance between minimizing the loss function and maintaining fairness across demographic subgroups in the resulting predictor model.
In summary, the main difference between the two algorithms is the approach to updating the ratio model. ROAD updates the ratio model Tw. through an iterative process, while BROAD calculates the likelihood ratio directly in the predictor model update step. This alternative algorithm simplifies the process and may result in faster computation times. However, when comparing the performance with respect to faimess trade-offs, it has been observed to underperform compared to the original algorithm (ROAD).
26

---

Published as a conference paper at ICLR 2024
Algorithm 2 BROAD: Boltzmann Robust Optimization for Adversarial Debiasing (Conditional normalization)
Require:
Training set r,
Individual loss functions by and Is.
Batchsize 6, Number of epochs Re, Neural Networks fwy and Iw Learning rates a, 0g and ar, Number of training iterations fgs Faireess control Ag E R and Temperature control T E R
1: for epoch E [1....te]
2: for all batch ( $1,01,)) (IH, sB, yB)} drawn from I do
3: for i Ng... do
4:
wg) + 1 Ei= Is(Sng (fwr (z:)), 5: w/ f wh
.;))),s)) s,) {Calculate the sensitive adversarial loss} {Update the sensitive adversarial model 9w, by gradient de- dwg
Og
scent) end for
6:
Sslong (Jay (1,)),90)
7: Vs {0, 1}
r(z,, s.)
(eyy {Calculate the like-
nama, 2,
lihood ratio per demographic subgroups}
8: J 1 wg) + 1 Σ ly (far (2.). vi) - Ar(a, 8;) * Ils (fary (I:)). s,) (Calculate the loss function of the predictor model]
aJ w {Update the predictor model}
9: Wy Wy a
Bwy
end for
11: end for
A.10 ADAPTING ROAD TO EQUALIZED ODDS
As presented in the section 2 Equalized Odds (EO) (Hardt al. 2016) is another group fairness criterion that requires the classifier to have equal true positive rates (TPR) and false positive rates (FPR) across demographic groups. This criterion especially relevant when misclassification can have significant impacts on individuals from different groups. The constraint of equalized odds can be expressed as follows, where é and é2 represent the allowed deviation from perfect TPR denoted as Arrn and FPR A Arpn denoted as respectively.
min "y
E Ly(Suy (x),y)
s.t. 18 BL (x) 1, = B(z.,.) (So, (z)ly y 1, = 0)] < <e (10)
EE ((..,,))pp(rr (x)ly 0, 1) 1) - -p (wy (z)ly 0, 0) < 62
As explained section 2.1 achieve EO 7hang et al. (2018) employs an adversarial learning ap- proach by concatenating the true outcome to the input of the adversary. The resulting relaxed opti- mization problem can expressed as mingy max Ep (f (f())) 3) - AEpCs( (fw, (x), s), This methodology balances minimizing the predictor model's loss and maximizing the adversarial model's 's loss while satisfying equalized odds.
As for demographic parity, propose to optimize accuracy while adhering to worst-case equal- ized odds constraint. We define the subpopulations of interest, for which we aim to optimize fairness, distributions q within uncertainty set Q, and present the DRO framework for the Equalized Odds criterion as follows:
min wy
E (xr...)) (z).8)
Biz,,,) (w, 1, E(e.y.)) Jee (x)]y 1, 0]
s.t.
max qEQ
< El, (11)
E Eizg,,)ng fuy (x)( 00, (xyg))y fus (x)ly 0, 0
<
The constraint ensures that the False Positive Rate and the False Negative Rate between the two sensitive groups remain less than a predefined threshold 6 and 62 respectively under the worst-case distribution g Q. Working with distribution g allows to enforce local fairness by targeting
27

---

Published as a conference paper at ICLR 2024
subpopulations of interest, thus creating more focused and adaptable model that addresses fairness problems both globally and at a granular level. As for the demographic parity task we restrict Q to the set of distributions that are absolutely continuous with respect to p. This allows us write q rp. with r: : x x y > IR+ a function that acts as a weighting factor.
Given training set T sampled from p. stituting g with rp and applying its Lagrangian relaxation, as an optimization problem on r e R = (rlrp 2}:
thus reformulate the overall objective, by sub-
Ag 1
min max- my PER n
Ly (
r(a 81, Ui Ls) (Swy B2),
(12)
with w = arg min Wa
Cslgwn (x:)) st)
With Ag a regularization parameter controlling the trade-off between accuracy and faimess in the predictor model.
The overall optimization problem of our Robust Optimization for Adversarial Debiasing (ROAD) framework can thus finally be formulated as (full derivation given in A.I
min max rER
1
Σ Ly ( (x:), 31) - Au Σ
ra 86, Ms 3:) (1wg (wy (1i). Wi), 8t)
+T
r(a 8t, Vol log(r 81, (1,9)))) v:))
(13)
Kl. constraint
with " arg min
Cslou, (1a)
(x.). v.),
As for the demographie parity task, to introduce more local smoothness in the faimess weights signed to training samples, we propose an implementation of the r function via a neural network architecture. Our goal is to ensure that groups of similar individuals, who might be neglected in the context of group fairness mitigation (e.g., due to their under-representation the training popula- tion, cf. Fig. [ receive similar level of attention during the training process.
To enforce the conditional validity constraint presented earlier, we employ an exponential for each demographic group. For each
parametrization with two batch-level normalizations, sample (I; yi, 8;) in the mini-batch, we define the normalized ratio as:
Bur (21.8.88)))
Vi, Twe (I, S5 31)
Pay E (n.. ehwrr (331))355))
with h X' x (0:1) + R neural network with weights
To train ROAD, we use an iterative optimization process, alternating between updating the predictor model's parameters wy and updating the adversarial models' parameters wy and w, by multiple steps of gradient descent.
To obtain better results, commonly done in the literature, we perform several training iterations of the adversarial networks for each prediction iteration. This approach results more robust ad- versarial algorithms, preventing the predictor classifier from dominating the adversaries.
The required inputs for ROAD include the training data containing X, Y, and S (please note that S is not required at the testing time), the loss functions (i.e., logloss function applied in exper- iment), batch size, number of epochs, neural networks architectures, learning rates, and the control parameters (T and A).
For each epoch, the algorithm Titerates through all the batches in the training data. each batch, the algorithm performs the following steps:
28

---

Published as a conference paper at ICLR 2024
Update the sensitive adversarial model 9w, The sensitive adversarial takes the current output predictions fay(6 (I) as input. The sensitive adversarial loss calculated based on the current parameters of the predictor and sensitive adversarial models. The sensitive adversarial model is then updated using traditional gradient descent.
Update the ratio model rw,: The likelihood ratio normalized across demographic sub- groups. The likelihood ratio cost function is calculated with respect to the average of the product between the weighting ratio and sensitive adversarial loss in addition to the KL constraint. Then, the adversarial model is updated using gradient descent.
Update the predictor model: The loss function for the predictor model fw, is calculated, taking into account the traditional loss function of the predictor and the average of the product between the weighting ratio and sensitive adversarial. The predictor model is then updated using gradient descent.
The process repeated for the specified number of epochs. By iteratively updating these three mod- els, the algorithm aims to achieve balance between minimizing the loss function and maintaining fairness across demographic subgroups in the resulting predictor model.
29

---

Published as a conference paper at ICLR 2024
Algorithm 3 ROAD for Equalized Odds
Require:
Training set
Individual loss functions by and Is.
Batchsize b, Number of epochs Re, Neural Networks Tw,,, fay and Iws, Learning rates f 0g and ar, Number of training iterations mg and ny. Fairness control Ag E R and Temperature control T E R
1: for epoch E [1,... do
2: for all batch {(01, $1 y1). y1....., (IB, SB, YB)} drawn from I' do
3: for i E [g.... do
4:
:W) + E 1s(gn, (Jey (r:); Mc), sc) {Calculate the sensitive adversarial loss}
5: Wy t Wy
ag
8J.(wy dul
{Update the sensitive adversarial model Iw, by gradient de-
scent) end for
6:
7:
for j
E [1. do Ys E {0, 1}
E e (spirii)))
8:
Tw. (I; s. ratio per demographic subgroups}
(Normalize the likelihood
2 c (a
J, (y(wr, (g, wy) + Ei Tw, (ar,, 81, U/) * 1s(gw (fay (ax)), U1), st) + Tw, (2i, 8ú, yi) log(Tw, w, (2i, W:)) {Calculate the likelihood ratio cost function]
9:
Wwl (Update the adversarial model rw,, by gradient descent) Owr
10:
Wp - wp Or
11:
end for
El Jowe (1s102i0))
12:
Vs E 10 Tw, (2i,88 8,11) f
Eizr em,, 10 (Normalize the likelihood
ratio per demographic subgroups}
(wy,Wyy. + b Ei (y (2:), vi)
(1 8i, Ui) Is (s( (frr (2:), M1), 8:)
(Calculate the loss function of the predictor model]
14:
W < Wy o
BJ ( wes wa) dwy
(Update the predictor model}
15:
end for
16: end for
A.11 LIMITATIONS
In this section, we discuss few limitations of our work.
A.11.1 TRADITIONAL GROUP FAIRNESS LIMITATIONS.
Several well-known limitations may hinder the efficacy of group faimess methods. In that regard, our approach may suffer equally.
Unavailability the sensitive attribute. The majority of group fairness methods rely on the sensitive attribute S being available. This is also the case for method, S is used as tar- get variable for the adversary g. Although a few works have proposed methods to circumvent the issue (Hashimoto al. 2018 Lahoti et al. 2020), this remains major problem for algorithmic fairness, as many real-world situations fall into this category (Tomasev et al. 202T)
Impossibility of Fairness The "impossibility fairness" refers to trade-offs that exist between faimess criteria (Chouldechoval 2017 Kleinberg et al. 2017). In practice, this means that criteria such as Demographic Parity and Equalized Odds cannot fully satisfied at the same time. More generally, strong dependence between the sensitive attribute S and the target variable Y has been shown to make the task enforcing fairness without sacrificing too much accuracy difficult.Like other group faimesss methods, proposed approach suffers from the same limitations.
30

---

Published as a conference paper at ICLR 2024
A.11.2 LIMITATIONS TO THE NOTION OF LOCAL FAIRNESS
One of the contributions of our work is the introduction of the notion of local faimess. Although intuitive, local fairness is difficult to formalize exactly, as partly discussed in Section 2.2 two main obstacles hurting the adoption of the notion practice:
We see
Although subgroups of interest do not need to be defined at training time, they obviously need to when evaluating the approach. For this purpose, experts need to be able to define, and agree, let al.] (2021), this is sometimes difficult in practice. In their work, Wachter et al. thus give examples of several European regulatory bodies (e.g. European Court of Justice and European Parlement) disagreeing on definitions of gender, age groups, or work categories.
exact definition for these subgroupes. Yet, discussed by Wachter
Group fairness criteria, such as Demographic Parity and Equalized Odds, are measured in all subgroups. For these measures to be significant, the subgroups need be of a certain minimum size. Yet, this minimum size may be difficult to determine.
Although ROAD was proposed as a solution circumventing these issues, we still believe these issues may hurt potential widespread adoption of the notion.
A.11.3 LIMITATIONS TO THE PROPOSED METHOD (ROAD)
Besides the sensitive attribute being unavailable, another limitation for our approach could be when the subpopulations of interest in specific usecases would be described with features that are unavail- able. For instance, when enforcing faimess in the context of text (e.g. tweet) toxicity prediction, one might be interested in making sure that the model fair (e.g. w.r.t. to employed racial dialect Ball- Burack et al. (2 not only globally, but also that this fairness holds independently of the, say, the context of the tweet. Yet, this context might not always be available.
31


Reviews:
[{"review_id": "rusTFuZqEn", "reviewer": "Reviewer_d8Gj", "summary": "This paper addresses a practical issue in methods that ensure group fairness in machine learning models. The methods that aim for group fairness can often be unfair to a subpopulation. To this end, authors combine adversarial learning and distributionally robust optimization (DRO) to learn globally fair classification models. DRO reweighs the sample, assigning more weights to samples that are easier to adversary. This ensures that fairness constraints would hold on even worst-case subgroups. Notably, the proposed method does not use subgroup labels. They propose two methods for reweighing the samples --- One exploits the loss of adversary to compute the weights (BROAD), and the other uses a separate neural network (which is optimized alongside during training) to compute the weights (called ROAD). \n\nThe authors performed experiments on three fairness datasets. They consider worst-case demographic parity on subgroups as a fairness constraint in one scenario. In the other scenario, they learn the fair model under distribution shift and use equalized odds as the fairness measure. In both cases, they show that ROAD provides a better trade-off between accuracy and fairness than other methods.", "questions": "- **ROAD vs BROAD**: It is quite unintuitive that BROAD, an exact solution, performs poorly than ROAD, which uses a neural network to predict weights. The authors argued that ROAD is better because of the smoothing effects. Could it be that what is easy and hard for adversaries may change rapidly (or adversaries' output may be noisy) due to noisy batch gradients, and using an NN in ROAD may just be smoothing that out? In that case, would it make sense to use an EMA-like estimate with BROAD?\n\n- Fig 4 (Left Fig): Are these unnormalized values? Isn't weight, i.e., $r$, positive and less than 1?\n- **Adapting the method for EO**: \n  - **Parameterization of weight network:** If we use BROAD for EO, it will use the adversary's loss, which uses labels $y_i$. Thus $r_i$ is a function of $x_i, s_i$ and $y_i$. Should the weight network (in ROAD) also use $y_i$ to predict r$?\n  - **Validity constraint:** Would the validity constraint change for EO? That is, would it require normalization over both $s_i$ and $y_i$ or only $s_i$ as explained in the paper?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper addresses an important practical issue in group fair learning approaches and is a step towards ensuring the reliability of fair learning methods. \n- ROAD is consistently better than the other approaches for different levels of fairness and all datasets. In particular, I like using trade-off curves to report the results. \n- The setting does not require specifying subgroups during training and can be fair w.r.t. to any choice of subgroup due to worst-case distribution consideration. To this end, combining DRO with adversarial learning is interesting.", "weaknesses": "- **Baseline**: In fig 3 and 4, some of the baselines do not span all ranges of fairness. Why do we see this behavior? Are the baselines tuned correctly?\n- The proposed approach can be tricky to implement with several additional hyperparameters and the adversarial nature (min-max optimization) of the problem. \n- **Results**: The experiments and reported results could be more elaborate:\n  - How does DRO affect global fairness compared to other baselines? Fig 3 only reports local fairness in the form of the worst DI. However, the global fairness results are not reported.  \n  - While extending this framework to EO measure is straightforward, results with EO and local fairness are not reported. Similarly, DI and distribution shift results would strengthen confidence in the method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iSpcGelT0M", "reviewer": "Reviewer_bLrb", "summary": "Most prior work on group fairness measure the fairness discrepancy using global averages across pre-defined groups (e.g. males vs females).  However, in practice, one may care about fairness discrepancies in sub-regions of the feature space (e.g. males and females belonging to a particular age group). In this paper, the authors enforce fairness across such local sub-regions, by employing a form of distributionally robust optimization, equipped with an adversary that seeks to predict the sensitive attribute from the model predictions. Importantly, the proposed method does *not* assume the sub-regions to be known during training time, and instead optimizes over a set of possible sub-regions defined by an uncertainty set.", "questions": "(1) **Complexity of problem for formulation**: The authors seek to impose a robust version of the demographic parity constraint in (3), by introducing a maximization over an uncertainty set in the constraint. Instead of directly seeking to solve this problem, they formulate a equivalent problem with an additional argmin on an adversary's reconstruction loss (4). *Could authors elaborate why they chose to work with (4) instead of the simpler form in (3)?* It appears that the trick of applying instance-level weighting function $r$ could be applied as well to solving (3).\n\n(2) **Assumptions on $q$**: Could the authors formally state the assumptions on the class of perturbed distributions $q$, preferably early on in the paper (e.g. Sec 2). My understanding is that the group priors are required to be the same for both q and p, i.e. $q(s) = p(s)$. However, its unclear if the validity constraint described in Sec 3.1 would include **all distributions $q$ for with the same group priors as $p$**. It might be cleaner to have the assumptions on $q$ described first, and then argue why the particular choice of weighting functions $r$ satisfy those assumptions.\n\n(3) **Rationale for assuming equal group priors:** Does the assumption $q(s) = p(s), \\forall s$ amount to saying e.g. that the proportion of male and female samples would be the same across all sub-regions (e.g. across all age groups)? If so, is that a reasonable assumption to make? I think the authors need to better explain how their choice of uncertainty set aligns with practical applications where one may want to enforce the form of local fairness they describe.\n\n(4) **Distribution shift in Sec 4.2:** In the experiments with the Adult/census distribution shift in Sec 4.2, do we have reasons to believe that uncertainty set used would capture the \"real-world\" distribution shift considered? I am again particularly curious about the relevance of the assumption $q(s) = p(s)$ here.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- A practically relevant problem statement with a clearly presented problem formulation\n- The solution approach presented makes for a satisfactory contribution\n- Strong experimental results\n- Well-written paper", "weaknesses": "- The problem formulation seems *overly complex*, with multiple levels of nested minimization problems. It is unclear if the complexity is absolutely necessary (see question 1).\n- Justifications needed for choice of uncertainty set $\\mathcal{Q}$.\n- *Lack of strong convergence guarantees* for the proposed optimization strategy. In fact, owing to the nested nature of the optimization problem, it appears (please correct me if I am incorrect) that it would be hard to provide theoretical guarantees even if under a simplistic scenario when $f$ and $g$ are linear models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7ml5Ip7Sum", "reviewer": "Reviewer_34eA", "summary": "This paper focuses on a notion of local fairness as opposed to group fairness. When local subgroups are unknown at training time, it is difficult to enforce local fairness constraints. This paper expands the method DRO to include an adversarial learning component that specifically minimizes the ability of an adversary to reconstruct sensitive attributes. The main idea is to boost the importance of regions where sensitive reconstruction is easiest. This is done at each optimization step. To do this efficiently, the paper introduces an adversarial importance re-weighting method. They provide the analytical formulation for this method as well as two implementations (one non-parametric and one parametric). Finally, they experimentally evaluate the proposed method including ablation studies.", "questions": "(1) Is there a reason why the first paragraph of the introduction omits citations in statements like \"models...have been shown to unintentionally perpetuate existing biases and discriminations\"? \n\n(2) What do you see as the main use case for this method? What are the limitations you anticipate other than sensitive attribute not being available?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "(1) The main contribution is novel and clearly specified. Combining DRO with adversarial learning is simple yet effective.\n\n(2) The paper is written with clarity and exposition is easy to follow throughout. Problem statement as well as the two implementations are concise.", "weaknesses": "(1) The paper does not directly address limitations. For instance, is there a computational scalability issue with this technique compared to other more well known group fairness methods?\n\n(2) The reference to applied fairness research could be strengthened. There are many related works that are omitted. In particular, work on fairness and adversarial learning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rjvcTxTQ0N", "reviewer": "Reviewer_Cpzk", "summary": "This paper addresses the problem of local fairness, which ensures the model fairness on the subregion split via unknown attributes.  They proposed an approach based on the idea of distributional robust optimization, under an adversarial learning paradigm. The desired fairness is implemented of instance-wise sample reweights and the experiments are conducted to demonstrate the proposed method can achieve Pareto dominance.", "questions": "1.\tThe motivation of this paper is that global fairness methods usually cannot guarantee a consistent fairness value on split subregions, shown as Fig. 1. However, such observations are based on fine-grained partition (e.g., age categories) of training data and each subregion might be with a relatively small size. In this case, the disparity tends to happen. For example, DI might be very large if a bin only includes two samples. How do we confirm that biases do widely exist and then precisely define “local fairness”?\n2.\tFollowing question 1, I think it might be necessary to use some prior for split subregions. For example, in DRO, they introduced a lower bound of the subgroup ratio. With accuracy as a utility, BPF demonstrated that very small group would lead to uniform classifier. But from sec 3.2, as Softmax weights can be regarded as applying a Shannon entropy regularization. It would be better if the authors can point out such behind insights.\n\n        Blind Pareto Fairness and Subgroup Robustness, ICML 2021.\n\n3.\tStarting from Eq. (2), the input of $g_{w_g}(.)$ directly takes the output of $f_{w_f}(.)$. So, how do you implement $w_g$? Considering a binary case, $f_{w_f}(.)$ is a only probability in output space. \n4.\tFollowing question 3, please check the constraint of eq. (4), where s should be properly positioned.\n5.\tAbove Eq. (4), why distribution $q$ can help characterize local fairness is not clear for me. Please note that in DRO, it serves any subregion’s upper bound.\n6.\tSince $\\lambda_g$ is not learnable (Sec A.5.4), are the best results selected to have a Pareto dominance as claimed?\n7.\tPlease justify the novelty/contribution compared to Michel et al 2022.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The local fairness introduced in this paper is interesting, and the analysis to existing literature is almost thorough and clear. The method looks good, and the writing is well prepared.", "weaknesses": "I have some doubts about concepts and formulations after reading the paper. Also, the technical contribution may need further clarifications.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)