Title: Regression under demographic parity constraints via unlabeled post-processing
Venue: NeurIPS 2024
Accepted: yes
Keywords: fairness

regression
Abstract: We address the problem of performing regression while ensuring demographic parity, even without access to sensitive attributes during inference. We present a general-purpose post-processing algorithm that, using accurate estimates of the regression function and a sensitive attribute predictor, generates predictions that meet the demographic parity constraint. Our method involves discretization and stochastic minimization of a smooth convex function. It is suitable for online post-processing and multi-class classification tasks only involving unlabeled data for the post-processing. Unlike prior methods, our approach is fully theory-driven. We require precise control over the gradient norm of the convex function, and thus, we rely on more advanced techniques than standard stochastic gradient descent. Our algorithm is backed by finite-sample analysis and post-processing bounds, with experimental results validating our theoretical findings.

Regression under demographic parity constraints via unlabeled post-processing

Abstract

We address the problem of performing regression while ensuring demographic parity, even without access to sensitive attributes during inference. We present a general-purpose post-processing algorithm that, using accurate estimates of the regression function and a sensitive attribute predictor, generates predictions that meet the demographic parity constraint. Our method involves discretization and stochastic minimization of a smooth convex function. It is suitable for online post-processing and multi-class classification tasks only involving unlabeled data for the post-processing. Unlike prior methods, our approach is fully theory-driven. We require precise control over the gradient norm of the convex function, and thus, we rely on more advanced techniques than standard stochastic gradient descent. Our algorithm is backed by finite-sample analysis and post-processing bounds, with experimental results validating our theoretical findings.

1 Introduction

Algorithmic fairness is an umbrella term for a subset of machine learning research that aims to better understand, quantify, mitigate, evaluate, and conceptualize negative and/or positive effects of datadriven algorithms on the society. At least one direction in this field falls within theoretical machine learning, where a form of fairness constraint, mainly inspired by common sense and formalized within mathematical framework, is proposed as an arguably reasonable proxy for a definition of ethical and non-discriminatory prediction. Even more particular sub-field of this research direction is formalized within a paradigm of group fairness, that aims at mitigating negative impact (or provide equal treatment to) towards sub-populations that share a common sensitive characteristic. Many works fall within this category (Barocas et al., 2018, Calders et al., 2009, Chiappa et al., 2020, Dwork et al., 2011, Feldman et al., 2015, Gordaliza et al., 2019, Hardt et al., 2016, Jiang et al., 2020, Lum and Johndrow, 2016, Zafar et al., 2017, Zemel et al., 2013, just to name a few).

Even without going into debates on the relevance of a given definition of fairness, many, purely mathematical and algorithmic questions remain unanswered in this field. The best theoretical understanding of the problem is available for the demographic parity constraint in case of awareness -the situation when the sensitive attribute is available at inference time (Agarwal et al., 2019, Chiappa et al., 2020, Chzhen and Schreuder, 2020b, Chzhen et al., 2019, Denis et al., 2024, Gaucher et al., 2023, Le Gouic et al., 2020). The latter case is well studies both in classification and regression setups. This is no longer the case for other fairness constraints or the unawareness setup-the situation when the sensitive attribute is not available at inference time. In particular, while the case of

Evgenii Chzhen

classification has been studied before from algorithmic and mathematical perspectives (Chzhen et al., 2019, Gaucher et al., 2023, Gordaliza et al., 2019, Hardt et al., 2016), the regression setup remains largely under explored and many methods lack strong theoretical evidences. In particular, to date, none of previous works effectively build computationally-efficient, fully theory-driven algorithm for the problem of regression under the demographic parity constraint in the case of unawareness. The present work fills this gap. Relying on previous ideas of discretization that goes back to Agarwal et al. (2019), we design a smooth convex objective function whose exact solution yields a fair and optimal prediction function. It turns out that this objective admits a first-order stochastic oracle that can be evaluated using only one independent sample of feature vector, thus allowing for stochastic optimization approach. Furthermore, despite the convexity, we show that the key quantity to control is the gradient (or rather a gradient-map) of this objective function, deviating from the more common setup of controlling the optimization error measured by the objective function. We deploy recent machinery of Allen-Zhu (2021) and Foster et al. (2019) that allows to achieve this goal, properly setting all the hyper-parameters and recovering the usual statistical rate 1 / √ T for both fairness and risk guarantees T being the number of samples.

Our work falls withing the realm of post-processing methods-another umbrella term that combines all the methods that perform a refitting of a base estimator to satisfy a certain constraint.

Importantly, due to the careful design of the above mentioned objective function, we can perform this post-processing in an online manner using a stream of i.i.d. unlabeled data without keeping it in memory, making it attractive in practice. Our approach is based on a combination of ideas from previous contributions to fairness from Agarwal et al. (2019) and Chzhen et al. (2020b) and recent stochastic optimization literature (Allen-Zhu, 2021, Foster et al., 2019) that deals with stationary point-type guarantees in the case of convex optimization.

Contributions Our contribution is three-fold: i) we significantly enhance the discretization strategy of Chzhen et al. (2020b) accommodating multiple sensitive features, relaxed fairness constraints, and unawareness setup; we introduce entropic regularization for this problem and design a dual convex objective from it; ii) we design a semi-supervised post-processing algorithm and show that it enjoys strong theoretical guarantees; iii) we perform numerical simulations demonstrating the relevance of our approach in practice.

Organization. This paper is organized as follows: in Section 2 we present the problem setup and introduce main problem-related notation; in Section 3 we describe our methodology step-by-step and highlight main challenges and relations to other results; in Section 4 we gives technical details of the proposed approach; Section 5 contains main theoretical results of the work; finally, Section 6 contains empirical evaluation of our method. All the proofs are postponed to the appendix.

Notation. Let us present generic notation that is used throughout this work. For a positive integer K , we write [ K ] to denote { 1 , . . . , K } and [ [ K ] ] to denote {-K,... , 0 , . . . , K } . For a > 0 denote by ⌊ a ⌋ largest non-negative integer that is smaller or equal to a . For a univariate probability measure µ , we denote by supp( µ ) its support. For every β > 0 , m ∈ N , and w = ( w 1 , . . . , w m ) ⊤ ∈ R m , we denote by LSE β : R m → R the log-sum-exp function, defined as

For every m ∈ N , w = ( w 1 , . . . , w m ) ⊤ ∈ R m , we denote by σ = ( σ 1 , . . . , σ m ) : R m → R m the soft-argmax as σ j ( w ) = exp( w j ) / ( ∑ m i =1 exp( w i )) . For any matrix A , the notation A ⩾ 0 means that A is positive coordinate-wise. For any a ∈ R and w ∈ R m we set ( a ) + = max { 0 , a } and ( w ) + = (( w 1 ) + , . . . , ( w m ) + ) ⊤ . The notation ˜ O hides (unimportant) constants and polylogarithmic factors. For a pair of random elements ( A,B ) , we denote by Law( A ) , the law of A , by Law( A | B ) , the conditional law of A given B , and we write A ⊥ ⊥ B to denote that variables A and B are independent. For two vectors w , w ′ ∈ R m , we write w / w ′ = ( w j /w ′ j ) j ∈ [ m ] ∈ R m to denote element-wise division. The Euclidean norm of a vector and the Frobenius norm of a matrix are denoted by ∥ · ∥ , while the spectral norm of a matrix is denoted by ∥ · ∥ op . We denote by B ( R ) , the Borel sigma-algebra on R , induced by the usual topology. We write log to denote the natural logarithm and log a , the base a > 0 logarithm.

2 Problem setup

Let ( X , S, Y ) be a triplet of nominally non-sensitive, nominally sensitive, and output characteristics, taking values in R d × [ K ] × R for some K ⩾ 2 . We assume that ( X , S, Y ) ∼ P , for some unknown distribution P . The main quantities of interest are the following: the regression function η ( x ) def = E [ Y | X = x ] ; the marginal distribution of sensitive vectors p def = ( p s ) s ∈ [ K ] with p s def = P ( S = s ) ; the conditional distribution of S given X , defined as τ ( x ) def = ( τ s ( x )) s ∈ [ K ] with τ s ( x ) def = P ( S = s | X = x ) . A randomized prediction function is a map π : B ( R ) × R d → [0 , 1] such that the map B ↦→ π ( B | x ) for B ∈ B ( R ) is a probability measure on ( R , B ( R )) for all x ∈ R d . For any prediction π we define a random variable ̂ Y π as

Remark 2.1. Note that if π ( · | x ) is a Dirac measure for all x ∈ R d , the above condition just means that ̂ Y π = g ( X ) almost surely for some deterministic g : R d → R . The above condition is not to be confused with the fairness constraint, which is not formulated point-wise. It is only viewed as an extension of the unawareness framework to the case of randomized predictions. The above condition completely specifies the distribution of the triplet ( X , S, ̂ Y π ) but leaves the relation between ̂ Y π and Y ambiguous. To be more formal, one needs to add the condition ( ̂ Y π ⊥ ⊥ Y ) | ( X , S ) , that is, the prediction ̂ Y π is independent from the true label Y , conditionally on ( X , S ) . That would define a complete joint distribution of ( X , S, Y, ̂ Y π ) ∼ P π = P ( X ,S ) ⊗ P Y | ( X ,S ) ⊗ π ( · | X ) .

We consider the following risk of a prediction function π

A prediction function π is said to satisfy the demographic parity constraint , if ̂ Y π ⊥ ⊥ S .

That is, ̂ Y π is stochastically independent of S viewed from the perspective of the joint distribution of ( X , S, ̂ Y π ) . On the high-level, the goal in this setup is to find a prediction function π , whose risk is small and whose violation of the demographic parity constraint is controlled as quantified by some measure of unfairness. The above problem is well understood in the case of awareness -the situation when π is expressed as π ( · | x , s ) (Chiappa et al., 2020, Chzhen et al., 2020a, 2021, Jiang et al., 2020, Le Gouic et al., 2020)-revealing an intimate connection of this problem with Wasserstein barycenters. Yet, when the sensitive attribute is not an input of the prediction function, the situation is drastically different. Some attempts have been made to either (so far only partially) characterise the optimal prediction function (Chzhen and Schreuder, 2020a, Gaucher et al., 2023, Zhao, 2021) or to design efficient algorithms for this problem (Agarwal et al., 2019, Maheshwari and Perrot, 2022, Narasimhan et al., 2020) that are only partially supported by a sound theory. One of the principal goals of this work is to design a computationally efficient algorithm that admits a (near) end-to-end theoretical guarantees. The main difficulty of the problem lies in very different natures of the risk and the fairness constraint-the latter involves image measures, while the former is a simple linear functional of π . In the case of awareness this issue can be bypassed by lifting the problem in the space of measures, working there directly and, then, returning to the initial space of prediction functions. Crucially, this is achieved only thanks to the fact that S is known at inference time, which is not the case for the considered problem.

Remark 2.2. In what follows we will exclusively focus on the squared risk and the regression setup. However, one can observe that the proposed methodology can be extended or even simplified for R ( π ) = E [ r ( X , ̂ Y π )] and multi-class classification respectively under the demographic parity constraint. Here r ( x , ̂ y ) quantifies fit of ̂ y for an individual x and can be either known or unknown.

3 Our methodology

The starting point of our work is similar to the one of Chzhen et al. (2020b) and relies on a simple observation-if | supp( π ( · | x )) | < ∞ and stays the same for all x , the independence constraint

is reduced to a finite amount of constraints that only involve the image of π ( · | x ) . In particular, assuming that supp( π ( · | x )) = ̂ Y ⊂ R for all x ∈ R d , ̂ Y π is independent from S iff P ( ̂ Y π = ̂ y | S = s ) = P ( ̂ Y π = ̂ y ) for all s ∈ [ K ] and all ̂ y ∈ ̂ Y . In view of the definition of ̂ Y π , the latter is equivalent to

which, assuming that ̂ Y is fixed, correspond to linear constraints on π . Combined with the observation that π ↦→ R ( π ) is also linear, we end up with a problem that is significantly easier to handle. Furthermore, again assuming that ̂ Y is fixed, the sketched direction gives a natural way to introduce some slack to the independence constraint-simply requiring an approximate equality in (1). Set

for all s ∈ [ K ] and ̂ y ∈ ̂ Y . Thus, for a fixed support (whose choice will be discussed in the next paragraph) and a fixed vector ε def = ( ε 1 , . . . , ε K ) ⊤ , our goal is to build an estimator of a solution to

Let us now describe the methodology for selecting ̂ Y and the trade-offs that are introduced.

Introducing discretization. Having in mind the above discussion, for every integer L ⩾ 0 and real B > 0 , we introduce a uniform grid ̂ Y L def = B · [ [ L ] ] /L on [ -B,B ] , so that | ̂ Y L | = 2 L +1 , which is viewed as a support of prediction functions π ( · | x ) . For the sake of simplicity, we will assume that the regression function η ( · ) is bounded in [ -B,B ] for some known B > 0 .

Assumption 3.1 (Bounded signal) . There exists B > 0 such that | η ( X ) | ⩽ B almost surely.

Thus, for a given B , the main parameter to tune is L ⩾ 1 -the higher the L is, the more accurate prediction functions can be produced, while lower values of L ensure that the demographic parity requirement reduces to a small number of constraints. Thus, there is a trade-off that is introduced by L . A natural attempt to tackle the problem of fairness in this context would be to estimate a solution to (3) with ̂ Y = ̂ Y L . Of course, L needs to be chosen so that the aforementioned solution attains the risk that is close to the risk of some benchmark prediction function that does not involve any discretization. This will be discussed later in the text. For now, let us address another subtle issue. Even assuming a complete knowledge of the underlying distribution P , solving (3) requires solving a linear program in dimension Ω( LK ) which can be infeasible in practice for large values of L and K . Instead of (3), we rather focus on the entropic regularized version of it. For β > 0 , we consider

where R β ( π ) = R ( π ) + 1 β E [Ψ( π ( · | X ))] and for any discrete univariate distribution µ , we define its negative entropy Ψ( µ ) def = ∑ ̂ y ∈ supp( µ ) µ ( ̂ y ) log( µ ( ̂ y )) .

Remark 3.1 (On abuse of notation) . Note that for every ̂ y ∈ ̂ Y L there is a unique ℓ ∈ [ [ L ] ] such that ̂ y = ℓB/L and we will write π ( ℓ | x ) instead of π ( ̂ y | x ) . Similarly, we write U s ( π, ℓ ) instead of U s ( π, ̂ y ) , defined in (2) , when no confusion is possible and the support ̂ Y L is fixed.

An extremely attractive feature of the problem in (4) is the fact that the solution to it can be written explicitly as a function of optimal dual variables, with the latter being a solution of a stochastic convex program with Lipschitz gradient-the main observation of our approach, that shares many similarities with the smoothing technique of Nesterov (2005). This is summarized in the following lemma.

Lemma 3.1. Let L ∈ N and β > 0 . Let Λ ⋆ = ( λ ⋆ ℓs ) ℓ ∈ [ [ L ] ] ,s ∈ [ K ] and V ⋆ = ( ν ⋆ ℓs ) ℓ ∈ [ [ L ] ] ,s ∈ [ K ] be two matrices that are solutions to

where t ( x ) def = 1 -τ ( x ) p , r ℓ ( x ) def = ( η ( x ) -ℓB L ) 2 , and λ ℓ = ( λ ℓs ) s ∈ [ K ] , ν ℓ = ( ν ℓs ) s ∈ [ K ] . Then, (4) admits a solution in the form

Assuming perfect knowledge of η and τ , the above lemma suggests a natural approach to estimating the π Λ ⋆ , V ⋆ -we can run a (version of) stochastic gradient descent on F ( · , · ) and then plug-in the resulting dual variables in the formula for π Λ ⋆ , V ⋆ . Notably, a stochastic gradient of F ( · , · ) can be obtained by simply sampling one X from P X -it does not require labels for this step. Yet, even in the above idealized case, it is not clear which optimization criteria would allow us to prove that the resulting solution would yield good properties in terms of risk and fairness. As we will see, despite the problem in (5) being convex with Lipschitz gradient, it is crucial to control the norm of the gradient of F for good statisitcal properties of the algorithm. That goes without saying that this relaxation has its price-the smaller the regularization parameter β the less accurate the resulting solution, but the resulting dual optimization problem is easier and vice-versa.

Properties of F and π Λ ⋆ , V ⋆ . Let us summarized key properties of the objects introduced in Lemma 3.1. The first two results concern the population properties of π Λ ⋆ , V ⋆ :

Lemma 3.2 (Fairness quantification) . Let L ∈ N , ε = ( ε s ) s ∈ [ K ] ∈ [0 , 1] K , β > 0 , and π Λ ⋆ , V ⋆ be defined in Lemma 3.1. Then, U s ( π Λ ⋆ , V ⋆ , ℓ ) ⩽ ε s for all s ∈ [ K ] , ℓ ∈ [ [ L ] ] .

In words, the optimal entropic-regularized prediction function is feasible for (3), that is, it satisfies the relaxed fairness constraints as quantified by (2). Furthermore, we can show that its risk is also controlled by the regularization parameter β > 0 .

Lemma 3.3 (Risk gain) . Let L ∈ N , β > 0 , and π Λ ⋆ , V ⋆ be defined in Lemma 3.1. For any π : B ( R ) × R d → [0 , 1] that is feasible for (3) , we have

The above result is rather instructive, it quantifies the price of the introduced regularization. Intuitively, one wants to set β high enough, so that the additive term in the above bound is vanishing. Unfortunately, we cannot set it arbitrarily high, since it will introduce instabilities from the optimization perspective-the function F becomes less regular as β growth. This is summarized below.

Lemma 3.4 (Regularity of F ) . Let σ 2 def = 2 ∑ s ∈ [ K ] 1 -p s p s . The objective function in (5) is convex and its gradient is ( βσ 2 ) -Lipschitz.

As mentioned, we see that the larger the β is, the less regular the function F is, making it harder to minimize. Thus, β ⩾ 0 controls the trade-off between the optimization error and statistical bias.

Gradient of F is crucial. Let us show that the control of the gradient of F is the most important and non-trivial part that allows to demonstrate strong statistical properties of the plug-in rule derived from the above strategy.

To this end, let us introduce parametric family of prediction functions, defined for any Λ , V ⩾ 0 as

We want to show that if Λ , V ⩾ 0 is nearly stationary point of F , then π Λ , V is nearly optimal in terms of risk and its violation of the demographic parity constraint is controlled. Note that the optimization problem in (5) is constrained, thus, unless the minimum lies in the interior of the domain, we cannot hope for the gradient of F to go to zero. Instead, we introduce gradient mapping-a quantity that shares many properties of the gradient in the case of constraint optimization problem. For α > 0 ,

Our main observation is summarized in the next lemma.

Lemma 3.5. Let σ 2 def = 2 ∑ s ∈ [ K ] 1 -p s p s , L ∈ N , Λ , V ⩾ 0 , then for any α > 0 , β > 0 , the unfairness of π Λ , V satisfies

Furthermore,

Lemma 3.5 is very instructive on its own-we can obtain a good estimator of π Λ ⋆ , V ⋆ in terms of risk and unfairness by performing stochastic optimization on F and controlling the norm of gradient mapping for a suitable parameter α > 0 . The final choice of the parameter α will depend on the optimization algorithm used and will be purely theoretical. In particular, for our purposes, it is sufficient to guarantee an existence of some value of α > 0 that yields desired statistical properties. A naive approach in doing so relies on a well-known relation between F ( Λ , V ) -F ( Λ ⋆ , V ⋆ ) and ∥ G α ( Λ , V ) ∥ 2 using the Lipshitzness of the gradient of F (see e.g., Beck, 2014, Lemma 9.11). More concretely, forgetting about the constraints 1 , one has

where M is the Lipschitz constant of ∇ F . Thus, the above inequality suggests that it is sufficient to control the standard optimization error in order to control the norm of the gradient. Unfortunately this approach is deemed to fail for two reasons: the first being that we control only the squared norm of the gradient map and not the norm itself, thus loosing in the rate of convergence; the second, and more subtle reason, is the separation of the purely 'statistical' rate that depends only on the variance of the stochastic gradient and scales as 1 / √ T , with T being the number of future samples from P X , and 'optimization' rate of convergence that depends on M and the diameter of the problem and typically scales as 1 /T or even 1 /T 2 if acceleration is used.

Indeed, in our setup, Lipschitz constant M of ∇ F is not a fixed constant, but a parameter to be set-it relates to β (cf. Lemma 3.4). Ideally, seeing Lemma 3.3, we want to set β = Θ( √ T ) , leading to M = Ω( √ T ) . Thus, in view of (9), a term of the form M/ √ T appears in the convergence rate, which destroys consistency of the resulting estimator. Arguably, this is less of an issue in case of convex optimization with constant Lipschitz constant M , especially if we only want the norm to go to zero. This discussion highlights that it is crucial to keep the separation between the statistical part of the rate and the optimization part of the rate, while controlling the norm of the gradient. Lucky for us, it is known that for convex problems one can indeed control the gradient mapping keeping this separation of the rate (Allen-Zhu, 2021, Foster et al., 2019). Note that it is not the case for non-convex problems as demonstrated by Arjevani et al. (2023).

Summary of our approach and why is it different from others. Now, keeping in mind the above, rather long justification, we are in position to sketch our approach and the formal presentation is deferred to the next section. For well selected parameters β > 0 , L ∈ N , we are going to perform stochastic optimization of F , relying on the SGD3 algorithm of Allen-Zhu (2021). In order to compute the stochastic gradient of F , we are simply going to sample one P X and it appears that this stochastic gradient has a well-behaved variance (see Appendix B-C for details). To make our approach completely data-driven (or at least to understand the order of magnitude of the parameters), we will compute or bound all the oracle quantities that appear in the used optimization algorithm (essentially related to the step-size tuning). We will show that for any sufficiently small α > 0 , the term E ∥ G α ( ̂ Λ , ̂ V ) ∥ 2 is controlled and then rely on Lemma 3.5 and some additional results to demonstrate that the resulting π ̂ Λ , ̂ V possesses good statistical properties.

Remark 3.2 (On the dynamic of algorithm) . Note that for Λ = V = 0 , the corresponding

That is, the above prediction puts the most amount of mass on the atom ℓ which minimizes ( η ( x ) -ℓB/L ) 2 -the most accurate, but unfair prediction. Since our algorithm is based on a SGD -type algorithm, initialized at Λ 0 = V 0 = 0 , then we expect that during the dynamic of the algorithm, the risk of π Λ t , V t increases, while the unfairness decreases. This phenomena coincides with the intuition of post-processing-we want to gain in fairness, while sacrificing some accuracy.

As it has been already mentioned, the idea of discretizing the image of (randomized) predictions is not novel and has been successfully deployed by Agarwal et al. (2019) for an in-processing estimator and by Chzhen et al. (2020b) for a post-processing estimator. We use this insight as a building block, but significantly deviate from both algorithms. Compared to Agarwal et al. (2019), our algorithm is positioned in the realm of post-processing and even online post-processing, where i.i.d. samples

from P X comes in a stream and we do not need to store them in memory. Also, while their algorithm is partially inspired by theory, the same theory suggests that this algorithm is not computationally efficient and it relies on some black-box parts that assume perfect solutions to some optimization problems. That being said, the algorithm of Agarwal et al. (2019) seem to be the gold standard method for the generic in-processing method in this problem. Compared to Chzhen et al. (2020b), we have made a sequence of improvements. First, our setup is unawareness, which is not the case in their paper; second, our algorithm is able to handle multiple protected attributes as well as approximate fairness constraints; finally, and most importantly, we do not make black-box assumptions about having access to exact minimizers of convex problems and provide an end-to-end analysis of out approach. Let us also remark that our method cannot be considered as a simple extension of Chzhen et al. (2020b) as we rely on different phenomenons and provide a very different algorithm. On a more subjective note, we believe that our approach is a nice example of a real convex optimization problem, where the norm of the gradient plays the central role, while the optimization error in term of the objective function does not matter 2 . This is precisely the phenomena highlighted by Nesterov (2012).

4 Proposed algorithm

Algorithm 1: DP post-processing ( L, T, β, p , B, η, τ )

1: Input: discretization parameter L ⩾ 1 ; regularization β > 0 , number of stochastic gradient evaluations T ⩾ 1 ; marginal distribution p of S ; regression function η ; conditional distribution τ of S | X ; bound B > 0 on η .

2: Build uniform grid ̂ Y L over [ -B,B ] ;

3: Set parameters: σ 2 = 2 ∑ s ∈ [ K ] 1 -p s p s , M = βσ 2 ;

4: Set ( Λ , V ) ↦→ F ( Λ , V ) as defined in Lemma 3.1

5: Run a black-box optimizer A ( F, σ 2 , M, T ) on function F having access to T stochastic gradient evaluations (see (11)) with variance σ 2 and smoothness parameter M to obtain ( ̂ Λ , ̂ V ) ;

6: return π ( ̂ Λ , ̂ V ) ( · | · ) as defined in (7);

In this section, we provide all the details about the proposed algorithm in case η and τ are known. If they are unknown, these quantities are replaced by their estimates ̂ η and ̂ τ that are constructed on a separate labeled dataset. First, for Λ = ( λ ℓs ) ℓ ∈ [ [ L ] ] ,s ∈ [ K ] , V = ( ν ℓs ) ℓ ∈ [ [ L ] ] ,s ∈ [ K ] , let us provide the expression for the gradient of F :

where □ ∈ { λ, ν } and △ = 1 if □ = λ and △ = -1 otherwise. Thus, a stochastic gradient g ( Λ , V ) = ( g λ ℓs ( Λ , V ) , g ν ℓs ( Λ , V )) ℓ ∈ [ [ L ] ] ,s ∈ [ K ] of F at a point ( Λ , V ) can be computed by erasing expectation in (10), i.e. , by sampling one X ∼ P X , using the same convention as above about □ , △ :

The next result controls the variance of the above stochastic gradient.

The proposed method is summarized in Algorithm 1. It uses a black-box stochastic optimization algorithm A , that operates on a convex function F and a stochastic first-order oracle. The stochasticfirst order oracle is implemented by (11) and only requires to sample X ∼ P in an i.i.d. manner. We also pass two additional parameters to this algorithm: namely, we pass the variance σ 2 from Lemma 4.1 and the Lipschitz constant of the gradient of F from Lemma 3.4. Then one can use any such algorithm. However, as shown in Lemma 3.5, those algorithms that are tailored to control expected norm of gradient mapping are preferred. For example, one can use SGD3 of Allen-Zhu (2021) or an improved version of Foster et al. (2019) that relies on restarted accelerated SGD of Ghadimi and Lan (2012).

5 Theoretical guarantees.

Let us first provide main results for Algorithm 1 assuming that η and τ are known. Note that Algorithm 1 can rely on any optimization algorithm. We provide a complete analysis using a refined version of SGD3 algorithm of Allen-Zhu (2021) that is due to Foster et al. (2019) with additional modifications taking into account the specific structure of our problem. We state the main result in existential form and postpone all the details on the implementation of the algorithm and a primer on optimization to the supplementary material (Appendix C-D).

Theorem 5.1. Let ε = ( ε s ) s ∈ [ K ] ∈ [0 , 1] K and σ 2 = 2 ∑ s ∈ [ K ] (1 -p s ) /p s . Setting β = T 8 log 2 ( T ) and L = √ T , there exists an optimizer A to be used in Algorithm 1 that, for T larger than some absolute constant, ensures

. Furthermore, if Assumption 3.1 is satisfied and let R ⋆ def = inf h : R d → [ -B,B ] { R ( h ) : sup t ∈ R | P ( h ( X ) ⩽ t | S = s ) -P ( h ( X ) ⩽ t ) | ⩽ ε s 2 , ∀ s ∈ [ K ] } (12) and E ( π ̂ Λ , ̂ V ) def = E [ R ( π ̂ Λ , ̂ V ) ] -R ⋆ , then for the same algorithm E ( π ̂ Λ , ̂ V ) ⩽ ˜ O (( σ √ T E 1 / 2 [ ∥ ( ̂ Λ , ̂ V ) ∥ 2 ] + ∥ ε ∥ T 5 / 4 )( 1 + σ √ T ∥ ( Λ ⋆ , V ⋆ ) ∥ ) + B √ T ) .

Theorem 5.1 gives two results: the first one being on the unfairness of the proposed estimator and the second one on the risk of thereof compared to a benchmark prediction function in (12). The benchmark that we pick is rather natural, we compare to the risk of a deterministic prediction that minimizes the risk and whose unfairness is controlled by a Kolmogorov-Smirnov distance. One first main observation is that both fairness and risk decrease at the rate 1 / √ T and T is the number of unlabeled data. From our numerical experiments, we observed that we can keep the number of unlabeled data unchanged and iterate several times through them. As a result, we increase artificially T -without generating new data-which gives a significant empirical improvement. We also remark that σ is the parameter that depends on the number of groups. For example, in the case of uniform distribution of sensitive groups σ = O ( K ) . We finally remark that both bounds involve a single unknown quantity-∥ ( Λ ⋆ , V ⋆ ) ∥ , which from standard duality argument can be shown to be bounded by O (1 / min s ∈ [ K ] { ε s } ) (see e.g., Nedi´ c and Ozdaglar, 2009, Lemma 3). Thus, having this norm multiplied by T -1 / 2 is a very attractive property of the bound. It allows to set ε ≈ T -1 / 2 without damaging the parametric convergence rate.

To derive the above result, we slightly extend the analysis of Foster et al. (2019), who, relying on the SGD3 algorithm of Allen-Zhu (2021), gave an optimal algorithm that controls the expected norm of the gradient in the convex case. More concretely, we incorporate a projection step into their analysis and extend the control to the squared norm of the gradient map. Interestingly, due to our smoothing step and the choice of the parameter β , we noticed that there is no need to restart the accelerated SGD as it is done by Foster et al. (2019) because it leads to identical statistical convergence rates. The interested reader can take a closer look into the Appendix C, where all the optimization results are either recalled or derived for the sake of completeness. Finally, having a control of the squared norm of the gradient map, the proof of Theorem 5.1 follows from Lemma 3.5 and a careful and practical choice of all the parameters of the algorithm.

Extension to unknown η and τ . In this part we show that if we replace η and τ with their estimates ̂ η and ̂ τ and run DP post-processing ( L, T, β, p , B, ̂ η, ̂ τ ) algorithm with the same choice of parameters, Theorem 5.1 remains if we pay additional price for the estimation of η and τ . From now on, we assume that ̂ η and ̂ τ are provided and are trained on its own labeled data sample, while the refitting is performed on an independent stream of i.i.d. data from P X . So, we essentially treat ̂ η and ̂ τ as deterministic functions. Let us introduce a family of prediction functions

Note that for fixed matrices Λ , V , the above prediction function is fully data-driven. With this plug-in strategy, our approach becomes fully data-driven and, in Appendix E, we show that the guarantees presented in the main body still hold paying additional price for estimation of η and τ . To be more precise, we consider the plug-in version of (5), defined as

Let us denote by ̂ F , the objective function of the above problem. Thus, main interesting part is to demonstrate that a control of the gradient map of ̂ F , denoted by ∥ G ̂ F,α ( ̂ Λ , ̂ V ) ∥ , gives a control of risk and unfairness of ̂ π ̂ Λ , ̂ V , quantifying the price induced by the plug-in estimation. This is precisely the purpose of the following two results:

Lemma 5.1. Let L ∈ N , Λ , V ⩾ 0 , then for any α > 0 , β > 0 it holds that

Lemma 5.2. Let ̂ σ 2 = 2 ∑ s ∈ [ K ] E X ( p s -̂ τ s ( X )) 2 p 2 s , L ∈ N , Λ , V ⩾ 0 , then for any α > 0 , β > 0 it holds that

Note that the two above results are extensions of Lemma 3.5, where both t and η were assumed to be known. These results are following the spirit of post-processing bounds-the quality of the final approach depends on the initial estimator and the optimization algorithm used to post-process and the two errors are clearly separated. Proofs of both results with additional details and discussions is provided in the supplementary material.

6 Numerical illustration

In this section we conduct empirical study of the proposed algorithm, denoted by DP-postproc , and demonstrate its relevance in practical problems 3 . We have implemented both SGD3 of AllenZhu (2021) and an improved version by Foster et al. (2019), observing that the latter significantly outperforms the former. We also tested the approach that is suggested by the theorySGD3 and accelerated SGD, without restart and it show nearly identical performance as the restarted version of Foster et al. (2019). Thus, for numerical evaluation, we stick to the latter.

We conduct our study on two datasets: Law School dataset (Wightman (1998)) and Communities and Crime dataset (Redmond (2009)). In the Law School dataset, the aim is to predict students' GPA on a scale of 0 to 4, normalized to [0 , 1] , while in the Communities and Crime dataset, we focus on predicting the normalized number of violent crimes per population within the range of [0 , 1] . In both datasets, ethnicity is a sensitive attribute, distinguishing between white and non-white individuals or communities (majority-wise).

Our pipeline is the following: First, we randomly split the data into training, unlabeled and testing sets with proportions of 0 . 4 × 0 . 4 × 0 . 2 . We use D train = { ( x i , s i , y i ) n i =1 } to train a base (unfair) regressor to estimate η and to train a classifier to estimate τ . We use simple LinearRegression and LogisticRegression from scikit-learn for training the regressor and the classifier. Finally, we use the trained regressor and classifier to train the Algorithm 1 with D unlabeled = ( x ) n + T i = n +1 for N (note that our theory suggests that N = T is enough, but we have noticed that larger N can be more beneficial in practice) iterations. We use D test = { ( x ′ i , s ′ i , y ′ i ) i m =1 } to collect evaluation statistics.

In Figure 1 we illustrate the post-processing dynamics of our method. We have 2 plots for each test dataset: the history of risk ( R ( ̂ π ) ) and of the unfairness ( U 0 ( ̂ π ) and U 1 ( ̂ π ) ) w.r.t. number of iterations. We illustrate the convergence for ε = (2 -8 , 2 -8 ) unfairness threshold. The explicit formulas of the evaluation measures are provided in Appendix G.

communities and crime

communities and crime

law school

law school

0.07

risk  MA

unf MA : S = 0

unf MA :S = 1

0.016

risk  MA

F SE0

unf MA :S = 0

unf MA :S =1

- 0E0

0.06

unfairness

0.4

0.015

unfairness

0.25

0.05

0.3

0.014

0.20

0.04

0.2

0.013

0.10

0.15

0.03

0.1

0.012

0.05

0.0

0.011 *

0.00 *

2000

4000

num of iterations

6000

8000

10000

12000

14000

0

2000

4000

num of iterations

6000

8000

10000

12000

14000

1000

num of iterations

2000

3000

4000

5000

1000

num of iterations

2000

3000

4000

5000

Comparison with Agarwal et al. (2019). Surprisingly, we were unable to find many open source competitors that target regression with demographic parity constraint in unaware setting, even the FairLearn -a popular python package-does not deal with the demographic parity constraint in regression. The only easily accessible algorithm that deals with our problem was kindly provided by Agarwal et al. (2019) (from now on referenced as ADW). We train ADW method in two ways: we use D train and D unlabeled as training set for ADW-1, whereas for ADW-2 we use only D train . The second situation is realistic, when unlabeled data is available and unlike ADW, our approach is able to take advantage of it. We take the set { (2 -i , 2 -i ) i ∈I } , where I = { 1 , 2 , 4 , 8 , 16 } as unfairness thresholds for training both datasets. We train ADW-1 and ADW-2 for each pair of epsilons for 10 times. With our available computing power and the code provided by the authors, the algorithm runs for 13.5 hours (see Appendix G for additional details). 4

On Figure 2 we illustrate the comparison of risk and unfairness between ADW-1, ADW-2, base (LinearRegression) and our model. We plot the mean and standard deviation of risk and unfairness for each epsilon threshold on both datasets. We observe that our method is competitive or eventually outperforms ADW in both training regimes.

communities and crime

law school

0.08

DP-postproc

0.024

-+-

DP-postproc

F 0'0

ADW -1

ADW-2

0.022

ADW - 1

ADW  2

0.06

base

0.020

base

0.018

0.05

0.016

0.04

0.014

0.03

0.010

0.02

0.008

0.1

0.2

0.3

0.4

0.5

0.05

0.10

0.15

0.20

0.25

0.30

unfairness unfairness

7 Conclusion

Deriving a dual convex surrogate, we have provided a generic way to build a post-processing estimator of any off-the-shelf method that achieves the demographic parity constraint. Our approach is fully data and theory driven, revealing a key role of stationary point guarantees in stochastic convex optimization. Following Remark 2.2, we intend to extend our approach, which is general enough, to other learning problems, beyond algorithmic fairness.

Limitations. From the theoretical perspective, the knowledge of B seems to be the main limitation. While it is available for many applications, it does not have to be the case all the time. Replacing this assumption with some tail conditions, could be more realistic. From the applied perspective, it would be beneficial to further investigate stationary point guarantees for convex optimization to yield a better practical performance.

Acknowledgements The work of Gayane Taturyan has been supported by the French government under the "France 2030' program, as part of the SystemX Technological Research Institute within the Confiance.ai project.

References

Agarwal, M. Dudik, and Z. S. Wu. Fair regression: Quantitative definitions and reduction-based algorithms. In International Conference on Machine Learning , 2019.

Allen-Zhu. How to make the gradients small stochastically: Even faster convex and nonconvex sgd, 2021.

Arjevani, Y. Carmon, J. Duchi, D. Foster, N. Srebro, and B. Woodworth. Lower bounds for non-convex stochastic optimization. Mathematical Programming , 199(1-2):165-214, 2023.

Barocas, M. Hardt, and A. Narayanan. Fairness and Machine Learning . fairmlbook.org, 2018.

Beck. Introduction to nonlinear optimization: Theory, algorithms, and applications with MATLAB . SIAM, 2014.

Bhatia and C. Davis. A better bound on the variance. The American Mathematical Monthly , 107 (4):353-357, 2000. doi: 10.1080/00029890.2000.12005203.

Boyd and L. Vandenberghe. Convex optimization . Cambridge university press, 2004.

Calders, F. Kamiran, and M. Pechenizkiy. Building classifiers with independency constraints. In IEEE international conference on Data mining , 2009.

Chiappa, R. Jiang, T. Stepleton, A. Pacchiano, H. Jiang, and J. Aslanides. A general approach to fairness with optimal transport. In AAAI , 2020.

Chzhen and N. Schreuder. An example of prediction which complies with demographic parity and equalizes group-wise risks in the context of regression. arXiv preprint arXiv:2011.07158 , 2020a.

Chzhen and N. Schreuder. A minimax framework for quantifying risk-fairness trade-off in regression. arXiv preprint arXiv:2007.14265v2 , 2020b.

Chzhen, C. Denis, M. Hebiri, L. Oneto, and M. Pontil. Leveraging labeled and unlabeled data for consistent fair binary classification. In Advances in Neural Information Processing Systems , 2019.

Chzhen, C. Denis, M. Hebiri, L. Oneto, and M. Pontil. Fair regression with wasserstein barycenters. NeurIPS 2020 , 2020a.

Chzhen, M. Denis, C.and Hebiri, L. Oneto, and M. Pontil. Fair regression via plug-in estimator and recalibration. NeurIPS 2020 , 2020b.

Chzhen, C. Denis, and M. Hebiri. Minimax semi-supervised set-valued approach to multi-class classification. Bernoulli , 2021.

Denis, R. Elie, M. Hebiri, and F. Hu. Fairness guarantees in multi-class classification with demographic parity. Journal of Machine Learning Research , 25(130):1-46, 2024.

Dwork, M. Hardt, T. Pitassi, O. Reingold, and R. Zemel. Fairness through awareness, 2011. URL https://arxiv.org/abs/1104.3913 .

Feldman, S. A. Friedler, J. Moeller, C. Scheidegger, and S. Venkatasubramanian. Certifying and removing disparate impact. In International Conference on Knowledge Discovery and Data Mining , 2015.

Foster, A. Sekhari, O. Shamir, N. Srebro, K. Sridharan, and B. Woodworth. The complexity of making the gradient small in stochastic convex optimization. In Conference on Learning Theory , pages 1319-1345. PMLR, 2019.

Gao and L. Pavel. On the properties of the softmax function with application in game theory and reinforcement learning. arXiv preprint arXiv:1704.00805 , 2017.

Gaucher, N. Schreuder, and E. Chzhen. Fair learning with wasserstein barycenters for nondecomposable performance measures. In Proceedings of The 26th International Conference on Artificial Intelligence and Statistics , volume 206 of Proceedings of Machine Learning Research , pages 2436-2459. PMLR, 25-27 Apr 2023. URL https://proceedings.mlr.press/v206/ gaucher23a.html .

Ghadimi and G. Lan. Optimal stochastic approximation algorithms for strongly convex stochastic composite optimization i: A generic algorithmic framework. SIAM Journal on Optimization , 22 (4):1469-1492, 2012.

Gordaliza, E. Del Barrio, G. Fabrice, and J. M. Loubes. Obtaining fairness using optimal transport theory. In International Conference on Machine Learning , 2019.

Hardt, E. Price, and N. Srebro. Equality of opportunity in supervised learning. Advances in neural information processing systems , 29, 2016.

Jiang, A. Pacchiano, T. Stepleton, H. Jiang, and S. Chiappa. Wasserstein fair classification. Uncertainty in Artificial Intelligence Conference , 2020.

Le Gouic, J.-M. Loubes, and P. Rigollet. Projection to fairness in statistical learning. arXiv preprint arXiv:2005.11720 , 2020.

Lichman. Adult. UCI Machine Learning Repository, 2013. URL http://archive.ics.uci. edu/ml .

Lum and J. Johndrow. A statistical framework for fair predictive algorithms. arXiv preprint arXiv:1610.08077 , 2016.

Maheshwari and M. Perrot. Fairgrad: Fairness aware gradient descent. arXiv preprint arXiv:2206.10923 , 2022.

Narasimhan, A. Cotter, M. Gupta, and S. Wang. Pairwise fairness for ranking and regression. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 34, pages 5248-5255, 2020.

A Nedi´ c and A Ozdaglar. Subgradient methods for saddle-point problems. Journal of Optimization Theory and Applications , 142(1):205-228, 2009.

Nesterov. Smooth minimization of non-smooth functions. Mathematical programming , 103(3): 127-152, 2005.

Nesterov. How to make the gradients small. Optima. Mathematical Optimization Society Newsletter , (88):10-11, 2012.

Redmond. Communities and Crime. UCI Machine Learning Repository, 2009. DOI: https://doi.org/10.24432/C53W3X.

Wightman. Lsac national longitudinal bar passage study. lsac research report series. 1998. URL https://api.semanticscholar.org/CorpusID:151073942 .

Zafar, I. Valera, M. Gomez Rodriguez, and K. Gummadi. Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In International Conference on World Wide Web , 2017.

Zemel, Y. Wu, K. Swersky, T. Pitassi, and C. Dwork. Learning fair representations. In International Conference on Machine Learning , 2013.

Zhao. Costs and benefits of fair regression. arXiv preprint arXiv:2106.08812 , 2021.

A Proofs for results in Section 3

First we explicit the first order optimality conditions for the problem in (5).

Lemma A.1. Let ( Λ ⋆ , V ⋆ ) ⩾ 0 be any minimizer of (5) and π ⋆ = π Λ ⋆ , V ⋆ be defined in (6) . Then, there exist Γ = ( γ ℓs ) ℓ ∈ [ [ L ] ] ,s ∈ [ K ] , Γ ′ = ( γ ′ ℓs ) ℓ ∈ [ [ L ] ] ,s ∈ [ K ] -element-wise non-negative matrices such that

where γ ℓ = ( γ ℓs ) s ∈ [ K ] , γ ℓ = ( γ ℓs ) s ∈ [ K ]

Proof. We first observe that the optimization problem in (5) is convex and smooth. Thus, Karush-Kuhn-Tucker conditions are sufficient for optimally. Furthermore, since Slatter's condition is satisfied, the latter is also necessary, as the strong duality holds. In particular, there exist Γ = ( γ ℓs ) ℓ ∈ [ [ L ] ] ,s ∈ [ K ] , Γ ′ = ( γ ′ ℓs ) ℓ ∈ [ [ L ] ] ,s ∈ [ K ] -element-wise non-negative matrices such that

To conclude, it is sufficient to evaluate the gradient on F , whose expression is given in (10) and use the definition of π ⋆ . ■

Proof of Lemma 3.1. To prove this result, we introduce the Lagrangian for the problem in (4).

where we used the fact, that using the definition of U s , for any randomized prediction function π , we can write

Thus, denoting by ( ⋆ ) the value in (4), we have

where the second equality holds thanks to Sion's minmax theorem. Let us solve the inner minimization problem on the right-hand-side. We can write

Thus, using the variational representation of LSE β , recalled in Lemma F.1, we have

and the optimum in the above problem for every Λ , V ⩾ 0 is achieved by π Λ , V , defined in (7). Thus, we have

■

The proof is concluded.

Proof of Lemma 3.2. As shown in (15), for any randomized prediction function π , we can write

Our goal is to show that π ⋆ satisfies the required fairness constraints. We are going to rely on Lemma A.1 Subtracting the first equation in (14) from the second one, we deduce that γ ℓ + γ ′ ℓ = 2 ε . Since γ ℓ , γ ′ ℓ ⩾ 0 , then we conclude that γ ℓs , γ ′ ℓs ∈ [0 , 2 ε s ] . The above implies that

as claimed.

Proof of Lemma 3.3. Fix some randomized prediction function π that is feasible for the problem in (3). In particular, it must be supported on ̂ Y . Then, we can bound its negative risk as follows

for (a) we used the assumption that π is fair ( i.e., U s ( π, ℓ ) ⩽ ε s ), which due to the fact that Λ ⋆ , V ⋆ ⩾ 0 implies

since every term in the summation is non-negative; (b) uses the fact that LSE β ( w ) ⩾ ⟨ w , p ⟩ for any probability vector p (see Lemma F.1 for details); (c) relies on the variational representation of the LSE β , recalled in Lemma F.1 and the definition of π ⋆ ( · | X ) ; (d) uses the uniform bound on the entropy; (e) the last equality relies on the complementary slackness condition (14) of Lemma A.1. It ensures that

implying that

The proof is concluded.

■

Proof of Lemma 3.5. Fix arbitrary Λ , V ⩾ 0 and consider π Λ , V , defined in (7). To ease the notation, in this proof, we write π instead of π Λ , V .

Part I. Let us first recall the definition of the gradient map G α given in (8). We have the following expression

where ( · ) + is to be understood entry-wise. Observing that for any α, a ⩾ 0 and b ∈ R , we have

we deduce that

Relying on (10) and the expression for π in (7), we observe that

and that U s ( π, ℓ ) = | E [ π ( ℓ | X ) t s ( X )] | as it is shown in (15). Using the fact that ( | a | -c ) 2 + = ( -a -c ) 2 + +( a -c ) 2 + for all a ∈ R and c ⩾ 0 , we deduce from above

Thus, we have shown

and (19) yields the claim.

Part II. We note that π ( Λ , V ) is a unique solution to

where L is the Lagrangian defined in (16). Furthermore, min π L ( π, Λ , V ) = -F ( Λ , V ) = L ( π ( Λ , V ) , Λ , V ) . Hence,

For the sake of simplicity let us denote by u def = ( Λ , V ) ∈ R 2 K (2 L +1) and recall the definition of gradient mapping G α given in (8). For any j ∈ [2 K (2 L +1)] and α > 0 , we have

To bound ⟨ u , ∇ F ( u ) ⟩ , let us examine each term of the scalar product. Denoting by ˜ u def = u -α ∇ F ( u ) ∈ R 2 K (2 L +1) , for any j ∈ [2 K (2 L +1)] and α > 0 , we have

Thus, it holds that

where the last inequality follows from Lemma F.5 that bounds ∥∇ F ( u ) ∥ .

To conclude the proof, we use the fact that It remains to observe that min Λ , V F ( Λ , V ) = -R β ( π ( Λ ⋆ , V ⋆ ) ) , the bound (19) on ∥ ( -∇ F ( Λ , V )) + ∥ and the fact that R β ( π ) ⩽ R ( π )+ log(2 L +1) β for all π . ■

B Bound on the variance of the stochastic gradient and its' smoothness

Proof of Lemma 4.1. We have

where the first inequality follows from the expressions of g Λ , V and ∇ Λ , V F ( Λ , V ) , and the fact that Var( X + a ) = Var( X ) ⩽ E [ X 2 ] ; the second inequality follows from the facts that ∥ ( a i b j ) i,j ∥ 2 2 = ∥ a ∥ 2 2 ∥ b ∥ 2 2 ⩽ ∥ a ∥ 2 1 ∥ b ∥ 2 2 and that ∥ σ ( · ) ∥ 1 = 1 ; and the last inequality follows from Lemma F.4. ■

Proof of Lemma 3.4. The goal of this proof is to show that the gradient of ( Λ , V ) ↦→ F ( Λ , V ) is M -Lipschitz. To this end, we first introduce some, rather heavy, but convenient, notation which will allow us to derive the announced result.

Introducing notation. We first vectorize the variables ( Λ , V ) and express them as

Furthermore, for each x ∈ R d , we introduce a matrix A ( x ) ∈ R (2 L +1) × 2 K (2 L +1) defined as

as well as

Hessian of F in the introduced notation. With the above introduce notation, we can express the function F as

That is, F is obtained from the LSE β by a point-wise affine transformation of the coordinates plus a linear term. Chain rule yields the following expressions for the Hessian of F :

Bounding the operator norm of the Hessian of F . To conclude the proof, we provide a uniform upper bound on the operator (spectral) norm of the Hessian of F . Using the Jensen's inequality and the fact that the operator norm is subordinate, we deduce that

Lemma F.2, implies that ∥∇ 2 LSE β ( A ( X ) z -b ( X )) ∥ op ⩽ β almost surely and for all z . Thus, it remains to bound E X ∥ A ( X ) ∥ 2 op to conclude the proof. To this end, consider a vector u , expressed 'block-wise' as

Using the definition of the operator norm and the expression for A ( X ) , we deduce that

where the last inequality combines the Cauchy-Schwartz inequality and the fact that ∥ v -w ∥ 2 2 ⩽ 2( ∥ v ∥ 2 2 + ∥ w ∥ 2 2 ) for all v , w ∈ R m . The proof is concluded using Lemma F.4 to bound E X ∥ t ( X ) ∥ 2 2 . ■

LemmaB.1 (Price of discretization) . Let Assumption 3.1 be satisfied. Let β, B > 0 , L ∈ N . Consider

Then, it holds that

Proof. Let us assume that R ⋆ = E ( h ⋆ ( X ) -η ( X )) for some h ⋆ : R d → [ -B,B ] . If it is not the case, the standard argument based on the minimizing sequence yields the same result.

Consider an operator T L , which maps a deterministic classifier h : R d → [ -B,B ] onto a deterministic classifier T L ( h ) : R d → ̂ Y L , which is defined point-wise as follows

where ⌊ a ⌋ is the closest integer smaller or equal to a ∈ R in absolute value. Notice, that for any ℓ ∈ {-L, . . . , L -1 } and any x ∈ R d , we have

Moreover,

Since h ⋆ satisfies ( ε / 2) -fairness constraints, one checks that for all ℓ ∈ [ [ L ] ] , s ∈ [ K ]

That is, T L ( h ⋆ ) is feasible for the problem in (4). Therefore, Lemma 3.3 implies that

Furthermore, since | T L ( h ⋆ )( x ) -h ⋆ ( x ) | ⩽ 1 /L and | η ( x ) -h ⋆ ( x ) | ⩽ 2 B , we have

The proof is concluded.

■

C Additional details on the algorithm

In this part of the appendix, we provide the analysis for the proposed algorithm. First, we introduce required notation and recall a result of Foster et al. (2019), who provided an algorithm for convex stochastic optimization. The provided algorithm is a refined version of the SDG3 algorithm of Allen-Zhu (2021). We note that Foster et al. (2019) give a control of the expected norm of a gradient, while we require a control of the expected squared norm of the gradient mapping. We introduce projection to the algorithm of Foster et al. (2019) based on the original algorithm of Ghadimi and Lan (2012) and provide a control of the expected squared norm of the gradient mapping of the final estimated solution.

The setup and notation.

Consider f : R d ×Z → R , such that w ↦→ f ( w , z ) is convex for each z ∈ Z . Let W ⊂ R d be a closed convex set. Let

for some probability distribution P on Z . In what follows, we assume that

always exists.

Assumption C.1. We assume that F is M -smooth and the variance of ∇ w f ( w , z ) is bounded. That is, for some M > 0 and σ > 0

Let us also define gradient mapping as

Let Proj W ( · ) be the Euclidean projection onto closed convex W .

C.1 Some known results.

We start by introducing the original AC-SA algorithm of Ghadimi and Lan (2012) and recall some of their results for the sake of completeness.

Algorithm 3: AC-SA 2 ( F, w 0 , µ, M, T )

1: Input: function F ; initial vector w 0 ; parameters µ, M ⩾ 0 ; number of iterations T ⩾ 1

3: w 2 ← AC-SA ( F, w 1 , µ, M, T 2 )

2: w 1 ← AC-SA ( F, w 0 , µ, M, T 2 )

4: return w 2

Algorithm 4: SGD3-refined ( F, w 0 , µ, M, T )

1:

Input: function F ; initial vector w 0 ; parameters 0 < µ ⩽ M ; number of iterations T ⩾ Ω ( M µ log 2 M µ )

2: F (0) ( w ) ← F ( w ) + µ 2 ∥ w -w 0 ∥ 2 ; ̂ w 0 ← w 0 ; µ 0 ← µ

3: for j = 1 to J = ⌊ log M µ ⌋ do

4: ̂ w j ← AC-SA 2 ( F ( j -1) , ̂ w j -1 , µ j -1 , 2( M + µ ) , T J )

5: µ j ← 2 µ j -1

6: F ( j ) ( w ) def = F ( j -1) ( w ) + µ j 2 ∥ w -̂ w j ∥ 2

7: end for

8: return ̂ w J

Theorem C.1. (Ghadimi and Lan, 2012, Proposition 9) Let w ⋆ ∈ arg min w ∈ W F ( w ) , w 0 ∈ W a starting vector. If F is µ -strongly convex and T ⩾ 1 then with

AC-SA ( F, w 0 , µ, M, T ) , defined in Algorithm 2, outputs ̂ w T satisfying

Foster et al. (2019) propose another version of AC-SA , called AC-SA 2 , which resets the stepsize halfway through the process.

Lemma C.1. (Foster et al., 2019, Lemma 1) Let W = R d , w ⋆ ∈ arg min w ∈ W F ( w ) , w 0 ∈ R d a starting vector. If µ > 0 , M ⩾ 0 and T ⩾ 1 then AC-SA 2 ( F, w 0 , µ, M, T ) , defined in Algorithm 3, outputs ̂ w satisfying

Remark C.1. Foster et al. (2019) do not consider constrained optimization throughout their work. However, the proof of Lemma C.1 follows analogous arguments.

Foster et al. (2019) also introduce a refined version of algorithm SGD3 of Allen-Zhu (2021).

In what follows, we will show that Algorithm 4, after T evaluations of the stochastic gradient, produces a point ̂ w such that E ∥ G F,α ( ̂ w ) ∥ 2 is controlled. This is a, rather mild, extension of Foster et al. (2019) and Allen-Zhu (2021).

C.2 Control of the expected squared norm

Most of the proof techniques are already present in the original contribution of Allen-Zhu (2021) and Foster et al. (2019), we slightly extend their proof, introducing modifications related to the control of the squared norm and the projection step. For some J ⩾ 1 , to be fixed later on, introduce

By construction, F ˜ µ is ˜ µ def = ∑ J j =1 µ j -strongly convex and ( M + ˜ µ ) -smooth. Let us also define F (0) def = F ( w ) and F ( j ) ( w ) def = F ( j -1) ( w ) + µ j 2 ∥ w -̂ w j ∥ 2 , for j = 1 , 2 , . . . , J . We will use the following results of Allen-Zhu (2021).

Lemma C.2. (Allen-Zhu, 2021, Lemma 2.3) Let ˜ F be an ˜ M -smooth and ˜ µ -strongly convex function. Let w , w ′ ∈ W and w + = w -α · G ˜ F,α ( w ) . For any α ∈ ( 0 , 1 ˜ M ] , we have

Lemma C.3. (Allen-Zhu, 2021, Lemma 5.1) Consider F ˜ µ and w ⋆ ˜ µ as defined in (22) and w ∈ W . For any α ∈ ( 0 , 1 M + ˜ µ ] , we have

Claim C.1. (Allen-Zhu, 2021, Claim 6.2) Suppose for every j = 1 , . . . , J the iterates ̂ w j of Algorithm 4 satisfy

then,

In addition to Claim C.1, we prove the following lemma.

Lemma C.4. Suppose for every j = 1 , . . . , J , µ j = 2 µ j -1 and the iterates ̂ w j of Algorithm 4 satisfy

then,

Proof. Let P j def = ∑ j t =1 µ t ∥ ∥ w ⋆ j -̂ w t ∥ ∥ , yielding that P J = ∑ J j =1 ( P j -P j -1 ) , with the agreement that P 0 = 0 . Cauchy-Schwartz inequality gives

Since P j is non-decreasing, to bound the above quantity, it suffices to bound each increment of the form P j -P j -1 . One can write

where (a) follows from the definition of P j , (b) from reverse triangle inequality and (c) uses triangle inequality and the fact that ∑ j -1 t =1 µ t ⩽ µ j as µ j = 2 µ j -1 . Therefore, using the fact that ( a + b ) 2 ⩽ 2 a 2 +2 b 2 , we deduce from the above that

Taking the expectation and applying Claim C.1(a) and Claim C.1(b), the latter is bounded as

Plugging (24) into (23) yields the claimed bound.

■

Remark C.2. Notice, that in Algorithm 4 we apply AC-SA 2 to F ( j -1) with starting point ̂ w j -1 and T/J iterations. Since F ( j -1) is M + ∑ j -1 t =1 µ t ⩽ 2 M -smooth and µ j -1 -strongly convex, applying Lemma C.1 and Claim C.1(b), we get E [ F ( j -1) ( ̂ w j ) -F ( j -1) ( w ⋆ j -1 )] ⩽ δ j and

We are in position to prove the main ingredient of this section.

Theorem C.2 (Control of the expected squared norm) . Let w ⋆ ∈ arg min w ∈ W F ( w ) , w 0 ∈ R d a starting vector. When µ ∈ (0 , M ] and T > 2 11 / 4 √ M µ ⌊ log 2 M µ ⌋ , then for α = 1 2 J +2 µ , with J = ⌊ log 2 M µ ⌋ , SGD3-refined ( F, w 0 , µ, M, T ) outputs ̂ w satisfying

Proof. Part I. At first, let us assume that F is µ 0 -strongly convex. Since the definition of F satisfies the definition given in (22) with J -1 , applying Lemma C.3 and using the fact that ( a + b ) 2 ⩽ 2 a 2 +2 b 2 , we get that for any α ∈ ( 0 , ( M + ∑ J -1 j =1 µ j ) -1 ] , it holds that

Note, that due to definition of µ j , we have ∑ J -1 j =1 µ j ⩽ M , thus, the derived inequality holds for any α ∈ ( 0 , (2 M ) -1 ] ⊂ ( 0 , ( M + ∑ J -1 j =1 µ j ) -1 ] . Lemma C.4 provides a control of the second term of (25). To control the first term, let us apply Lemma C.2 with F ( J -1) and w = w ′ = ̂ w J , getting α 2 ∥ ∥ G F ( J -1) ,α ( ̂ w J ) ∥ ∥ 2 ⩽ F ( J -1) ( ̂ w J ) -F ( J -1) ( ̂ w + J ) ⩽ F ( J -1) ( ̂ w J ) -F ( J -1) ( w ∗ J -1 ) , ∀ α ∈ (0 , 1 2 M ] . Meaning, that ∥ ∥ G F ( J -1) ,α ( ̂ w J ) ∥ ∥ 2 ⩽ 2 δ J α . Let us recall, that J = ⌊ log 2 M µ 0 ⌋ and µ J = 2 J µ 0 ⩽ M ⩽ 2 µ J . Hence, choosing α = 1 4 µ J and substituting the derived bound into (25), we deduce that

Now, let us substitute the bound on δ j from Remark C.2 and replicate the steps of Foster et al. (2019) to control the above. We get

where the last inequality comes from the facts that ∑ J j =1 1 2 j -1 ⩽ 2 and ∑ J j =2 δ j -1 µ j -1 ⩽ ∑ J j =1 δ j µ j ⩽ 1 µ 2 0 ∑ J j =1 µ j δ j . Rearranging the terms and multiplying both sides by 144 J , we get

Choosing T > 2 11 / 4 J √ M µ 0 ensures that 1 1 -2 10 M 2 J 4 µ 2 0 T 4 ⩽ 2 . Finally, substituting the derived bounds and the value of J = ⌊ log 2 M ⌋ , we conclude that

µ 0

Part II. When F is not strongly convex, let F µ ( w ) def = F ( w ) + µ 2 ∥ w -w 0 ∥ 2 and w ⋆ µ ∈ arg min w { F µ ( w ) } . Applying (26) and Lemma C.3 with J = 1 and ̂ w 1 = w 0 , we get

Since µ 2 ∥ w ⋆ -w 0 ∥ 2 -µ 2 ∥ ∥ w ⋆ µ -w 0 ∥ ∥ 2 = ( F µ ( w ⋆ ) -F ( w ⋆ )) + ( F ( w ⋆ µ ) -F µ ( w ⋆ µ )) ⩾ 0 , then ∥ ∥ w ⋆ µ -w 0 ∥ ∥ 2 ⩽ ∥ w ⋆ -w 0 ∥ 2 . The proof is concluded. ■

Remark C.3. Notice, that in Algorithm 4 we apply AC-SA to F ( j -1) with starting point ̂ w j -1 and T/J iterations. Since F ( j -1) is M + ∑ j -1 t =1 µ t ⩽ 2 M -smooth and µ j -1 -strongly convex, applying Lemma C.1 and Claim C.1(b), we get E [ F ( j -1) ( ̂ w j ) -F ( j -1) ( w ⋆ j -1 )] ⩽ δ j and

Theorem C.3 (Control of the expected squared norm) . Let w ⋆ ∈ arg min w ∈ W F ( w ) , w 0 ∈ R d a starting vector. When µ ∈ (0 , M ] and T > 4 √ M µ ⌊ log 2 M µ ⌋ , then for α = 1 2 J +2 µ , with J = ⌊ log 2 M µ ⌋ , SGD3-refined ( F, w 0 , µ, M, T ) with AC-SA outputs ̂ w satisfying

Proof. Part I. At first, let us assume that F is µ 0 -strongly convex. Applying Lemma C.3 and using the fact that ( a + b ) 2 ⩽ 2 a 2 +2 b 2 , we get

Lemma C.4 provides a control of the second term of the above inequality. To control the first term, let us apply Lemma C.2 with F ( J -1) and w = w ′ = ̂ w J , getting α 2 ∥ ∥ G F ( J -1) ,α ( ̂ w J ) ∥ ∥ 2 ⩽ F ( J -1) ( ̂ w J ) -F ( J -1) ( ̂ w + J ) ⩽ F ( J -1) ( ̂ w J ) -F ( J -1) ( w ∗ J -1 ) , ∀ α ∈ (0 , 1 2 M ] . Meaning, that ∥ ∥ G F ( J -1) ,α ( ̂ w J ) ∥ ∥ 2 ⩽ 2 δ J α . Let us recall, that J = ⌊ log 2 M µ 0 ⌋ and µ J = 2 J µ 0 ⩽ M ⩽ 2 µ J . Hence, choosing α = 1 4 µ J and substituting the derived bound into (27), we deduce that

Let us substitute the bound on δ j from Remark C.2 to control the above. We get

Rearranging the terms and multiplying both sides by 144 J , we get

Choosing T > 4 J √ M µ 0 ensures that 1 1 -8 MJ 2 µ 0 T 2 ⩽ 2 . Finally, substituting the derived bounds and the value of J = ⌊ log 2 M µ 0 ⌋ , we conclude that

Part II. When F is not strongly convex, let F µ ( w ) def = F ( w ) + µ 2 ∥ w -w 0 ∥ 2 and w ⋆ µ ∈ arg min w { F µ ( w ) } . Applying (26) and Lemma C.3 with J = 1 and ̂ w 1 = w 0 , we get

D Proofs of statistical guarantees

In order to derive statistical guarantees for the proposed method, we are going to instantiate the extension provided in the previous appendix.

Proof of Theorem 5.1. Let us instantiate Theorem C.2. According to Lemma 4.1 and Lemma 3.4 we have that σ 2 = 2 ∑ s ∈ [ S ] 1 -p s p s and M = βσ 2 . Setting β = T 8 log 2 T and µ = σ 2 /β , ensures that µ ⩽ M and that T > 4 √ M µ ⌊ log 2 M µ ⌋ = T log 2 T ⌊ log 2 T 8 log 2 T ⌋ , ∀ T ⩾ 2 . For T larger than some large enough absolute constant, the conditions of Theorem C.2 are satisfied for the function F .

Fairness guarantee. Theorem C.2 yields

Therefore, we have shown that

Hence, the first part of Lemma 3.5 implies the fairness guarantee.

Fairness guarantee with AC-SA . Theorem C.3 yields

Therefore, we have shown that

Hence, the first part of Lemma 3.5 implies the fairness guarantee.

Risk guarantee. The second part of Lemma 3.5 states that

Taking the expectation and applying the Cauchy-Schwartz inequality, we obtain

Recalling the value of α = 1 2 J +2 µ J ⩽ 1 2 M from Theorems C.2 and C.3 and the fact that M = βσ 2 , we get ασ ⩽ 1 2 βσ . Finally, applying (29)-(30) and setting L = √ T , we obtain

Above combined with Lemma B.1 yields the claimed bound.

E Unknown η and τ

In this section we consider the case, when η and τ are unknown and estimated by ̂ η and ̂ τ . We denote by ̂ t ( x ) def = 1 -̂ τ ( x ) p and ̂ r ℓ ( x ) def = ( ̂ η ( x ) -ℓB L ) 2 . We consider the plug-in version of (5), defined as

Let us denote by ̂ F , the objective function of the above problem and introduce

The gradient of ̂ F is given for any Λ , V ⩾ 0 by

for ℓ ∈ [ [ L ] ] , s ∈ [ K ] . Let us denote by ̂ g ( Λ , V ) the stochastic gradient of ̂ F , defined as

for X ∼ P X and ℓ ∈ [ [ L ] ] , s ∈ [ K ] . We also define, by the analogy with the main body, a family of plug-in estimators

Our goal is to derive analogous optimization results for the new plug-in objective. Inspecting the proofs of Lemma 4.1 and Lemma 3.4, which bound variance of and the Lipschitz constant of the gradient respectively, we observe that those proofs only depend on the nature of ̂ t via Lemma F.4. In particular, the key quantity to control is

Before, when we assumed the perfect knowledge of τ , the above was controlled by the Bhatia-Davis inequality, leveraging the fact that variance of τ s ( X ) appears in the numerator. It is no longer the case here. However, if one can build calibrated estimators, that is, estimators for which E X [ ̂ τ s ( X )] = p s , the same machinery is applicable. In any case, even without requiring calibrated predictions, one can have a reasonable control of ̂ σ 2 building sufficiently accurate estimator ̂ τ s .

That being said, results of Lemma 4.1 and Lemma 3.4 generalize line-by-line, replacing σ 2 by ̂ σ 2 and give

Moreover, the result of Lemma F.5 generalizes as well and gives sup Λ , V ⩾ 0 ∥ ∥ ∥ ∇ Λ , V ̂ F ( Λ , V ) ∥ ∥ ∥ ⩽ ̂ σ .

As in (19), we can show that

where the gradient mapping G ̂ F,α is defined by analogy with G α = G F,α , discussed in the main body.

Considering the SGD3 algorithm with the same choice of parameters, but replacing σ 2 by ̂ σ 2 , results in a control of ∥ ∥ ∥ G ̂ F,α ( ̂ Λ , ̂ V ) ∥ ∥ ∥ .

Proof of Lemma 5.1. Fix Λ , V ⩾ 0 . To ease the notation, we write ̂ π to denote ̂ π Λ , V within this proof. Similarly to the proof of (21) from Lemma 3.5, one shows that for all Λ , V ⩾ 0

Recalling that U s ( ̂ π, ℓ ) = | E [ ̂ π ( ℓ | X ) t s ( X )] | , triangle's inequality combined with the above yields

Cauchy-Schwartz inequality gives

We conclude using (34).

■

Proof of Lemma 5.2. Fix Λ , V ⩾ 0 . To ease the notation, we write ̂ π def = ̂ π Λ , V and π ⋆ def = π Λ ⋆ , V ⋆ , within this proof.

As in the second part of the proof of Lemma 3.5, we have

Furthermore, since ∥∇ LSE β ( · ) ∥ 1 ≡ 1 , we have

and | ̂ R β ( ̂ π ) - R β ( ̂ π ) | ⩽ E [max ℓ ∈ [ [ L ] ] {| r ℓ ( X ) -̂ r ℓ ( X ) |} ] . The last two displays combined with (35), gives

Observe that min Λ , V ⩾ 0 F ( Λ , V ) = -R β ( π ⋆ ) . Using triangle's inequality and the fact that max ℓ ∈ [ [ L ] ] ∥ λ ℓ -ν ℓ ∥ ⩽ √ 2 ∥ ( Λ , V ) ∥ , we conclude recalling that R ( π )+ log(2 L +1) β ⩾ R β ( π ) ⩾ R ( π ) for any randomized prediction function. ■

F Auxilliary results

In this appendix, we collect some standard auxiliary results, that are used to derive main claims of the paper.

Lemma F.1 (Boyd and Vandenberghe (2004)) . It holds that

where ∆ is the probability simplex in R m and Ψ( p ) = ∑ m i =1 p i log( p i ) . Furthermore, -Ψ( p ) ∈ [0 , log( m )] and the optimum in the above optimization problem is achieved at p ⋆ = σ ( β w ) .

Lemma F.2 (Gao and Pavel (2017)) . Let a = ( a 1 , · · · , a m ) and β > 0 . Define log-sum-exp and softmax functions respectively as

The LSE property is as follows

Moreover, σ ( β a ) = ∇ LSE β ( a ) , and σ ( β a ) is β -Lipschitz.

Lemma F.3 (Bhatia and Davis (2000)) . Let m and M be the lower and upper bounds, respectively, for a set of real numbers a 1 , · · · , a n , with a particular probability distribution. Let µ and σ 2 be respectively the expected value and the variance of this distribution. Then the Bhatia-Davis inequality states:

Proof. We have E X [ τ s ( X )] = p s and 0 ⩽ τ s ( X ) ⩽ 1 almost surely. Using Bhatia-Davis inequality written in Lemma F.3, we deduce that

The proof is concluded.

Proof. By Jensen's inequality

Recalling the definition of g Λ , V , given in (11), we have

where the last inequality comes from the proof of Lemma 4.1.

The proof is concluded.

■

■

G Additional details on experiments

Evaluation measures. We use D test = { ( x ′ i , s ′ i , y ′ i ) } i m =1 to collect the following statistics of any (randomized) prediction π

which correspond to the empirical risk and the empirical group-wise unfairness quantified by the Kolmogorov-Smirnov distance of a randomized. We note that our classifier is supported on a finite grid, thus all the integrals involved transform into weighted sums.

Agarwal et al. (2019) build multi-class classifiers h k : R d ↦→ Θ , where Θ is some finite grid over R and k = 1 , . . . , K , that come with weights ( w 1 , . . . , w K ) ⊤ such that w k ⩾ 0 and ∑ K k =1 w k = 1 . Then, they build a randomized classifier π ( · | · ) such that supp( π ( · | x )) = Θ and for each θ ∈ Θ

Thus, integrals appearing in ̂ R and ̂ U s reduced to finite sums for both methods.

Additional details on the experiments on Communities and Crime and Law School datasets. Communities and Crime dataset has 1994 instances, however we use 1984 examples with 120 features after preprocessing. Law School dataset has 20649 instances, thus we use a smaller sub-sample of 2000 points with 11 features after preprocessing.

We take the sets of unfairness thresholds { (2 -i , 2 -i ) i ∈I } , where I = { 1 , 2 , 4 , 5 , 6 , 8 , 16 , 32 , 128 , 512 } for Communities and Crime dataset, and I = { 0 , 1 , 2 , 4 , 5 , 6 , 8 , 16 , 32 , 64 , 128 } for Law School dataset. We train Communities and Crime dataset for N=15000 iterations and Law School dataset for N=5000 iterations for each pair of epsilons. We use parameters L = √ T , β = √ T log √ T and B = 1 for both datasets. We repeat the aforementioned pipeline 10 times to ensure more reliable statistical summary.

Discussion on other algorithms. We conduct additional experiments to observe the behaviors of the more straightforward algorithms discussed in Appendix C. We illustrate the comparison in Figure 3. In conclusion, all of the algorithms perform similarly in the middle to high unfairness regime, while those based on SGD3 are more stable in the low unfairness (high fairness) regime.

050'0

0.045

0.040

communities and crime

ACSA2

SGD3 + ACSA2

base

GOS

ACSA

SG03 + ACSA

0.014 -

0.013

law school

ACSA2

base

dos

ACSA

SG03 + ACSA

SGD3 + ACSA2

0.035

0.012

F 0E0°0

0.011

0.010

0.020

600°0

T'0

Z'0

E'0

0.4

0.5

00°0

S0'0

0.10

0.15

0.20

0.25

0E'0

unfairness unfairness

Additional experiments on Adult dataset. We conduct further experiments on Adult dataset (Lichman (2013)). Classically, Adult is used for classification, however we use it to predict individual's age on a scale of 0 to 100 , normalized to [0 , 1] . We factor in sex as a sensitive attribute, distinguishing between male and female individuals. Adult dataset has 48842 instances, however we clean and preprocess it, and use a smaller sub-sample of 2000 points with 8 features throughout our experiments.

The pipleline of the experiments is the same as the one for Law School and Communities and Crime datasets in the main body. We randomly split the data into training, unlabeled and testing sets

with proportions of 0 . 4 × 0 . 4 × 0 . 2 . We use D train = { ( x i , s i , y i ) n i =1 } to train a base (unfair) regressor to estimate η and to train a classifier to estimate τ . We use simple LinearRegression and LogisticRegression from scikit-learn for training the regressor and the classifier, and give them to Algorithm 1 with D unlabeled = ( x ) n + T i = n +1 for N = 10000 iterations. We use D test = { ( x ′ i , s ′ i , y ′ i ) } i m =1 to collect statistics. We take the sets of unfairness thresholds { (2 -i , 2 -i ) i ∈I } , where I = { 0 , 1 , 2 , 4 , 5 , 6 , 8 , 16 , 32 , 64 , 128 } for. As in the experiments in the main body, we set L = √ T , β = √ T/ log √ T and B = 1 . We repeat the pipeline 10 times.

Wecompare our method with the ADW method. We train ADW 2 times: we use D train and D unlabeled as training set for ADW-1, whereas for ADW-2 we use only D train . We take the set { (2 -i , 2 -i ) i ∈I } , where I = { 1 , 2 , 4 , 8 , 16 } as unfairness thresholds for training both datasets. We train ADW-1 and ADW-2 for each pair of epsilons for 10 times.

In Figure 4 we illustrate the convergence of the risk and the unfairness for convergence for ε = (2 -8 , 2 -8 ) unfairness threshold. We also illustrate the comparison of our model with ADW-1, ADW-2 and base models.

adult

adult

adult

risk-MA

unf-MA :S = 0

DP-postproc

0.020

0.10

unf-MA :S =1

0.019

ADW -1

ADW-2

0.018

base

0.019

80°0

unfairness

90°0

risk

0.017

C

0.018

0.016

0.04

0.017

0.015

0.02

0.016

00°0

0.014

0

2000

num ofiterations

4000

6000

8000

10000

0

2000

num ofiterations

4000

6000

8000

10000

0.000

0.025

0.050

0.075

unfairness

0.100

0.125

0.150

0.175

Running time. Additional details about training time for Communities and Crime , Law School and Adult datasets are presented in Table 1.

Additional experiments on a synthetic dataset. We conduct an additional experiment to demonstrate the results of Algorithm 1 in the case of multiple sensitive attributes. We generate a synthetic dataset D n = ( X i , S i , y i ) n i =1 of n = 2000 points, where ( X i ) n i = ( X i 1 , X i 2 , X i 2 ) n i ∼ N (0 , 1) . We choose S i = 0 if X i 1 ⩽ -0 . 7 , S i = 1 if X i 1 < 0 , S i = 2 if X i 1 < 0 . 7 and S i = 4 if X i 1 ⩾ -0 . 7 . For i ∈ [ n ] , we generate y i = 4 ∑ 3 j =1 X ij + X i 1 + ξ i , where ξ = ( ξ i ) n i ∼ N (0 , 1) . We split D n into train , unlabeled and test datasets with proportions of 0 . 4 × 0 . 4 × 0 . 2 . We use ( X train , y train ) to train the base estimator, ( X train , S train ) to train the classifier and X unlab to train the fair regression model. We evaluate our model on test dataset. In Figure 5 we illustrate the distributions of the predictions (scaled to [ -1 , 1] ) of the fair and base models.

This experiment is for visual representation of the Algorithm 1 in the case of multiple sensitive attributes, thus we do not collect further statistics.

Fair model

Basemodel

3.0

S=0.0

S=0.0

2.5

S=1.0

S=1.0

S=2.0

2.5

S=2.0

2.0

S=3.0

S=3.0

2.0

1.5

1.5

1.0

1.0

0.5

0.5

0.0

0.0

0.6

0.4

0.2

0.0

0.2

0.4

0.6

0.6

0.4

0.2

0.0

0.2

0.4

0.6

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We translate the fairness problem to a smooth and convex problem as stated in Lemma 3.1. Theorem 5.1 gives the main theoretical control on fairness and risk and Section 6 illustrates the performance of the method and a comparison to a state-of-the-art method.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: A limitation section is included in the paper.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We extensively developed our methodology in the main body and decided to postpone all the proofs to the Appendix.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: All the experiments are reproducible. We provide the link to the source code in Section 6.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: All the datasets that are used are benchmark datasets. Moreover, as previously mentioned, we point to a GitHub link in Section 6 to make our experiments reproducible.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Section 6 gives all necessary information on the datasets splits and on the tuning parameters.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: All our experiments include and illustrate the standard deviations reported over 10 repetitions.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We reported all this information in a footnote in Section 6.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We only used open source data and codes. Therefore, we believe that there are no issues form the ethical point of view.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: The contribution falls within the scope of fairness and the goal is to mitigate bias in decision making. The paper focus on the general notion of approximate fairness ε -fairness. While approximate fairness is desired in general, this setting allows for a control of the amount of unfairness that we allow or accept. From this point of view, providing the control on the fairness to some user may generate ethical issues.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA] .

Justification: All the datasets are open source and widely used by the fairness community.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We provided references for the two datasets we are using. In addition, we also referred the papers that we considered for a numerical comparison.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: The main contribution is a novel approach for imposing demographic parity fairness in the unawareness case. This is challenging problem for which we spend the whole core of the paper to explain the method and its new considerations.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA] .

Justification: The paper considered public data that does not involve human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [No]

Justification: The paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper considers post-processing regressors to satisfy the group fairness criterion of demographic parity, in particular, under the attribute unaware setting.\n\n1. The authors begin by analyzing the constrained optimization problem (\\*) for fair post-processing, and showing in lemma 3.1 that the solution, i.e., the post-processed regressor, can be represented and parameterized by the dual variables of (\\*); and provided that the base models are accurate and (\\*) is solved to optimiality, the resulting regressor will be optimally fair.\n\n2. Then the authors discussed optimization and statistical aspects of (\\*). Specifically, the convergence rate is analyzed, as well as the conditions for non-perfect solutions to satisfy fairness, namely that the \"gradient mapping\" of (\\*) needs to be lipschitz. For this reason, the authors recommended using the SGD3 algorithm for optimizing (\\*).\n\n3. The paper closes with empirical evaluations of the proposed algorithm.\n\nSTRENGTHS:\n1. To my knowledge, this is the first paper that studies fair post-processing regressors in the attribute unaware setting.\n2. The primal-dual analysis that leads to the representation result in lemma 3.1 is interesting. Because the proposed procedure involves discretization, the formulation of (\\*) is based on the support of the regressor's output space. This is different from somewhat similar works for the classification setting [1, 2] where the optimization problem is based on the scores of the training examples (without discretization).\n3. The theoretical analysis is thorough, hence the proposed procedure is well-supported, including the choice of SGD3.\n\n[1] https://arxiv.org/pdf/2310.05725  \n[2] https://arxiv.org/pdf/2405.04025\n\nWEAKNESSES:\n1. Regarding lipschitzness of $F$. The reviewer feels that some discussions are curt. The authors introduced the \"gradient mapping\" $G_\\alpha$ in eq. (8), with a hyperparameter $\\alpha$. It is not mentioned: how $\\alpha$ should be chosen, in theory or in practice; how exactly SGD3 controls the norm of $G_\\alpha$ (lines 237-239) in the main body. At least a brief discussion should be included in the main body for the latter, since theorem 5.1 would depend on whether SGD3 can provably reduce this quantity.\n2. Regarding the use of SGD3, an ablation study would have been appreciated at illustrating the importance of using SGD3 over other optimizers; is it absolutely necessary? Also, could the authors discuss potential limitations with SGD3?\n3. The authors mentioned that there are several hyperparameters associated with SGD3 (line 236), and with discretization (i.e., the number of bins). But it does not seem to be discussed how these are chosen for the experiments in section 6, and whether mis-specification of these hyperparameters would impact performance.\n\n[3] https://arxiv.org/pdf/2006.07286\n\nQUESTIONS:\n- Does the analysis rely on properties unique to the demographic parity fairness criterion? If not, is there a path to extend to other criteria (potentially taking inspirations from [1, 2])?\n\nLIMITATIONS:\n- See weaknesses.\n- Minor, but the proposed procedure requires discretizing the support, which is a limitation; even though the reviewer is aware that, in practice, discretization helps with generalization hence gives better performing models compared to non-parametric methods [3].", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-06-25"}

comments: SUMMARY:
This paper proposes an algorithm that takes in an fitted regression function and a sensitive attribute predictor and output a prediction function satisfying the demographic parity constraint. It designs a smooth convex objective function with discretization to solve for a prediction function with small risk and controlled violation of the demographic parity constraint. Stochastic minimization techniques are applied to solve the proposed optimization problem and recover the statistical rate $1/\sqrt{T}$.

STRENGTHS:
The proposed algorithm is supported by theoretical analysis and error bounds. The proofs are well-organized and clear to read. The algorithm deploys suitable stochastic minimization techniques to achieve statistical guarantees. Moreover, based on the experiment results, the proposed algorithm is much more computationally efficient than existing methods.

WEAKNESSES:
Typos: 
* falls withing -> falls within (page 2)
* statisitcal properties -> statistical properties (page 5)
* out approach -> our approach (page 7)
* phenomenons -> phenomena (page 7)
* outperfomce -> outperform (page 8)

QUESTIONS:
In practice, how do we pick algorithm parameters such as L, B and \beta?

LIMITATIONS:
Yes.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2024-07-10

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a post-processing algorithm designed to enforce demographic parity in regression tasks without access to sensitive attributes during inference.\n\nSTRENGTHS:\nThe solution is versatile solution for enforcing demographic parity as it can be applied on different optimizers.\n\nThe paper provides a rigorous theoretical foundation to quantify the risk.\n\nThe fairness problem is relevant.\n\nWEAKNESSES:\nWhile the algorithm presented in the paper appears promising, its evaluation is hampered by the presentation style, which tends to obscure clarity. Technical details such as the reliance on discretization and sophisticated methods for controlling the gradient norm are emphasized in the abstract, which could be streamlined to focus on broader impacts and significance instead. \n\nThe paper contains numerous remarks that disrupt the flow of discussion; for instance, Remark 2.1 seems tangential and could be relegated to an appendix or omitted entirely. Other generalizations and discussions, such as those in Remark 2.2 and the paragraph on line 304, should be consolidated and presented at the end of the paper to maintain focus. The 'abuses notation' in Remark 3.1 could be resolved by providing clear definitions Comparisons with other literature are inconsistently integrated within the text, appearing at disparate locations such as lines 56, 228, and 334, which could be better organized to aid in comparative analysis and enhance readability.\n\nThere is excessive use of sub-titles or mini-sections. Some of these sections are notably brief (line 176, for example), where the short content under each title does not justify the need for a separate heading.\n\nQUESTIONS:\nThe paper focuses on demographic parity as a fairness metric. Are there impacts on other fairness metrics like Equalized Odds and Equal Opportunity when using the proposed algorithm? Does it improve these metrics, or could it potentially worsen them?\n\nRegarding the risks plot in Figure 1, could you clarify its purpose given there are no comparative benchmarks provided? How should the unfairness score presented in the plot be interpreted?\n\nReference [1] addresses a similar topic with a minimax approach. Is this minimax result applicable or relevant to the methodology used in your paper?\n\nIn Algorithm 1, \"DP\" is mentioned in the name. Could you specify what \"DP\" stands for?\n\nThe algorithm \"fairname\" appears in line 305 without a prior definition. Could you explain what this term means within the context of your study?\n\n[1]  Fukuchi, Kazuto, and Jun Sakuma. \"Demographic parity constrained minimax optimal regression under linear model.\" Advances in Neural Information Processing Systems 36 (2024).\n\nLIMITATIONS:\nNot applicable.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 1, "DATE": "2024-07-11"}