Conformalized Fairness via Quantile Regression
Authors: Meichen Liu
Lei Ding
Dengdeng Yu
Wulong Liu
Linglong Kong
Bei Jiang
Venue: NeurIPS 2022 Accept (2022)
Keywords: Fairness
Quantile regression
Conformal prediction
Optimal transport
Functional synchronization

Abstract: Algorithmic fairness has received increased attention in socially sensitive domains. While rich literature on mean fairness has been established, research on quantile fairness remains sparse but vital. To fulfill great needs and advocate the significance of quantile fairness, we propose a novel framework to learn a real-valued quantile function under the fairness requirement of Demographic Parity with respect to sensitive attributes, such as race or gender, and thereby derive a reliable fair prediction interval. Using optimal transport and functional synchronization techniques, we establish theoretical guarantees of distribution-free coverage and exact fairness for the induced prediction interval constructed by fair quantiles. A hands-on pipeline is provided to incorporate flexible quantile regressions with an efficient fairness adjustment post-processing algorithm. We demonstrate the superior empirical performance of this approach on several benchmark datasets. Our results show the model’s ability to uncover the mechanism underlying the fairness-accuracy trade-off in a wide range of societal and medical applications.

Conformalized Fairness via Quantile Regression
Meichen Liu', Lei Ding', Dengdeng Yu?, Wulong Liu Linglong Kong' Bei Jiang"
Department of Mathematical and Statistical Sciences, University of Alberta
Department of Mathematics, University of Texas at Arlington
3 Huawei Noah's Ark Lab Canada
(meichen1 1ding1, lkong, meil)aualberta.c
[dengdeng yutaut.eddee [Liumulon>>@huwwei...
Abstract
Algorithmic fairness has received increased attention in socially sensitive domains. While rich literature on mean fairness has been established, research on quantile fairness remains sparse but vital. To fulfill great needs and advocate the significance quantile fairness, we propose novel framework to learn a real-valued quantile function under the fairness requirement of Demographic Parity with respect to sitive attributes, such as race or gender, and thereby derive a reliable fair prediction interval. Using optimal transport and functional synchronization techniques, establish theoretical guarantees of distribution-fies coverage and exact fairness for the induced prediction interval constructed by fair quantiles. A hands-on pipeline is provided to incorporate flexible quantile regressions with an efficient fairness adjustment post-processing algorithm. We demonstrate the superior empirical performance of this approach on several benchmark datasets. Our results show the model's ability to uncover the mechanism underlying the fairness-accuracy trade-off in a wide range of societal and medical applications.
Introduction
We are increasingly leaning on machine learning systems to tackle human problems. A primary objective is develop intelligent algorithms that can automatically produce accurate decisions which also enjoy equitable properties, as unintended social bias has been identified as a rising concern in various fields
13 18 [4].
As means of providing quantitative measures fairness, a number metrics have been proposed. These metrics be categorized into three broad categories: group fairness (3), individual fairness 25), and causality-bass fairness [31] In contrast to causality-based fairness that requires domain knowledge develop fair causal structure and individual fairness that seeks equality only between similar individuals, group fairness does not require any prior knowledge and seeks equality for groups whole (6). Among the metries defined for group fairness such as equalized odds 29] and predictive rate parity [10], demographic parity (DP) is generic since it does not allow prediction results in aggregate to depend on sensitive attributes 12 39) In particular, an algorithm said to satisfy DP if its prediction is independent of any given sensitive attribute I
There have been number of studies algorithmic fairness concerning DP In the context regression analysis, much attention have been paid on conditional mean inferences I exhibit heterogeneity, contain extreme outliers, or do not meet satisfactory distributional assumptions,
39]
12 32 few concerned with conditional quantiles
37 39). As real-world data often
CCo-Corresponding Authors
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

like Gaussianity, a fairness discussion on conditional quantiles may be more rational and essential since they are able to provide a more complete understanding of the dependence structure between response and explanatory variables [39. as well as better accommodate asymmetry and extreme tail behavior [8 It should also be noted that bias or unfairness that arises in mean regression may also be propagated through quantile regression, therefore it must be properly dealt with separately: a graphic demonstration can be found in Figure [ More intuitively, we may take an example from Spanish labor market study [7 [8). The study found that in Spain, also in line with other countries, the mean wage gap between men and women is quite substantial: on average, women earn around 70 percent of what men earn. While wage gaps are not uniform across all pay scales, they are greater at higher quantiles than at lower quantiles. As biases and disparities at different quantiles tend to be overshadowed by the mean behavior of the entire population, propose novel framework to seek fair predictions at different quantiles. It uses optimal transport techniques 2 12) by transforming bias-affectee distributions into only-fair Wasserstein-2 barycenter through a kernel-based functional synchronization method g 45). in order to provide fair quantile estimators.
Deasity ofa45-1
Denstr of Dessity of
Qui--ot) Quit) Qus--10)
Figure 1: An illustration of quantile fairness: for skewed and heteroscedastic quantile estimation ( affected by the sensitive attribute S E {0, 1}, for example, the higher quantile of the salary distribution, the optimal fair quantile prediction Qaa (t), t E (0, 1) is derived through a convex combination of the conditional quantile functions of Qg, SS= and Qu.
Since quantile fairness poses number of theoretical challenges, no previous literature has been able to provide any inference results such as prediction intervals concerning quantile fairness. It is imperative to keep in mind that fairness is only one of two legs of the primary goal of modern machine learning algorithms, the other being accuracy. Building reliable prediction with valid confidence is a significant challenge that is encountered by many machine learning algorithms (44). Towards this end, we propose the conformalized fair quantile prediction (CFQP) inspired by the works of Romano et al. B3 34 Our analysis demonstrates, both mathematically and experimentally, that CFQR provides finite sample, distribution-fiee validity, DP fairness for different quantiles, and precise control of the miscoverage rate, regardless of the underlying quantile algorithm.
Contributions and Outlines. In this paper, we propose new quantile based method with valid inference that enhances both accuracy and fairness while maintaining balance hetween the two. novel framework that allows an exact control of prediction miscoverages while ensuring quantile fairness simultaneously. The main contributions are summarized as follows:
i. We successfully transform the problem of searching quantiles under DP fairness to the construc- tion of marti-marggial Wasserstein-2 barycenters via the optimal transport theory 2 12 19). We incorporate novel kernel smoothing step into the preceding method, which is particularly advantageous for subgroups whose sample sizes are too small to obtain reliable quantile function estimations.
ii In Section we propose a conformalized fair quantile regression prediction interval (CFQP) inspired by the works of Romano et al. [33] 34]. It is mathematically proved to achieve distribution-free validity, demographic parity on different quantiles, and an exact control of
miscoverage rates, regardless of the quantile algorithm used. The theoretical validity of prediction interval constructed by CFQP and exact DP of the fair quantile estimators are given Section and the supplement.
ii.. The experimental results presented Section / include a numerical comparison of the proposed CFQP and fair quantile estimation with both sfate-of-the-aar conformal and fairness-oriented

---

methods. By reducing the discriminatory bias dramatically, our method outperforms the state-of- the-art methods while maintaining reasonable short interval lengths.
Related works. Existing approaches for building a fair mean regression broadly fall into three classes: pre pre-processing in in-processing and post-proc.s.ing. In particular, preprocessing methods focus on transforming the data to remove any unwanted bias [ 31. 43); in-processing methods aim to build
in fairness constraints into the training step [ 4 25 29|; post-processing methods target to modify
the trained predictor [ 12 28. As few previous works have focused on the quantile fairness of and fair prediction interval, the most related are Yang et al. [39], where a different fairness measure was used. While Agarwal et al. [ mentioned that their reduction-baset approach can be adapted into quantile regression, Williamson and Menon 33 brought forward a novel conditional variance at risk faimess measure aiming to control the largest subgroup risk. For interval faimess measure, the approach by Romano et al. [33] achieved equalized coverage among groups without fairness on interval endpoints. Methodologically. integrating algorithmic fairness with Wasserstein distance
based barycenter problem has been studied in [2] I 19 231. Both in in-prceessing [ 23 and
post-processing [11 12] methods were proposed solve classification and mean regression problems. As post-processing method, our work distinct from above- monee-eet methods by constructing the DP-fairness for each population quantile, and generating fair prediction interval accordingly.
Notations. We denote by [K] the set (l...., K) for arbitrary integer K. |S] represents the cardinality for finite set S. E and P represent the expectation and probability and 1[-} is the indicator function. Let (Zp) 100 n=1 be a sequence of random variables, and [ l be a sequence of positive numbers, say that Z,,-
(k,),, if lim lim supn->oc P(|Zal > Tk,) 0, then Zn/ kkn Op(1). To
denote the equality in distribution of two random variables A and B, we write A d B.
2 Problem statement
Consider the regression problem where a "sensitive characteristic" S is available, which by the U.S. law [19] 33] can be enumerated as sex, race, age, disability, etc. We observe the triplets S1, ........ (X,,, S,, Y,), denote (X,, S,, Yi) by Z,, i = 1...., n and Z, is random variable in IRP x [K] R. The aim is to predict the unknown value of Yn+1 at a test point X S.+1. Let P be the joint distribution of Z, we assume that all the samples {2, / drawn exchangeable, where i..d. is special case.
Our goal is to construct marginal distribution-free prediction band C C(X++1,, S.+1) C R that is likely to cover the unknown response Yn+1 with finite-sample (nonasymptotic) validity. Formally, given a desired miscoverage rate the predicted interval satisfies
P (Y E C ( 2-
(1)
for any joint distribution P and any sample size n. while the left and right endpoint of C (X Sn+1) satisfies the fairness constraint of Demographic Parity concerning the sensitive variable S.
Demographic Parity. We introduce the quantitative definition of DP in fair regression and connect the DP-faireess with quantile regressor ga- The result that qa can be projected to the fair counterparts using optimal transport will be invoked later.
Given a fixed quantile level a (it may refer to Og or ahi indicating the upper and lower quantile estimates for the prediction band C(X,+1, 1 )) Let qa : R2 x [K] + IR represent an arbitrary conditional quantile predictor. Denote by 1s the distribution of (g. (X, S) S = s), the Cumulative Distribution Function (CDF) of Va.1s is given by
Feealo (t) = P(qaa S St|S=s)
(2)
Pggla" : [00 1] + R ,namely, the generalized inverse of F Fe....: can The quantile function Qve.. = -Fo11 thus be defined as for all levels t E (0, 1].
Qu y.m (4) inf{y E lR: Feraie (v) 2 O with Quaa (0) = -u (0+).
(3)
To simplify the notations, we will write Fa.l= and Qaal1s instead of Fuo.. and Quaato respectively, for any prediction rule ga-
In the following, we introduce the definition of Demographic Parity (DP), which is most commonly used in the context of faimess research
21 29
3

---

Definition 1 (Demographic Parity). An arbitrary prediction 9: Rd [K] + R satisfies demographic
parity under a distribution over (X,SS YY). g(X,)) statistically independent of the sensitive attribute S. Formally, for every 8, 8' E [K]:
sup P(g(X S) S t S = P (g((X st|s= == =0) =. EER
Demographic Parity (DP) requires the predictions to be independent of the sensitive attribute, and it demands the Kolmogorov-Smng -iiinov distance [26] (the difference between CDFs measured in the l norm) between "s|s and Vggls" to vanish for all categories s, s'.
3 Quantile Regression and Conformal Prediction
In this section, we recall the CQR approach for finite sample. distribution- prediction interval inference. Quantile regression was proposed by Koenker and Bassett (24) to estimate the a-th quantile of the conditional distribution of Y given X == (X, S) for some quantile level a (0, 1), since then it has become more pervasive with various applications, such as providing prediction intervals, detecting outliers, or perceiving the entire distribution [27 (0). Denote the conditional cumulative distribution of Y given X by Fly X = 8) == P(Y S y X = æ.. The a-th conditional quantile prediction is defined as go (T) :- inf E R Fly X = σ) a). Quantile regression can be cast
as an optimization problem 27 30
41,, by minimizing the expected check loss function
F(Pa)) = (y x)lx T). where
( aly (ā) (1- (ā)) if < (
if 2 (ā),
Pa (a
(4)
Quantile regression offers a principled way of judging the reliability of predictions by building a prediction interval for the new observation (x++1,Yn+1) In contrast to asymptopia, Romano al. [33 34 brought forward the conformalized quantile regression (CQR) by combining the merits of robust quantile regression with conformal prediction, thus finite sample validity in Eq. (I) is guaranteed. Inspired by the split conformal method. split CQR likewise starts with split- ting the data into proper training set and a calibration set, indexed by IT In respectively. Given any quantile regression algorithm Q, we then fit two conditional quantile functions Ωa. and Jam on the proper training set: {gom 3mmn + 2 (((...). : Z The conformity scores are calculated to quantify the error made by the plug-in prediction interval (( = [a (7), Jona (i) We evaluate the scores on the calibration set as Es = max { (an. (Xa) - Yn, Y, 4om (8x) for each i E Iz, where both undercoverage and overcoverage of the interval consideration [34] Given new input data Xn+I, construct the prediction interval for Y as C {x+1) = aou (xn+1) Qi-a I2). Laus (Kn+1) + +Qia (E,Z2) where QE, = (1- 1/ empirical quantile of [Ex k E Σ2} conformalizes the plug-in prediction interval. Note that the constructed interval C(X,+11) could be highly influenced by the sensitive variable S.
taken into
4 Conformal fair quantile prediction (CFQP)
We formally describe our proposed conformal fair prediction (CFQP) framework for constructing DP fairness constrained prediction intervals in this section. A kernel smoothing quantile function is introduced during the functional synchronization, which can improve the estimation when some subgroups are too small to give reliable sample quantile function estimations.
Definition 2 2Wasserstein-2 distance). Let f and be two univariate probability measures with finite second moments. The squared Wasserstein-2 distance between μ and v is defined as
W inf { Rxx |x (x,
where Faw is the set of probability measures (couplings) on IR x IR having i and marginals.

---

Proposition 1 (Fair optimal prediction [2]). Assume, for each E [K], that the univariate measure Valls has density and let ps = P(S s). Then,
min E Bu fair
-ax min
E W2 (vg (s:
(5)
sE(K]
Moreover, if Ia and solve the l.h.s. and the r.h.s. problems respectively, then Vg. and specifically,
(2,
Σ S'E[K]
Ps QurQu Faols ls o (a
(6)
Proposition I implies that the optimal fair quantile predictor for an input (I, s) is obtained by non- linear transformation of the vector (ga (2, (ar {) linking a Wasserstein barycenter problem (2] [2]
The explicit closed form solution comes from 12 16), which relies on the classical characterization of optimal coupling in one dimension of the Wasserstein-2 distance. A rigorous proof is given in
[12 23.. It shows that a minimizer Ja of the L2-risk can be used to construct v and vice-versa, given
v, there is a explicit expression Eq. (6) for the multi-marginal Wasserstein barycenter [2]
First, we provide sketch of the CFQP approach. We start with splitting the whole training data into proper training set I and a calibration set Iz, then fit arbitrary quantile regression algorithm Q
on IT (fm. I..) < 2 ({(x.... : We apply the fitted quantile algorithm Q on the
calibration set I2 obtain the predicted {âo. (X:), (X:) Since the quantile estimates for Iz will be used for conformalization, it is essential to transform them into fair ones, i.c. Ia,i Vi E Zz (Eq. (9)), through Algorithm |
Finally, for test point Xn+1, we will predict two quantile estimates aa (x, s) affected by the sensitive variable S by Q. then apply the functional synchronization (details in Algorithm? 2 and calibration (Algorithm T steps in turn to generate the fair constraint prediction interval C(X,+1) for Yn+1.
Next, we explicate in detail how remove the effect of the sensitive variable for the predicted quantile estimates. By Proposition]] T the optimal fair quantiles take the form of Eq. (6). Therefore, we propose an empirical optimal fair quantile estimator å. that relies on the plug-in principle. In particular, Eq. (6) indicates that for each quantile level a and each category s [K], we only need estimators for the regression function Ga. the proportion P.. the cumulative distribution function Fa and the quantile function Qgals-
Note that we can empirically estimate the CDF and quantile function for each sensitive group in the calibration set Iz separately. Hence for each quantile level a, let N, == [Zg]. and the quantile estimators (ği- 02..92..... ........9. Vi, P are calculated through the fitted quantile regression Q with training set I, We define the augmented random variable for each data point in Iz,
T = of (f((=a,)) Vi I2 [K].
where U. are i.i.d. random variables, uniformly distributed on [-a, a) for some small positive a, and independent from all the previously introduced random variables. It serves smoothing random variable, for the random variables of are i.i.d. continuous for any PXY[X) and Q. Otherwise, the original of might have atoms resulting in a non-zero probability to observe ties the group {Øf) for 1,..., K. This trick, also called jittering 7 [2] is often used for data visualization for tie- bebbrraking Using the above quantities, we build the CDF and quantile function estimators for each subgroup s' [K] follows,
N.
(a)lo((1) = No1
Σ 1
s(). for all & R,
Q2. (1) 1/¹ (w) Kn v)de, LE (0,
(8)
The smoothed kernel estimator (8) was firstly proposed by Cheng and Parzen (8 where K-(() K (-/h)/h h a kernel function chosen as a probability density function that symmetric around zero with bandwidth parameter h > 0.
of depends the quantile level a, suppress for notational simplification.
5

---

If the quantile functions Q2.aals' is differentiable, the derivative (',(t) := Q2 2. (t) for E (0, is the quantile density function g 45 We hereby give an estimation bound for Eq. (8) using kernel smoothing. For this purpose, we invoke the conditions (A1) (A3) that are needed for deducing the following proposition. They can also be found from 45 and are included in the supplementary material.
Proposition 2. Under conditions (AI), (A2), and (A3), have
12... (t) Qg or (t) Op (N-V)), 1'= K. rE
sup sup
The motivation for including smoothing step is twofold: First, smoothing the quantile function eliminates the troublesomeness in defining arbitrary quantiles from the empirical one when the sample sizes of subgroups small. Second, the proposed kernel smoothing improves second-order efficiency by alleviating the relative deficiency 15 45
Remark One can utilize various kernels such as the Gaussian or Epanechnikov kernel with adaptive bandwidth for better practical performance. Other smoothing methods such splines or local linear fitting can likewise be applied with equal effectiveness.
Consequently, for each quantile level a, the functional synchronized quantile estimator is
K
Ia.s Σ
p.. P.. Fa.l. of Vi Iz.
(9)
The proposed estimator can be deemed as the empirical counterpart with additional randomization of the explicit fair optimal formula Eq.
To conformalize the adjusted fair quantiles Eq ( we need to compute the conformity scores E3 for each E Iz that quantify the error made by the plug-in fair prediction interval (* @o (z), Jor. (i)]. The scores are evaluated on the calibration as
E max ( Y,,Y, - Jam.i).
(10)
At the last stage, for new data point X, (x, 8), and {al, anj) by defining
7 कf + UF([-o, If and To (a, 8) (x, + U(-ooo)).
We use the empirical CDF of training set
Fr.eal s +
EE
{H <1 + U(O, 1)) (101)) -1))
(II)
estimate the location Fi.gals qa (z, s). Thus the fair quantile estimator is built follows
K
Da (x, 8) = Σ
ps Q2 o Pigale o ao (x, s), Va E {Al, oni)
(12)
The fair prediction interval for Yn+1 is constructed as
C(En+1) = [9 (x, 8) Q1-a(E, 9am. (I, QEI-a(ETT2)
(13)
where Q1-o (E, In) = (1- 10001111//000 1/(Z2))-th empirical quantile of {E; i E I2} will adjust the plug-in fair prediction interval. We present the pseudo-codes of CFQP well as the construction of 9a for Eq. 9 in Algorithm respectively.
Theoretical results
We provide statistical analysis of the proposed algorithm with coverage and Da-fairness guarantees in this part.
Still. of depends quantile level

---

Algorithm Split Conformal Fair Prediction (CFQP)
Input: == [(X;, Sc, Y:)] miscoverage level a E (0. 1): quantile regression algorithm Q
1: Randomly split [n]] into disjoint proper training and calibration indices Z1, Iz.
2: Fit two conditional quantile functions the training set ( } + Q(((,, i E
3: Call functional Synchronization (Algorithm 2 to calculate (gam Geni for each Zz.
Compute E, <- max (3 (X;) Y, Y; Jaw (Xx)) for V I2.
5: Compute Q1-a(E, I2] + (1 a)(1 + //zal)-th empirical quantile of {Ea : Iz}.
6: For new test point (z, s), compute (θar (x, 8), Jass (2. through Algorithm
Output: Fair prediction interval C(r,, s)
[d (2, 8) Q1 I2): Jom (I, Q (E,a(E for
S441) s).
Algorithm 2 Functional Synchronization
Input:
Calibration set {(X,, S:) Jez, new point (z, s); base quantile estimator Q slack parameter training set (.),,,)))
I: if Calibration set {(X,, S:)) then
a E {cno: anh) } do
3:
(X,, S.)} + (XX, U((=))
> U([-o, are used for bre-kie
4: for s' E [K] do
5:
Compute Faalor (t), and Ā-1 2. by Eq. and
6:
Obtain (X, S.) +- 20'=1 Ds Pr- 2 Pan famle' (X,, Sc), Vi E I2
7:
end for end for
8:
else New test point (x. then
10: for a E {Ano: Crps do
11: {?i.a) + {% + U[[_o, aolo,,lleerr and + 2a (a( UU-[a
12:
Compute 9. (z. Es po P 2g Pianis (x, s) by Eq. and
13:
end for
14:
end if
Output: fair quantile prediction ja for calibration set test point (27, s).
Theorem (Prediction coverage guarantee). If (X, = n + exchangeable, then the prediction interval C(Xx+1) constructed by the split CFQP algorithm satisfies
P(Yn+1 C(Xn+1)) 2
Moreover, if the conformity scores E; are almost surely distinct, the prediction interval is nearly exactly calibrated,
P(Y E c(En+1)) 1 -1-a+1/(12 +
Remark 2. Corollary 1 in the supplementary material gives extension for the conformalization step which allows coverage errors to be spread arbitrarily over the left and right tails. Controlling the left and right tails independently yields stronger coverage guarantee.
Theorem (Demographic parity guarantee). For any joint distribution P of (X, Y), any o &a defined in Eq. (12) satisfies
as well as the base quantile estimator âa RP [K] + R constructed on labeled data, the estimator
(X, S= d (X, Vs, E [
Quantile DP guarantees provided by Theorem 2 are derived directly from distribution-free properties on rank and order statistics in Lemma E., Theorem 7.2, of Chzhen and Schreuder [T]] Further information can be found in their papers and the references they provide. We extend the estimator ý for quantile regression based on the estimators developed in their work with solid theoretical foundations. In contrast to the seminal work of Chzhen and Schreuder [II], we regard the densities and quantile functions as functional data, namely, as samples of stochastic processes. Typically, this approach used by economists when dealing with the densities of income distribution across populations. As result of introducing the kernel smoothing step, potential performance improvements are demonstrated in the supplemental material.
7

---

Experiments
To evaluate our proposed method we report the performance of post-processing fairesss adjustment on quantiles through four benchmark datasets: Law School (LAA)), Community& (CRIME), MEPS 2016 (MEPS), Government Salary (GOV). A detailed description of these datasets can be found in the supplementary material. The code for reproducing our results is avaiable at https //github.. com/Lei Lei -ingg7 Conf oataloomaaaaaaaa Quantileeeeeeeaae _ele__FFFress
LAW
CRIME
Coverage
Longth
KS(lo)
KS(hi)
Coverage
Length
KS(Jo)
KS(hi)
La-COR
90.16:1::47
0.46: 004
0.3910003
0.11
90.22:1:.88
1.30:1:0.05
0.62:1:0066
0.53:10.06
La-CFQP
90.02.10.51
0.46 004
0.02
0.02110.01
90.44.1.888
1.64.10.05
0.11:10.33
0.12.10.04
RF-CQR
90.25-1:055
0.39 005
0.20-:0.02
0.15::0.02
90.27::
1.15440..3
0.64::0005
0.59:10.05
RF-CFQP
90.11=00.48
0.38= 004
0.02#.008
0.02#t0009
90.3411.88 1.84
1.54+0004
0.12#00.4
0.12::0.03
NN-CQR
90.00:10.50
0.40=0.02
0.407
0.18:1:005
90.01
1.16::0.05
0.70:1:.05
0.63::0.06
NN-CFQP
90.01
0.39
0.0211.09
0.03.1.009
1.695
1.54:10.11
0.12:10.04
0.12:10.03
MEPS
GOV
Coverage
Length
(l)
KS(hi)
Coverage
Length
KS (lo)
KS(hi)
La-CQR
89.92:1:0.66
0.66=0.01
0.09:0.03
0.33:::.05
90.00::0.19
0.794.0002
0.264.014
0.44:1:.22
La-CFQP
89.99:10.69
0.66:0.01
0.03
0.03:1:001
90.0210.119
0.784.002
0.05100.01
0.04:10.01
90.07-10.65
0.38 009
0.19:10.2
0.30110.03
90.03.10.17
0.28-10.02
RF-CFQP
90.38::0.60
0.39=0.01
0.02
0.03
90.03
0.62:4.002
0.05::0.01
0.04
NN-CQR
89.95.10.68
0.07
0.24100.99
0.37.L 0,06
90.0111..119
0.58.10.01
0.28.10.03
0.32.10.04
NN-CFQP
89.97:1:.611
0.37=0.04
0.03+20.11
0.04
90.01
0.59::0.01
0.05::0.01
0.05:10.01
Table I: Results reported
of 200 repeated experiments with a 0.1. CQR refers the
conformalized quantile regression (34). Ln, RF, and NN denote the linear, random forest, and neural network quantile regression models. Our methods shown in bold.
LAY
CIP
0.134
332
0.13
51N
STM
336
3.100
6.00
6.0
6.2
0.0
042
082
311
022
C35
C315
1.3
2.5
2.15
5.0
tclmogome Sminns Dieante
023
033
0.29 0.2
3.1
Ours_o
jo
agarwal_u
Ourshh
Unfiirni
Aganwal
Figure 2: Results for estimating the lower (at)) and upper (aa)) quantiles using ttate-of-the-art DP-fairness requirement methods on all the datasets. 'Unfair', 'Chzhen', and 'Agarwal' stand for the linear quantile model without fairness adjustment, barycenter method [12] and reduction-based algorithm [ respectively. We present the MAE and KSS of lower and upper quantile estimations. A Linear quantile model is implemented in this comparison.
measure the violation DP-fairness of the quantiles required by Definition I through the empirical Kolmogorov-Smimo (KS) distance. The value represents the disparity between groups
'We utilize the local linear fitting smoothing method in the experiments.

---

2 = ((X,8,)) E S:S=s/ for all s E [K],
KS(g.)= max sup [, ER
Σ (X,
(X, 5 1
E (9 s (x,5.Y)
2S
Experiment results. In Table 1 we report the average performance of the proposed CFQP over 200 randomly training-test splits as well the baseline model CQR by the coverage rate, length of prediction interval, and the KS distance of the interval endpoint. We split the training data into proper training and calibration sets of equal sizes. Throughout the experiments. the nominal miscoverage rate is fixed to a 0.1. Among pre-existing quantile algorithms, we select three leading variants: linear model 24 random forests (27 and neural networks [35] Overall, our CFQP likewise CQR constructs prediction bands attaining desirable coverage around 90%, as claimed in Theorem Random forest based approaches tend to be slightly more conservative than the other two w.r.t the coverage rate among all four datasets.
In the KS column concerning the DP fairesss of interval endpoints, our CFQP method greatly reduces the discriminatory bias (quantified by KS) by 70% up to 90% compared to that of CQR. In addition, the lengths of the prediction intervals mostly remain the same except for the Crime dataset, probably due to its inherent high discriminatory bias between sensitive groups. In addition, CFQP is more robust according to the standard errors over experiment repetitions.
Figure presents the comparison of our post-process procesrroessing faimesss adjustment procedure on quantiles dn using the test set Ziest - {(,,, S., )=) with some state-of-the-art fairness algorithms. Since most of the algorithms are targeted at mean prediction, there is no direct comparison with our quantile fairness method; we accordingly modified the existing methods into quantile versions for comparison. detailed description can be found in the supplementary material.
The points in Figure Prepresents the mean of 200 repeated experiments with r-axis as KS distance and y-axis as Mean Absolute Error(MAEE, where MAE(ga) = 1/ne.r Ezneg |Y - Ja (X, S)] measures the prediction error of quantiles, the bars the standard error on both axes. The optimal points should locate at the bottom left corner of the graph, where smaller KS distance and smaller MAE are achieved. In each subplot, our method consistently performs better with the smallest KS distance while keeping the MAE equal or slightly higher than the others or the unfair version. Note that due to the highly right skewness of real datasets, the MAE of the upper quantile estimation is larger than that of the lower quantile for all quantile approaches. Additional ablation studies for computational time analysis and testing the kernel smoothing approach are incorporated in the supplementary material.
Conclusion and future work
Conformal fair quantile regression is novel approach for creating fair prediction intervals that attain valid coverage and reach independence between sensitive attributes while making minimal modifications to the quantile endpoints simultancously. It becomes superior within heteroskedastic and/or asymmetric datasets and robust to outliers.
Our method is supported by rigorous distribution- free coverage and exact fa-fairness guarantees, as proved in theoretical parts. We conducted several real data examples demonstrating the effectiveness of our method in achieving exact coverage while imposing DP-fairness in practice. The method outperforms several state-of-the-art approaches by comparison.
limitation in our numerical experiments is that we simply utilize the local linear smoothing method in defining quantile functions of the subgroups; believe incorporating flexible kernel smoothing approaches 40 45) would improve the experimental performances.
As potential future works, it would be valuable to introduce DP relaxation framework based on an unfaimess measure in a similar manner as [1 37], allowing controlling the level of unfairness in quantile estimates. We also expect to extend the scope to other potential faimess metrics which is dependent on the underlying response like equalizing quantile loss across groups by incorporating a fairness penalty term in training. or the fairness metric defined for conditional variance-at-risk, In addition, it worthwhile exploring the quantile fairness other areas of AI, such as NI.P [4 Computer Vision, Recommendation systems, etc.

---

Acknowledgements
This work was supported by the Economic and Social Research Council (ESRC ES/T01238211 and the Social Sciences and Humanities Research Council (SSHRC 2003-2019-00000 under the scheme of the Canada-UK Artificial Intelligence Initiative. The project title is BIAS: Responsible AI for Gender and Ethnic Labour Market Equality. We thank Yang Hu, Xiaojun Du and Matthew Pietrosanu for their helpful discussion and valuable input and all the constructive suggestions and comments from the reviewers.
References
[1] A. Agarwal, M. Dudík, and S. Wu. Fair regression: Quantitative definitions and reduction sdduetion-based algorithms. In International Conference on Machine Learning, page 120 129.. PMLR, 2019.
[2] M. Aguch and Carlier. Barycenters in wasserstein space. SIAM Journal on Mathematical Analysis, (3(2):904-244 2011. doi: 10.1137/100805741.
[3] S. Barocas, M. Hardt, and A. Narayanan. Fairness in machine learning. Nips tutorial, 1:2, 2017.
[4] R. Berk, H. Heidari, S. Jabbari. M. Joseph, M. Kearns, J. Morgenstern, S. Neel, and A. Roth. A convex framework for fair regression. arXiv preprint arXiv: 7006 02400, 2017.
[5] F. Calmon, D. Wei, B. Vinzamuri, K. Natesan Ramamurthy, and K. R. Varshney, Optimized pre- processing for discrimination prevention, Advances neural information processing systems, 30, 2017.
[6] A. Castelnovo, R. Crupi, Greco, D. Regoli, Penco, and C. Cosentini. clarification of the nuances in the faireess metrics landscape. Scientific Reports, 11:12,, 2022.
[7] J. M. Chambers, W. S. Cleveland, B. Kleiner, and P. A. Tukey, Graphical methods for data analysis. Chapman and Hall/CRC, 2018.
[8] C. Cheng and F. Parzen. Unified estimators of smooth quantile and quantile density functions. Journal of statistical planning and inference, 59(2):2913307, 1997.
19] J. Cho, G. Hwang, and C. Suh. A fair classifier using kernel density estimation. Advances in Neural Information Processing Systems, 33:15088- 15099, 2020.
[10] A. Chouldechova. Fair prediction with disparate impact: A study of bias in recidivism prediction instru- ments. Big data, 5(2):153-163, 2017.
[11] E. Chzhen and N. Schreuder. A minimax framework for quantifying risk-fairesss tradc-off in regression. The Annals of Statistics, 50(4):2416-2442, 2022.
[12] E. Chzhen, C. Denis, M. Hebiri, L. Oneto, and M. Pontil. Fair regression with wasserstein barycenters. Advances in Neural Information Processing Systems, 33:7321-7331, 2020.
[13] P. Czarnowska, Y. Vyas, and K. Shah. Quantifying social biases in nip: A generalization and empirical comparison of extrinsic faimess metries. Transactions of the Association for Computational Linguistics, 1249-1267, 2021.
[14] L. Ding, D. Yu, J. Xie, W. Guo, S. Hu, M. Liu, L. Kong. H. Dai, Y. Bao, and B. Jiang. Word embeddings via causal inference: Gender bias reducing and semantic information preserving. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 11864-11877 11872, 2022.
[15] M. Falk. Relative deficiency of kernel type estimators of quantiles. The Annals of Statistics, pages 261-268. 1984.
[16] S. Gallón, J.-M. Loubes, and E. Maza. Statistical properties of the quantile normalization method for density curve alignment. Mathematical biosciences. 242(2):129 1421 2013. doi: 10.1016/j.mbb mbw.mmml...2.....m 2012. 12.007.
[17] García, Hernández, and López-Nicolás. How wide the gap? an investigation of gender wage differences using quantile regression. Empirical Economics, 16(1):149-1 167, Mar 2001. doi: 10. 10000//001100000055
[18] J. Gardeazabal and A. Ugidos. Gender wage discrimination at quantiles. Journal of Population Economics, 18(1):165-177 Mar 2005. doi: 00.100777000144-003-007..2. 003-0172-2.
10

---

[19] T. L. Gouic. J.-M. Loubes, and P. Rigollet. Projection to fairness in statistical learning, arXiv:2005.11720 [cs, math, stat], Jun 2020. arXiv: 2005.11720.
[20] P. Han, L. Kong, J. Zhao, and X. Zhou. A general framework for quantile estimation with incomplete data. Journal of the Royal Statistical Society: Series (Statistical Methodoloyy). 81(2):005-333, 2019.
[21] H. J. Holzer and D. Neumark. Affirmative action: What do we know? Journal of policy analysis and management, 25(2)4433 0400033-9000000 2006.
[22] S. Hu, A. Al-Ani, K. D. Hughes, N. Denier, A. Konnikov, L. Ding. J. Xie, Y. Hu, M. Tarafdar, B. Jiang, et al. Balancing gender bias in job advertisements with text-level bias mitigation. Frontiers in big Data, 2022.
[23] R. Jiang, A. Pacchiano, T. Stepleton, H. Jiang, and S. Chiappa. Wasserstein fair classification. In Proceedings of The 35th Uncertainty in Artificial Intelligence Conference, page 862-872. PMLR, Aug 2020.
[24] R. Koenker and G. Bassett. Regression quantiles. Econometrica: journal of the Econometric Society, pages 33-50, 1978.
[25] M. J. Kusner, J. Loftus, C. Russell, and R. Silva. Counterfactual fairness. Advances in neural information processing systems, 30, 2017.
[26] E. L.. Lehhannn, J. P. Romano, and G. Casella. Texting statistical hypotheses, volume 3. Springer, 2005.
[27] N. Meinshausen and G. Ridgeway. Quantile regression forests. Journal of Machine Learning Research, 7 (6), 2006.
[28] S. Milli, J. Miller, A. D. Dragan, and M. Hardt. The social cost of strategic classification. In Proceedings of the Conference Fairmess, Accountability, and Transparency, pages 230-239, 2019.
[29] L. Oneto, M. Donini, and M. Pontil. General fair empirical risk minimization. 19Xi::1911110000 [cs, stat]. Dec 2019. arXiv: 1901.10080.
[30] M. Pietrosanu, J. Gao, L. Kong, B. Jiang, and D. Niu. Advanced algorithms for penalized quantile and composite quantile regression. Computational Statistics, 36(1):333-346, Mar 2021. ISSN 0943-4062,, 1613-9658. doi: 10. 00.1007/00180-0000 80-020-01010-1.
[31] D. Plecko and N. Meinshausen. Fair data adaptation with quantile preservation. Journal of Machine Learning Research, (242)): 2020.
[32] D. Plečko, N. Bennett, and N. Meinshausen, fairadapt: Causal reasoning for fair data pre-proeee pre-processing. arXiv preprint 1xXiv:2110.10200, 2021.
[33] Y. Romano, R. Barber, Sabatti, and Candès. With malice towards none: Assessing uncertainty via equalized coverage. ArXiv, 2019. doi: 10. 11119996081922000005522
[34] Y. Romano, F., Patterson, and Candes. Conformalized quantile regression. Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
[35] J. W. Taylor. A quantile regression neural network approach to estimating the conditional density of multiperiod returns. Journal of Forecasting, 19(4):299-3) 311, 2000.
[36] Y. Wang, L. Kong. B. Jiang. X. Zhou, S. Yu, L. Zhang, and G. Heo. Wavelet-based lasso in functional linear quantile regression. Journal of Statistical Computation and Simulation, 89(6): 1 111-1130, 2019.
[37] R. Williamson and A. Menon. Fairesss risk measures. In International Conference on Machine Learning, pages 6786-6797. PMLR, 2019.
[38] Z. Xiao and R. Koenker. Conditional quantile estimation for generalized autoregressive conditional heteroscedasticity models. Journal of the American Statistical Association, 1696 2009.
[39] D. Yang, J. Lafferty, and D. Pollard. Fair quantile regression. arXiv preprint arXiv:1907.08644 2019.
[40] S.-S. Yang. A smooth nonparametric estimator of quantile function. Journal of the American Statistical Association, 80(392): 100--1011 1985. ISSN 0162- 1459. doi: 10.2307/2288567.
[41] D. Yu, L. Kong, and I. Mizera. Partial functional linear quantile regression for neuroimaging data analysis. Neurocomputing, 195:74 87, 2016.

---

[42] D. Yu, L. Zhang, L. Mizera, B. Jiang, and L. Kong, Sparse wavelet estimation in quantile regression with multiple functional predictors. Computational statistics & data analysis, 136:12-29, 2019.
[43] R. Zemel, Y. Wu, K. Swersky, T. Pitassi, and C. Dwork. Learning fair representations. In International conference machine learning, pages 225-333. PMLR, 2013.
[44] G. Zeni, M. Fontana, and S. Vantini. Conformal prediction: unified review of theory and new challenges. arXiv preprint arXiv:2005.07977 2020.
[45] 7. Zhang and H.-G. Müller. Functional density synchronization. Computational Statistics & Data Analysis, 55(7):2234-2249) 2011. doi: 10. 1..1016ff.ssaa22111.1.0000
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contribu- tions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Section
(c) Did you discuss any potential negative societal impacts of your work? [No] We currently don't t foresee the potential negative societal impacts of work, however, anticipate this method be used the future by practitioners real-life scenarios involving job markets, medical science, etc., and to potentially help making decisions that would protect the vulnerable and propel social harmony.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs all theoretical results? [Yes]
3. If you experiments...
(a) Did you include code, data, and instructions needed to reproduce main experimental results (cither in the supplemental material or as URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (c.g., type of GPUs, internal cluster, cloud provider)? [N/A]
4. If you are using existing assets (c.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you're 'r us- ing/curating? [Yes]
(c) Did you discuss whether the data you are using/curating contains personally identifiable informa- tion or offensive content? [N/A]
1f you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
12


Reviews:
[{"review_id": "jvSZ9A8Y6jf", "reviewer": "Reviewer_3BdP", "summary": "The work proposes a method to guarantee fair prediction intervals in regression tasks. It takes the approach of quantile regressions to obtain the prediction intervals and uses a previous result on the form of the fair quantile regressor to develop an approach for fair prediction intervals. Additionally, it uses conformal inference approach to provide intervals that have valid coverage in finite samples. The experiments on 4 real datasets show that the method maintains competitive coverage, length of prediction intervals, and prediction error compared to baselines while reducing unfairness.", "questions": "Questions:\n\nRegarding presentation:\n\n1. The relation between DP fairness defined for the interval prediction function (Def. 1) and DP fairness defined for quantile functions (between \\nu’s) is not clear. How does one ensure another?\n\nThe importance or challenge in building a method that gives both fair quantile regression and conformal prediction intervals is not presented convincingly.\n\nRegarding related work:\n\n2. How does the proposed method for fair quantile regression relate to the method in Agarwal et al. [1]? Their method also covers the case of quantile regression with check loss and DP fairness. Does the addition of the split-conformal step applicable to their method as well to get miscoverage guarantees?\n\n3. How does the work differ from Romano et al. [42]? Is the difference due to the fairness objective?\n\nRegarding experiments:\n\n4. The abstract claims that the experiments uncover the fairness vs coverage frontiers. Table 1 reports the coverage and fairness metrics but does not show the frontier. Does the claim refer to Figure 2 which reports the frontier of fairness vs error.\n\n---\nSuggestions (not expecting a response):\n\nI would suggest tightening up the motivations for fairness (between group-specific quantile regression functions) and uncertainty estimates in the Introduction (first page). It does not clearly answer why these are required. For example what is the issue with heteroscedastic data or in the case of uncertainty estimates, do the high-stakes areas require prediction intervals or confidence intervals.\n\nPlease describe why it is challenging to combine fair regression functions with conformal inference such that the significance of the proposed method and analysis can be conveyed.\n \nThe example of salary prediction in Introduction is unclear. What is the fairness objective?\n\nThe \\alpha_lo and \\alpha_hi needs to be explained at page 4, first equation. Do these refer to upper and lower quantiles, \\alpha/2 and 1-\\alpha/2 for a miscoverage rate of \\alpha?\n\nThe description of the method can be improved. Consider giving a sketch of the method before hand before describing each of the steps.\n\nConsider discussing other potential fairness metrics in case of real-valued predictions like equalizing calibration metrics across groups.\n\nDiscuss related work Williamson and Menon 2019, Fairness risk measures. The fairness metric defined for conditional variance-at-risk is another possibility to go beyond mean predictions.\nhttps://proceedings.mlr.press/v97/williamson19a.html\n\nPlease define DP fairness briefly to give context to Introduction and Related work.\n\nThe left hand side of Proposition 1 that is the squared loss term is not clear.\n\nA tip on writing style is to limit the use of superlatives like tremendous as they are not precise and redundant sometimes.", "limitations": "Limitations are adequately discussed. Consider discussing whether the method allows for relaxing the fairness constraint that is the KS distance can be less than some non-zero value.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- addresses a new problem.\n- the work seems to be important given the importance of providing valid prediction intervals in safety-critical settings while ensuring some form of fairness.\n- skilfully combines advances from fair quantile regression and conformal inference.\n\nWeaknesses\n- both the problem definition and motivation is not clear from the Introduction.\n- there seems to be a discrepancy between the fairness goal and the method. The aim is to define fairness of prediction intervals but the fairness metric is defined for the quantile regression functions which are one of the ways of constructing the intervals but not the only way.\n- presentation is not easily understandable to readers not familiar with quantile regression-based prediction intervals and conformal inference.\n\nThe main concern is the presentation of the method. The theoretical analysis seems sound but I am not familiar with the techniques and literature so I cannot comment on its validity and significance. In my opinion, improving the presentation would take substantial changes to the writing than is possible to verify from the author response.\n\n---\n\n## After the response\n\nThe response adequately addresses my concerns. The presentation of the problem statement and overview of the method has been greatly improved. I appreciate the honest response to comparison with related work [1] and meticulous comparison in running time. \n\nThe work addresses an important problem via a principled approach. Thus, moving the fair learning field forward beyond notions based on averages to quantiles. Hence I have increased my score to 7, Accept.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "fix4PEEGQk9", "reviewer": "Reviewer_H28y", "summary": "This paper studies Demographic Parity in Quantile Regression. The authors extend from previous work in Conformalized Quantile Regression and use optimal transport and functional synchronization techniques to incorporate DP into Quantile Regression. Experiments show that the fair version of Quantile Regression significantly reduces the gap in DP while keeping a good coverage rate and length of the interval.", "questions": "1. It seems that the template is not in submission mode.", "limitations": "Limitations and negative societal impacts are adequately addressed.", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros:\n1. Consider the fairness issue in quantile regression is novel. It seems that there is no other paper considering this specific problem.\n2. The proposed techniques look sound and make sense to me.\n3. Good experimental results. Compared to the non-fair version, the proposed method gets a small gap in terms of KS at a minimal cost of coverage rate and interval length.\n\nCons:\n1. There is limited context characterizing the fairness issue in quantile regression. Probably the authors can describe without a careful design how unfairness can be propagated in quantile regression through a vivid example, and how such unfairness is different from other problems, e.g., ordinary regression.\n2. Some key information in experiments is missing. The authors mention in Fig. 2 they compare to 'some state-of-the-art fairness algorithms' but there is no reference and discussion.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "58k6k-3CCDc", "reviewer": "Reviewer_NTAY", "summary": "In this paper, the authors study the fairness problem in quantile regression settings. Based on the theories from the Wasserstein barycenter problem and two smoothing strategies, the authors propose the CFQP algorithm. They also theoretically analyze the results of the algorithm. Finally, they conduct experiments on four real-world datasets to prove the effectiveness of the algorithm.", "questions": "See details in the Strengths And Weaknesses section, especially the points in the Originality, Quality, and Clarity parts.", "limitations": "More limitations are demonstrated in the Strengths and Weaknesses section.\n", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Originality**\n\n\\+ The authors propose to study the fairness problem in the quantile regression setting, which is a new problem.\n\n\\- The methods proposed by the authors seem a combination of the Wasserstein barycenter problem studied in [10] and two smoothing strategies in [6][7], which makes the contribution marginal.\n\n**Quality**\n\n\\- The advantages of the two smoothing strategies are not clear. On the one hand, more discussions about the two smoothing strategies are needed. For example, why the jittering strategy works when the sample sizes of subgroups are small? And what are the second-order efficiency and relative deficiency brought by the kernel smoothing strategy? On the other hand, the authors are encouraged to conduct the ablation study by testing the experimental results if removing either or both of the smoothing strategies.\n\n**Clarity**\n\n\\- The paper is not well-written. Several unclear parts need further clarification. There are also several typos to fix.\n  1. The three conditions in Section 5 are confusing. What do these conditions mean and are they practical? More discussions are needed.\n  2. As mentioned above, the advantages of the two smoothing strategies should be clarified.\n  3. In Section 2, it is not clear what the subscript $n+1$ means since there are $n$ training samples. I understand this may refer to the unknown test sample, but please clarify it.\n  4. Figure 1 caption: \"affacted\" --> \"affected\", \"quartile\" --> \"quantile\".\n\n**Significance**\n\n\\+ The method is generally sound with both theoretical and experimental results.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "3jBtJxMJtc0", "reviewer": "Reviewer_kfJM", "summary": "This work investigates the incorporation of group-fairness into the established learning problem of conformalized quantile regression. In particular, the authors extend previously developed quantile regression techniques to handle demographic parity constraints between groups. To do this,  the authors develop a postprocessing technique for modifying a given quantile regressor such that it becomes fair with respect to demographic parity. Within this framework the authors provide theoretical gurentess with respect to the efficacy of their method. In particular Theorems 1, 2 demonstrate that the proposed technique produces a model which has both reasonable predictive power as well as fairness. Further, the authors compare their technique to two other fair regression methods and show that their method iid frequently superior in terms of fairness and error.\n\n", "questions": "## Questions and Comments \n\n\n- **Slack in fairness**: In practice achieving perfect fairness (i.e. equal DP across all groups) is infeasible or at the very least undesirable. As such it is useful to have a so called \"tunable slackness variable\" $\\varepsilon$, such that fairness can be framed as $|M_i - M_j| \\leq \\varepsilon$ between any two groups $i, j$, where $M$ is a statistical metric applied to both groups. The current model requires $\\varepsilon=0$. Can we say anything theoretically about when $\\varepsilon > 0$. Similarly, is a model designer able to control the desired unfairness (i.e. $\\varepsilon$) in the practice? If so, are there any experimental results on error-fairness tradeoffs as we have vary $\\varepsilon$?\n\n- **Equation 2**: Equation 2 seems to be misspecified compared to [33]. Also the suppression of the dependence that $q$ has on $x$ and $\\alpha$ in this definition may lead to confusion. It may be more clear to use $q_{\\alpha}(x)$ instead of simply $q$, especially since $q$ is defined to be a function two lines earlier.\n\n- **Citations for legal justifications**: Legal justification is given for both demographic parity as well as choice of sensitive feature S. In both instances the authors cite computer science papers (not legal works). Although the claims are mentioned in these citations, it would be better to directly cite the actual source, thus avoiding any whisper-down-the-lane issues that may arise.\n\n- **Distribution notation**: The variable $\\mathbb{P}$ is used to denote distributions. Perhaps $P_{X}$, $D$, or $\\mathcal{D}$ would be more standard. Note that  [33, 41] do not use $\\mathbb{P}$ to denote distributions.\n\n- **Interpretation of DP**: Prior to the definition of demographic parity, the authors mention that demographic parity can be thought of as a stronger version of the so called \"four-fifths rule\". While this is a reasonable interpretation for DP in the context of classification, this example does not necessarily translate to regression. For example in a regression problem what does it mean for a candidate to \"receive a job\"?. In [1] (the paper from which this interpretation is pulled) the authors are quite careful to point out that they are referring to DP in the context of classification during this example. This distinction should either be made more precise, or an alternative interpretation should be given. \n\n     \n- **Comparison to other fair models**: In the experiments related to Figure 2, the authors mention that the models they compare two are not quantile regressors so they adapt these models to the quantile regression paradigm. Can the authors give a more percicse description of this adaptation procedure? Is this as simple as altering the labels in the training set, then running the original learning scheme, or is there something else being done? Additionally, the comparisons in this figure are for linear models only do we see similar results for non-linear models?\n\n\n- **Minor comments/issues**:\n        \n    - NeurIPS latex format appears to be partially incorrect (no space for authorship and line numbers are missing).\n\t\n    - The first sentence of the abstract “Fairness and impartiality have been among the predominant pursuits in human society” comes across as a slight exaggeration. One could make the argument that fairness is more often than not, treated as an afterthought rather than primary goal, i.e. ensuring systems are just fair enough to be acceptable for the time. As such I think some readers are likely to take issue with this claim as-is. \n        \n    - Page 1, second to last line: \"categories ... would\" should have a common instead of \\dots\n\t\n    - Page 2, last paragraph: The first sentence \"While current approaches ... \" is incomplete.\n       \n    - Page 3, second paragraph: \"$\\mathbb{E}$ and $\\mathbb{P}$ represent for the generic ...\" should be \"$\\mathbb{E}$ and $\\mathbb{P}$ represent the generic ...\". Also this sentence starts with math notation. \n    - Page 3, Section 3, first paragraph: \"... since then it becomes more and more pervasive...\", should be \"... since then it has become more pervasive ...\"\n    - Page 3, Section 3, first paragraph: \"As a brief introduction to the quantile regression\", this sentence is incomplete and also should not have \"the\" before \"quantile regression\".\n\n    - Definition 1: It is more standard to give these definitions as the difference between a sententive group and the population, rather than between two group, e.g. $|M_g - M|$ for all groups $g$, where $M_g$ is the statistical metric applied to $g$ and $M$ is the statical metric applied to the whole population.\n         \n    - Page 4, last paragraph: \"Proposition 1 pronounces\", pronounces is out of place, perhaps \"implies\" would be better.\n\n    - Page 7, Remark 2: \"...tails as Corollary 1 in Appendix\", missing \"the\" before \"Appendix\".\n\n    - Page 6, last paragraph: \"...presented in Lemma 3in the ...\", missing a space between \"3\" and \"in\".\n\n   - Page 8, Government Salary: \"The yearly salary for over 200 hundred examples...\", I believe this should be \"The yearly salary for over 200,000 examples...\", based on the size of gov_census.rda in the supplement.\n\n    - Figure 2: It may be better to set the minimum of the $x$-axis to $0$.\n\n     - Inconsistent use of \"supplement\" and \"appendix\" throughout the paper.\n\n", "limitations": "The authors have adequately addressed the limitations of their work.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n- The paper is well written (barring a few minor issues mentioned later).\n- The authors make non-trivial contributions to a well established problem, namely introducing a notion of group fairness to the quantile regression problem.\n- A mix of theoretical experimental results are provided.\n- The proposed method is shown to outperform two state-of-the-art methods in experiments.\n- The authors make a point to address the practicality of their method (e.g. Remark 1).\n \n## Weaknesses\n- The experiments performed are somewhat limited. \n- Running time and computability of certain quantities appears to be missing. What is the worst case running time of Algorithm 1. How feasible is it to compute equation 8 (does this have a closed form solution or must we rely on numerical approximations)?\n- The proposed model only works with one fairness metric (namely DP). It would be nice to see similar results for a fairness metric which is dependent on the underlying $Y$.\n- Allowed unfairness does not appear to be a tunable parameter.\n- See Questions section for more details comments.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept