Understanding Fairness and Prediction Error through Subspace Decomposition and Influence Analysis
Authors: Enze Shi
Pankaj Bhagwat
Zhixian Yang
Linglong Kong
Bei Jiang
Venue: NeurIPS 2025 poster ()
Keywords: Algorithmic Fairness
Fairness–Utility Trade-off
Sufficient Dimension Reduction
Influence Functions
Primary area: social_and_economic_aspects_of_machine_learning
TL;DR: We balance fairness and predictive accuracy by decomposing feature representations and analyzing their impact using influence functions.

Abstract: Machine learning models have achieved widespread success but often inherit and amplify historical biases, resulting in unfair outcomes. Traditional fairness methods typically impose constraints at the prediction level, without addressing underlying biases in data representations. In this work, we propose a principled framework that adjusts data representations to balance predictive utility and fairness. Using sufficient dimension reduction, we decompose the feature space into target-relevant, sensitive, and shared components, and control the fairness–utility trade-off by selectively removing sensitive information. We provide a theoretical analysis of how prediction error and fairness gaps evolve as shared subspaces are added, and employ influence functions to quantify their effects on the asymptotic behavior of parameter estimates. Experiments on both synthetic and real-world datasets validate our theoretical insights and show that the proposed method effectively improves fairness while preserving predictive performance.

Understanding Fairness and Prediction Error through Subspace Decomposition and Influence Analysis
Enze Shi, Pankaj Bhagwat, Zhixian Yang, Linglong Kong, Bei Jiang"
Department of Mathematical and Statistical Science
University of Alberta paaaippbhgwwat,hhheoe zhixian, ,kong bei beteeeeiili@@lleae Qualberta.cccco
Abstract
Machine learning models have achieved widespread success but often inherit and amplify historical biases, resulting in unfair outcomes. Traditional faimess methods typically impose constraints at the prediction level, without addressing underlying biases in data representations. In this work, we propose a principled framework that adjusts data representations to balance predictive utility and fairness. Using sufficient dimension reduction, we decompose the feature space into target-relevant, sensitive, and shared components, and control the fairness-uthe utility trade-off by selectively removing sensitive information. We provide theoretical analysis of how prediction error and fairness gaps evolve as shared subspaces are added, and employ influence functions to quantify their effects the asymptotic behavior parameter estimates. Experiments on both synthetic and real-world datasets validate theoretical insights and show that the proposed method effectively improves fairness while preserving predictive performance.
Introduction
Machine learning (ML) models have achieved remarkable success across wide range high- stakes applications, including finance Hardt et al. [2016). Liu et al. [2018) healthcare Potash
jet al. [0155] Rudin and Ustun (2018). and criminal justice Van Dijck 2022 Billi et al.) 2023]]
Despite these advances, growing evidence highlights that ML systems often inherit and reinforce historical biases, leading to unfair outcomes Tolan et al. [2019], |Merabi et al. 2021]. Biases in data
collection Liang et al. 22200) Pagano al. 2023 and disparities in group representation De-Arteagal
et al. 201 9] Dablain et al. [2024] can manifest in model predictions, ultimately amplifying social
inequities Bolukbasi et al.
2016 Hassani 2021 Hu et al. 2024 Ding al. 2024
To address fairness concerns, researchers have introduced range of formal definitions and algo- rithmic interventions. Early work focused on ensuring statistical criteria such as Demographic Parity Kamishima et al. [0012] Jiang al. [2020], and more recent developments extend fairness guarantees multiple sensitive attributes Tian al. (2024]. Chen et al. [2024] or local prediction regions Jin et al. 2024). Many approaches operational ize faimess by adding constraints or regulariza- tion terms to learning objectives Hardt al. [2016] Li et al. 2023]. However, this strategy typically treats fairness as an external correction layered on top of predictive modeling, without addressing the root causes of bias encoded within data representations themselves.
This work proposes new perspective: instead of adjusting model predictions post hoc, we seek to understand and manage the trade-off between utility and fairness at the level of data representation. Specifically, we focus on how to construct representations that balance predictive accuracy for the
"Corresponding Author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

target variable and independence from sensitive attributes. This viewpoint naturally connects faimess with the framework of sufficient dimension reduction (SDR) Cook and Li 2002). Adragni and Cook [200), where the goal is to project high-dimensional features onto lower-dimensional subspaces that preserve essential information.
We consider a structured setup in which both the prediction target Y and the sensitive attribute Z admit low-dimensional sufficient reductions with respect the covariates X. By analyzing the subspaces associated with Y and Z, we separate two types of directions: (I) directions informative about Y but orthogonal Z and (2) directions jointly informative about both Y and Z. This decomposition enables a principled subspace selection: start with Z-orthogonal directions for fairness, then add shared ones boost accuracy as needed.
To quantify the uailits trade-off, we develop a theoretical framework showing how prediction error and group-wise disparities evolve shared directions added. While incorporating Z- related directions improves accuracy, it reintroduces unfairness, which is captured through variance decomposition and gap metrics. We further apply influence functions to analyze the impact of partially fair representations on parameter estimation and prediction. The contributions of this paper are summarized as follows:
1. We propose a general framework for fairness-aware learning by directly manipulating data repre- sentations. It enables controlled removal of sensitive information without relying specific fairness definition.
We theoretically characterize how prediction risk and fairness gaps evolve as shared informa- tion is gradually introduced, and use influence functions analyze the effect of partially fair representations on estimation and prediction.
3. We validate our approach through experiments on synthetic and real-world datasets, demonstrating that subspace elimination achieves trade-offs between faimess and predictive performance.
The structure of this paper is as follows. Section 3 introduces the motivation and problem setup, along with theoretical analysis of the utility- trade-off under subspace elimination. Section details the estimation procedure. Section anaalyzes the asymptotic behavior of the estimators and predictors using influence functions. Section presents simulation and real-world experiments that demonstrate the effectiveness of our approach.
2 Related Work
A major approach to fairness in machine learning focuses on enforcing fairness during model training,
known on-processing Wang al. [2022a] Berk et al. (2023). Caton and Haas (2024). These
methods incorporate fairness constraints Zafar et al. 2019) Agarwal et al. [2019] into the optimization objective to satisfy predefined criteria such Demographic Parity Dwork et al. [012) Equality of
Opportunity Hardt et al.) [2016), Shen et al. [2022). and have been applied across various paradigms
including contrastive learning Wang al. [2022b]. Zhang al.) [2022], adversarial training Han
et al. [022]. domain adaptation Wang et al. 22020. and balanced supervised learning Han et al. 20022 However, fairness guarantees obtained this way are often task-specific and may not generalize across downstream applications. A complementary direction focuses on achieving fairness at the representation level by removing sensitive information from feature embeddings. Notable methods such as Iterative Null-space Projection (INLP) Ravfogel et al. [2020] 2023) and Relaxed Linear Adversarial Concept Erasure (RLACE) Raafogel et al. 2022] aim to systematically eliminate sensitive information. More recently, sufficient dimension reduction (SDR) techniques have been adapted for debiasing (Shi et al. [2024), identifying and removing subspaces associated with sensitive attributes while offering theoretical guarantees, with extension of non-linear SDR via deep neural networks as proposed in Shi et al. 2025).
3 Methodology
3.1 Motivation
We study the problem of removing sensitive information from vector representations while preserving task-rclevant content. Let (X, Y. Z) be random variables where X IR" denotes the representation,

---

Y E RK is the prediction target, and Z E R is the sensitive attribute. Our goal is to learn transformation IR" R such that the transformed representation h satisfies: (1) Minimal dependence on the sensitive attribute Z and (2) Sufficient information for predicting the target variable Y. direct approach involves constructing linear transformation h (X) PX, where P E TRPXP is projection matrix. This induces a decomposition of the input space as X = PX + (T X) with P projecting onto a subspace S S IR" and P projecting onto its orthogonal complement S2 = S The representation space is thus decomposed as R Si + S2. For effective debiasing, Si should minimize information about Z, while S2 captures the Z-relevant components to be removed.
This formulation is introduced by Shi et al. [2024], who adopt the sufficient dimension reduction (SDR) framework to identify and eliminate sensitive information in representations. Given X and Z, their goal is to find matrix B= E R2x with orthonormal columns such that
Z X,
ensuring that B X contains all the information in X relevant to predicting Z. The column space of Bz, known as the SDR subspace of X with respect to Z, see Cook and Lij [2002] Adragni and Cook (2009), thus serves as natural candidate for S2 with dim = Its orthogonal complement, Then h(X) = PX removes
spanned by P = I B.B., defines S1 with dim (S1) =
information associated with Z. yielding fair projection.
While the sufficient projection method in Shi et al. (2024) performs well across downstream tasks, it primarily focuses removing Z-related information without explicitly preserving information relevant to Y. This can lead to substantial utility loss when Y and Z share overlapping subspaces. To address this, we propose finer decomposition of the representation space IR", separating directions informative about Y but orthogonal to Z, directions shared by both, and directions unrelated to either. This more granular perspective allows for better control of the Fairtes--utiltyy trade-off by retaining lask-relevant features while minimizing bias from sensitive attributes.
3.2 Problem Setup
We adopt a SDR framework for both the target variable Y and the sensitive attribute Z, modeled as
Y = [(8 x, X
X xx),,
(2)
= u x, 21 X 1. x,ez)
for some measurable functions f and g, where {Bel2=1 C R' and 1, C R R2 are orthonormal direction vectors, and Ey, E2 are noise terms independent of X. The SDR assumption implies that all information relevant for predicting Y and Z is captured by the low-dimensional projec- tions (,, X) ==== and {tb X )/==1 respectively. Equivalently, the models in and 3 imply the conditional independence statements:
Y TT X My
I X MzX,
(4)
where My, Mz E PPP are matrices with rank q and The subspaces spanned by these matrices are referred to as the central subspaces, defined by
Srix Span(My) Span( ........ (B,....,B,yk Sz|x Span(Mz) Span }.
Assume that the central subspaces Sy and S2ix intersect subspace
Srix n Sz (pann(1,.....44 Φz),
where s I min(g.r.. When = 0, the subspaces intersect only the origin. In the special case where Sy x - the two subspaces are completely separable. In this setting, removing all information associated with Z does not affect the information relevant for predicting Y, and thus fairness can be achieved without sacrificing utility.
In practice, the subspaces Syjx and Sz often overlap, with a nontrivial intersection (s > 0). In such cases, removing all Z-related components may also eliminate valuable information for predicting Y. To address this, we decompose Sy x into two orthogonal parts: one shared with Szix and one independent of it. This decomposition enables a principled approach to balancing fairness and utility by selectively retaining target-relevant features uncorrelated with the sensitive attribute.
3

---

Without loss of generality, we assume that the shared basis vectors satisfy Ø, = 3g g--ati = -r for
i == ,,...,s. S. Define the projection matrix onto Sz x as P= = Ej=1 4 and Q= = 1p- 1--Pt Pp-P=. where
Q: projects onto the orthogonal complement of the sensitive subspace. Let B = (3,,... ...., Bg) R and Φ (di...., ..) E IRxx*. Since Sy1x n Sz1x = Span ()), the uncorrelated component
SY x n x can be identified by finding a matrix B such that
Y Q x
(5)
Theorem 3.1. Let B be an orthonormal matrix satisfying condition (Q. Syxx x n nSZ 2|X
5 then Span B)
Theorem 3.1 provides a direct link between the central subspace of Y and its component orthogonal to the sensitive subspace S21x. Building on this decomposition, define sequence of partially
fair projection matrices { 1s
m==0 as
p(m) BB pm T
where Pm (01, ... m.).
Each p(m) projects X onto subspace that retains B-based directions uncorrelated with Z, along with m of the shared directions between and Z. Let {(m) p(m) X denote the resulting representation. When m = 0, the representation is entirely uncorrelated with Z; when m = the representation spans the full central subspace of Y. preserving complete utility but potentially reintroducing bias.
To study the predictive behavior under this fairness-uie utility trade-off, we define the Bayes optimal predictor using the partially fair representation:
j((m (E (m)) E Y
mm).
This formulation allows gradual control over the balance between fairness and accuracy by varying m, i.e., the number of shared components included. In the following, we analyze the theoretical properties of f(mm characterizing how prediction risk and faimess evolve with subspace selection.
3.3 Utility and Fairness Trade-off
Let = (mm) p(m) X denote the reduced representation. The following result characterizes the prediction error of the Bayes optimal predictor based on the reduced representation.
Theorem 3.2. Let (E(m)) Y
(m)) denote the Bayes predictor using the partially fair representation =(m), and let (x) = E X] denote the Bayes optimal predictor using the original representation. Then, the expected squared prediction error satisfies
[( f(() (=())) = VaVar((' (x) (m))
BiEY]
== A(m) + +o²
Approximation
Irreducible noise
Moreover, the approximation error ((m) is non-increasing in m, i.e. A(m + 1) < (mm) for all E {O...., 1).
This decomposition follows from the orthogonality principle in L space, which ensures that the conditional expectation minimizes mean squared error. The term [[[ar[[**XX
= {(m))] quantifies the loss in predictive information due reducing the representation to E (m) As more shared directions are included, the reduced representation becomes increasingly informative, and the error A (m) decreases. When m = 8, the full central subspace for Y recovered, yielding ( = 0.
Quantifying unfairness theoretically inherently challenging, as it often stems from disparities in prediction outcomes across sensitive subpopulations. Instead of relying on specific fairness metrics like TPR or demographic parity (DP) gaps, we adopt a distributional perspective by measuring the statistical dependence between the predictor and the sensitive attribute using distance covariance (dCov) Székely et al. [207]]. which quantifies the discrepancy between joint and marginal charac-
teristic functions. The following result illustrates how the reduced representation (mm) mitigates unfairness by weakening the dependency between the predictor and the sensitive attribute. While the result is stated for binary Z, it naturally extends to multivariate cases with Z e Rd

---

Theorem 3.3. Let Z E {0,1} be a binary sensitive attribute with PZZ = =1), and let fim) 1 B|Y (i be the Bayes predictor using the reduced SDR representation E E(m). Then the squared population distance covariance between f(()) and Z satisfies
dCov² dCov²(f(m)) Z) = 2p(11 (EIL)TT) - f I f(mm))) (1).
where 7(m) ~ f(lm Z = for E {0, 1}. and fimm is an independent copy of fimm), Specifically, when O. we have dCCv" (f(m)) Z) 0 and thereby f(m) 11 Z.
This expression reveals that distance covariance is determined by the discrepancy between within- group and between-group variations in predictions. When the reduced representation ((m) sufficiently removes dependence on Z, the term E[]fim) [/ approaches the population- variation Ellf(m)) - f(m))) []. leading to a smaller dCov and improved fairness.
4
Subspace Estimation and Algorithm Implementation
4.1 Estimation of Projections
We describe the procedure to estimate the sufficient directions for predicting Y. The first step is to estimate the intersection subspace Sy n nSzix, spanned by f This is equivalent to estimating matrix + RPx* such that
Y
1 Z
TX,
(6)
which corresponds to dimension reduction with respect to the interaction between response variables as proposed in Luo 2022
Let Σ denote the covariance matrix of X. Note that My and Mz are the symmetric candidate
matrices from SDR methods that satisfy (4). with estimates My and Mz. Define the cross-matrix
My,z My EM2, and let rank (M, ). Luo 2022 show that & satisfies (6) if and only if
My E Pr. My,z,
where PE,B = BBB BB Σ. The following theorem characterize the estimation of intersection subspace as a generalized eigenvalue decomposition problem.
Theorem 4.1. Suppose Span( S R2 an s-dimensional subspace. Then Span 4) given by the span of the leading eigenvectors of the following generalized eigenvalue decomposition problem: My EM VEv, The candidate symmetric matrix for (6) is My,z My Σ Mz.
Then the estimation procedure of + is as follows. First, construct My and My using any exhaustive inverse regression method (e.g.. SIR [i [199T]. SAVE Cook and Weisberg [1991], or directional regression Li and Wang 2007 ) and compute the sample covariance matrix Σ. Next, form My.z
My EM2 and estimate the dimension & rank My,2) 2)) using the ladle estimator [Luo and Li 2016).
Once @ is estimated, the remaining directions relevant for predicting Y can be obtained via projection. Rather than estimating .I,..., Po- directly, we estimate their projections onto the orthogonal complement of Sz|x- Define the estimated projection matrix Q_ Ip - N5 then apply any SDR method to (Y, x. obtain By.q, such that Y 1T P.X By X. This matrix approximates the projected directions BT Pz,..... PT. Together, the estimated shared directions + and the unshared components By.P, form a sufficient projection p(m)) The consistency and 112 convergence rates of the estimated directions and projections are well-established in the SDR literature; we omit these details for brevity. The complete procedure is summarized in Algorithm|] Remark 4.2.
The computational cost of obtaining the fair projection matrix primarily arises from estimating the candidate matrices My, .M and My.z, each typically constructed as a weighted covariance matrix. These computations scale linearly with the sample size n. In addition, the procedure includes an eigen-decomposition step for p p matrices, and the rank estimation step scales linearly with dimension
5

---

Algorithm Estimation of Sufficient Projections
1: Input: Data (X,, Y;, Z:) SDR method for candidate matrix construction
2: Output: Estimated sufficient projections p(m), m ....,....
3: Compute My. Mz using SDR method; compute Σ = cov cXv(X).
4: Form My.z My EM2; apply ladle estimator to estimate rank s.
5: Obtain estimator T TR and get projection Q..
6: Apply SDR to (Y. Qx X) and obtain By.q, with rank dy.q.
7: Obtain p(m) By By Ps 4m 4 for m .,...
4.2 Algorithm Implementation
The sufficient variables {(m) ((m)) X can then be used to fit regression or classification models for downstream tasks. Note that the columns of & are ordered by the eigenvalues of My reflect their predictive power for both Y and Z. By gradually adding these columns in the projection, can incrementally build models with varying levels of sensitive information included.
This setup allows users to manually control the amount of sensitive information retained when predicting Y. The post-SDR training procedure is summarized Algorithm 2 using a classification task with accuracy (Acc) the utility metric and demographic parity (DP) the faimess metric. The optimal feature set and fair model are chosen to achieve at least 95% of the validation accuracy from the full dataset while minimizing unfairness.
Remark 4.3. Algorithm 2 presents replaced by any fairness and utility metrics, and the 95% threshold generalized to any r E (0, 1). The shared dimension acts as a tuning parameter analogous to regularization coefficient, offering two advantages: (1) 8 is selected from a finite set, avoiding continuous grid search (one can step every 2-3 dimensions use bisection); and (2) each added component remains interpretable, enabling clear insight into the fairness-performatte trade-off.
example of post-SDR training. In practice, DP and Acc can be
Algorithm Sequential Fair Projection: Post-SDR Fair Model Training
Input: Sufficient variables [(m) ((m) X, training and validation datasets
2: Output: Fair model Muai and selected feature set = (m²)
3: for = 0 to do
4: Train model M(()
fair based on = [(0) and record (t) and DP(1) validation
5: end for
6: Let arg min; (DP(1) |Aco 95% Accloris)), where (or(g))g) is the accuracy trained
by original datasets X.
7: return Selected model Mair = Mg and corresponding feature set E (mm))
5 Theoretical Analysis
In this section, we analyze the impact of using sufficient variables in the post-SDR training procedure, with focus how they influence parameter estimation and expected errors. We employ influence functions to trace model's prediction through the learning algorithm and back its input features.
5.1 Influence Functions
Let (X, 2) ~ Fo be the joint law and we denote the empirical distribution on n samples of (X,Y, Z) F,. Define the SDR functional My and Mz(FF) for estimating central subspaces Syix and S21x respectively, and let (FF) = Var(X) ) be the functional for the covariance matrix. Therefore, the intersection subspace estimation for corresponds to reduction functional My,x(F) My (F))
In the following, an asterisk on a symbol always indicates the influence function of a statistical functional represented by that symbol. For sample point S (x, 2) drawn from Fo, let ôg be the

---

Dirac measure at S. The influence function of the functional R is defined as
de ① R[( F) + Sds] I==0
R (*)
For notation simplicity, we abbreviate R (S) by R In the following, an asterisk on symbol always indicates the influence function of a statistical functional represented by that symbol. For example, we denote the influence functions of functionals My (F), Mz My (F) and Σ(F) as My. M2- My 2 and Σ", respectively. For notation simplicity, we omit the Using the product rule for Gateaux derivatives, we obtain the influence function of My
Lemma 5.1. Suppose (F)F), Mz (F)) and (FF) are Hadamard differentiable. Then, the influence function of the reduction functional My, (F) is
My.z M2 E(F) (F) (y Mz(F) My(F)E(P)) M2.
5.2 Asymptotic Normality of Estimators
Let ((X;0) be the differentiable predictive function parametrized by and let L(x, y; 0) be the differentiable loss function with respect to 6, where we fold in any regularization terms into L. For notation simplicity, we denote L(x, y/ 0) by L(S;0). Then the population and empirical risk minimizer of the parameter is given by
1
i = (Fo) arg min E[((S,)), BEB
0, = (FF,) arg min =
I(S,,)) 0)
BEB n2
1=1
Similarly, we define the parameters estimated by sufficient variables ((m) (p(m) x, y, 2) as
@lm) glm) (Fo, p(m) (Fo)) - arg min ([L(S(m), .0]
BEB
mm)) glm) (Fns p(m) (Fr)) A arg min BEB ni
Σ
L
(S!T)... ,),
where p(m) (F.) ((m) and S(mm) (p(m) xr y, z). We refer to 0, the original estimator and Himi as the fair estimator. The entire post-SDR learning procedure can be viewed as a composition of three mappings:
Fn
reduction
p(m) (Fr) estimation
((m) p(m) (())(F)) prodiction
f(-: ((m)) p(m) (Fx)))
The composition mapping allows us to derive the asymptotic normality of both the estimators and predictors after applying fair projections, stated in the following theorems.
Theorem 5.2. Suppose all the statistical functionals presented above are Hadamard differentiable, and their inftuence functions has zero expectation and finite variance, then we have the following asymptotic normality for the estimator g
Vn(0(m) Øiml) D N (0. Var - ((m)) ((m)) veel (oo(p))))))
(8)
g(m)
where Blon) (SL(S))) Olm)) the gradient, D(m) (00(Fo. plm) (Fo)) vec (ee(P))))) vec(.) vector- ization of the matrix, and p(m)) denotes the influence function of p(m)(F) whose explicit form depends on the choice of SDR method and is provided in the Appendix.
E(V {( is the expected Hessian of loss function, G(m)
Corollary 5.3. Under the same assumptions stated in Theorem 5.2
we have
E(1@@mm) m) - 13. = (m))) - 0112 +
1 Tr (ar Har H.' G-- H (()) + (m)) vec(p(m))))). (9) ((m))
where H E V 0)) and G (S,
Theorem 5.4. Let fxx; 8) be the predictor evaluated at covariate x. Under the same assumptions
stated in Theorem 5.2 we have the following asymptotic normality for the predictor
fn f( (z; 0m)) f::0))-f(;;0000 f(z;0lm))
2, N (0.g* Var - H=1 ((m)
(m)) + ((m) vec( (ec(p())))))) (10)
where g Vof(z;ūm))).

---

Theorems 5.2 and 5.4 provide feasible inference procedures for the fair estimators corresponding each m, which are useful for subsequent statistical inference or hypothesis testing. Corollary [.3] follows directly from Theorem 5.2 by comparing the asymptotic distributions of the fair and original estimators. shows that the expected distance between the fair and original estimators is upper bounded by the distance between their respective true parameters and the sampling variability.
Moreover, the projection matrix P(m) plays a crucial role in shaping the asymptotic variance of the fair estimator. By restricting the estimation to a subspace spanned by p(m), it reduces the dimensionality of the parameter space, leading smaller asymptotic variance. However, smaller m also implies that important predictive directions truncated and inflate the |0(m) 0 term. Thus, p(m) provides natural mechanism to balance variance reduction and bias introduction.
Experiments
6.1 Simulation Studies
In this section, we use simulation results to justify the behavior of using the sufficient variables under the fairness setting. Consider the multivariate linear regression, simulate a dataset (X, Y, Z). where the sensitive attribute E {0, 1} is binary.
Let A be a randomly generated matrix with columns are orthonormal. Denote Ay E RRxx is the first q columns of A and Az R is the q to q s + r columns of A, which means Ay and Ag has shared columns. Define the latent variables Uy X A'yy Ay and U = XAzAA. Let X E R' drawn from a standard multivariate normal distribution: X ~ (0, The response Y e R is generated from
Y Uy ey (0.00.55Ikk
where the coefficients O E RPx are randomly generated with each entry samples from MV 1). The sensitive variables Z is generated based on the following latent score
1 P j=1
Z = 1f 1, and Z = 0 otherwise; where & =
tanh((U2)) + ez and eg ~ ~(O, 1).
Finally, introduce distributional shift between groups by applying non-linear transformation to the X samples based only on the span of Az. Specifically, for all samples X with Z = 1,
X + x + 0.5 exp (X (xAzA))
We generate synthetic data with parameters n = 5000, p = 10, K = 5, =, 8, and s = 6. The dataset is randomly split into 4000 training samples and 1000 testing samples. We fit multivariate ordinary least squares (OLS) model on both the original features X and the projected representations p(m) X to estimate the model parameters O.
To assess performance, trial, we compute the root mean squared error (RMSE) on the test set overall and separately for the subgroups Z 0 and Z 1, along with the RMSE gap between groups. We also report the
repeat the entire process over 30 independent replications. For each
parameter distance ||0, (m) 0.IlI where 0,, the OLS estimator obtained from the projected data p(m)) X, and 0,, the baseline estimator from the original data. Results are averaged over the 30 trials and summarized in the left panel of Figure We also project X onto the direction that best discriminates the sensitive attribute Z using Linear Discriminant Analysis (LDA), in order to visualize the distributional discrepancy between the two sensitive groups.
The simulation results clearly support our theoretical findings Specifically, as the shared dimension increases, more directions in Φ are used to predict the target variable, incorporating additional sensitive and predictive information. This leads to reduction the MSE of the predictor and parameter distance to the original estimator, but an increase in the MSE gap between sensitive groups Both the distribution discrepancy and the Wasserstein distance also increase, indicating growing divergence between groups more sensitive information is included.

---

2-9
2-1
0, WD
C.102
2.WD 0.116
0.6
0.3
PRSE Al
PNSE (20))
BMSE (2-1)
4. WD C.170
6. WD 0.285
PNSE Oup
* Perementa Dedance
0.1
00
0.0
Stared Dimereaan [m)
LDA direction
LD4 direction
Figure I: Left panel: Trends RMSE and parameter distance as the number of shared dimensions increases, averaged over 30 replications. Right panel: Distributional discrepancy between sensitive groups visualized via LDA, along with the average Wasserstein distance (WD) across all dimensions between the original and projected data the shared dimension increases in one replication.
6.2 Real Data Applications
We evaluate our proposed Sequential Fair Projection (SFP) method on two tabular datasets: Adult Kohavi [996] and Bank Moro et al. [014]. The Adult dataset contains personal data from 40K individuals, with the task of predicting whether annual income exceeds $50K; gender is used as the sensitive attribute. The Bank dataset originates from Portuguese marketing campaigns, where the goal is to predict whether client will subscribe to deposit; age (over/under 25) is treated as the sensitive attribute. The data is standardized during preprocessing and split into training, validation, and test sets with a ratio of 70%: 20%%..
We compare SFP against the following baselines: Logistic Regression (LR). AdvDebias Zhang ctt al. 2018), FairMixup Chuang and Mrouch [2021], DRAlign Li et al. [2023], DifI Jnn
2024 INLP Ravfogel et al. [0200, RLACE Ravfogee et al. 2022]. and SUP Shi et al. [2024] Fairness is evaluated using TPR gap, DP, and MCDP, and utility is evaluated using accuracy. The detailed experimental settings shown in Appendix B All methods repeated 20 times, and
al.
report the average performance with standard deviations. The results are presented in Table
Table 1: Performance metrics on the Adult and Bank datasets over 20 replications. Optimal results in bold, and sub-optimal results are underlined.
Bank Dataset
Method
Accuracy
DP 1
IPR
MCDP
Method
Accuracy 1
DP
TPR
MCDP
LR
84.90
17.37
6.87
35.12
LR
91.17
6.72
2.62
26.28
AdvDebins
76.49 (0.65)
12.45 (1.56)
5.27 (0.53)
29.32 (2.311
AdvDehias
(1.56)
2.01 (1.36)
2.20 (0.13)
17.55 (1.39)
FairMixup
(0.43)
(0.34)
(1.58)
FairMixup
(0.24)
DRAlign
75.01 (0.41)
7.58 (1.03)
4.78 (0.29)
22.04
DRAlign
(1.52
(..6(0)44)
1.96 (0.21)
13.61 (1.23)
DiAMCDP
73.93 (0.31)
5.92 (1.25)
5.33 (0.46)
11.50 (1.09)
DiAMCDP
60.01 (1.83)
(0.39)
2.18
11.00 (0.97)
INLP
(0.57)
(0.51)
8.58 (1.24)
INLP
70.13
(0.21)
(0.31
10.44
RLACE
72.79 (0.83)
5.24 (0.56)
4.56 (0.31)
7.45 (0.86)
RLACE
72,51 (0.53)
T.02 (0.35)
2.26 (0.27)
8.98 (0.38)
(0.36)
(0.27)
(0.29)
8.16())25)
SUP
70.82 (0.47)
0.96 (0.16)
2,51 (0.22)
10.63 (0.42)
SFP (Ours)
76.88 (0.47) 3.78 (0.15) 4.43 (0.26)
10.52 (0.72)
SEP (Ours)
89.66 (0.27)
0.51 (0.26)
2.33 (0.27)
10.57 (0.34)
Our results show that SFP consistently strikes an effective balance between predictive accuracy and fairness. On both the Adult and Bank datasets, SFP achieves competitive or superior accuracy compared to existing fairness-aus aware methods while substantially reducing group-level disparities. On the Adult dataset, SFP attains the highest accuracy (76.8%%) and the lowest DP gap. demonstrating strong control over statistical bias. also performs well terms of TPR gap and MCDP, with only minor trade-offs relative to sub-optimal methods. On the Bank dataset, SFP also achieves the highest accuracy (89.66%, (6.6)))) and the lowest DP gap (0.51), indicating minimal disparity. Although RLACE slightly outperforms in MCDP, SFP maintains competitive fairness across all metrics. These findings

---

2-9
2-1
m = 1, WD = 0.377
m 10, WD 0.302
0.4
0.
0.1
NREAA
PUGE (2-0)
NMGE (2+1)
m 19. 3.542
m 28. WD 0.872
TUEZE Cup
Parametrr Delatce
0.5
0.3
XXXXXX
8.1
0.0
10
Shared Dimeendon (n)
LDA cirection
LDA direction
Figure 2: Trends RMSE, parameter distance and distributional discrepancy as the number of shared dimensions increases when the linear SDR assumption is violated.
confirm that SFP is practical and flexible framework for fairness-aware learning, enabling users to suppress sensitive information while preserving task-relevan predictive power.
6.3 Model Misspecification
In practice, the linear SDR assumption could be violated and the estimated central subspaces will fail to capture the full conditional independence structure. Therefore, conditional independence tests can be applied to verify whether the estimated projections preserve sufficiency. When the linear SDR assumption is strongly violated, the estimated matrices My and Mz may lose their low-rank structure. This indicates that low-dimensional linear subspace (X.Y) or (X, Z). such cases, one practical remedy is to retain a subset of directions corresponding the leading eigenvalues and My,z and construct fair projection by climinating leading directions.
capture the dependence between
To examine the trade-off between utility and fairness of SFP under model misspecification, we conduct additional experiments based the simulation setup described in Section 6.1 with added nonlinear term X /l//p in the generating processes of both Y and &. This modification violates 30, making it impossible to recover low low-rank representation through SDR. After applying SFP, the estimated shared dimension is 3 = 28, indicating that the rank of My.z is 28. We then sequentially add the directions obtained from My.2 and report the resulting trend in Figure
the linear SDR assumption. We set p 30, K 5, and
The overall RMSE and RMSE gap using the original representation are 3.18 and 3.06, respectively. As shown in Figure there is clear trend that, the number of shared dimensions used to construct the projection matrix increases, both the overall MSE and the parameter distance decrease, indicating that the projected representation gradually approaches the information contained in the original features. Meanwhile, the RMSE gap and the distributional discrepancy increase as more sensitive information is included. This demonstrates that SFP can still capture the trade-off between utility and fairness even when the linear SDR assumption is violated.
Discussion
We propose principled and model-agnostic framework for fairness-aware learning through subspace decomposition of data representations. Our method manipulates the representation space by selec- tively removing shared information between the target and sensitive attributes, which enables flexible control over the faimess-utility trade-off. We provide theoretical guarantees on how prediction error and fairness metrics evolve as more sensitive information incorporated, and further apply influence function analysis to characterize the impact of partially fair representations on estimator behavior. Empirical results on both synthetic and real-world datasets validate our theoretical insights, demonstrating that the proposed SFP method achieves good performance.
10

---

Acknowledgements
Bei Jiang and Linglong Kong were partially supported by grants from the Canada CIFAR AI Chairs program, the Alberta Machine Intelligence Institute (AMII), and Natural Sciences and Engineering Council of Canada (NSERC), and Linglong Kong was also partially supported by grants from the Canada Research Chair program from NSERC.
References
Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. Advances in neural information processing systems, 29, 2016.
Lydia T Liu, Sarah Dean, Esther Rolf, Max Simchowitz, and Moritz Hardt. Delayed impact of fair machine learning. In International Conference on Machine Learning, pages 3150-3158. PMLR, 2018.
Eric Potash, Joe Brew, Alexander Loewi, Subhabrata Majumdar, Andrew Reece, Joe Walsh, Eric Rozier, Emile Jorgenson, Raed Mansour, and Rayid Ghani. Predictive modeling for public health: Preventing childhood lead poisoning In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 2039-2047, 2015.
Cynthia Rudin and Berk Ustun. Optimized scoring systems: Toward trust in machine learning for healthcare and criminal justice. Interfaces, 48(5):449-466 2018.
Gijs Van Dijck. Predicting recidivism risk meets ai act. European Journal on Criminal Policy and Research, 28(3):407-444 423, 2022.
Marco Billi, Thiago Raulino Dal Pont, Isabela Cristina Sabo, Francesca Lagioia, Giovanni Sartor, and Aires José Rover. Supervised learning, explanation and interpretation from pretrial detention decisions by italian and brazilian supreme courts. In International Conference Conceptual Modeling, pages 131-140. Springer, 2023.
Songiil Tolan, Marius Miron, Emilia Gómez, and Carlos Castillo. Why machine learning may lead to unfairness: Evidence from risk assessment for juvenile justice in catalonia. In Proceedings of the seventeenth international conference on artificial intelligence and law, pages 83-92, 2019.
Ninarch Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey bias and fairness in machine learning. ACM computing surveys (CSUR), 54(6):1-35 2021.
Paul Pu Liang, Irene Mengze Li, Emily Zheng, Yao Chong Lim, Ruslan Salakhutdinov, and Louis- Philippe Morency. Towards debiasing sentence representations. Proceedings of the 58th Anmual Meeting of the Association for Computational Linguistics, 2020.
Tiago P Pagano, Rafael B Loureiro, Fernanda VN Lisboa, Rodrigo M Peixoto, Guilherme AS Guimarães, Gustavo OR Cruz, Maira M Araujo, Lucas L Santos, Marco AS Cruz, Ewerton LS Oliveira, et al. Bias and unfairness in machine learning models: a systematic review on datasets, tools, fairness metrics, and identification and mitigation methods. Big data and cognitive computing, 7(1):15, 2023.
Maria De-Arteaga, Alexey Romanov, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, and Adam Tauman Kalai. Bias in bios: A case study of semantic representation bias in high-stakes setting. In proceedings of the Conference on Fairness, Accountabiliry, and Transparency, pages 120-128, 2019.
Damien Dablain, Bartosz Krawezyk, and Nitesh Chawla. Towards holistic view of bias in machine learning: bridging algorithmic fairness and imbalanced learning. Discover Data, 2(1):4, 2024.
Tolga Bolukbasi, Kai- Wei Chang, James Zou, Venkatesh Saligrama, and Adam Kalai. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems, 29, 2016.
Bertrand Hassani. Societal bias reinforcement through machine learning: credit scoring perspec- tive. Al and Ethics, 1(3):239-244 2021.

---

Yang Hu, Nicole Denier, Lei Ding, Monideepa Tarafdar, Alla Konnikov, Karen D Hughes, Shenggang Hu, Bran Knowles, Enze Shi, Jabir Alshehabi Al-Ani, al. Language job advertisements and the reproduction of labor force gender and racial segregation. PNAS nexus, 3(1 3(12))pgae526, 2024.
Lei Ding, Yang Hu, Nicole Denier, Enze Shi, Junxi Zhang, Qirui Hu, Karen Hughes, Linglong Kong, and Bei Jiang. Probing social bias in labor market text generation by chatgpt: masked language model approach. Advances in Neural Information Processing Systems, 37:139912 12-
12-22-119999 139937, 2024.
Toshihiro Kamishima, Shotaro Akaho, Hideki Asoh, and Jun Sakuma. Fairness-aware classifier with prejudice remover regularizer. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2012, Bristol, UK, September 24-28, 2012. Proceedings, Part 23, pages 35-50. Springer, 2012.
Ray Jiang, Aldo Pacchiano, Tom Stepleton, Heinrich Jiang, and Silvia Chiappa. Wasserstein fair classification. In Uncertainty in artificial intelligence, pages 862-872. PMLR, 2020.
Huan Tian, Bo Liu, Tianqing Zhu, Wanlei Zhou, and S Yu Philip. Multifair: Model fairness with multiple sensitive attributes. IEEE Transactions on Neural Networks and Learning Systems, 2024.
Zhenpeng Chen, Jie M Zhang, Federica Sarro, and Mark Harman. Faimess improvement with multiple protected attributes: How far are we? In Proceedings of the IEEE/ACM 46th International Conference Software Engineering, pages 1-13, 2024.
Jinqiu Jin, Haoxuan Li, and Fuli Feng. On the maximal local disparity of faimess-aware classifiers. In Forty-first International Conference on Machine Learming. 2024.
Tianlin Li, Qing Guo, Aishan Liu, Mengnan Du, Zhiming Li, and Yang Liu. Fairer: fairness as decision rationale alignment. In International Conference on Machine Learning, pages 19471- 19489. PMLR, 2023.
Dennis Cook and Bing Li. Dimension reduction for conditional mean regression. The Annals of Statistics, 30(2):455-44 2002.
Kofi P Adragni and R Dennis Cook. Sufficient dimension reduction and prediction in regression. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 367(196)):4385 4405, 2009.
Xiaomeng Wang. Yishi Zhang, and Ruilin Zhu. A brief review on algorithmic faireess. Management System Engineering. 1(1):7, 2022a.
Richard A Berk, Arun Kumar Kuchibhotla, and Eric Tchetgen Tchetgen. Fair risk algorithms. Annual Review of Statistics and Its Application, 10(1):165-187, 2023.
Simon Caton and Christian Haas. Fairness in machine learning: A survey. ACM Computing Surveys, 56(7))1-38, 2024.
Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez- Gomez-Rodriguee Rodriguez, and Krishna P Gummadi. Fairness constraints: A flexible approach for fair classification. Journal of Machine Learning Research, 20 (75):1-42, 2019.
Alekh Agarwal, Miroslav Dudík, and Zhiwei Steven Wu. Fair regression: Quantitative definitions and reduction-based algorithms. In International Conference on Machine Learning, pages 120-129. PMLR, 2019.
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Faimess through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference, pages 214-226, 2012.
Aili Shen, Xudong Han, Trevor Cohn, Timothy Baldwin, and Lea Frermann. Optimising equal oppor- tunity fairness in model training. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4073-4084, 2022.
Ruijia Wang, Xiao Wang, Chuan Shi, and Le Song. Uncovering the structural fairness in graph contrastive learning. Advances in neural information processing systems, 35:32465-32472 2022b.
12

---

Fengda Zhang, Kun Kuang, Long Chen, Yuxuan Liu, Chao Wu, and Jun Xiao. Fairness aarness-aware contrastive learning with partially annotated sensitive attributes. In The Eleventh International Conference on Learning Representations, 2022.
Xudong Han, Timothy Baldwin, and Trevor Cohn. Diverse adversaries for mitigating bias in training. In Proceedings the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 2760-2765, 2021.
Zeyu Wang, Klint Qinami, Ioannis Christos Karakozis, Kyle Genova, Prem Nair, Kenji Hata, and Olga Russakovsky. Towards fairness visual recognition: Effective strategies for bias mitigation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8919 8928, 2020.
Xudong Han, Timothy Baldwin, and Trevor Cohn. Balancing out bias: Achieving fairness through balanced training. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1335-11350, 2022.
Shauli Ravfogel, Yanai Elazar, Hila Gonen, Michael Twiton, and Yoav Goldberg. Null it out:
Guarding protected attributes by iterative nullspace projection. In Proceedings of the 58th Anmual Meeting of the Association for Computational Linguistics, pages 7237-7256, 2020.
Shauli Ravfogel, Yoav Goldberg, and Ryan Cotterell. Log-lineat guardedness and its implications. In Proceedings of the sl Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 9413-9431, 2023.
Shauli Ravfogel, Michael Twiton, Yoav Goldberg, and Ryan D Cotterell. Linear adversarial concept erasure. In International Conference on Machine Learning, pages 18400-18421, PMLR, 2022.
Enze Shi, Lei Ding, Linglong Kong, and Bei Jiang. Debiasing with sufficient projection: A general theoretical framework for vector representations. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 1960--975,, 2024.
Enze Shi. Linglong Kong, and Bei Jiang. Deep fair learning: A unified framework for fine-tuning representations with sufficient networks. arXiv preprint arXiv:2504.064700 2025.
Gábor Székely, Maria L Rizzo, and Nail K Bakirov. Measuring and testing dependence by correlation of distances. The Annals of Statistics, 35(6):2769, 2007.
Wei Luo. On efficient dimension reduction with respect to the interaction between two response variables. Journal of the Royal Statistical Society Series B: Statistical Methodology, 84(2):269 294, 2022.
Ker-Chau Li. Sliced inverse regression for dimension reduction. Journal of the American Statistical Association, 86(414):316-3) 1991.
Dennis Cook and Sanford Weisberg. Discussion of sliced inverse regression for dimension reduction. Journal of the American Statistical Association, 86(414): (3:2244)32--33,, 1991.
Bing Li and Shaoli Wang. On directional regression for dimension reduction. Journal of the American Statistical Association, 102(479);97-11088 2007.
Wei Luo and Bing Li. Combining eigenvalues and variation of eigenvectors for order determination. Biometrika, 17744))875-887 2016.
Ron Kohavi. Scaling up the accuracy of naive-bayes classifiers: a decision-tree hybrid. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, pages 202 207, 1996.
Sérgio Moro. Paulo Cortez, and Paulo Rita. data-driven approach to predict the success of bank telemarketing. Decision Support Systems, 62:22-31, 2014.
Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. Mitigating unwanted biases with adversarial learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pages 335-340, 2018.
13

---

Ching- Yao Chuang and Youssef Mroueh. Fair mixup: Fairness via interpolation. In International Conference on Learning Representations, 2021.
Li-Xing Zhu and Kai-Tai Fang. Asymptoties for kernel estimate of sliced inverse regression. The Annals of Statistics, 24(3):1053-1065 1996.
Kyongwon Kim, Bing Li, Zhou Yu, and Lexin Li. On post dimension reduction statistical inference. Annals of Statistics, 1567--5566 2020.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification; All the claims made in the abstract and introduction are established in Method- ology and Theoretical Analysis sections and demonstrated on both simulations and real datasets.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
Limitations are discussed in Section D of the Appendix
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
15

---

Answer: [Yes]
Justification:
The assumptions are clearly stated and all the proofs are provided in Section C
of Appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proof' in the paper should be numbered and cross referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide all the technical details of our experimental setup. Furthermore, provide the code necessary to reproduce results in the supplementary materials.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
16

---

The authors should provide scripts to reproduce all experimental results for the new reproducible, they Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: All datasets used are publicly available, and we have uploaded code in the supplementary materials.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
public/guides/ Saarddese/ooeeSummsssssooore
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ https/cccppblligggddpeeeeeeeeeee Coaee//dddeSebmmmcia ssi onPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide all the technical details of our experimental setup.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: All the results are reported with mean and standard deviation for multiple replicates.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
17

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma enror bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We conduct all our experiments on an Ubuntu Server with CPU AMD Ryzen Threadripper PRO 3995WX 64-Cores Processor and 256G RAM.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics htttp:://neurip::::: http://ps...p.ppoooocccccccccala
Answer: [Yes]
Justification: The research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss the broader impact in Section E Appendix.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
18

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All the methods, codes and datasets used in the paper are properly cited.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperswithcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
19

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
This paper propose new algorithm and does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
20

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: LLM used only for editing and grammatical refinements.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to L.LM policy htttaa/a/eaaarrsssec////ofeeeeee for what should or should not be described.
21

---

A Influence Function of Projections
In this section, we give explicit form of the influence function of projection p(w), which can used to derive the asymptotic normality of the parameter estimators.
Let {(As.i; +;)] 1=1 be the eigenvalues and associated eigenvectors of My,z, {(Ag, Bi) be the eigenvalues and associated eigenvectors My and {Av,, vi)) be the eigenvalues and associated eigenvectors of Mz.
Then, by Zhu and Fang 1996], the influence function of the directions estimated by SDR techniques can be written as, for
,,
O50, My
(11)
Ao,i Ao.5
And for 1: = 1,...,
(Q2Bk)"
Q=3% + Q Q=3% 3r
1p Σ ( (55bgT))
(12)
5=1
where
BeB3 My
Ve M2
Σ
and v/
A8,K A3,e
Ab A
1=1, Eetck
t=1, Edj
Then we have
p(m)) = (Q= (Q.36)(23aa))
1x x +
where 4h = (ci o".).
The influence functions for the candidate matrices My and Mz estimated via SDR techniques have been well studied. For details, we refer readers to Section 4 Kim et al. 20200 and omit the derivations here.
B Experiments Details
B.1 Experiment Setup
Throughout the experiments,
the MSAVE method to estimate the candidate matrices My and Mz, with the number of slices set to p 1, where denotes the dimensionality of the input X. the post-SDR training procedure (as described in Algorithm 2), apply multivariate linear regression for synthetic simulations and logistic regression for weal-world datasets. The MCDP metric used to select the optimal fair model M( To estimate the rank of cach candidate matrix, adopt the ladle estimator with 30 bootstrap replications.
For baseline methods, we use the official code released by the respective authors. Specifically, for AdvDebias, FairMixup, DRAlign, and DiffMCDP, we use the implementation provided in Din et al.) [2024). For INLP, we set the number of iterations 100. For RLACE, we follow the training hyperparameters used the original paper. For SUP, we adopt the same number of slices (p + as in our method, and select the dimension to remove based on 10-fold cross-validation, choosing the model that achieves the lowest MCDP
B.2 Fairness Measurement
We consider multi-class classification setting where the target label Y IRK is one-hot encoded vector, i.c., Y; = 1 if the true label corresponds to class j and 0 otherwise. We denote the predicted probability vector Ý. and the sensitive attribute a binary variable Z {0, 1}.
22

---

Demographic Parity (DP) Gap. Demographic Parity requires that the predicted output is indepen- dent of the sensitive attribute. For each class j E {l..... K. define the group-wise expected predicted score as:
DP E| YY,
Z z/1
and the corresponding DP gap for class as:
DP gap.j DP
DPo.j
The overall DP gap across classes is then aggregated as:
K-1 (DP spp x 100%.
DP
gap
K
- ==1
When K = 2, this definition reduces to the binary case, where DP gap
[DP1, DPo.j.
True Positive Rate (TPR) Gap. For each class j E {l...., (4...,KK, K], the TPR for group {0, 13 is defined as:
TPR = :R=.=P((Y
2, Y, 1),
and the corresponding TPR gap is:
TPR TPRo.j We aggregate TPR gaps across all classes into a single fairness metric:
TPRg
TPR
gap
TPPR (TPR gop.) g x 100%.
K
J=1
Maximal Cumulative Ratio Disparity along Predictions (MCDP). For each class j. define the group-wise cumulative distribution function:
Fu)(() P(Y, S Z = =).
The MCDP for class then the Kolmogorov-Smie -miinoy distance between the distributions of predicted probabilities for the two sensitive groups:
MCDP,
max yε(0,
(ii (o)(
Since the components of Y sum to I due the softmax constraint, only of them are linearly independent. Therefore, we define the overall MCDP gap as:
K-1 (MCDP,)2 x 100%.
MCDP gap
K
j=1
When K = 2, this definition reduces to the binary case as in Jin et al. 2024
C Proof of Theorems
Proof of Theorem 3.1
Let B be orthonormal basis for the central subspace Sy that E[Y
X Y B X]. Since Q2 X is a measurable function of X, and Y 1 X B X, it follows
that E Y 1Q = Y B X). Noting that B Q.X (Q. X, conclude that
Y Q E|Y (Q x).
which implies that Q. B spans a sufficient subspace for Y with respect Q.X.
By definition, the central subspace for Q is the minimal subspace satisfying this conditional independence. Therefore, if B is any orthonormal matrix such that E[Y Q:X] = E X), must hold that
Span (3) C (pan(Q.F)
and Span (B) Span(Q.B) if and only if rank B) (ank(Q.B))
23

---

Proof of Theorem 3.2 We begin by substituting the model into the expected emror:
B((Y-f(m) =
(
f(m))2 + (X) f(m))yy + E
Take expectations term- ferm-by-term by-term. For the first term, we observe that f(m) B|y = |. which is the projection of Y onto coarser sigma-algebra. Therefore fimm) B (x)
= ] and
E(( - f(m)) | B|Var(f" (f*()) (X) E {])
For the second term, since Eleyy
E [m)] E X = (m))
0, have:
[[["(X]] (m)) )ey = 0.
Finally, the third term is simply E[ = o.- Putting everything together gives:
E[[(Y - f(m) )²] [[ E (m)) +
Note that ((m)( (m)) is the orthogonal projection of (X( == E| X onto (E)))) in L
Since m (m++) the projection becomes finer, and by the Pythagorean identity in L
(X) f(mm 112 (X) f(m> 1|:2.
Hence, the approximation error A (mm) E[Var( X) E (m)) satisfies A(m + 1) S A(m).
Proof of Theorem 3.3 From the definition of distance covariance for scalar X and binary Z, consider the V-statistic form:
dCov? dCCov2(X, 2) E[Jx X" -- - 2'1] + B|| x'I XII - z'1] - 2B|]-2[[ X'I 2 -
Specializing Z E {0, 1), we know:
EliZ Z'|] = BllZ = p),
and I Z"1 only when Z + Z', meaning cross-group expectation. Then:
B[X- X' 12 Z'] = 2p(1 - p) B X1 [] E|X X' IZ - z"1] = 2p(1 - p) E|| - X'[].
So:
dCov² (0, Z) 2p(1 -) (EIIX, X1I| - X') Substituting X x= f(m), obtain the stated identity. Specifically, when 0, we have fim)
((m) ((m) and thereby the distance covariance equals zero.
Proof of Theurem 4.1
Following Luo 2022
Span ( ( is the unique solution of
My Pr.o Mz My MM,,
(13)
where, for any full rank B, Pr, the E-orthogonal projector onto Span B}.
Observe that
== Po /2/ where U E1E2/ Pu P ---
Substituting into yields
My 1/2 Pt 1/22 Mz = My EM
Left- and right-multiplying by Σ 1/2 gives
AP E, A= My 1/2
== == - 1/2 = AB - 1/22Myy2222m EMz EMNΣ-12
24

---

Since range(AA) C rUnge(U) and range B) C range(U), we have A = A and Po B = B. It
follows that
E Po E.
Because E' is symmetric, its nonzero-eigenvalue subspace coincides with range(U). Writing the spectral decomposition
E ==VAV, vVv=1,
and setting 인 = د (/2)) (the s leading eigenvectors), we recover the equivalent generalized
eigenproblem
(My Σ M2) Ø = AΣΦ,
whose top 8 eigenvectors {or} span Span( (+).
Proof of Theorem 5.2 It is well-known that, if (((z) (F) is Hadamard differentiable, then satisfies the following expansion
glm) (Fn, p(m) (Fn)) = ((m²) (Fo, ((mm)((o)
(Fo)) )= E. [0(m))] + +p (/13
n
= B(()) (Fo, pmm) (Fo)) + g(m)a (S.. (lm)) (Fr)) + (n
By Theorem and Proposition
Kim et al. 2020). since L(S, 0) is differentiable, we have
[D0(Fu, p(m) (Fo)) (D
(S,) p(m) (Fa) ((m)) (S, p(m)) (Fo))
veel (p((()))(()))
① vee (((m)
= -- V (SL() alm) + (80(Fo, Fo, p(m)) (Fo)))
@ve((((m) )
veo veo(p(m)=(s))
where H-1 ((m)
LLS {im))) Therefore, have
@m)) = m)
1
mlm) (S (()) (Fx)) + op
+
n2
i=1
= mm) +
1
H=1 0lm)
VoL(S(m),,mn)) Ttm)) + (00(Fo, P(m) (Fo))
vee( (ee((p)))(ss)) + to 1/2)
i ((me)
1=1
Consequently, by the Central Limit Theorem
Vn(O ( m(m)) 24 N (o, Var - ((m)
VOL(S)m) ((m)) + D(m) veo (ec(p()))))
where ((m) (00(Fo, p(m)
Proof of Corollary 5.3 Similar as shown in Theorem 5.2
the original estimator satisfies the follow-
ing expansion
0, = 0
(S.) + on
1=1
n 0 + n
H VeL(S + on
Then we have (g(m)
θ.)) (0(m)) -0)
1
11 V VoL(S,, mim)) Him) VeL(S!!M ((m))+ + (a0(Fo, p(m) (ec(p(m))(ss)
vecl pim))
n1
Therefore, by Central Limit Theorem, we have
1 B((0Am)) 0..13) < [|(m)] 012 Tr (Var g
G H-11 ((m) + D(m) veo (p(m)))))
Ø(m)
n
25

---

Proof of Theorem 5.4
Consider the first-order Taylor expansion of fx;; 0)
[w; 0lm))
[(x; ((m) Va (x; (()) (0,mm) T(m) al o(1 1).
From Theorem 5.2 we know |0 @( = (n 1/2).. Then, following the same expansion shown in the proof of Theorem
5.2. along with Slutsky's theorem, gives the results.
D Limitations
The current work builds upon linear SDR, which assumes that both the target label and the sensitive attribute depend on linear projections of the representation. While this is a relatively mild assumption, limits our ability capture and mitigate nonlinear dependencies that may be embedded in the representation.
A promising future direction is to extend the framework using nonlinear SDR theory. In this approach, the representation is first mapped into a nigher-dimensional feature space using predefined kernel function, denoted by Ker(X ) The goal is then find a projection matrix B₂ such that
IL B Ker(X),
enforcing conditional independence in the nonlinear space. Importantly, the core derivations and structural components of our method remain applicable in this generalized setting. Developing this extension a promising direction for future research.
E
Impact Statement
The framework proposed in this paper tackles the fundamental challenge of fairness in machine learning by directly mitigating bias learned data representations. Departing from post-hoc de- biasing methods, enforce faimess for new representations, enabling scalable and generalizable solutions for sensitive domains such as employment, healthcare, and finance. By balancing fairness and predictive utility through subspace decomposition, SFP helps reduce systemic disparities and promotes the development of trustworthy Al systems. This work underscores the importance of rigor- fairness evaluation and responsible deployment advance equitable and accountable machine learning technologies.
26


Reviews:
[{"review_id": "e8wDA2P2pd", "reviewer": "Reviewer_L3zY", "summary": "In this work the authors argue to look at fairness through representational decomposition, analyzing the separate subspaces\nof a model representation that contain information pertaining to a sensitive attribute, to a target variable, and to both. Their\nanalysis provides a method of interpolating between maximum utility (and worst fairness) and minimum utility (and best\nfairness) by removing more and more of the shared subspace between the two variables. Through extensive theoretical\nanalysis, they demonstrate the utility of their decomposition as a way to debias representations and propose an algorithm to\ncreate fair predictors.", "questions": "* How do these estimators scale with data and representation space size? In your setting you focused on dimensionality\nof 16 in the synthetic setting and of a similar scale for Adult. However, some of your feature embedding baselines scale\nup to LLM feature spaces on the order of hundreds if not thousands of dimensions. Would this method be competitive there?", "limitations": "The authors have not discussed the limitations, but I do not think there are any critical points missing.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n * The paper provides rigorous theoretical analysis of their approach, demonstrating that their approximation results in non-\nincreasing error for the Bayes predictor as they include more of the shared subspace. Additionally, the authors have\nexperiments (both synthetic and on real-world data) which validate the efficacy of their approach on established fairness\nmetrics.\n\n## Weaknesses\n * In the predictive settings evaluated in the paper, there are many existing (strong) methods for fairness. In fact, part of your\nargument is that moving beyond predictive intervention is important. I think a key piece of evidence missing in this paper is a\nsetting where predictive interventions fail, or where representational fairness is important.\n * Additionally , you may want to look to work on privacy and specifically the privacy funnel [1], where a similar task is at hand:\nhow can we modify X such that it provides no information about a private attribute Z while maintaining maximum utility for Y.\n\n[1] Calmon, F. P., Makhdoumi, A., & Médard, M. (2015, June). Fundamental limits of perfect privacy. ISIT 2015", "was_revised": true, "final_justification": "I think this paper will be a good contribution to NeurIPS and deserves an accept. The authors answered my questions during the review process as well and communicated clearly about the message and motivation of the work.", "rebuttal": "We thank the reviewer for recognizing that our work is well-motivated and grounded in solid theoretical foundations, and we appreciate the constructive feedback. We will address the remaining clarifications below point by point.\n\n## Weakness\n\n**W1**: We do not claim that predictive approaches are inadequate or must be entirely replaced. Instead, our contribution lies in offering additional insights into achieving intermediate levels of fairness and providing interpretability for the fair representations we construct. While the predictive setting is the most commonly used approach for achieving fairness, typically by incorporating a fairness penalty term during training, it often suffers from optimization instability. This is because the prediction loss and the fairness loss can be inherently conflicting, leading to convergence issues and fluctuating training dynamics. Moreover, predictive fairness methods are usually task-specific: if one wishes to apply the learned model to a different downstream task or to enforce a different type of fairness constraint, it often requires retraining the model from scratch.\n  \nIn contrast, our approach focuses on representational fairness, which seeks to remove information about the sensitive attribute directly from the learned representation. A key advantage of this approach is that it offers a clear and interpretable trade-off between fairness and utility: by controlling the number of dimensions, we can balance fairness constraints against predictive performance. Moreover, once such a fair representation is obtained, it can be reused across multiple downstream tasks or fairness definitions without requiring retraining. This flexibility makes representational fairness particularly appealing in multi-task or evolving fairness scenarios, as it decouples fairness enforcement from task-specific prediction objectives.\n\nWe will incorporate this discussion into the related work section in the revised version of the manuscript.\n\n**W2**: We thank the reviewer for suggesting this reference. Indeed, it shares a similar motivation, which aims to ensure privacy by minimizing the information content related to sensitive variables in the learned representation. This aligns well with our fairness perspective, and it opens an interesting direction for extending our framework. Specifically, we can consider non-linear transformations of representations and formalize fairness objectives using dependence measures such as mutual information or distance covariance. For example, by minimizing the dependence between the representation and the sensitive attribute (e.g., minimizing $I(X,Z)$ while preserving utility (e.g., maximizing $I(X,Y)$, we can construct fair representations in a more general, potentially non-linear setting. We will incorporate this perspective into the conclusion and future work section of our revised manuscript.\n\n## Questions\n\n**Q1**: The computational cost of obtaining the fair projection matrix primarily arises from estimating the candidate matrices $M_Y$, $M_Z$ and $M_{Y,Z}$, each typically constructed as a weighted covariance matrix. These computations scale linearly with the sample size $n$. In addition, the procedure includes an eigen-decomposition step for $p\\times p$ matrices, and the rank estimation step scales linearly with dimension $p$. Notably, our method remains applicable in high-dimensional settings, as SDR techniques are specifically designed to extract low-dimensional structure from high-dimensional data. \n\nAs for performance, it largely depends on the structure of the original representation. Since our current framework focuses on linear transformations, it may fail to capture non-linear dependencies. One potential solution is to extend our approach using non-linear SDR techniques, such as kernel-based methods, as discussed in the limitations section of the appendix. Alternatively, as mentioned in our response to your comment on weaknesses, we can explore information-theoretic approaches (e.g., mutual information) to handle non-linear relationships in the representation learning process.\n\n## Paper Formatting Concerns\n\nWe thank the reviewer for pointing out the typos and potential confusion notations, we will revise them in the final manuscript. In line 147, $B$ is the standardized eigenvectors of $M_Y$, which makes $BB^{\\top}$ a projection matrix without losing any information relates to $Y$.\n\n**We appreciate your feedback, which has helped strengthen our work, and we hope our responses have addressed all of your concerns. Please let us know if there is any additional information or clarification we can provide to further support your understanding and confidence in our work.**"}, {"review_id": "01bl6MSHHd", "reviewer": "Reviewer_Bsk4", "summary": "In the present work, the authors introduce Sequential Fair Projection (SFP), a representation-level debiasing framework based on linear sufficient dimension reduction (SDR): the approach entails (i) decomposing the feature space into label-only, sensitive-only, and shared subspaces; (ii) supplying a family of projection matrices that can re-inject up to m shared directions to trade off fairness and accuracy; (iii) analyzing the prediction error vs distance covariance-based unfairness performance. The practical recipe proposed by the authors is to pick the smallest shared sub-space projection whose validation accuracy is at least 95 % of the original model's performance. The authors validate SFP on synthetic data and two tabular benchmarks (Adult and Bank), where it attains competitive fairness with modest accuracy loss.", "questions": "On mixture of label/group information: Can the authors justify and empirically test the assumption that a linear projector might suffice when label and sensitive information are non-linearly entangled? What kind of issues could appear if there is a strong violation of this hypothesis?\n\nOn the chosen fairness-accuracy trade-off: Why choose the ad-hoc “≥ 95 % accuracy” cut-off? How sensitive are results to this hyperparameter? Is there some guarantee that one obtains the \"fairest\" model with that accuracy level? \n\nOn scalability in high-dimensional settings: What is the wall-clock cost of estimating SDR subspaces and looping over m on datasets at the scale of ImageNet, or transformer embeddings?\n\nOn the practical use of Theorems 5.2-5.4: How should a practitioner act on the given asymptotic variance bounds?\n\nEvaluation scope – Why restrict experiments to two small tabular datasets with binary sensitive attributes? Have the authors tried multi-attribute settings or modern vision/NLP benchmarks?\n\nAccuracy–fairness trade-off – On Adult, LR is far more accurate yet only moderately less fair. Could simpler post-hoc methods (e.g., threshold adjustment) close the gap?\n\nReproducibility – Code is promised, but hyper-parameters for SDR (slice count, rank estimation) and optimisation are missing from the main paper.\n\n4. Limitations (as I see them)", "limitations": "Some limitations are discussed in a paragraph at the end of the appendix. I believe that, in the spirit of transparency, they should be moved to the main text.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths** \n- The manuscript tackles fairness within representations, a worthwhile direction supported by clear notation and an SDR-based derivation. \n- The presented influence-function analysis appears technically sound and may interest theory-minded readers. \n\n**Weaknesses**\n- The contribution feels somewhat incremental because prior work (e.g., Shi et al. 2024) already projects away sensitive subspaces, and iterative “add-back” heuristics are common in fairness fine-tuning. \n- Methodologically, all results hinge on linear SDR projections, both for disentangling and for intersecting subspaces. This assumption is not necessarily realistic in modern representation learning, yet taken for granted in the main (a quick discussion is presented in the limitation section at the end of the appendix). Quantifying the degree of departure from this hypothesis in realistic settings would seem to be a necessary discussion topic in this work.   \n- The empirical evaluations appear limited: experiments cover only two small tabular datasets; no large-scale or pretrained-model study supports the broader narrative, or the competitiveness of the proposed method. \n- The 95 % accuracy threshold and the feature-by-feature loop add a degree of arbitrariness and manual tuning (as many other methods that strike a balance between performance and fairness) and could largely increase computational cost in large-dimensional problems.\n- There are some repetitions and typos, and some observables that would be useful for the reader to interpret the empirical evaluation of the proposed method are left in the appendix.", "was_revised": true, "final_justification": "With all the promised additions, I believe my main concerns have been addressed adequately in the rebuttal, and I am inclined to raise my score to a 4:borderline accept.", "rebuttal": "First, we thank the reviewer for recognizing that our work is well-motivated and grounded in solid theoretical foundations, and we appreciate the constructive feedback.\n\nMoreover, we would like to respectfully clarify that the primary concern regarding our use of linear SDR methods has already been discussed in the Limitations section. As discussed below, our work represents the **first attempt** to address the fairness–utility trade-off through a **theoretically grounded representation learning framework**, closing the gaps in existing methods: lack of explicit control over fairness–accuracy trade-offs and the absence of theoretical guarantees. We fully agree that extending the framework to nonlinear setting would be a valuable direction for future work; we view these as natural and promising evolutions of our approach rather than shortcomings of the current contribution.\n\nWe will address the remaining clarifications below point by point.\n\n## Weakness\n\n**W1**: We appreciate the reviewer’s comments that our work shares a common tool, i.e., Sufficient Dimension Reduction, with Shi et al. (2024), and that related add-back methods also aim to achieve an intermediate level of fairness by balancing fairness and predictive accuracy. However, our contribution differs from these approaches in two important ways:\n1. First, unlike Shi et al. (2024), which completely removes the influence of the sensitive attribute $Z$ from the representation of covariates $Z$ without leveraging label information $Y$, our method addresses a fundamentally different problem setup. We adopt a supervised approach that explicitly incorporates $Y$ to decompose the predictive information in $X$ into two components: (i) the part that is orthogonal to $Z$, and (ii) the part that is shared with $Z$. This label-informed decomposition enables more effective identification of useful predictive signal.\n\n2. Second, while existing ``add-back’’ heuristics attempt to recover lost utility after fairness constraints, they are typically ad hoc and lack theoretical guarantees. In contrast, our approach formalizes the decomposition of shared and orthogonal information among $X$, $Y$ and $Z$, resulting in a method that is both interpretable and supported by statistically theory. And importantly, our principled formulation enables one to explicitly tune the fairness–accuracy trade-off while providing transparency on retained versus discarded information into the learned.\n\n**W2**: You are right that our proposed method is built on linear sufficient dimension reduction (SDR) projections, and as discussed in the Limitation section, violations of this assumption may affect its effectiveness. However, quantifying the degree of departure from this assumption is a non-trivial problem and depends on several factors, including the data distribution, signal-to-noise ratio, and the nature of the downstream task, as widely acknowledged in the SDR literature. We will clarify this point further and acknowledge it as an important direction for future research.\n\n**W3**: We appreciate the reviewer’s comment. Given the theoretical and methodological focus of our work, our goal in this paper is to present a principled framework rather than to conduct extensive, large-scale benchmarking. As is common in theory-oriented papers, we validate our approach on representative tabular datasets to illustrate the key properties and practical performance of the method. While we agree that evaluating on large-scale or pre-trained models would further support the broader narrative, we view this as a natural next step and plan to pursue such empirical validation in future work to complement the current theoretical contribution.\n\n**W4**: We thank the reviewer for this thoughtful observation. However, we would like to clarify that our method does not require the use of any accuracy threshold. The 95\\% threshold is chosen to ensure a fair comparison across different methods. All methods and results reported in Table 1 follow this same criterion for hyperparameter tuning. Specifically, this threshold allows us to assess how fair the trained models are when maintaining a reasonable level of predictive performance. Users can easily adjust this threshold based on their specific tolerance for accuracy sacrifice in exchange for fairness.\n\nRegarding the feature-by-feature loop, it is designed to systematically identify and incorporate informative components by allowing a controlled level of unfairness, thereby achieving an intermediate point in the fairness-utility trade-off. This loop functions as a tuning parameter with improved interpretability. Unlike optimization-based methods that also require tuning a penalty coefficient $\\lambda$, our approach offers explicit control and provides two main advantages: (1) the shared dimension is selected from a finite set of values, avoiding the need for grid search over a continuous $\\lambda$ range, and (2) the interpretability of each added component allows practitioners to better understand the trade-off dynamics.\n\nImportantly, the number of shared dimensions is determined by the degree of entanglement between $Y$ and $Z$, rather than the dimensionality of the representation space. Thus, even in high-dimensional settings, the number of shared components may remain small.\n\n**W5**: We thank the reviewer for pointing out the repetitions, typos, and the placement of key observables in the appendix. We will carefully revise the paper to eliminate redundant text and correct all typos. We will move key summaries and metrics into the main text and clarify their relevance to the evaluation of our method.\n\n## Questions\n\n**Q1**: Conditional independence tests can be employed to assess whether the estimated central subspaces are sufficient to capture all relevant information. For example, one can statistically test whether $Y \\perp X \\mid M_Y X$ and $Z \\perp X \\mid M_Z X$ hold. A number of existing works focus on this problem, such as the paper \\textit{``Kernel-based Conditional Independence Test and Application in Causal Discovery'' (UAI 2011)}.\n\nIf the linear SDR assumptions are strongly violated, then the estimated central subspaces may not adequately represent the conditional structure, and the matrices $M_Y$ and $M_Z$ may become full rank (i.e., rank $p$), indicating the need to include all dimensions. In such cases, we can still manually choose how many directions to retain in the post-SDR training based on the ordering of the eigenvalues, and form a fair projection accordingly. \n\n**Q2**: Our theoretical analysis and fairness utility trade-off does not rely on this 95\\% threshold. It is intended solely to ensure a fair comparison between different methods. We will clarify this in our revised version.\n\n**Q3**: The computational cost of obtaining the fair projection matrix primarily arises from estimating the candidate matrices $M_Y$, $M_Z$ and $M_{Y,Z}$. These computations scale linearly with the sample size $n$. In addition, the procedure includes an eigen-decomposition step for $p\\times p$ matrices, and the rank estimation step scales linearly with the ambient dimension $p$. For reference, the computation times for obtaining the all SDR subspaces $M_Y$, $M_Z$ and $M_{Y,Z}$ and their ranks on the Adult and Bank datasets are 80.47 seconds and 50.22 seconds only use CPU, respectively. We did not conduct additional experiments on large scaled image and contextual datasets as our work focuses on providing theoretical analysis and explanation, but it is an interesting task worth exploring for nonlinear SDR setting in the future work. \n\n**Q4**: Theorems 5.2--5.4 provide practitioners with statistical uncertainty quantification results for parameters estimated using the fair representation as well as their impact on predictive performance. These results enhance the interpretability of our framework by elucidating how the choice of dimension and the specific SDR technique employed influence the resulting parameter estimates. The practitioners can also benefit from constructing confidence intervals for the estimated parameters.\n\n**Q5**: The purpose of our simulation and real data experiments is to demonstrate that our proposed framework is effective in practice and that the underlying idea is sound. Our main contribution lies in the theoretical analysis and the insights it provides into the fairness-utility trade-off through the lens of subspace interactions.\n\nIn more complex scenarios, such as those embeddings with intricate non-linear relationships, our linear SDR methods may be less effective in producing fair representations. Nonetheless, the core idea behind our approach remains applicable. By extending our framework to incorporate non-linear structures, e.g., through kernel-based SDR or information-theoretic formulation, we aim to close this gap in future work.\n\n**Q6**: Although the value of TPR gap is low for both the Adult and Bank datasets when using logistic regression, the DP and MCDP gaps remain high, indicating substantial group-level unfairness in terms of both expectations and distributions. Therefore, simply using post-hoc methods, such as threshold adjustment, can not address the underlying fairness issues.\n\n**Q7**: The number of slices is chosen as $p+1$, as stated in Section B.1. The parameters for rank estimation follow the same settings as presented in the original paper. For post-SDR training, we apply the baseline logistic regression model directly on the fair representations for both Adult and Bank datasets. We will clarify these implementation details in the revised version of the manuscript.\n\n**We appreciate your feedback, which has helped strengthen our work, and we hope our responses have addressed all of your concerns. Please let us know if there is any additional information or clarification we can provide to further support your understanding and confidence in our work.**"}, {"review_id": "pyGIGNAGiK", "reviewer": "Reviewer_6n9b", "summary": "This work proposes a new approach to improve the fairness of algorithmic predictors by addressing the biases in the learned representations of the underlying data. The authors propose a novel approach using *sufficient dimension reduction* that decomposes the data space into sensitive attribute relevant, target variable relevant, and sensitive-target shared components. Using this decomposition, the work shows how the fairness-accuracy tradeoff can be navigated by leveraging these separate representation components. The authors provide a theoretical analysis and also empirically validate their approach on two popular fairness datasets.", "questions": "* How does the proposed approach's efficiency, specifically using SDR for dimension reduction, compare to other neural representation learning based approaches that look into disentanglement or fairness?\n* Can the proposed approach with linear SDR decomposition also be applied in conjunction with a strong representation learning method for vision and textual settings?", "limitations": "Limitations are not discussed in the main paper. However, a brief discussion on technical limitations is provided in the appendix.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "## Strengths\n- The work provides a formal framework to study fairness from the perspective of representation subspaces by decomposing it into relevant subspaces. By having separate subspaces for sensitive features, target variable, and also having a subspace focused on the shared relations between the sensitive and target variables, the authors show how we can effectively navigate the accuracy fairness tradeoff.\n- The approach is theoretically well-motivated, and the authors also provide a detailed theoretical analysis of their approach regarding predictive error and fairness tradeoffs. After using SDR to reduce dimensionality and finding the relevant subspaces, finding the most optimal fair predictor reduces to finding the feature subset that can provide high accuracy and low unfairness.\n- Using synthetic data, the authors validate the theoretical results and show how increasing the number of shared components in the predictive process can improve the predictive performance *for all sensitive groups* but also *increase the performance gap* across the different groups, leading to potential unfairness.\n- The authors also show how their method is effective in achieving good accuracy and fairness results in two real-world datasets (Adult and Bank).\n\n## Weaknesses\n- Since this work is highly related to prior works on *fair representation learning*, a more thorough discussion and comparison with more fair representation learning works would have been effective to position the authors' approach. Specifically, it would have been nice to have a discussion and also comparisons to approaches that have looked at fair representation learning, representation disentanglement, e.g., [1] and [2]. Works that have looked at *causal representation learning*, e.g., the work in [3], could also be relevant for a discussion and/or comparison.\n- The author's approach relies on a linear SDR method, and it is unclear how well the method can work if it were necessary to extend it to more nonlinear or complex settings. To this end, the empirical evaluations are limited, focusing on only two tabular datasets without looking into image (e.g., CelebA) or text (e.g., BiasBios) datasets.\n- It would be nice to have an understanding of the efficiency of the proposed method when compared to competing approaches mentioned by the authors, e.g., AdvDebias, and other deep-learning-based representation learning approaches cited here.\n\n[1] Creager, Elliot, et al. \"Flexibly fair representation learning by disentanglement.\" International conference on machine learning. PMLR, 2019.\n\n[2] Louizos, Christos, et al. \"The variational fair autoencoder.\" arXiv preprint arXiv:1511.00830 (2015).\n\n[3] Yang, Mengyue, et al. \"Causalvae: Disentangled representation learning via neural structural causal models.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021.", "was_revised": true, "final_justification": "The authors provide a new theoretically grounded application to improve fair representation learning that can maintain accuracy. After the rebuttal, I think the work is interesting, but the paper can be improved by having slightly more evaluations, and at a minimum, a more intuitive discussion of the theoretical results and analysis conducted.", "rebuttal": "First, we thank the reviewer for recognizing that our work is well-motivated and grounded in solid theoretical foundations, and we appreciate the constructive feedback.\n\nMoreover, we would like to respectfully clarify that the primary concern regarding our use of linear SDR methods has already been acknowledged in the Limitations section. As discussed below, our work represents the **first principled attempt** to address the fairness–utility trade-off through a **theoretically grounded representation learning framework**, directly targeting key gaps in existing methods, namely, the lack of explicit control over fairness–accuracy trade-offs and the absence of theoretical guarantees. As is often the case with methodological contributions, it is expected that subsequent work will address the limitations of the initial framework. Indeed, we fully agree that extending the framework to nonlinear and large-scale domains (e.g., image or text data) would be a valuable direction for future work; we view these as natural and promising evolutions of our approach rather than shortcomings of the current contribution.\n\nWe will address the remaining clarifications below point by point.\n\n## Weakness\n\n**W1**: We thank the reviewer for pointing us to these additional references, which are highly relevant. However, we would like to clarify that our method differs significantly from these prior works, as outlined below:\n\nFirst, prior methods such as Creager et al. (2019) and Louizos et al. (2015) **lack explicit control** over the fairness–accuracy trade-off. These approaches impose fairness constraints through disentangled variational autoencoders and rely on tuning hyperparameters to achieve different levels of fairness and utility. However, they do not offer explicit guarantees or clear interpretability regarding how these trade-offs are managed. In contrast, our method is grounded in a statistical decomposition of the covariates $X$ into components that are predictive of the target label $Y$ and either orthogonal to or shared with the sensitive attribute $Z$. This results in a representation that is both **interpretable and transparent** on the information discarded and retained, **allowing for explicit and principled control** over the fairness–accuracy trade-off.\n\nSecond, these prior methods rely on **optimization-based frameworks** involving variational autoencoders with adversarial or mutual information-based regularization. Such approaches are often sensitive to hyperparameter tuning and prone to convergence issues. In contrast, **our approach requires no model training, and is supported by theoretical guarantees**, thereby avoiding the instability, lack of transparency, and limited interpretability associated with adversarial learning.\n\nCausalVAE (Yang et al., 2021), while conceptually related in its goal of learning disentangled and causally interpretable representations, is not directly focused on fairness. Its framework is aimed at generative modeling through causal factorization and intervention, rather than fair prediction. When applied to fairness settings, CausalVAE would likely encounter the same two limitations noted above: **lack of explicit fairness–utility control and reliance on complex optimization procedures.**\n\nWe will incorporate this discussion into the related work section of the revised paper.\n\n**W2**: We appreciate the reviewer’s insightful comment. It is true that our approach is built on a linear sufficient dimension reduction (SDR) framework, and as discussed in the Limitation, this assumption may limit applicability in highly nonlinear settings. However, our primary goal in this work is to develop a principled and interpretable framework for balancing fairness and utility through explicit statistical information decomposition.\n\nIn line with our theoretical contribution, we evaluated the method on two widely used tabular datasets that allow for clear, interpretable assessments of fairness–utility trade-offs and are commonly used in fairness research. While our current implementation focuses on structured tabular data, we agree that evaluating the method on higher-dimensional and unstructured domains (such as CelebA or BiasBios) would be valuable. We view this as an important next step, and note that the subspace decomposition ideas we introduce could be extended using nonlinear SDR techniques (e.g., kernel or deep SDR), which we plan to explore in future work. \n\nNon-linear SDR techniques are indeed promising and are expected to outperform linear SDR methods, particularly on complex datasets such as those involving images and text. A straightforward extension would be to perform fair representation learning using neural network architectures that explicitly decorrelate the learned representations from sensitive variables. However, analyzing such models theoretically poses greater challenges. We believe this is a highly promising direction and plan to explore it in future work.\n\n**W3**: We are unclear about which sense of efficiency the reviewer is referring to, as statistical and computational efficiency are conceptually distinct.\n\nIf the comment concerns ***statistical efficiency***, this typically refers to the convergence rate or asymptotic variance of an estimator. Our method achieves a convergence rate of $1/\\sqrt{n}$ for all subspace estimators and post-SDR parameters. In contrast, most deep learning-based approaches lack theoretical guarantees on convergence or asymptotic behavior, making direct statistical efficiency comparisons infeasible.\n\nIf the concern is about ***computational efficiency***, we note that our method avoids any optimization during subspace estimation. It relies only on matrix multiplications and closed-form calculations, enabling efficient execution on CPUs without the need for GPU acceleration. For instance, on the Adult and Bank datasets, our method takes 80.47 and 50.22 seconds, respectively, using only CPU. In comparison, deep learning-based methods such as AdvDebias, FairMixup, DRAlign, and DiffMCDP require iterative training and take approximately 110 and 90 seconds, while RLACE takes 155 and 127 seconds, respectively. This highlights that our method is more lightweight and accessible in terms of computational resources.\n\n## Questions\n\n**Q1**: See our reply to **W3**.\n\n**Q2**: Yes, our method can be applied to high-dimensional settings, as SDR techniques are designed to extract low-dimensional structure from high-dimensional data. The performance of our approach depends on the structure of the original representation, and since our framework is limited to linear transformations, it may not fully capture non-linear dependencies. To address this, one could extend our method using non-linear SDR techniques such as kernel-based methods (as discussed in the appendix), or adopt information-theoretic approaches (e.g., mutual information) to better model non-linear relationships.\n\nWe acknowledge that in complex settings, such as image and text embeddings, the assumptions of linear SDR may not hold, where our method may not be the most effective one. Nonetheless, the core idea of subspace disentanglement remains applicable. Developing a theoretical and interpretable non-linear extension is a promising yet challenging direction, which we will highlight in the revised manuscript.\n\n**We appreciate your feedback, which has helped strengthen our work, and we hope our responses have addressed all of your concerns. Please let us know if there is any additional information or clarification we can provide to further support your understanding and confidence in our work.**"}, {"review_id": "t4oG4OFYre", "reviewer": "Reviewer_MkjP", "summary": "This paper proposes sequential fair projection, a fairness-aware learning framework that (i) decomposes feature space via sufficient dimension reduction into three orthogonal sub-spaces (Y-specific, Z-specific, Y-Z correlated) and (ii) controls fairness-utility trade-offs by incrementally adding shared components to the projection. Theoretical guarantees and numerical justifications are provided.", "questions": "(i) How to extend the current results to the more general $Z$ settings other than the current binary one?\n\n(ii) The SDR procedure may be costly for a high-dimensional covariate $X$. What is the complexity of your algorithm?\n\n(iii) Is it possible to derive a finite sample guarantee (asymptotic results are already provided in the paper)?\n\n(iv) Please discuss potential intersectional fairness. For example, multiple sensitive attributes exist.", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths: The paper is well-written and easy to follow. The theoretical results are rigorous, with comprehensive numerical results. And the idea is also novel.\n\nWeaknesses:  (i) The sensitive variable is binary in the theoretical and numerical results; (ii) The SDR may be costly for high-dimensional X; (iii) Finite-sample performance is not discussed.", "was_revised": true, "final_justification": "I suggest accepting the paper.", "rebuttal": "We thank the reviewer for recognizing that our work is novel, well-motivated and grounded in solid theoretical foundations, and we appreciate the constructive feedback. We will address the remaining clarifications below point by point.\n\n## Weakness\n\nSince the reviewer raised the same concerns in both the weaknesses and questions sections, we provide a consolidated response under the questions section only.\n\n## Questions\n\n**Q1**: Our framework does not assume any specific structure for $Z$ in the model assumptions (Equation (3)). In particular, $Z$ is not restricted to being binary; the SDR framework naturally allows $Z$ to be scalar or vector-valued, and either categorical or continuous. However, since most existing fairness evaluation metrics (e.g., EO or DP) are defined for binary sensitive attributes, and most benchmark datasets contain only binary sensitive attributes, our experiments focus on that setting.\n\n**Q2**: The computational cost of obtaining the fair projection matrix primarily arises from estimating the candidate matrices $M_Y$, $M_Z$ and $M_{Y,Z}$, each typically constructed as a weighted covariance matrix. These computations scale linearly with the sample size $n$. In addition, the procedure includes an eigen-decomposition step for $p\\times p$ matrices, and the rank estimation step scales linearly with dimension $p$.\n\n**Q3**: The finite sample analysis would be extremely challenging due to the chain of mappings described in Section 5.2, line 233. Existing theoretical results show that each individual step in the process achieves a $1/\\sqrt{n}$ convergence rate to the population level, allowing us to drop higher-order terms in asymptotic analysis. \n\nHowever, for a finite sample analysis, it would require us to carefully characterize the error propagation across multiple stages of the estimation pipeline. This includes quantifying the cumulative effect of the estimation errors in the SDR subspaces, the projection matrices, and the final predictive model. Such an analysis would involve bounding the composition of errors and developing non-asymptotic guarantees that depend explicitly on the sample size, dimensionality, and possibly the eigenvalue gaps in the estimated subspaces. We recognize this as an important and nontrivial direction for future work.\n\n**Q4**: Our setting already accommodates intersectional fairness. When considering multiple sensitive attributes, we can simply concatenate them to form a multidimensional $Z$ without altering the model assumptions in Equation (3). This requires no changes to our framework or estimation procedure, and all asymptotic results remain valid. We will clarify and illustrate this point in the revised manuscript.\n\n**We appreciate your feedback, which has helped strengthen our work, and we hope our responses have addressed all of your concerns. Please let us know if there is any additional information or clarification we can provide to further support your understanding and confidence in our work.**"}]

Author rebuttal:

Meta-review: This paper proposes a fairness framework that uses sufficient dimension reduction to decompose feature representations into target-relevant, sensitive, and shared components, thereby allowing explicit control of the fairness–utility trade-off. Reviewers found the work theoretically rigorous, noting the influence-function analysis and theoretical guarantees as strengths. They appreciated the clarity of the problem formulation and the interpretability of the trade-off.

Concerns were raised about the limited scope of the experiments, which were restricted to synthetic data and two tabular datasets, and about the reliance on linear SDR, which may not hold in more complex or nonlinear settings. Reviewers also pointed out that the contribution feels somewhat incremental given related work, and that the fairness–accuracy trade-off involved ad hoc choices such as the 95% accuracy threshold. Some reviewers requested additional discussion of related work on fair representation learning and privacy.

In the rebuttal, the authors clarified that their goal is to complement predictive fairness methods by offering representational fairness with interpretable trade-offs. They emphasized computational efficiency, provided further detail on how fairness metrics can be adapted, and acknowledged the linear SDR assumption as a limitation while suggesting extensions to nonlinear settings. Reviewers responded positively to these clarifications, with some maintaining their accept scores and others raising to borderline accept, while still noting that broader evaluations would strengthen the work.

Overall, reviewers agreed that the theoretical contribution is solid and the paper is well-motivated, despite the limited empirical scope. After the rebuttal, all reviewers supported acceptance, with ratings in the accept to borderline accept range. I therefore recommend acceptance.
Decision: Accept (poster)