Fair Canonical Correlation Analysis
Authors: Zhuoping Zhou
Davoud Ataee Tarzanagh
Bojian Hou
Boning Tong
Jia Xu
Yanbo Feng
Qi Long
Li Shen
Venue: NeurIPS 2023 poster (2023)
Keywords: Fairness
Canonical Correlation Analysis
Riemannian Optimization
Pareto Optimization
TL;DR: The paper introduces a framework to address fairness and bias in Canonical Correlation Analysis, ensuring comparable correlation levels across groups without sacrificing accuracy.

Abstract: This paper investigates fairness and bias in Canonical Correlation Analysis (CCA), a widely used statistical technique for examining the relationship between two sets of variables. We present a framework that alleviates unfairness by minimizing the correlation disparity error associated with protected attributes. Our approach enables CCA to learn global projection matrices from all data points while ensuring that these matrices yield comparable correlation levels to group-specific projection matrices. Experimental evaluation on both synthetic and real-world datasets demonstrates the efficacy of our method in reducing correlation disparity error without compromising CCA accuracy.

Fair Canonical Correlation Analysis
Zhuoping Zhou", Davoud Ataee Tarzanagh", Bojjan Hou Boning Tong, Jia Xu, Yanbo Feng, Qi Long", Li Shen
University of Pennsylvania
(zhuopinz@sas, ,, tarzanaqe, boningt@seas. jiaxu70, yanoofosaaaanoaaeeeee [bojian hou, glong, 1i. shen] opennmedicine www....e....n.n.nn.m.mmmmmmmm edu
Abstract
This paper investigates fairness and bias in Canonical Correlation Analysis (CCA), a widely used statistical technique for examining the relationship between two sets of variables. We present framework that alleviates unfairness by minimizing the correlation disparity error associated with protected attributes. Our approach enables CCA learn global projection matrices from all data points while ensuring that these matrices yield comparable correlation levels to group-specific projec- tion matrices. Experimental evaluation on both synthetic and real-world datasets demonstrates the efficacy of our method in reducing correlation disparity error without compromising CCA accuracy.
1 Introduction
Canonical Correlation Analysis (CCA) is a multivariate statistical technique that explores the rela- tionship between two sets of variables [30]. Given two datasets X NRD, and Y E Nx, on the same of N observations, CCA seeks the R-dimensional subspaces where the projections of X and Y are maximally correlated, i.e. finds U E RD.xR and V E R such that
maximize trace (U X YV) subject to U = VTY = IR. (CCA)
CCA finds applications in various fields, including biology [51]. neuroscience [2]. medicine [79], and engineering [14], for unsupervised or semi-supervised learning. It improves tasks like clustering, classification, and manifold learning by creating meaningful dimensionality-redue
representations [70]. However, CCA exhibit unfair behavior when analyzing data with protected attributes, like sex or race. For instance, in Alzheimer's disease (AD) analysis, CCA can establish correlations between brain imaging and cognitive decline. Yet, if it does consider the influence of sex, may result in disparate correlations among different groups because AD affects males and females differently, particularly in cognitive decline [36, 81].
The influence of machine learning on individuals and socicty has sparked growing interest in the topic of fairness [42]. While faimness techniques well-studied in supervised learning [5, 18, 20], attention is shifting to equitable methods in unsupervised learning [11, 12, 15, 34, 49, 55, 64]. Despite extensive work on fairness in machine learning, fair CCA (F-CCA) remains unexplored. This paper investigates F-CCA and introduces new approaches to mitigate bias (CCA).
For further discussion, we compare CCA with our proposed F-CCA in sample projection, as illustrated Figure Figure 1(a), we have samples xı and x₂ from matrix X, and Figure 1(b), their corresponding samples y and Y2 are from matrix Y. CCA learns U and V to maximize correlation,
Equal contribution a Corresponding authors
The columns of X and Y have been standardized.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

X c A.D.
Fomale Pair
XU, YV E
Male
R Paa
(d) Legend for srross
(c) Random Projection
CCA
samplee
Y e A
Two pairs samples aftrr projection
(g) CCA for Male
(h) CCA for Femalo
( Fair CCA
(b) Two samples in the second data metrix
Figure 1: Illustration of CCA and F-CCA, with the sensitive attribute being sex (female and male).
Figures (a)(c) (c) demonstrate the general framework of CCA, while Figures (d)- provide com- parison of the projected results using various strategies. It is important to note that the correlation between two corresponding samples is inversely associated with the angle formed by their projected vectors. F-CCA aims to equalize the average angles among different groups.
inversely related to the angle between the sample vectors. Figure I(c) demonstrates the proximity within the projected sample pairs (U x1, y1) and (U X2, V V y2). In Figure (dd)-ii, compare the results of different learning strategies. There are five pairs of samples, with female pairs highlighted in red and male pairs shown blue. Random projection (Figure ((e)) leads to randomly large angles between corresponding sample vectors. CCA reduces angles compared to random projection (Figure 1(f)), but significant angle differences between male and female pairs indicate bias. Using sex-based projection matrices heavily biases the final projection, favoring one sex over the other (Figures (g)) and I(h)). To address this bias, our F-CCA maximizes correlation within pairs and ensures equal correlations across different groups, such as males and females (Figure (()). Note that while this illustration represents individual fairness, the desired outcome practice is achieving similar average angles for different groups.
Contributions. This paper makes the following key contributions:
We introduce fair CCA (F-CCA), a model that addresses fairness issues in (CCA) by considering multiple groups and minimizing the correlation disparity eиror of protected attributes. F-CCA aims to learn global projection matrices from all data points while ensuring that these projection matrices produce similar amount of correlation group-specific projection matrices.
We propose two optimization frameworks for F-CCA: multi-objective and single-objective. The multi- oojeii-o framework provides an automatic trade-off between global correlation and equality in group-specific correlation disparity errors. The single-objective framework offers simple approach to approximate fairness in CCA while maintaining a strong global correlation, requiring tuning parameter balance these objectives.
We develop gradient descent algorithm the generalized Stiefel manifold to solve the multi- objective problem, with convergence guarantees to Pareto stationary point. This approach extends Riemannian gradient descent [8, multi-objective optimization, accommodating broader range of retraction maps than exponential retraction [23, 6]. Furthermore, we provide a similar algorithm for single-objective problems, also with convergence guarantees to a stationary point.
We provide extensive empirical results showcasing the efficacy the proposed algorithms. Com- parison against the CCA method on synthetic and real datasets highlights the benefits of the F-CCA approach, validating the theoretical findings 2
Section covers related work. Our proposed approach is detailed in Section 3, along with its theoretical guarantees. Section showcases numerical experiments, while Section 5 discusses implications and future research directions.
Organization:
"ode is available https: https://githul: cwwp/PemmSheenata/aaa......

---

2
Related work
Canonical Correlation Analysis (CCA), CCA was first introduced by [28, 29]. Since then, has
been utilized to explore relations between variables in various fields of science, including economics [72]. psychology [19, 27], geography [45], medicine [39], physics [76]. chemistry [69], biology
[62], time-series modeling [26], and signal processing [57]. Recently, CCA has demonstrated its applicability modern fields of science such as neuroscience, machine learning, and bioinformatics
[59, 60J. CCA has been used to explore relations for developing brain-computer interfaces [10, 46] and the field of imaging genetics [22]. CCA has also been applied for feature selection [47], feature extraction and fusion [61], and dimension reduction [71]. Additionally, numerous studies have applied CCA in bioinformatics and computational biology, such as [54, 56, 58). The broad range of application domains highlights the versatility of CCA in extracting relations between variables, making it a valuable tool in scientific research.
Fairness. Fairness in machine learning has been growing area of research, with much of the work focusing on fair supervised methods [5, 16, 18, 20, 67, 78]. However, there has also been increasing attention on fair methods for unsupervised learning tasks [11, 12, 15, 34, 33, 49, 55, 64, 50, 66]. In particular, Samadi et al. [55] proposed a semi- demineef programming approach to ensure fairness in PCA. Kleindessner et al. [33, 34] focused on fair PCA formulation for multiple groups and proposed a kernel-based fair PCA. Kamani et al. [32] introduced an efficient gradient method for fair PCA, addressing multi-objective optimization. In this paper, propose novel multi-objective framework for F-CCA, converting constrained F-CCA problems to unconstrained ones on a generalized Rieman- nian manifold. This framework enables the adaptation of efficient gradient techniques for numerical optimization on Riemannian manifolds.
Riemannian Optimization. Riemannian optimization extends Euclidean optimization smooth manifolds, enabling the minimization of fxx on a Riemannian manifold M and converting con- strained problems into unconstrained ones [1, 8]. It finds applications in various domains such as matrix/tensor factorization [31, 631, PCA [21], and CCA [77]. Specifically, CCA can be formulated as Riemannian optimization on the Stiefel manifold [13, 43]. In work, utilize Riemannian optimization to develop a multi-objective framework for F-CCAs on generalized Stiefel manifolds.
3
Fair Canonical Correlation Analysis
This section introduces the formulation and optimization algorithms for F-CCA.
3.1
Preliminary
Real numbers are represented as R, with R4 for nonnegative values and R++ for positives. Vectors and matrices use bold lowercase and upperease letters (c.g.. a, A) with elements a, and aty. For x. y e RT X < y and x I mean y - X E IT+ and y - x E RT, respectively. For a symmetric matrix A E R N A > 0 and A > 0 denote positive definiteness and positive semidefiniteness (PSD), respectively. In. D. and On are D D identity, all-ones, and all-zeros matrices. A;(A) stands for || = Eis |ayll. ||
the i-th singular values of A. Matrix norms are defined as ||
max; (A(A),
and ||A|| || == (Eij |asy 12)1/2 We introduce some preliminaries on manifold optimization [1, 6, 8].
Given PSD matrix B DxD, the generalized Stiefel manifold is defined as
St(D, R. B) {Z E R D
BZ IR]
The tangent space of the manifold M = St R) at Z E M is given by
TzM = (W E Dx Z BW + W - 0
(2)
The tangent bundle of a smooth manifold M, which consists of Tz-M at all Z E M, is defined as
TM {ZZ W) M, W Tz.).
Definition A retraction on differentiable manifold M is a smooth mapping from its tangent bundle T TM to M that satisfie the following conditions, with Te being the retraction of R to T
1. R²(()) (0) for all Z E M, where 0 denotes the zero element of M.
3

---

1Z+ IR*(E) (E)
Ez+Eill 1|Ell
For any Z E M, it holds that lim M2
O.
In the numerical experiments, this work employs a generalized polar decomposition based retraction. Given a PSD matrix E RDDD, for any 5 E Tz.M with M = (D, R. B). it is defined as:
(²(()) - U(QA -
where ÜΣVT E is the singular value decomposition of &, and Q, A obtained from the eigen- value decomposition QAQ" =U BÜ. Further details on retraction choices are in Appendix A.I.
3.2 Correlation Disparity Error
As previously mentioned, applying CCA to the entire dataset could lead to biased result, as some groups might dominate the analysis while others are overlooked. To avoid this, we can perform CCA separately on each group and compare the results. Indeed, we can compare the performance of CCA on each group's 's data with the performance of CCA the whole dataset, which includes all groups' data. The goal is to find a balance between the benefits and sacrifices of different groups so that each group's contribution to the CCA analysis is treated fairly. In particular, suppose the datasets X E R NxD= and Y E R Nx on the same set of observations, belong to K different groups {(X*, } with X R.NxD and Y R.NexD, based on demographics some other semantically meaningful clustering. These groups need not be mutually exclusive; each group can be defined as a different weighting of the data.
k==
To determine how each group is affected by F-CCA, we can compare the structure learned from each group's data (X* Y²) with the structure learned from all groups' data combined (X, Y). A fair CCA approach seeks to balance the benefits and drawbacks of each group's 's contribution to the analysis. Specifically, if we train global subspaces U E D. and V E IR P. on k-tth group dataset (X*, Y²), we can identify the group-specific (local) weights represented by (U V*) ) that has the best performance weights (U, V) on all data points while ensuring that each group's 's correlation on the CCA learned by the whole dataset is equivalent to the group-specifhe subspaces learned only by its own data.
that dataset. Thus, F-CCA algorithm should be able learn global
To define these fairness criteria, we introduce correlation disparity error as follows:
Definition 2 (Correlation Disparity Error). Consider a pair of datasets (X, Y) with K sensitive groups with data matrix {(X*,
K ) representing each sensitive group's 's data samples. Then, for k=1
any (U, V), the correlation disparity error for each sensitive group E [K] is defined as
8h "U, == -trace (Uk+* x yhykit) -trace (UTx*
(U))
1SKSK. (5)
Here, (Uk.,, Vk.*) is the maximizer of the
the following group-specific CCA problem:
maximize trace (u* (U*"** x v"v) subj.
U X X
v Y Ykvk IR. (6)
This measure shows how much correlation we are suffering for any global (U, V). with respect to the loss of optimal local (UR+*, V..*) that we can learn based on data points (X*, Y
Using Definition 2, we can define F-CCA as follows:
Definition 3 (Fair CCA). A CCA pair (U*, V*) is called fair the correlation disparity error among K different groups is equal, i.e..
Ek (U*,V*) E (U*,
vk s, k, s E [K].
(7)
A CCA pair (U*, V*) that achieves the same disparity error for all groups is called a fair CCA.
Next, we introduce the concept of pairwise correlation disparity error for CCA, which measures the variation in correlation disparity among different groups
Definition 4 (Pairwise Correlation Disparity Error). The pairwise correlation disparity error for
any global (U, V) and group-specific subspaces ((Uh.,, *k is defined as
k V) = (8* (U, V) 8d (U, V)).
Vk + s, k, [K].
(8)
Here, Φ o: + R4 is function such (xx) exp(x), 6(z) x² or 6(z) = |z].

---

Algorithm 1: A Multi-Objective Gradient Algorithm 2:
A Single-Objective Gradient Method for F-CCA (SF-CCA)
Method for F-CCA (MF-CCA)
1: Input: (X, Y) (Un, Vo), (R
Input: (X, Y), (Vo, Vo), (R, E N N;
and stepsizes {(n mT²) Jiii
A E R4++ and stepsizes {(Th' mi ))==i
2: Find the R-rank subspaces for each group,
2: Find the R-rank subspaces for each group,
{(Uk, )k using (6).
)
{(Uk. V**)) using (6).
3:
for 00 ...., 11-1dd do
3: for 0, 0,.... T do
4: Find (PI PY) by solving (10).
4: Find (G," GT) ) by solving (12).
5: U R" (n)
U f- RD (n)
6:
V4+1* + Ry (1) P))
6: V4+1 + R (m) G))
7:
end for
7: end for
8:
Output: (Ur, Vr). ((UR,, V 1E
8: Output: (Ur, Vr), ((UR,, V
The motivation for incorporating pairwise correlation disparity error in our approach can be attributed to the work by [40, 55] in the context of PCA. To facilitate convergence analysis, we will primarily consider smooth penalization functions, such as squared or exponential penalties.
3.3 Multi-Objective Framework for Fair CCA
In this section, we introduce an optimization framework for balancing correlation and disparity errors. Let fi (U, V) == trace (U" X YV) f2 (U, V) := 1.2 (U. ....... far (U, V) A KK (U, V). The optimization problem finding an optimal Pareto point of F is denoted by
minimize U.V
F(U, V) == [fi (U, V) ,2 (U, v).,.,, V)], UEU, VEV,
subj.
where U = -U U XU- = IR] and V VV= P YVV
point (U, E U x V satisfying Im( (U,, (-BM)) = 0 is called critical Pureto. Here, Im denotes the image of Jacobian of F. An optimum Pureto point of F is point (U*, V*) E x V such that there exists no other VU E U x with F(U, V) < F(U*, V*). Moreover, a point U V U X V is weak optimal Pareto of F if there is no (U.V) E u V with F(U, V) = F(U", V*). The multi-objective framework (9) addresses the challenge of handling conflicting objectives and achieving optimal trade-offs between them.
To effectively solve Problem (9), we propose utilizing a gradient descent method on the manifold U x V that ensures convergence to a Pareto stationary point. The proposed gradient descent algorithm for solving (9) is provided in Algorithm 1. For each (U, V) l x V, let P == (P", P*) with pu E Tull and P E Vv The iterates (P", PY) in Step 4 are obtained by solving the following subproblem the joint tangent plane Tull x TvV:
(P),, where (p(P) = max trace (P PV V.))) - Pl (10) PETUMITTV
min
[E[M]
If (U,, V.) E x V not a Pareto stationary point, Problem (10) has a unique nonzero solution Pa (see Lemma 7), known as the steepest descent direction for F at (U,, V.). In Steps and 6, Ru and R denote the retractions onto the tangent spaces Tull and TvV. respectively; refer to Definition 1. Assumption A. For a given subset S of the tangent bundle Tu TV, there exists a constant LE such that, for all (Z, P) E S, we have (())(P)) F(Z) + V (LF/2) |P 1 M, where
V = ( P). V = V / E R and R" is the retraction.
The above assumption extends [8, A 4.3] to multi- exponential map (exponential retraction) if the gradient of F is L -Liischhhttz continuous [23, 6].
optimization, and it always holds for the
Theorem 5. Suppose Assumption A holds. Let (U,, V.) be the sequence generated by MF-CCA.
Let f := inf (U. V): (U, V) U V}. for all i E [M] and define ft. (Uo, Vo) fī,
min (U, Vo) - ft i [M]]. If n = nt 17 < LL for all E {0, T 1}. then
fa. (Uo, Vo) ft.
min | t = 0, O,...., T 1 <
T
5

---

Proof Sketch. We employ Lemma 7 to establish the unique solution P, for subproblem (10). Lemmas 9 and 10 provide estimates for the decrease of function F along P,: For any 1) 2 0, we have F(U+,,, V(+1) < F(U,, Vt) (n. - L /2) PPPle Pll 1m. Summing this incquality over 0,1, T - 1 and applying our step size condition yields the desired result.
Theorem 5 provides a generalization of [8, Corollary 4.9] to the multi-objective optimization, showing that the norm of Pareto descent directions converges to zero. Consequently, the solutions produced by the algorithm converge to a stationary fair subspace. It is worth mentioning that multi-objective optimization in [23, 6] relies on the Riemannian exponential map, whereas the above theorem covers broader (and practical) retraction maps.
3.4 A Single-Objective Framework for Fair CCA
In this section, we introduce straightforward and effective single-objective framework. This ap- proach simplifies F-CCA optimization, lowers computational requirements. and allows for fine-tuning fairnes--ccurracy trade-offss using the hyperparameter x. Specifically, by employing a regularization parameter À > our proposed fairness model for F-CCA is expressed as follows:
minimize U.V subj. to
1(U, V) trace (UTXTVV) + (U,
(11)
Ueu, VeV,
where A - Ad (U, V); see Definiton 4.
The choice of À in the model determines the emphasis placed on different objectives, When is large, the model prioritizes fairness over minimizing subgroup errors. Conversely, À is small, the focus shifts towards minimizing subgroup correlation errors rather than achieving perfect fairness. In other words, it is possible to obtain perfectly F-CCA subspaces; however, this may come at the expense of larger errors within the subgroups. The constant À in the model allows for flexible trade-off between fairness and minimizing subgroup correlation errors, enabling us to find a balance based on the specific requirements and priorities of the problem at hand.
The proposed gradient descent algorithm for solving (11) is provided as Algorithm 2. For each (U, V) E u x V, let G == (Gu, GY) with G" E Tull and GY E V. The iterates (G), Gý) are obtained by solving the following problem the joint tangent plane Tull TvV:
min GETul(X
ge (G(G), where qe (G) trace (GTVf((U, VO)) + G
(12)
The solutions (GH,, GY) are maintained on the manifolds using the retraction operations R" and RY.
Assumption B. For subset S C Tu x TV, there exists constant Ly such that for all (Z, G) E S. f(R (G)) < F(Z)+ (Vf(Z), G) + (2) |GG]|, with R as the retraction.
Theorem 6. Suppose Assumption holds. Let (U,, V.) be the sequence generated by SF-CCA Let /" == (VU.
Eu V}. if V. = S 1/L, for all E (T), then
min [IGH t= o....... T 1 < 2 f(U. Do Vo)
T
Comparison between MF-CCA and SF-CCA: MF-CCA addresses conflicting objectives and
achieves optimal trade-off's automatically. but it necessitates the inclusion of (K) additional objectives. SF-CCA, on the other hand, provides a simpler approach but requires tuning an extra hyperparameter
A. When choosing between the two methods, it is crucial to consider the trade-off between complexity and simplicity, well the number of objectives and the need for hyperparameter tuning.
4 Experiments
In this section, we provide empirical results showcasing the efficacy of the proposed algorithms.

---

4.1 Evaluation Criteria and Selection of Tuning Parameter
F-CCA's performance evaluated on correlation and fairness for each dimension of subspaces. Let U [u1. ugl E IR Da x and V = [Vi,'. ,Rl E IR P,XR, The r-th canonical correlation is defined as follows:
Yv,
Pr
u Yu r= R.
(13a)
Next, terms of fairness, we establish the following two key measures:
A max,r
max
1 (u., V,) 8d (u,, vr)l, r 1,..., R,
(13b)
[JE(K]
sum,
Σ [JE(K]
E (u,,
1, R.
(13c)
Here, Amax.r measures maximum disparity error, while Asu.. represents aggregate disparity error. The aim is to reach A Amax, and Asumr of 0 without sacrificing correlation (p,)) compared to CCA. We conduct a detailed analysis using component-wis measurements (13) instead of matrix versions; for more discussions, see Appendix C.2.
The canoncorr function from MATLAB and [35] is used to solve (CCA). For MF-CCA and SF- CCA, the learning rate is searched on a grid in {le - 1, 2, Ie 1..2.... . le 5), and for SF-CCA, is searched grid {le 2, le- 0.5, 2,2, 2, ..., 10]. Sensitivity analysis of À is provided in Appendix B.2. The learning rate decreases with the square root of the iteration number. Termination of algorithms occurs when the descent direction norm below le 4.
4.2 Dataset
4.2.1 Synthetic Data
Following [4, 44], our synthetic data are generated using the Gaussian distribution
X Y
N
μx
Ex Σyx
Σxy
Σy xx)
Here, px E RDxx1 and JyY Ryyx1 are the means of data matrices and Y, respectively; covariance matrices Σx, Ey and the cross-covariance matrix Σxy are constructed as follows. Given ground truth projection matrices E R V D and canonical correlations (p1, ......, PR) defined in (13a). Let U = QxRx and V QyRy be the QR decomposition of and V, then we have
Σxy Σx U diag(p) V ExE,,
(14a)
Σx QxRx Rx -Qx T + T Tx QxQx T)xx T
(14b)
Ey = Qy -TR -0x T + TY - Qr T)TY" T.
(14c)
Here, Tx E R2 DD= and Ty E D2x Dy are randomly generated by normal distributions, and Ta and Ty 0.001 are scaling hyperparameters. For subgroup distinction, we added noise to canonical vectors and adjusted sample sizes: 300, 350, 400, 450, and 500 observations each. In the numerical experiment, different canonical correlations are assigned to each subgroup alongside two global canonical vectors U and V to generate five distinct subgroups.
4.2.2 Real Data
National Health and Nutrition Examination Survey (NHANES). We utilized the 2005-2006 subset of the NHANES database https://wwit edc wwc.gov/.oo.oooooooooooooooo www.goo//cccie gov/nchs///////oooceeeeppc/eec nhanes, including physical mea- surements and self-reported questionnaires from participants. We partitioned the data into two distinct subsets: one with 96 phenotypic measures and the other with 55 environmental measures. Our objective was to apply F-CCA to explore the interplay between phenotypic and environmental factors in contributing to health outcomes, considering the impact of education. Thus, we segmented the dataset into three subgroups based on educational attainment (i.e., lower than high school, high school, higher than high school), with 2,495, 2,203, and 4,145 observations in each subgroup.

---

Table I: Numerical results in terms of Correlation (p-), Maximum Disparity (Amax.r). and Aggregate Disparity (Aum,r) metrics. Best values are in bold, and second- are underlined. We focus on the initial five projection dimensions, but present only two dimensions here; results for other dimensions are in the supplementary material. We put the results of other projection dimensions in the supplementary material. "1" means the larger the better and "4" means the smaller the better. Note that MHAAPS has only 3 features, we report results for its 1 and 2 dimensions.
Pr t
Amax. +
Asm,r
Dataset
Dim. (r)
CCA
MF-CCA SF-CCA|
CCA
MF-CCA SF.CCA
CCA
MF-CCA SF-CCA
Synthetic
0.7533
0.7475
0.7309
00.5555
0.2866
0.2241
3,3802
2.8119
2.2722
Data
0.4717
0.4681
0,4581
0.4385
0.3313
0.2424
4,1649
3.1628
2.2304
0.6392
0.6360
0.6334
0.0485
0.0359
0.0245
0.1941
0.1435
0.0980
NHANES
5
0,4416
0.4393
0,4392
0.1001
0.0818
0.0824
0,4003
0.3272
0.3297
0.4464
0.4451
0.4455
0.00033
0,0076
0.0044
0.0187
0.0152
0,0088
MHAAPS
2
0.1534
0.1529
0,1526
0.0061
0.0038
0.0019
0.0122
0.0075
0.0039
2
10.7778
0.7776
0.7753
0.0131
0.0119
0.0064
0.0263
0.0238
0.0127
ADNI
5
0.6810
0.6798
0.6770
0.0477
0.0399
0.0324
0.0954
0.0799
0.0648
Table 2: Mean computation time in seconds (+:ttd) of 10 repeated experiments for R = 5 on the real dataset and R- 7 on the synthetic dataset. Experiments are run on Intel(R) (e) CPU E5-2660.
Dataset
CCA
MF-CCA
SF-CCA
Synthetic Data
0.0239+00.0266
109. 109.0693-55.5418
29, 1387=+200828
NHANES
0.0483110.0059
1..445
14.9156.11.8411
MHAAPS 0.0021+:00047
3.5235::2.0955
0.8238=0.8155
ADNI 0.0039110.0332
2.7297:10.5136
1.8489=1.0519
Mental Health and Academic Performance Survey (MHAAPS). This dataset is available at tetrtttttttttteeeeeeeeeeeee
data. It consists of three psychological variables, four academic variables, as well as sex information for a cohort of 600 college freshmen (327 females and 273 males). The primary objective of this investigation revolves around examining the interrelationship between the psychological variables and academic indicators, with careful consideration given to the potential influence exerted by sex.
https://8s thub. cww/marks/
Alzheimer's Disease Neuroimaging Initiative (ADNI). We utilized AV45 (amyloid) and AV AV1451 (tau) positron emission tomography (PET) data from the ADNI database (http:/ / //w:ni.//ii.i/// 1oni usc. edu) [73, 74]. ADNI data are analyzed for fairness in medical imaging classification [41, 53, 81], and sex disparities in ADNI's CCA study can harm generalizability, validity, and intervention tailoring. We utilized F-CCA to account for sex differences. Our experiment links 52 AV45 and 52 A41451 features in 496 subjects (255 females, 241 males).
4.3 Results and Discussion
In the simulation experiment, we follow the methodology described in Section 4.2.1 to generate two sets of variables, each containing two subgroups of equal size. Canonical weights are trained and used to project the two sets of variables into a 2-dimensional space using CCA, SF-CCA and MF-CCA. From Figure 2, it is clear that the angle between the distributions of the two subgroups, as projected by SF-CCA and MF-CCA, is smaller in comparison. This result indicates that F-CCA has the ability to reduce the disparity between distinct subgroups.
Table 1 shows the quantitative performance of the three models: CCA, MF-CCA, and SF-CCA. They are evaluated based on Pr. Amax.r. and Asum,r defined in (13) across five experimental sets. Table 2 displays the mean runtime of each model. Several key observations emerge from the analysis. Firstly, MF-CCA and SF-CCA demonstrate substantial improvements in fairnsss compared to CCA. However, it is important to note that F-CCA, employed in both MF-CCA and SF-CCA compromises some degree of correlation due to its focus on faimess considerations during computations. Secondly,
8

---

Group
Group
(a) XU of CCA
(b) XU of MF-CCA
(c) XU SF-CCA
Group Group
Group Group
Group Group? 2
(d) YV of CCA
(e) YV of MF-CCA
(1) YV of SF-CCA
Figure 2: Scatter plot of the synthetic data points after projected to the 2-dimensional space. The distributions of the two groups after projection by CCA are orthogonal to each other. Our SF-CCA and MF-CCA can make the distributions of the two groups close each other.
4.0
3.5
NF-CCA
MF CGA
3.0
SP-CGA
5D
2.5
2.0
1.3
3.0
1.0
4.3
0.0
(a) Synthetic Data
(b) NHANES
2.00
1.75
1.25
1.00
6.50
5.29
(c) MHAAPS
(d) ADNI
Figure 3: Aggregate disparity of CCA, MF-CCA, and SF-CCA (results from Table 1).
SF-CCA outperforms MF-CCA in terms of fairness improvement, although it sacrifices correlation. This highlights the effectiveness of the single-objection optimization approach in SF-CCA. Moreover, the datasets consist of varying subgroup quantities (5, 3, 2, and 2) and an imbalanced number of samples distinct subgroups. F-CCA consistently performs well across these datasets, confirming its inherent scalability. Lastly, although SF-CCA requires more effort to tune hyperparameters, SF-CCA still exhibits a notable advantage in terms of time complexity compared to MF-CCA, demonstrating computational efficiency. Disparities among various CCA methods are visually represented in Figure 3. Notably, the conventional CCA consistently demonstrates the highest disparity error. Conversely, SF-CCA and MF-CCA consistently outperform CCA across all datasets, underscoring their efficacy in promoting faimess within analytical frameworks.
In Table 1, we define the percentage change of correlation (pr), maximum dispar- ity gap (Amax,), and aggregate disparity (Aum,r), respectively, as follows: Ppr Pr of Cp, of CCA) x 100, PA 100, and PA = - -asum, of F-CCA
(pr of F-CCA
-(Amax, of F-CCA
m of CCA)/( (A (Amax,r of CCA)
sum.r of CCA)/ (CA)/(aum,r of CCA) x 100. Here, F-CCA is replaced with either MF-CCA or

---

A ue
55
CEN (MF-CX))
Playy HMl CCA
Maw. CCA)
Plane CCM)
29
10
10
(a) Synthetic Data
(b) NHANES
F-CA
"NCoage hom CZAIO F-EAA
Plove ooA
50
80
Mhou CO.)
(30
Plan
20
2d
10
(c) MHAAPS
(d) ADNI
Figure 4: Percentage change from CCA F-CCA (results from Table 1). Each dataset panel shows two cases with projection dimensions (r). Pp, is slight, while PAmax,r and P PAsum, changes are substantial, signifying fairness improvement without significant accuracy sacrifice.
SF-CCA to obtain the percentage change for MF-CCA or SF-CCA. Figure illustrates the percentage changes of each dataset. Pp, is slight, while PAmax,r and PAsumr changes substantial, signifying fairness improvement without significant accuracy sacrifice.
Conclusion, Limitations, and Future Directions
We propose F-CCA, novel framework mitigate unfairness in CCA. F-CCA aims to rectify the bias of CCA by learning global projection matrices from the entire dataset, concurrently guaranteeing that these matrices generate correlation levels akin group- specific projection matrices. Experi- ments show that F-CCA is effective in reducing correlation disparity error without sacrificing much correlation. We discuss potential extensions and future problems stemming from work.
While F-CCA effectively reduces unfairness while maintaining CCA model accuracy, its potential to achieve minimum achievable disparity correlation remains unexplored. A theoretical exploration this aspect could provide valuable insights.
F-CCA holds promise for extensions diverse domains, including multiple modalities [80], deep CCA [3], tensor CCA [44], and sparse CCA [25]. However, these extensions necessitate novel formulations and in-depth analysis.
Our approach of mujti-objecctie optimization smooth manifolds may find relevance in other problems, such as fair PCA [55]. Further, bilevel optimization approaches [37, 68, 65] can be designed on a smooth manifold to learn a single Pareto-efficient solution and provide automatic trade-off between accuracy and fairness.
With applications encompassing clustering, classification, and manifold learning, F-CCA ensures fairness when employing CCA techniques for these downstream tasks. can also be jointly analyzed with fair clustering [15, 66, 34] and fair classification [78, 18).
6
Acknowledgements
This work was supported in part by the NIH grants U01 AG066833, UO1 AG068057, RFI AG063481, R01 LM013463. P30 AG073105, and UO1 CA274576, and the NSF grant IIS 1837964. The ADNI data were obtained from the Alzheimer's Disease Neuroimaging Initiative database (https: usc. edu), funded by NIH U01 AG024904. Moreover, the NHANES data were sourced from the NHANES database (https: :/wpw cdc wwwp:wddc.goo/wne.. saad..go//nchsshaaaes)))
//adni.
loni. loni.
We appreciate the reviewers' valuable feedback, which significantly improved this paper.
10

---

References
[1] P-A Absil. Optimization algorithms on matrix manifolds. Princetonn University Press, 2008.
[2] Fares Al-Shargie, Tong Boon Tang, and Masashi Kiguchi. Assessment of mental stress effects prefrontal cortical activities using canonical correlation analysis: an fnirs-eeg study. Biomedical optics express, 865):2583-5598, 2017.
[3] Galen Andrew, Raman Arora, Jeff Bilmes, and Karen Livescu. Deep canonical correlation analysis. In International Conference on Machine Learning, pages 1247-1255, 2013.
[4] Francis R. Bach and Michael I. Jordan. probabilistic interpretation of canonical correlation analysis. 2005.
[5] Solon Barocas, Moritz Hardt, and Arvind Narayanan. Fairness and machine learning: Limita- tions and opportunities. MIT Press, 2023.
[6] GC Bento, Orizon Pereira Ferreira, and P Roberto Oliveira. Unconstrained steepest descent method for multicriteria optimization on riemannian manifolds. Journal of Optimization Theory and Applications, 154:88 154:8--107, 2012.
[7] Glaydston C Bento, Orizon P Ferreira, and Jefferson G Melo. Iteration-complextty of gradient, subgradient and proximal point methods on riemannian manifolds. Journal of Optimization Theory and Applications, 173(2):548-562 2017.
[8] Nicolas Boumal. An introduction to optimization on smooth manifolds. Cambridge University Press, 2023.
[9] Nicolas Boumal, Pierre-Antoine Absil, and Coralia Cartis. Global rates of convergence for nonconvex optimization on manifolds. IMA Journal of Numerical Analysis,
2019.
[10] Cao, Z Ju, J Li, R Jian, and Jiang. Sequence detection analysis based on canonical correlation for steady-state visual evoked potential brain computer interfaces. Journal of neuroscience methods, 253:10-17, 2015.
[11] Simon Caton and Christian Haas. Fairness in machine learning: A survey. arXiv preprint arXiv:2010.04053. 2020.
[12] L Elisa Celis, Damian Straszak, and Nisheeth K Vishnoi. Ranking with fairesss constraints. arXiv preprint arXiv: 704.06840, 2017.
[13] Shixiang Chen, Shiqian Ma, Lingzhou Xue, and Hui Zou. An alternating manifold proximal gradient method for sparse principal component analysis and sparse canonical correlation analysis. INFORMS Journal on Optimization, 2(3): 208, 2020.
[14] Zhiwen Chen, Steven X Ding, Tao Peng, Chunhua Yang, and Weihua Gui. Fault detection for non-gaussian processes using generalized canonical correlation analysis and randomized algorithms. IEEE Transactions on Industrial Electronics, (5(2):1559911
2017. [15] Flavio Chierichetti, Ravi Kumar, Silvio Lattanzi, and Sergei Vassilvitskii. Fair clustering through fairlets. Advances neural information processing systems, 30, 2017.
[16] Alexandra Chouldechova and Aaron Roth. The frontiers of fairness in machine learning. arXiv preprint arXiv: 1810.088/0, 2018
[17] Vien Ngoc Dang, Anna Cascarano, Rosa H. Mulder, Charlotte Cecil, Maria A. Zuluaga, Jerónimo Hemández-Gonnzine
and Karim Lekadir. Fairness and bias correction in machine learning for depression prediction: results from four different study populations, 2023.
[18] Michele Donini, Luca Oneto, Shai Dev-Da John Shawe- Taylor, and Massimiliano Pontil. Empirical risk minimization under faimess constraints. arXiv preprint arXiv: (80iv::8002.0000 1802.08626, 2018.
[19] Ralph B Dunham and David J Kravetz. Canonical correlation analysis in a predictive system. The Journal of Experimental Education, 43(4):35-42, 1975.
[20] Cynthia Dwork. Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference, pages 214-226, 2012.
[21] Alan Edelman, Tomás A Arias, and Steven T Smith. The geometry of algorithms with or- thogonality constraints. SIAM journal on Matrix Analysis and Applications, 20(2):003-333 1998.
11

---

[22] J Fang, D Lin, SC Schulz, Xu, VD Calhoun, and Y-P Wang. Joint sparse canonical correlation analysis for detecting differential imaging genetics modules. Bioinformatics, 32822)3348--3488 2016.
[23] Orizon P Ferreira, Mauricio S Louzeiro, and Leandro F Prudente. Iteration-compleiitt and asymptotic analysis of steepest descent method for multiobjective optimization on riemannian manifolds. Journal of Optimization Theory and Applications, 184:507-533, 2020.
[24] Gembicki and Y Haimes. Approach to performance and sensitivity multiobjective optimiza- tion: The goal attainment method. IEEE Transactions on Automatic control, 20(6):769-771, 1975.
[25] David Hardoon and John Taylor. Sparse canonical correlation analysis. Machine Learning, 83(3):331-353, June 2011.
[26] Heij and Roorda. A modified canonical correlation approach to approximate state space modelling. In Decision and Control, 1991., Proceedings of the 30th IEEE Conference on, pages 1343-1348, IEEE, 1991.
[27] CE Hopkins.
Statistical analysis by canonical correlation: a computer application. Health services research, 4(4):304, 1969.
[28]
Hotelling. Relations between sets of variates. Biometrika, 28(3/4):3213377 1936.
[29] Harold Hotelling. The most predictable criterion. Journal of educational Psychology, 26(2):139,
1935.
[30] Harold Hotelling. Relations between two sets of variates. Breakthroughs in statistics: method- ology and distribution, pages 162-190, 1992.
[31] Mariya Ishteva, P-A Absil, Sabine Van Huffel, and Lieven De Lathauwer. Best low multilinear rank approximation of higher-order tensors, based on the riemannian trust-region scheme. SIAM Journal on Matrix Analysis and Applications, 32(1):115-135, 2011.
[32] Mohammad Mahdi Kamani, Farzin Haddadpour, Rana Forsati, and Mehrdad Mahdavi. Efficient fair principal component analysis. Machine Learning, pages 1-32, 2022.
[33] Matthäus Kleindessner, Michele Donini, Chris Russell, and Muhammad Bilal Zafar. Efficient fair pea for fair representation learning. In International Conference on Artificial Intelligence and Statistics, pages 5250-5270. PMLR, 2023.
[34] Matthäus Kleindessner, Samira Samadi, Pranjal Awasthi, and Jamie Morgenstern. Guarantees for spectral clustering with fairness constraints. In International Conference on Machine Learning, pages 3458-3461 PMLR, 2019.
[35] Wojtek J Krzanowski. Principles of multivariate analysis. Oxford University Press, 2000.
[36] Keith R Laws, Karen Irvine, and Tim M Gale. Sex differences in cognitive impairment in alzheimer's 's disease. World Psychiatry, 6(1):6( 65, March 2016.
[37] Mingchen Li, Xuechen Zhang, Christos Thrampoulidis, Jiasi Chen, and Samet Oymak. Au- tobalance: Optimized loss functions for imbalanced data. Advances in Neural Information Processing Systems, 01:3166-31777 2021.
[38] Tianyi Lin, Chenyou Fan, Nhat Ho, Marco Cuturi, and Michael Jordan. Projection robust wasserstein distance and riemannian optimization. Advances in neural information processing systems, 33:9383-9397. 2020.
[39] H Lindsey, JT Webster, and S Halpern. Canonical correlation as discriminant tool in periodontal problem. Biometrical journal, 27(3)2557-26, 1985.
[40] Mohammad Mahdi Kamani, Farzin Haddadpour, Rana Forsati, and Mehrdad Mahdavi. Efficient fair principal component analysis. arXiv e-prints, pages arXiv-1911, 2019.
[41] Carolyn M Mazure and Joel Swendsen. Sex differences in alzheimer's disease and other dementias. The Lancel Neurology, 15(5):451-45, 2016.
[42] Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey on bias and faimess in machine learning. ACM Computing Surveys (CSUR), 54(6):1-35, 2021.
12

---

[43] Zihang Meng, Rudrasis Chakraborty, and Vikas Singh. An online riemannian pea for stochastic canonical correlation analysis. Advances in neural information processing systems, 34:14 14056- 14068, 2021.
[44] Eun Jeong Min, Eric C Chi, and Hua Zhou. Tensor canonical correlation analysis. Stat, 8(1)xe253, 2019.
[45] Mark S Mommonier and Fay Evanko Finn. Improving the interpretation of geographical canonical correlation models. The Professional Geographer, ((22):440142, 1973.
[46] Masaki Nakanishi, Yijun Wang, Yu-Te Wang, and Tzyy-Ping Jung. A comparison study of canonical correlation analysis based methods for detecting steady-state visual evoked potentials. PloS one, 10(10)==01 407003, 2015.
[47] Toru Ogura, Yasunori Fujikoshi, and Takakazu Sugiyama. A variable selection criterion for two sets of principal component scores in principal canonical correlation analysis. Communications in Statistics-Theory and Methods, 42(12):21 11::21188-1335 2013.
[48] Aileme Omogbai. Application of multiview techniques to NHANES dataset.
CoRR,
0bs//608.007788 2016.
[49] Luca Oneto and Silvia Chiappa. Fairness in machine learning. In Recent Trends in Learning From Data, pages 195-196. Springer, 2020.
[50] Jaakko Peltonen, Wen Xu, Timo Nummenmaa, and Jyrki Nummenmaa. Fair neighbor embed- ding. 2023.
[51] Harold Pimentel, Zhiyue Hu, and Haiyan Huang. Biclustering by sparse canonical correlation analysis. Quantitutive Biology, 6(1):56-67, 2018.
[52] Miao Qi, Owen Cahan, Morgan A Foreman, Daniel M Gruen, Amar K Das, and Kristin P Bennett. Quantifying representativeness in randomized clinical trials using machine learning fairness metrics. JAMIA Open, 4(3):0oab077, 09 2021.
[53] Mathew J Reeves, Cheryl D Bushnell, George Howard, Julia Warner Gargano, Pamela W Duncan, Gwen Lynch, Arya Khatiwoda, and Lynda Lisabeth. Sex differences in stroke: epidemiology, clinical presentation, medical care, and outcomes. The Lancet Neurology, 7(10):115 926, 2008
[54] Juho Rousu, David D Agranoff, Olugbemiro Sodeinde, John Shawe- Taylor, and Delmiro Fernandez-Reye Reyes. Biomarker discovery by sparse canonical correlation analysis of complex clinical phenotypes of tuberculosis and malaria. PLoS Comput Biol, 9(4):e100301 2013.
[55] Samira Samadi, Uthaipon Tantipongpipat, Jamie H Morgenstern, Mohit Singh, and Santosh Vempala. The price of fair pea: One extra dimension. Advances in neural information processing systems, 31, 2018.
[56] Biplab K Sarkar and Chiranjib Chakraborty, Dna pattern recognition using canonical correlation algorithm. Journal of biosciences, 40(4):709-719, 2015.
[57] Steven Schell and William A Gardner. Programmable canonical correlation analysis: A flexible framework for blind adaptive spatial filtering. IEEE Transactions on Signal Processing, 43(12))2898-299 29088, 1995.
[58] Jose A Seoane, Colin Campbell, Ian NM Day, Juan P Casas, and Tom Gaunt. Canonical correlation analysis for gene-based pleiotropy discovery. PLoS Comput Biol, :0(10):e103387, 2014.
[59] J. Sha, J. Bao, K. Liu, S. Yang, Z. Wen, J. Wen, Y. Cui, B. Tong. J. H. Moore, A. J. Saykin, C. Davatzikos, Q. Long, I.. Shen, and Alzheimer's Disease Neuroimaging Initiative. Pref- erence matrix guided sparse canonical correlation analysis for mining brain imaging genetic associations in alzheimer's 's disease. Methods, 218:27-38. 2023.
[60] Li Shen and Paul M. Thompson. Brain imaging genomics: Integrated analysis and machine learning. Proceedings of the IEEE, 108(1):125-162, 2020.
[61] Xiao-Bo Shen, Qi-Song Sun, and Yuan-Hai Yuan. Orthogonal canonical correlation analysis and its application in feature fusion. In Information Fusion (FUSION), 2013 16th International Conference on, pages 151-157. IEEE, 2013.
[62] Michael Sullivan. Distribution of edaphic diatoms in a missisippi salt marsh: A canonical correlation analysis. Journal of Phycology, 18(1):100-33, 1982.
13

---

[63] Mingkui Tan, Ivor W Tsang, Li Wang, Bart Vandereycken, and Sinno Jialin Pan. Riemannian pursuit for big matrix recovery. In International Conference on Machine Learning, pages 1539-1547. PMLR, 2014.
[64] Uthaipon Tantipongpipat, Samira Samadi, Mohit Singh, Jamie H Morgenstern, and Santosh S Vempala. Multi-criteria dimensionality reduction with applications to fairness. Advances in neural information processing systems, (32). 2019.
[65]
Davoud Ataee Tarzanagh and Laura Balzano. Online bilevel optimization: Regret analysis of online alternating gradient methods. arXiv preprint arXiv:2207.02222 2022.
[66] Davoud Ataee Tarzanagh, Laura Balzano, and Alfred O Hero. Fair structure learning in heterogeneous graphical models. arXiv preprint arXiv:21 /2.05/28, 2021.
[67] Davoud Ataee Tarzanagh, Bojian Hou, Boning Tong. Qi Long, and Li Shen. Fairness-aware class imbalanced learning on multiple subgroups. In Uncertainty in Artificial Intelligence, pages 2133.2133. PMLR, 2023.
[68] Davoud Atace Tarzanagh, Mingchen Li, Christos Thrampoulidis, and Samet Oymak. Fednest: Federated bilevel, minimax, and compositional optimization. In International Conference on Machine Learning, pages 21 1414-121120 179. PMLR, 2022.
[69] Xin M Tu, Donald S Burdick, Debra W Millican, and I. Bruce McGown. Canonical corre- lation technique for rank estimation of excitation-emission matrixes. Analytical chemistry, 61(19):2219-2224 1989.
[70] Viivi Uurtio, João M Monteiro, Jaz Kandola, John Shawe-Taylor, Delmiro Fernandez-Reyes, and Juho Rousu. tutorial on canonical correlation methods. ACM Computing Surveys (CSUR), 50(6):1-33, 2017.
[71] GC Wang, N Lin, and B Zhang. Dimension reduction in functional regression using mixed data canonical correlation analysis. Stat Interface, 6: 6:187-196, 2013.
[72] FV Waugh. Regressions between sets of variables. Econometrica. Journal of the Econometric Society, pages 290-310, 1942.
[73] M. W. Weiner, P. P. Veitch, P. S. Aisen, al. The alzheimer's disease neuroimaging initiative: a review of papers published since its inception. Alzheimers Dement, 9(5)e1 111)))11194,, 2013.
[74] Michael W Weiner, Dallas P Veitch, Paul S Aisen, et al. Recent publications from the Alzheimer's Disease Neuroimaging Initiative: Reviewing progress toward improved AD clinical trials. Alzheimer's & Dementia, 13(4)xel-e85, 2017.
[75] Zaiwen Wen and Wotao Yin. A feasible method for optimization with orthogonality constraints. Mathematical Programming, 142(1-2):397-433 2013.
[76] KW Wong, PCW Fung, and CC Lau. Study of the mathematical approximations made in the basis-correlation method and those made in the canonical canonical-transformn transformation method for an interacting bose gas. Physical Review A. 22(3):1272, 1980.
[77] Florian Yger, Maxime Berar, Gilles Gasso, and Alain Rakotomamoniy, Adaptive canonical correlation analysis based on matrix manifolds. arXiv preprint arXiv:1206. 64106,64,,, 2012.
[78] Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna P Gummadi. Fairness constraints: Mechanisms for fair classification. In Artificial intelligence and statistics, pages 962-970. PMLR, 2017.
[79] Yu Zhang, Guoxu Zhou, Jing Jin, Yangsong Zhang, Xingyu Wang, and Andrzej Cichocki. Sparse bayesian multiway canonical correlation analysis for eeg pattern recognition. Neurocomputing, 225:103-110, 2017.
[80] Xiaowei Zhuang, Zhengshi Yang, and Dietmar Cordes. A technical review of canonical correlation analysis for neuroscience applications. Human Brain Mapping, 41(13):3807-3833 2020.
[81] Yongshuo Zong, Yongxin Yang, and Timothy Hospedales. Medfair: Benchmarking fairness for medical imaging. arXiv preprini 10..0222 2022.
14

---

Roadmap, The appendix is organized as follows: In Section A, we present the subproblem solvers for MF-CCA and SF-CCA. along with the proofs for Theorems 5 and In Section B, we provide comprehensive insights into the three real datasets utilized and present supplementary experimental results. Furthermore, Section outlines the experimental specifics and the procedure for selecting hyperparameters.
Contents
Addendum to Section
16
A.I Preliminaries on Retractions
16
A.2
Subproblem Solver for MF-CCA
16
A.3 Subproblem Solver for SF-CCA
17
A.4 Auxiliary Lemmas
18
A.5 Proof of Theorem 5
18
A.6 Proof of Theorem
19
B Addendum to Section 4
20
B.1 Detailed Description of Datasets
20
B.1.1 National Health and Nutrition Examination Survey (NHANES)
20
B.1.2
Mental Health and Academic Performance Survey (MHAAPS)
21
B.1.3
Alzheimer's Disease Neuroimaging Initiative (ADNI)
21
B.2
Trade-off Analysis of Correlation and Disparity
22
B.2.1
Sensitivity of Correlation and Disparity Error to K
22
B.2.2
Sensitivity of Correlation and Disparity Error to Sensitivity of Correlation and Disparity of SF-CCA ..........
22
B.2.3
24
B.3
Runtime Sensitivity MF-CCA and SF-CCA
25
B.3.1
Runtime Sensitivity of MF-CCA and SF-CCA to d
25
B.3.2
Runtime Sensitivity of MF-CCA and SF-CCA to N
25
B.3.3
Runtime Sensitivity of MF-CCA and SF-CCA to
26
C Experimental Details and Hyperparameter Selection Procedure
26
C.I Hyperparameters Selection
27
C.2 Fairness and Correlation Measures
28
15

---

A Addendum to Section
A.1 Preliminaries on Retractions
In reference to Definition 1, the following options together with generalized polar decomposition defined in (4) represent commonly employed retractions[1, 6, 7, 21, 23, 38, 75] of a matrix & p
Exponential mapping. takes 8DR + O(Ie)) flops and has the closed-form expression:
(|R
- & R
)
In
Rexp (E) Z
Q exp
0
where QR -I, zZ ) is the unique QR factorization.
Polar decomposition. It takes 3DR O(R²) flops and has the closed-form expression:
polar (
(Z (In + + ()
QR decomposition. takes 2DR + O (R$) flops and has the closed-form expression:
Ry (E) = qr 7 E),
where qr(A) is the Q factor of the QR factorization of A.
Cayley transformation. It takes 7DR O(RS) flops and has the closed-form expression:
Re laylley (E) = I - w(()
IR + w(() z,
2
where W(E) = (IR zz /2) ZE (IR - ZZ /2).
This work specifically focuses on generalized polar decomposition defined in (4) for retraction.
A.2 Subproblem Solver for MF-CCA
To solve the optimization problem (9), we introduce method for finding the descent direction on the joint manifold, as described in Equation (10). In this section, we will provide a detailed explanation of the relevant calculations. We expand one for updating the canonical weight U and the other for updating the canonical weight V. Here, we focus on outlining the process of updating U, which is described below.
optimization problem into two sub-optimization problems,
In the t-th iteration, the gradient of the multi-objective function (9) with respect to U, is computed as follows:
VoJ(U, V.) = (U, V.) = -VV
aU,
VU,,UU, V,) 04 h. (U, V.)
(15a)
aU,
(OE*(U,, V.)
a V.)
o' (8 (U, V) 8 (U.
JU:
for M 2j 2 2. k, s [K], + s, where
k (U, Vi) trace (U. x² T ykvkett - yvVV.
SU,
(15b)
x 30 (U,, Vt) trace T Tx -x- -
Y Y
aU,
Similarly, we compute the gradient of f w.r.t Vt.
Lemma 7, inspired by [23, 6], states that the direction P, can be expressed as linear combination of gradients (U, Vt). (U, V.)...., VfM(U, V.)].
16

---

Lemma 7. The unconstrained optimization problem in (10) has unique solution. Moreover, Pt is the solution of the problem in (10) if only if there exist μ4 2 0, E [M], such that
P. - Σ VV
Σ KE[M]
p4 2 0, for 1 Sis M.
(16)
MEIM
Proof. Recall that for each (U,V) u we have P (P", PV) with P" E Tuu and Pt E V. It can be easily seen that the first term of Q(P) (defined in (10)) is summation of the maximum of linear functions in the linear space Tull VvV. Hence, it is convex, which implies that Q(P) is strongly convex. Thus, the solution is unique.
From the convexity of the function, it is well known that P, is the solution of (10) if only if E aQ(P)Pt.
or equivalently,
P. e max trace (PVAA(,, Vo)) P.-
[E|M]
Therefore, the second statement follows from the formula for the subdifferential of the maximum of convex functions.
The (sub)-gradient formula (15) can be substituted into Equation (10), and subsequently, the fminimax funetion from the MATLAB optimization toolbox be applied to solve for the descent direction Pt- We found that the Goal Attainment Method [24] gives a more stable and accurate solution for our subproblem Given PH. U, can be updated by:
U++1 = R (m.
(17)
where is the adaptive stepsize and Ru is the retraction operator onto u I xD..R X
{U
=
Various retraction options discussed in Sections 3.1 and A.I. Specifically, the generalized polar decomposition method defined in (4) is employed in our numerical experiments described in Section The symmetric nature of the update process for canonical weights U and V in CCA allows us to employ identical procedures for updating V. By substituting X with Y and U with V in the aforementioned steps, we obtain the process for updating V. In each iteration, U and V updated once in sequence.
A.3 Subproblem Solver for SF-CCA
To solve the optimization problem (11), the method of finding the descent direction is introduced in Equation (12). Detailed calculations will be explained below. We expand it into two sub-optimization problems to update the canonical weight U and the canonical weight V, respectively. First, to update the canonical weight U, iteration 4, we can take gradient of f(U,, V.) with respect U, as follows:
(U, V.) = af(Un, V.) OU, - AA, V.) aU,
(18a)
--
o' ( (DeM(U,)) V.)
(E,(U, V.)
+1 Σ
8
aU,
aU,
where
08* (E,, Vt) aU; O Vt) trace (U+++x** aU,
= trace U T x )+ x* y
(18b)
x= x=V)
Xx* Y"V.. T
html
17

---

Given the subgradients (18) and the single-objective subproblem (12), the direction G. (G) G can be effectively computed using subgradient-bart based methods such as the fmincon function from the MATLAB optimization toolbox. Given G the update for U, can be obtained as follows:
U441 R (n)
(19)
Here, Th the adaptive stepsize and Ru is the retraction operator onto u X
{U
R PU. x = In}.
In our numerical experiments, we utilize polar decomposition as the retraction operator; see (4).
Similarly, the canonical weight V can be updated using the same procedure.
A.4 Auxiliary Lemmas
Lemma 8. Suppose Assumption A holds. For any (U,, Vi) E eu x let P, be the solution of (10).
Then, for any 2 O, we have the following:
LE
F(U++1, Vet1) F(U,, Vi) + V. + Th
PPll IM.
P TvV,
where Vu == (V,, V.). and V = Vi.' Mi T ERM.
Proof. The proof is straightforward consequence of Assumption A. Lemma's 9, For any (U,, Vi) E u x v, let Pt be the solution of (10). Then,
max trace (P. VU, V.)) = - =PI M
(20)
Hence, trace (P, V,, V.) 1 Pill F Further Pt is critical Pareto point of F if, and only if. |Pell 00
Proof. It follows from (10) that
P trace (P. Σ AVA.U. V.)
Σ ps trace P V.V...((.
(E[M]
RE[M]
Hence, by the second equality in (16), easy to verify that (20) holds. The second statement follows by using the definitions of trace (P." (FU,U, V.)). We proceed with the proof of the third statement of the lemma. Assuming that P, is a critical Pareto, follows from the definition that there exists i [M] such that trace(P, V fU,, V.) 0. Then, by the first part of the lemma, have ||P||| = 0. The converse (only if) follows from the application of [6, Lemma 4.2).
The following result provides estimate for the decrease of function F, along P. This result is crucial in determining the iteration-oomplxxtt bounds for the gradient method on a general Riemannian manifold.
Lemma 10 (Descent Property of MF-CCA), Suppose Assumption A holds. For any (U,, V.) EUXv, let P, the solution of (10). Then, for any ni 2 O, we have
F(U,+1, V.+1) = F(U, V.) ( LFTI
|Prll 1M
nt
2
Proof.
The proof is straight combination of Lemmas 8 and 9.
A.5 Proof of Theorem 5
Proof. It follows from Lemma 10 that
Len
|P 1M 3 F(U,, V.) -F(U,+, V(1),
1/
2
for all 0, o,.., T-1. 1.
18

---

Setting and summing both sides of this inequality for = 0, 0.1,.... 1, we get
Lon
T-1
Σ
Pulp
IM 3 F(U,, Vo) - FFU
2
1=0
Thus, by the definition of i,, we obtain
(Lon -1) min (IPrIE: 1 0,,,..,, S (U Vo) -7.
which together with 7 5 1 implies
2 fú. (Uo Vo) - E.
min ( t-
T
Remark 11. As consequence, if we consider tolerance level of e, we can bound the number of iterations required by the gradient method to obtain PT such that || S This bound can be expressed as O ((fs. (Uo, -o)- f(.)/e), where ft. (Uo, Vo) - ft. min [fo Vo) - f. E [M]). In other words, the number of iterations needed by the gra- dient method to achieve a solution (UT, VT) with [[P||p I c is proportional to the difference between the initial objective function value and the optimal objective function value, divided by the square of the lolerance level e. The "big O" notation, O, indicates that the bound is an upper bound, providing estimate of the worst-case behavior of the algorithm. This result showcases the convergence rate of the gradient method and provides useful guideline for choosing the tolerance level and estimating the computational resources required achieve the desired accuracy.
A.6 Proof of Theorem
Proof. This proof is slightly modified version of the proof of Theorem 5. Using Assumption B and oingle-objetive variant Lemma 10, we obtain
f (UL-1, V.1) (U,, V.)
$ (V, V)U,(( V++1) (U,, V()) T 2 [(U++1. V++1) - (U,, V.
Lym²
=-n (U,, V.II
[V, V.
2
=-n (1 T
VV (U, V.
Summing both sides of this inequality for = 0, 1, 1,....,T 1, we get
T.pmp
T-1
Σ
| & f(U,, Vo) - f(Ur, Vr).
2
Hence, using the fixed stepsize 1)- = S T. we get
S(Uo, Vo) f*
min fl t O........................ T 1)
T
Remark 12. a result, given a tolerance e, the number of iterations required by the gradient method to obtain (UT, VT) such that ||Gr||e S e bounded by O ((f(Uo, Vo) )
19

---

Table 3: Statistic summary of real datasets. the ADNI database, AV45 is used twice in two different datasets but with a different number of features. In AV45 vs. AV1451, we use the shared ROI (region of interest), thus, the two views have the same number of features. While in AV45 vs Cognitive Score, AV45 contains the total number of features.
Number of Number of Sensitive Attribute
Group Distribution
Datbase
Dataset
Features
Samples
Phenotypic Measures
96
<High School: 2495
NHANES
8843
Education
High School: 2203
Environmental Measures
55
>Hih School: 4145
Phenotypic Measures
96
White: 4576
NHANES
8136
Race
Black: 1793
Environmental Mcasures
55
Mexican: 1767
Psychological Performance Academic Performance
3
Male: 273
MHAAPS
Sex
4
Female: 327
AV45
52
Male: 241
ADNI
496
Sex
AV1451
52
Female: 255
AV4S
68
Male: 431
ADNI
785
Sex
Cognitive Score
46
Female: 354
B
Addendum to Section
B.1 Detailed Description of Datasets
Table 3 provides comprehensive overview of the relevant details associated with each real dataset. These details include the number of samples, the number of features, sensitive attributes, and group distribution. Notably, within the group distribution, it is important to acknowledge that in certain datasets, the total number of samples within different sensitive attributes may not be equivalent. This discrepancy arises due to the presence of missing information, as well as the insufficient sample sizes certain subgroups. Figure 5 visualizes the distribution of the groups in each dataset. We can see that the groups are imbalanced in synthetic data, NHANES (Education) and NHANES (Race), while balanced in MHAAPS (Sex), ADNI AV45/AV 1451 (Sex), and ADNI AV4S/Cognition (Sex). Imbalanced groups make the problem more complicated, which is good challenge to validate the effectiveness of our methods.
In the following, we provide comprehensive descriptions of all the datasets we used.
B.1.1 National Health and Nutrition Examination Survey (NHANES)
The NHANES dataset extensively used for CCA analysis [48] and fairness studies [17, 52). The dataset's diverse attributes give rise to fairness considerations, underscoring the importance of conducting equitable CCA experiments to tackle discrepancies in sample representation stemming from these inequalities.
In our study, we narrow our focus to the 2005-2006 subset of the NHANES database available at https://wwet cdc. www.gggo/////iinneeeee /aaas//hane.... This subset comprises physical measurements and self- reported questionnaires from participants. To address missing data concerns, we employ the Multiple Imputation by Chained Equations (MICE) Forest methodology. Afterward, two distinct subsets: the "Phenotypic-Meassie
divide the data into dataset and the "Environmental- Meeauree" " dataset, based on the inherent nature of the features. This stratified approach enables us to leverage F-CCA to gain a nuanced understanding of how phenotypic and environmental factors interact influencing health outcomes while also accounting for the potential influence of education and race.
In our numerical experiments, we utilize education and race as sensitive attributes to partition the two datasets. In the initial experiment, we split the dataset into three subgroups based on participants' educational attainment. These subgroups comprise 2495, 2203, and 4145 observations, respectively. For the subsequent experiment, work with total of 8 136 observations, dividing them into three
20

---

3000
200
203
Gropt
Gra.p1
Coug3
<18g
School
Group Name
Group Name
(a) Synthetic Data
(b) NHANES (Education)
Samples
5000
200
Mack
Mokea
(c) NHANES (Race)
(d) MHAAPS (Sex)
200
Samples
300
023
of
Group Name
Group Name
(c) ADNI AV45/AV1451 (Sex)
ADNI AV45/Cognition (Sex)
Figure 5: Group distributions the studied datasets.
subgroups based on racial categories. Specifically, there are 4576 white subjects, 1793 black subjects, and 1767 Mexican subjects. The Phenotypic-Measurg dataset encompasses 96 distinct features, while the "Envirommental-Mrre
dataset includes features.
B.1.2 Mental Health and Academic Performance Survey (MHAAPS)
The dataset employed this study obtained from the online repository available https: //github. thub. com/ com/marks/ m/aa//ra//////aa////aaaa/a/a httes/covetteeeeeeeeeeeeeeeeeeee wwwp/tseee/sstteeeeeeeeeeeeeeeee wttp///atter//ssaeeeeee paassttrrs/mpleee
This particular dataset includes three distinct psychological variables and four academic variables in the form standardized test scores, as well as gender information for cohort of 600 individuals classified college freshmen. The primary objective of this investigation revolves around examining the interrelationship between the aforementioned psychological variables and academic indicators, with careful consideration given to the potential influence exerted by gender. Specifically, the dataset consists of 327 female samples and 273 male samples.
B.1.3 Alzheimer's Disease Neuroimaging Initiative (ADNI)
We utilize AV45 (amyloid), AV1451 (tau) positron emission tomography (PET), and Cognitive Score data from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database (http://ade) .loni.. usc.ed. edu) three of our experiment datasets. Over- or inder-representation of age, sex, or biometric groups might affect ADNI data faimess. Disease distribution among sensitivity features, such as higher AD occurrence in women, can also impact faimess [41]. The unfairness in sex representation within the CCA study of ADNT dataset arises from an imbalance in number of male and female participants. This disparity undermines the generalizability and validity of the study findings, as it fails to account for potential sex-related differences Alzheimer's disease progression and response to treatments. The skewed representation limits the ability draw accurate conclusions
21

---

MF-CCA SF-CCA
Aggregate Disparity
8
Number of Subgroups (K)
Figure 6: Aggregate disparity of 1st projection dimension synthetic data comprising varying numbers of subgroups (K).
and develop tailored interventions for both sexes. perpetuating sex bias in research and healthcare. Consequently, becomes imperative to utilize F-CCA as means to acknowledge and integrate sex disparities within research initiatives.
In the conducted numerical study, distinct experiments carried out utilizing the ADNI dataset, where the sensitive attribute considered is the sex of the participants. The first experiment aims explore the correlation between AV45 and AV 1451, whereas the second experiment focuses on investigating the correlation between AV45 and Cognitive Scores. In the first experiment, both AV45 and AV145 1451 include 52 regional features and comprise total of 496 observations. Among these observations, 255 belong to the female subgroup, and 241 belong to the male subgroup. In the analysis involving the correlation between AV45 and Cognitive Scores, subset of 785 common samples is extracted, consisting of 431 male samples and 354 female samples. This analysis employs 68 regional features associated with AV45 and 46 cognitive score features.
B.2
Trade-off Analysis of Correlation and Disparity
B.2.1
Sensitivity of Correlation and Disparity Error to
We conducted an analysis to examine the sensitivity of fairness in relation to the number of subgroups K. To investigate this, we utilized synthetic data consisting of varying numbers of subgroups and compared the resulting aggregate disparity on the first projection dimension.
Figure provides visual representation of our findings. The results clearly demonstrate that the number of subgroups increases, there is substantial amplification in the aggregate disparity when employing CCA. However, both MF-CCA and SF-CCA effectively address this issue and mitigate the observed phenomenon. Specifically, they successfully reduce the level of disparity even as the number of subgroups grows.
Moreover, is worth noting that the aggregate discrepancy of SF-CCA exhibits a certain degree of sensitivity beyond threshold of six subgroups. Beyond this point, the effectiveness of SF-CCA in reducing disparity begins to diminish. Conversely, MF-CCA remains consistently unaffected by variations in the number of subgroups, consistently maintaining its fairness performance. This analysis underscores the robustness and scalability of MF-CCA in handling an increasing number of subgroups, it consistently maintains fairness irrespective of this parameter. On the other hand, SF-CCA demonstrates effective fairness mitigation but exhibits limitations when faced with large number of subgroups beyond certain threshold.
B.2.2 Sensitivity of Correlation and Disparity Error to
Table 4 presents the numerical results three measures described Section 4.1, namely correlation (pr), maximum disparity (Amax,r), and aggregate disparity (.) The results
with respect to
22

---

Canunical Correlation
0.9
Group
Canonical Correlation
Group
Canonical Correlation
Group
0.8
0.8
0.8
Group
Group
Group
0.7
Group
Group 4
Group
0.6
Group
0.8
Group
0.5
0.5
0.5
0.3
0.2
2
3
5
0.2
3
0.2
2
4
6
(a) Correlation of CCA
(b) Correlation of MF-CCA
(c) Correlation of SF-CCA
Figure 7: Visualization of the canonical correlation results synthetic data for the total five projection dimensions (r). All the methods are applied to both the entire dataset and individual subgroups. The closer each subgroup's 's curve is to the overall curve, the better.
0.80
0.80
0.80
Canonical Correlaiion
0.75
0.75
Correlation
0.75
0.70
0.70
0.70
Canonical Correl
0.85
0.50
Overall
Overall
Overall
High School
High Schoo
High School
High Schoo
High
High School
0.35
High Schoo
0.35
High School
(a) Correlation of CCA on NHANES (Education)
(b) Correlation of MF-CCA on NHANES (Education)
(c) Correlation of SF-CCA on NHANES (Education)
Canonical Correlation
White
Canonical Correlation
0.75
White
0.75
White
Black
0.70
Black
0.70
Black
0.85
Mexican
0.85
Moxican
0.60
0.50
0.60
0.55
0.55
0.55
Camonical
0.50
0.50
0.50
0.43
0.45
0.45
0.40
0.40
0.40
0.35
0.35
0.35
0.30
3
0.30
3
0.30
2
(d) Correlation of CCA
(e) Correlation of ME-CCA NHANES (Race)
(f) Correlation of SF-CCA on NHANES (Race)
NHANES (Race)
Figure 8: Visualization of the canonical correlation results NHANES (Education & Race) for the total five projection dimensions (r). All the methods are applied to both the entire dataset and individual subgroups. The closer each subgroup's curve is to the overall curve, the better.
the first r projection dimensions. an extension of Table 1. Specifically, we present the results of the first seven projection dimensions for synthetic data; the first five projection dimensions for NHANES and ADNI; and the first two projection dimensions for MHAAPS, respectively. Overall, we have consistent results and conclusions in Table 1. Firstly, MF-CCA and SF-CCA show substantial improvements in fairness compared to CCA with mild compromises of correlation. Secondly, SF- CCA outperforms MF-CCA in terms of faimess improvement, although it sacrifices correlation. This highlights the effectiveness the single-objective optimization approach in SF-CCA F-CCA consistently performs well across these datasets, confirming its inherent scalability.
23

---

Canonical Correlation
Mole
Corrciation
Mule
Canonical Corcelation
Mole
0.35
0.35
0.30
0.30
0.25
0.55
0.5
0.20
020
0.2
0.15
0.25
0.15
0.10
0.0
0.10
0.35
0.05
0.55
0.00
0.00
2
0.00
(a) Correlation of CCA
(b) Correlation of MF-CCA
(c) Correlation of SF-CCA
MHAAPS (Sex)
MHAAPS (Sex)
MHAAPS (Sex)
Figure 9: Visualization of the canonical correlation results of MHAAPS (Sex) for the total two projection dimensions (r). All the methods are applied both the entire dataset and individual subgroups. The closer each subgroup's s curve is to the overall curve, the better.
Table 5 quantifies the improvement in fairness and the minor compromises accuracy presented in Table 4 by depicting the percentage performance shift from baseline CCA MF-CCA and SF-CCA. The table is reference and an extension of Figure percentage shifts of the second and fifth projection dimension for Synthetic Data, NHANES, and ADNI and the first two projection dimensions for MHAAPS regarding the results presented in Table for coherence. In Table 5, the metries are formulated as: Pp. P.. F.CCA- pp. CCA x 100,
Section 4. Figure 4 only visualizes the
ofCCA
PAmax.r Amax,r O-C-CA- Amax, CCA x 100, PAsum.r
A of F-CCA- - of
x 100, where F-CCA is switched to MF-CCA and SF-CCA to obtain the corresponding shift results. The formulations are designed regarding the properties of the metrics where p, the larger the better while Amax.r and Asm,r are the smaller the better. According to the table, compared to CCA, MF. CCA, and SF-CCA sacrifice slightly the performance in correlation (p,) in exchange for significant fairness improvements in terms of maximum disparity (A (Amax.r) and aggregate disparity (Asum,r) across all datasets general, which demonstrate the good performance of F-CCA.
Amax. oCA
Asan.y of CCA
We also provide visualizations demonstrating how the correlation changes as the dimensionality changes. Figures 7, 9, and Figure 10 display these correlation curves. Our expectation is that the correlation curves of our methods will show stronger tendency to concentrate around the overall correlation curve compared the baseline method. The findings indicate that the utilization of both MF-CCA and SF-CCA approaches yields a convergence of subgroup performance towards the overall canonical correlation. Furthermore, the overall canonical correlations derived from these two models exhibit a comparable level of performance to that of the CCA model.
Upon examining the synthetic data in Figure 7, we observe clear concentration tendency. However, in the case of real data, this tendency is less pronounced. This is due to the fact that the disparities between different groups in the real data already quite small. As result, all the dashed curves in the CCA are already close to the solid red curve.
B.2.3 Sensitivity of Correlation and Disparity of SF-CCA to À
The SF-CCA experiment incorporates the hyperparameter À to strike balance between correlation and fairness considerations. This hyperparameter allows us to modulate the emphasis placed on fairness within the SF-CCA model. Figure 11 provides visual representation of the effects of different À values on both correlation and fairness metrics.
The illustration clearly demonstrates that increasing the magnitude of À enhances the emphasis on faireess within the SF-CCA model. Consequently, this leads to a reduction in aggregate disparity, indicating improved fairesss. Conversely, lower values of X prioritize correlation, emphasizing the preservation the underlying relationship between the datasets.
This experimental finding aligns with the discussion presented in Section of our work, where we discussed the trade-off between correlation and fairness. By adjusting the value of X, we can effectively control the balance between these two objectives in the SF-CCA model.

---

Canunical Correlation
Male
Canonical Correlation
Male
Canonical Correlation
0.85
Nule
Female
Farale
0.90
0.90
0.75
0.75
0.75
0.70
0.70
0.70
0.69
0.85
0.88
0.60
0.60
0.50
2
(a) Correlation of CCA
(b) Correlation of MF-CCA ADNI AV45/AV144 1451 (scx)
(c) Correlation of SF-CCA
ADNI AV45/AV145 (sex)
ADNI AV45/AV145 (sex)
0.80
0.90
0.90
Canonical Correlation
0.75
Ovarall
Canonical Comelation
0.75
Overall
Canonical Comelation
0.75
Ovarall
Female
Female
0.70
Female
0.65
0.55
0.65
0.60
0.00
0.50
0.50
0.50
0.50
0.45
0.45
0.45
(d) Correlation CCA
(e) Correlation of MF-CCA on ADNI AV4S/Cognition (Sex)
(f) Correlation SF-CCA on ADNI AV4S/Cognition (Sex)
on ADNI AV45/Cognition (Sex)
Figure 10: Visualization of the canonical correlation results of ADNI for the total five projection dimensions (r). All the methods are applied to both the entire dataset and individual subgroups. The closer each subgroup's curve the overall curve, the better.
B.3 Runtime Sensitivity MF-CCA and SF-CCA
When it comes to the issue of running time, it is important consider the trade-off between computational efficiency and hyperparameter tuning effort. While SF-CCA may require more effort in tuning its hyperparameters, it still demonstrates notable advantage in terms of time complexity compared to MF-CCA. To conduct a sensitivity analysis concerning the sample count (N), the feature count (d), and the subgroup count(K) ["), conduct series of experiments ensuring the hyperparameters remained constant, These experiments are performed in three distinct settings, each replicated times, employing techniques including CCA, MF-CCA, and SF-CCA. The specifics of outcomes are depicted in Figures 12, 13, and 14.
synthetic datasets,
B.3.1 Runtime Sensitivity of MF-CCA and SF-CCA to d
In the first experimental setup, with the sample size held constant at N = 2000 and the number of groups held constant K 5, the dimensionality of features varied across the set [50, 100, 150, 200, 250, 300, 350, 400J. According to Figure 12, it is discerned that as feature dimensionality increases, the runtime of MF-CCA exhibits a concomitant augmentation. Conversely, the runtime associated with SF-CCA remains comparatively stable throughout the varying feature sizes.
B.3.2 Runtime Sensitivity of MF-CCA and SF-CCA to
In the subsequent experimental configuration, the feature size is held constant at d = 100, while the sample size varies across the set [600, 800, 1000, 1200, 1400, 1600, 1800, 2000]. Notably, according to Figure 13, the increment in the sample size exerts minimal impact on the runtime duration of SF-CCA This observation can rationalized by considering that an increase in the number of features would lead to covariance matrix of greater dimensionality. Consequently, the
25

---

Table Numerical results regarding three metrics including Correlation (pr), Maximum Disparity (A r)), and Aggregate Disparity rA. The best ones in each row are bold, and the second best one is underlined. The analysis focuses on the initial five projection dimensions for NHANES and ADNI, the initial seven projection dimensions for Synthetic Data, and the initial two projection
dimensions for MHAAPS. "T" means the larger the better and "4" means the smaller the better.
Dataset
Dim. (r)
Pr
Amax, +
Aum.r +
CCA
MF-CCA SF-CCA|
CCA
MF-CCA SF-CCA|
CCA
MF-CCA SF-CCA
0.8003
0.7934
0.7757
0.3003
0.2386
0.1829
2.8647
2.2836
1.7258
2
0.7533
0.7475
0.7309
0.3555
0.2866
0.2241
3,3802
2.8119
2.2722
0.7032
0.6980
0.6853
0.3215
0.2591
0.2033
3.0974
2.5135
1.9705
Synthetic
0.5872
0.5818
0.5677
0.3784
0.2791
0.197
3.6619
2.6939
1.8134
Data
0.4717
0.4681
0,4581
0.4385
0.3313
0.2424
4.1649
3.1628
2.2304
6
0,4186
0.4161
0.4131
0.1163
0.0430
0,0000
1.161
0.4142
0.0000
0.4012
0.3989
0.3928
0.2156
0.1336
0.0488
2.0224
1.1771
0.4117
0.7360
0.7305
0.7330
0.0149
0.0113
0.0092
0.0596
0.0450
0.0366
0.6392
0.6360
0.6334
0.0485
0.0359
0.0245
0.1941
0.1435
0.0980
NHANES
0.6195
0.6163
0.6147
0.0423
0.0322
0.0187
0.1691
0.1287
0.0747
(Education)
0,5027
0.4961
0.4990
0.0506
0.0342
0.0243
0,2022
0.1367
0.0974
0,4416
0,4393
0.4392
0.1001
0.0818
0.0824
0.4003
0.3272
0.3297
0.7376
0.7359
0.7365
0.0482
0.0479
0.0468
0..197
0.1916
0.1874
0.6400
0.6374
0,6383
0.0101
0.0097
0,0048
0.0402
0.0389
0.0191
NHANES
0.6221
0.6216
0.6195
0.0739
0.0708
0.0608
0.2955
0.2834
0.2432
(Race)
0.5001
0.4990
0.4980
0.0887
0.0774
0.0685
0.3549
0.3096
0.2742
0,4459
0.4454
0.4442
0,1561
0.1451
0.1413
0,6244
0,5805
0,5653
MHAAPS
0.4464
0.4451
0.4455
0.0093
0.0076
0.0044
0.0187
0.0152
0.0088
(Sex)
0.1534
0.1529
0.1526
0.0061
0.0038
0.0019
0.0122
0.0075
0.0039
0.8472
0.8468
0,8450
0.0190
0.0180
0.0165
0.038
0.036
0.0329
ADNI
2
0.7778
0.7776
0.7753
0.0131
0.0119
0.0064
0.0263
0.0238
0.0127
AV45 and
0.7369
0.7360
0.7332
0.0460
0.0371
0.0269
0.0919
0.0743
0.0538
AV1451
0.7022
0.7003
0.6985
0.0083
0.0046
0.0018
0.0167
0.0092
0.0037
(Sex)
0.6810
0.6798
0.6770
0.0477
0.0399
0.0324
0.0954
0.0799
0.0648
0.7391
0.7386
0.7373
0.0146
0.0149
0.0135
0.0293
0.0299
0.0270
ADNI
0,5232
0.5224
0.5223
0.1212
0.1127
0.1116
0.2424
0.2254
0,2233
AV45 and
0.5189
0.5185
0.5182
0.1568
0.1508
0.1497
0.3135
0.3017
0.2994
Cognition
4
0.4999
0.4979
0.4985
0.12
0.1061
0.1084
0.2485
0.2123
0.2168
(Sex)
0.4904
0.4886
0.4896
0.0249
0.0124
0.0165
0.0498
0.0248
0.0329
corresponding numerical computations, such as the resolution of eigenvalues, become increasingly time-intensive.
B.3.3 Runtime Sensitivity of MF-CCA and SF-CCA to K
Finally, we examined the sensitivity of both SF-CCA and MF-CCA to the number of subgroups. Synthetic data was generated with varying numbers of subgroups, allowing us to assess the corre- sponding running time. The results are presented in Figure 14. From the figure, it is evident that conventional CCA is not significantly sensitive to the number of groups. However, MF-CCA exhibits a higher level of sensitivity the number of groups compared SF-CCA
C Experimental Details and Hyperparameter Selection Procedure
In this section, we delve into the particulars of our experiments, which encompass the choice of the penalty function and hyperparameters. We initiate discussion with the penalty function & outlined in Equation (8). This function takes on various forms, such as absolute, square, and exponential functions. For our experimental purposes, we concentrate specifically on the absolute function.
The distinctive characteristic of the absolute function lies in its robustness against minor discrepancies, opposed to the square function, which tends to converge rapidly towards zero. It's worth noting
26

---

Table Performance change from baseline CCA to F-CCA algorithms (MF-CCA and SF-CCA) in the format of percentage regarding three metrics: Correlation (p,), Maximum Disparity (A (Amax,)) and Aggregate Disparity r.. The changes are computed using the numerical results from
Table 4: Ppr
Pr. Pf F-CCA--AAEE PeCA-p.ue CCA X 100, PAmax,"
Amax, of Amaxx CCA X 100, Amax, of
pr, CCA
PA
sumrr Aam, M-CCA-AAum, of CCA 100. Here, F-CCA is replaced with either MF-CCA Aun, CCA
or SF-CCA to obtain the results in each column. The formulations are determined based on the properties of the metrics where Pr is the larger the better while Amax.r and Aum, are the smaller the better.
Dataset
Dim. (r)
Ppr
PAmax,
PAum.
ME-CCA SF-CCA
ME-CCA SF-CCA
MF-CCA SF-CCA
-0,6688
-3.0817
20,8021
39.2877
20.2834
39.7572
2
-0,7726
-2.9710
19.3625
36.9495
16.8132
32.7792
-0,7366
-2.5461
19.3916
36.7677
18.8495
36.3813
Synthetic
-0.9243
-3.3148
26.245
47.9246
26.4333
50.4784
Data
-0,7660
-2.8896
24.4426
44.7114
24.0612
46.4484
6
-0,5865
-1.3126
63.0023
99.9997
64.3202
99 99.9997
7
-0.5767
-2,0838
38.0292
77.3821
41.7986
79.6436
-0,7511
0.4184
24.5248
38.5621
24.5248
38.5621
-0.5131
0.9070
26.0899
49.5206
26.0899
49.5206
NHANES
-0.5243
0.7835
23,8580
55.7921
23 23,8580
55.7921
(Education)
-1,3118
-0.3099
32,4151
51.833
32,4151
51.833
5
-0.5175
-0.5422
18.2615
17.6268
18.2615
17.6268
-0,2329
-0.1550
0,5481
2.7539
0.5481
2.7539
2
-0,3972
-0.2625
3,3229
$2.6573
3.3229
$2.6573
NHANES
-0.0931
-0.4209
4.1097
17.7082
4.1097
17.7082
(Race)
-0.2334
-0.4317
12.7708
22.7437
12.7708
22.7437
-0.1113
-0.3914
7.0315
9.4561
7.0315
9.4561
MНAAРS
-0,2917
-0.2084
18.6150
$2.8984
18.6150
52.8984
(Sex)
2
-0,2724
-0.4941
38.2692
68.1768
38,2692
68.1768
-0.0444
-..6004
5,3213
13.4811
5.3213
13,4811
ADNI
2
-0,0222
-0.3188
9,5217
51.4446
9,5217
51,4446
AV45 and
3
-0.1294
-0.4999
19.1779
41.4136
19.1779
41,4136
AV1451
-0.2648
-0.5307
44.6624
77.8152
44.6624
77.8152
(Sex)
-0.1737
-0.5918
16.288
32.0918
16.288
32.0918
-0.0631
-0.4416
220189
7.7392
-2.1199
-7.7392
ADNI
-0.1436
-0.1690
7.0116
7.8657
7.0116
7.8657
AV45 and
3
-0.0832
-0.1386
3.7799
4.5125
3.7799
4.5125
Cognition
4
-0.4074
-0.2842
14.5679
12.7538
14.5679
12.7538
(Sex)
5
-0.3663
-0.1533
50.094
33.8674
50.094
33.8674
that due to the non-differentiabiiity the absolute function at the origin, its identification within the MATLAB environment requires the utilization of the sign function. When the input to the penalty function reaches zero, the sign function indicates that the discrepancy error has already reached zero, leading to the termination of the training process.
C.1 Hyperparameters Selection
When it comes selecting hyperparameters, the key decisions revolve around providing the appro- priate learning rate and the parameter À within the SF-CCA framework. Our approach consists of two-step process. Firstly, we ascertain the optimal À for each dataset through an extensive sensitivity analysis, as detailed in Section B.2.3. With these derived values in hand, we then conduct a compre- hensive grid search determine the best-fitting learning rate. The unique parameter combinations utilized for each experimental set are elucidated as follows:
Synthetic Data: 10, learning rate of 2c-2 SF-CCA, and 4e-1 in MF-CCA.
NHANES (Education): À = =,, learning rate of 2e-3 in SF-CCA, and 5e-2 in MF-CCA.
27

---

1-1
A-05
Am
A-1A-03
Corelaton
2--
Opneation
0.72
1.5
0.30
0.720
17.0
-1
0.724
20
2.3
2.0
13
4.0
4.3
3,0
33
0.0
Agprogati Dippriity
(x13"))
(a) Synthetic Data
(b) NHANES (Education)
0.4461
Curdicion
Cordlition
0.4660
7.35
1.15
0.445
Aggregate Dicooitry Oac
Aggregate resparity Eo:10
(c) NHANES (Race)
(d) MHAAPS (sex)
0.730
10.1
Conellaten
0.737
0.42
0.810
0.73
f1
1.5
2.0
2.3
3.0
3,5
4.2
2.03
2.70
2.75
2.00
2.50
Disperity
(c) ADNI AV45/AV1451 (sex)
(f) ADNI AV45/Cognition (sex))
Figure 11: Sensitivity correlation and disparity error to À in SF-CCA framework. Higher emphasizes fairesss correlation (accuracy). Moving right left, accuracy drops fairness improves (smaller disparity). A notable trend links higher correlation with reduced faimesss.
NHANES (Race): À 5, learning rate of 1e-3 in SF-CCA, and 2e-2 in MF-CCA.
MHAAPS: À 10, learning rate of 2e-2 in SF-CCA, and 4e- in ME-CCA.
ADNI (AV45 and AV1451): À learning rate of le-3 in SF-CCA, and 1e-2 in MF-CCA.
ADNI (AV45 and Cognition): À 5, learning rate of 1e-3 SF-CCA, and 2e-2 in MF-CCA.
C.2
Fairness and Correlation Measures
The correlation and fairness between X and Y under projections of U and V within R-dimensional spaces can be quantitatively measured by
trace X
(21a)
tU X tUaCeaCeeY Y Y YV)
max
(U, (U, max [j([K]
(21b)
- A sum Σ -(U,V)-EE(U) -E( [JE[K]
(21c)
In Section 4.1, instead of employing matrix-ban based measurements (21). we adopted component-wise measurements (13) to facilitate detailed observations on each individual projection dimension r. In the following discussion, we demonstrate that the component-we wisse measurements, across all r dimensions, provide more aggressive measurement approach compared the matrix variants (21). In other words, it is evident that having large component-wit wise correlation values pr and small
28

---

700
CCA
600
MF-CCA
SF-CCA
500
Runtime
400
300
200
100
50
100
150
200
250
300
350
400
Number of Features (d)
Figure 12: Computation time (tean=std) of 10 repeated experiments for the total three projection dimensions on synthetic data comprising four subgroups (K). The number of samples fixed at N 2000, while the number of features varies.
CCA
SF-CCA
120
100
Runtime
80
60
40
20
1600
Number of Samples (n)
Figure 13: Computation time (tean=std) of 10 repeated experiments for the total three projection dimensions synthetic data comprising four subgroups (K). The number of features is fixed at 100, and the number of groups is held constant at K = 5, while the number of samples varies.
disparity errors Max,r and Asm, for all r [R] can guarantee large values of p and small values of Amax and Agm-
The following lemma rigorously supports this observation.
Lemma 13. Let Pr, Amax,r A be defined as in (13), and let p. Amax: Armm be defined as in (21). Then,
R
R
Pri
S
Amxxrr
sum
A
sum,
29

---

600 MF-CCA
SF-CCA
Runtime
200
7
8
Number of Subgroups (K)
Figure 14: Computation time (mea of 10 repeated experiments for the total seven projection dimensions on synthetic data comprising varying numbers of subgroups (K). The number of features fixed at d 100.
Proof. Since [u uRl R2xx and V [V
vRl R.
we have
R trace(U" Σ u
x
Y
R trace(U xrace(UxxxXX] = Σ u,
x
Xu,,
(22)
r=1
trace Y = Σ Y
Note that u [u|U x
XU= IR} and == ( Y YV= Y
Using the implementation
of the retraction operation on U and V (see, (19) and (17)), we obtain
u Xu, X = v,, 1,
U x XU V IRa
(23)
trace U _ XU) trace (V y YV) = trace(IR)* = R².
Thus, for the womponent-wise measure defined in (13a) and the matrix variant p defined in (2la),
we have
trace U T YV)
p Vrace( U XU) trace ( YTYV)
u, X SY,,
R
u x Yv.
R
1
(24)
R
Vu, X Xu, Yv,
R
Pr.
30

---

For component wise measure Amax, and matrix measure Amax. we have
! max
max []j[[]]
(U, ( ())
(traee U T x vivii)) trace (oTxxXiiv)
max []][K]
trace (USin x Y'V..)
trace (UX xvv))
n u
n Σ
Σ
x T Y'v!""
u X Y'v.
max [JE[K]]
Y Σ ur T x xim))
u
x
[ T x Y Y'v=' ur _ viv.)
max i.jE[K]
(25)
(u-- x Y ur T x Y'v.)
(u.t x T Y v.- u, _x x'vr))
x
max (JE(K]
[1. Txxxx x
v - u, Txd riv.))
v,) - 8d (ur.
max (E((K]
A max,r
Similarly, for Asm,, and Asmm we have Aum $ E/ sum.
31


Reviews:
[{"review_id": "Bsm47fIvob", "reviewer": "Reviewer_kQSh", "summary": "This paper addresses a fairness issue that arises in CCA, proposing a fair CCA that well trade-offs correlation disparity errors w.r.t. sensitive attributes against correlation w.r.t. global projection subspaces. It introduces two optimization frameworks (multi-objective and single-objective), then developing corresponding efficient algorithms based on the generalized Stiefel manifold together with convergence analysis. Experimental results both on synthetic and real datasets are provided to validate the theoretical findings. ", "questions": "See Weakness in the above. ", "limitations": "See Weakness in the above. ", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "S1. The paper is very well written. Many illustrations (e.g., Fig. 1) are greatly helpful in grasping the ideas. \n\nS2. The proposed optimization frameworks are convincing, and the corresponding translation techniques enable the use of efficient algorithms in the manifold literature. In addition, the theoretical analysis (Theorems 3 and 4) provides convergence guarantees of the algorithms. \n\nS3. Experimental results emphasize the efficacy of the proposed approach, and the discussions therein are insightful.\n", "weaknesses": "W1. The translations for efficient algorithms, (8) and (9), can further be detailed for those who are not familiar with the manifold literature. \n\nW2. For Theorems 3 and 4, the proof sketches (or technical contributions if any) are preferred to be included in the main body. \n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> **W1:** The translations for efficient algorithms, (8) and (9), can further be detailed for those who are not familiar with the manifold literature.\n\n**Response:**  Thank you for your valuable comment. We have addressed the efficient solutions to subproblems (8) and (10) in Appendices A2 and A3, and discussed the manifold retraction operator used in these sections in Appendix A1. To provide detailed explanations on the subproblems, we will add a sentence in the main text referring readers to Appendix A.\n\nRegarding the translations for efficient algorithms for solving subproblems (8) and (10), we understand the importance of providing additional details, particularly for readers who may not be familiar with the manifold literature. Following your valuable suggestions, we will include comprehensive discussions about subproblems (8) to enhance clarity and provide necessary context. We note that since subproblem (10) can be regarded as a specific case of subproblem (8) with $M=1$, where the function $f$ is replaced by its regularized counterpart on a smooth manifold, we focus on the latter. We will primarily include a summary of the following discussions in the paper. \n\n\nWe provide a more detailed explanation of the multi-objective subproblem (8) and its connection to the convergence of standard gradient descent on Euclidean space. Notably, the definition of Pareto critical on Manifold extends the classical critical condition $\\nabla f (x) = 0$ for the single objective case on Euclidean space (i.e., $M = 1$) to vector optimization on Manifold.\n\nIndeed, following single-objective optimization, we are looking for the gradient-based directions $\\mathbf{P}_t$ such that $ \\left\\Vert\\mathbf{P}_t\\right\\Vert \\rightarrow 0$.  To achieve this goal, we follow existing literature on multi-objective optimization  [[FS00](https://link.springer.com/article/10.1007/s001860000043)], and first use Lemma 7 (appendix) to show that the optimization subproblem (8) has a unique solution $\\mathbf{P}_t$ on the manifold and can be expressed in closed form. Hence, the solution to subproblem (8) can be computed using a variety of minmax methods, and we found that the Goal Attainment Method [[G75](https://ieeexplore.ieee.org/document/1101105), [FP86](https://www.infona.pl/resource/bwmeta1.element.ieee-art-000004256366), [F86](https://www.sciencedirect.com/science/article/abs/pii/B9780080316659500122)]  gives a more stable and accurate solution for our subproblem (8). Then, we provide Lemmas 9-10 (appendix) to show that $\\mathbf{P}_t$ is actually a descent direction and provides an estimate for the decrease of the function $\\mathbf{F}$ along the solution of (8). These results correspond to the descent of the objective in the single objective case and are well-studied in the optimization literature for gradient descent-type methods.\n\n \n> **W2:** For Theorems 3 and 4, the proof sketches (or technical contributions if any) are preferred to be included in the main body.\n\n**Response:** Thank you for the invaluable suggestion. Fortunately, NeurIPS permits an extra content page for the camera-ready version, which allows us to incorporate the following proof sketch:\n\nWe provide the proof of Theorem 3 in three steps:\n\n* **Step I**: We use Lemma 7 to show that the optimization subproblem (8) has a unique solution $\\mathbf{P}_t $.\n* **Step II**:  We provide Lemmas 9-10 to show an estimate for the decrease of the function $\\mathbf{F}$ along the solution of (8); that is  for any $\\eta_t \\geq 0$, we have\n$\n\\mathbf{F}(\\eta_t\\mathbf{P}_{t}) \\preceq \\mathbf{F}(\\mathbf{U}_t, \\mathbf{V}_t) - ( \\eta_t-L_F \\eta_t^2/2) \\left \\Vert \\mathbf{P}_t \\right \\Vert^2 \\mathbf{1}_M.\n$ \nThis can be seen as an extension from single-objective manifold optimization [[Nicolas23](https://www.nicolasboumal.net/book/IntroOptimManifolds_Boumal_2023.pdf)]  to the multi-objective counterpart. \n* **Step III**:  Finally, summing both sides of the inequality in Step II for $t=0, 1, \\ldots, T-1$, and using our step size condition, gives the desired result.\n\n\nThe proof of Theorem 4 follows steps similar to Theorem 3:\n\n* **Step I**:  We use a single objective variant of Lemma 7 to establish the unique solution $\\mathbf{G}_t$ for the optimization subproblem (10).\n* **Step II**:  A single objective variant of Lemmas 9-10 shows that for any $\\eta_t \\geq 0$, we have a descent inequality as follows:\n$\nf(\\eta_t\\mathbf{G}_{t}) \\leq f(\\mathbf{U}_t, \\mathbf{V}_t) - ( \\eta_t-L_f \\eta_t^2/2)  \\left \\Vert \\mathbf{G}_t \\right \\Vert^2.\n$ \n* **Step III**:  Finally, by summing both sides of the inequality in Step II for $t=0, 1, \\ldots, T-1$, and using our step size condition, we obtain the desired result."}, {"review_id": "Ed7xAMMgqL", "reviewer": "Reviewer_MdZ1", "summary": "This paper investigates the concept of Fair CCA, focusing on addressing the potential bias that arises when analyzing the relationship between two sets of variables using CCA, a widely utilized statistical technique. The conventional application of CCA fails to account for the impact of sensitive attributes like gender or race, leading to potential biases. In response, this study aims to bridge this gap by integrating fairness principles into CCA. The authors introduce the fairness issue within the context of CCA and propose two distinct methods to tackle it: a multi-objective approach and a single-objective approach, each offering unique strengths. The effectiveness of the proposed methods is substantiated through empirical and theoretical analyses, confirming their value in addressing the fairness concerns in CCA.\n", "questions": "1. Clarification on \"critical Pareto\" (line 164): The definition of \"critical Pareto\" is unclear. It would be helpful to provide a more intuitive explanation for better understanding.\n2. Elaboration on optimization problem (8) and steepest descent direction (lines 174-175): The statement regarding the optimization problem and steepest descent direction requires further elaboration to enhance clarity.", "limitations": "The method is currently limited to two modalities. Even though this is the most common scenario in CCA, it would be interesting to see how the method can be extended to more than two modalities.\n\nOverall, this is a well-written and informative paper that makes a significant contribution to the field of machine learning. The proposed methods are novel and effective, and the experimental results are convincing. However, the method is currently limited to two modalities, and it would be interesting to see how it can be extended to more than two modalities.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The problem addressed in this paper holds significant importance. Given the increasing influence of machine learning algorithms and methods on individuals and society, it becomes crucial to delve into the study of fairness within this domain. By mitigating bias issues in machine learning, we can contribute to a more equitable outcome and benefit vulnerable groups. While numerous works have explored fairness in machine learning, the majority of them focus on the supervised learning scenario. In contrast, this paper ventures into uncharted territory by examining the fairness issue in CCA, an unsupervised learning approach. This unique perspective underscores the urgency and significance of studying fairness within the context of CCA.\n2. The concepts and methods presented in this paper exhibit a high degree of novelty. To the best of my knowledge, this is the first study to explore fairness within the context of CCA. Fairness, being a multifaceted concept, encompasses various definitions. In the realm of supervised learning, researchers have proposed different definitions such as demographic parity, equalized odds, and group sufficiency. Therefore, establishing a practical and reasonable definition becomes crucial. This paper introduces the notion of fairness criteria through correlation disparity error, which takes into account both global and group-wise correlations. The resulting fairness definition is intuitive and reasonable. The incorporation of fairness as additional objectives (in the multi-objective framework) or constraints (in the single-objective framework) is accomplished seamlessly, aligning with the natural progression of the problem. Furthermore, the authors introduce the Riemannian manifold in their solutions, which promotes convergence and facilitates computation, thereby introducing a novel aspect to the research.\n3. The paper exhibits good writing quality, characterized by clarity and soundness. It effectively guides readers through its content, ensuring easy comprehension from the motivation and definition to the methods and solutions. Notably, Figure 1 provides a clear and intuitive visualization that enables immediate understanding of the proposed method's functionality. The effectiveness of the methods is supported by robust experimental results on synthetic and real data. Furthermore, Figure 3 serves as a compelling validation of fairness, as it visually demonstrates the improved proximity between the two groups after the projection using the proposed methods. Overall, the paper is meticulously crafted, maintaining a high level of clarity and rigor throughout.\n\n", "weaknesses": "1. Limited discussion on multiple modalities: While CCA is not restricted to two modalities, the paper primarily focuses on this scenario. It would be beneficial to discuss a more general setting involving multiple modalities and computing correlations under the fairness setting.\n2. Figure 4 lacks an obvious trend: The authors could consider including Figure 9 from the supplementary file in the main body, as it provides a more intuitive demonstration of the method's effectiveness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> **W1:** Limited discussion on multiple modalities: While CCA is not restricted to two modalities, the paper primarily focuses on this scenario. It would be beneficial to discuss a more general setting involving multiple modalities and computing correlations under the fairness setting.\n\n**Response:**  Thank you for your valuable suggestion. We will discuss extending F-CCA to multiple modalities or generalized scenarios in our paper as future work.  Existing multiset CCA methods mostly revolve around the concept of maximizing pairwise correlation sums across subsets. We intend to leverage this concept to accommodate multiple modalities within our framework. Yet, these extensions come with notable challenges. Effectively handling fairness, bias, and solving single/multi-objective optimization problems across multiple modalities or sets requires fresh formulations and algorithmic approaches. This entails optimizing correlation or generalized correlation among sets while factoring in fairness, stability, and computational complexity, particularly when dealing with high-dimensional data issues [[ZYC20](https://pubmed.ncbi.nlm.nih.gov/32592530/),[TT11](https://link.springer.com/content/pdf/10.1007/s11336-011-9206-8.pdf),[X68](https://psycnet.apa.org/record/473742008-115?doi=1)]. Hence, developing a fair Generalized CCA approach requires addressing numerical stability and algorithm design, which we consider as future research problems.\n\n> **W2:**  Figure 4 lacks an obvious trend: The authors could consider including Figure 9 from the supplementary file in the main body, as it provides a more intuitive demonstration of the method's effectiveness.\n\n**Response:** Thank you for your suggestion. We'll replace Figure 4 with Figure 9, illustrating aggregate disparities across multiple datasets with varying projected dimensions. In our discussion, we'll detail SF-CCA and MF-CCA in connection with Figure 9. Notably, CCA consistently exhibits greater disparities than our SF-CCA and MF-CCA, highlighting our framework's effectiveness.\n\n\n> **Q1:**  Clarification on \"critical Pareto\" (line 164): The definition of \"critical Pareto\" is unclear. It would be helpful to provide a more intuitive explanation for better understanding.\n\n**Response:** Thank you for your comment.   In multi-objective optimization, conflicting objectives are simultaneously optimized, leading to trade-offs between solutions. The Pareto front comprises solutions that can't be enhanced in one objective without worsening another, defining Pareto optimal solutions. Pareto critical solutions lie on this front, representing vital trade-offs between objectives and providing optimal choices with distinct trade-offs for decision-makers.\n\nIn Section 3.3, we introduced the concept of critical Pareto points. A point is considered critical Pareto if the image of the gradient of $\\mathbf{F}$ at that point does not intersect with positive $M$-dimensional real numbers. In other words, the solution $\\mathbf{P}$, as obtained from the optimization problem $(8)$, satisfies $\\max_{i \\in M} \\langle \\nabla f_i(\\mathbf{U}, \\mathbf{V}), \\mathbf{P} \\rangle = 0$. In simpler terms, there are no feasible directions $\\mathbf{P} \\neq \\mathbf{0}$ that can simultaneously satisfy $\\langle \\nabla \\mathbf{F}_i(\\mathbf{U}, \\mathbf{V}), \\mathbf{P} \\rangle \\geq 0$ for all objectives while having at least one objective $i$ with $\\langle \\nabla \\mathbf{F}_i(\\mathbf{U}, \\mathbf{V}), \\mathbf{P} \\rangle > 0$.\n\nIn summary, Pareto critical points ensure that no feasible directions can reduce the norm of the component-wise gradient. This concept is similar to the single-objective case, where critical points prevent feasible descent directions from reducing the gradient norm.\n\n\n> **Q2:**  Elaboration on optimization problem (8) and steepest descent direction (lines 174-175): The statement regarding the optimization problem and steepest descent direction requires further elaboration to enhance clarity.\n\n\n**Response:** Thank you for your question. We have addressed subproblem (8) efficiently in Appendix A2 and discussed the manifold retraction operator in Appendix A1. A reference to Appendix A will be added in the main text, along with a summary of the following discussions.\n\nWe elaborate on subproblem (8) and its relation to the convergence of standard gradient descent in Euclidean space. The definition of Pareto critical on Manifold extends the classical critical condition $\\nabla f (x) = 0$ for the single objective case on Euclidean space ($m = 1$) to vector optimization on Manifold. Indeed, following single-objective optimization, we are looking for the directions $\\mathbf{P}_t$ such that $ \\left\\Vert\\mathbf{P}_t\\right\\Vert \\rightarrow 0$.  To achieve this goal, we follow existing literature on multi-objective optimization  [[FS20](https://link.springer.com/article/10.1007/s001860000043)], and first use Lemma 7 (appendix) to show that the unconstrained optimization subproblem (8) has a unique solution $\\mathbf{P}_t$ on the manifold and can be expressed in closed form. Hence, the solution to subproblem (8) can be computed using a variety of minimax methods, and we found that the Goal Attainment Method [[G75](https://ieeexplore.ieee.org/document/1101105), [FP86](https://www.infona.pl/resource/bwmeta1.element.ieee-art-000004256366), [F86](https://www.sciencedirect.com/science/article/abs/pii/B9780080316659500122)]  gives a more stable and accurate solution for our subproblem (8).  Next, Lemmas 9-10 (appendix) demonstrate that $\\mathbf{P}_t$ is a descent direction and offer an estimate for the function $\\mathbf{F}$'s decrease along the solution of (8). These findings mirror the descent of the objective in the single objective case and are extensively explored in the optimization literature for gradient descent-type methods.\n"}, {"review_id": "FBKe4RsLTM", "reviewer": "Reviewer_1Xpv", "summary": "This paper addresses fairness and bias in Canonical Correlation Analysis (CCA). The authors propose a framework that minimizes correlation disparities associated with protected attributes, reducing unfairness without compromising accuracy. Experimental evaluation validates the effectiveness of the approach. The findings emphasize the importance of fairness in CCA applications. ", "questions": "- In the experimental setup, what considerations were made in selecting the synthetic and real-world datasets? Were there any specific characteristics of these datasets that influenced the results or generalizability of the findings? \n\n- While the paper focuses on fairness and bias in CCA, could you elaborate on the potential implications of the findings for other statistical methods or broader machine learning applications? How transferable do you believe the proposed framework is beyond the scope of CCA?\n\n- Are there any additional factors or considerations that should be taken into account when applying the proposed framework in practical settings? For instance, how would the framework handle missing data, outliers, or high-dimensional datasets?\n\n- Given the goal of reducing unfairness, how does the proposed framework balance the trade-off between fairness and overall predictive accuracy? Were there any cases in the experiments where the framework significantly sacrificed accuracy to achieve fairness? ", "limitations": "- Fairness Metrics: The paper could provide a more detailed discussion on the fairness metrics used to evaluate the proposed framework. Further elaboration on the choice and justification of these metrics would enhance the clarity and interpretability of the experimental results.\n\n- Generalizability: The generalizability of the findings may be limited by the specific characteristics and distribution of the datasets used in the experiments. The authors could discuss the potential challenges or variations that may arise when applying the framework to other datasets or domains.\n\n- Trade-off between Fairness and Accuracy: The paper briefly mentions that the proposed framework minimizes unfairness without compromising the accuracy of the CCA model. However, a more in-depth analysis of the potential trade-off between fairness and accuracy would provide a clearer understanding of the framework's limitations and its impact on prediction performance.\n\n- Real-world Application Challenges: While the framework demonstrates effectiveness in reducing unfairness, the paper does not extensively discuss the practical challenges that may arise when applying the proposed approach to real-world scenarios, such as handling missing data, complex feature distributions, or scalability issues. ", "rating": 7, "confidence": 2, "soundness": 4, "presentation": 2, "contribution": 3, "strengths": "- Novel Contribution: The paper introduces a framework to address fairness and bias concerns in Canonical Correlation Analysis (CCA), making a valuable contribution to the field.\n\n- Practical Relevance: By focusing on CCA, a widely used statistical technique, the paper addresses a real-world problem and emphasizes the importance of considering fairness in data analysis.\n\n- Experimental Validation: The authors conduct experiments on both synthetic and real-world datasets, providing empirical evidence of the effectiveness of their proposed framework in reducing unfairness without compromising the accuracy of the CCA model.\n\n- Clear Presentation: The abstract provides a concise overview of the paper's objectives, approach, and findings, making it easy to understand the key contributions of the research. ", "weaknesses": "- Lack of Detailed Metrics: The paper could benefit from providing more specific details about the metrics used to measure unfairness and bias in CCA. This would enhance the transparency and reproducibility of the experimental evaluation. \n\n- Limited Comparison: The paper does not explicitly compare the proposed framework with existing fairness-aware CCA methods. Including such comparisons would provide insights into the relative performance and advantages of the proposed approach. \n\n- Scope and Generalizability: While the paper addresses fairness concerns in CCA, the focus is limited to this specific technique. It would be beneficial to discuss the potential implications of the findings for other statistical methods or broader machine learning applications. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "\n> **Q1:**  In the experimental setup, what considerations … real-world datasets? Were there any specific characteristics of these datasets that influenced the results or generalizability of the findings?\n\n**Response:**  We carefully selected synthetic and real-world datasets to evaluate the fair CCA method across various scenarios. Real-world datasets from diverse fields further validated its applicability. More details about these datasets can be found in Appendix B. For example, ADNI data is analyzed for fairness in medical imaging classification [[ZYH22](https://arxiv.org/abs/2210.01725)]. Fairness concerns with ADNI image data could stem from under-representation of specific populations (ethnic, cultural, or economic backgrounds). Moreover, over- or under-representation of age, sex, or biometric groups might affect data fairness. Disease distribution among sensitivity features, such as higher AD occurrence in women, can also impact fairness [[MS16](https://www.thelancet.com/journals/laneur/article/PIIS1474-4422(16)00067-3/fulltext)].\n\n> **Q2:** While the paper focuses on fairness and bias in CCA, could … machine learning applications? How transferable do you believe the proposed framework is beyond the scope of CCA?\n\n**Response:**  Thank you for this valuable question. \n-  As suggested by reviewer MdZ1, our work holds potential for extension to multiple modalities. CCA has been adapted for intricate scenarios like multiset CCA, kernel CCA for nonlinear situations, and deep CCA for imaging data  [[ZYC20](https://pubmed.ncbi.nlm.nih.gov/32592530/)].  \n\n- Our algorithm for smooth manifold multi-objective optimization can also tackle other problems like fair PCA, as both can be framed as optimization challenges on smooth manifolds [[Nicolas23](https://www.nicolasboumal.net/book/IntroOptimManifolds_Boumal_2023.pdf)].\n\n -  As CCA finds applications in downstream tasks like clustering, classification, and manifold learning, our approach can effectively ensure fairness in such scenarios when utilizing CCA methods [[ZYC20](https://pubmed.ncbi.nlm.nih.gov/32592530/)]. \n\nThus, our framework has potential beyond CCA, but these extensions could introduce novel optimization and computation challenges, left for future research.\n\n> **Q3:** Are there any additional factors or considerations … in practical settings? For instance, how would the framework handle missing data, outliers, or high-dimensional datasets?\n\n**Response:**   Thanks for your question. \n\n- Extending fair CCA to handle missing data is notably more challenging than addressing missing data within standard CCA, given the potential for biases and distorted correlations. This challenge is emphasized in [[F21](https://onlinelibrary.wiley.com/doi/full/10.1002/int.22415),[ZL21](https://proceedings.neurips.cc/paper_files/paper/2021/hash/85dca1d270f7f9aef00c9d372f114482-Abstract.html)]. The added complexity arises from the integration of fairness considerations with missing data biases. \n- Outliers have the capacity to disrupt CCA, affecting reliability and interpretations, and potentially steering fair CCA toward suboptimal outcomes. \n- Sparse CCA addresses high-dimensional data using techniques like $L_1$ penalty to induce sparsity in canonical vectors. However, incorporating Sparse CCA into multi-objective optimization introduces challenges tied to nonsmooth optimization issues.\n\nIn addressing these concerns, especially in our multi-objective context, we need novel formulations and analysis for future work.\n> **Q4:**  Given the goal of reducing unfairness, how does the proposed framework balance the trade-off between fairness and overall predictive accuracy?\n\n**Response:** Thank you for your question. \n\n-  Our experiments explored the impact of the $\\lambda$ parameter in the SF-CCA method on correlation and disparity, as depicted in Figure 12 of the appendix. The results confirmed our expectations: higher $\\lambda$ values led to reduced disparity and correlation. Interestingly, we observed a gradual decline in correlation as $\\lambda$ increased, contrasted by a rapid reduction in disparity. This showcases our framework's ability to enhance fairness while preserving accuracy. In Figure 12 (a), the correlation plateaued between $\\lambda$ values of 0.001 and 10, while disparity rapidly decreased to near-zero at $\\lambda = 1$. Even at $\\lambda = 10$, the correlation remained reasonable. Notably, the optimal balance between correlation and fairness was achieved at $\\lambda = 1$.\n\n-  We extended our experiments to real datasets, illustrated in Figure 2 of the attached PDF in the general rebuttal. A consistent pattern emerged: as fairness improved (disparity decreased), accuracy (correlation) declined. Yet, by pinpointing an optimal $\\lambda$, we significantly improved fairness without compromising accuracy.\n\nIn summary, our approach attains fairness with competitive accuracy, distinguishing it from traditional CCA methods that overlook fairness concerns.\n\n\n> **Q5:** Were there any cases in the experiments where the framework significantly sacrificed accuracy to achieve fairness?\n\n**Response:**   Our empirical results show that our framework maintains accuracy while enhancing fairness. In NHANES, MF-CCA and SF-CCA correlations are only 0.5% and 1% lower than CCA in 2 dimensions. Max disparity drops by 26.4% and 29.5% compared to CCA, indicating fair enhancement with minimal accuracy loss across dimensions (Table 1).\n\nFigure 3 in the attached PDF further validates this benefit. Each dataset panel displays two cases for specific dimensions. MF-CCA and SF-CCA columns show changes in correlation ($\\rho$), max disparity ($\\Delta_{\\max}$), and disparity sum ($\\Delta_\\Sigma$). Pearson correlation $\\rho$ change is slight, while $\\Delta_{\\max}$ and $\\Delta_\\Sigma$ changes are substantial, signifying fairness improvement without significant accuracy sacrifice.\n"}, {"review_id": "k9FNh7ludq", "reviewer": "Reviewer_6yqF", "summary": "This paper proposes a fair CCA algorithm that aims to find fair CCA projection matrices. The authors claim that it is necessary to develop an appropriate algorithm for CCA with fairness guarantee. In the presence of sensitive attribute and unfairness in observed data, the proposed algorithms successfully reduces certain unfairness of the learned projection matrices. Specifically, the two proposed algorithms, MF-CCA and SF-CCA, are optimized using gradient descent algorithms on Stiefel manifolds. The convergences of these algorithms are also theoretically guaranteed, and several experiments conducted provide empirical support for the proposed algorithms.", "questions": "- Which penalty function, $\\phi$, was used in the experiments?\n- Is the computation time more significantly impacted by the size of the training data or the dimension of the input feature?\n- Could the authors provide the definition of a 'componentwise Lipschitz continuous function' as stated in Assumption A?", "limitations": "- Unlike the vanilla CCA which solves the objective by optimizing two matrices, $U$ and $V$, the proposed algorithms require more matrices to be optimized, the number of which increases with the number of sensitive attributes, $K.$\n- Naturally, the computational cost of these proposed algorithms is higher than that of the vanilla CCA due to the fairness objectives they must minimize. Any future work aimed at reducing this computational cost would certainly be novel.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is generally well-written and easy to follow in overall.\n- To the best of my knowledge, this work is the first to address the issue of unfairness issue in CCA, which could be a milestone.\n- The fairness metric targeted (i.e., Correlation Disparity Error in Definition 1) is well-defined and fits well with other general fairness notions used in fair prediction tasks (e.g., demographic parity).\n- The proposed algorithms (MF-CCA and SF-CCA) align well with the theoretical studies presented (Theorems 4 and 5).\n- The roles of the two algorithms well-specified. MF-CCA finds the optimal solution by minimizing CCA error and unfairness losses simultaneously, while SF-CCA provides an advantage in controlling the trade-off between CCA error and unfairness losses.\n- Empirical results validate the efficacy of MF-CCA and SF-CCA, not only on synthetic data but also on real datasets.", "weaknesses": "- There is no theoretical guarantee for fairness provided. If the authors could theoretically demonstrate that the solutions to equations 7 and 9 have low $\\mathcal{E}^{k} (U, V),$ as has already been empirically shown, the contribution would be more novel.\n- To illustrate the ability of SF-CCA in controlling the trade-off between error and fairness, a visualization such as Pareto-front lines (commonly used in fair classification problems [1, 2]) would be beneficial. Table 1 only presents results using a single $\\lambda$ selected from the set [1e-2, 1e-1, 1, 10, 100].\n\n[1] https://arxiv.org/abs/1802.06309\n\n[2] https://arxiv.org/abs/2103.06503\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> **W1:** There is no theoretical guarantee for fairness provided. If the authors could theoretically demonstrate that the solutions to equations 7 and 9 have low \\(\\epsilon^k(\\mathbf{U},\\mathbf{V})\\) as has already been empirically shown, the contribution would be more novel.\n\n**Response:** Thank you for your comment.  The CCA problem's similarity to PCA in its non-convex nature means that Theorem 4 (multi-objective) causes the Pareto descent direction norm to diminish, converging algorithm solutions to stationary fair subspaces. To make the CCA objective convex, we can introduce regularization such as $\\ell_2$-sqaure type penalty with hyperparameter $\\alpha>0$,  denoted as $R_{\\alpha}(\\mathbf{U},\\mathbf{V})$, that maintains positive semi-definite Hessian matrices of objectives. This adjustment aims to minimize the fairness error $\\tilde{f}(\\mathbf{U},\\mathbf{V}) - \\tilde{f}(\\mathbf{U}^{\\star},\\mathbf{V}^{\\star})$ toward zero, where $\\tilde{f}(\\mathbf{U},\\mathbf{V}) = f(\\mathbf{U},\\mathbf{V}) + R_{\\alpha}(\\mathbf{U},\\mathbf{V})$, and $f$ represents multi or single objectives as in (7) or (9) respectively. However, the regularization-based approach requires more hyperparameters, leading us to primarily focus on the standard non-convex method in CCA literature. Thus, we offer theoretical first-order stationary analysis using $\\left\\Vert \\mathbf{P} \\right\\Vert$ and experimental evidence of algorithmic convergence toward fair subspaces. \n\n\n> **W2:**  To illustrate the ability of SF-CCA in controlling the trade-off between error and fairness, a visualization such as Pareto-front lines (commonly used in fair classification problems [1, 2]) would be beneficial. Table 1 only presents results using a single selected from the set $ \\lambda$ in $ [1e-2, 1e-1, 1, 10, 100]$.\n\n**Response:** Thank you for your valuable comment. \n\n-  Our experiments examined the influence of the $\\lambda$ parameter in SF-CCA on correlation and disparity (Figure 12 in the appendix). As expected, higher $\\lambda$ values led to decreased disparity and correlation. Notably, a gradual decline in correlation was observed with increasing $\\lambda$, while disparity decreased rapidly. This underscores our framework's ability to boost fairness without significant accuracy loss. In Figure 12 (a), correlation plateaued between $\\lambda$ values of 0.01 and 10, while disparity swiftly neared zero at $\\lambda = 1$. Even at $\\lambda = 10$, the correlation remained reasonable. Importantly, the optimal correlation-fairness balance was attained at $\\lambda = 1$.\n\n-  We extended our experiments to real datasets, illustrated in Figure 2 of the attached PDF. A consistent pattern emerged: as fairness improved (disparity decreased), accuracy (correlation) declined. Yet, by pinpointing an optimal $\\lambda$, we significantly improved fairness without compromising accuracy.\n\nHence, our approach attains fairness with competitive accuracy, distinguishing it from traditional CCA methods that overlook fairness concerns.\n\n\n> **Q1:**  Is the computation time more significantly impacted by the size of the training data or the dimension of the input feature?\n\n**Response:** We've presented time computations for algorithms on real and synthetic datasets in Table 2. CCA's efficiency stems from its singular eigenvalue decomposition, while MF-CCA requires extensive gradient direction searches and repeats of gradient descent. SF-CCA, with a fixed trade-off parameter, runs faster than MF-CCA despite hyperparameter tuning. While comparing MF-CCA and SF-CCA is challenging due to their trade-offs, the extra time invested aligns with our goal of balancing fairness enhancement with accuracy retention.\n\nFurther examining time complexity, we assessed SF-CCA and MF-CCA sensitivity to subgroup count. Figure 11 in Appendix B.3 reveals MF-CCA's increased sensitivity to subgroup count compared to SF-CCA. This corresponds to their performance in Table 2, where SF-CCA excelled in computational efficiency. SF-CCA's trade-offs are apparent: it achieves a balance between computational efficiency and fairness, while MF-CCA guarantees stronger fairness at the cost of longer computation times for more subgroups.\n\n​​For sensitivity analysis on sample count ($n$) and feature count ($d$), we conducted two sets of  news experiments on synthetic data with fixed hyperparameters. Each experiment, repeated 20 times using CCA, MF-CCA, and SF-CCA, is detailed in Figure 1 of the attached PDF.\n\n- For the first set, we maintained sample size and varied feature size [50, 100, 150, 200, 250, 300, 350, 400]. MF-CCA exhibited extended runtimes with increased features, while SF-CCA's runtime remained relatively steady.\n\n- In the second set, fixing feature size, we varied sample size [600, 800, 1000, 1200, 1400, 1600, 1800, 2000]. The impact of   larger sample size on SF-CCA's runtime was relatively minimal.\n\n\n> **Q2:** Which penalty function, $\\phi$ was used in the experiments?\n\n**Response:** Thank you for pointing this out. The function $\\phi$ can take various forms, including absolute, square, exponential, and more. In our experiments, we specifically focus on the absolute function. Its strength lies in its resilience to minor disparities, unlike the square function, which tends to rapidly approach zero. We will ensure that these specific details are included in the paper for improved clarity.\n\n> **Q3:** Could the authors provide the definition of a 'componentwise Lipschitz continuous function' as stated in Assumption A?\n\n**Response:** Thank you for your question. A componentwise Lipschitz continuity means that each $\\nabla f_i$ for $i \\in [M]$ has Lipschitz continuity on the manifold $\\mathcal{M}$ with constant $L_{i,M}$, and $\\nabla \\mathbf{F}$ is componentwise Lipschitz continuous on $\\mathcal{M}$ with constant $L_F:=\\max_{i=1,\\ldots, M} L_{i,M}$. We will clarify this in Assumption A.\n"}]

Author rebuttal:
We thank all the reviewers for their valuable comments and suggestions. We summarize all the most-concerned questions raised by the reviewers below and present **new experiments**, accompanied by detailed explanations for the corresponding **attached Figures 1-3**. 

* Reviewer **kQSh**  has highlighted the need for more detailed explanations of the efficient algorithms utilized to solve subproblems (8) and (10). 

* Reviewer **MdZ1**  has emphasized the importance of addressing multiple modalities for CCA, suggesting an extension of our fair CCA approach to handle such scenarios.

* Reviewer **1Xpv** has expressed concerns about the generalizability of our framework and its applicability to various machine learning techniques or domains.

* Reviewer **6yqF** has observed that our approaches are more time-consuming compared to standard CCA , particularly in optimizing multi-objective problems.

* Reviewers **1Xpv** and **6yqF** have raised concerns regarding the trade-off between correlation disparity error (fairness) and correlation (accuracy).


To address these concerns, we have taken the following actions:

**a1:** We have provided more comprehensive descriptions and explanations of efficient algorithms utilized to solve subproblems (8) and (10), streamlining them for greater clarity and self-consistency within the main body of the paper. Although these descriptions were previously covered in Appendices A.2 and A.3, we aim to enhance their integration into the main text.

      Further details are provided in the response to Reviewer kQSh.

**a2:** We have given discussions regarding the extension of our fair CCA framework to accommodate multiple modalities. This includes exploring the application of our approach to scenarios such as multiset CCA, while acknowledging the challenges posed by optimization, scaling, and numerical stability issues. 

      Further details are provided in the response to Reviewer MdZ1. 

**a3:**  We have elaborated on the generalizability of our optimization framework to diverse machine-learning techniques and domains. This includes its potential extension to more generalized CCA variations including kernel CCA, deep CCA, and multiset CCA as suggested by Reviewer MdZ1. We also explore its application in subsequent CCA-based tasks like clustering and classification. Our approach of enhancing fairness by encouraging global projection matrices to align centrally with local projection matrices on the smooth Manifold has broader applicability and can be extended effectively to enhance other methods such as fair PCA.

      Further details are provided in the response to Reviewer 1Xpv. 

**a4:**   We have conducted additional experiments to assess the time complexity of our approach, focusing on its sensitivity to the number of samples ($n$) and features ($d$). 
    
- The findings, illustrated in Figure 1 of the attached PDF, highlight that MF-CCA’s runtime is notably sensitive to these factors, contrasting with the relative stability of CCA and SF-CCA runtimes. This sensitivity stems from MF-CCA's multi-objective nature, which necessitates optimizing $2K+2$ matrix variables for $M$ component objectives. Yet, this complexity offers the advantage of minimizing hyperparameter ($\lambda$) tuning while still achieving strong outcomes. 

- We would like to emphasize that a runtime comparison of our methods against baseline CCA is indeed included in Table 2 of the paper. Additionally, the runtime sensitivity concerning the number of subgroups ($K$) is discussed in Figure 11 of Appendix B.3. 

In response to the valuable suggestion by Reviewer 6yqF, we have incorporated new figures in the attached file to enhance the comprehensiveness of our runtime-related analysis.

    Further details are provided in the response to Reviewer 6yqF. 

**a5:**  We have conducted new experiments, expanded discussions, and provided insights into the trade-off between fairness and accuracy. We show that our methods maintain accuracy while enhancing fairness, as evidenced by percentage changes. 

  - In Figure 12 of the appendix, we demonstrate how performance reacts to the trade-off parameter $\lambda$ using synthetic data. Increasing $\lambda$ improves fairness but may slightly reduce accuracy, a reasonable outcome given its role in fairness regularization. Notably, the accuracy decline is modest despite significant fairness gains, showcasing our methods' capability to balance accuracy and fairness for SF-CCA. 

  - Extending our experiments to real datasets, we present analogous results in Figure 2 of the attached PDF. Additionally, Figure 3 of the attached PDF depicts the performance changes across various datasets, underscoring our methods' ability to enhance fairness without sacrificing accuracy to a significant degree.

Therefore, the combined analysis of attached Figures 2-3 along with Figure 12 in the appendix offers a comprehensive understanding of the trade-off between fairness and accuracy.

     Further details are provided in the responses to Reviewer 1Xpv and 6yqF. 


**Remark:** A few cited references in the response are absent from our paper's reference list. We assure their inclusion in the final version. 

We appreciate the chance to address any concerns and questions raised by the reviewers during the discussion period.

Meta-review: The reviewers unanimously accepted the paper and agreed that this was a noteworthy contribution to algorithmic fairness. I encourage the authors to consider the reviewers' suggestions, particularly in terms fo clarifying the theoretical guarantees and motivating the choice of fairness metrics.
Decision: Accept (poster)