SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Theorem 5. Suppose Assumption A holds. Let  $(\mathbf{U}_t,\mathbf{V}_t)$  be the sequence generated by MF-CCA. Let  $f_{i}^{*}\coloneqq \inf \{f_{i}(\mathbf{U},\mathbf{V}):(\mathbf{U},\mathbf{V})\in \mathcal{U}\times \mathcal{V}\}$ , for all  $i\in [M]$  and define  $f_{i_*}(\mathbf{U}_0,\mathbf{V}_0) - f_{i_*}^*\coloneqq \min \left\{f_i(\mathbf{U}_0,\mathbf{V}_0) - f_i^*:\quad i\in [M]\right\}$ . If  $\eta_t^{\mathbf{u}} = \eta_t^{\mathbf{v}} = \eta \leq 1 / L_F$  for all  $t\in \{0,\ldots ,T - 1\}$ , then
$$
\min  \left\{\| \mathbf {P} _ {t} \| _ {\mathrm {F}}: t = 0, \dots , T - 1 \right\} \leq \frac {2}{\eta} \left[ \frac {f _ {i _ {*}} (\mathbf {U} _ {0} , \mathbf {V} _ {0}) - f _ {i _ {*}} ^ {*}}{T} \right] ^ {\frac {1}{2}}.
$$
Proof Sketch. We employ Lemma 7 to establish the unique solution  $\mathbf{P}_t$  for subproblem (10). Lemmas 9 and 10 provide estimates for the decrease of function  $\mathbf{F}$  along  $\mathbf{P}_t$ : For any  $\eta_t \geq 0$ , we have  $\mathbf{F}(\mathbf{U}_{t+1}, \mathbf{V}_{t+1}) \preceq \mathbf{F}(\mathbf{U}_t, \mathbf{V}_t) - (\eta_t - L_F \eta_t^2 / 2) \| \mathbf{P}_t \|_F^2 \mathbf{1}_M$ . Summing this inequality over  $t = 0, 1, \ldots, T-1$  and applying our step size condition yields the desired result.
Theorem 5 provides a generalization of [8, Corollary 4.9] to the multi-objective optimization, showing that the norm of Pareto descent directions converges to zero. Consequently, the solutions produced by the algorithm converge to a stationary fair subspace. It is worth mentioning that multi-objective optimization in [23, 6] relies on the Riemannian exponential map, whereas the above theorem covers broader (and practical) retraction maps.
3.4 A Single-Objective Framework for Fair CCA
In this section, we introduce a straightforward and effective single-objective framework. This approach simplifies F-CCA optimization, lowers computational requirements, and allows for fine-tuning fairness-accuracy trade-offs using the hyperparameter  $\lambda$ . Specifically, by employing a regularization parameter  $\lambda > 0$ , our proposed fairness model for F-CCA is expressed as follows:
$$
\underset {\mathbf {U}, \mathbf {V}} {\text {m i n i m i z e}} \quad f (\mathbf {U}, \mathbf {V}) := - \operatorname {t r a c e} \left(\mathbf {U} ^ {\top} \mathbf {X} ^ {\top} \mathbf {Y} \mathbf {V}\right) + \lambda \Delta (\mathbf {U}, \mathbf {V}), \tag {11}
$$
where  $\Delta (\mathbf{U},\mathbf{V}) = \sum_{i,j\in [K],i\neq j}\Delta^{i,j}(\mathbf{U},\mathbf{V})$  ; see Definiton 4.
The choice of  $\lambda$  in the model determines the emphasis placed on different objectives. When  $\lambda$  is large, the model prioritizes fairness over minimizing subgroup errors. Conversely, if  $\lambda$  is small, the focus shifts towards minimizing subgroup correlation errors rather than achieving perfect fairness. In other words, it is possible to obtain perfectly F-CCA subspaces; however, this may come at the expense of larger errors within the subgroups. The constant  $\lambda$  in the model allows for a flexible trade-off between fairness and minimizing subgroup correlation errors, enabling us to find a balance based on the specific requirements and priorities of the problem at hand.
The proposed gradient descent algorithm for solving (11) is provided as Algorithm 2. For each  $(\mathbf{U},\mathbf{V})\in \mathcal{U}\times \mathcal{V}$ , let  $\mathbf{G} := (\mathbf{G}^{\mathbf{u}},\mathbf{G}^{\mathbf{v}})$  with  $\mathbf{G}^{\mathbf{u}}\in \mathcal{T}_{\mathbf{U}}\mathcal{U}$  and  $\mathbf{G}^{\mathbf{v}}\in \mathcal{T}_{\mathbf{V}}\mathcal{V}$ . The iterates  $(\mathbf{G}_t^{\mathbf{u}},\mathbf{G}_t^{\mathbf{v}})$  are obtained by solving the following problem in the joint tangent plane  $\mathcal{T}_{\mathbf{U}}\mathcal{U}\times \mathcal{T}_{\mathbf{V}}\mathcal{V}$ :
$$
\min  _ {\mathbf {G} \in \mathcal {T} _ {\mathbf {U}} \mathcal {U} \times \mathcal {T} _ {\mathbf {V}} \mathcal {V}} q _ {t} (\mathbf {G}), \text {w h e r e} q _ {t} (\mathbf {G}) := \left\{\operatorname {t r a c e} \left(\mathbf {G} ^ {\top} \nabla f \left(\left(\mathbf {U} _ {t}, \mathbf {V} _ {t}\right)\right)\right) + \frac {1}{2} \| \mathbf {G} \| _ {\mathrm {F}} ^ {2} \right\}. \tag {12}
$$
The solutions  $(\mathbf{G}_t^{\mathbf{u}},\mathbf{G}_t^{\mathbf{v}})$  are maintained on the manifolds using the retraction operations  $R^{\mathbf{u}}$  and  $R^{\mathbf{v}}$ .
Assumption B. For a subset  $S \subseteq \mathcal{T}\mathcal{U} \times \mathcal{T}\mathcal{V}$ , there exists a constant  $L_{f}$  such that for all  $(\mathbf{Z},\mathbf{G}) \in S$ ,  $f(R^{\mathbf{z}}(\mathbf{G})) \leq \mathbf{F}(\mathbf{Z}) + \langle \nabla f(\mathbf{Z}),\mathbf{G}\rangle + (L_{f} / 2)\| \mathbf{G}\|_{\mathrm{F}}^{2}$ , with  $R^{\mathbf{z}}$  as the retraction.
Theorem 6. Suppose Assumption  $B$  holds. Let  $(\mathbf{U}_t,\mathbf{V}_t)$  be the sequence generated by SF-CCA. Let  $f^{*}\coloneqq \inf \{f(\mathbf{U},\mathbf{V}):\mathbf{(U,V)}\in \mathcal{U}\times \mathcal{V}\}$ . If  $\eta_t^{\mathbf{u}} = \eta_t^{\mathbf{v}} = \eta \leq 1 / L_f$  for all  $t\in [T]$ , then
$$
\min  \left\{\| \mathbf {G} _ {t} \| _ {\mathrm {F}}: t = 0, \dots , T - 1 \right\} \leq \frac {2}{\eta} \left[ \frac {f (\mathbf {U} _ {0} , \mathbf {V} _ {0}) - f ^ {*}}{T} \right] ^ {\frac {1}{2}}.
$$
Comparison between MF-CCA and SF-CCA: MF-CCA addresses conflicting objectives and achieves optimal trade-offs automatically, but it necessitates the inclusion of  $\binom{K}{2}$  additional objectives. SF-CCA, on the other hand, provides a simpler approach but requires tuning an extra hyperparameter  $\lambda$ . When choosing between the two methods, it is crucial to consider the trade-off between complexity and simplicity, as well as the number of objectives and the need for hyperparameter tuning.
In this section, we provide empirical results showcasing the efficacy of the proposed algorithms.
4.1 Evaluation Criteria and Selection of Tuning Parameter
F-CCA's performance is evaluated on correlation and fairness for each dimension of subspaces. Let  $\mathbf{U} = [\mathbf{u}_1,\dots ,\mathbf{u}_R]\in \mathbb{R}^{D_x\times R}$  and  $\mathbf{V} = [\mathbf{v}_1,\dots ,\mathbf{v}_R]\in \mathbb{R}^{D_y\times R}$ . The  $r$ -th canonical correlation is defined as follows:
$$
\rho_ {r} = \frac {\mathbf {u} _ {r} ^ {\top} \mathbf {X} ^ {\top} \mathbf {Y} \mathbf {v} _ {r}}{\sqrt {\mathbf {u} _ {r} ^ {\top} \mathbf {X} ^ {\top} \mathbf {X} \mathbf {u} _ {r} \mathbf {v} _ {r} ^ {\top} \mathbf {Y} ^ {\top} \mathbf {Y} \mathbf {v} _ {r}}}, \quad r = 1, \dots , R. \tag {13a}
$$
Next, in terms of fairness, we establish the following two key measures:
$$
\Delta_ {\max , r} = \max  _ {i, j \in [ K ]} | \mathcal {E} ^ {i} \left(\mathbf {u} _ {r}, \mathbf {v} _ {r}\right) - \mathcal {E} ^ {j} \left(\mathbf {u} _ {r}, \mathbf {v} _ {r}\right) |, \quad r = 1, \dots , R, \tag {13b}
$$
$$
\Delta_ {\operatorname {s u m}, r} = \sum_ {i, j \in [ K ]} | \mathcal {E} ^ {i} \left(\mathbf {u} _ {r}, \mathbf {v} _ {r}\right) - \mathcal {E} ^ {j} \left(\mathbf {u} _ {r}, \mathbf {v} _ {r}\right) |, \quad r = 1, \dots , R. \tag {13c}
$$
Here,  $\Delta_{\mathrm{max},r}$  measures maximum disparity error, while  $\Delta_{\mathrm{sum},r}$  represents aggregate disparity error. The aim is to reach  $\Delta_{\mathrm{max},r}$  and  $\Delta_{\mathrm{sum},r}$  of 0 without sacrificing correlation  $(\rho_r)$  compared to CCA. We conduct a detailed analysis using component-wise measurements (13) instead of matrix versions; for more discussions, see Appendix C.2.
The canoncorr function from MATLAB and [35] is used to solve (CCA). For MF-CCA and SF-CCA, the learning rate is searched on a grid in  $\{1e - 1,5e - 2,1e - 2,\dots ,1e - 5\}$ , and for SF-CCA,  $\lambda$  is searched on a grid in  $\{1e - 2,1e - 1,0.5,1,2,\dots ,10\}$ . Sensitivity analysis of  $\lambda$  is provided in Appendix B.2. The learning rate decreases with the square root of the iteration number. Termination of algorithms occurs when the descent direction norm is below  $1e - 4$ .
4.2 Dataset
4.2.1 Synthetic Data
Following [4, 44], our synthetic data are generated using the Gaussian distribution
$$
\left( \begin{array}{c} \mathbf {X} \\ \mathbf {Y} \end{array} \right) \sim N \left(\left[ \begin{array}{c} \mu_ {\mathbf {X}} \\ \mu_ {\mathbf {Y}} \end{array} \right], \left[ \begin{array}{c c} \boldsymbol {\Sigma} _ {\mathbf {X}} & \boldsymbol {\Sigma} _ {\mathbf {X Y}} \\ \boldsymbol {\Sigma} _ {\mathbf {Y X}} & \boldsymbol {\Sigma} _ {\mathbf {Y}} \end{array} \right]\right).
$$
Here,  $\mu_{\mathbf{X}} \in \mathbb{R}^{D_x \times 1}$  and  $\mu_{\mathbf{Y}} \in \mathbb{R}^{D_y \times 1}$  are the means of data matrices  $\mathbf{X}$  and  $\mathbf{Y}$ , respectively; covariance matrices  $\boldsymbol{\Sigma}_{\mathbf{X}}, \boldsymbol{\Sigma}_{\mathbf{Y}}$  and the cross-covariance matrix  $\boldsymbol{\Sigma}_{\mathbf{XY}}$  are constructed as follows. Given ground truth projection matrices  $\mathbf{U} \in \mathbb{R}^{D_x \times R}, \mathbf{V} \in \mathbb{R}^{D_y \times R}$  and canonical correlations  $\rho = (\rho_1, \rho_2, \ldots, \rho_R)$  defined in (13a). Let  $\mathbf{U} = \mathbf{Q}_{\mathbf{X}}\mathbf{R}_{\mathbf{X}}$  and  $\mathbf{V} = \mathbf{Q}_{\mathbf{Y}}\mathbf{R}_{\mathbf{Y}}$  be the QR decomposition of  $\mathbf{U}$  and  $\mathbf{V}$ , then we have
$$
\boldsymbol {\Sigma} _ {\mathbf {X Y}} = \boldsymbol {\Sigma} _ {\mathbf {X}} \mathbf {U} \operatorname {d i a g} (\boldsymbol {\rho}) \mathbf {V} ^ {\top} \boldsymbol {\Sigma} _ {\mathbf {Y}}, \tag {14a}
$$
$$
\boldsymbol {\Sigma} _ {\mathbf {X}} = \mathbf {Q} _ {\mathbf {X}} \mathbf {R} _ {\mathbf {X}} ^ {- \top} \mathbf {R} _ {\mathbf {X}} ^ {- 1} \mathbf {Q} _ {\mathbf {X}} ^ {\top} + \tau_ {x} \mathbf {T} _ {\mathbf {X}} \left(\mathbf {I} _ {D _ {x}} - \mathbf {Q} _ {\mathbf {X}} \mathbf {Q} _ {\mathbf {X}} ^ {\top}\right) \mathbf {T} _ {\mathbf {X}} ^ {\top}, \tag {14b}
$$
$$
\boldsymbol {\Sigma} _ {\mathbf {Y}} = \mathbf {Q} _ {\mathbf {Y}} \mathbf {R} _ {\mathbf {Y}} ^ {- \top} \mathbf {R} _ {\mathbf {Y}} ^ {- 1} \mathbf {Q} _ {\mathbf {Y}} ^ {\top} + \tau_ {y} \mathbf {T} _ {\mathbf {Y}} \left(\mathbf {I} _ {D _ {y}} - \mathbf {Q} _ {\mathbf {Y}} \mathbf {Q} _ {\mathbf {Y}} ^ {\top}\right) \mathbf {T} _ {\mathbf {Y}} ^ {\top}. \tag {14c}
$$
Here,  $\mathbf{T}_{\mathbf{X}} \in \mathbb{R}^{D_x \times D_x}$  and  $\mathbf{T}_{\mathbf{Y}} \in \mathbb{R}^{D_y \times D_y}$  are randomly generated by normal distributions, and  $\tau_x = 1$  and  $\tau_y = 0.001$  are scaling hyperparameters. For subgroup distinction, we added noise to canonical vectors and adjusted sample sizes: 300, 350, 400, 450, and 500 observations each. In the numerical experiment, different canonical correlations are assigned to each subgroup alongside two global canonical vectors  $\mathbf{U}$  and  $\mathbf{V}$  to generate five distinct subgroups.
4.2.2 Real Data
National Health and Nutrition Examination Survey (NHANES). We utilized the 2005-2006 subset of the NHANES database https://www.cdc.gov/nchs/nhanes, including physical measurements and self-reported questionnaires from participants. We partitioned the data into two distinct subsets: one with 96 phenotypic measures and the other with 55 environmental measures. Our objective was to apply F-CCA to explore the interplay between phenotypic and environmental factors in contributing to health outcomes, considering the impact of education. Thus, we segmented the dataset into three subgroups based on educational attainment (i.e., lower than high school, high school, higher than high school), with 2,495, 2,203, and 4,145 observations in each subgroup.
Table 1: Numerical results in terms of Correlation  $(\rho_r)$ , Maximum Disparity  $(\Delta_{\mathrm{max},r})$ , and Aggregate Disparity  $(\Delta_{\mathrm{sum},r})$  metrics. Best values are in bold, and second-best are underlined. We focus on the initial five projection dimensions, but present only two dimensions here; results for other dimensions are in the supplementary material. We put the results of other projection dimensions in the supplementary material. “↑” means the larger the better and “↓” means the smaller the better. Note that MHAAPS has only 3 features, so we report results for its 1 and 2 dimensions.
Dataset	Dim. (r)	ρr↑	Δmax,r↓	Δsum,r↓
CCA	MF-CCA	SF-CCA	CCA	MF-CCA	SF-CCA	CCA	MF-CCA	SF-CCA
Synthetic Data	2	0.7533	0.7475	0.7309	0.3555	0.2866	0.2241	3.3802	2.8119	2.2722
5	0.4717	0.4681	0.4581	0.4385	0.3313	0.2424	4.1649	3.1628	2.2304
NHANES	2	0.6392	0.6360	0.6334	0.0485	0.0359	0.0245	0.1941	0.1435	0.0980
5	0.4416	0.4393	0.4392	0.1001	0.0818	0.0824	0.4003	0.3272	0.3297
MHAAPS	1	0.4464	0.4451	0.4455	0.0093	0.0076	0.0044	0.0187	0.0152	0.0088
2	0.1534	0.1529	0.1526	0.0061	0.0038	0.0019	0.0122	0.0075	0.0039
ADNI	2	0.7778	0.7776	0.7753	0.0131	0.0119	0.0064	0.0263	0.0238	0.0127
5	0.6810	0.6798	0.6770	0.0477	0.0399	0.0324	0.0954	0.0799	0.0648
Table 2: Mean computation time in seconds (±std) of 10 repeated experiments for  $R = 5$  on the real dataset and  $R = 7$  on the synthetic dataset. Experiments are run on Intel(R) Xeon(R) CPU E5-2660.
Dataset	CCA	MF-CCA	SF-CCA
Synthetic Data	0.0239±0.0026	109.0693±5.5418	29.1387±2.0828
NHANES	0.0483±0.0059	42.3186±1.9045	14.9156±1.8941
MHAAPS	0.0021±0.0047	3.5235±2.0945	0.8238±0.8155
ADNI	0.0039±0.0032	2.7297±0.5136	1.8489±1.0519
Mental Health and Academic Performance Survey (MHAAPS). This dataset is available at https://github.com/marks/convert_to_csv/tree/master/sample_data. It consists of three psychological variables, four academic variables, as well as sex information for a cohort of 600 college freshmen (327 females and 273 males). The primary objective of this investigation revolves around examining the interrelationship between the psychological variables and academic indicators, with careful consideration given to the potential influence exerted by sex.
Alzheimer's Disease Neuroimaging Initiative (ADNI). We utilized AV45 (amyloid) and AV1451 (tau) positron emission tomography (PET) data from the ADNI database (http://adni.loni.usc.edu) [73, 74]. ADNI data are analyzed for fairness in medical imaging classification [41, 53, 81], and sex disparities in ADNI's CCA study can harm generalizability, validity, and intervention tailoring. We utilized F-CCA to account for sex differences. Our experiment links 52 AV45 and 52 AV1451 features in 496 subjects (255 females, 241 males).
In the simulation experiment, we follow the methodology described in Section 4.2.1 to generate two sets of variables, each containing two subgroups of equal size. Canonical weights are trained and used to project the two sets of variables into a 2-dimensional space using CCA, SF-CCA, and MF-CCA. From Figure 2, it is clear that the angle between the distributions of the two subgroups, as projected by SF-CCA and MF-CCA, is smaller in comparison. This result indicates that F-CCA has the ability to reduce the disparity between distinct subgroups.
Table 1 shows the quantitative performance of the three models: CCA, MF-CCA, and SF-CCA. They are evaluated based on  $\rho_r$ ,  $\Delta_{\mathrm{max},r}$ , and  $\Delta_{\mathrm{sum},r}$  defined in (13) across five experimental sets. Table 2 displays the mean runtime of each model. Several key observations emerge from the analysis. Firstly, MF-CCA and SF-CCA demonstrate substantial improvements in fairness compared to CCA. However, it is important to note that F-CCA, employed in both MF-CCA and SF-CCA, compromises some degree of correlation due to its focus on fairness considerations during computations. Secondly,
(a) XU of CCA
(b) XU of MF-CCA
(c) XU of SF-CCA
(d) YV of CCA
(e) YV of MF-CCA
(f) YV of SF-CCA
Figure 2: Scatter plot of the synthetic data points after projected to the 2-dimensional space. The distributions of the two groups after projection by CCA are orthogonal to each other. Our SF-CCA and MF-CCA can make the distributions of the two groups close to each other.
(b) NHANES
(a) Synthetic Data
(c) MHAAPS
(d) ADNI
Figure 3: Aggregate disparity of CCA, MF-CCA, and SF-CCA (results from Table 1).
SF-CCA outperforms MF-CCA in terms of fairness improvement, although it sacrifices correlation. This highlights the effectiveness of the single-objective optimization approach in SF-CCA. Moreover, the datasets consist of varying subgroup quantities (5, 3, 2, and 2) and an imbalanced number of samples in distinct subgroups. F-CCA consistently performs well across these datasets, confirming its inherent scalability. Lastly, although SF-CCA requires more effort to tune hyperparameters, SF-CCA still exhibits a notable advantage in terms of time complexity compared to MF-CCA, demonstrating computational efficiency. Disparities among various CCA methods are visually represented in Figure 3. Notably, the conventional CCA consistently demonstrates the highest disparity error. Conversely, SF-CCA and MF-CCA consistently outperform CCA across all datasets, underscoring their efficacy in promoting fairness within analytical frameworks.
In Table 1, we define the percentage change of correlation  $(\rho_r)$ , maximum disparity gap  $(\Delta_{\mathrm{max},r})$ , and aggregate disparity  $(\Delta_{\mathrm{sum},r})$ , respectively, as follows:  $P\rho_r := (\rho_r$  of F-CCA  $-\rho_r$  of CCA) /  $(\rho_r$  of CCA)  $\times 100$ ,  $P\Delta_{\mathrm{max},r} := -(\Delta_{\mathrm{max},r}$  of F-CCA  $-\Delta_{\mathrm{max},r}$  of CCA) /  $(\Delta_{\mathrm{max},r}$  of CCA)  $\times 100$ , and  $P\Delta_{\mathrm{sum},r} := -(\Delta_{\mathrm{sum},r}$  of F-CCA  $-\Delta_{\mathrm{sum},r}$  of CCA) /  $(\Delta_{\mathrm{sum},r}$  of CCA)  $\times 100$ . Here, F-CCA is replaced with either MF-CCA or
(a) Synthetic Data
(c) MHAAPS
(d) ADNI
Figure 4: Percentage change from CCA to F-CCA (results from Table 1). Each dataset panel shows two cases with projection dimensions  $(r)$ .  $P\rho_{r}$  is slight, while  $P\Delta_{\max,r}$  and  $P\Delta_{\mathrm{sum},r}$  changes are substantial, signifying fairness improvement without significant accuracy sacrifice.
SF-CCA to obtain the percentage change for MF-CCA or SF-CCA. Figure 4 illustrates the percentage changes of each dataset.  $P\rho_{r}$  is slight, while  $P\Delta_{\max,r}$  and  $P\Delta_{\mathrm{sum},r}$  changes are substantial, signifying fairness improvement without significant accuracy sacrifice.
5 Conclusion, Limitations, and Future Directions
We propose F-CCA, a novel framework to mitigate unfairness in CCA. F-CCA aims to rectify the bias of CCA by learning global projection matrices from the entire dataset, concurrently guaranteeing that these matrices generate correlation levels akin to group-specific projection matrices. Experiments show that F-CCA is effective in reducing correlation disparity error without sacrificing much correlation. We discuss potential extensions and future problems stemming from our work.
- While F-CCA effectively reduces unfairness while maintaining CCA model accuracy, its potential to achieve a minimum achievable disparity correlation remains unexplored. A theoretical exploration of this aspect could provide valuable insights.
- F-CCA holds promise for extensions to diverse domains, including multiple modalities [80], deep CCA [3], tensor CCA [44], and sparse CCA [25]. However, these extensions necessitate novel formulations and in-depth analysis.
- Our approach of multi-objective optimization on smooth manifolds may find relevance in other problems, such as fair PCA [55]. Further, bilevel optimization approaches [37, 68, 65] can be designed on a smooth manifold to learn a single Pareto-efficient solution and provide an automatic trade-off between accuracy and fairness.
- With applications encompassing clustering, classification, and manifold learning, F-CCA ensures fairness when employing CCA techniques for these downstream tasks. It can also be jointly analyzed with fair clustering [15, 66, 34] and fair classification [78, 18].
This work was supported in part by the NIH grants U01 AG066833, U01 AG068057, RF1 AG063481, R01 LM013463, P30 AG073105, and U01 CA274576, and the NSF grant IIS 1837964. The ADNI data were obtained from the Alzheimer's Disease Neuroimaging Initiative database (https://adni.loni.usc.edu), funded by NIH U01 AG024904. Moreover, the NHANES data were sourced from the NHANES database (https://www.cdc.gov/nchs/nhanes).
We appreciate the reviewers' valuable feedback, which significantly improved this paper.
Roadmap. The appendix is organized as follows: In Section A, we present the subproblem solvers for MF-CCA and SF-CCA, along with the proofs for Theorems 5 and 6. In Section B, we provide comprehensive insights into the three real datasets utilized and present supplementary experimental results. Furthermore, Section C outlines the experimental specifics and the procedure for selecting hyperparameters.
A Addendum to Section 3 16
A.1 Preliminaries on Retractions 16
A.2 Subproblem Solver for MF-CCA 16
A.3 Subproblem Solver for SF-CCA 17
A.4 Auxiliary Lemmas 18
A.5 Proof of Theorem 5 18
A.6 Proof of Theorem 6 19
B Addendum to Section 4 20
B.1 Detailed Description of Datasets 20
B.1.1 National Health and Nutrition Examination Survey (NHANES) 20
B.1.2 Mental Health and Academic Performance Survey (MHAAPS) 21
B.1.3 Alzheimer's Disease Neuroimaging Initiative (ADNI) 21
B.2 Trade-off Analysis of Correlation and Disparity 22
B.2.1 Sensitivity of Correlation and Disparity Error to  $K$  22
B.2.2 Sensitivity of Correlation and Disparity Error to  $r$  22
B.2.3 Sensitivity of Correlation and Disparity of SF-CCA to  $\lambda$  24
B.3 Runtime Sensitivity of MF-CCA and SF-CCA 25
B.3.1 Runtime Sensitivity of MF-CCA and SF-CCA to  $d$  25
B.3.2 Runtime Sensitivity of MF-CCA and SF-CCA to  $N$  25
B.3.3 Runtime Sensitivity of MF-CCA and SF-CCA to  $K$  26
C Experimental Details and Hyperparameter Selection Procedure 26
C.1 Hyperparameters Selection 27
C.2 Fairness and Correlation Measures 28
A Addendum to Section 3
A.1 Preliminaries on Retractions
In reference to Definition 1, the following options together with generalized polar decomposition defined in (4) represent commonly employed retractions[1, 6, 7, 21, 23, 38, 75] of a matrix  $\pmb{\xi} \in \mathbb{R}^{D \times R}$ :
- Exponential mapping. It takes  $8DR^{2} + \mathcal{O}(R^{3})$  flops and has the closed-form expression:
$$
R _ {\exp} ^ {\mathbf {z}} (\boldsymbol {\xi})   =   [ \mathbf {Z} \quad \mathbf {Q} ] \exp \left(\left[ \begin{array}{c c} - \mathbf {Z} ^ {\top} \boldsymbol {\xi} & - \mathbf {R} ^ {\top} \\ \mathbf {R} & 0 \end{array} \right]\right) \left[ \begin{array}{c} \mathbf {I} _ {R} \\ 0 \end{array} \right],
$$
where  $\mathbf{QR} = -(\mathbf{I}_r - \mathbf{Z}\mathbf{Z}^\top)\boldsymbol{\xi}$  is the unique QR factorization.
- Polar decomposition. It takes  $3DR^{2} + \mathcal{O}(R^{3})$  flops and has the closed-form expression:
$$
R _ {\text {p o l a r}} ^ {\mathbf {z}} (\boldsymbol {\xi}) = \left(\mathbf {Z} + \boldsymbol {\xi}\right) \left(\mathbf {I} _ {R} + \boldsymbol {\xi} ^ {\top} \boldsymbol {\xi}\right) ^ {- 1 / 2}.
$$
- QR decomposition. It takes  $2DR^{2} + \mathcal{O}(R^{3})$  flops and has the closed-form expression:
$$
R _ {\mathrm {q r}} ^ {\mathbf {z}} (\boldsymbol {\xi}) = \operatorname {q r} (\mathbf {Z} + \boldsymbol {\xi}),
$$
where  $\mathrm{qr}(\mathbf{A})$  is the Q factor of the QR factorization of  $\mathbf{A}$ .
- Cayley transformation. It takes  $7DR^{2} + \mathcal{O}(R^{3})$  flops and has the closed-form expression:
$$
R _ {\text {c a y l e y}} ^ {\mathbf {Z}} (\boldsymbol {\xi}) = \left(\mathbf {I} _ {R} - \frac {1}{2} \mathbf {W} (\boldsymbol {\xi})\right) ^ {- 1} \left(\mathbf {I} _ {R} + \frac {1}{2} \mathbf {W} (\boldsymbol {\xi})\right) \mathbf {Z},
$$
where  $\mathbf{W}(\pmb {\xi}) = (\mathbf{I}_R - \mathbf{Z}\mathbf{Z}^\top /2)\pmb {\xi}\mathbf{Z}^\top -\mathbf{Z}\pmb{\xi}^\top (\mathbf{I}_R - \mathbf{Z}\mathbf{Z}^\top /2)$
This work specifically focuses on generalized polar decomposition defined in (4) for retraction.
A.2 Subproblem Solver for MF-CCA