SHGR: A Generalized Maximal Correlation Coefficient
Authors: Samuel Stocksieker
Denys Pommeret
Venue: NeurIPS 2025 poster ()
Keywords: Maximal Correlation
HGR
Non Linear Correlation
Neural Network
Primary area: general_machine_learning
TL;DR: We introduce GHGR, a robust neural estimator of the Hirschfeld-Gebelein-Rényi (HGR) maximal correlation, which efficiently detects nonlinear dependencies, satisfies Rényi's axioms, and outperforms state-of-the-art methods.

Abstract: Traditional correlation measures, such as Pearson’s and Spearman’s coefficients, are limited in their ability to capture complex relationships, particularly nonlinear and multivariate dependencies. The Hirschfeld–Gebelein–Rényi (HGR) maximal correlation offers a powerful alternative by measuring the highest Pearson correlation achievable through nonlinear transformations of two random variables. However, estimating the HGR coefficient remains challenging due to the complexity of optimizing arbitrary nonlinear functions. We introduce a new coefficient, satisfying Rényi's axioms, based on the extension of HGR with Spearman's rank correlation: the Spearman HGR (SHGR). We propose a neural network-based estimator tailored to estimate (i) the bivariate correlation matrix, (ii) the multivariate correlations between a set of variables and another one, and (iii) the full correlation between two sets of variables. This estimate effectively detects nonlinear dependencies and demonstrates robustness to noise, outliers, and spurious correlations (hallucinations). Additionally, it achieves competitive computational efficiency through designed neural architectures. Comprehensive numerical experiments and feature selection tasks confirm that SHGR outperforms existing state-of-the-art methods.

SHGR: A Generalized Maximal Correlation Coefficient
Samuel Stocksieker CNRS, 12M Aix Marseille University Marseille, France
Denys Pommeret CNRS, 12M Aix Marseille University Marseille, France
Abstract
Traditional correlation measures, such as Pearson's 's and Spearman's coefficients, are limited in their ability to capture complex relationships, particularly nonlinear and Rényi (HGR) maximal corre-
multivariate dependencies. The lation offers powerful alternative by measuring the highest Pearson correlation achievable through nonlinear transformations of two random variables. However, estimating the HGR coefficient remains challenging due to the complexity of opti- mizing arbitrary nonlinear functions. We introduce a new coefficient, satisfying Rényi's axioms, based on the extension of HGR with Spearman's rank correlation: the Spearman HGR (SHGR). We propose a neural network- based estimator tailored estimate (i) the bivariate correlation matrix, (ii) the multivariate correlations between set of variables and another one, and (iii) the full correlation between of variables. This estimate effectively detects nonlinear dependencies and demonstrates robustness to noise, outliers, and spurious correlations (halluci- nations). Additionally, achieves competitive computational efficiency through designed neural architectures. Comprehensive numerical experiments and feature selection tasks confirm that SHGR outperforms existing stale-of-the-art methods.
1 Introduction
Understanding variable dependencies is a central challenge in machine learning, statistics, and data science, with critical applications including feature selection, dimensionality reduction, fairness assessment, causal inference, and multimodal learning. Classical measures, especially Pearson's 's and Spearman's correlations, are widely used but limited to linear, monotonic, and bivariate rela- tionships, often failing to capture complex multivariate or higher-order dependencies. To address these limitations, some generalized dependence measures have been proposed. Among them, the Hirschfeld-GGebeliin Gebelein-Rényi (HGR) maximal correlation stands out a theoretically principled tool for quantifying nonlinear dependence between random variables, whether univariate or multivariate. Introduced by Hirschfeld (2I), extended by Gebelein [10, and formalized by Rényi [37]. HGR defines correlation as the maximum linear correlation between transformed versions of the variables. Despite its strong theoretical appeal, estimating the HGR coefficient remains challenging due to the complexity of identifying optimal transformations. Existing alternative methods suffer from similar drawbacks, including limited interpretability. high computational complexity, difficulty in extending to multivariate configurations.
In this paper, introduce extension of the Hirschfeld-Gehelede Deeilld--Geeeei--nnnnn Rényi (HGR) maximal correlation that we shall call Spearman HGR (SHGR). is based on Spearman instead of the Pearson coefficient. SHGR offers scalable and flexible way estimate nonlinear dependencies in both bivariate and multivariate settings: by estimating pairwise, multivariate (set-to-target), and groupwise correlation matrices. This extension builds on two key components: (i) neural approximation tailored for simultaneous estimation of correlations, and (ii) a copula-based formulation that operates on ranks, ensuring invariance to monotonic transformations while improving robustness and stability against noise and extreme values. Our main contributions are summarized as follows:
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

We revisit HGR maximal correlation by proposing Spearman based extension: SHGR
We present first estimator of SHGR that is differentiable, fast, efficient, and robust to noise, outliers, and spurious dependencies. Unlike some existing methods, it recovers the optimal nonlinear transformations and enables formal significance testing.
We introduce a stacked cross-encoder architecture specifically designed to estimate multiple correlations simultaneously in both bivariate, multivariate, and groupwise contexts.
We establish comprehensive evaluation protocol for assessing maximal correlation estima- tors in terms of performance, robustness to noise, hallucinations (respecting Rényi's Axiom 4; see Appendix A), extreme values, as well as the estimation of bivariate, multivariate, and full correlations, significance testing, and computational efficiency.
We validate SHGR on synthetic and real-world tabular datasets. We demonstrate that SHGR outperforms existing state-of-the-arr methods in terms of performance and robustness.
The remainder of the paper is organized as follows: Section 2 reviews prior work on maximal correlation. Section introduces SHGR, defining the proposed coefficient and its neural estimator based on stacked encoders. Section 4 presents comprehensive empirical evaluation, comparing SHGR to tfate-of-the-arr methods, especially in terms of detection power and robustness. Section S shows the application of SHGR to feature selection tasks on real datasets. Dectionfoccoclueee with discussion of the strengths and limitations of our approach and future directions. Code and data are available at: httpp:: //gtthub..t com/ http://stocsiiekeeeeeeee
2 Background
2.1 Related Work
Numerous nonlinear dependence measures have been proposed to extend classical coefficients such as Pearson's 's r. Kendall's r, and Spearman's Canonical Correlation Analysis (CCA) 23 is
foundational method for extracting linear dependencies between sets of variables, but its linearity
limits its capacity to capture complex relationships. Its extensions (142 49 include Kernel CCA
26 20), which applies CCA to nonlinear feature spaces, and Deep CCA (DCCA) 45 2. which uses deep networks to model nonlinear projections. The Randomized Dependence Coefficient (RDC)
31 offers a flexible and efficient alternative using random nonlinear projections followed by CCA.
Rényi (HGR) maximal correlation, originally developed by 21 [0 37), aims to quantify dependence via maximal Pearson correlation after applying measurable transformations. HGR has been widely studied in machine learning. notably through the Alternating Conditional Expectations (ACE) algorithm (6). which itcratively estimates transformations via conditional regressions. While partially interpretable, ACE is sensitive noise and the quality of the estimators 23 Recently, HGRRbased methods have been applied in Fairness-aware learning
In contrast, the Hirschfeld-Gebelee
36
33 16 13), multimodal learning 33 44) regression [47. linear-time independence criterion
a generalized ACE for unlabeled data 46 or in feature selection [24]. Hellinger Correlation is recent alternative, though limited two variables. Other notable dependence measures include Distance Correlation (dCor) or Brownian Correlation 288. the Hilbert-SShmidt Independence
48
Criterion (HSIC and CHSIC) [19 18). and the Maximum Mean Discrepancy (MMD) 17. These kernel-based methods, widely used in hypothesis testing, capture a broad range of dependencies, but are sensitive kernel choice and do not yield interpretable scores in [0, 1]. The Mutual Information Coefficient (MIC) (38] can detect various functional relationships but lacks axiomatic properties like those of HGR and is sensitive to binning heuristics. GeDI [14, recently proposed for fairness applications, offers an interpretable. transparent measure. MMD and its copula- based variant CMMD [35], as well as GeDI, are not included our evaluation as they yield scale-free measures not confined to [0, 1). limiting interpretability and comparability. Table in Appendix B compares the different methods, theoretically and empirically.
2.2 Hirshfeldd-Gebellin--iie Correlation coefficient
The MirschfelddGehhelen--Reenn maximum correlation (HGR) a statistical measure theoretically designed to quantify the relationship between two (univariate or multivariate) random variables. Unlike the standard correlation coefficients, the HGR is capable of detecting both linear and nonlinear associations. It is computed the maximum correlation coefficient obtained after transforming the

---

variables using measurable functions. This makes the HGR particularly useful in situations where the relationship between variables is complex. The HGR coefficient has the advantage of being between O (independence) and (strong correlation). As defined by Rényi ((77), the maximal coefficient presents interesting properties, described in Appendix A While the HGR correlation coefficient holds great potential in theory, its estimation can be quite challenging due to the limitless possibilities for transformations. Moreover, the relationships between variables can be highly complex and require substantial computational resources, making its application difficult.
DEFINITION 1 (HGR maximal correlation coefficient). Let U (~ Du) and V (~ Dv) be two continuous random variables taking values in U and V. respectively. Let E(U) (resp. E(V)) denote the set of measurable functions from 4 (resp. V) to R. Let r() denote the Pearson correlation coefficient, The
Renyi (HGR) maximal correlation coefficient is defined as
follows ((21) 10 37
HGR(U.V) = sup ((()))))
sup JUCERA).yEEE
EUEER) fVEE(V)
2.3
Neural HGR
In the context of fairness analysis, 15 proposed estimating the HGR transformations using neural networks to capture nonlinear relationships. This estimation be formulated general optimiza- tion problem, where the goal is to map two random variables into a space where their linear correlation maximized. The algorithm takes u, a sample of U, and v,a sample of V, inputs, and returns as output the estimated Pearson correlation ((f.(u). fo (v)), where fu and fo are parameterized by a compact domain O This measure is then estimated using a neural network by minimizing the following loss function:
LNHGr= = - sup r(Ja, (w), fa. (u)),
where Jo. (resp. fo.) denotes neural estimator of Su (resp. f.). This estimator of the maximal correlation coefficient HGR(u, v), denoted NHGR (for Neural-HGR) is defined as:
NHGRo(u, v) r(fū, (u), (u)), with (fr.; fo.)) = arg max r r(fo, (u), fo. fo,. Eo
This method enables the approximation of the maximal correlation between two variables, or between one variable and a set of variables (15), 31). Nevertheless, by construction, this approach is based on the Pearson correlation coefficient and therefore inherits its known limitations.
3 SHGR: A Robust and Efficient Maximal Correlation Coefficient
3.1 Neural Spearman HGR
The HGR coefficient is a valuable tool for identifying potential correlations but may. in practice, exhibit two key limitations similar to those of Pearson's correlation: sensitivity to extreme values, and (ii) restricted to capturing linear dependencies of transformed variables (illustration in Appendix C. In practice, the constraints of centering and unit variance are insufficient exclude outiers, which could bias the HGR estimator completely. Using neural networks could lead to overfitting behaviors, such as learning extreme values to inflate the correlation score artificially. The sensitivity of neural HGR estimators to outliers can lead to overfitting behaviors and artificially inflated correlation scores. To address these limitations, we propose a natural extension of the HGR coefficient. Our approach is inspired by the Spearman coefficient and copula-based methods, which rely on cumulative distribution functions and are inherently robust to both extreme values and monotonic transformations. This substitution offers two main advantages: (i) it improves robustness to extreme values and distributional shifts, and (ii) it extends the scope of measurable dependence to include monotonic but nonlinear relationships. Importantly, it remains consistent with the HGR principle of maximizing correlation between nonlinear transformations the variables, but reframes the notion of dependence in terms of rank-based monotonicity rather than raw linearity. This could help reduce transformation efforts and optimize calibration to avoid seeking linear correlation, just monotonic correlation. Recall here
Its estimate is also designated by simplify notation.
3

---

that the ith rank statistic from n-sample, (21, 2n), drawn from a random univariate variable Z, given by:
rank(z;) = Σ 12yS = (x))) with Fz(2)) = Σ 3=1
where F= denotes the empirical cumulative distribution function. By extension, the rank a random vector yields the vector of all ranks. We recall that the Spearman correlation coefficient between two iid paired samples u = (u1,". Un) and v = (U1,-.. ".,))) Un) from U and V, denoted by p(u, v). is defined as the Pearson correlation coefficient based on the ranks:
p(u,v) (nF (u)((t))) - (Fr with Pv(u) (Frrlu),, (un)))
DEFINITION (Spearman-HGF (SHGR) coefficient). Let U and be two paired continuous random variables taking values in U and V, respectively. Let E(L) (resp. E(V)) denote the set of measurable functions from U (resp. V) to The Spearman-HGR (SHGR) coefficient associated to (U, V) is defined by
SHGR(U,V) =
r ((l,)))( (. L((()E)LL
max A.EE(U).,
The SHGR is related the notion of grade correlation between two random variables U and V (see [12), which is the limit of their Spearman coefficients and is defined as the correlation between their copula transformations:
(U.V) ((F))))(( = lim E(p(u, v)),
where vu, = (u, 02) u.) are n-iid paired samples from (U,V). So, we have:
SHGR(U, V) E.EE) ((),
important to note that the copula transformation preserves the dependence between the original vectors U and V (see for instance, (34).
Using the empirical estimator F of F, we obtain an estimator of the SHGR and its Neural version:
SHGR=
r
max F.EER). =L T
(H,
r(F (1) (fo, (u)), (v))). max p(fo. (u), [o)( fo, se'
fon, Ee
Note that the rank transformation is applied only the correlation computation, not the input variables themselves. This preserves the information of the original inputs while benefiting from the robustness of rank-based objectives.
Consistency of SHGR estimates
SHGR is empirically motivated as rank-based reformulation of the HGR coefficient ensuring ro- bustness outliers and extreme values. theoretically relies on copula transformations preserve dependency structures. We now prove consistency guarantees through convergence results and Rényi's axioms. The proofs are given in Appendix[])
PROPOSITION 1. For all 0, there exists family of continuous neural networks parametrized by a compact domain O such that
[SHGRo(u,)) - SHGR(u, S
PROPOSITION 2. Let (u, be independent sequences of iid samples of size drawn by (U, V). We have the following convergence in law as n tends to infinity:
SHGR(u, v) -SHGRUU V).

---

PROPOSITION 3. Let U, V be two continuous random variables. All the following Rényi axioms for nonlinear dependence measures (modified by 44 are satisfied by the SHGR coefficient.
1. SHGR(U, V) is defined for all pairs of non-constant continuous random variables U, V.
2. SHGR(U, V)= = SHGR(V,U).
3. o SHGR(U, V) S
4. SHGR(U V) = 0 if and only U and V are independent.
5. For all Borel-measurable bijective functions f RP - and g RR - R, SHGR(U,V) SHGR(J(U),((V)
6. SHGR(U,V) = U f(V) V g(U), for some Borsl-measurable function g.
7. If VU. ~ (N, 2) then SHGR(U, V) is strictly increasing function of |r(U, V)J.
3.2 Generalized Neural SHGR
While pairwise correlation analysis useful, many real-world applications require assessing depen- dencies between group of variables and one or more target variables. In similar manner to the Canonical Correlation Analysis (CCA) and the Randomized Dependence Coefficient (RDC), we propose here a natural generalization of our rank-based HGR estimator to this multivariate setting. For instance, it is often necessary to analyze several pairwise correlations rather than focusing on a single variable pair. Since training neural networks can be time-consuming, estimating each correlation independently may be inefficient. We address this limitation by introducing a neural architecture specifically designed for this purpose. To construct a matrix of SHGR estimates, we extend the neural SHGR estimator using a stacked cross cross-encoder design. This architecture is illustrated in Figure Ia for pairwise (1-vs-1 1) analysis and in Figure Tb] for full (p-vs-q) correlation matrices. An architecture for multivariate (p-vs-) 1) settings is provided in Appendix (Figure 9.. Typically, for bivariate estimation, the algorithm considers p input variables u1.-..: Upp and aims optimize all pairwise entries in the p x p correlation matrix. The stacked encoder design enables marginal transformations that capture complex, nonlinear dependencies between variables. The objective function for bivariate estimations (pairwise correlations) is defined as follows:
LSHGR(U) = Σ 02 [ (us); fow, (u,) 1/21
P
(1or,
with > O.
i/==1
The objective function for multivariate analysis (correlations of any order) is thus:
LSHGR(U) = Σ o² (Fo., (ux), 90.,, (fu, (1()))))) 1/770 with go.. : RP-1
We also extend this approach obtain full correlation between two sets of variables u and v of dimensions p and respectively (for example, two datasets):
LSHGR(U, u, v) == [p² (fe. (u), fe, (2)) with fe, RR + R, fe, R9 R.
Similar to the absolute value, we apply a square root transformation of the squared coefficient to obtain differentiable operation. We introduce hyperparameter a that allows weighting the coefficients: weight greater than gives more importance to stronger correlations, and their maximization will be prioritized during training, enabling faster maximization of potential correlations. It interesting to note that the Canonical Correlation Analysis (CCA) is special case of SHGR, corresponding to single linear layer without activation and rank transformation.
Practical Remarks (1) Since our loss function depends on rank- estimation, we use TorchSort
approach 44) to obtain a differentiable approximation of the sorting operation (2) Empirically,
SHGR exhibits convergence, with no substantial gain beyond a threshold (illustrated in Figure 12a in
We used the implementation available htt.ail:atttttsssgttttbb... ailttpes//ggtaboooo/////aaattttt
5

---

Alo
Eosrs (PUL (o2
(a) for bivariate correlations estimation
(b) for full correlation estimation
Figure Cross-Encoder Architecture
Appendix F.3.4), motivating the use of an early stopping to balance accuracy and efficiency. (3) Our estimator supports significance testing via a Spearman- test. To improve robustness, we apply masking rule: correlations not significant with the test set to zero. (4) An extended version of SHGR handles mixed-type data by incorporating the correlation ratio (numerii-to-caategorica) and Cramér's V (categorical-orcategorraa))) (illustration in Appendix F.3.4..
4
Experiments
propose to evaluate the SHGR and its competitors with a complete protocol following the analysis
below (defining the multivariate power of correlation measures): (42.1l Performance: ability to capture complex nonlinear correlations without noise (4.2.2) Robustness to noise: ability to identify
complex nonlinear correlations with noise; (42.3) Robustness hallucination (spurious correlations):
null correlation in the case of independence; ; (4.2.4) Robustness to extreme values: null correlation in the case of independence, in the presence of extreme values; (4.2.6) Bivariate power of a dependence measure as proposed by [31] (4.2.7 Computation time: fast to estimate correlations ; (4.2.5) Significance test analysis: possibility of performing significance test the coefficient and ability to reject the null hypothesis (of null correlation) in the presence of noise.
4.1 Experimental Protocol
We apply the multivariate power of correlation measures protocol in three settings: (i) bivariate (pairwise) correlations (1-vs-1). 1). (ii) multivariate correlations (p-vs-1), and (iii) full (groupwise) correlations (p-vs-q). For (i), generate six independent Gaussian variables and five others that depend nonlinearly on them. For (ii), we simulate 20 variables, including nonlinear dependencies involving than two variables. For (iii), generate datasets with varying global correlation structures. Details on data generation are provided in Appendix F3 Figure 2a presents the pairplot for bivariate correlations. We estimate nonlinear correlations using SHGR and compare them with the following alternative methods [ (some methods are unavailable in the multivariate and full settings):
The Pearson correlation coefficient: Pearson
The Spearman correlation coefficient: Spearman
The Kendall correlation coefficient: Kendall
The Randomized Dependence Coefficient (31): RDC
The Mutual Information Criterion ((88)) MIC
The Distance Correlation [1 dCor
Canonical Correlation Analysis 23 CCA
Kernel Canonical Correlation Analysis (2)): &CCA
Alternating Conditional Expectations (17): ACE
HGR estimation with kemel (33) HGRkde
HGR estimation with Neural Net (16)): HGRnn
HGR estimation with Lattice (14)) HGRlat
HGR estimation with double kernel (14 [14): dk
'The used librarics are listed Appendix F.4 'This approach was not integrated because unstable and too time-consuming calculate

---

HGR estimation with simple kernel ((14) ): sk
The Normalized Hilbert-Schmine Schhmdd Independence Criterion (188 NHSIC
The Hellinger Correlation ((II):: H1R
We perform our SHGR model for 100, 200, and 500 epochs and with early stopping. A sensitivity analysis of the SHGR architecture and hyperparameters was performed (details provided in Appendix F22.). A single architecture and hyperparameter configuration were then used consistently across all illustrations and experiments. In the analyses below, generate K 10 synthetic datasets and compare the estimated correlations from each method to the true, known, correlations, referred to reference correlations. We evaluate the deviation from these reference correlations using: the pairwise correlation matrix for bivariate correlation, the vector multiple correlations for multivariate correlation, and the correlation coefficient for full correlation analysis.
(c) Reference Correlation Matrix
00000 8 001 0 n M
o 001
1003
0010 0910 0
(a) Inputs Pairplot
(b) SHGR Transformations Pairplot
(d) SHGR Correlation Matrix
Figure 2: Illustration of SHGR on bivariate correlations
4.2
Results
4.2.1
Performance
Our goal this experiment is to evaluate the ability of the different approaches to detect perfect nonlinear correlations in noise-free setting. Figure shows the results for both bivariate and multivariate settings. We observe that SHGR closely matches the reference correlation values and performs par with the strong baseline ACE. Morcover,
find that varying the number of training epochs for SHGR (early stopping, 100, 200, or 500 epochs) has little to no effect on performance, suggesting that the method converges efficiently. In the full-correlation setting, SHCR, together with RDC, CCA, dk, and sk, successfully captures nonlinear dependencies, whereas other approaches such HGRnn, NHSIC, and dCor fail to do (see Figure in Appendix F.5.3). Further implementation details and numerical results are provided in Appendix
evere pot soord
saice deanton
thsRt jonel
GooR 200 stans
Ric
Bist 13
(a) Bivariate Correlation
(b) Multivariate Correlation
Figure Performance Results: distance to Reference Correlations (lower better)
TThis approach not integrated because inefficient, limited to pairwise correlations and tine-consuming

---

4.2.2 Robustness to Noise
We now evaluate the robustness of correlation measures with respect additive white noise (zero mean), assessing their ability to detect significant dependencies as noise increases. To this end, we reproduce the previous simulations while gradually introducing higher levels of noise in each trial. In this setting, the true correlation is unknown. Following the approach of [31], we consider a method more robust it yields higher correlation values under increasing noise. We therefore adopt the same evaluation metric as in the previous section: the distance to the reference correlations. Results are shown Figure 4 detailed experimental settings and additional visualizations are given in Appendix F.6 As noise increases (left to right on the x-axis), SHCR consistently outperforms state-of catte-ofthee- --tth-rrt baselines across bivariate, multivariate, and foll-correlation scenarios. Notably, while ACE performs well in noise-free settings, its accuracy drops significantly under noise. In the full-correlation setting, some methods even exhibit hallucinated correlations when applied independent variables, which the case when the value of k on the x-axis exceeds 10.
(a) Bivariate Correlation
(b) Multivariate Correlation
(c) Full Correlation
Figure 4: Robustness to Noise (a) and (b): distance to Reference Correlations (lower is better): (c): Decreasing correlation context on the left side (higher better) and independence context on the right side (lower is better))- Results with SHGR are red
4.2.3 Robustness to Hallucination
We evaluate here the ability of correlation measures to avoid ballucinations, that is, refrain from reporting spurious correlations on fully independent data. For each scenario, we simulate independent variables and compute the corresponding estimated correlation coefficients. As illustrated in Figure 5a our method, SHGR, remains stable and does not produce artificial correlations, unlike several competing approaches that yield inflated values. Additional details and experimental results provided in Appendix F7
4.2.4 Robustness to Extreme Values
We evaluate the robustness of correlation measures hallucinated dependencies induced by extreme values. Following the previous setup, we simulate independent variables and inject extreme values in all iterations except the first, which serves as a baseline reference. We then apply various correlation estimators to assess their stability under such perturbations. As shown in Figure 5b] SHGR remains stable in the presence of extreme values, while several competing methods, including HGRnn, exhibit inflated or unstable correlation estimates. Additional results and implementation details are provided in Appendix F88
Heternnc
(a) Robustness to hallucination
(b) Robustness to Extreme Values
Figure 5: Analysis of the distances the Bivariate Reference Correlation Matrix (lower is better)

---

4.2.5 Significance Test
SHGR optimizes the Pearson correlation coefficient over nonlinear transformations of the input variables, in line with the theoretical definition of HGR. As such, it is possible to perform an asymptotic significance test the obtained correlation coefficient (when is sufficiently large or by applying a Gaussian quantile transformation, that is monotonic) test can also be applied. Specifically, we the null and alternative hypotheses:
[29]. A non parametrii permutation
Ho SHGRe(u,
versus H1 SHGRa(u, v) +
We assess the effectiveness of this test across various bivariate and multivariate relationships, each per- turbed with increasing levels of noise. The results are compared to the significance tests implemented for MIC, RDC, and dCor, over nine representative types of dependencies. Full details are provided in Appendix F.9 Overall, the SHGR-based test consistently outperforms the alternatives, demonstrating greater robustness and higher ability correctly reject the null hypothesis of independence.
4.2.6 Bivariate Power of Dependence Measure
In addition to the previous evaluations, assess the bivariate power of dependence measure. Following the protocol introduced [3] and adopted in subsequent works such as [32] and (15), we evaluate the robustness of our method by progressively perturbing several nonlinear bivariate and multivariate relationships. An ideal dependence measure should maintain high correlation values noise increases. Detailed results are provided in Appendix [F10 Overall, SHGR demonstrates strong robustness, consistently producing higher correlation scores than competing methods across all tested configurations.
4.2.7 Computation time
SHGR exhibits highly competitive computation times. Unlike many existing approaches, it avoids iterative loops for estimating multiple correlations, instead performing joint optimization over all coefficients, which significantly reduces computational overhead. In contrast, baseline methods often rely on sequential estimation, leading to increased runtimes. On typical illustrative dataset (2,000 observations and 10 variables). SHGR computes the full pairwise correlation matrix in approximately seconds with early stopping. It consistently outperforms NHSIC, MIC, dk, and HGRlat for bivariate corrllttions. In the multivariate setting, its runtime is par with RDC and CCA. We also evaluate the method's 's scalability with respect to the number of variables and sample size. SHGR remains efficient in -igh-dimnniional settings: for instance, it requires only 3.5 minutes to estimate multivariate correlations on a dataset with 10,000 observations, faster than both RDC and CCA, and just I minute to compute bivariate correlations across 1,000 variables. Further computational details provided Figure presents the computation time for illustration. In the case of bivariate
in Appendix E.11
Computition Time
Compustion Time
suoct d8 count
gooaie und (owec)
gecien pur poo tounty
(oun))
BaSaR ES (uus)
dieat 280 caumd
Haltee
Mabed
Metood
(a) Bivariate correlations
(b) Multivariate correlations
Figure 6: Computational time Comparison
correlation estimates, i.e., the correlation matrix, the computation time of SHGR, with early stopping. is around 6 seconds average. This is faster than MIC, HGRlat, dk and NHSIC. In the case of multivariate correlation estimates, the computation time of SHGR is 23 seconds on average and is slightly faster than the CCA, RDC, and dCor methods (33 seconds). The dk method takes far too long, with almost 14 minutes. The ACE method is instantancous.

---

5 Real-World Applications
To evaluate the practical relevance of our method, we apply it to feature selection on nine real- woal-world tabular datasets (see Appendix G). For each method, we select the top & features most correlated with the target y. and assess predictive performance via RMSE on test set (30%), using a random forest regressor. To test robustness to limited data, all models are trained on at most 500 samples. Some results are shown in Figure Across most datasets, SHGR-selected features consistently yield lower RMSE than competing methods. We also evaluate the multivariate SHGR using leave-one-out strategy to estimate each variable's 's contribution to the global X X-vs-y correlation. Complete results reported in Appendix G confirming the effecttivnness of SHGR for real-world feature selection.
(a) Abalone dataset
(b) Appliance dataset
(c) Boston dataset
Figure 7: RMSE for test set prediction with feature selection on real-world datasets (lower is better); Based on maximal (bivariate) correlation Results with SHCR are red
6 Discussion
The problem of estimating the maximal correlation between two variables has long been a central challenge in statistics and machine learning. Several approaches have been proposed, most of which are based on the Hirschffld-Gebeleede Gebelein-éény: (HGR) definition, where the goal is to learn transformations of the input variables that maximize their linear correlation. In this work, introduce a new maximal correlation coefficient, SHGR, which generalizes the HGR framework using rank-based transformations and satisfies Rényi's s axioms. Its estimation relies dedicated neural architecture. Our approach offers several key advantages: (1) it directly optimizes a full matrix of correlation coefficients in non-iterative manner, significantly reducing computation time; (2) the estimator is accurate, robust, and computationally efficient; (3) it enables statistical significance testing of the estimated correlations: (4) outperforms state-of-the-ard methods across multiple evaluation settings and real-world experiments; Interestingly, our approach can be combined with existing methods to improve their results, as shown in Appendix [.14 and, (5) SHGR is fully differentiable, making it easy to integrate into modern deep learning pipelines. As highlighted by [13), JIGR-baser methods may produce "hallucinated" correlations as a result of overfitting. In our illustrations, we showed that SHGR remains robust to such artifacts. Nevertheless, caution is still required; therefore, it important to carefully tune the stacked neural network our approach, sufficiently expressive to capture complex dependencies, but not overly flexible to prevent overfitting and artificially inflated correlation scores. Our experimental design includes a variety of nonlinear dependencies, inspired by the Power of Dependence Measure protocol suggested by (31). We assessed performance in both noise-free and noisy settings.
Further exploration of other types of nonlinear dependencies would be a valuable extension. [13 also points out that HGR is difficult to interpret. Although this criticism is generally valid, SHGR provides access to the learned transformations of the input variables, which can provide information about the nature of the dependency. Indeed, the SHGR approach allows for graphical analysis of the function
applied transformations by plotting the transformations of stacked neural networks of the inputs, as illustrated in Figure 37 Nevertheless, interpretability is not the main objective of HGR-based measures. Exploring the interpretability of learned functions is interesting avenue, and a posteriori visualizations could help to better understand the adjusted transformations and therefore the dependencies. Although detaching normalization terms reduces theoretical gradient bias, we found that doing so significantly harmed empirical performance, We thus keep the differentiable form and leave bias-corrected alternatives for future work. Analyzing its behavior in high-dimensioonl settings remains an open question. Extending the framework to large-scale neural architectures and multimodal learning tasks, as well as applying fairness-awe aware scenarios, represents promising directions for future research.
10

---

Acknowledgment
The authors would like to express their sincere gratitude to the reviewers for their helpful comments, which have contributed improving the manuscript. S. Stocksieker would like acknowledge the support of the Research Chair DIALog under the aegis of the Risk Foundation, a joint initiative by CNP Assurance. D. Pommeret would like to acknowledge the support received from the Research Chair ACTIONS under the aegis of the Risk Foundation, an initiative by BNP Paribas Cardif and the Institute of Actuaries of France.
References
[1] Galen Andrew, Raman Arora, Jeff Bilmes, and Karen Livescu. Deep canonical correlation analysis. In International conference machine learning, pages 1247-1255. PMLR, 2013.
[2] Francis R Bach and Michael I Jordan. Kernel independent component analysis. Journal of machine learning research, 3(Jul):1-44 2002.
[3] Yves Ismaël Ngounou Bakam and Denys Pommeret. Smooth test for equality of copulas. Electronic Journal of Statistics, 18(1):895 - 941, 2024.
[4] Mathieu Blondel, Olivier Teboul, Quentin Berthet, and Josip Djolonga. Fast differentiable sorting and ranking. In International Conference Machine Learning, pages 950-959. PMLR, 2020.
[5] Paula Branco, Luis Torgo, and Rita P Ribeiro. Pre-processing approaches for imbalanced distributions in regression. Neurocomputiing, 343:76-99 096776999600 2019.
[6] L. Breiman and J.H. Friedman. Estimating optimal transformations for multiple regression and correlation. Journal of the American Statistical Association, page 580-619, 1985.
[7] Leo Breiman and Jerome H Friedman. Estimating optimal transformations for multiple re- gression and correlation. Journal of the American statistical Association, 80(391):580-59) 1985.
[8] Luis Candanedo. Appliances Energy Prediction. UCI Machine Learning Repository, 2017. DOI: http://doi.oggll.2333//////
[9] Zhiwen Chen, Siwen Mo, Haobin Ke, Steven X Ding, Zhaohui Jiang, Chunhua Yang, and Wei- hua Gui. Canonical correlation guided deep neural network. arXiv preprint arXiv:2409. 19396, 2024.
[10] Hans Gebelein. Das statistische problem der korrelation als variations- und eigenwertprob- lem und sein zusammenhang mit der ausgleichsrechnung Zuim-reitschrifi Fur Angewandle Mathematik Und Mechanik, 21:364-379, 1941.
[11] Gery Geenens and Pierre Lafaye de Micheaux. The hellinger correlation. Journal of the American Statistical Association, 117(538):639-655 2022.
[12] Jean Dickinson Gibbons and Subhabrata Chakraborti. Nonparametric statistical inference: revised and expanded. CRC press, 2014.
[13] Luca Giuliani, Eleonora Misino, and Michele Lombardi. Generalized disparate impact for configurable fairness solutions in ML. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 1444-11458. PMLR, 23-29 Jul 2023.
[14] Luca Giuliani, Eleonora Misino, and Michele Lombardi. Generalized disparate impact for configurable fairness solutions in In International Conference on Machine Learning, pages 1 11443-11458. PMLR, 2023.

---

[15] Vincent Grari, Oualid EI Hajouji. Sylvain Lamprier, and Marein Detyniecki. Learning unbiased representations via rényi minimization. In Nuria Oliver, Fernando Pérez-Cruz, Stefan Kramer, Jesse Read, and Jose A. Lozano, editors, Machine Learning and Knowledge Discovery in Databases. Research Track, pages 749-764, Cham, 2021, Springer International Publishing.
[16] Vincent Grari, Boris Ruf, Sylvain Lamprier, and Marcin Detyniecki. Fairness-aware neural N'eyni minimization for continuous features. arXiv preprint arKiv:1911.049900 2019.
[17] Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. The Journal of Machine Learning Research, 13(1):723-773, 2012.
[18) Arthur Gretton, Kenji Fukumizu, Choon Teo, Le Song, Bernhard Schölkopf, and Alex Smola. kernel statistical test of independence. Advances in neural information processing systems, 20, 2007.
[19] Arthur Gretton, Ralf Herbrich, Alexander Smola, Olivier Bousquet, and Bernhard Schölkopf. Kernel methods for measuring independence. J. Mach. Learn. Res., 6:2075- 2129,, December 2005.
[20] David Roi Hardoon and John Shawe-Taylor. Convergence analysis of kernel canonical correla- tion analysis: theory and practice. Machine Learning, 74:23-38, 2008
[21] H. O. Hirschfeld. A connection between correlation and contingency. Mathematical Proceedings of the Cambridge Philosophical Society, 51(4):520-522 1935.
[22] K. Hornik, M. Stinchcombe, and H. White. Multilayer feedforward networks are universal approximators. Neural Netw. ((5):359-366, July 1989.
[23] Harold Hotelling. Relations between two sets of variates. In Breakthroughs in statistics: methodology and distribution, pages 162-190. Springer, 1992.
[24] Shao-Lun Huang, Anuran Makur, Lizhong Zheng, and Gregory W Wornell. An information- theoretic approach to universal feature selection in high-dimensional inference. In 2017 IEEE International Symposium on Information Theory (ISIT), pages 1336-1340. IEEE, 2017.
[25] Shao-Lun Huang and Xiangxiang Xu. On the sample complexity of hgr maximal correlation functions for large datasets. IEEE Transactions Information Theory, 6753)1151-19, 2020.
[26] Su-Yun Huang, Mci-Hsien Lee, and Chuhsing Kate Hsiao. Kernel canonical correlation analysis and its applications to nonlinear measures of association and test of independence. Institute of Statistical Science: Academia Sinica, Taiwan, 2006.
[27] Henrik Hult and Filip Lindskog. Multivariate extremes, aggregation and dependence in elliptical distributions. Advances in Applied Probability, 34(3)-587-608, 2002.
[28] Michael R. Kosorok. Discussion of: Brownian distance covariance. The Annals of Applied Statistics, 3(4): 1278, 2009.
[29] Charles Kowalski. On the effects of non-normality on the distribution of the sample product- moment correlation coefficient. Journal of the Royal Statistical Society: Series C (Applied Statistics), 1121 1972.
[30] Yihua Liang, Fei Ma, Yang Li, and Shao-I un Huang. Person recognition with hgr maximal correlation on multimodal data. In 2020 25th International Conference on Pattern Recognition (ICPR), pages 1-8, 2021.
[31] David Lopez-Paz. Philipp Hennig, and Bernhard Schölkopf. The randomized dependence coefficient. Advances in neural information processing systems, 26, 2013.
[32] Jérémie Mary, Clément Calauzenes, and Noureddine El Karoui. Fairness-aware learning for continuous attributes and treatments. International conference on machine learning, pages 4382-4391. PMLR, 2019,
12

---

[33] Jeremie Mary, Clément Calauzénes, and Noureddine El Karoui. sermees--waace learning for continuous attributes and treatments. In Kamalika Chaudhuri and Ruslan Salakhutdinov. editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 1382243911 PMIR, 09-15 Jun 2019.
[34] Roger B. Nelsen. An introduction to copulas. Springer, New York, 2006.
[35] Barnabás Péczos, Zoubin Ghahramani, and Jeff Schneider. Copula-based kernel dependency measures. arXiv preprint arXiv: arXiv: /206.4682, 2012.
[36] Meisam Razaviyayn, Farzan Farnia, and David Tse. Discrete rényi classifiers. In C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc., 2015.
[37] Alfréd Rényi. On measures of dependence. Acta Mathematica Academiae Scientiarum Hungar- ica, 10:441-451, 1959.
[38] David N. Reshef, Yakir A. Reshef, Hilary K. Finucane, Sharon R. Grossman, Gilean Mcan Peter J. Turnbaugh, Eric S. Lander, Michael Mitzenmacher, and Pardis C. Sabeti. Detecting novel associations in large data sets. Science, 334466662::518-11222 2011.
[39] Pablo Repository, 2014. DOI:
and Francisco Zamoraaaarriiine,,,,,, SML2010. UCI Machine Learning
[40] B. Schweizer and E. F Wolff. On nonparametric measures of dependence for random variables. Ann. Statist., 91): 885, 1981.
[41] Gábor J Székely, Maria L Rizzo, and Nail Bakirov. Measuring and testing dependence by correlation of distances. 2007.
[42] Viivi Uurtio, João M Monteiro, Jaz Kandola. John Taylor, Delmiro Fernandez-Reyer and Juho Rousu. A tutorial on canonical correlation methods. ACM Computing Surveys (CSUR), 50(6):1-3,, 2017.
[43] Saverio Vito. Air Quality. https//ttb.or//102244////KK
UCI
Machine
Learning Repository, 2008.
DOI:
[44] Lichen Wang, Jiaxiang Wu, Shao-Lun Huang, Lizhong Zheng, Xiangxiang Xu, Lin Zhang, and Junzhou Huang. An efficient approach to informative feature extraction from multimodal data. 19/EA/99/EAAAT 19. AAAI Press, 2019.
[45] Weiran Wang, Raman Arora, Karen Livescu, and Jeff A Bilmes. Unsupervised learning of acoustic features via deep canonical correlation analysis. In 20/5 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 4590-4594 IEEE, 2015.
[46] Xiangxiang Xu and Shao-Lun Huang. On the asymptotic sample complexity of hgr maximal correlation functions in semi-supervised learning. In 2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pages 879-886. IEEE, 2019.
[47] Xiangxiang Xu and Shao-Lun Huang. Maximal correlation regression. IEEE Access, 8:26591- 26601, 2020.
[48] Longfei Yan, W Bastiaan Kleijn, and Thushara Abhayapala. A linear-time independence criterion based on finite basis approximation. In International Conference on Artificial Intelligence and Statistics, pages 202-212. PMLR, 2020.
[49] Xinghao Yang, Weifeng Liu, Wei Liu, and Dacheng Tao. A survey on canonical correlation analysis. IEEE Transactions Knowledge and Data Engineering, (3(6))2349-233) 2021.
[50] I-Cheng Yeh. Concrete Compressive Strength. UCI Machine Learning Repository, 1998. DOI: http:://ooo.or//1122442/////
13

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No]. or [NA].
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will published with the paper.
The reviewers of your paper will be asked use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] providedd proper justification is given (e.g.. "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
"[No] or "[NA]" is not grounds for rejection. While the questions are phrased acknowledge that the true answer is often more nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Contributions are listed in the introduction and summarized in the abstract. made in the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Limitations are presented in the discussion section
14

---

Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: All demonstrations of the proposals in the paper are rigorously detailed in the Appendix, with references where necessary. Unfortunately, due to space constraints, unable to give a short proof sketch.
Guidelines:
The answer NA means that the paper does include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
The code and datasets are not provided submission time but will be released upon acceptance, along with the camera-ready version. The synthetic data generation process is fully detailed, and the sources of the real-world datasets used in our experiments
Justification:
15

---

are explicitly cited. Additionally, we provide a pseudo-code description of the algorithm and illustrate the construction of the stacked cross-encoder architectures for all three correlation settings.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No] / [Yes]]
Justification: The code and datasets are not provided at submission time but will be released in case of acceptance, along with the camera-ready version. The synthetic data generation process fully detailed, and the sources of the real-world datasets used in our experiments are explicitly cited. Additionally, provide a pscudo-code description of the algorithm and illustrate the construction the stacked cross-encoder architectures for all three correlation settings.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new opun-s benchmark).
16

---

reproducible, they The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
stiips.cc/pullic/uudeee
setpssummssssssiiiiii osiliiicccc for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: All hyperparameters documented in the Appendix, including network architectures, training settings, and data simulation parameters. We also conduct sensitivity analyses to assess the robustness of method with respect these choices.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: For each analysis, we performed at least 10 randomized runs (with fixed seeds for reproducibility) ensure robust and reliable results. We report outcomes using various formats: heatmaps of correlation scores, ranking tables, and visualizations such as boxplots or iefiddenceinneeree curves, depending on the context. Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
17

---

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Computational ressources and computation time are detailed in the Appendix. NOus avons également réalisés une étude de sensibilités de temps de caleul par rapport taille de I' I'échantillon et nombre de vairales.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics httppp////ooorooscccoo//////hhht
Answer: [Yes]
Justification: the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Proposition of a new maximal correlation coefficient, without societal impact.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinfornation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
18

---

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e-g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All assets (code, data, and models) are properly referenced, with their original papers. The datasets are sourced from public repositories such as the UCI Machine Learning Repository, with URLs provided. The models used in the baselines are implemented using publicly available or Python libraries, retrieved from open-access GitHub repositories (with URL). Licenses are indicated wherever applicable.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paweessipppweiiiia....... thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
19

---

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: details about training, license, limitations, etc. will be communicated in case of acceptance
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/model as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact core methodology, scientific rigorousness, originality of the research, declaration is not required.
20

---

Answer: [NA]
Justification: The use of large language models (LLMs) was limited strietly to writing assistance, editing, and formatting. No LLMs were used for developing core research ideas, designing experiments, or generating results.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non sta-standard components.
Please refer to our LLM policy hatp:p/:/hearrrssccc/////oeerree /LLM) for what should or should not be described.
21

---

A Renyi's axioms
1. General definition: D(X,Y, Y) is defined for all pairs of non-constant random variables X and Y.
2. Symmetry: D(X, X)
3. Natural bounds: - D) I
4. Zero under independence: D(X, = 0 if and only if X and Y are statistically independent.
5. Invariance under bijective transformations: For all Borcl-measurable bijective functions
f.g: : + IR, D(X,Y) -
6. Unit value under deterministic functional dependence: D(X,Y) = 1f f(X) or
g(Y), for some Borel-measee measurbeaee function f or g-
7. Agreement with Pearson correlation the Gaussian case: If (X,Y) ~ NN4, E), then D(X,Y)= |p(X,Y)
8. where p(X,Y, Y) denotes the Pearson correlation coefficient.
B Related Work
Table 1: Comparison of correlation estimation approaches across multiple criteria (*** for good score, for bad score, based bivariate illustrations)
Approach
Non-Lin.
Mult.
Perf.
Noise
Halluc.
Extreme
Rényi
Marg. Inv.
Pearson
X
X
Spearman
X
X
X
Kendall
X
X
X
RDC
#38
d
MIC
488
dCor
CCA
X
X
ACCA
X A
ACE
X
49
600
600
HGRkde
X
X
40
HGRM
HGRlar
X
X
dk
48
#88
48
388
SHGR (ours)
S8*
C Pearson coefficient Analysis
The following figure illustrates the differences between Pearson and Spearman correlation coefficient estimates. In Figure 8b) the relationship is purely monotonic with the exponential function and the Pearson coefficient poorly captures the correlation. distorted if an extreme value is present (Figure (Fi. Finally, both coefficients do not capture nonlinear relationships, such quadratic relationship, as shown Figure &c
also observed that the Pearson coefficient
Sensitivity of Pearson's correlation to extreme values We demonstrate the sensitivity of Pearson's correlation coefficient to extreme values using simple counterexample. Let X and Y be two independent, centered random variables, such that E[X] = E[Y] and Cov(X, Y) an i.i.d. sample of n pairs (I; 3/ > drawn from these variables, with empirical Pearson correlation coefficient Pn, close
Consider
due to independence. Now suppose we add a single extreme data point (x" yy" = (K, K) with K > 1. Let 5.+1 and Dn+1 denote the empirical means computed over the n + observations (including the extreme point). The updated empirical Pearson correlation becomes:
22

---

> xi= (I; - (y - In+1) + (K - In+1)(K - (m+1))
Pa+1
/ Di Inti)² + (K Bnt1)2 12 v / (Vi - Dnt1)² + (K In+1)
Now observe that:
Int1
Σ i=1
2j K
= n
K
n+1
so that:
K
nK
nK En+1)² ~
K - En+1
K
K
n+1 = (K
++11 = O
n1 +
and similarly for Y.
This shows that both the covariance term and the variances are asymptotically dominated by the contribution of the extreme point. Therefore, as K - 00:
(K In+1)(( Dn+1))
Pa+1
V(K 1n v(K -in+1)
This confirms that a single well-aligned extreme point can drive the Pearson correlation arbitrarily close to I, even when the underlying variables are entirely independent. This illustrates the high sensitivity of Pearson's correlation coefficient to outliers and highlights the potential for misleading empirical estimates in the presence of extreme values.
r == 0.003 rho = 0.005
r = 0.356 rho = 1.0
(a) Independence
(b) Monotonic correlation
r = -0.007 rho -0.011
= 0.908 rho -0.005
(c) Quadratic correlation
(d) Independence with an extreme value
Figure 8: Pearson (r) and Spearman (rho) coefficients for different cases
D Proofs
In the proofs, when there is no ambiguity, we simplify notation by writing place of sus 9 place of go. F in place of Fy or F),(U), G' in place of Gy or Gg.(V), depending of the situation.
23

---

D.1 Proof of Proposition
We have
SHGR - SHGRe
sup (up((()),(()) sup pl (u),
f.a
1'
where 1",9 9 stand for the neural approximation functions.
Let 5 > By definition of the supremum, there exist functions for 90 in (()) and (VV) such that:
p(Jo(u))o(v)
sup ((()()())()))) f.
Therefore:
-IGR-SHGRR S p(fo(u). sup g(f +
6" S (o(fo(u).g(v))) (() 96 g6))))++ +
(1)
= =r(F (fo())) (9o - r(( r(F (w) (u)), (o(o)) + +,
where fö and 96 stand for the neural approximations of fo and go respectively. Since the random prove by the
variables U and V are bounded on the compact supports {(4)) and E(V)), universality of the neural approximation [22] that when n tends to infinity,
r( fo (w) (u)): go (x))) r (
(o)
0.
The argument such a proof similar to those used in 15 Finally, since the inequality I holds for every O, obtain the result.
D.2 Proof of Proposition 2
We have
SHGR - SHGR
(pp(P(((()) -npr(((((())))((((()))
f.9
f.9
Let { > By definition of the supremum, there exist functions fo.90 in El4) and E(V) such that:
((oo())))GGG(())))
supr((((()))((o(o))) E -
5.9
Therefore:
SHGR- - SHGR = (((ol())) - -uDp(FF((())))))))))
f.9
s ((( (rFd))))))=(((()))))C))))))))))
(2)
(F +
A +
where, for abbreviation, (x) (F(x) UF) uF)/OF denotes the normalized version (centered and reduced) of F. can decompose
Σ A
Σ
1=1
B;
+
Σ
C.
1=1
24

---

We have IF' ((ool)))(( - F² (foo)))))
(folu)))) - (fo(u.)) sup C (v.))
[[G(oo(v=))- - (g0o)))))) FF
(fo(w)))
Since G' and Ft are bounded, by the Sereenko-Canellly Theorem obtain the uniform convergence: sup B; + 0, as n. tends to infinity and by the Cesàro Lemma we deduce that
B
0.
Since the C are iid centered random variables, for = 1,...........................1 n, by the law of large numbers, we have almost surely
C + 0,
and finally
A 0.
1=1
Since inequality 2 holds for every
0, we have:
SHGR- SHGR + 0,
tends
D.3
Proof of Proposition 3
Points 1-3 are immediate.
Point 4. If and V are independent, we obviously have SHGR 0. Conversely, let U and V be two continuous dependent random variable. implies that Cv.V the copula associated (U, V) not the independent copula. By Proposition I in [3] their exists non null copula coefficient associated to Cu. that is there exists two functions f and such that p E(J(U)((V)) = o Write X [(U), Y vVV). We get
(x,
+ O,
where fx,y denotes the joint density (X, Y). Without loss of generality, we can assume 6 0. Then there exists set S such that for all x, y E S, fx. (z. > 0 and Fx (x) > 0, Fy (1) O. It implics that
((Fx(X)B<())
(x 9ddddy
2 | ( y1d > 0.
which gives the result.
Point 5 follows from the construction of the SHGR.
Point 6. U f(V), choosing fa identity and fo f. we obtain
By construction of the SHGR, by normalizing with a function g such that E(g(U)) 0 and E(g²(U)) we obtain the result.
(F),
Point 7. In the Gaussian case we have the following relation (see for instance 27
6
Ps Xy)
arcsin
Pss
which yields the result.
25

---

fi(u1)
ú1
02
Φ1 g1(u2,
0g
0
f2(uz)
U2O
012
E. (F((((()))9((()))))))))
U3O
4/1
g2(u1, us)
U3
fa(ua)
10
161
g3(U1, u2)
0/2
Figure 9: A Cross-Encoder Architecture for multivariate case (illustration with p variables)
E SHGR Algorithm
In addition figures la and 1b figure shows example of architecture set of variables.
Based on neural networks, the SHGR algorithm defined two stages: one for building the architec- ture and the other for training the model. At each epoch, the correlation is measured on the inputs (in whole), and the model retains the model with the lowest loss (i.e. the highest correlation) on the inputs. If the results no longer improve (to within an epsilon) during given number of iterations, then learning stops.
Input:
Input matrices X, [Y], number of epochs E, batch size B, learning rate hidden layer dimensions dimHL. L, early stopping threshold E, maximum patience Pmaxx correlation type (Spearman or Pearson)
Output: Trained model M encoded outputs, loss history
Initialize model M with input dimensions of X [and Y], hidden layer dimension dim DI; Initialize optimizer (AdamW) with learning rate 77,
Convert X [and Y];
for epoch = 1 to E do
Shuffle X [and Y] and create mini-batches;
foreach butch (zal, yol] do
Compute encoded representation using M M
Compute loss
If Spearman: use + SHGR rank correlation yol)
If Pearson: use L SHGR linear conrelation(2:]110
Perform backpropagation and update model weights;
Record loss:
end
Compute full-batch loss on YX as Lval;
Apply early stopping if Tova does not improve beyond e for Pauax epochs; If L.val improves, save current model as best model M*
end
Compute encoded outputs using best model M all inputs; return M encoded outputs, loss history, best epoch Algorithm 1: train_SHGaR SHGR: Training of the SHGR model
Remark: as is classic with neural networks, it's customary to apply a standard scaler to the input data. Even though the algorithm also works with raw data, we noticed an improvement when using a standard scaler. The encoded is then standardized, allowing the HGR constraint to be respected: the transformations are indeed of zero expectation and unit variance. Examples of possible architecture are defined below in the illustration.
26

---

Experiments Details
Computational resources
The computations were performed personal desktop computer with the following specifications: NVIDIA GeForce RTX 4080 graphics card, 64GB memory (but the memory usage did not exceed 30GB), Intel 19-14900KF processor.
F.2 Model Architecture and hyperparameters
F.2.1
Hyperparameter
For the illustration, we have chosen the following hyperparameters:
epoch number: 200 maximum
batch size: 64
learning rate: 10e -3
hidden layer dimensions: [64, 32, 16, 8]
epsilon for early stopping 0.5
iteration max for patience early stopping: 20
penalization for differentiable ranks (as defined in 4
a power parameter SHGR loss function: 2.0
F.2.2 Architectures
The architecture of our SHGR for bivariate (pairwise) correlations between variables u1, Up is defined as follows:
an encoder for each with / = 1, consisting of hidden Layers:
Hj of dimensions (1, 64)
H L2 of dimensions (64, 32)
H L3 of dimensions (32, 16)
HL of dimensions (16, 8)
HH of dimensions (8, 1)
activation function are all Tanh() expect the first one that is Relu.
The architecture of our SHGR for multivariate correlations between variables un, Up is defined as follows:
an encoder for w; with = 1,... , consisting of 5 hidden Layers:
H of dimensions (1, 64)
H L2 of dimensions (64, 32)
L3 of dimensions (32, 16)
H4 of dimensions (16, 8)
HH of dimensions (8, 1)
encoder for {u,} # with j = 1, 1..., consisting of 5 hidden Layers:
H of dimensions (p 1 1, 64)
HL2 of dimensions (64,
HTg of dimensions (32, 16)
L4 of dimensions (16, 8)
HTs of dimensions (8, 1)
activation function are all Tanh() expect the first one that is Relu.
27

---

The architecture of our SHGR for full correlations between u (of dimensions p) and (of dimensions q) is defined follows:
encoder for consisting of 5 hidden Layers:
H1 of dimensions (p, 64)
H L2 of dimensions (64, 32)
H 2I of dimensions (32, 16)
H of dimensions (16, 8)
Ls of dimensions (8,
encoder for consisting of 5 hidden Layers:
H1 of dimensions (g, 64)
H2 of dimensions (64, 32)
H Hs of dimensions (32, 16)
HT of dimensions (16,8)
Lo of dimensions (8, 1)
activation function are all Tanh() expect the first one that Relu.
F.2.3
Sensibilities
Even though this criterion has been analyzed, as the calculation time is very satisfactory, does not come into play in the following evaluation. Our aim here is rather to analyze and optimize the architecture and hyperparameters the reference correlation. Here we use the simulations shown in the numerical illustration and the bivariate correlations.. For the architecture, we tested the following parameters:
the estimated correlation, which should as close as possible to
batch size: 32, 64 and 128
learning rate: -0e-2, 2 10e-3 and 10c-1
hidden layer dimensions: [5,5,5, 5). [10, 10, 10, 10). [128, 64, 32, 16]
[64, 32, 16, 8] and
2.10
2.15
2.10
2.05
2.10
2:5
180
1.96
1.90
32
120
1.50
0.001
0.01
(a) Batch size and hidden layer di- mensions
(c) learing and hidden layer di- mensions
(b) Batch size and learning rate
Figure 10: Sensitivities of architecture and hyperparameter of cross encoders
Concerning the learning rate, except with batch size of 128, which seems too high to capture correlations on a dataset, using 10e seems preferable as it reduces the distance to the reference correlation (Figures 10b and 10cl Concerning batch size, using 64 seems more stable than 32 and 128 (Figures T0a and 10b). even if 32 seems slightly better median. Finally, for the dimensions of hidden layers, 3, 12,,,,,6,,8 seems more stable than the others. (Figures [nd Toc..
We also analyzed the sensitivity and impact of the results to the hyperparameter power the loss function and to the differentiable rank regularization parameter as defined in [4] Figure [] presents results obtained. We can clearly see benefits of using a power of for correlations. This allows the networks to quickly focus the strongest correlations. With regard to differentiable rank regularization, the values 0.1, 0.5 and seem fairly comparable. We choose to use the default value of 1.
28

---

(h) Distance reference correlation matrix (focus) cording to power (a) regularization differen- tiable ranks
(a) Distance reference correlation matrix according to power (a) and regularization of differentiable ranks
Figure 11: Sensitivities of power and regularization of differentiable ranks; with bivariate correlations
F33 Synthetic Dataset Generation
F.3.1 Bivariate correlations
We generate synthetic dataset composed of samples and 11 input features (Xo:--.. X10). where the first 5 variables are drawn independently from a standard normal distribution. The remaining features nonlinearly correlated with the first ones. For performance analysis, generate noise- free data (s 0). For robustness analysis, the s noise increases progressively. Specifically, the data generation process is follows:
Xo to X4 A(0, 1) independently.
X5 (X 3s): cubic relationship with Xo plus Gaussian noise,
Xg (((sin(2:)) s/5): sinusoidal transformation of X1 plus noise,
Xy is discretized, piecewise-constant function of X2, mapped into 7 quantile bins, and then perturbed with /O, s/10) noise,
Xs -N(exp(xs). s): exponential transformation of X3 with additive noise.
X9 ~ ~N(X3, s): squared transformation of X4 with noise,
X,0 ~ NO 2 squared standard normal variable, i.e., x1())ddistributee.
Figure 12 gives an illustration of correlations with the pairplot (2a). the SHGR transformation (with 2b) and loss functions. Figure compare the reference correlation matrix and estimation with SHGR.
SHGR Loss Function on batch
(b) SHGR Function input
Figure 12: SHGR training: loss function for bivariate correlations
29

---

F.3.2 Multivariate correlations
We generate synthetic dataset composed of n samples and 20 input features (Xo..... X19). where the first 11 variables are drawn independently from a standard normal distribution. The remaining features include nonlinear and additive combinations, introducing structured dependencies. For performance analysis, we generate noise-free data (s 0). For robustness analysis, the a noise increases progressively. Specifically, the data generation process is follows:
Xo to X10 ~N(0,) 1) (independent standard normal variables).
X11 X3 + X3 + S11,
. X12 3X,, + 2X, + 0.9X2 +
X13 sin(X;) - 0.5.x + 613.
X14 to X17 ~ ~N(0011 1) (independent noise),
Xis Zis with Zis ~ (/,0,)) 1).
X9 Zig with Z19 N/ 1),
where each E; s²) is an optional noise term added introduce variability. The seed is randomly initialized unless otherwise specified. Figure 13 gives an illustration for multivariate correlations.
tns Mith unoanaardes
Mmn
(a) SHGR Loss Function on batch
(b) SHGR Loss Function input
(c) Reference Correlation
0.95
0.90
0.09
0.41
0.59
(d) SHGR Correlation
Figure 13: Multivariate correlations: reference vs SHGR
E.3.3 Full correlations
We generate synthetic dataset consisting of samples and p input variables (X1y.... Xp). The data constructed as a weighted combination of two independent standard Gaussian matrices, controlled by a mixing parameter a [0, 1]. Specifically, we first generate base matrix Z NO Ip) of size n x then add a perturbation to create X as follows:
X = a- 2 (1-
where Z' is an independent Gaussian matrix of same dimension. When a = data purely Gaussian with no noise: when 0, the data is entirely random. This allows control over the signal-to-noise ratio in the generated features. The final output is returned as a labeled DataFrame with columns (X))
Xp).
30

---

F33.4 Mixed bivariate correlations
A mixed-data version was designed to handle both quantitative and categorical variables within dataset. Specifically, the correlation ratio used to assess correlations between numerical and categorical variables, while measures such as Cramér's V are employed for correlations between categorical variables. By way of illustration, we generate dataset with mixed correlations (between numeric, between numeric and categorical, and between categorical). We then apply the SHGR approach, applicable mixed data, and obtain the matrix below, comparison matrix. Figure 14 gives an illustration of the mixed correlation matrix estimation.
true (reference)
D O 9 Y
MO O
(a) Reference mixed bivariate correlations
(b) SHGR mixed bivariate correlations
Figure 14: Application of SHGR on mixed data
F,4 State-of-the-Art Methods
We estimate nonlinear correlations using SHGR and compare them with alternative methods; note that some baselines unavailable in the multivariate and full cases.
The Pearson correlation coefficient from python library numpy: Pearson
The Spearman correlation coefficient from python library numpy: Spearman
The Kendall correlation coefficient from python library numpy: Kendall
The Randomized Dependence Coefficient
311 from R library AlterCorr: RDC
The Mutual Information Criterion (38) from R library AlterCorr. MIC
The Distance Correlation
41 from R library AlterCorr: dCor
Canonical Correlation Analysis from library stats: CCA
Kernel Canonical Correlation Analysis ((2)) from R library kernlab: kCCA [but not inte-
grated because unstable and too time-consuming to calculate]
Alternating Conditional Expectations
(77) from library acepack: ACE
HGR estimation with kernel (133) from python package maxcorr and from github reposi-
tory HGR_NN [ multivariate correlations: HGRkkle
HGR estimation with Neural Net (16) from python package maxcorr and from python github HGR_M for multivariate and full correlations: HGRnn
HGR estimation with Lattice
[4] from python package maxcorr: HGRlat
HGR estimation with double kernel from python package maxcorr.
HGR estimation with simple kernel (14) from python package maxcorr: sk
The Normalized Hilbert-Schmidt Independence Criterion (18)) from python library HSIC
NHSIC
31

---

The Hellinger Correlation ((II) from R library HellCor: HR [but not integrated because inefficient, limited to pairwise correlations and too consuming calculate] We perform our SHGR model for 100, 200, and 500 epochs with early stopping. A sensitivity analysis of the
The python package maxcorr is available from this github repository with the following license: MIT License, Copyright (c) 2024 Luca Giuliani, Permission is hereby granted, free of charge, to any person obtaining copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and permit persons whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software THE SOFTWARE IS PROVIDED "AS 1S", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
The python package numpy is available with the following license: Coppyrghh (c) 2005-2024, NumPy Developers. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are mel: i) Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. ti) Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. tii) Neither the name of the NumPy Developers nor the names of any contributors may be used to en- dorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILIT AND FITNESS FOR A PARTICULAR PURPOSE ARE DIS- CLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE. GOODS OR SERVICES; LOSS OF USE DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY. WHETHER IN CONTRACT, STRICT LIABILITY. OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
The License of package AlterCorr GPL2 The github repository HGR_NN is public.
The license of R package acepack is as follow: License: MIT + file LICENSE Copyright (c) <YEAR>, <COPYRIGHT HOLDER> Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software") "), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Saftware is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS". WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY FITNESS A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER IJABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
F.5 Performance analysis details
In this section, we analyze the performance of the methods, i.e. their ability to identify perfectly non-linear correlations. These performances are analyzed for the estimation of bivariate, multivariate and full correlations.
32

---

F.5.1 Bivariate Correlation
The objective is that:
i) the estimated correlation matrix as close as possible to the reference correlation matrix. We therefore analyze the results based on the distance to the reference correlation matrix:
d_M == -E ('Meelli - CMrrsliss]l
where CMest | E [0, 1] (resp. C.Mres ( [0, 1]) is the estimated (resp. reference) correlation between variables and j.
ii) the nonlinear correlations are exactly identified. We therefore analyze the results based on the focused distance to the reference correlation matrix:
d M Foeus
>E CMsselli CM x mask
where mask filters the 5 coefficients of interest. It is indeed important to measure perfor- mance these correlations, some coefficients introduce spurious correlations in their estimates, which can degrade overall results.
can observe that the methods ACE and SHGR achieve the best scores regarding the distance to the reference correlation matrix (d_ ML_M__re When focusing on the five non-linear correlations, we notice that the methods ACE, RDC, MIC and our approach SHGR yield the best results. This can be explained by the fact that these methods are able to effectively identify non-linear correlations. However, the methods RDC and MIC tend to introduce false correlations where none exist, which degrades their performance relative to the reference matrix.
In Figure 15
F.5.2 Multivariate Correlation
In Figure [6]
we observe that SHCR gives good results but not as good as the ACE method. It is, however, better than the other methods RDC, dCor, CCA, HGRnn, dk, sk and NISIC.
F5.. Full Correlation
In Figure 17, we see that only the methods SHGR, CCA, RDC, and sk manage to capture perfectly non-linear correlations between the two datasets. The methods HGRnn, NHSIC and dCor show deviations from the reference correlation.
F.6 Robustness to Noise
Here, we analyze the robustness to noise of methods: their performance in the presence of white noise (null average). We therefore propose to reproduce the previous simulations while introducing noise that increases with each simulation. Under these conditions, the true correlation is unknown. However, following the analysis of [3I], the metrics producing the highest coefficients will be considered the most robust. For this reason, retain the metric in the previous section: the distance reference correlations. In this scenario, the noise is progressively introduced (the x-axis represents increasing noise from left right).
F66.1 Bivariate Correlation
For the analysis of bivariate correlations, through the correlation matrix, we use the same two metrics for performance: distance to the reference correlation matrix and distance to the 5 non-linear correlations only. In Figure we can see that the SHGR approach outperforms the other methods. Indeed, it gives the lowest distances reference correlations, even though the ACE method still delivers good results. However, it's interesting to note that the performance of ACE deteriorates rapidly with the addition noise: the SHGR approach remains below, i.e., closer to the reference correlations. By analyzing the distances to the focus matrix, i.e. to the 5 non-linked correlations, we can see that the MIC method excels in the absence of noise, but its performance deteriorates sharply with increasing noise.
33

---

Ditteoo
10728
SHOrTe sear pooes)
socoit pest cownt)
Mebel
(a) Lineplot of distances Reference Correlation (d_ (_CM)eff)
(b) Boxplots of distances Reference Correlation focus (d_C MefCMrefoocuu
Desgen Beleaae Oumakk Mase
Mana
00A4
21
0.3
73
1.6
15
54
83
50
C154
62
E2
15
62
60
63
19
55
60
89
83
833
59
65
HGR
71
0.5
84
68
67
61
02
67
166
HSRk
23
20
MG
113
125
112
13
53
T25
195
19t
112
139
VBC
00
00
0.5
60
0.5
7.9
16
25
122
95
89
10.1
13.0
93
10.1
0.1
51
01
61
211
0.1
02
0.7
0.5
KGR
00
Spcerner
51
0.0
76
91
75
60
10:
80
6.2
Ran
(c) Heatmap of distances to Reference Correlation (d) Hcatmap of distances Reference Correlation focus (d_C Mref Focus
(d_ odeM)
Depaee Befeence Oupnta Mase
Delaace Relerman Oure 150 M999 Foox
00A
110
110
12D
90
120
140
12D
1.5
11.
0154
14.0
140
140
155
140
145
128
150
120
150
05
160
1E0
H0R
60
8.5
69
110
60
86
82
68
60
68
HSRk
12.
11.
100
1100
08
40
00
10
01
MGC
T60
160
180
90
99
160
160
16.
96
VBC
1310
150
12.
13.0
155
15.
150
15.0
12.0
13.5
15.3
IDC
30
20
20
25
23
2.0
20
20
10
1.0
KGRL
Spcerner
5.0
7.8
4.0
60
5.0
5.
82
89
00
50
12.
120 120
12.0
124
124
22
120
12.00
11.2
100
50
33
90
00
Ran
(e) Hcatmap of ranks of distances Reference Corre- lation (d_ Mref)
(f) heatmap of ranks of distances Reference Correla-
tion focus (d_CMreff Focus)
Figure 15: Performance: distance to Reference Correlation Matrix (lower is better) results for bivariate correlations
62
HGRan
40
49
68
68
HGR
RBC
50
30
44
54
20
50
55
54
54
5.0
RDC
2.5
2,8
3.6
30
39
24
4.
40
19
34
100
45
25
2.
100
18
SHGR
45
25
4.2
28
89
80
6/0
87
88
80
8.0
81
64
84
160
15.
130
910
100
160
160
1
13
10.
930
10
100
160
160
110
910
100
100
15.0
(a) Hcatmap of ranks of distances Reference Corre- lation (d_ (d_C)rref)
(b) Heatmap of distances to Referenee Correlation (d_ (d_CMref)
Figure 16: Performance: distance to Reference Correlation Matrix (lower is better) results for multivariate correlations
F.6.2 Multivariate Correlation
In Figure 19 we see that the SHGR method gives the best results: smallest distance to reference correlations. Once again, the ACE method is significantly better in the absence of noise, but results deteriorate with the addition of noise; our SHGR approach remains more stable.
34

---

Delonnce Rebornso Correiionn Marla
CCA
24
02
08
0.
0.
00
OO
RDG
0.
0.
0.8
0.
0.
08
08
00
60
00
SHOR Jouns
0.0
0.
08
05
03
08
00
25
03
88
05
02
22
02
08
05
02
00
0d
00
60
00
Vehool
Pan
Heatmap of distances Reference Correlation Figure 17: Performance: distance to Reference Correlation Matrix (lower is better); results for full correlations
(a) Boxplots of distances Reference Correlation
Degance todeeror Centage Pooco
Kondal
(a) Lineplot of distances to Reference Correlation (b) Lineplot distances to Reference Correlation focus (d C. F Focus)
(d CMref)
Debaee io Comeation tert: Sehewecs
Detacce to Ecmbbao Matte Defrenck Esga
00A
08
87
168
150
1:3
11.5
C04
132
12A
125
126
112
11.
75
18
84
122
76
83
84
114
191
48
93
55
57
104
Kendul
NHISC
4.
03
73
80
82
35
8.
0.
00
011
50
10
13
53
15
27
85
125
155
13.2
1355
16.
SH-3R: (Fam))
02
022
0.00
04
15
41
Speamer
8:
013
011
10V
103
11.2
Eooaman
11.8
11.5
13.
12.3
6.0
70
7:1
8.5
87
10A
127
11:3
12.3
5.
Ran
(c) Hcatmap of distances to Reference Correlation (d) Heatmap of distances to Reference Correlation focus (d_C (fcus
(d_ id_C)
Dolamo Corclfoor Mulls Pufenne
Ookone Doncklon Mutis Rddcoss Faxe
11.
K0
103
120
120
913
14.
125
125
#300
140
140
14.0
14.
150
1400
1400
140
14
600
40
48
40
T0
10
60
70
HBRmn
8.8
83
12.
60
HGRin
100
42
11.
80
70
82
63
1.0
Kendal
NiIScc
0.0
12.
100
130
120
180
150
(20)
12.0
150
180
140
120
120
186
SHHR 100 (cam)
1.5
15
10
10
15
23
1.9
SHOR, 100 Jouns
20
10
10
1.3
13
30
300
15
1.5
Spearer
Epcaman
110
110
110
11.0
91
11.0
81
81
80
11.0
76
11.6
11.
110
(c) Hcatmap of ranks of distances to Reference Corre- lation (d_ Mrer))
(f) heatmap of ranks of distances to Reference Correla- tion focus (d_ (d_MMrefFaa
Figure 18: Robustness to noise: distance Reference Correlation Matrix (lower is better) results for bivariate correlations
F.7 Robustness to Hallucination
In this section, we evaluate the correlation measures to avoid hallucination, indicating correlation on independent data (ten independent Gaussian variables). For each context, we simulate independent data and apply the correlation coefficients. Figure 20) we observe that our approach SHGR remains very robust to hallucinations, whatever the type of correlations estimated.
35

---

Reboo
ACE
21
6.3
622
68
95
97
RDC
48
43
68
8.
F3
F8
F9
822
ALT
6-6
SSHORES Ioans
24
35
43
69
0.
F.1
Y:T
7.9
120
120
120
100
12.
12.
100
12.
10.0
10.0
Pan
Figure 19: Robustness to noise: distance to Reference Correlation Matrix (lower is better) results for multivariate correlations
Datance
Natrs
AGE
02
0
08
0.5
87
07
05
us Reterree Commaation
HGRan
25
25
76
HGRen
48
43
47
63
40
47
48
11
61
5.4
Kenda
21
1.8
19
146
NHSC
Plarmon
2,5
RBC
104
105
10
13.3
10.0
11.0
10.1
10.6
12.4
11.1
SHGR, 100 (ong)
01
05
00
0.1
0.5
02
0.5
(009))
sprca donryy guecin speonn Paarrii
conrgy
querd
Hosten
dite
6.0
3.0
04
7.5
81
81
8.6
8.5
8.0
8.5
2.4
6.8
6.
6.7
82
6.8
7.0
6.7
7.0
Dittwos to Relemenca Commitiinn TOaTY
ACE
03
03
033
0.5
C33
0.3
6.5
5.3
C/3
NHSIC
0.1
0.1
0.1
0.1
0.1
0.1
0.1
0/1
0.1
8H0R_100 (ous))
0.6
08
0.7
0.5
63
0,8
6.5
(ouns)
0.6
C9
cCor
1.0
1.0
1.8
1.0
1.1
1.D
1-0
1.5
1.0
AED2
4.5
4.5
4.7
4.7
Run
0.35
Distance to Reference Correlation Matrix
Distanoo Refe Cormertion Morti
0.30
CCA
astanee
0.25
HG3tm
RC
C.I
8D0
0
BC
GD
0.0
241
0.0
0.20
0.15
0.10
RDC
03
02
99
03
0.3
0.2
0.05
0.00
0.
0.0
SHGR ES (our))
RDI
dEar
CCA
HoRRn
NHSIC
0.3
0.3
0.5
333
85
0.3
0.5
0.3
231
0.3
Ran
Figure 20: Robustness to hallucination: distance to Reference Correlation Matrix (lower is better); for bivariate correlations (up), multivariate correlations and full correlations (down)
E.8 Robustness to Extreme Values
In this section, we evaluate the correlation measures avoid hallucinations due extreme values. As above, we simulate independent data with extreme values (except for the first iteration, to measure the impacts). Figure 21 we see that the approaches ACE, CCA, HGRmn, HGRlat, Pearson, dk, sk and dCor obtain values of 30 and are therefore very sensitive extreme values. Methods HGRkde, MIC and RDC show values significantly different from (due to hallucinations). Methods with low values are Spearman, Kendall, NHSIC and SHGR.
36

---

Palue Compasson
Delanee Heterano
00A
304
2010
33.
30.5
33.0
26.0
300
30.0
3010
HGRR
50.0
20:0
301
30.0
501
30.8
3.0
rxaten
35.
30.0
30.0
30.5
31.0
30.0
310
30.0
M0
18:00
8:3
8.3
33
4.1
Pearson
550
50.0
33.0
50.0
90.0
36.3
90.0
30.3
$0.0
26
310
286
2.0
2/8
2.8
Spaiit 203 pare
8.0
0.
0.0
6.8
0.2
6.0
0/8
8.2
0.8
BHGRt ES (our))
0.0
68
0.6
59
0.0
60
8.0
Spaemrnn
28
0.6
2888
29
30.0
334
38.5
31.0
38.0
310
30.0
30.
Vertod
(a) Boxplots of distance Reference Correlation Matrix
Run
Heatmap distance Reference Correlation Matrix
Figure 21: Robustness to Extreme Values: distance to Reference Correlation Matrix (lower is better): for bivariate correlations
F.9 Significance Test
SHGR optimizes the Pearson linear correlation coefficient based transformations of the original variables, in line with the theoretical definition of HGR. As such, it is possible to perform an asymptotic significance test on the obtained correlation coefficient (when n is sufficiently large) [29 Specifically, we test the following null and alternative hypotheses:
(Ho) SHGRo u)
versus (H1) (H) #0
We apply the existing significance test for the Spearman coefficient (more robust than the Pearson test, also sensitive to extreme values), using on the transformed variables provided by the method. We evaluate the performance of this test across several bivariate and multivariate correlations, which we progressively add noise to. This test is compared to the ones available for MIC, RDC and RDC for nine types of relationships. Figure 22 shows the p-value (y-axis) and the noise addition (x-axis) (increasing towards the right). We can see that the significance test fairly low for the SHCR approach, ie., the test generally rejects the null hypothesis of the correlation coefficient fairly robustly.
Figure 22: Performance Significance test: for bivariate correlations (lower is better)
E.1 Power of Dependence Measure details
addition the previous analyses, we now perform the bivariate power of a dependence measure. we assess the robustness of our method by progressively perturbing several nonlinear bivariate/mullivariate correlations. The coefficient should ideally remain as high as possible.
Following the analyses conducted in 31. 32 and [5]
37

---

Ees
Figure 23: Power of Dependence Measure: results for bivariate correlations (higher is better)
Figure 23 shows correlation coefficients (ordinate) function of added noise (abscissa): increasing towards the right. We can see that our approach SHGR outperforms the other methods: higher and manages capture correlations even in the presence of noise.
W
Figure 24: Power of Dependence Measure: results for multivariate correlations (higher is better)
Figure 24 shows correlation coefficients (ordinate) as a function of added noise (abscissa): increasing towards the right. In this scenario, unlike the bivariate case where only 2 variables are generated, we generate variables 21; 22 T3 as reduced-centered Gaussian. We then define y correlated non- linearly with æ1 + Z2, I' being spurious variable. We can see that our approach SHGR outperforms the other methods: higher so manages capture correlations even in the presence of noise.
F.11 Computation time
Here we compare the computation times of SHGR with those of other methods. These analyses are carried out for bivariate and multivariate correlations (the full ones being quite fast).
38

---

Corpurror
100
sbnck cowe))
poeart und (oomd)
guclft 2nd coon))
socoil poo court)
SHEIR moum?
aod tound
Vetee
Mehrod
(a) Bivariate correlations
(b) Multivariatc correlations
Figure 25: Computational time Comparison
F.12 Sensitivity to sample size
This section presents analysis of the performance and computation time of the SHGR method, compared with other methods, as a function sample size. More precisely, we present an analysis that allows us to analyze computation time and performance indicators by varying the number of observations as follows: [50, 100, 500, 1000, 5000, 10000) (resp. [1000, 10000, 100000, 1000000),, with the number of variables always fixed at 11. These results analyzed first bivariate correlations and then on multivariate correlations.
F.12.1 Bivariate correlations
Computation Time they take respectively 10 minutes and 47 minutes for 1000 observations, and 30 minutes and 274 minutes for 10,000 observations. The dCor method also takes 18 minutes for 10, 000.
We can see in Figure 26
that that the MIC and NHSIC methods are not usable, as
10e
21
21
IKc
20
120
47/
100
5000
10000
Sample Ebs
Figure 26: Computational time according to sample size: for bivariate correlations
60
Renill
50
Holky
40
HOlee
30
Paee
80
30
1000
10000
100000
1000000
Samole Size
Figure 27: Computational time according sample size: for bivariate correlations (bis)
In this second analysis (Figure 27
where time-consuming methods are removed, we estimate bivari-
ate correlations by varying the number observations as follows: [1000, 10000, 100000, 1000000), with the number of variables always fixed at 11. The SHGR approach is quite long for 100,000 observations but faster for 1,000,000 observations: faster than RDC and as fast as ACE, for example. The dk method is far too long for a large number of observations. It's interesting to note that the SHGR method, being based on neural networks, can be parameterized optimize computation time (notably batch size and epochs parameters).
39

---

Performance Figures 28 and 29 show the performance results obtained on bivariate correlation estimates. They show that the performance of the SHGR approach outperforms that of competitors. regardless of sample size.
34
10
to Rufrenta
30
500
1000
5000
12900
Sample
Sice
Figure 28: Performance according to sample size: for bivariate correlations: distance to reference correlations (lower better)
Disannoe Comeaadon
02
10
10000
100000
1000000
Sample Size
Figure 29: Performance according to sample size: for bivariate correlations (bis): distance to reference
correlations (lower better)
F.122 Multivariate correlations
Computation Time
Figure 30 presents the computation time estimate multivariate correlations
by varying the number of observations as follows: [50, 100, 500, 1000, 5000, 10000), with the number of variables always fixed 20. Our method SHGR is quite fast, even faster than RDC dCor CCA. The dk method takes far too long to calculate.
100
Compuration Time (mR)
80
35
USC
20
Sample
Figure 30: Computational time according to sample size: for multivariate correlations
Performance Figure 31 presents the performance to estimate multivariate correlations by varying the number of observations as follows: [50, 100, 500, 1000, 5000, 10000], with the number of vari- ables always fixed 20. The results show that the SHCR method gives very good results, whatever the sample size.
E.13 Sensitivity to the number of variables
This section presents an analysis the performance and computation time of the SHGR method, compared with other methods, function of number of variables.
40

---

10.0
MBV
8.0
7.5
100
500
1000
5000
10000
Samble Size
Figure 31: Performance according to sample size: for multivariate correlations: distance to reference correlations (lower better)
E.1311 Bivariate correlations
Here present analysis that allows to analyze computation time and performance indicators by varying the number of variables as follows: [1, 2, 5, 10] x = 11 with the number observations always fixed at 1000. These results are analyzed first on bivariate correlations and then on multivariate correlations.
32 shows the computation times for estimating multivariate correlations by varying the number of observations. The results show that the SHGR method has very satisfactory levels. The methods IIGRlat, NHSIC and dk present relatively long times.
Computation Time Figure 32
10ckc
Number
Figure 32: Computational time according number of variables: for bivariate correlations
Performance Figure 33 shows the performance for estimating multivariate correlations by varying the number of observations. The results show that the SHGR method outperforms its competitors.
1600
604
291
1200
1o Rufrrese
200
261
400
11
22
55
110
Namber vertables
Figure 33: Performance according to number of variables: for bivariate correlations
E.13.2 Multivariate correlations
Computation Time Figure 34 presents the computation time estimate multivariate correlations by varying the number of variables as follows: [20,30, 30, 40, 70, 120). with the number of observations always fixed at 2000. SHGR shows fairly long computation times, higher than ACE, CCA, RDC and dCor but faster than NHSIC and HGRmn.
41

---

Computaiioal
Thme
1
30
120
Number of varaba
Figure 34: Computational time according to number of variables: for multivariate correlations
Performance Figure 35 presents the performance to estimate multivariate correlations by varying the number of variables as follows: [20,30] 30, 40, 70, 120], with the number of observations always fixed at 2000. Compared with other methods, the results of SHGR deteriorate slightly the number of variables increases. To improve results, it might be more efficient loop over the estimation of correlations, like the other methods, accepting slight loss in calculation time.
QCA
1M
Dilaace Refenna
48
10
20
01
19
ou var abree
Figure 35: Performance according to number of variables: for multivariate correlations
E.14 Improving existing methods
To guarantee the relevance of our approach, we compare the results obtained with the best competitors: using input data transformed data from the SHGR. Figure shows the gains (correlation SHGR encoded correlation on inputs) obtained with the benchmark applied on the SHGR encoded rather than inputs (with the simulations as bivariate Power of Dependence Measure above). All results are enhanced: correlation is greater when combined with SHGR than when applied directly to inputs.
Exchhmuk conteca En
Dosc WoB ourralo
Denc
met Wct Dverae
(LNCK KLACk_eec Jeca
JoWw
15
21
2.4. H.HAA Ja66
8
8
10
22
JUMEIC JANN
35
10
hode
6,00A
07 00
7677666
57
47
0,200 SHOR
07
63
50
(L00L PHOK
11 14 15
13
28.7
MMMR 2H0W
53
87
60
50
(JHeK SHOW
10
56
53
47
36
30
76
18
SHOR
64
40
35
33
14
12
7B
MHOR
12
44
17
10
TARAA
13
17
14
27
23
33
25
4,200_500R
98 96
81
70
52
46
4,000,5058
11 1322
28
7
22
23
20
27
17
cJ SWAR
60
54
27
11
11 12
12
10
19
17
0,80c SHAR
54 53
47
40
30
09
30
28
1023325
20
Figure 36: benchmark methods gain (%) when applied on SHGR encoded rather than inputs
42

---

E11
Transformation Visualization
Our SHGR approach allows us to recover the transformations of the inputs (encoded by stacked neural nets). This makes it easy to graphically analyze the transformations applied by plotting the outputs of the stacked neural nets as a function of the inputs. Figure 37 illustrates a graphical analysis of the transformations in bivariate context.
-120
Figure 37: Illustration of the graphical analysis of the transformations: bivariate setting
43

---

G Real-World Applications details
To assess the relevance of method, we apply it to 9 real-world datasets for feature selection.
The experiments are conducted on the following datasets:
Abalone, composed of 9 variables and 4177 observations. Available in the imbalanced re- gression datasets benchmark ((5) in the public repository: https://paobrancee .githubthuu io/DataSets-I1 IR/
AirQuality, composed of 15 variables and 9358 observations (43) This dataset is licensed under a Creative Commons Attribution 4.0 International (CC BY 4.0) license. Available at
https://archive. ics uci wttpe///aaaaatee//////aaaaa
copposee of 28 variables and 19735 observations (()). This dataset is licensed under a Creative Commons Attributtion Internatiion (CC BY 4.0) license. Available at https://archive. ics uci htt..e/cc..caaaaeee//a/aaaaaaaaa
Bank8fim, composed of 9 variables and 4499 observations. Available in the imbalanced re- gression datasets benchmark ((5) in the public repository: lattps://paobrance .eoooggitiig github
io/DataSets-I1 IR/
Boston, composed of 14 variables and 505 observations. Available in the imbalanced regres- sion datasets benchmark ((5) in the public repository: luttps://paobranco. -itggttthhb..... io/
DataSets-IR/
concrete, composed of variables and 1030 observations (50))
This dataset
is licensed under Creative Commons Attribution 4.0 International (CC BY 4.0)
license. Available at hattss ics uci woc..ddu//eee edu/dataset/165 concrete compressiveessteenthh
CpuSm, composed of 13 variables and 8192 observations. Available in the imbalanced re- gression datasets benchmark ((5) in the public repository: huttps://paobrance gi ritb
io/DataSets-1n IR/
NO2, composed of 8 variables and 500 observations. Available in the imbalanced regression datasets benchmark (5) in the public repository: Inttps://paobaanco wwwpobbrrrnnniiiguu..u... 1o/
Dataßets- IR/
Temperature, composed of 24 variables and 4137 observations (39). This dataset is licensed under Creative Commons Attribution 4.0 International (CC BY 4.0) license. Available https: petps://arrhiiee ar archiie..... ics. uci. edu/ http/daasse//22//5 2atpsett/2/ssmmm2111
Note that we only consider numerical features.
G.1 Maximal Correlation
We perform regression tasks using the top features selected by each method, i.c., the ls features most correlated with the target variable y. We then analyze the root mean squared error of test set predictions for each value of and each method, Predictions are made on a test set (30% of the original dataset), randomly sampled, using random forest model. The training consists of most 500 observations for all datasets (to assess the robustness the methods to sampling). Figures 38 present the obtained results.
G.2 Contribution to Maximal Correlation
We perform regression tasks using the top features selected by each method, i.c., the l features that contribute most to the maximum correlation of features with the target variable y (Contr(æ;) := SHGR(y, X) SHGR X \ X2)). We then analyze the root mean squared error of test set predictions for each value of l and each method. Predictions original dataset), randomly sampled, using a random forest model. The training set consists of at most 500 observations for all datasets (to assess the robustness of the sampling methods).
made
test set (30% of the
44

---

(a) Abalone datasel
(b) AirQuality datasel
(c) Appliance dataset
400
(d) Bank8FM dataset
(e) Boston dataset
(I) Concrete dataset
(g) CpuSm dataset
(h) NO2 dataset
(i) Temperature datasct
Figure 38: Feature selection real-world datasets with SHGR shown in red (lower is better): based
on maximal correlation (bivariate correlation)
(a) Abalone dataset
(b) AurQuality dataset
(c) Appliance dataset
400
(d) Bank8FM datasct
(c) Boston datasct
(f) Conerete dataset
(g) CpuSm dataset
(h) NO2 dataset
ii) Temperature datasct
Figure 39: Feature selection real-world datasets with SHGR shown in red (lower is better): based on maximal correlation contribution (multivariate correlation)
45


Reviews:
[{"review_id": "XiDfMHt8Nz", "reviewer": "Reviewer_vdVN", "summary": "This paper introduces a new coefficient for a given random variable pair, called SHGR (Spearman HGR), which combines some advantages of Hirschfeld-Gebelein-Rényi (HGR) maximal correlation and the Spearman rank correlation. The authors provided theoretical properties of the SHGR correlation and discussed its estimation based on neural networks. Numerical experiments were presented to support the analyses.", "questions": "1. Compared with HGR, SHGR can be more robust to outliers. However, the ranking operation also loses the information about its original numerical values. There is a tradeoff between robustness and keeping accurate information. Can the authors briefly comment when SHGR is favorable to HGR?\n2. The proof in Appendix D.4 (point 6) does not hold. It is not even proving what the proposition states.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "### Strengths\nThis paper is on an important topic of quantifying the relation between two random variables. The proposed SHGR correlation appears to be novel, although it is related to other formulations, such as copulas. The numerical experiments are comprehensive. \n\n### Weaknesses\n1. The paper did not discuss the details of neural SHGR. specifically:\n      + L140, if directly used as the objective function, would suffer from a biased gradient estimation. This is due to the normalization factors in computing the correlation coefficient $r$.\n      + Sec 3.4, the multivariate generalization can be too arbitrary and lack a good motivation.\n      + L175, the usage of torchsort should be further justified.\n2. Overall, the theoretical properties of SHGR and how these properties guide the estimator design should be discussed in more detail.\n3. The presentation throughout the paper can be greatly improved. For example, before Def. 2, the samples should be paired $\\{(u_i, v_i)\\}$, rather than separate samples of $U$ and $V$. There are also some technical problems, see the question section.", "was_revised": true, "final_justification": "The authors have provided detailed illustrations of both technical issues and related work. I believe the manuscript will provide valuable insights if these points are incorporated in the final version. Therefore, I raise my rating to 4.", "rebuttal": "We sincerely appreciate the reviewer’s careful reading and constructive feedback. Below, we provide detailed responses to each point that requires clarification or revision.\n\n**Weaknesses**\n1. \n- We thank the reviewer for this insightful comment.\nWe are aware of the gradient bias introduced by the normalization terms and have experimented with detaching them during training. However, this led to a significant degradation in empirical performance, particularly in recovering correlation. Given our focus on practical effectiveness, we chose to retain the differentiable formulation. We have clarified this trade-off in the revised version with the following sentence in the discussion section: \"*Although detaching normalization terms reduces theoretical gradient bias, we found that doing so significantly harmed empirical performance. We thus keep the differentiable form and leave bias-corrected alternatives for future work.*\" It is also worth noting that similar neural correlation estimators, such as HGR_NN [1] and MaxCorr [2] do not explicitly address this gradient bias either. Finally, we would like to mention that our code will be made publicly available if the paper is accepted, which may help the community further explore and potentially address this limitation. We remain open to any additional suggestions the reviewer may have on mitigating this issue.\n\n\n[1] Grari, V., Ruf, B., Lamprier, S., & Detyniecki, M. (2019). Fairness-Aware Neural R\\'eyni Minimization for Continuous Features. arXiv preprint arXiv:1911.04929.\n\n[2] Giuliani, L., Misino, E., & Lombardi, M. (2023, July). Generalized disparate impact for configurable fairness solutions in ML. In International Conference on Machine Learning (pp. 11443-11458). PMLR.\n\n- We appreciate this helpful observation.\nWe agree that the transition to the multivariate case could be better motivated in the current version. We revised the paper to clarify that our generalization is guided by well-established approaches such as Canonical Correlation Analysis (CCA) and the Randomized Dependence Coefficient (RDC), which aim to capture dependencies between multidimensional variables. Indeed, in many practical applications, it is essential to estimate correlations not only between individual variables, but also between a group of variables and one or more targets, as is often the case in neuroscience [3], sociology [4], biology [5], medicine [5], and engineering [5]. Our multivariate formulation, while concisely presented in the submitted version, is grounded in extending rank-based dependence estimation to objectives that maximize correlation between multivariate variables. This motivation has been made explicit in the revised version by adding the following sentence: “*While pairwise correlation analysis is useful, many real-world applications require assessing dependencies between a group of variables and one or more target variables. In a similar manner to the Canonical Correlation Analysis (CCA) and the Randomized Dependence Coefficient (RDC), we propose here a natural generalization of our rank-based HGR estimator to this multivariate setting.*” \n\n\n[3] Zhuang, X., Yang, Z., & Cordes, D. (2020). A technical review of canonical correlation analysis for neuroscience applications. Human brain mapping, 41(13), 3807-3833.\n\n[4] Li, L., Wang, P., Li, S., Liu, Q., Yu, F., Guo, Z., ... & Wang, X. (2023). Canonical correlation analysis of depression and anxiety symptoms among college students and their relationship with physical activity. Scientific Reports, 13(1), 11516.\n\n[5] Zhou, Z., Ataee Tarzanagh, D., Hou, B., Tong, B., Xu, J., Feng, Y., ... & Shen, L. (2023). Fair canonical correlation analysis. Advances in neural information processing systems, 36, 3675-3705.\n\n- We thank the reviewer for pointing this out. Due to space limitations, the motivation behind this choice was not sufficiently explained in the paper. Since our loss relies on rank-based measures, we require a differentiable approximation of the ranking operation to allow gradient-based optimization. We use TorchSort [6] to enable differentiable sorting within our objective function. The following clarification has been added in the revised version: \"*Since our loss function depends on rank-based estimation, we use TorchSort approach \\cite{blondel2020fast} to obtain a differentiable approximation of the sorting operation.*\"\n\n[6] Blondel, M., Teboul, O., Berthet, Q., & Djolonga, J. (2020, November). Fast differentiable sorting and ranking. In International Conference on Machine Learning (pp. 950-959). PMLR.\n\n2. We thank the reviewer for this valuable comment, which gives us the opportunity to clarify the theoretical motivations behind SHGR.\nThe estimator is designed to be both robust and theoretically grounded. Typically, we initially adopted a rank-based formulation to improve robustness to outliers and extreme values. From a theoretical perspective, as described in Section 3, SHGR can be interpreted as a reformulation of the Hirschfeld-Gebelein-Rényi (HGR) coefficient, where the classic Pearson correlation is replaced by a grade correlation. This substitution is made possible by a copula transformation, i.e., applying the cumulative distribution function to each transformed variable which maps them to the uniform distribution on [0,1] while preserving the underlying copula, and thus the full dependence between variables. Moreover, several theoretical results support the validity of our estimator: Proposition 1 shows that the neural estimator $SHGR_\\theta$ converges to the empirical version; Proposition 2 establishes that the empirical estimator $\\widehat{SHGR}$ is consistent and can be reliably used in practice; Propositions 3 and 4 demonstrate that SHGR satisfies Rényi's axioms and is a valid dependence measure. Taken together, these results justify our design choices: robustness through rank transformation, dependence preservation via copula invariance property, and practical reliability through proven convergence.\nWe have revised the manuscript to make this theoretical foundation more explicit, at the beginning of the part \"Consistency of SHGR estimators\": \"*SHGR is empirically motivated as a rank-based reformulation of the HGR coefficient ensuring robustness to outliers and extreme values. It theoretically relies on copula transformations to preserve dependency structures. We now prove consistency guarantees through convergence results and Renyi's axioms.*\"\n\n\n3. We appreciate this helpful remark. We agree that the samples should be described as paired, not as independent samples. We have revised the notation to make this clearer. We also appreciate the technical remarks and will address them carefully in the updated version.\n\n**Questions**\n1. We thank the reviewer for this observation which is a key point of our proposition. In our approach, the rank transformation is applied only during the optimization of the correlation, not on the input variables themselves. As noted in Practical Remark 5 (line 183), the original inputs remain unchanged; they are optimized to maximize the correlation between their rank-based representations. This means there is no direct loss of information due to the rank transformation, unlike methods such as RDC, which explicitly transform the data into ranks. SHGR is thus particularly favorable in settings with outliers or non-linear relationships, while still preserving the expressiveness of the original inputs. We have clarified this point in the revised version by adding the following sentence: \"*Note that the rank transformation is applied only to the correlation computation, not to the input variables themselves. This preserves the full expressiveness of the original inputs while benefiting from the robustness of rank-based objectives.*\". Additionally, the analysis presented in Appendix F.12 – Improving existing methods shows that competing methods actually perform better when applied to the learned transformations from our estimator, rather than directly to the raw inputs. We added this point in discussion section with the following sentence: \"*our approach can be combined with existing methods to improve their results, as shown in the appendix F.12.*\".\n\n2. We thank the reviewer for pointing out this mistake. We have reviewed and corrected the proof, and the revised version is now included in the updated appendix. We apologize for the oversight. Here, the revised demonstration: Point 6. If $U=f(V)$, choosing $f_u = identity$ and $f_v=f$, we obtain $r(F_{f_u(U)}(f_u(U)),F_{f_v(V)}(f_v(V))=r(F_U(U),F_{U}(U))=1$. By construction of the SHGR, by normalizing with a function $g$ such that $\\mathbb{E}(g(U))=0$ and $\\mathbb{E}(g^2(U))=1$, we obtain the result. \n\nWe appreciate your attention to our work. Should you have any further concerns, we would be glad to address them."}, {"review_id": "JWuKQ3tPMQ", "reviewer": "Reviewer_G8iJ", "summary": "The authors propose a novel neural network based correlation based correlation estimator that provides a neural approximation of the HGR estimator. Note that the HGR estimator is known to capture non-linear dependencies that the more commonly used Pearson correlation coefficient does not. However, it does so through an exhaustive search through pairwise comparisons of non-linear mappings of random variables to determine the maximum Pearson coefficient. It thus imposes a high computational burden and does not scale well. (It is reminiscent of the Hausdorff distance metric in that sense and thus equally computationally intense.) \n\nPast work on this problem has recognized the importance of capturing non-linear dependencies using neural techniques. One class of methods does not use the HGR and thus lacks the theoretical backing that HGR provides. Approximations of the HGR have been proposed in the literature but they are sensitive to noise, lack normalization, do not permit significance testing etc. The authors provide a table that compares various existing techniques across multiple assessment criteria. The table shows why a new approach would be required that would cover all the requirements mentioned earlier and thus motivates the authors' proposed SHGR. \n\n\nThe authors propose a neural network based variation on the HGR estimator called the Generalized Neural HGR that limits itself to bivariate correlation matrices, multivariate correlations between a set of predictors and a target, and full correlations between two sets of variables.  It can be thought of as a reformulation of the HGR that is inspired by the Pearson coefficient and copula methods that uses cumulative distributions thus providing robustness to extreme values and non-linear relationships. It replaces raw linearity with rank-based monotonicity which is a way to account for the holistic correlation without the undue influence of extreme values that a linear measure would suffer from.  The authors go through rigorous analysis to show that their proposed estimator has many of the useful properties that HGR does while it is much more tractable computationally. They propose a first estimate of their proposed SHGR that is differentiable, easy to compute and robust to noise. The proposed first estimate motivates them to propose a stacked cross-encoder based architecture that enables parallel computation of multiple correlations which is ideally suited for the multiple correlation exploration required by the HGR. Note that the Canonical Correlation Analysis (CCA) is a special case of the SHGR which is intuitively satisfying. \n\nThey propose a new assessment protocol for assessing maximal correlation based measures based on performance, robustness to noise and hallucinations, significance testing and computational efficiency.\nThey run experiments on real-world and synthetic tabular datasets and show that their proposed estimator clearly improves upon the state of the art in detection accuracy, detection accuracy and stability. They cover a variety of past estimators including those that attempt to approximate the HGR. They enable comparison with existing methods by using the SHGR as a mapping so Pearson coefficients can be compared.  In their discussion section they go into the limitations of their proposed estimator and provide some useful insights for future work. All in all a well written and interesting paper on neural approximation of a computationally complex but effective correlation measure.", "questions": "The authors describe a neural HGR before describing their SHGR measure. How much of an innovation over the neural HGR is the proposed measure? Clarifying that would help strengthen the novelty/originality of the paper.\nHow do you decide you have enough training data for your SHGR measure? I see the danger of overfitting to your training data.", "limitations": "The authors adequately cover the limitations of their work in the discussion. I did not see anything about societal impact but their work is largely neutral to societal impact. If anything their proposed measure should help tighten up current efforts to improve alignment of AI with norms.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. Well written.(Clarity)\nThe logical flow of the paper is good.  Each section nicely leads into the next. There are clear illustrations as well as copious material in the appendices which helps the reader get the gist before diving into the details. The mathematical proofs are written systematically with concise but helpful textual descriptions. \n2. Good literature review.(Clarity and significance)\nThe authors review linear and non-linear measures of correlation thoroughly. They go through past non-HGR and HGR based methods that capture non-linear dependencies and find that the state of the art falls short in terms of robustness to noise, interpretability etc. They find that every past measure falls short along one or more such requirements. After leading the reader through a systematic review, they finally provide a table that clearly shows the limitations of the state of the art along the new assessment criterial that they have developed. \n3. Well motivated novel correlation measure.(Quality and Originality)\nThe authors nicely motivate their proposed new assessment criteria through their literature review. Once the requirements are laid out in that manner, the authors build upon the HGR step by step by first reviewing the Neural HGR and finding that it is still prone to extreme values and linear dependencies, and then showing how a simple reformulation of the HGR in terms of monotonic dependency enables their proposed SHGR which is computationally tractable while maintaining much of the desirable property of the HGR viz capture of non-linear dependencies. They use their this reformulation to propose a clever neural realization they term the Generalized Neural HGR which enables simultaneous computation of cross-correlations the capturing the spirit of the HGR without its computational burden and vulnerabilities. \n4. Convincing experimental results.(Quality, Significance, Originality)\nThe authors compare their proposed SHGR with a broad set of measures that cover linear and non-linear methods including those that approximate the HGR. Their proposed new measures of efficacy are reasonable as explained above because they constitute a theoretically backed evaluation framework that also intuitively convincing. Their proposed technique performs better than the state of the art along all the proposed new measures of correlation efficacy such as noise robustness, robustness to extreme values, computational efficiency etc. They have a detailed discussion of performance along each of those measures that thoroughly and soundly explains the superiority of their proposed measure. \n5. Good discussion of limitations. (Clarity, Originality)\n\nThe authors point out that while their method outperforms the state of the art, it has room for improvement. They mention overfitting and fine-tuning of parameters as a concern. They point out its lack of interpretability and heavy reliance on numerical optimization. They state that further extension to high dimensional spaces represents a significant and rich avenue for future research. My view is that the authors' discussion of the limitations of SHGR shows a thorough awareness of the strengths and limitations of their proposed approach and makes enough of a case that their current work represents a worthy effort towards capture of non-linear dependencies while meeting multiple efficacy criteria. The limitations they mention are applicable to any neural approximation but given their work it is easy to imagine how they would build upon their current mathematical and architectural framework to address the concerns around overfitting, parameter tuning and interpretability. \n\nWeaknesses\n1. The authors describe a neural HGR before describing their SHGR measure. How much of an innovation over the neural HGR is the proposed measure? Clarifying that would help strengthen the novelty/originality of the paper. The authors do say that the neural HGR is still vulnerable to extreme values and other limitations they have laid out. \n2. How much data does the SHGR require? A neural approximation might have overfitting issues. The authors do mention overfitting in passing in their final discussion but do not seem to directly address the challenges posed by overfitting. I don't necessarily see this as a major weakness since that could be a topic for a future follow up paper.", "was_revised": true, "final_justification": "I am satisfied with the author's response to my review as well as to the other reviewers. I think the finally revised paper will a good contribution to Neurips 2025. I asked them about novelty over the baseline neural HGR and overfitting dangers of their technique. They have provide a convincing list of advancements over the baseline Neural HGR. They have also acknowledged the concern on overfitting and provided a convincing response.\nThe other reviewers had other detailed comments, most of which the authors have addressed convincingly. I am sure that they will improve their style and clarity based on the quality of their response to the reviews. I am therefore staying with my original rating of accept.", "rebuttal": "We sincerely appreciate the reviewer’s careful reading and constructive feedback. Below, we provide detailed responses to each point that requires clarification or revision.\n\n**Weaknesses**\n1. We agree that this is an important aspect and clarify it as follows. While our method builds on the neural framework, it introduces several key innovations. First, we reformulate the objective in terms of monotonic (rank-based) dependence rather than raw values, which enhances robustness to outliers and extreme values. Second, we propose a scalable estimator (SHGR) that retains the core properties of HGR but is more robust to hallucinations, and, as the neural HGR, is fully differentiable, making it suitable for integration into modern learning pipelines. Finally, we introduce a novel stacked cross-encoder architecture, which enables efficient computation of multiple correlations simultaneously, something the original neural HGR estimator does not support. If helpful, we could further clarify these contributions in the introduction of the revised manuscript.\n\n2. We acknowledge this valuable point. As with most neural estimators, SHGR could be sensitive to overfitting, particularly in low-data settings. In our experiments, with sample sizes as small as 1000 or 2000, we mitigate this effect using an early stopping and a significance-mask mechanism, which allow SHGR to perform correctly estimate correlations, while avoiding hallucinations and being more robust to extreme values and outliers. In Appendix F.5.2, we evaluate runtime as a function of sample size (ranging from 50 to 10,000). We also tracked performance across sample sizes, though these results were not included in the current submission. We have added them to the revised version, as they show that SHGR outperforms competing methods across all sample sizes, including very small ones. This confirms that SHGR does not overfit and effectively captures correlations in practice. We also conducted complementary analyses with respect to the number of variables (Appendix F.5.3) and in the multivariate setting (Appendices F.5.2 and F.5.3).  \n\n**Questions**\nSee above\n\nWe appreciate your attention to our work. Should you have any further concerns, we would be glad to address them."}, {"review_id": "56H2PSeDJD", "reviewer": "Reviewer_1fNq", "summary": "This paper presents a novel adaptation of the HGR correlation measure by incorporating rank based spearman correlation while estimating the maximal correlation coefficient. Authors also present a neural network based encoder architecture for estimating the correlations in both bivariate and multivariate setups. Empirical results demonstrate superior performance on sensitivity to outliers and computational efficiency to alternative approaches in designing correlation coefficients for measuring non linear correlations.", "questions": "1. Paper could benefit from ablation analysis regarding the model choices like encoder architecture, loss functions to indicate the robustness of correlation estimations.\n\n2. The discussion around how does rank transformation of variables impacts the properties of maximal correlation is missing. can you provide  some discussion around conditions under which SHGR may fail or perform poorly compared to classical HGR", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths - Paper introduces novel theoretical contributions extending HGR from Pearson to spearman correlation with strong empirical evidence. Innovative stacked cross encoder architecture design also allows for simultaneous estimation of multiple correlations avoiding iterative optimizations resulting in computation efficiency. The measure also satisfies Renyi's axioms for non linear dependence measures.\n\nWeaknesses - Paper could benefit from ablation analysis regarding the model choices like encoder architecture, loss functions to indicate the robustness of correlation estimations. The discussion around how does rank transformation of variables impacts the properties of maximal correlation is missing which the paper could benefit from.", "was_revised": true, "final_justification": "I am satisfied with the author's response and the changes they have made addressing my concerns. I have updated my ratings based on the comments.", "rebuttal": "We sincerely appreciate the reviewer’s careful reading and constructive feedback. Below, we provide detailed responses to each point that requires clarification or revision.\n\n**Weaknesses**\nSee Questions\n\n**Questions**\n1. We appreciate the suggestion. As detailed in the “Sensitivities” subsection of Appendix F.2, we analyzed the impact of several architectural and training choices, including hidden layer dimensions, batch size, and learning rate.\nRegarding the loss function, we conducted sensitivity analyses on its key hyperparameter $\\alpha$ and the differentiable rank regularization (as proposed in [1]). This analysis is reported in line 264: “We also analyzed the sensitivity and impact of the results to the hyperparameter power α in the loss function and to the differentiable rank regularization parameter as defined in [4]. Figure 4 presents the results obtained. We can clearly see the benefits of using a power of 2 for correlations. This allows the networks to quickly focus on the strongest correlations. Regarding differentiable rank regularization, the values 0.1, 0.5 and 1 seem fairly comparable. We choose to use the default value of 1.”\nIf helpful, we would be happy to include additional ablations in the revised version to indicate the robustness of correlation estimations.\n\n[1] Blondel, M., Teboul, O., Berthet, Q., & Djolonga, J. (2020, November). Fast differentiable sorting and ranking. In International Conference on Machine Learning (pp. 950-959). PMLR.\n\n\n2. We acknowledge this insightful question which is a key point of our proposition. In our approach, the rank transformation is applied only during the correlation computation, not directly on the input variables. As noted in Practical Remark 5 (line 183), the original inputs remain unchanged and are optimized to maximize the correlation between their rank-based representations. This avoids the direct loss of information typically associated with full rank-based preprocessing, as in methods like RDC. We have clarified this in the revised version, before subsection 3.4, by adding: \"*Note that the rank transformation is applied only to the correlation computation, not to the input variables themselves. This preserves the full expressiveness of the original inputs while benefiting from the robustness of rank-based objectives.*\". From a theoretical standpoint, SHGR satisfies Rényi’s axioms and leverages the fact that copula transformations, specifically $(F_U(U), F_V(V))$, preserve the dependency structure of the pair $(U, V)$. This means that, in theory, SHGR should retain the essential properties of HGR while improving robustness. Our empirical results show that SHGR performs consistently well in all evaluated settings. Appendix F.12 (Improving Existing Methods) shows that the SOTA methods perform better when applied to the learned representations from the neural SHGR rather than to raw inputs, further reinforcing their usefulness. We have added this idea in the discussion with the following sentence: \"*Interestingly, our approach can be combined with existing methods to improve their results, as shown in the appendix F.12.*\"\n\nWe appreciate your attention to our work. Should you have any further concerns, we would be glad to address them."}, {"review_id": "VShzlfxwwA", "reviewer": "Reviewer_Q6UW", "summary": "This paper presents a Spearman version to the HGR maximal correlation (SHGR), and the neural architecture to estimate it. Some interesting propositions are made regarding SHGR, as well as links to traditional metrics such as Pearson and copula-based correlations. Finally, extensive simulations are held, on simulated and real datasets, to show SHGR's ability to: (a) detect correlation under non-linear transformation of variables, in both 1-to-1 bivariate relations, p-to-1 and p-to-q correlation matrix; (b) demonstrate its robustness under noise and independence (avoiding hallucinated correlations); and (c) perform quality feature selection.\n\nTo the authors: I would like to say upfront that I love the idea and I am rooting for this paper, so please read my criticism from this viewpoint. If the paper is not accepted please use my comments, though they are many, to improve it.", "questions": "As said, what would make me increase scores for this paper is more the editing, less the idea or demonstration themselves which I liked, but here are a few things I did not understand:\n\n(1) l121: \"that addresses these limitations\" -- I am not clear on what those limitations are, could you clarify more clearly or point to the exact place in the Appendix where you discuss these and/or demonstrate them?\n\n(2) Why did you mention copulas? Like, at all. Am I correct that in the bottom-line, apart from acknowledging the link between the Spearman correlation (and therefore SHGR) and the Pearson correlation on the copulas, you are not using this insight in any way later on?\n\n(3) l201: What is \"nonlinear dependencies of order greater than 2\"?\n\n(4) l245: Why in this setting \"the true correlation is unknown\", is it because of the noise? Needs to be clearer, you of course know the true correlation, you are comparing to it for your metric.\n\n(5) In Figure 2 the metric made sense, on the y-axis we see \"distance\". Here we see on the y-axis \"MCref\" -- I realize lower is better but I have no idea what this means. What is MCref?", "limitations": "The authors have addressed the limitations of their work, and there do not seem to be any negative societal impact.\n\nTwo points may be better addressed, but this is not that critical:\n\n(1) l328: \"HGR is difficult to interpret\" true but you know, there are tools to at least start exploring what your neural networks have learned once trained, it's a very busy research area, have you tried using any tools to at least visualize the fitted f functions?\n\n(2) l285: \"4.2.7 Computation time\" -- I've been waiting for this paragraph and it was a bit disappointing. Whenever you suggest researchers use a method for calculating a metric with your method which involves neural networks (!) instead of say ACE in R, you should really supply some clear comparisons with a table or graph in the main text. It is very difficult for the reader to summarize from this anecdotal writing that \"SHGR exhibits highly competitive computation times\"...", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "I would say the paper's strength are its neat and simple idea, the clear definition of the its goal, and extensive simulations to demonstrate the method's strengths. The paper's main weakness is its editing and style, but these can be repaired.\n\nNow in more detail:\n\n* Quality: \n  - Strengths: The paper is of high quality when it comes to experimental setting. They test many baselines, in many scenarios, reporting many adequate results.\n  - Weaknesses: The paper is of a (relatively) low quality in terms of writing and flow, many missing parts which could make it easier on the eye. Examples: (1) Reference to appendix is quite bad, it is always a general \"go search\" statement (l117: \"(illustration in the Appendix)\"). You should be specific on what is illustrated and where exactly to find it in this huge appendix; (2) The entire \"Related Work\" section is without a single reference. At a certain point it feels like name dropping (HSIC, MMD, MIC, GeDI -- all of these are mentioned without even explaining what they do, what the initials are), this should be heavily edited. (3) Then, under section 3 which is supposed to show the authors' work they start with defining HGR and neural HGR (4) Footnotes on page 6 start with lower letters, are grammatically incorrect and start with \"but\".\n\n* Clarity:\n  - Strengths: SHGR itself clearly defined, the motivation is clear, all mathematical expressions are clear.\n  - Weaknesses: (1) The Figures are extremely hard to read, even at zoom 120% I found them hard to read (2) Very lazy captions (we need a little bit more than \"lower is better\") (3) Flow between paragraphs and sections is poor, e.g. l141: \"Consistency of SHGR estimators\" -- Please give the reader one or two lines on what is about to take place, this is an article, not a summary of a uni class. Especially since these propositions are so cool and important to realize. Another example: l157: \"It is often necessary to analyze\" -- again, what are we doing, tell the reader what we just did and what is about to occur. (4) Poor choice of titles: section 4 is called \"Numerical Illustration\" and section 5 is called \"Experiments\" -- did we not just see a (huge!) set of experiments? (5) Missing important details in experiments: e.g. what are your functions? I realize there is little room but one of the most interesting things in these simulations is what functions (e.g. f(U)) you used to simulate the transformations, it is really hard to be impressed without knowing how difficult they would be to approximate for e.g. a simple MLP\n\n* Significance: I enjoyed reading this paper, it seems like an important addition to the (old) correlation analysis field, starting with Pearson, continuing with CCA and now neural HGR. It is not groundbreaking earth-shattering, but I would want to see it accepted after editing issues fixed.\n\n* Originality: To the best of my knowledge this is an original work, it highlights important aspects with very good demonstrations. It is a relatively small trick to implement but a well-motivated trick nonetheless.", "was_revised": true, "final_justification": "I have increased the paper's final rating to 5. The authors have assured us in their rebuttal that a major revision has been made to the main issues of this paper which were style and clarity.", "rebuttal": "We sincerely appreciate the reviewer’s careful reading and constructive feedback. Below, we provide detailed responses to each point that requires clarification or revision.\n\n**Strengths And Weaknesses:**\n  - *Quality*:\nWe appreciate the reviewer’s comments regarding the writing quality and overall presentation. We acknowledge that several aspects of the paper can be significantly improved for clarity and flow, and we have addressed them carefully in the revised version. (1) We agree that references to the appendix were too vague. All such references have been revised to clearly indicate what is being shown and where exactly to find it, to make navigation easier for the reader. (2) The submitted version of the Related Work section is a condensed summary of the more complete discussion found in Appendix B, due to space constraints. We acknowledge that it lacks references and sufficient explanation. In the revised version, we replaced it with a rewritten version based on Appendix B, which includes proper citations and concise descriptions of the mentioned methods improving clarity. (3) We introduced HGR and neural HGR in Section 3 to motivate our formulation. However, we agree that this content would be more appropriate as background. In the revised version, we moved this part to the Related Work section and renamed it “Related Work and Background,” so that Section 3 can focus more directly on our contributions. (4) We also thank the reviewer for pointing out formatting and grammatical issues in the footnotes. These are corrected in the revised version. Improving the clarity, structure, and overall presentation of the paper is a priority for the revision.\n\n- *Clarity*: We thank the reviewer for the detailed and constructive feedback regarding the paper’s presentation. We carefully revised the manuscript to address each of the points raised. (1) We acknowledge that the figures are currently difficult to read. We improved them by increasing the font size and enhancing clarity to ensure readability even at standard zoom levels. (2) We agree that the current captions are too minimal. We have rewritten them to clearly describe the results, provided space permits. (3) We recognize the lack of transition between certain sections and agree that this affects the overall flow. We revised the paper to introduce the purpose of each subsection more clearly. For example, to introduce “Consistency of SHGR estimators,” we added the following sentence: “*SHGR is empirically motivated as a rank-based reformulation of the HGR coefficient ensuring robustness to outliers and extreme values. It theoretically relies on copula transformations to preserve dependency structures. We now prove consistency guarantees through convergence results and Renyi's axioms.*”. To introduce the generalization on line 157, we propose adding the following sentence: “*While pairwise correlation analysis is useful, many real-world applications require assessing dependencies between a group of variables and one or more target variables. In a similar manner to the Canonical Correlation Analysis (CCA) and the Randomized Dependence Coefficient (RDC), we propose a principled generalization of our rank-based HGR estimator to this multivariate setting.*”. (4) We agree that the current section titles are confusing. To make the structure clearer, we propose renaming Section 4 to “Experiments” and Section 5 to “Real-World Applications,” which better reflects the distinction between synthetic benchmarks and applied evaluations. (5) This is an important point. The functions used in our simulations were initially detailed in the main paper, but we had to move them to the appendix due to space constraints.  We have moved the pairplots (Figure 5 from the appendix) to the main paper. This figure provides a visual illustration of the simulated dependencies in the bivariate analysis (functions detailed in lines 272–284 in the appendix). If the reviewer has a preference, we would be happy to follow it. We appreciate the reviewer’s suggestions and will make improving readability, structure, and transparency a priority in the revision.   \n\n**Questions**\n1. We acknowledge the need for clarification at line 121. The limitations referred to concern those previously discussed with respect to the neural HGR estimator, particularly its sensitivity to extreme or outlier values, which may impair its ability to capture non-linear dependencies. As stated in lines 119–120: “Using neural networks could lead to overfitting behaviors, such as learning extreme values to artificially inflate the correlation score.”. We added the following indication in the revised version “*The sensitivity of neural HGR estimators to outliers can lead to overfitting behaviors and artificially inflated correlation scores. To address these limitations, we propose a natural extension of the HGR coefficient.*”\n\n2. We agree that we do not use copulas in a formal mathematical sense throughout the paper, but our results rely on a key invariance property of copulas. We use the term “copula transformation” to refer to the application of $F_U(U)$, where $U$ is a continuous random variable and $F_U$ is its cumulative distribution function. This terminology becomes meaningful when considering the copula $C(U,V)$ between two variables $U$ and $V$, for which the invariance property holds: $C(U,V)=C(F_U(U),F_V(V))$. \nThis property justifies, from a theoretical standpoint, that transforming the variables through their marginal CDFs does not alter their dependency structure. We added a sentence on this point in subsection 3.1 to clarify that this insight theoretically justifies the rank-based formulation used in SHGR: “*It is important to note that the copula transformation preserves the dependence between the original vectors $U$ and $V$ (see for instance, \\cite{Nelsen2006}).*”. We also note that this terminology has been used similarly in prior work, including the RDC.\n\n[Nelsen2006] Nelsen, R. B. (2006). An introduction to copulas. New York, NY: Springer New York.\n\n\n3. We acknowledge that the phrasing at line 201 was unclear. By “nonlinear dependencies of order greater than 2,” we intended to refer to dependencies involving more than two variables. We revised the sentence in the paper to avoid confusion.\n\n4. We agree that this point needs clarification. While we know the true correlation before noise, the actual correlation after adding noise is lower and unknown. In our evaluation, we use the correlations computed before adding noise as a reference to assess the robustness of different estimators. This kind of analysis is inspired by [1].\n\n[1] Lopez-Paz, D., Hennig, P., & Schölkopf, B. (2013). The randomized dependence coefficient. Advances in neural information processing systems, 26.\n\n\n5. We appreciate the reviewer’s observation. “MCref” refers to the absolute difference between the estimated maximal correlation and the true (reference) correlation matrix. Thus, lower values indicate better estimation accuracy. This is explained in lines 224–226: “In the analyses below, we generate K = 10 synthetic datasets and compare the estimated correlations from each method to the true, known correlations, referred to as reference correlations. We evaluate the deviation from these reference correlations.”\nWe agree that this should be more clearly stated in the y-axis label, and we revised the figure accordingly.\n\n**Limitations**\n1. We appreciate this excellent suggestion. We agree that neural approximations, including SHGR, can benefit from post hoc interpretability tools. We have included this important point in the discussion section: “*Exploring the interpretability of the learned functions is an interesting direction, and post hoc visualizations could help better understand the fitted transformations.*”. We also plan to explore visualizations of the fitted functions, and if feasible, we will include them in the appendix of the revised version.\n\n2. We agree with this comment and acknowledge that the discussion on computational time was relatively limited in the main paper due to space constraints. A more detailed analysis is provided in Appendix F.5, which includes sections on “Computation time of numerical illustrations,” “Sensitivity to sample size,” and “Sensitivity to the number of variables.” This section was not expanded in the submitted version (e.g., with figures) to prioritize other content. However, the results show that SHGR has computational times comparable to other methods, which is very satisfactory for a neural-based approach. In particular, Figure 9 in the Appendix, which presents runtime in both the bivariate and multivariate settings, has been added to the revised version.\n\nWe sincerely appreciate your careful attention to our work and your helpful suggestions. We hope that our responses and proposed revisions adequately address your concerns. Should you have any further concerns, we would be glad to address them."}]

Author rebuttal:

Meta-review: This paper addresses the fundamental problem of measuring associations between variables. It proposes combining the Spearman rank correlation with the well-established Hirschfeld–Gebelein–Rényi (HGR) maximal correlation, leading to an efficient, modern NN-based estimator that can be applied to a wide range of scenarios, from bivariate cases to correlations between two sets of variables.

The paper is overall well written. The technical contribution is solid and sufficiently novel, and the effectiveness of the proposal has been empirically demonstrated. Although the reviewers initially raised concerns regarding presentation and evaluation, the authors provided strong rebuttals. Most of the reviewers’ concerns have been satisfactorily addressed, and all reviewers are now positive toward acceptance.

I have one suggestion: it would be valuable to include comparisons with mutual information in the experiments, as MI is known to be effective in many practical cases. Nevertheless, this omission does not prohibit acceptance. Since the reviewers have provided constructive feedback, I strongly encourage the authors to revise the manuscript in line with the reviews and rebuttal for the camera-ready version.

Overall, I believe this paper makes a strong contribution, and I recommend its acceptance.
Decision: Accept (poster)