Title: SHGR: A Generalized Maximal Correlation Coefficient
Venue: NeurIPS 2025
Accepted: yes
Keywords: Maximal Correlation

HGR

Non Linear Correlation

Neural Network
Abstract: Traditional correlation measures, such as Pearson’s and Spearman’s coefficients, are limited in their ability to capture complex relationships, particularly nonlinear and multivariate dependencies. The Hirschfeld–Gebelein–Rényi (HGR) maximal correlation offers a powerful alternative by measuring the highest Pearson correlation achievable through nonlinear transformations of two random variables. However, estimating the HGR coefficient remains challenging due to the complexity of optimizing arbitrary nonlinear functions. We introduce a new coefficient, satisfying Rényi's axioms, based on the extension of HGR with Spearman's rank correlation: the Spearman HGR (SHGR). We propose a neural network-based estimator tailored to estimate (i) the bivariate correlation matrix, (ii) the multivariate correlations between a set of variables and another one, and (iii) the full correlation between two sets of variables. This estimate effectively detects nonlinear dependencies and demonstrates robustness to noise, outliers, and spurious correlations (hallucinations). Additionally, it achieves competitive computational efficiency through designed neural architectures. Comprehensive numerical experiments and feature selection tasks confirm that SHGR outperforms existing state-of-the-art methods.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a Spearman version to the HGR maximal correlation (SHGR), and the neural architecture to estimate it. Some interesting propositions are made regarding SHGR, as well as links to traditional metrics such as Pearson and copula-based correlations. Finally, extensive simulations are held, on simulated and real datasets, to show SHGR's ability to: (a) detect correlation under non-linear transformation of variables, in both 1-to-1 bivariate relations, p-to-1 and p-to-q correlation matrix; (b) demonstrate its robustness under noise and independence (avoiding hallucinated correlations); and (c) perform quality feature selection.\n\nTo the authors: I would like to say upfront that I love the idea and I am rooting for this paper, so please read my criticism from this viewpoint. If the paper is not accepted please use my comments, though they are many, to improve it.\n\nSTRENGTHS AND WEAKNESSES:\nI would say the paper's strength are its neat and simple idea, the clear definition of the its goal, and extensive simulations to demonstrate the method's strengths. The paper's main weakness is its editing and style, but these can be repaired.\n\nNow in more detail:\n\n* Quality: \n  - Strengths: The paper is of high quality when it comes to experimental setting. They test many baselines, in many scenarios, reporting many adequate results.\n  - Weaknesses: The paper is of a (relatively) low quality in terms of writing and flow, many missing parts which could make it easier on the eye. Examples: (1) Reference to appendix is quite bad, it is always a general \"go search\" statement (l117: \"(illustration in the Appendix)\"). You should be specific on what is illustrated and where exactly to find it in this huge appendix; (2) The entire \"Related Work\" section is without a single reference. At a certain point it feels like name dropping (HSIC, MMD, MIC, GeDI -- all of these are mentioned without even explaining what they do, what the initials are), this should be heavily edited. (3) Then, under section 3 which is supposed to show the authors' work they start with defining HGR and neural HGR (4) Footnotes on page 6 start with lower letters, are grammatically incorrect and start with \"but\".\n\n* Clarity:\n  - Strengths: SHGR itself clearly defined, the motivation is clear, all mathematical expressions are clear.\n  - Weaknesses: (1) The Figures are extremely hard to read, even at zoom 120% I found them hard to read (2) Very lazy captions (we need a little bit more than \"lower is better\") (3) Flow between paragraphs and sections is poor, e.g. l141: \"Consistency of SHGR estimators\" -- Please give the reader one or two lines on what is about to take place, this is an article, not a summary of a uni class. Especially since these propositions are so cool and important to realize. Another example: l157: \"It is often necessary to analyze\" -- again, what are we doing, tell the reader what we just did and what is about to occur. (4) Poor choice of titles: section 4 is called \"Numerical Illustration\" and section 5 is called \"Experiments\" -- did we not just see a (huge!) set of experiments? (5) Missing important details in experiments: e.g. what are your functions? I realize there is little room but one of the most interesting things in these simulations is what functions (e.g. f(U)) you used to simulate the transformations, it is really hard to be impressed without knowing how difficult they would be to approximate for e.g. a simple MLP\n\n* Significance: I enjoyed reading this paper, it seems like an important addition to the (old) correlation analysis field, starting with Pearson, continuing with CCA and now neural HGR. It is not groundbreaking earth-shattering, but I would want to see it accepted after editing issues fixed.\n\n* Originality: To the best of my knowledge this is an original work, it highlights important aspects with very good demonstrations. It is a relatively small trick to implement but a well-motivated trick nonetheless.\n\nQUESTIONS:\nAs said, what would make me increase scores for this paper is more the editing, less the idea or demonstration themselves which I liked, but here are a few things I did not understand:\n\n(1) l121: \"that addresses these limitations\" -- I am not clear on what those limitations are, could you clarify more clearly or point to the exact place in the Appendix where you discuss these and/or demonstrate them?\n\n(2) Why did you mention copulas? Like, at all. Am I correct that in the bottom-line, apart from acknowledging the link between the Spearman correlation (and therefore SHGR) and the Pearson correlation on the copulas, you are not using this insight in any way later on?\n\n(3) l201: What is \"nonlinear dependencies of order greater than 2\"?\n\n(4) l245: Why in this setting \"the true correlation is unknown\", is it because of the noise? Needs to be clearer, you of course know the true correlation, you are comparing to it for your metric.\n\n(5) In Figure 2 the metric made sense, on the y-axis we see \"distance\". Here we see on the y-axis \"MCref\" -- I realize lower is better but I have no idea what this means. What is MCref?\n\nLIMITATIONS:\nThe authors have addressed the limitations of their work, and there do not seem to be any negative societal impact.\n\nTwo points may be better addressed, but this is not that critical:\n\n(1) l328: \"HGR is difficult to interpret\" true but you know, there are tools to at least start exploring what your neural networks have learned once trained, it's a very busy research area, have you tried using any tools to at least visualize the fitted f functions?\n\n(2) l285: \"4.2.7 Computation time\" -- I've been waiting for this paragraph and it was a bit disappointing. Whenever you suggest researchers use a method for calculating a metric with your method which involves neural networks (!) instead of say ACE in R, you should really supply some clear comparisons with a table or graph in the main text. It is very difficult for the reader to summarize from this anecdotal writing that \"SHGR exhibits highly competitive computation times\"...", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-01"}

comments: SUMMARY:
This paper presents a novel adaptation of the HGR correlation measure by incorporating rank based spearman correlation while estimating the maximal correlation coefficient. Authors also present a neural network based encoder architecture for estimating the correlations in both bivariate and multivariate setups. Empirical results demonstrate superior performance on sensitivity to outliers and computational efficiency to alternative approaches in designing correlation coefficients for measuring non linear correlations.

STRENGTHS AND WEAKNESSES:
Strengths - Paper introduces novel theoretical contributions extending HGR from Pearson to spearman correlation with strong empirical evidence. Innovative stacked cross encoder architecture design also allows for simultaneous estimation of multiple correlations avoiding iterative optimizations resulting in computation efficiency. The measure also satisfies Renyi's axioms for non linear dependence measures.

Weaknesses - Paper could benefit from ablation analysis regarding the model choices like encoder architecture, loss functions to indicate the robustness of correlation estimations. The discussion around how does rank transformation of variables impacts the properties of maximal correlation is missing which the paper could benefit from.

QUESTIONS:
1. Paper could benefit from ablation analysis regarding the model choices like encoder architecture, loss functions to indicate the robustness of correlation estimations.

2. The discussion around how does rank transformation of variables impacts the properties of maximal correlation is missing. can you provide  some discussion around conditions under which SHGR may fail or perform poorly compared to classical HGR

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-08

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a novel neural network based correlation based correlation estimator that provides a neural approximation of the HGR estimator. Note that the HGR estimator is known to capture non-linear dependencies that the more commonly used Pearson correlation coefficient does not. However, it does so through an exhaustive search through pairwise comparisons of non-linear mappings of random variables to determine the maximum Pearson coefficient. It thus imposes a high computational burden and does not scale well. (It is reminiscent of the Hausdorff distance metric in that sense and thus equally computationally intense.) \n\nPast work on this problem has recognized the importance of capturing non-linear dependencies using neural techniques. One class of methods does not use the HGR and thus lacks the theoretical backing that HGR provides. Approximations of the HGR have been proposed in the literature but they are sensitive to noise, lack normalization, do not permit significance testing etc. The authors provide a table that compares various existing techniques across multiple assessment criteria. The table shows why a new approach would be required that would cover all the requirements mentioned earlier and thus motivates the authors' proposed SHGR. \n\n\nThe authors propose a neural network based variation on the HGR estimator called the Generalized Neural HGR that limits itself to bivariate correlation matrices, multivariate correlations between a set of predictors and a target, and full correlations between two sets of variables.  It can be thought of as a reformulation of the HGR that is inspired by the Pearson coefficient and copula methods that uses cumulative distributions thus providing robustness to extreme values and non-linear relationships. It replaces raw linearity with rank-based monotonicity which is a way to account for the holistic correlation without the undue influence of extreme values that a linear measure would suffer from.  The authors go through rigorous analysis to show that their proposed estimator has many of the useful properties that HGR does while it is much more tractable computationally. They propose a first estimate of their proposed SHGR that is differentiable, easy to compute and robust to noise. The proposed first estimate motivates them to propose a stacked cross-encoder based architecture that enables parallel computation of multiple correlations which is ideally suited for the multiple correlation exploration required by the HGR. Note that the Canonical Correlation Analysis (CCA) is a special case of the SHGR which is intuitively satisfying. \n\nThey propose a new assessment protocol for assessing maximal correlation based measures based on performance, robustness to noise and hallucinations, significance testing and computational efficiency.\nThey run experiments on real-world and synthetic tabular datasets and show that their proposed estimator clearly improves upon the state of the art in detection accuracy, detection accuracy and stability. They cover a variety of past estimators including those that attempt to approximate the HGR. They enable comparison with existing methods by using the SHGR as a mapping so Pearson coefficients can be compared.  In their discussion section they go into the limitations of their proposed estimator and provide some useful insights for future work. All in all a well written and interesting paper on neural approximation of a computationally complex but effective correlation measure.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1. Well written.(Clarity)\nThe logical flow of the paper is good.  Each section nicely leads into the next. There are clear illustrations as well as copious material in the appendices which helps the reader get the gist before diving into the details. The mathematical proofs are written systematically with concise but helpful textual descriptions. \n2. Good literature review.(Clarity and significance)\nThe authors review linear and non-linear measures of correlation thoroughly. They go through past non-HGR and HGR based methods that capture non-linear dependencies and find that the state of the art falls short in terms of robustness to noise, interpretability etc. They find that every past measure falls short along one or more such requirements. After leading the reader through a systematic review, they finally provide a table that clearly shows the limitations of the state of the art along the new assessment criterial that they have developed. \n3. Well motivated novel correlation measure.(Quality and Originality)\nThe authors nicely motivate their proposed new assessment criteria through their literature review. Once the requirements are laid out in that manner, the authors build upon the HGR step by step by first reviewing the Neural HGR and finding that it is still prone to extreme values and linear dependencies, and then showing how a simple reformulation of the HGR in terms of monotonic dependency enables their proposed SHGR which is computationally tractable while maintaining much of the desirable property of the HGR viz capture of non-linear dependencies. They use their this reformulation to propose a clever neural realization they term the Generalized Neural HGR which enables simultaneous computation of cross-correlations the capturing the spirit of the HGR without its computational burden and vulnerabilities. \n4. Convincing experimental results.(Quality, Significance, Originality)\nThe authors compare their proposed SHGR with a broad set of measures that cover linear and non-linear methods including those that approximate the HGR. Their proposed new measures of efficacy are reasonable as explained above because they constitute a theoretically backed evaluation framework that also intuitively convincing. Their proposed technique performs better than the state of the art along all the proposed new measures of correlation efficacy such as noise robustness, robustness to extreme values, computational efficiency etc. They have a detailed discussion of performance along each of those measures that thoroughly and soundly explains the superiority of their proposed measure. \n5. Good discussion of limitations. (Clarity, Originality)\n\nThe authors point out that while their method outperforms the state of the art, it has room for improvement. They mention overfitting and fine-tuning of parameters as a concern. They point out its lack of interpretability and heavy reliance on numerical optimization. They state that further extension to high dimensional spaces represents a significant and rich avenue for future research. My view is that the authors' discussion of the limitations of SHGR shows a thorough awareness of the strengths and limitations of their proposed approach and makes enough of a case that their current work represents a worthy effort towards capture of non-linear dependencies while meeting multiple efficacy criteria. The limitations they mention are applicable to any neural approximation but given their work it is easy to imagine how they would build upon their current mathematical and architectural framework to address the concerns around overfitting, parameter tuning and interpretability. \n\nWeaknesses\n1. The authors describe a neural HGR before describing their SHGR measure. How much of an innovation over the neural HGR is the proposed measure? Clarifying that would help strengthen the novelty/originality of the paper. The authors do say that the neural HGR is still vulnerable to extreme values and other limitations they have laid out. \n2. How much data does the SHGR require? A neural approximation might have overfitting issues. The authors do mention overfitting in passing in their final discussion but do not seem to directly address the challenges posed by overfitting. I don't necessarily see this as a major weakness since that could be a topic for a future follow up paper.\n\nQUESTIONS:\nThe authors describe a neural HGR before describing their SHGR measure. How much of an innovation over the neural HGR is the proposed measure? Clarifying that would help strengthen the novelty/originality of the paper.\nHow do you decide you have enough training data for your SHGR measure? I see the danger of overfitting to your training data.\n\nLIMITATIONS:\nThe authors adequately cover the limitations of their work in the discussion. I did not see anything about societal impact but their work is largely neutral to societal impact. If anything their proposed measure should help tighten up current efforts to improve alignment of AI with norms.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a new coefficient for a given random variable pair, called SHGR (Spearman HGR), which combines some advantages of Hirschfeld-Gebelein-Rényi (HGR) maximal correlation and the Spearman rank correlation. The authors provided theoretical properties of the SHGR correlation and discussed its estimation based on neural networks. Numerical experiments were presented to support the analyses.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\nThis paper is on an important topic of quantifying the relation between two random variables. The proposed SHGR correlation appears to be novel, although it is related to other formulations, such as copulas. The numerical experiments are comprehensive. \n\n### Weaknesses\n1. The paper did not discuss the details of neural SHGR. specifically:\n      + L140, if directly used as the objective function, would suffer from a biased gradient estimation. This is due to the normalization factors in computing the correlation coefficient $r$.\n      + Sec 3.4, the multivariate generalization can be too arbitrary and lack a good motivation.\n      + L175, the usage of torchsort should be further justified.\n2. Overall, the theoretical properties of SHGR and how these properties guide the estimator design should be discussed in more detail.\n3. The presentation throughout the paper can be greatly improved. For example, before Def. 2, the samples should be paired $\\{(u_i, v_i)\\}$, rather than separate samples of $U$ and $V$. There are also some technical problems, see the question section.\n\nQUESTIONS:\n1. Compared with HGR, SHGR can be more robust to outliers. However, the ranking operation also loses the information about its original numerical values. There is a tradeoff between robustness and keeping accurate information. Can the authors briefly comment when SHGR is favorable to HGR?\n2. The proof in Appendix D.4 (point 6) does not hold. It is not even proving what the proposition states.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}