Title: Nonlinear Laplacians: Tunable principal component analysis under directional prior information
Venue: NeurIPS 2025
Accepted: yes
Keywords: principal component analysis

random matrix theory

spiked matrix models

low-rank estimation
Abstract: We introduce a new family of algorithms for detecting and estimating a rank-one signal from a noisy observation under prior information about that signal's direction, focusing on examples where the signal is known to have entries biased to be positive. Given a matrix observation $\mathbf{Y}$, our algorithms construct a *nonlinear Laplacian*, another matrix of the form $\mathbf{Y} + \mathrm{diag}(\sigma(\mathbf{Y1}))$ for a nonlinear $\sigma: \mathbb{R} \to \mathbb{R}$, and examine the top eigenvalue and eigenvector of this matrix. When $\mathbf{Y}$ is the (suitably normalized) adjacency matrix of a graph, our approach gives a class of algorithms that search for unusually dense subgraphs by computing a spectrum of the graph "deformed" by the degree profile $\mathbf{Y1}$. We study the performance of such algorithms compared to direct spectral algorithms (the case $\sigma = 0$) on models of sparse principal component analysis with biased signals, including the Gaussian planted submatrix problem. For such models, we rigorously characterize the strength of rank-one signal, as a function of the nonlinearity $\sigma$, required for an outlier eigenvalue to appear in the spectrum of a nonlinear Laplacian matrix. While identifying the $\sigma$ that minimizes the required signal strength in closed form seems intractable, we explore three approaches to design $\sigma$ numerically: exhaustively searching over simple classes of $\sigma$, learning $\sigma$ from datasets of problem instances, and tuning $\sigma$ using black-box optimization of the critical signal strength. We find both theoretically and empirically that, if $\sigma$ is chosen appropriately, then nonlinear Laplacian spectral algorithms substantially outperform direct spectral algorithms, while retaining the conceptual simplicity of spectral methods compared to broader classes of computations like approximate message passing or general first order methods.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces the Nonlinear Laplacian, a method within the PCA family of algorithms, for detecting and recovering low-rank signals from noisy data. The technique is specifically designed for scenarios where the signal is known to be highly correlated with the vector of all ones, leveraging this bias as prior information. The authors demonstrate that by applying and tuning the Nonlinear Laplacian, the largest eigenvalue corresponding to the low-rank signal is isolated as a distinct outlier from the other eigenvalues, enabling its detection.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The paper is well-structured, and the problem is introduced clearly. \n- The simple experiment and the analytical results support the claim.\n\nWeaknesses:\n- Some of the paper's conjectures could be tested with an experiment (see question #2).\n- The comparison with prior work for the $\\beta>0$ case is not clear. As noted in the paper's limitations, the AMP algorithm can outperform their method on the chosen evaluation data. It is also unclear why the authors claim AMP is \"more complex,\" as no analytical or wall-clock time comparison between the two methods is provided.\n- Lack of experiments to show the applicability of their method.\n\nQUESTIONS:\nA experiment with a real-world dataset would strengthen the practical aspect of the work.\n\nCould you provide an experiment to test Conjecture 3.7?\n\nRegarding the generalized directional priors mentioned as future work in Appendix F.2: How would the method's performance be affected when using a more general prior instead of the all-ones vector? Specifically, could you provide a high-level intuition on how changing the direction of the prior would impact the phase transition threshold, $\\beta$? Is this direction handled in the previous works introduced in your paper?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

comments: SUMMARY:
The paper studies the problem of rank-one spiked model when a prior directional information of the signal $x$ is known. The authors propose a family of spectral methods using the nonlinear Laplacian with some nonlinear mapping $\sigma$. The authors derive exact phase transitions and detection thresholds in this scenario, and shows a lower detection threshold compared to the vanilla BBP phase transition.

STRENGTHS AND WEAKNESSES:
Strengths:

1.	The paper is well-written, clearly organized and easy to follow. The results using nonlinear Laplacians are interesting to me.
2.	The theorems are well presented and are sound for the parts that I checked.

Weakness:

1.	The optimal detection threshold is not known and only numerically computed for family of nonlinear functions $\sigma$.

QUESTIONS:
1.	It would be very interesting to study the sharp detection threshold optimized over all possible transformation function $\sigma$. Does the authors have an intuition what the optimal function $\sigma$ could be?

LIMITATIONS:
See "Strengths and Weakness".
TITLE: Official Review
CLARITY: 4
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies a model called \"Sparse Biased PCA\". Here, one observes $Y = \\beta \\sqrt{n} xx^\\top  + W$ where $x$ is an unknown signal vector and $W$ is i.i.d. Gaussian noise. The author consider a sparse prior for the signal $x$ where approximately $p_n$ fraction of the entries are non-zero where  $\\log^6(n)/n \\ll p_n \\ll 1$. Moreover, the mean of the non-zero coordinates is positive. The authors focus primarily on the detection problem, where the goal is to test if $\\beta = 0$ or if $\\beta$ takes a given non-zero value. \n\nIntuitively, one expects that the largest eigenvector of $Y$ can help us detect the signal. Since the signal prior has positive mean, the vector $Y 1$ might also be helpful. The authors propose to combine these two sources of information using the non-linear Laplacian matrix $L = Y + \\mathrm{diag}(\\sigma(Y \\cdot 1))$ where $\\sigma: \\mathbb{R} \\rightarrow \\mathbb{R}$ is a user-specified function. \n\nThe primary contributions of the authors are as follows:\n\n1) The authors characterize asymptotic spectrum of the non-linear Laplacian matrix (as $n$, the dimension tends to infinity). This includes a characterization of the bulk spectrum and the outlier eigenvalues (if any). This provides a consistent test for the problem, provided that $\\beta$ is above a critical threshold. \n\n2) A special case of their model is the Gaussian planted submatrix problem. The authors show that their proposed test succeeds when $\\beta > 0.79$. On the other hand, any test based on the eigenvalues of $Y$ alone would require $\\beta > 1$, and any test that depends only on $Y1$ will always fail. Hence, the non-linear Laplacian, which combines both these sources of information, improves on both of them. \n\n3) The authors also explore some numerical approaches to find a good $\\sigma$ function.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1) The paper is very well-written and clear. The authors have made a commendable effort to include statements of random matrix theory results and other mathematical preliminaries to make the paper self-contained. \n\n2) I found the overall message of the paper quite appealing: there are regimes where neither the leading eigenvector of $Y$ nor the vector $Y1$ by themself can solve the detection problem, but when combined together in a non-linear Laplacian matrix, they can. This seems quite original. \n\nWeaknesses:\n\n1) The paper seems to have a few \"loose ends\". I might be mistaken, but it is quite plausible that some of these \"loose ends\" can be resolved using existing techniques:\n-- The authors prove their result for a \"theoretically convenient\" modification of the Laplacian matrix rather than the Laplacian matrix.\n-- A characterization of the eigenvector overlap is missing.\n\nQUESTIONS:\nHere are some suggestions and questions:\n\n1) There has been some work in combining linear and spectral estimators for generalized linear models, which seems related. It would be good to cite and discuss the relationship. See: M. Mondelli, C. Thrampoulidis, R. Venkataramanan, \"Optimal Combination of Linear and Spectral Estimators for Generalized Linear Models\"\nThere are, of course, important differences: the work by Mondelli et. al. combines a spectral estimator and a linear estimator, whereas this paper incorporates a \"linear estimator\" into the matrix and computes a spectral estimator on the resulting matrix.\n\n2) It would be good to discuss the bottleneck in proving Theorem 3.3 for the Laplacian matrix rather than the \"compressed Laplacian matrix\". If there are no significant bottlenecks, it would be good to tie up this loose end. \n\n3) Regarding the optimal $\\sigma$: the optimization problem over $\\sigma$ has some similarities to problems in prior work. See: Optimal Spectral Initialization for Signal Recovery with Applications to Phase Retrieval by Wangyu Luo, Wael Alghamdi, Yue M. Lu. \nIs there a possibility of adapting their arguments to find the optimal $\\sigma$ mathematically?\n\n4) Regarding Eigenvector Overlap: I might be mistaken but a simple argument on top of the results of the authors should also give a characterization of the eigenvector overlap. Let $\\lambda_{n}(\\beta)$ denote the largest eigenvalue of $Y$, viewed as a function of $\\beta$. Using the standard argument to differentiate eigenvalues (see Section 4 in https://terrytao.wordpress.com/2010/01/12/254a-notes-3a-eigenvalues-and-sums-of-hermitian-matrices/), one can show that the derivative of $\\lambda_n(\\beta)$ with respect to $\\beta$, will be the squared overlap between the signal and the eigenvector. One can swap the limit and derivative because the largest eigenvalue is a convex function of beta, and conclude that the overlap should converge to the derivative of the limit of the largest eigenvalue. \nAnother option could be to try the argument given in Proposition 2 of https://arxiv.org/pdf/1702.06435.\n\n5) Is there an absolute value missing on Pg 23, first equation on the page (RHS)? \n\n6) Section C: If I am not misunderstanding, I think some of the arguments here can be simplified. The basic task seems to be to show that the root of $G_n(\\lambda) = 1$ converges to the root of $G(\\lambda) = 1$. Because $G_n, G$ are strictly decreasing functions, one doesn't need uniform convergence for this. If the function $G = 1$ has a root $\\theta$, then for any for any $\\epsilon>0$, $G(\\theta - \\epsilon)$ will be positive and $G(\\theta+\\epsilon)$ will be negative. For sufficiently large $n$, these signs will be reflected in $G_n$. This means that $G_n$ must cross zero at least once in $(\\theta-\\epsilon, \\theta+\\epsilon)$, and this root must be unique because $G_n$ is strictly decreasing.\n\n7) I also think it would be helpful to motivate the model studied in the paper in a better way. Currently, the reader is left wondering where did this specific model came from and why one should study it?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a new familty of algorithms for detecting and estimating a rank-one matrix from a noisy observation, assuming access to prior information about the hidden signal's direction. Specifically, they consider a non-linear Laplacian constructed using a non-linear functional $\\sigma$, and characterize the spectral properties of such non-linear Laplacian matrix. As an application of their main theorem, they show that the proposed algorithms outperform direct spectral methods for the Gaussian Planted Submatrix problem.\n\nSTRENGTHS AND WEAKNESSES:\nThis is a nice theoretical paper exploring the use of non-linear Laplacian for detecting and estimating a rank-one matrix from noisy observation. The presentation is neat and clear, and the theoretical results are pretty solid. My main concern is that the proposed algorithm does not seem to offer clear advantages over previous algorithms, and I would love to see a more detailed discussion in this regard.\n\nQUESTIONS:\n1. I am wondering if this result can be generalized to non-Bernoulli prior, say, a prior distribution that has non-zero expectation? \n2. I am not quite sure if the proposed algorithm is much simpler than the AMP algorithm: AMP is an iterative algorithm that involves mainly matrix-vector multiplication and entry-wise transformation. Specifically, spectral initialization is not needed for the sparse biased PCA setting that is considered in the current paper. On the other hand, the proposed algorithm requires taking the top eigenvalue of a matrix. \n3. As shown in [EAKJ20], the information-theoretic detection threshold for rank-one spiked model is much smaller than the current theoretical guarantee. I think the authors should discuss this point in the paper. \n4. For the motivation of their proposed algorithm, the authors gave the following explanation: \"both the spectrum of Y and the vector Y 1 carry information about x\". I am wondering if there is a specific reason for constructing a diagonal matrix using Y 1 and using it to assist recovery? In other words, I am curious about how the authors came up with this specific proposal.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-09"}