Title: Partial Correlation Network Estimation by Semismooth Newton Methods
Venue: NeurIPS 2025
Accepted: yes
Keywords: graphical model selection

semismooth Newton methods

parallel algorithm

global convergence

$\ell_1$-regularization

convex optimization
Abstract: We develop a scalable second-order algorithm for a recently proposed  $\ell_1$-regularized pseudolikelihood-based partial correlation network estimation framework. While the latter method admits statistical guarantees and is inherently scalable compared to likelihood-based methods such as graphical lasso, the currently available implementations rely only on first-order information and require thousands of iterations to obtain reliable estimates even on high-performance supercomputers. In this paper, we further investigate the inherent scalability of the framework and propose locally and globally convergent semismooth Newton methods. Despite the nonsmoothness of the problem, these second-order  algorithms converge at a locally quadratic rate, and require only a few tens of iterations in practice.  Each iteration reduces to solving linear systems of small dimensions or linear complementary problems of smaller dimensions, making the computation also suitable for less powerful computing environments. Experiments on both simulated and real-world genomic datasets demonstrate the superior convergence behavior and computational efficiency of the proposed algorithm, which position our method as a promising tool for massive-scale network analysis sought for in, e.g., modern multi-omics research.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper focuses on the problem of partial correlation network estimation. The classical approach to this is the graphical lasso problem.\nUnfortunately, as the authors note, the graphical lasso struggles to scale to datasets where p>1000, owing to cubic scaling that results from the KKT condition. This is problematic, as in areas like genetics one routinely encounters problems where p>100000, for which graphical lasso would be prohibitively expensive. \n\nRecently, a framework called ACCORD proposes solving an equivalent problem that is much easier to solve for large p. Existing approaches for solving this problem are based on the proximal gradient method, which while possessing scalable iterations, is slow to converge to an acceptable solution. \nIndeed, a problem with p~300000, required more than 200 node hours of compute.\n\nTo address this the problem, the authors develop a bespoke semismooth-Newton method for solving the equivalent problem. Unlike the proximal gradient approach, the proposed method exhibits local quadratic convergence which ensures rapid convergence to a high accuracy solution once the iterates are close enough to the solution. In addition, global convergence from any initialization is ensured. \nDespite being a Newton type method, the authors show that thanks to the structure of the problem, the resulting subproblems reduce to solving small linear systems or linear complementarity problems.\nNumerical experiments validate the theory in the paper and demonstrates superior performance to the current state-of-art proximal gradient algorithm for the problem.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n- The presentation in the paper is excellent. The authors thoroughly explain the limitations of existing methodology for solving the partial correlation network estimation problem, and carefully explain each component of their algorithm and how it addresses the problem. Similarly, the proofs of the main results are presented clearly and with sufficient detail.\n\n- The algorithm developed in the paper is efficient and comes with strong theoretical guarantees that address the limitations of existing algorithms.\n\n- Numerical experiments show that the proposed algorithm outperforms the existing baseline on a real world large-scale genetics task, which shows its potential to yield better performance on real world tasks that are of interest to practitioners. \n\n**Weaknesses**\n\n- My main issue with the paper is that it misses some important related work. Semismooth Newton methods have been used for solving sparse learning problems for sometime, see for example [1-3].\nThus, the authors should cite these works and clarify how their approach differs from this prior work.\n\n**Overall**\n\nThe paper makes a good contribution by developing a better solver for an important problem that frequently arises in large-scale genomics. I believe the method will be of interest to members of the community working in this area. \nThe main issue I would like to see the authors address is my question about how their method compares prior semismooth Newton methods for large-scale sparse learning.\n\n**References**\n\n[1] Li, X., Sun, D. and Toh, K.C., 2018. A highly efficient semismooth Newton augmented Lagrangian method for solving Lasso problems. SIAM Journal on Optimization, 28(1), pp.433-458.\n\n[2] Lin, M., Liu, Y.J., Sun, D. and Toh, K.C., 2019. Efficient sparse semismooth Newton methods for the clustered Lasso problem. SIAM Journal on Optimization, 29(3), pp.2026-2052.\n\n[3] Zhang, Y., Zhang, N., Sun, D. and Toh, K.C., 2020. An efficient Hessian based algorithm for solving large-scale sparse group Lasso problems. Mathematical Programming, 179, pp.223-263.\n\nQUESTIONS:\n- Can you clarify the relation of your use of the semismooth Newton method in relation to references [1,3] in the above? \nThis would solidify the novelty of the approach taken by the paper and would make connections with prior uses of semsimooth Newton for sparse learning problems.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a semismooth Newton method for partial correlation network estimation, establishing a framework that ensures both local and global convergence while mitigating the high iteration costs of first-order methods. The method reformulates the KKT conditions as a nonlinear equation, employs Bouligand derivatives for global convergence, and addresses linear complementarity problems to optimize iteration direction. Experiments on synthetic and liver cancer genomic data demonstrate the method's quadratic convergence and computational efficiency, achieving a significant speedup over proximal gradient methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The background analysis is comprehensive, focusing on computational bottlenecks in high-dimensional network estimation, such as matrix inversion complexity, with a clearly defined objective.\n2. The innovative framework combines second-order information from semismooth Newton methods with B-derivative globalization, theoretically guaranteeing local quadratic convergence and global stability.\n3. The experimental design is compelling, demonstrating effectiveness and scalability through comparisons with proximal gradient methods, tests across different graph densities, and applications to large-scale genomic data.\n\nWeakness and Questions：\nMajor problems:\n1. Lack of systematic analysis for hyperparameter selection and algorithm switching criteria: The paper does not clarify how regularization parameters (e.g., λ) or line search step sizes are chosen (e.g., via cross-validation). The switching conditions between the two-stage algorithm (local Newton vs. global B-derivative strategy), such as error thresholds or active set rank criteria, lack theoretical justification, potentially affecting convergence stability.\n2. Incomplete ablation studies: Core components (e.g., semismooth Newton iteration, linear complementarity problem solving) are not ablated, precluding quantification of each module’s contribution to convergence speed (e.g., how removing B-derivative globalization affects convergence rate).\n3. Unexplored applicability of quasi-Newton methods: While semismooth Newton methods avoid matrix diagonalization, the paper does not compare quasi-Newton methods (e.g., L-BFGS) for optimizing semismooth functions, missing a cross-validation of computational complexity and convergence properties.\n4. Limited generalizability validation: Experiments are confined to partial correlation network estimation, without verifying the method’s applicability to unstructured optimization problems (e.g., general convex/non-convex functions). Dependencies on problem structures (e.g., row-wise separability) need further clarification.\n\nMinor Problems:\nIntuitive explanations for theoretical derivations (e.g., generalized Jacobian construction) are insufficient, potentially hindering reader comprehension.\n\nQUESTIONS:\nPlease see the detailed comments on Weaknesses and Questions.\n\nLIMITATIONS:\nNo. Please see the detailed comments on Weaknesses and Questions.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies partial correlation network estimation by applying graph lasso within the recently proposed ACCORD framework (Lee et al., 2019). While the method in Lee et al. (2019) is computationally scalable, it faces a bottleneck for graph lasso due to sparse-dense matrix-matrix multiplication. The authors further explore the inherent scalability of this framework by employing semismooth Newton methods and propose a scalable second-order algorithm that achieves superlinear convergence. Theoretically, they demonstrate local quadratic convergence. Simulations and real data applications comparing their method with the proximal gradient method in Lee et al. (2019) are also provided.\n\nSTRENGTHS AND WEAKNESSES:\nStrength: The paper is well-organized with some theory to support their method and demonstrate the phenomenon appears in simulation. The improvement compared to Lee et al., 2019 is significant.\n\nWeakness:\n\n1. The writing of theoretical part is hard to understand. More explanations on the lemma and theorem and clearly state the connection of the lemmas could help improve the readability given the abundant notations crowded in lemmas or theorems. \n\n2.  The numerical part only compare their method with the one in Lee et al., 2019. How about other algorithms and also the algorithms beyond glasso framework? In addition, only convergence rate or computational cost is compared, nothing is\nreported about accuracy. \n\n3. The motivation of the paper is this: Glasso is a great method, therefore, we wish to improve it. However, despite the popularity of the glasso, study on the glasso is largely theoretical, with limited numerical success for high dimensional data (simulated and real). In fact, it was reported that the glasso estimation of the precision matrix tends to be overly sparse, and alternative approaches may have much better performance.    My question is,  if the glasso is not an effective algorithm, how to justify the work that tries to improve its (and only its) computing side? It seems that the more important problem is to improve its accuracy.  In fact, the paper only investigate the computing speed of the proposed method, without investigation on the estimation accuracy.\n\nQUESTIONS:\n1.  While the proposed method and algorithm does significantly improve the cur- rent algorithms based on glasso, it is known that glasso has its own problem in partial correlation network estimation. For instance, it relies on assumption of equal regularization for all nodes but this is not appropriate for real network, and the performance is not good as it may retrieves very small false edges and sensitive to outliers. See some references here:\n\n[1] Huang. S, Jin. J, & Yao.Z. (2016). Partial correlation screening for estimating large precision matrices, with applications to classification.” Ann. Statist. 44 (5) 2018 - 2057.\n\n[2] Cheng, C., Ke, Y., & Zhang, W. (2025). Large Precision Matrix Estimation with Unknown Group Structure. Journal of the American Statistical Association, 1–24.\n\nGiven this, how can you justify such improvement really contribute to the study of partial correlation network estimation? People are more likely to use other methods that has a fair computational cost but more accurate performance.\n\n2. The locally quadratic convergence result is quite attractive. How can we heuristic under stand it? What is the intuition behind this quadratic convergence?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a new semismooth Newton method for efficiently estimating partial correlation networks using an ℓ₁-regularized pseudolikelihood-based framework. Building on the existing ACCORD framework (which itself is scalable but relies on slower first-order methods), the authors derive a second-order optimization algorithm that significantly improves convergence speed and reduces reliance on expensive hardware like supercomputers.\nThe method is applied to both synthetic and real-world datasets, showing improved performance over the state-of-the-art proximal gradient method (ACCORD-FBS) in terms of convergence rate, iteration count, and wall-clock time.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The authors develop a semismooth Newton approach with a strong theoretical foundation.\n2. The algorithm is shown to have locally quadratic convergence, which is a major improvement over linear convergence of previous methods.\n3. The paper provides clear derivations, conditions for global convergence, and strategies for damping and line search.\n\nWeekness\n\n1. The method assumes submatrices like X_A have full column rank. While reasonable in many high-dimensional biological datasets, this may not hold in low-sample or noisy regimes. Include a robustness analysis or empirical study on how violations of this assumption affect convergence and solution accuracy would be helpful\n2. One motivation for pseudolikelihood methods is robustness to non-Gaussian data, but this is not empirically evaluated in the paper.\n\nQUESTIONS:\n1.  How sensitive is the method to the choice of regularization parameter λ, particularly in real-world datasets?\n2. Have you tested the algorithm on non-Gaussian data distributions? How does performance compare in those settings? Include experiments on non-Gaussian synthetic data (e.g., t-distributions or contaminated normals) to validate robustness.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}