Title: Low-Rank Graphon Learning for Networks
Venue: NeurIPS 2025
Accepted: yes
Keywords: low-rank graphon

subgraph counts

connection probability matrix

nonparametric statistics

network analysis
Abstract: Graphons offer a powerful framework for modeling large-scale networks, yet estimation remains challenging. We propose a novel approach that leverages a low-rank additive representation, yielding both a low-rank connection probability matrix and a low-rank graphon--two goals rarely achieved jointly. Our method resolves identification issues and enables an efficient sequential algorithm based on subgraph counts and interpolation. We establish consistency and demonstrate strong empirical performance in terms of computational efficiency and estimation accuracy through simulations and data analysis.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses the problem of estimating the graphon function  $f: [0,1]^2 \\to [0,1]$ that governs the edge probabilities in an undirected random graph. Each of the $n$ nodes in the graph is associated with an unobserved latent variable $U_i$, drawn independently from the uniform distribution on $[0,1]$. Conditional on these latent variables, the presence of an edge between nodes $i$ and $j$ is determined by a Bernoulli trial with success probability $f(U_i, U_j)$.\n\nThe authors propose new algorithms for estimating the graphon $f$, and establish that, under certain structural assumptions, the estimator $\\hat{f}$ achieves a convergence rate of  $\\sqrt{\\frac{\\log(n)}{n}}$ in the supremum norm. A key assumption is that the true graphon $f$ admits a decomposition as a finite sum of $r$ orthogonal, rank-one symmetric functions, i.e., functions of the form $ G(u)G(v)$, where each $G$ is a univariate function. Furthermore, it is assumed that the functions $G$ are Lipschitz continuous and that one of them is known to be monotone increasing. These structural constraints play a central role in enabling consistent and computationally efficient estimation.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. I enjoyed reading this paper. It is well polished and pleasant to read.  \n2. All the math details that I checked appear to be correct.  \n3. The obtained rates of estimation match the best known rates in the problem of graphon estimation.\n\n**Weaknesses**\n1. The conditions imposed on the true graphon appear to be quite strong, especially when it comes to considering the case $r > 1$. Indeed, both the condition that the $G_i$s are orthogonal and that they are orthogonal to the constant function is a strong assumption that has no reason to be satisfied.  \n2. The condition 3.7 that essentially requires that $G_1$ is an increasing function is overly restrictive. I cannot think of a relevant example of application in which such a condition could be acceptable.  \n3. The paper would be much better if more intuitions were given about the construction of the estimators. Here are some examples of intuitive explanations that can be added, but of course this is just a sample:  \n   a. The estimator provided at line 150 is based on the fact that the $i$th order statistic of the sample of uniform in $[0,1]$ random variables is close to $i/(n+1)$.  \n   b. The rate of convergence $n^{-1/2}$ corresponds to the standard nonparametric rate $N^{-\\beta/(2\\beta + d)}$, where $\\beta$ is the smoothness, $d$ is the dimension and $N$ is the sample size. Since in the problem under consideration the observations are the edges, the sample size $N$ is $n^2$. The Lipschitz assumption implies that $\\beta = 1$, and the dimension is obviously $d = 2$.\n\n4. Many relevant references on graphon estimation are missing. I am not claiming that these references have a direct connection to the obtained results, but they have to be discussed in order to situate the current work in the landscape of already existing results. The papers I have in mind are:\n- E. Donier-Meroz, A. Dalalyan, F. Kramarz, P. Choné, X. D'Haultfoeuille (2023). *Graphon Estimation in bipartite graphs with observable edge labels and unobservable node labels*. arXiv:2304.03590  \n- O. Klopp and N. Verzelen. *Optimal graphon estimation in cut distance*. *Probability Theory and Related Fields*, 174(3), pp. 1033–1090 (2019)  \n- O. Klopp, A. Tsybakov and N. Verzelen. *Oracle inequalities for network models and sparse graphon estimation*. *Annals of Statistics*, 45(1), pp. 316–354 (2017)\n\nQUESTIONS:\n1. Could you provide some hints on why the conditions required in Section 3 could be acceptable in practice?  \n\n2. How should the equations (5) be solved?  \n\n3. You claim that your proposal based on counting subgraphs is computationally efficient. I did not find the justification of this claim in the main body. Did I miss something? If not, and if it is a crucial property of your method as compared to past work, it might be beneficial to move it to the main body of the paper.\n\n4. Is it easy to extend the results of Section 3.2 to the case where $r$ might depend on $n$ and tend to infinity when $n$ tends to infinity? If yes, how then does $r$ enter into the rate?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-21"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper offers a framework for estimating the connectivity matrix and the underlying function of a low-rank graphon through subgraph counts (AKA motifs). The authors' main theorem is an error guarantee in the sup-norm for both the connectivity matrix $P$ (Theorem 3.6) and the graphon function $f$ (Theorem 3.8). The former is noted to match pre-existing minimax error rates for smooth graphon estimation. Additionally, the authors perform simulation studies to compare their method with various prior algorithms, in terms of mean-squared errors and runtimes.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper's main strength is presenting and addressing graphon estimation in a new framework: namely, estimating both the graphon connectivity matrix and the underlying function, through efficient algorithms that leverage motifs. Additionally, the numerical experiments are extensive, and demonstrate their method is advantageous (at least in a very low sparsity regime of $\\rho = 1/\\sqrt{n}$). The algorithm is clean, scalable, and intuitive. \n\nThe main weaknesses are:\n\n1. Unclear scope for their model of low-rank graphons, which seem quite similar to the well-studied case of piecewise Holder-smooth graphons. Experiments show that prior methods are mostly competitive even for the authors' chosen simulated graphons (see my questions below), so it is not clear what sort of graphon problems really fit their specific theoretical assumptions and algorithm.  \n\n2. Lack of real-world experiments, which appear in Appendix A.6 but in very limited form (see my questions) below.\n\nQUESTIONS:\n- Literature review seems sparse: can you cite more papers on matrix sensing / matrix completion, as well as Holder-smooth graphon estimation? Putting in an appendix would be okay. \n\nChen, Yudong, et al. \"Completing any low-rank matrix, provably.\" The Journal of Machine Learning Research 16.1 (2015): 2999-3034.\n\nKLOPP, OLGA, ALEXANDRE B. TSYBAKOV, and NICOLAS VERZELEN. \"ORACLE INEQUALITIES FOR NETWORK MODELS AND SPARSE GRAPHON ESTIMATION.\" The Annals of Statistics 45.1 (2017): 316-354.\n\nJalan, Akhil, et al. \"Transfer Learning for Latent Variable Network Models.\" The Thirty-eighth Annual Conference on Neural Information Processing Systems.\n\n- Eq. 1 seems to imply that the graphon is “almost” Lipschitz, since each G_k is L^2, so it must be bounded outside a set of measure zero in [0,1]. So, I understand the main difference between your model and smooth graphons to be this measure zero set. \n\nHow critical is this set of measure zero for \n(a) real-world modeling applications such as the biological networks you cite, and \n\n(b) theoretical work such as your estimation algorithms, as compared to USVT or Gao’s averaging technique? \n\nSome discussion on both points would be helpful.\n\n- The discussion of theoretical contributions on line 42 begins with Lemmas A8-A10. Could these be consolidated into a single statement and put in the main body? If they are a core contribution of the work, I feel they should be at least discussed in the main text, even if space constraints prevent the proof from being included therein. \n\n- Line 126: Can Xu 2018 not also take advantage of the rank-1 structure to do better than O(n^3) time? If I recall, Xu’s algorithm is bottlenecked by computing the SVD itself. \n\n- Theorem 3.1: You are citing previous minimax lower bounds later for smooth graphon estimation, so I understand these are a special case of your model. What is an example of a graphon that falls into your model but does not meet the piecewise Holder assumption? \n\n- Theorem 3.2: The use of a canonical G_1 seems similar to the idea of estimating graphons up to equivalence relations as defined in e.g. Klopp 2017, among others. Can you explain the differences?  \n\n- Line 187: What is the citation is needed for the complexity of matrix multiplication? I think would be more readable if you just wrote $O(n^\\omega)$. \n\n- Line 193-194: You emphasize that your method does not require smoothness assumptions, but Theorem 3.2 requires a Lipschitz G_1, which would imply a Lipschitz f in the rank-1 case. Lipschitz smoothness is in fact stronger than Holder smoothness as in Xu 2018. Can you clarify how your method does not require smoothness? \n\n- Assumption 3.5(iii): Similarly, the assumption of bounded G_k functions, along with them being in L^2, seems to imply a lot of regularity, since the G_k must be Lipschitz. How precisely does your model differ? I apologize for being repetitive. \n\n- Table 1 would benefit from bolds / italics to emphasize the best performing methods in each scenario \n\n- Table 1: Is the runtime an average across all 100 runs? Reporting stdev or quantile interval of runtimes would be helpful. \n\n- Table 1: It seems USVT is comparable in MSE across all scenarios except #4; however, even here it is comparable in terms of max error. Why is USVT so effective in all of these examples? Are there any motivating examples of graphons where USVT is clearly worse? \n\n- Table 2: Why does sparsity make your method so much better than others? Discussion would be helpful. \n\n- Table 2: It would be helpful to see how methods’ performance scales with sparsity, ranging from $\\rho = 1/\\sqrt{n}$ up to $\\rho = 1$. \n\n- Appendix A6: Can you report the MSE numbers and comparisons to your baseline methods (USVT, neighborhood smoothing, etc) for these experiments? With the figures as they are now, it is very unclear why your method should be preferable for these real-world graphs.\n\nLIMITATIONS:\nThere is not much discussion of the limitations of their work, aside from a suggestion on future work in the conclusion. I would suggest a deeper discussion of: \n\n- The unique aspects of their model as opposed to smooth graphon models. \n\n- The real-world applicability of their model, perhaps to the graphons they study in Appendix A.6 or others. \n\n- A more extensive discussion of why their method only seems to outperform baselines in a sparse graphon regime.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 1, "ORIGINALITY": 2, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors proposed a methodology that estimates both the connection probability matrix and its underlying graphon at low computational cost. The approach leveraged the rank-r eigen-decomposition of a graphon into r eigenvalues and eigenfunctions. These quantities were recovered by counting cycles and paths of prescribed length, which leads to an estimate of the connection probability matrix. Identification issue was handled via sorting, and linear interpolation was used to extend the discrete estimates of the eigenfunctions to a continuous graphon.\n\nSTRENGTHS AND WEAKNESSES:\n<Strengths>\n\n1. Mathematically well-founded:\nThe methodology was supported by rigorous proofs; e.g., Theorems 3.2, 3.6, and 3.8 establish convergence rates.\n\n2. Solid methodology:\nTo my knowledge, the authors introduced a new methodology to estimate both a connection probability matrix and graphon of a graph while remaining computationally efficient.\n\n<Weaknesses>\n\n1. Writing is hard to follow\n- The word “learning” in the title is potentially misleading; the work concerns “estimation”, not machine learning.\n- Key background concepts such as graphon, identification issue, connection probability matrix were introduced too briefly.\n- No visual aids were provided to help readers grasp the pipeline.\n- Several derivations, particularly those involving interpolation, omitted explanatory text between equations.\n- Practical relevance was under-explained: concrete downstream tasks or benchmarks would help. \n\n2. Insufficient experimental verification\n- Table 1 and Table 2 were inconsistent: Table 2 omitted runtime results and the entries for IDs 1, 6, and 7.\n- Experiments appeared limited to rank less than or equal to 3.\n- Graphon estimation was assessed only qualitatively (Fig. 1); no quantitative error metric on graphon (not connection probability matrix) is reported.\n- Real-data experiments were presented, but there is no evaluation of how accurate the estimated graphons or matrices are.\n\n3. The source code is not available.\nThus, there is limitation in reproducing the proposal.\n\nQUESTIONS:\n1. How can this work tangibly benefit the AI research community or How can this work be related to the scope of the conference?\n\n2. Can you provide the missing results for IDs 1, 6, 7 and the runtime column in Table 2?\n\n3. What happens if the target graphon is high-rank? Is the method scalable or accurate in that regime?\n\n4. Is low-rank (less than or equal to 3) approximation applicable to real-world data?\n\n5. Could you compare computational cost as a function of rank?\n\n6. Could you report quantitative graphon-estimation errors (e.g., norm)?\n\n7. How do you validate the quality of the real-data estimates?\n\nLIMITATIONS:\n- Unclear practical impact\n- Limited experimental verification", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe author(s) propose a low-rank graphon modelling for networks. The estimation approach reconcile the estimation of the graphon function and the corresponding connection probability matrix. The author(s) propose a sequential efficient algorithm baed on subgraph counts and interpolation. They also prove theoretical consistency results in sup-norm for the connection matrix and the graphon function estimation. The proposed approach is validated on simulated data examples and on real world network data.\n\nSTRENGTHS AND WEAKNESSES:\nstrengths:\n\n1. A novel modelling approach of network data via low-rank graphon modelling\n2. Estimating jointly the graphon function and the connection matrix via an efficient algorithm based on subgraph counts and interpolation\n3. Strong theoretical results on sup-norm consistency of graphon function estimation and connection matrix.\n4. Scalable algorithm that has favourable time complexity over its neighbours.\n5. The assumptions in the theoretical framework often matches with those in SBM or RDPG network models.\n\nweakness\n\n1. For clarity and understanding, the section on low-rank modelling with rank $r > 2$ should be rewritten especially the graphon function estimation. The subgraph count based idea along with sorting may facilitate reader's understanding with a toy example.\n2. Some motivation/discussion was lacking why graphon function estimation would be useful beyond estimation the connection matrix.\n3. As it seems, the motivational example of $r=1$ uses the method of moments approach in estimating the $p_{ij}$. Some discussion on that would have been good as method of moments has been used in network parameter estimation earlier in the literature as well.\n\nQUESTIONS:\nI think the low-rank modelling approach using graphon models for networks is a novel formulation indeed. Especially, the contribution in the direction of joint estimation of connection matrix and the graphon function itself is a long-standing problem in the literature of networks. So indeed this is valuable.\n\nI have a couple of minor queries for the author(s)\n\n-I may have missed this point but how does one ensure the uniqueness of the estimated graphon function $\\hat{f}^{+}(u,v)$ ?\n-What happens if there are ties in the degree sequence $\\lbrace d_i\\rbrace_{i=1}^n$? How do you do the sorting then i.e. how breaking of ties is implemented?\n-Does the additional assumption 3.7 means in certain cases one can only estimate $P$ and not the graphon function $f$?\n-Why ratio is being used in (A.5) to select unknown rank $r$ instead of the difference $|\\lambda_{k+1}-\\lambda_k|$? \n-Theorem 3.8 says the sup-norm rate is independent of $r$ but practically speaking, not all networks can have same rate of estimation since the rank may vary. Am I missing something here?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}