Title: A Few Moments Please: Scalable Graphon Learning via Moment Matching
Venue: NeurIPS 2025
Accepted: yes
Keywords: Graphon estimation

Moment matching

implicit neural representations

MomentMixup
Abstract: Graphons, as limit objects of dense graph sequences, play a central role in the statistical analysis of network data. 
However, existing graphon estimation methods often struggle with scalability to large networks and resolution-independent approximation, due to their reliance on estimating latent variables or costly metrics such as the Gromov-Wasserstein distance. 
In this work, we propose a novel, scalable graphon estimator that directly recovers the graphon via moment matching, leveraging implicit neural representations (INRs). Our approach avoids latent variable modeling by training an INR--mapping coordinates to graphon values--to match empirical subgraph counts (i.e., moments) from observed graphs.
This direct estimation mechanism yields a polynomial-time solution and crucially sidesteps the combinatorial complexity of Gromov-Wasserstein optimization. 
Building on foundational results, we establish a theoretical guarantee: when the observed subgraph motifs sufficiently represent those of the true graphon (a condition met with sufficiently large or numerous graph samples), the estimated graphon achieves a provable upper bound in cut distance from the ground truth. Additionally, we introduce MomentMixup, a data augmentation technique that performs mixup in the moment space to enhance graphon-based learning. 
Our graphon estimation method achieves strong empirical performance--demonstrating high accuracy on small graphs and superior computational efficiency on large graphs--outperforming state-of-the-art scalable estimators in 75\% of benchmark settings and matching them in the remaining cases. Furthermore, MomentMixup demonstrated improved graph classification accuracy on the majority of our benchmarks.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes MomentNet, an implicit-neural-representation (INR) estimator that fits a graphon by matching a fixed set of induced-motif densities (“moments”) from observed graphs.  This approach is claimed to be more scalable and avoids the need for latent variable estimation. They claim a finite-sample bound on the cut distance under an assumption of INR. And they also introduce MomentMixup, a data augmentation technique operating in moment space. Experiments on synthetic graphons and four small graph-classification datasets show modest gains over USVT and IGNR and parity with SIGL.\n\nSTRENGTHS AND WEAKNESSES:\n### **Strengths**  \n1. **Clear Idea, Simple Pipeline** : Replacing Gromov-Wasserstein optimisation with direct moment matching is conceptually clean and easy to implement. It reduce optimization overhead present in methods like IGNR [1].\n2. **Continuous Representation** – Using an INR yields a resolution-free graphon that can be sampled at an arbitrary size.  \n\n### **Weaknesses:**\n\n1. **Vacuous Theoretical Guarantees** I carefully checked the theory and the provided supplementary material. Theorem 1 relies fundamentally on **Assumption 1**, which requires the INR to drive every motif error below $\\delta$. However: \n\n\ti) No optimization bounds are provided showing when this assumption can be satisfied. \n\n\tii) No sample complexity analysis demonstrates how many samples are needed in practice. \n\n\tiii) The condition requires $\\Omega(k^2/\\delta)$ samples per motif but provides no guidance on achievable $\\delta$ values.\n\n2. **Severe Scalability Limitations**  ($O(n^4)$ Dense Complexity) I read and checked the paper and Appendix D. There's a fundamental scalability problem: \n\n\ti) **Dense graphs**: Even with full parallelization, Stage 1 costs $\\Theta(n^4)$ due to 4-node graphlet counting. \n\n\tii) **Comparison with SIGL**: SIGL achieves $\\Theta(n^2)$ complexity, making it asymptotically superior for dense graphs. So the **\"Scalable\" claim is misleading**: The method only scales well for sparse graphs where degree $d = O(1)$.\n\n3. **Mixed and Questionable Empirical Results**\n\n\ti) **Graphon estimation**: MomentNet loses to SIGL on $4/13$ benchmark graphons.\n\n\tii) **Statistical significance**: MomentMixup improvements of $\\leq 2$ percentage points fall within $1\\sigma$ error bars on most datasets.\n\n\tiii) **No ablation studies**: No analysis of motif selection impact or dictionary size effects.\n\n4. **Issuses on Experimental Validation**\n\n\ti) **Limited motif analysis**: While ORCA can handle 5-node motifs, authors use only 9 motifs up to 4 nodes without ablation studies on motif dictionary size or selection impact.\n\n\tii)  **Limited scale**: Largest experiment far below the range where the number of nodes becomes prohibitive ($n > 5000$ nodes).\n\n\tiii) **Missing dense graph evaluation**: No experiments on realistically large dense networks.\n\n5. **Poor Writing and Citation Issues** During review, I found that,\n\n\ti) **Duplicate citations**: References [27] and [28] are identical papers with different labels.\n\n\tii) **Wrong citation item**: Reference [22] is in wrong format and seems to mix two papers in one item.\n\n\tii) **Notation inconsistencies**: Switching between $W$, $W^*$, $\\hat{W}_{\\theta}$ without clear distinction.\n\n6. **Methodological Concerns**\n\n\ti) **Inverse weighting scheme**: The loss function (Eq. 5) uses $w_i = 1/m_i$ weights without theoretical justification, potentially causing instability for small moment values.\n\n\tii) **Limited motif selection guidance**: No principled approach for choosing appropriate moments for different graph types.\n\n\tiii) **MomentMixup justification**: While Proposition 1 shows convex combinations of graphons differ from moment combinations, practical implications for data augmentation remain unclear.\n\n### Reference\n[1] Xia, X et al. (AISTATS 2023) Implicit Graphon Neural Representation.\n\n[2] Ali Azizpour et al. (AISTATS 2025) Scalable Implicit Graphon Learning.\n\nQUESTIONS:\n1. **Theoretical validity**: Can you provide empirical evidence of when Assumption 1 holds? What are typical motif errors δ achieved in practice, and how do they relate to the theoretical requirements?\n\n2. **Large-scale evaluation**: Can you demonstrate performance on graphs with >5000 nodes to validate scalability claims? Include timing breakdown between Stage 1 (motif counting) and Stage 2 (INR training).\n\n3. **Motif selection**: What is the rationale for using only 4-node motifs when 5-node motifs are available? How does motif dictionary size affect estimation quality and computational cost?\n\n4. **Dense graph performance**: How does the method perform on dense graphs (edge density >0.5) with realistic node counts?\n\n5. **Statistical significance**: Can you provide paired t-tests for MomentMixup improvements and confidence intervals for comparisons?\n\nLIMITATIONS:\n### **Major Technical Issues**\n\ni) **Assumption 1 Analysis**  \n   The theoretical guarantee fundamentally depends on the INR’s ability to approximate empirical motif densities within tolerance $\\delta$. Refer Weakness 1.\n\nii) **Complexity Contradiction**  \n   The paper claims scalability but reveals $\\Theta(n^4)$ complexity for dense graphs. Refer Weakness 2. For the dense regime, we have:\n   $\n   T = O \\Bigl(\\bigl\\lceil P / M\\bigr\\rceil n_T^4 + N_e \\ (L \\cdot C_{INR} + |\\theta|)\\Bigr)\n   $\n   while SIGL [2] gives\n   $\n   T = O(P \\cdot N_e \\cdot n_T^2).\n   $\n   This makes SIGL asymptotically faster for large dense graphs, contradicting the main scalability claim.\n\n### **Limitations Assessment**\n\nThe authors acknowledge some limitations but miss critical issues:\n\ni) **Unaddressed**: Theoretical assumption verification, large-scale evaluation gaps, motif-selection methodology.\n\nii) **Insufficient**: No discussion of $\\Theta(n^4)$ complexity implications or energy consumption for large graphs.\n\n### **Writing Issues**\n\nSee Weakness 5: duplicated citations for [27] and [28], wrong reference [22]; unclear symbols.\n\n### **Review Summary**\n\nAccording to the above weaknesses and limitations, I maintain a Borderline Reject rating as there are several fundamental issues that prevent this work from meeting the NeurIPS standards before being resolved.\n\n### Reference\n\n[1] Xia, X et al. (AISTATS 2023) Implicit Graphon Neural Representation.  \n[2] Ali Azizpour et al. (AISTATS 2025) Scalable Implicit Graphon Learning.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes MomentNet, a scalable graphon estimator that employs moment matching with implicit neural representations (INRs). The method estimates graphons by matching empirical subgraph counts (\"moments\") derived via counting from observed graphs. The authors demonstrate theoretically that when the neural network can accurately approximate the motifs, the overall graphon estimation error remains bounded. Through moment matching, they circumvent computationally expensive Gromov-Wasserstein loss estimations.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths:\n\n**Overall:**\n\nThe central insight connecting motif approximation to graphon estimation quality (Theorem 1) is elegant and well-motivated. The idea of bypassing expensive Gromov-Wasserstein optimization through direct moment matching represents a meaningful algorithmic contribution. The paper is generally well-written.\n\n**Clarity** while good in general could be improved at two points:\n- ORCA and graphlets are fundamental to the approach but barely explained.\n- Strange wording: l14-15 “the estimated graphon achieves a provable upper bound in cut distance from the ground truth”\n\n**Reproducibility:** Code is provided and results seems reproducible \n\n## Weaknesses:\n\n**Critical moments-selection limitation not addressed:**\nThe authors acknowledge that relying on pre-selected moments is a major limitation but defer this to future work. While I appreciate this honest discussion, for a venue like NeurIPS with its strong practical focus, this seems like a fundamental issue that should be addressed in the current work. Without adaptive moment selection, the method's applicability remains quite limited, and it raises concerns about potential cherry-picking of motifs. This limitation significantly reduces the paper's practical significance, as the effectiveness of the approach becomes heavily dependent on domain expertise for motif selection. The contribution would be substantially more impactful if an automatic moment selection procedure were introduced and demonstrated to work well across different graph types.\n\n**Limited practical impact of computational gains:**\nWhile the polynomial runtime is nice theoretically, the experiments show SIGL running in ~250 seconds, which isn't prohibitive for most applications. The authors should demonstrate scenarios where existing methods become truly infeasible to strengthen their computational efficiency claims.\n\n**Missing technical validation:**\nNo discussion of Monte Carlo gradient estimation stability, which could be problematic for training. \nThe theoretical bounds in Theorem 1 are never computed for the experimental settings. Showing whether these bounds are tight would significantly strengthen the claims. \n\n## Summary:\nWhile the core idea has merit and Theorem 1 provides compelling theoretical motivation, the paper needs to address the fundamental moment selection problem and demonstrate clearer practical advantages before meeting the bar for NeurIPS. The theoretical contributions, though heavily building upon existing work, are sound and well-executed. The experimental gains are modest but show promise for the approach's potential.\n\nQUESTIONS:\n- Can you provide concrete examples where the computational advantage over SIGL becomes critical?\n- How sensitive is the method to Monte Carlo sampling variance during training?\n- Can you compute the actual bounds from Theorem 1 for your experimental settings?\n\nLIMITATIONS:\nWhile not covered in an explicit \"Limitations\" section, this is honestly discussed in the Conclusion.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes MomentNet, a novel and theoretically principled approach for graphon estimation using implicit neural representations (INRs) trained via moment matching. It avoids costly latent variable inference and combinatorial Gromov-Wasserstein (GW) alignment by learning from empirical motif statistics extracted from observed graphs. The authors further introduce MomentMixup, a graph data augmentation strategy that interpolates motif vectors and generates synthetic graphs via the learned graphon.\nThe paper is well-structured and contains meaningful theoretical insights. It also reports performance improvements in both graphon estimation (on synthetic benchmarks) and graph classification (on small real datasets). However, I believe the paper in its current form does not provide sufficient experimental validation or analysis to support its claims fully.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. Theoretically grounded: The paper offers a rigorous upper bound on the cut distance between estimated and true graphons using a moment-based INR approximation, which is both novel and elegant.\n2. Conceptual simplicity: The use of empirical motifs to bypass latent node estimation is clever, computationally efficient, and scalable.\n3. Clear writing and organization: The overall exposition is solid and follows logical structure from motivation to theory to experiments.\n\n**Weaknesses**\n1. Weak Evaluation of MomentMixup\nWhile MomentMixup is a key contribution of the paper, the empirical support is limited: Only one baseline is considered (G-Mixup); Only four relatively small datasets are used (IMDB-B, IMDB-M, REDDIT-B, AIDS); The performance improvements presented are marginal in most settings.\n2. Fixed and Unanalyzed Motif Design\nThe method relies on a fixed set of motifs (up to 5 nodes) computed using the ORCA library, but provides no justification, selection criteria, or ablation analysis. The performance may heavily depend on motif relevance, redundancy, or sparsity. This weakens the generality and robustness of the proposed method, especially when applied to diverse graph types.\n3. Lack of Graphon Complexity in Experiments\nAll graphon estimation experiments are conducted on relatively smooth, synthetic graphons. The INR used is a shallow 1-layer MLP. There is no investigation into whether MomentNet can recover more complex or high-frequency graphons, or how its performance scales with harder tasks. This limits the scope and applicability of the method.\n\nQUESTIONS:\nSuggestions for Improvement\n1. Include ablation experiments that isolate the effect of MomentNet (without mixup) vs. MomentMixup (with simple generators).\n2. Perform motif selection analysis, including importance ranking, sensitivity motif size/type, and robustness to noise.\n3. Test MomentNet on more complex graphons (e.g., high-frequency, oscillating, or hierarchical structures).\n4. Try estimating graphons from real-world graphs and visualizing or interpreting the learned functions.\n5. Add stronger augmentation baselines such as DropEdge, GraphTransplant, or SubgraphMix, to better situate MomentMixup.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes MomentNet, a novel, scalable graphon estimation method using moment matching and implicit neural representations (INRs). Instead of inferring latent positions or minimizing costly distances like Gromov-Wasserstein (GW), it uses motif count statistics (graphon moments) as supervisory signal. The key idea is to optimize an INR to match empirical subgraph densities from observed graphs. The authors further propose MomentMixup, a mixup-based data augmentation strategy in moment space, which enables structural interpolation of graphs via synthetic moment vectors decoded through MomentNet.\n\nThe paper offers a new path in graphon estimation by combining solid theoretical foundations (e.g., cut-distance guarantees via inverse counting lemma) with practical scalability. It demonstrates strong empirical results on synthetic graphons and several real-world graph classification datasets.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**:\n1. *Novelty*: The idea of matching motif densities via an INR is new and elegant. It sidesteps latent position estimation—a common bottleneck in graphon estimation.\n\n2. *Scalability*: The method is inherently scalable. It reduces both memory and computational overhead by discarding raw graphs after extracting moments and works in polynomial time.\n\n3. *Theory-backed*: The paper offers a clear theorem bounding the cut distance between the estimated and true graphon under realistic assumptions (with number of motifs, sample size, etc.).\n\n4. *Empirical Performance*: Strong results on synthetic graphon estimation (13 benchmark graphons) and real-world graph classification via MomentMixup. It outperforms or matches SOTA in most cases.\n\n5. *MomentMixup is principled*: The paper identifies a limitation in G-Mixup (moment vectors of interpolated graphons ≠ interpolated moment vectors) and proposes a better method grounded in structure.\n\n6. *Clear Writing and Presentation*: The exposition is mostly accessible and well-structured for a technical audience. Figure 2 and the breakdown of training steps help clarify the process.\n\n\n**Weaknesses**:\n1. *Choice of Moments is Fixed*: As admitted by the authors, the set of motifs is predefined (up to 5-node motifs via ORCA). There’s no adaptive selection or theoretical guidance on which motifs best characterize a given graphon. Poor motif choice can hurt performance.\n\n2. *Single-Graphon Assumption for Each Class*: MomentMixup assumes each class corresponds to a single graphon. This is limiting for datasets with high intra-class variation (e.g., Reddit-Binary), where performance lags.\n\n3. *No Comparison with Recent Diffusion-Based or Equivariant Methods*: SIGL is a strong baseline, but there are recent graph representation models (e.g., equivariant GNNs, diffusion-based models) that could offer complementary perspectives.\n\n4. *No Ablation on INR Architecture*: The INR used is a shallow 1-layer MLP. It's unclear whether deeper architectures improve estimation quality or overfit due to noise in empirical moments. An ablation would be informative.\n\n5. *MomentMixup’s Label Assignment*: The paper does not detail how the class labels for generated graphs via MomentMixup are assigned. Are they soft labels or nearest neighbor class labels? This has implications for supervised training.\n\nQUESTIONS:\n1. *On Moment Selection*: Can you elaborate on how robust your method is to the choice or number of motifs? Would adaptive motif selection (e.g., using entropy or redundancy) improve performance?\n\n2. *On MomentMixup Semantics*: When combining moment vectors from two classes, how is the label assigned to the generated graph? Is it a soft label, or do you assign based on dominant α weight?\n\n3. *On INR Generalization*: Have you evaluated whether the trained MomentNet generalizes well to moment vectors outside the training distribution (e.g., interpolations or extrapolations)? How stable is the mapping from moments to graphons?\n\n4. *On Heterogeneous Graph Classes*: How would your method adapt to datasets with significant intra-class structural variability? Could a mixture-of-graphons model address this?\n\n5. *On Motif Sampling Efficiency*: The runtime and sampling complexity of computing motif densities (especially with ORCA) might still be high. Could you consider approximating them via graphlet sampling or neural surrogates?\n\nLIMITATIONS:\nThe authors acknowledge key limitations in Section 6, including:\n\n  - Sensitivity to the choice of moments.\n\n  - Assumption of a single graphon per class in MomentMixup.\n\n  - Lack of adaptation to dynamic or attributed graphs.\n\nThese are valid and well-stated. I would also add:\n\n  - Lack of interpretability in the learned INR.\n\n  - Training instability when motif estimates are noisy (especially for small motifs or sparse graphs).", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}