Title: Where Graph Meets Heterogeneity: Multi-View Collaborative Graph Experts
Venue: NeurIPS 2025
Accepted: yes
Keywords: Graph neural networks

heterogeneous data

multi-view learning
Abstract: The convergence of graph learning and multi-view learning has propelled the emergence of multi-view graph neural networks (MGNNs), offering strong capabilities to address complex real-world data characterized by heterogeneous yet interconnected information. 
While existing MGNNs exploit the potential of multi-view graphs, the inherent conflict persists between the two critical inductive biases of multi-view learning, consistency and complementarity. Consequently, the challenge of defining and resolving this tension in the new context of multi-view graphs remains largely underexplored. To bridge this gap, we propose Multi-view Collaborative Graph Experts (MvCGE), a novel framework grounded in the Mixture-of-Experts (MoE) paradigm. MvCGE establishes architectural consistency through shared parameters while preserving complementarity via layer-wise collaborative graph experts, which are dynamically activated by a graph-aware routing mechanism that adapts to the structural nuances of each view. This dual-level design is further reinforced by two novel components: a load equilibrium loss to prevent expert collapse and ensure balanced specialization, and a graph discrepancy loss based on distributional divergence to enhance inter-view complementarity. Extensive experiments on diverse datasets demonstrate MvCGE’s superiority.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Multi-view Collaborative Graph Mixture of Experts (MvCGE), a novel framework that unifies multi-view learning and graph neural networks. MvCGE employs a pool of layer-wise graph experts that are dynamically routed to each view via a graph-aware top-K gating mechanism. Consistency is enforced by sharing expert parameters across views, while complementarity is preserved through two regularizers. Extensive experiments on five multi-view and six single-view benchmarks demonstrate consistent performance gains over strong baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The work is clearly motivated by the trade-off between consistency and complementarity in multi-view GNNs.  \n2. The proposed framework seems to be novel and bridge the research gap in multi-graph learning.  \n3. Most of the descriptions and formalizations in the methods are clear, and it is convinced that the proposed MoE-based framework can be a good solution.  \n4. Evaluation across ten datasets (including large-scale Freebase and arXiv in the appendix) with both multi-view and single-view settings demonstrates broad applicability.\n\nWeaknesses:\n\n1. As per my understanding, introducing multiple GNNs on multiple graphs will increase the runtime and memory footprint by a significant amount compared to the baselines. However, the authors claim that the proposed method is efficient and does not impose a significant computational burden. The authors should clarify this issue.  \n2. The model requires tuning on several hyperparameters $\\alpha$, $\\beta$, total experts $M$, and top-$k$. although explored their impact, the need to tune four interdependent hyperparameters may hinder practical adoption.  \n3. Some equations (e.g., the generalized MMD in Eq. 16 vs. 17) and definitions (routing mechanism) could be streamlined in the main paper and elaborated in the appendix, or readers may struggle with these.\n\nQUESTIONS:\nDoes the combination of multiple GNNs and multiple graphs result in a high computational burden? If so, does the proposed approach include any improvement for scalability?\n\nLIMITATIONS:\nYes. The authors note the omission of homophily/heterophily discussion. I recommend the authors to include a discussion (or empirical analysis) of how their method performs under varying graph homophily levels.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a MoE-inspired framework to harmonize consistency and complementarity in multi-view graph learning. By employing shared collaborative graph experts and a graph-aware router, the model dynamically selects experts per view while maintaining architectural consistency. Two regularization strategies, load balancing and distributional divergence, prevent expert dominance and enhance view-specific representations. Experiments show state-of-the-art performance across 10 datasets.\n\nSTRENGTHS AND WEAKNESSES:\n## Strength\n\n- The paper is well-motivated and well-written. The authors do identify a critical challenge in multi-view graph learning and the design limitations of existing multi-view graph neural networks. The design principles of this work could inspire MoE applications in other graph tasks.\n- One principal strength of this work is the proposed architecture decouples expert count from view count, which achieves dynamic adaptation to multi-view graphs and without composing much computational burden.\n- Providing executable source code is encouraged, I run the code and can reproduce the reported results. BTW, the code is well-organized and easy to read.\n\n## Weakness\n\n- As the author mentioned in the conclusion, one limitation of this paper is the absence of a discussion on the interaction between homophily/heterophily and the proposed MvCGE. Addressing this issue may provide more insights into this work.\n- I note that the authors provide some theoretical analysis in Appendix C, can the authors clarify the necessity of Proposition 1 and Corollary 1 and how they demonstrate the validity of the proposed approach?\n- The discussion of graph discrepancy appears to lack sufficient definitional clarity. Regarding the distributions $\\mathbb{P}\\_{Q}$ and $\\mathbb{P}\\_{Q’}$, what exact probability measures do $\\mathbb{P}\\_{Q}$ and $\\mathbb{P}\\_{Q’}$ represent in the proposed framework? How to sample instances from these distributions to compute the MMD-based graph discrepancy $M_{H}(\\mathbb{P}\\_{Q}, \\mathbb{P}\\_{Q’})$?\n\nQUESTIONS:\n- How does homophily/heterophily interact with the proposed model?\n- What do proposition 1 and corollary 1 suggest?\n- See weaknesses for some questions.\n\nLIMITATIONS:\nYes the authors discuss limitations in the conclusion.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the challenge of balancing consistency and complementarity in multi-view graph neural networks (MvGNNs) by introducing a Mixture-of-Experts–inspired framework called Multi-view Collaborative Graph Mixture of Experts (MvCGE). At each layer, a set of shared “graph experts” (i.e., distinct GNN filters) is dynamically routed per view via a graph-aware top-k gating mechanism. To prevent expert collapse and encourage diversity, the authors also propose two losses. Experiments on five multi-view and six single-view benchmarks demonstrate that MvCGE consistently outperforms baselines.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\nS1. Novel and well-motivated. The framework innovatively addresses the key challenge of balancing consistency and complementarity in multi-view learning by combining MoE with GNN, has a practical impact and makes sense for the community.\n\nS2. It is interesting that the work designs the graph discrepancy and the graph-aware router, which are capable of regularizing the experts and learning awareness of graph structures.\n\nS3. The evaluation of the proposed method is comprehensive, experiments include performance comparisons over both single-view and multi-view graphs, sensitivity analyses, and visualization.\n\n**Weaknesses**\n\nW1. Reproducibility is doubtful. The authors have included only an appendix in the supplemental material and have not provided any source code or any form of pseudo-code for reproduction. This may affect the credibility of the experimental results.\n\nW2. The Methodology section is hard to follow. The complex notations made me spend more time reading about the methodology, especially for the collaborative experts and graph discrepancy regularization. I list the questions in the following part.\n\nW3. Lack of some literature. In related work, the authors only introduce a few MoE-GNN papers, and some recent works are not discussed or compared as baselines, leaving unclear how this work positions itself in the literature.\n\nW4. Limited theoretical analysis. Beyond intuitive motivation, the paper lacks a formal or theoretical analysis of the proposed method.\n\nQUESTIONS:\nQ1. How exactly is the routing function calculated? Equations 8, 10 and 11 are a bit confusing, can you provide a step-by-step description or a table of algorithm for better clarity?\n\nQ2. In Equation 13, why does the superscript $(l)$ disappear and why is it inconsistent with Equation 8?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper identifies that existing multi-view GNNs either share parameters across views (over-emphasizing consistency but losing view-specific nuance) or allocate separate channels per view (preserving complementarity but sacrificing inter-view synergy). They propose MvCGE, which at each layer maintains a shared pool of graph-convolution experts and uses a graph-aware top-K routing to dynamically assign each view’s nodes to a subset of experts. A load-equilibrium loss ensures no expert is neglected, and a graph-discrepancy loss (via MMD) enforces that different experts learn distinct representations. Experiments on five multi-view and five single-view citation/dataset benchmarks show MvCGE consistently outperforms strong baselines in Macro-F1 and Micro-F1.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- MOE for  multi-view GNN learning\nBy decoupling expert count from view count and dynamically routing, MvCGE achieves both architectural consistency (shared experts) and view complementarity (expert specialization) in a unified framework.\n-   Novel Regularizers\nThe load-equilibrium loss prevents expert collapse (ensuring each expert is used), and the graph-discrepancy loss leverages MMD to keep expert representations distinct. \n-   Strong Empirical Gains \nOn five multi-view graphs (e.g., ACM-M, YELP), MvCGE achieves good performance compared to other baselines, and it also improves over state-of-the-art single-view GNNs on standard benchmarks.\n\n**Weaknesses**\n- The Ldsp loss:\n This loss is to minimize the discrepancy between distributions from different graph experts. THIS DESIGN is actually against the orginal design to perform distinct filtering. Thus, this loss is questionable.  \n- Complexity issues:\n   This solution introduce considerable complexity which might not be effective on large scale graphs. The authors fail to provide the complexity analysis and the trainning/execution time of the proposed solution.  The Top-K gating per node incurs extra computation and may be hard to scale to very large graphs or high expert counts without further efficiency tricks.\n● Limited Task Scope\n    Evaluation focuses solely on semi-supervised node classification. It remains unclear how MvCGE would perform on regression, link prediction, or graph-level tasks.\n● Lack of reproducity, Source code and Settings are not provided\n     This paper provide no source code with very complex design. The detailed settings are also missing, e.g. how to generate multi-view for single-view graphs , how to define the feature similarity threshold for connections? Settings on designs of differnt experts, number of experts, topK are not mentioned.\n\nQUESTIONS:\n1.  Scalability: How does inference time grow with graph size and expert count?\n2.  How to define the view of multi-view graphs, is there any different designs of different experts ? Would different experts receive graph inputs only from that views, are different combinationn of designs have impacts towards final results ?\n3.  Expert specialization: Have you observed that particular experts consistently specialize on certain views or substructures? Can these specializations be interpreted?\n4.   Robustness to view noise: If one view is very noisy or missing, how does the routing mechanism compensate?\n5.    Loss weighting: How sensitive is performance to the choice of α (load-equilibrium) and β (discrepancy)? Is there a principled way to set them?\n6.    Generalization to other tasks: Can the same MvCGE framework be applied to link prediction or graph regression problems? What changes would be needed?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}