Title: Symmetry-Preserving Conformer Ensemble Networks for Molecular Representation Learning
Venue: NeurIPS 2025
Accepted: yes
Keywords: Molecular Conformers

Geometric Deep Learning

Mixture-of-Experts
Abstract: Molecular representation learning has emerged as a promising approach for modeling molecules with deep learning in chemistry and beyond. While 3D geometric models effectively capture molecular structure, they typically process single static conformers, overlooking the inherent flexibility and dynamics of molecules. In reality, many molecular properties depend on distributions of thermodynamically accessible conformations rather than single structures. Recent works show that learning from conformer ensembles can improve molecular representations, but existing approaches either produce unphysical structures through averaging or require restrictive molecular alignment. In this paper, we propose SymmetryPreserving Conformer Ensemble networks (SPiCE), which introduces two key innovations: (1) geometric mixture-of-experts for selective processing of scalar and vector features, and (2) hierarchical ensemble encoding that combines ensemblelevel representation with cross-conformer integration. Crucially, SPiCE ensures physically meaningful representations by maintaining joint equivariance to geometric transformations of individual conformers and conformer permutations. Extensive experiments demonstrate that SPiCE consistently outperforms existing conformer ensemble methods and state-of-the-art structural aggregation models across quantum mechanical and biological property prediction tasks.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents Symmetry-Preserving Conformer Ensemble networks (SPiCE), a model achieving equivariance to permutations of conformer ensemble and geometric transformations of the molecules. The model is tested using different GNN backbones and various property prediction problems. The method's main contribution is a hiearchical GNN that uses Geometric Mixture of Experts (GMoE) to create an ensemble of weight-shared equivariant 3D GNNs that model intra-conformer interactions. GMoE is an extension of Mixture of Experts, using two sets of experts to process invariant and equivariant features separately.\n\nSTRENGTHS AND WEAKNESSES:\nI thank the authors for the well-written paper.\n\nThe SPiCE framework is implemented so that it can handle different GNNs and the authors tested it for various models. They also compare the method to other ensemble methods. (The code seems to be built using that of Ref. [51] as mentioned by the current paper.) Table 1 shows that SPiCE often achieves the best performance with one of the backbone models; however, other ensemble-based methods often achieve performances very close in absolute values (<0.03 differences for the MAE), which correspond to 2.8-8 % improvements as discussed by the paper.\n\nThe studied properties of Table 1 are only described in the Appendix, many of them are only referred by abbreviation in the main text (e.g. EA, IP). It would help readability to write these out in the paper. Moreover, units of the quantities are missing.\nAlso, there seems to be a mistake in Table 1: it is stated that the bold value is the best model but for B_5, another value is bold, the underlined one seems to be the best.\n\nNote that from the name choice, readers might associate it with the dataset of the same name (see: Eastman, Peter, et al. \"Spice, a dataset of drug-like molecules and peptides for training machine learning potentials.\" Scientific Data 10.1 (2023): 11.).\n\nQUESTIONS:\n1) In Table 1, what are the units of the different quantities? \n2) Can you explain the increase of the training loss around 100 epochs in Fig. 2b?\n3) How does the computational cost of SPiCE compare to the baseline GNN models? (For training and evaluation?)\n4) The Experimental setting states: \"Training terminates if the loss shows no improvement for 400 consecutive epochs.\" - Is 400 epochs without improvement really necessary to achieve good models? Could there be a more computationally efficient strategy?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors address the (often-overlooked) role of 3-D atomic arrangements in molecular machine-learning tasks. They introduce SPiCE, a symmetry-preserving architecture that leverages conformer ensembles instead of collapsing them into a single structure. Across several real-world datasets, SPiCE reportedly outperforms existing approaches.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1. SPiCE outperforms baselines on reported benchmarks.\n1. The paper's argument, \"preserving conformer information is beneficial for accurate molecular property prediction,\" sounds convincing\n\nWeaknesses\n1. Notation and definitions. Some symbols lack formal mathematical definitions, and dimension mismatches (e.g., between H^s and r^s) create confusion. A concise notation table would help readers.\n1. Domain-specific shortcuts. The manuscript assumes prior knowledge of certain molecular-representation conventions. Brief primers or citations would benefit readers outside the subfield.\n1. Rationale for design choices. Key architectural decisions (e.g., differing functional forms for r^s vs. r^v, adoption of Mixture-of-Experts) need clearer theoretical or empirical justification.\n1. Compute/accuracy trade-off. Modeling multiple conformers inevitably increases computational cost, but the paper does not quantify this overhead or compare run-time vs. baseline methods.\n\nQUESTIONS:\n1. Are the conformers $\\{c_i\\}$ supplied by the datasets or generated by the authors? Please clarify the protocol.\n2. Is $R = \\mathrm{SO}(3)$ or $\\mathrm{O}(3)$?\n3. Why is $r^s$ linear in $h$ while $r^v$ is quadratic?\n4. Line 196: Does $R^n$ denote $\\mathrm{SO}(n)$, $\\mathrm{O}(n)$, or something else?\n5. Line 180: $H^s$ is $N \\times d$, yet $r^s$ is $N_I$. Where does the reduction from $N$ to $N_I$ occur?\n6. What specific bottleneck does the expert-routing mechanism alleviate in this context?\n7. Line 224: Why is $\\bar{H}$ claimed to be $S_n$-invariant rather than $S_n$-equivariant?\n8. Table 1: What are the parameter counts for each model? Were they matched across methods?\n9. Fig. 3d: Performance drops for both too few and too many experts; can the authors provide intuition?\n10. How much wall-clock time or GPU hours does SPiCE add relative to its single-conformer backbone?\n\nMinor Comments\n1. Line 185: Should $\\hat{r}_i$ lie in $\\mathbb{R}^{2 \\times (N_I + N_E)}$?\n1. Fig 2: There is a loss spike near epoch 100; any explanation?\n1. Fig 3: Font size is too small; please enlarge axis labels for readability.\n\n---\n\nLIMITATIONS:\nThe limitations are not discussed. It would be nice to include the Compute/accuracy tradeoff mentioned in Strengths.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 1, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Symmetry-Preserving Conformer Ensemble networks (SPiCE), a new framework designed to enhance molecular representation learning by leveraging ensembles of molecular conformers rather than relying on single static structures. SPiCE designs an architecture integrating three core components: (1) shared conformer encoding with weight-tied 3D GNNs that preserve geometric equivariance, (2) a Geometric Mixture-of-Experts (GMoE) layer that processes scalar and vector features through specialized expert networks with type-aware selective routing, and (3) a hierarchical ensemble encoder that incorporates molecular-level context via cross-conformer attention. The model also provides theoretical properties of aggregations, such as R-equivariant, etc., showing the model is physically meaningful, symmetry-aware representations, and outperforms existing methods across a wide range of molecular property prediction tasks, from quantum mechanics to biology.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\ni) Method Novelty\n\nThough the method leverages existing techniques, its overall design is well-designed, adapted to the molecular representation learning. There are also some interesting parts like differentiable top-k with a mixture of experts to selectively fuse 3D conformer to 2D representations.\n\nAuthors also present properties about the permutation invariance of the ensemble and geometric symmetries of individual conformers. \n\nii) Experiments:\nProvided experiments are very details where SPiCe is validated on four different architectures (PaiNN, ClofNetEquiformer, and VisNet), confirming the improved performance and generalized ability of the method.\nThere are also experiments on model scaling on the Drug-75k dataset, at 10% and 100% training rates. A detailed of ablation studies is also provided in Figure 3.\n\nLimitations Weaknesses:\n\n\nAmbiguity in the definitions of scalar and vector features: The paper defines h_{i}^{s} ​\\in R^{1\\times d} as scalar features and h_{i}^{v} \\in  R^{3 \\times d} as vector features (line 181). However, this naming can be confusing, especially when d>1, as it may not align with the conventional understanding of scalars and vectors in the molecular context. The confusion is exacerbated by the early introduction of these terms in Figure 1. The reviewer suggests the authors consider alternative, more intuitive terminology or provide clearer justification and explanation for these definitions. Similarly, the definitions of X^s and X^v (line 151), which are inputs to the message-passing network that generates H (line 180), are not clearly explained and would benefit from further clarification.\n\n\nClarity and readability of Section 3.2.2: This section contains dense notation and is challenging to follow. The reviewer recommends augmenting this part with a relevant subfigure from Figure 1 to visually support the equations and improve reader comprehension.\n\n\nLack of discussion on limitations: The paper does not adequately discuss the limitations of SPiCE. Including such a discussion would help readers better understand the potential constraints of the method and set appropriate expectations for its applicability.\n\n\nUnclear attention mechanism in Figure 1c: It is unclear why only the 3D scalar features are used as queries in the attention mechanism with 2D features. The specific role of vector features within the GMoE module should be clarified to improve the interpretability and transparency of the design choice.\n\n\nRedundant discussion on equivariance and invariance: The method section repeatedly revisits the concepts of equivariance and invariance, which, while important, may distract from the understanding of other architectural components. The reviewer suggests summarizing these theoretical properties in a dedicated subsection or theorem early in the method section, allowing the rest of the space to focus on describing the novel contributions and architectural details more clearly.\n\nQUESTIONS:\n-\n\nLIMITATIONS:\n-", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a new architecture for conformer-based molecular property prediction. This architecture involves a Mixture of Experts for analyzing the latent features of the molecule. Care is taken to differentiate between the vector and scalar components, and it is shown that the architecture preserves the equivariance/invariance of the underlying experts (GNNs). The equivariant experts are 3D GNNs, and the invariant experts are 2D topological GNNs. Attention is used to combine the information of both types of experts.\n\nSTRENGTHS AND WEAKNESSES:\nQuality: The submission takes care to explain the technical details of the architecture and all of the mathematical proofs that the architecture is invariant/equivariant in the appropriate places. The work did not really cover weaknesses of the introduced architecture. In particular, the datasets evaluated seem relatively small.\n\nClarity: The writing was generally clear, and the level of detail in section 3 was good. Sections 4 and 5 felt brief in comparison to how dense section 3 was.\n\nSignificance: The work improves the state of the art on some molecular property prediction tasks, and the details are explained well enough that this could be extended in the future. The method of this work is more general than many previous works, and so the mathematical foundation developed in this work can likely be used in later works to extend and improve the results described here.\n\nOriginality: The comparisons to other similar conformer-based and non-conformer-based models are helpful in providing context. It improves on and generalizes the previous works.\n\nQUESTIONS:\nIs it possible that the size of the datasets used in evaluation impacted the performance of the model? At the largest, 75k molecules/560k conformers is only a moderately sized dataset, and most of the comparisons were done with a model trained on a fraction of those data.\n\nPossible confusion may arise with the similarly named SPICE dataset (https://www.nature.com/articles/s41597-022-01882-6).\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-06"}