Symmetry-Preserving Conformer Ensemble Networks for Molecular Representation Learning
Authors: Yanqiao Zhu
Yidan Shi
Yuanzhou Chen
Fang Sun
Yizhou Sun
Wei Wang
Venue: NeurIPS 2025 poster ()
Keywords: Molecular Conformers
Geometric Deep Learning
Mixture-of-Experts
Primary area: machine_learning_for_sciences

Abstract: Molecular representation learning has emerged as a promising approach for modeling molecules with deep learning in chemistry and beyond. While 3D geometric models effectively capture molecular structure, they typically process single static conformers, overlooking the inherent flexibility and dynamics of molecules. In reality, many molecular properties depend on distributions of thermodynamically accessible conformations rather than single structures. Recent works show that learning from conformer ensembles can improve molecular representations, but existing approaches either produce unphysical structures through averaging or require restrictive molecular alignment. In this paper, we propose SymmetryPreserving Conformer Ensemble networks (SPiCE), which introduces two key innovations: (1) geometric mixture-of-experts for selective processing of scalar and vector features, and (2) hierarchical ensemble encoding that combines ensemblelevel representation with cross-conformer integration. Crucially, SPiCE ensures physically meaningful representations by maintaining joint equivariance to geometric transformations of individual conformers and conformer permutations. Extensive experiments demonstrate that SPiCE consistently outperforms existing conformer ensemble methods and state-of-the-art structural aggregation models across quantum mechanical and biological property prediction tasks.

Symmetry-Preserving Conformer Ensemble Networks for Molecular Representation Learning
Yanqiao Zhu" Yidan Shi" Yuanzhou Chen Fang Sun Yizhou Sun Wei Wang
Department of Computer Science, University of California, Los Angeles Kyzhu, adriaddinnaaa adrianchen, fts, yzsun, weiwang Wcs ucla. aola.ed edu A.danshiuucla.. edu
Abstract
Molecular representation learning has emerged as promising approach for model- ing molecules with deep learning chemistry and beyond. While 3D geometric models effectively capture molecular structure, they typically process single static conformers, overlooking the inherent flexibility and dynamics of molecules. In reality, many molecular properties depend on distributions of thermodynamically accessible conformations rather than single structures. Recent works show that learning from conformer ensembles can improve molecular representations, but existing approaches either produce unphysical structures through averaging or require restrictive molecular alignment. In this paper, we propose Symmetry- Preserving Conformer Ensemble networks (SPiCE), which introduces two key innovations: (1) geometric mixture-of-experts for selective processing of scalar and vector features, and (2) hicrarchical ensemble encoding that combines ensemble- level representation with cross-conformer integration. Crucially, SPiCE ensures physically meaningful representations by maintaining joint equivariance to ge- ometric transformations of individual conformers and conformer permutations. Extensive experiments demonstrate that SPICE consistently outperforms existing conformer ensemble methods and state-of-the-an structural aggregation models across quantum mechanical and biological property prediction tasks.
Introduction
Molecular Representation Learning (MRL) has emerged as a powerful tool at the intersection of chem- istry and machine learning, offering data-driven approach to encode discrete molecular structures into continuous feature representations [1, 2]. This enables efficient predictions of molecular proper- ties without expensive quantum mechanical calculations for various downstream tasks including drug discovery, materials design, and chemical reaction prediction [3-5].
The field has evolved from early appraoches using simplified molecular representations such as SMILES strings [6] and molecular fingerprints [7, 8] to sophisticated geometric deep learning methods that directly incorporate 2D topological and 3D geometric information [9]. Among these advances, 3D Graph Neural Networks (GNNs) have gained popularity by learning from molecular geometries while respecting fundamental physical symmetries. These models can be categorized as invariant (producing identical outputs regardless of molecular orientation and translation) [10-12], SF(3)--quivarian (transforming consistently under rotations and translations) [13, 14), and E(3)- equivariant (additionally respecting reflection symmetry) [15, 16). By incorporating these inductive biases, geometric models achieve both improved sample efficiency and better generalization.
However, current 3D MRL models face fundamental limitation: they typically encode individual conformer structures as if molecules were rigid, static entities. In reality, molecules exist as dynamic
'These authors made equal contribution this research.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Conformer Ensembles
Shared Conformer Encoding
Scaler Foalures
Veclor Features
Router
Rouler
Excoding
EE
EE,
EE
H- H-Eqqiisarinn Interaction Block
GMoi:
GMaE
GMoE
GNN
Geometrie Mixture-of-Esperts (GM)))
Cross Conformer Integration
Invoriant Experr (IR)
Equivariant Experr (8X)
Confomer- evel Pooling
(b) E-Equivarrian Interaction Block
3D Sealar Features
2D Feutures
Molecule-Level Pooling
O0
Lincar
MLP Lincar
Lincar
SiLUI
Product
Equivariant
Prediction Head
Scaled Dol-Product Allention
[ Layer Norm
Layer Norm
(4) Overall pipeline
Cross-Conformer Infegration Layer
Invariant Equivariant Experts
Figure 1: (a) SPiCE processes conformer ensembles through multiple interaction blocks, followed by node-level pooling to obtain conformer representations and ensemble-level pooling for final molecular embeddings. (b) H-equivariant interaction block that maintains joint equivariance conformer permutation (S,) and geometric transformations (G"), composed shared conformer encoding, geometric mixture-of-expert. and cross-conformer integration. (c) Cross-conformer integration layer combining ensemble encoding through 2D GNN with attention based integration. (d) Geometric mfixxereof-eeperts (GMoE) with separate routing for scalar (type-0) and vector (type-l) features. (e) Invariant and equivariant expert networks: equivariant experts preserve rotational symmetry through specialized linear transformations with scalar feature gating for enhanced expressivity.
systems that continuously interconvert between different conformational states through bond rotations, vibrational motions, and intermolecular interactions [17]. Therefore, a conformer ensemble, which refers to the collection of thermodynamically accessible molecular conformations equilibrium, provides a more complete molecular representation [18]. This is particularly important because many experimentally observable properties depend on the entire distribution of conformers rather than single static structure. For example, pootein-ligand binding often involves conformational selection where proteins recognize specific ligand conformations, and reaction mechanisms frequently depend on the accessibility of particular geometric configurations [19, 20].
Existing approaches conformer ensemble modeling fall into three categories. Traditional chemin- formatics methods like 4D-QSAR [21-23] extend classical mesh-based 3D-QSAR by incorporating conformational information into grid-based molecular descriptors. These methods map molecular properties onto regular 3D lattices for each conformer to produce molecular shape spectra, but require rigid molecular alignment and are limited to datasets with common substructures. Multi- instance learming methods have been adapted from computer vision to treat conformer ensembles as bags of instances [24-28]. However, these approaches typically process conformers independently and then aggregate conformer representations through simple pooling operations, which discard cross-conformer interactions and cannot maintain geometric symmetrices. Additionally, structural aggregation methods [29-31] attempt to combine information from multiple conformers through averaging or clustering procedures. While these can leverage geometric models, they often produce unphysical structures and are highly sensitive to alignment methods.
To address these limitations, we present Srrmmetry-Preserveee Conformer Ensemble networks (SPiCE), a novel model that achieves joint equivariance to both permutations conformer ensemble (S,,) and geometric transformations of individual conformers (G"). SPiCE processes conformer ensembles through a series of H/ -eequuvaiiannt interaction blocks, where H = Sn, x G". Each block is composed of three key components: (1) shared conformer encoding with weight-tied 3D GNNs that process each conformer while preserving geometric equivariance, (2) Geometric Mixture-off Mixture-of-Experts (GMoE) layer that separately routes scalar and vector features through specialized expert networks, enabling type-aware selective processing, and (3) hierarchical ensemble encoding that combines molecular-level context with selective cross-conformer integration through attention mechanisms. Importantly, SPiCE is able to capture both geometric and topological relationships while respecting fundamental geometric symmetries, effectively processing conformer ensembles without requiring alignment generating unphysical intermediate structures.

---

We validate SPiCE through comprehensive experiments across diverse molecular property prediction tasks, ranging from quantum mechanical properties to biological activities. Using various 3D GNN backbones and datasets of different scales, we demonstrate consistent improvements over existing conformer ensemble methods. Our analysis reveals scaling behaviors with dataset size, suggesting reliable deployment across different data regimes in real-world applications. Through extensive ablation studies, we justify key architectural choices including the GMoE design, sparse upcycling strategy, and optimal expert granularity for conformer ensemble modeling.
2 Preliminaries
2.1 Problem Definition
A molecule is represented as molecular graph G = X),, where V = {v. [ denotes the node set of atoms, and Ç V x V represents chemical bonds between atoms. The node attributes are represented X E Rlvxd. For a given molecule, we consider a set of ri discrete conformers C= -C where each C, R represents atomic 3D coordinates. These conformers are sampled from the with a Boltzmann weight p: = conformer C, kg is the Boltzmann constant, and T is temperature. We note that these Boltzmann weights are not provided to the model but rather are used to compute ground-trute ensemble properties.
conformational space. Each conformer is associated E, exp(-e//kis] where 6, is the energy of
Our goal is to learn mapping from the conformer ensemble (G to molecular properties while preserving both permutation invariance of the ensemble and geometric symmetries of individual conformers. This requires learning functions that are equivariant to the product group H S,, x G where S, is the permutation group over n conformers and G is the geometric symmetry group. Typically, G is taken as the Euclidean group E(3 of translations, rotations and reflections, or the special Euclidean group SE(3) of translations and rotations. We allow individual transformations on each conformer individually and independently, hence the overall geometric transformation G".
2.2 Basics of Symmetry Properties
On permutation group S,, Our work builds on the characterization of linear S, Sqqqquivariant layers [32], which be decomposed into intra-conformer interactions and a global G-invariant aggregation. We formalize this as a theorem and prove it in Appendix C.I. This provides a practical construction of H --equiiaria layers, by processing individual conformers through G-equivariant functions and capturing ensemble interactions via a G-invariant function applied to aggregated features. This generalizes Deep Sets [33] geometric symmetries in molecular conformers.
On geometry group G. We decompose geometric transformations into translations and rotations (including reflections for E(3)): G = T x R. Following Equiformer [34], rype-0 and rype-1 to express R- R-invariant and equivariant features and list basic symmetry-present operations below.
Lemma 1. The following operations preserve symmetry:
(1) Any function (s) of an R-invariant feature & is R-invariant;
(2) The product between R-invariant feature and R-equivariant feature is R-equivariant.
(3) The inner product (V1, 02) between two R-equivariant features U1, U2 is R-invariant.
3 Method
The overall architecture of SPiCE is illustrated in Figure 1. SPiCE employs a hierarchical ar- chitecture that enables structural interaction across atoms and conformers. Its core is series of !I-equivariant interaction blocks designed to process conformer ensembles while maintaining the rel- evant symmetries. intra-conformer learning is performed using weight-shared equivariant 3D GNNs, which jointly handle scalar and vector features. To support type-specific and selective information processing, we design Geometric Mixture-of of of-Experts (GMoE) module. The resulting features are then pooled to form an ensemble representation with 2D GNNs that capture the topological structure of the molecular system. These molecule- and conformer-leve features are subsequently combined through a cross-conformer integration module. Following the interaction blocks, we apply node-level pooling within each conformer to generate conformer-leved embeddings and conformer-leve pooling
3

---

across the ensemble to obtain the final molecular embedding. This embedding is then passed through prediction head to produce the output property prediction. A PyTorch-like pseudocode of SPiCE is provided in Appendix B.
Overall H--equivariance. We specify the input and output invariance and equivariance types for every module in the following sections. Since each module preserves symmetry and the output symmetrical properties of each module match the input properties of each subsequent module, the overall symmetry with respect to / is guaranteed through the composition of all modules.
Other remarks. We batch over both conformers and molecules, hence our features shapes often start with x-> We denote N n x [V] the total number of atoms across all conformers and interchangeably use N to express dimensionality. Typically, the second-to-lan dimension of our features (dim=-2) is either 1 for R-invariant scalar features 3 for R equiviiaai vector features. We often concatenate these features into a single hybrid feature with size 4 over this dimension, and call hybrid feature It-equivariant when the corresponding type-0 feature is Ri-ivvriian and type- feature is Reequivariant. Many modules of our model perform on each conformer separately, in which case S, equivaiaaee automatically holds, and individual G G-symmetries lead to global G" symmeert....
3.1 Input Representation and Feature Construction
For conformer ensemble, each H eequivaaria interaction block operates on graph node features that combine atomic and geometric information. For a molecule with m atoms, the input consists of: (a) atomic numbers E 2|! which are embedded into initial node features X E R through learnable embedding layer, and (b) atomic coordinates {c. )m where E TR33 From the coordinates, we compute relative position vectors Pij = C; Cj.
The //-uquivariant interaction blocks process both features æ; and geometric vectors Pij to construct messages that respect the underlying symmetries. To encode distance information effectively, we transform the relative distances r = Piil using set of radial basis functions. Specifically, we employ Gaussian radial basis functions (RBF) dx (r) spaced centers and controls the width of the Gaussians [10]. These radial features are then processed through a three-layer neural network, with layer normalization [35] and SiLU activation [36] after the first two linear layers. The output of this network parametrizes the message construction in the interaction blocks. Thanks to the RBF positional encoding, our encoded features are all T T-invariant, therefore the invarriiee of the entire model already guaranteed.
(r )²x))), where {ur } are equally
3.2 H Eqqivaaria Interaction Blocks
As the model is equipped with L interaction blocks, here we analyze the architecture within the I-th interaction block, and mostly omit the layer index | in the following discussions for simplicity.
3.2.1 Shared Conformer Encoding
With conformer set with hybrid node feature X = (X* X") E [xxx/xxxxdd employ shared 3D GNN layers to process conformer geometric information. A 3D GNN layer consists of two key learnable functions: message construction & and feature update V. which can be formulated as
((x,, 21, cuj)
h; =
jENs
where h; = (h;, hy), h E R 1xd rrppresents the scaler (type-0, rotationally invariant) feature for atom i with tensor component and d channels, and hy E Rgddd represents the vector (type-1, rotationally cquivariant) feature for atom i with 3 tensor components and d channels. For atom i in an arbitrary conformer, Mi denotes the set of its neighbors, Φ; = (æ), x;) E 124xd represent the concatenated scalar and vector features, Cij- = (cly. cyy) E 2 are concatenated scalar and vector messages between atom and j, d, dm are feature dimensions, and Q denotes message aggregation over neighbors N2 (typically summation).

---

For the first interaction block (1 = 0), X (type-O) contains initial molecular features including embeddings of atomic numbers z, distance-based radial basis functions, and topological descriptors. while X" (type-1) 1) contains initial geometric features including embeddings of relative position vectors and directional bond features. For subsequent interaction blocks (1 > 0), the features include both the
nn initial features above plus the processed outputs H and H from the previous interaction block.
The 3D GNN interaction layer outputs hybrid features H (H², H") E [xx[M[xxxd where h; denotes the hybrid node embedding for the i-th atomic node within this tensor. We also process H with a residual connection and an Equivariant Layer Normalization (ELN), which is a generalized layer normalization for G-equivariant features [34]. Here each conformer is operated separately, therefore the output is -q-eqivvrrinnt; plus, all 3D GNNs are pre-selected to be equivariant, so also independently Reequivariant for each conformer.
We note that our framework is agnostic to the specific architecture of the message and update functions. They can be either Cartesian equivariant models (e.g., PaiNN [37]) that operate directly in Cartesian coordinates to construct messages and update features, or spherical equivariant models that employ spherical harmonics and tensor products for message passing (c.g., Equiformer [34]), with channel-ne mixing and gated non-linear functions for feature updates.
3.2.2 Geometric ceature-offExperty (GMoE)
When processing molecular conformers, certain atoms or structural features may be more relevant or important than others for specific properties. Graph pooling approaches [38-40] achieve this selective processing by focusing computation on important nodes through attention mechanisms. This naturally extends neural networks for selective information passing with a learnable routing mechanism.
Eixture-of-Expertt (MoE) architectures [41, 42], which specialize multiple
Since scalar and vector features require distinct handling due their different transformation properties under symmetry operations, we propose GMoE as shown in Figure 1(d) that maintains cquivariance for vector features while ensuring invariance for scalar features through distinct router and expert groups. Note that all operations in this section operate on each node (and hence each conformer) separately, therefore Sq-equivariaace is already guaranteed, and any claim on rotational R- invariance/ / equivariance automatically extends to R"
GMoE processes the two types of features in H (H", H") E Nx4xd using separate modules. For given node i, let h; R Ixd and hy e 3xdd denote its scalar and vector features, respectively. We extend the standard Mixture of Experts (MoE) formulation [43, 44] to handle scalar and vector features independently, introducing two sets of routers and experts that process invariant and equivariant features respectively. The numbers of invariant and equivariant experts are denoted by Nj and Ng.
router design. For each node the concatenated routing weights A, (A), (A) E R NN+NNe for scalar and vector branches are computed through two-stage process. First, the corresponding routing networks r² r'(h)) and r" (h")), both with R-invariant outputs, compute initial scores r; (ri. r ) R MitNNN NE of (N, Ng) experts for scalar and vector features. Following SAGPool [38], our scalar router leverages GCN layer [45] compute routing scores, and our vector router calculates a linearly transformed inner product of features:
crrmuutatio-equivariiii
(h;)) = softmax
Σ JEN()
/ddd, +
(2)
r"(h)) = softmax
hh: W
(3)
where de represents the number of neighbors of node i. The R-invariance of 7 due to (3) of Lemma I: operation hy performs inner products over the equivariant dimension. The scores then undergo top-k selection and normalization:
rr..0
if if E top-k (op-k((r2,)) ())))
A8 where ra, refers the score for the i-th atom of the e-th expert, and l is the pre-set number of selected experts for each type of hybrid features. The computation for fx shares the same logic as ps The resulting (A: (f), ff)e - -invvaaria from (1) of Lemma 1.
0, otherwise,
5

---

Invariant and equivariant experts. Each Invariant Expert (IE) is implemented as two-layer MLP with SiLU activations followed by layer normalization. The output of the e-th IE can be defined as F = (H) E Rxxxxd These outputs are also Rin-invariant from (1) of Lemma 1. Meanwhile, for maintaining geometric symmetries, each Equivariant Expert (EE) is formulated as:
f h; We diag(MLP(ht)).
(5)
where f is the output of the e-th expert on node and the concatenated output F² = [H' (H,, H)) E
RN xxxxx Here We Rd is learnable weighting matrix, and diag constructs diagonal matrix from a vector. EE is R" -quiviivari from (2) of Lemma since h is equivariant, while both W. and diag(MLP(h:)) are invariant, the latter being consequence of (1) of Lemma 1.
Node-wise weighted summarization. With selected expert weights and expert output for both scalar and vector features, node-wise weighted summarization is employed for dynamic feature ensemble. For node C C x V, the scalar and vector representations can be calculated as:
h Σ Pe fajees
R 1xd
h
Σ
Pr Ses 3xd
(6)
For molecular conformer ensemble, the output of MoE module is represented as H = (H HH Nx4xd, where the H' and H" are R -nniinvarant and R -euequivariant, respectively from (1) and (2)
of Lemma 1. Hence, the hybrid feature H is tt -'u-euiiariaat..
Router training and regularization. To ensure balanced expert utilization, we employ three key mechanisms: (1) Router z-loss to punish extreme routing decisions [46]:
NE
Cn(r)
log Σ exp(re,)) ) + log
Σ
exp(re,))
where re, and r²., represents the Ri"iinvaiint score from invariant and equivariant routers for the i-th node of the e-th expert, and Nj and NE are the number invariant and equivariant experts.
(2) Top-k expert selection with Gumbel-Sigmoid sampling [47, 48), where the Gumbel noise allows the experts be better differentiated in the process of training:
Gumbel-Sigmoid(()) = Sigmoid(I + + G""
(8)
where routing logits ri.e or ri. are input x. and G' and G" are independent Gumbel noise samples.
(3) Expert upcyeling strategy [49] that gradually increases the number of active experts during training. Upeycling combined with the Gumbel-Sigmoid technique allows us to overcome the limitations of static expert selection and achieve improved performance in our geometry- aware sparse architecture.
3.2.3 Ensemble Encoding
The ensemble encoding block enables cross-conformer interactions while preserving S, -euuivariiinc.e 1 denote the mean-pooled representation of S,, S..-qquivarian,, h?"- Let H = (1/n) Σ Ha
invariant scalar features H' across all conformers, which is S,,-ivvrriat and iin-ivariant. This is processed by a Graph Isomorphism Network (GIN) layer [50], incorporated with 2D topological graph edge information. The propagation rule updates node embeddings as:
h: (a+c)h
Σ h JEN(O
(9)
where e is learnable and yp is two-layer perceptron with ReLU activation. Let HP E R denote the final node embedding matrix after the GIN interaction layer, which aggregates information across the conformer ensemble, and is -i-iirariant and R -iiinvariant from (1) of Lemma 1.
3.2.4 Gated Aggregation
While the GMoE mechanism handles self-attention within each conformer, we employ gated ag- gregation to facilitate information passing between conformer-level and molecular- features.

---

This cross-attention mechanism integrates conformer-level features H' E R(nx and the aggregated molecular-level GIN output features H' E R RMxxxx For ease of discussion, we squeeze the scalar dimension for both representations (dim=-2) in the following matrix multiplications. The gated integration is defined X Attn(H', H*), where the attention mechanism computes:
Q = HH
K HW
V H'W,,
(10)
QK
Att H* = softmax
V.
(11)
vdk
Here, We Rdxd, WK E Idxda, Wy E Rdd are learnable projection matrices, dę is the dimension of the attention space. After reshaping, we have the output X E R"*jy x x1xd This mechanism enables each conformer to selectively incorporate molecular-level and inter-conformer information, facilitating better ensemble representation learning. Since every input here are R=- invariant, the output X is also R -i-invvaiant from (1) of Lemma 1. We further prove in Section C.2 that X* is also -quivvarri.
Finally, we denote X ((++) = H" E R Vx4xx as the next-level bybrid input feature that satisfies -queevvivrr and R - thus completing our single H H-equivariant layer architecture.
3.3 Model Training
The proposed architecture consists of an equivariant graph neural network with L interaction layers. Each interaction layer contains N1 invariant experts and Ng equivariant experts process type-O and type-1 feature separately. For regression tasks, the objective is a weighted sum of the standard Mean Squared Error (MSE) loss and an auxiliary z-loss term. For classification tasks, binary classification head equipped with sigmoid activation is attached to the final molecular representations and we use a binary cross entropy loss for model training.
Experiments
In this section, we present comprehensive experimental evaluations of SPiCE across multiple molecular datasets and tasks. We first describe our datasets and experimental setup, then present results and analyses. We aim to answer the following research questions:
RQ1: Performance. How does SPiCE perform across different molecular property prediction tasks compared to state-of-the-ar conformer ensemble methods?
RQ2: Scalability. How does the performance of SPiCE scale with dataset size?
RQ3: Architecture design. What is the impact of key architectural choices in SPICE?
4.1 Experimental Setup
Datasets. We evaluate SPiCE on four datasets spanning both regression and classification tasks: (1) Drugs-7.5K, obtained by downsampling 10% of Drugs-75K [51] with a fixed random seed due to computational constraints, with three quantum mechanical properties: IP, EA, and x. (2) Kraken [52] with four 3D ligand descriptors: (Sterimol B,, Sterimol L., BurBs, BurL., and (3-4) CoV2 and CoV2- 3CL from BEOM-Druph [53]: CoV2 measures general inhibition in human cells, while CoV2-3CL specifically targets the 3CL protease inhibition. Prior to training, we perform preprocessing including conformer deduplication, clustering, and selection following the methodology described in prior works [30, 51J. We present detailed statistics and descriptions of datasets Appendix E.
Metrics. Following prior works, use Mean Absolute Error (MAE) for regression tasks and Receiver Operating Characteristic (ROC) area under the curve for classification tasks, particularly appropriate for the highly imbalanced datasets CoV2 and CoV2-3CL.
Baselines. We evaluate SPiCE against three categories of baselines. (1) 1D string-baser and 2D topological models, including fingerprints with random forest [54], extended 3D fingerprints with random forest [55]. and 2D GNN models (GIN [50], GIN with virtual nodes [56], and GraphGPS [57]).
(2) 3D single-conformer GNNs with random conformer sampling [51], including PaiNN [37]. ViSNet [58], ClofNet [13), and Equiformer [34]. (3) Conformer ensemble methods: For each
3D GNN backbone, we implement three conformer aggregation strategies following [51]: mean
7

---

Table Performance terms of MAE (4) for seven regression tasks (Drugs-7.5K, Kraken) and ROC scores (1) for two classification tasks (CoV2. CoV2-3CL). Bold and underlined values indicate best and second-best overall performance, respectively.
Ensemble
Drugs-7.50 (MAE, L)
Kraken (MAE, L)
CoV2
3CL
Backbone
Strutegy
IP
EA
X
L
BurBs
BurL
ROC (1)
ROC (1)
ID String-based and 2D Topological Approaches
Fingerprint-RFisa
0.5833
0.5277
0.3130
0.4760
0.4303
0.2758
0.1521
0.6071
0.9013
E3FP+RF
0.6217
0.5774
0.3464
0.6249
0,5535
0,3692
0.1908
0.6046
0.7676
GIN
0,5575
0.5116
0.2892
0.3128
0.4003
0.1719
0.1200
0.3708
0.5942
GIN-VN [56]
0.5398
0.5160
0.2937
0.3567
0.4344
0.2422
0.1741
0.4832
0.7387
GraphGPS 1571
0,5480
0.5054
0.2863
0.3450
0.4363
0.2066
0.1500
0.5601
0.8387
3D
Graph Neural Networks with Random Conformer Sampling 1512
PaiNN pn)
0.5557
0.5127
0.2924
0.3443
0.4471
0.2395
0,1673
0.2997
0.8368
[lo
0.6316
0.6008
0.3615
0.4473
0.6369
0.3216
0.2426
0.5233
0.7562
Equiformer(4) (34)
0.5471
0.4898
0.2887
0.2709
0.3759
0.2019
0.1526
0.4577
0.8035
ViSNet (58)
0.5393
0,4855
0,2985
0.3828
0.4495
0.2400
0.1755
0.5011
0.4774
Conformer Ensemble Approaches
ConfNet (60)
0.5760
0.5359
0.3057
0.4469
0.4680
0.2686
0.1657
0,5010
0.4930
ConAN-FGW Bi
0,5471
0.4945
0.2891
0.3242
0,5178
0.2026
0,1492
0.6340
0,9180
Mean
0.5410
0.4966
0.2963
0.2877
0.3950
0.1817
0.1472
0.5722
0,8850
PaiNN [3]]
DeepSets
0.5396
0.5091
0,2982
0.2225
0.3619
0.1693
0.1324
0.5802
0.6808
Attention
0.6318
0.5985
0.3488
0,3496
0.4109
0.2123
0.1506
0.4179
0.6984
SPICE
0.5281
0.4929
0,2792
0,2178
0.3548
0.1564
0,1292
0,5910
0.8880
Mean
0.5935
"0.441
0.3121
0.3986
0.5674
".2887
0.2327
0.3900
0,7580
ClofNet
DeepSets
0.5912
0,5533
0,3153
0.3314
0.5375
0.2532
0,1983
0.6208
0.7628
Attention
0.6694
0.5949
0.3578
0.4979
0.6118
0.3353
0.2502
0.3707
0.8182
SPiCE
0.5747
0.5283
0.3059
0.3193
0.4903
0,2477
0.1913
0.6730
1,0000
Mcan
0.5457
0.4932
0.2977
0.2303
0.3830
0.1680
0.1259
0.5601
0.8387
Equiformer
DeepSets
0.5404
0.4888
0.2990
0.2564
0.3772
0.1782
0,1234
0,5125
0.7134
Attention
0.5488
0.4923
0.2896
0.3187
0.4508
0.1673
0.1425
0.3882
0.7881
SPiCE
0.5318
0.4830
0,2816
0.2241
0.3456
0.1611
0.1229
0,5650
0,8405
Mcan
0.5593
0.4927
0.2862
0.2811
0.3970
0.1874
0.1469
0.6035
0.7447
ViSNet
DeepSets
0.5280
0.4987
0.2846
0.3104
0.4113
0.1716
0.1314
0.6626
0.4160
Attention
0.5593
0.4988
0.2944
0.3755
0.4195
0.2384
0.1394
0.5262
0.7158
SPICE
0.5384
0.4538
0.2814
0.2715
0.3807
0.1657
0.1277
0.6890
0.7195
pooling, DeepSets [33], and self-attention [59]. We also compare against specialized ensemble models: ConfNet [60] and ConAN-FGW [31]. Please refer to Appendix F for details of baselines.
Note that SPiCE is designed with equivariant representations. While we primarily use equivariant models due to their higher representational capabilities, we also evaluate invariant 3D models as baselines in Appendix H.
and--annd-lla framework that requires compatible backbones
Experimental settings. Following prior works, for regression datasets, we randomly partition data into training, validation, and test sets with 7:1:2 ratio, while classification datasets use fixed public splits. We optimize models using AdamW [61] with cosine decay scheduler. Training terminates the loss shows no improvement for 400 consecutive epochs. To ensure consistent comparison, the latent feature dimension to 128 and limit each molecule to a maximum of 20 conformers. For classification tasks, we address label imbalance by maintaining 1:1 ratio of positive to negative samples during training. Other configurations follow the original settings from respective papers. The details of experimental settings are provided in Appendix D.
set
4.2 Main Results (RQ1)
Table summarizes the performance across all tasks. Previous studies have established that conformer ensemble learning presents unique challenges: explicit set encoding can improve performance but makes training more challenging due to computational complexity, and model performance often shows strong task dependencies [30, 51]. Despite these challenges, SPiCE consistently
8

---

achieves superior performance compared to baseline methods, outperforming in 34 out of 36 total experimental configurations across all tasks and 4 base models and often surpassing the state-of-the-art model ConAN-FGW.
Notably, SPiCE demonstrates robust performance across datasets of varying sizes, from the smaller Kraken to the larger Drugs-7.5K and CoV2 datasets, suggesting that our architecture effectively balances computational efficiency with modeling capacity. The improvements are particularly significant with PaiNN, though performance gains vary across tasks, which is consistent with past observation in conformer ensemble modeling where different structural features may dominate different properties.
Regression tasks. On the Drugs-7.5K and Kraken datasets, SPiCE demonstrates consistent improve- ments across all backbone architectures. For Drugs-7.5K, SPiCE achieves relative MAF reductions of 2.79%, 7.89%, and 5.78% for IP, EA, and x respectively, compared to the second-hest strategy. This is due to its strong geometric feature extraction capabilities complementing our selective in- formation processing. The mid-sized Kraken dataset shows even more improvements, with MAE reductions of 3.66%, 8.78%, 7.61% %, and 3.53% across its four targets (Bs. L, BurBs, and BurL). The most significant improvement is observed with ClofNet BurBs, where MAE reduces from 0.5375 to 0.4903. It demonstrates that SPiCE can effectively enhance even simple yet powerful equivariant backbones, pattern we also observe in classification results.
Classification tasks. SPiCE shows strong performance on our highly imbalanced classification tasks. On the moderately imbalanced CoV2-3CL dataset (~1:10 positive-negative ratio), the PaiNN backbone achieves the most notable improvements, with ROC-AUC increasing by 10.95% and reaching perfect precision. The more challenging CoV2 dataset (~1::00 ratio) shows similar trends, increasing ROC-AUC by 8.41 141 % and thus highlighting the effectiveness of SPiCE and its robustness extreme class imbalance.
These comprehensive results highlight the effectiveness of our geometry-aware interaction networks in selectively integrating geometric and topological molecular information. The consistent im- provements across diverse tasks, backbones, and data distributions suggest that SPiCE successfully captures meaningful patterns in conformer ensembles.
4.3 Analysis of Model Scaling (RQ2)
Processing conformer ensembles introduces significant computational overhead, making it essential to understand how model performance scales with dataset size. To investigate the scalability of SPICE, conduct experiments
0.50
MAB EA
0.48
the EA task using the PaiNN backbone. Starting with the full Drugs-75K dataset (75,099 molecules, 558,002
0.46
0.44
conformers) [51], we create random subsets ranging from 10% to 100% of the data at 10% intervals.
1075
40%
70%
100%
(a) Dataset Size
Our analysis reveals two findings. First, SPiCE shows consistent im- provements with increasing data size, reducing MAE from 0.4929 (7.5K molecules) to 0,4386 (75K molecules). As shown in Fig- ure 2(a), the approximately linear scaling relationship suggests that our geometry-awee architecture effectively leverages additional training data without encountering performance plateaus that often character- ize capacity-limiie limited models. Second, we plot the loss trajectories at 30%, 70%, and 100% dataset sizes for the first 200 epochs in Fig- ure 2(b). They all show similar patterns despite varying data scales. This consistent optimization behavior suggests that SPICE maintains stable learning characteristics across different dataset sizes, making it suitable for applications with varied data.
Traizing Loss
2-
100
150
200
(b) Eporia
Figure (a) Performance as the dataset size increases from 7.5K to 75K molecules. (b) Training loss trajectories of the first 200 epochs for 30%, 70%, and 100% of 75K molecules.
4.4 Ablation Studies (RQ3)
To better understand the key design choices, we conducted extensive ablation studies using PaiNN as the backbone and EA as the task. We disabled or modified individual components of SPICE while keeping other parts unchanged. All experiments used the same protocol to ensure fair comparison.

---

Figure 3 presents our ablation results. First, the type-1 MoE component for vector feature processing demonstrates the highest importance, with its removal causing a substantial performance drop. This shows the necessity of dedicated equivariant feature han-
(n)
Victor fentuess only (1)pe1)
000003
Scaler Sorrme only (3ype-0)
(15125
(b)
Withont Gembee--ggrade stmpling
(0.0076
Aluas vie routing * Coulomcc-iie couling
Laveraa'e
Jumper
dling. Similarly, the topological aggregator proves crucial, confirming that molecular topology provides essential information for conformer ensemble rep- resentations. Third, the router mechanisms, both Gumbel sampling and nonlinear activation, signifi- cantly influence model performance, indicating that sophisticated expert selection mechanisms are funda- mental for effective specialization. Lastly, the gated aggregation component validates the selective cross- conformer integration, while the relatively smaller impact of atom-wise routing and layer-wise MoE suggests some implementation flexibility in these ar- chitectural aspects.
Withon spryding
0,3550
Gatel sgampric to Simple sumuutioo
Wilkoar topokigiol epprepiie
Redood expertx
n.coss
lacessed expects (12)
3050
Fewa activleed eaputa
Vor actiisted
0,4070
0,450
0.475
0.500
Performance (MAE)
Figure 3: Results of ablation studies regarding (a) feature processing, (b) routing mechanism, (c) training & integration, and (d) expert granularity.
4.5 Additional Experiments
We conducted several extra experiments further validate our approach. Ablation studies on MoE positioning and router z-loss regularization support our architectural choices. The experiments show that conformer-we routing with layer-wise MoE placement achieves optimal performance, and moderate z-loss regularization (A le -e= effectively balances expert utilization with training stability. Additionally, we evaluated SPiCE with invariant backbones and on the BDE dataset [51]. These additional experiments are documented in Appendices G and H.
5 Conclusions
We present SPiCE, a hierarchical framework for molecular conformer ensemble learning that com- bines three key architectural innovations: (1) shared conformer encoding for geometry-preserving molecular processing, (2) geometric mixture-of-exppeee features, and (3) hierarchical ensemble encoding that integrates molecular topology with selective cross-conformer communication. SPiCE maintains essential symmetries, both in conformer permu- tation and geometric transformations. Our comprehensive evaluation demonstrates the effectiveness of SPiCE across diverse molecular prediction tasks.
for specialized handling of scalar and vector
Acknowledgments
This work was partially supported by NSF Center for Computer Assisted Synthesis (2202693), National Artificial Intelligence Research Resource (NAIRR) Pilot (240280, 240443), National Science Foundation (2106859, 221 2557, 2119,43, 2200274, 2303037, 2312501), National Institutes of Health (U54HG012517, U24DK097771, U54OD036472), NEC, Optum AI, SRC JUMP 2.0
Center, Amazon Research Awards, and Snapchat Gifts.
References
[1] Daniel S. Wigh, Jonathan M. Goodman, and Alexei A. Lapkin. A Review of Molecular Representation the Age of Machine Learning. WIREs Comput. Mol. Sci., 12(5)xe1603, 2022.
[2] Jun Xia, Yanqiao Zhu, Yuanqi Du, and Stan 7. Li. A Systematic Survey of Chemical Pre-trained Models. In IJCAI, 2023.
[3] Laurianne David, Amol Thakkar, Rocío Mercado, and Ola Engkvist, Molecular Representations in AI-Driven Drug Discovery: A Review and Practical Guide. Cheminformatics, (15):
2020.
10

---

[4] Patrick Reiser, Marlen Neubert, André Eberhard, Luca Torresi, Chen Zhou, Chen Shao, Hous- sam Metni, Clint van Hoesel, Henrik Schopmans, Timo Sommer, and Pascal Friederich. Graph Neural Networks for Materials Science and Chemistry. Commun, Mater, 3(1):93, 2022.
[5] Mandana Saebi, Bozhao Nan, John E. Herr, Jessica Wahlers, Zhichun Guo, Andrzej M. Zurański, Thierry Kogej, Per-Ola Norrby, Abigail G. Doyle, Nitesh V. Chawla, and Olaf Wiest. On the Use of Real-Worke Datasets for Reaction Yield Prediction. Chem. Sci., 14(19):4977-5005, 2023.
1
[6] David Weininger. SMILES, A Chemical Language and Information System. J. Chem. Inf. Comput. Sci., 28(1):31-36, February 1988.
[7] H. Morgan. The Generation of a Unique Machine Description for Chemical Structures Technique Developed at Chemical Abstracts Service. J. Chem. Doc., 5(2):107-113, 1965. 1
[8] Robert C. Glem, Andreas Bender, Catrin H. Arnby, Lars Carlsson, Scott Boyer, and James Smith. Circular Fingerprints: Flexible Molecular Descriptors with Applications from Physical Chemistry to ADME. IDrugs, 9(3):199-204 2006.
[9] Michael M. Bronstein, Joan Bruna, Taco Cohen, and Petar Veličkovié. Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges. arXiv.org, April 2021. 1
[10] Kristof Schütt, Pieter-Jan Kindermans, Huziel Enoc Sauceda Felix. Stefan Chmicla, Alexandre Tkatchenko, and -lbbrrtobert Müller. SchNet: Continuous- Convolutional Neural Network for Modeling Quantum Interactions. In NIPS, pages 991-1001, 2017. 1, 16
[1I] Johannes Klicpera, Janek GroB, and Stephan Günnemann. Directional Message Passing for Molecular Graphs. In ICLR, 2020. 16
[12] Johannes Gasteiger, Florian Becker, and Stephan Günnemann. GemNet: Universal Directional
Graph Neural Networks for Molecules. In NeurlPS, pages 0790,6802,,00 2021. 1, 16
[13] Weitao Du, He Zhang, Yuanqi Du, Qi Meng, Wei Chen, Nanning Zheng, Bin Shao, and Yan Liu. SE(3) Equivariant Graph Neural Networks with Complete Local Frames. In ICML, pages 5583-5608, 2022. 1, 7, 22, 26
[14] Fabian Fuchs, Daniel Worrall, Volker Fischer, and Max Welling. SE(3)- (e(()))Taanformers, 3D Noto-Traansltion Equivariant Attention Networks. In NewrIPS, pages 1970-1981, 2020.
[15] Simon Batzner, Albert Musaelian, Lixin Sun, Mario Geiger, Jonathan P. Mailoa, Mordechai Kornbluth, Nicola Molinari, Tess E. Smidt, and Boris Kozinsky. Equivaaaar Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials. Nat. Commun.. 13(1):2453, 2022. 1,25
[16] Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E(n) Equivariant Graph Neural Networks. In ICMI., pages 9323-9332, 2021. 16
[17] Bharath Ramsundar, Peter Eastman, Patrick Walters, and Vijay Pande. Deep Learning for the Life Sciences: Applying Deep Learning to Genomics, Microscopy, Drug Discovery, and More. O' O'Reilly Media, 2019.
[18] Dmitry Zankov, Timur Madzhidov, Alexandre Varnek, and Pavel Polishchuk. Chemical Com- plexity Challenge: Is Multi-Instance Machine Learning A Solution? WIREs Comput. Mol. Sci., page e1698, November 2023. 2
[19] Emanuele Perola and Paul S. Charifson. Conformational Analysis of Drug-Like Molecules Bound to Proteins: An Extensive Study of Ligand Reorganization upon Binding. J. Med. Chem., 47(10):2499-2510, 2004. 2
[20] R.D. Levine. Molecular Reaction Dynamics. Cambridge University Press, 2009. 2
[21] Andrew F. Zahrt, Jeremy J. Henle, Brennan T. Rose, Yang Wang, William T. Darrow, and Scott E. Denmark. Prediction of Trigher-Selectvvng Catalysts by Computer-Driven Workflow and Machine Learning. Science, (66366644)caau56633 1, January 2019. 2

---

[22] Carolina H. Andrade, Kerly F. M. Pasqualoto, Elizabeth I. Ferreira, and Anton J. Hopfinger. 4D-QSAR: Perspectives Drug Design. Molecules, 155))3321-3324, 2010. 2
[23] Malini Ravi, Anton J. Hopfinger, Robert E. Hormann, and Laurence Dinan. 4D- Analysis of Set of Ecdysteroids and Comparison to CoMFA Modeling, J. Chem. Inf. Comput. Sci., 41 (6):1587-1604, November 2001.
[24] Dmitry Zankov, Timur Madzhidov, Pavel Polishchuk, Pavel Sidorov, and Alexandre Varnek. Learning Approach the Modeling of Enantioselectivity Conformationally Flexible Organic Catalysts. J. Chem. Inf. Model., 63(21)::629-6644 November 2023. 2
[25] Thomas G. Dietterich, Richard H. Lathrop, and Tomás Lozano-Pérez. Solving the Multiple Instance Problem with Axis-Parallel Rectangles. Artif. Intell., 89(1-2)31--1, 1997. 2
[26] Oded Maron and Tomás Lozano-Pérez. A Framework for Multiple-Instance Learning. In NIPS, pages 570-576, 1997.
[27] Maximilian llse. Jakub M. Tomezak, and Max Welling. Attention- bason Deep Multiple Instance Learning. In ICML, pages
2018. 2
[28] Charles Ruizhongtai Qi, Hao Su, Kaichun Mo, and Leonidas Guibas. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. In CVPR, pages 77-85, 2017. 2
[29] W. Kabsch. A Solution for the Best Rotation to Relate Two Sets of Vectors. Acta Cryst., 32(5): 922-923, Sep 1976. 2
[30] Simon Axelrod and Rafael Gómez-Bombarelli. Molecular Machine Learning with Conformer Ensembles. Mach. Learn. Sci. Technol., 4(3):35025, 2023. 2, 7,8
[31] Duy Minh Ho Nguyen, Nina Lukashina, Tai Nguyen, An T. Le, TrungTin Tin Nguyen, Nhat Ho, Jan Peters, Daniel Sonntag, Viktor Zaverkin, and Mathias Niepert. Structure-Aware E(3)-Invariant Molecular Conformer Aggregation Networks. In ICML, 2024. 2, 8, 22
[32] Haggai Maron, Or Litany, Gal Chechik, and Ethan Fetaya. On Learning Sets of Symmetric Elements. In ICML, pages 6734-6744, 2020. 3, 22
[33] Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabás Póczos, Ruslan R. Salakhutdinov, and Alexander J. Smola. Deep Sets. In NIPS, pages 3391-3401, 2017. 3, 8, 22
[34] Yi-Lun Liao and Tess Smidt. Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs. In ICLR, 2023. 3, 5, 7, 8, 17, 22, 24, 26
[35] Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. Layer Normalization. arXiv.org, July 2016.
[36] Stefan Elfwing, Eiji Uchibe, and Kenji Doya. Sigmoid- Linear Units for Neural Network Function Approximation in Reinforcement Learning. Neural Netw,, 107:3-11, 2018.
[37] Kristof Schütt, Oliver T. Unke, and Michael Gastegger. Equivariant Message Passing for the Prediction of Tensorial Properties and Molecular Spectra. In ICML, pages 9377-9388, 2021. 5, 8, 16, 22, 24, 26
[38] Junhyun Lee, Inyeop Lee, and Jaewoo Kang. Self-Attention Graph Pooling. In ICML, pages 3734-3743. PMLR, 2019. 5
[39] Hongyang Gao, Yi Liu, and Shuiwang Ji. Topology-Aware Graph Pooling Networks. arXiv.org, October 2020. 5
[40] Rex Ying. Jiaxuan You, Christopher Morris, Xiang Ren, William L. Hamilton, and Jure Leskovec. Hierarchical Graph Representation Learning with Differentiable Pooling. In NIPS, pages 4801-4811. 2018. 5
[41] Robert A. Jacobs, Michael I. Jordan, Steven J. Nowlan, and Geoffrey Hinton. Adaptive Mixtures of Local Experts. Neural Comput.. 3(1):79-87, 1991. 5, 17
12

---

[42] David Eigen, Mare' Aurelio Ranzato, and Ilya Sutskever, Learning Factored Representations in a Deep Mixture of Experts. In ICLR, 2014. 5
[43] William Fedus, Barret Zoph, and Noam Shazeer. Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity. J. Mach. Learn, Res., 23:120:1-120:39 2022. 5, 17
[44] Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, and Zhifeng Chen. GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding. In ICIR,, 2021. 5, 17
[45] Thomas N. Kipf and Max Welling. Semi-Supervised Classification with Graph Convolutional Networks. In ICLR, 2017. 5
[46] Barret Zoph. Irwan Bello, Sameer Kumar, Nan Du, Yanping Huang, Jeffrey Dean, Noam Shazeer, and William Fedus. ST-MoE: Designing Stable and Transferable Sparse Expert Models. arXiv.org, February 2022. 6, 17, 18
[47] Wouter Kool, Herke van Hoof, and Max Welling. Stochastic Beams and Where To Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacement. In ICML,, pages 3499-3508, 2019.
[48] Xi Victoria Lin, Akshat Shrivastava, Liang Luo, Srinivasan Iyer, Mike Lewis, Gargi Ghosh, Luke Zettlemoyer, and Armen Aghajanyan. MoMa: Efficient Early-Fusion Pre-training with Mixture of Modality-Awar Experts. arXiv.org, July 2024. 6, 17
[49] Aran Komatsuzaki, Joan Puigcerver, James Lee-Thorp, Carlos Riquelme Ruiz, Basil Mustafa, Joshua Ainslie, Yi Tay, Mostafa Dehghani, and Neil Houlsby. Sparse Upcycling: Training coxture-of-Expertt from Dense Checkpoints. ICLR, 2023. 6,
[50] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How Powerful Are Graph Neural Networks? In ICLR, 2019. 6, 7,
[51] Yanqiao Zhu, Jechyun Hwang, Keir Adams, Zhen Liu, Bozhao Nan, Brock Stenfors. Yuanqi Du, Jatin Chauhan, Olaf Wiest, Olexandr Isayev, Connor W. Coley, Yizhou Sun, and Wei Wang. Learning Over Molecular Conformer Ensembles: Datasets and Benchmarks. In ICLR, 2024. 7,
8, 9, 10,22 22
[52] Tobias Gensch, Gabriel dos Passos Gomes, Pascal Friederich, Ellyn Peters, Théophile Gaudin, Robert Pollice, Kjell Jorner, AkshatKumar Nigam, Michael Lindner-D' Addario, Matthew Sigman, and Alán Aspuru-Guzik. A Comprehensive Discovery Platform for Organophosphorus Ligands for Catalysis. J. Am. Chem. Soc., 144: 1205-1217,
January 2022.
[53] Simon Axelrod and Rafael Boanez--ommbrell... GEOM, Energy-Annotated Molecular Con- formations for Property Prediction and Molecular Generation. Sci. Data, 91)):185, 2022.
7
[54] Joseph L. Durant, Burton A. Leland, Douglas R. Henry, and James G. Nourse. Reoptimization of MDL Keys for Use in Drug Discovery. J. Chem. Inf. Compul. Sci., 1236) 2002. 7,8
[55] Seth D. Axen, Xi-Ping Huang, Elena L. Cáceres, Leo Gendelev, Bryan L. Roth, and Michael J. Keiser. A Simple Representation of Dimensionnn Molecular Structure. J. Med. Chem., 60(17):7393-7409, 2017. 7, 8
[56) Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open Graph Benchmark: Datasets for Machine Learning on Graphs. In NewrlPS, pages 22118 10111122133,3 2020. 7,8
[57] Ladislav Rampášek, Mikhail Galkin, Vijay Prakash Dwivedi, Anh Tuan Luu, Guy Wolf, and Dominique Beaini. Recipe for a General, Powerful, Scalable Graph Transformer. In NewrIPS, pages 14501-14515, 2022. 7,8
13

---

[58] Yusong Wang, Tong Wang. Shaoning Li, Xinheng He, Mingyu Li, Zun Wang, Nanning Zheng. Bin Shao, and Tie- Yan Liu. Enhancing Geometric Representations for Molecules with Equiv- ariant Vector-Scalar Interactive Message Passing. Nat. Commun., 15(1):313, 2024. 7, 8, 22, 26
[59] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Uszkoreit Kaiser, and Illia Polosukhin. Attention is All You Need. In NIPS, pages 5998-6008,
2017. 8, 17, 22
[60] Meng Liu, Cong Fu, Xuan Zhang, Limei Wang, Yaochen Xie, Hao Yuan, Youzhi Luo, Zhao Xu, Shenglong Xu, and Shuiwang Ji. Fast Quantum Property Prediction via Deeper 2D and 3D Graph Networks. arXiv.org, June 2021. 8, 22
[61]
Ilya Loshchilov and Frank Hutter. Decoupled Weight Decay Regularization. In ICLR, 2019.
[62] Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural Message Passing for Quantum Chemistry. In ICML, pages 1263- 1272, 2017. 16
[63] Johannes Brandstetter, Rob Hesselink, Elise van der Pol, Erik Bekkers, and Max Welling. Geometric and Physical Quantities Improve E(3) Equivariant Message Passing, In ICLR, 2022.
17
[64] Yi-Lun Liao, Brandon Wood, Abhishek Das, and Tess Smidt. EquiformerV2: Improved Equivariant Transformer for Scaling Higher-Degree Representations. arXiv.org, June 2023.
17
[65] Alexandre Duval, Simon V. Mathis, Chaitanya K. Joshi, Victor Schmidt, Santiago Miret, Fragkiskos D. Malliaros, Taco Cohen, Pietro Lio, Yoshua Bengio, and Michael M. Bronstein. A Hitchhiker's Guide Geometric GNNs for 3D Atomic Systems. arXiv.org, December 2023. 17
[66] Juho Lee, Yoonho Lee, Jungtaek Kim, Adam R. Kosiorek, Seungjin Choi, and Yee Whye Teh. Set Transformer: A Framework for Attention-bases Permutation-Invariae Neural Networks. In ICML, pages 3744-3753, 2019. 17
[67] Ryan L. Murphy, Balasubramaniam Srinivasan, Vinayak A. Rao, and Bruno Ribeiro. Janossy Pooling: Learning Deep Permutation-Innarrianl Functions for Variable-Size Inputs. In ICLR, 2019. 17
[68] Mathieu Blondel, Olivier Teboul, Quentin Berthet, and Josip Djolonga. Fast Differentiable Sorting and Ranking. In ICML, pages 950-959, 2020. 17
[69] Yan Zhang, Jonathon S. Hare, and Adam Prigel-Benneee FSPool: Learning Set Representations with Featurewise Sort Pooling. ICLR, 2020.
17
[70] Chirag Pabbaraju and Prateek Jain, Learning Functions over Sets via Permutation Adversarial Networks. arXiv.rrg 2019. 17
[71] Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc V. Le, Geoffrey Hinton, and Jeffrey Dean. Outrageously Large Neural Networks: The Sparsely-Gated Mixture- of-Experts Layer. In ICLR, 2017. 17
[72] Basil Mustafa, Carlos Riquelme, Joan Puigcerver, Rodolphe Jenatton, and Neil Houlsby. Multi- modal Contrastive Learning with LIMoE: the Language- Image Mixture of Experts. In NeurIPS. 2022. 17
[73] Damai Dai. Chengqi Deng, Chenggang Zhao, R X Xu, Huazuo Gao, Deli Chen, Jiashi Li, Wangding Zeng, Xingkai Yu, Y Wu, Zhenda Xie, Y K Li, Panpan Huang, Fuli Luo, Chong Ruan, Zhifang Sui, and Wenfeng Liang. DeepSeekMoF: Towards Ultimate Expert Specialization in Mixture-of-Fippetss Language Models. arXiv.org. January 2024. 18, 23
[74] Yanqi Zhou, Tao Lei, Hanxiao Liu, Nan Du, Yanping Huang, Vincent Y. Zhao, Andrew M. Dai, Zhifeng Chen, Quoe V. Le, and James Laudon. Mixture-of-Experts with Expert Choice Routing. In NewrIPS, 2022. 18
14

---

[75] Ted Zadouri, Ahmet Üstün, Arash Ahmadian, Beyza Ermis, Acyr Locatelli, and Sara Hooker. Pushing Mixture Experts the Limit: Extremely Parameter Efficient MoE for Instruction Tuning. In ICIR, 2024. 18
[76] Joan Puigcerver, Carlos Riquelme Ruiz, Basil Mustafa, and Neil Houlsby. From Sparse to Soft Mixtures of Experts. ICLR, 2024. 18
[77] Benjamin Meyer, Boodsarin Sawatlon, Stefan Heinen, O. Anatole von Lilienfeld, and Clémence Corminboeuf. Machine Learning Meets Volcano Plots: Computational Discovery of Cross- Coupling Catalysts. Chem. Sci., 9:7069-7077, 2018. 24
[78] Noel M. O'Boyle, Michael Banck, Craig A. James, Chris Morley, Tim Vandermeersch, and Geoffrey R. Hutchison. Open Babel: An Open Chemical Toolbox. J. Cheminform., 3(1):1-14, 2011. 24
[79] Johannes Gasteiger, Shankari Giri, Johannes T. Margraf, and Stephan Günnemann. Fast and Uncertainty-Awan Directional Message Passing for Non-Equilibrium Molecules. In MLAMolecules @ NeurlPS 2020, 2020. 24, 25
[80] Julia Balla, Siddharth Mishra-Sharma Carolina Cuzsta-Lazaro, Tommi Jaakkola, and Tess Smidt. A Cosmic-Scale Benchmark for October 2024. 25
Data Processing. arXiv.org,
[81] Chaitanya K. Joshi, Cristian Bodnar, Simon V. Mathis, Taco Cohen, and Pietro Liò. On the Expressive Power of Geometric Graph Neural Networks. In ICML, pages 15330-15355, 2023. 25
15

---

Supplementary Material for SPiCE
A Related Work
16
A.1 Geometric Graph Neural Networks
16
A.2 Learning over Sets
17
A.3 Mixture-of-Experts
17
py Torch-like Pseudocode
18
C Theoretical Details
18
C.1 equuiivaria Linear Layer Structure
18
C22 S, equivarrae Cross-Attention Output X
20
D Details of Experimental Settings
21
Details of Datasets
F Details of Baselines
22
G Detailed Ablation Studies
23
H Additional Experiments
24
Code and Dataset Availability
25
Limitations and Future Work
25
K Raw Performance Data
25
A Related Work
A.1 Geometric Graph Neural Networks
Message-passing neural networks have been widely adopted in modeling atomistic systems, where nodes represent atoms and edges represent chemical bonds [62]. When incorporating 3D geometric information, these models can be broadly categorized into invariant and equivariant architectures, based on how they handle geometric transformations.
Invariant models satisfy the condition:
<g =) (G)), Vg E E(3)/SE(3),
These models transform equivariant coordinates X into invariant scalar features (type-O) that re- main unchanged under Euclidean transformations. Common invariant features include pairwise distances [10], triplet angles [11], torsion angles [12], etc.
Equivariant models, which satisfy:
Qgg = (GG), Ag E E(3)/SE(3),
can be further divided into two subcategories. Cartesian equivariant models (e.g.. EGNN [16]. PaiNN [37]) operate directly in Cartesian coordinates, processing scalar (type-0) and vector (type-1)
16

---

features while preserving equivariance through restricted operations. In contrast, spherical equivariant models (e.g., SEGNN [63], Equiformer [34, 64J) convert geometric information into steerable features using spherical harmonics, process rotations through Wigner-D matrices, and combine features using Clebsch-Gordan tensor products. This formulation naturally extends to higher-order geometric features, making it particularly effective for capturing complex geometric patterns in molecular structures.
Readers of interest may refer to Duval et al. [65] for a complete review of geometric graph neural networks for 3D atomic systems. We would like to remark that our SPiCE framework agnostic to the choice of geometric GNN architecture, allowing practitioners to select the most suitable backbone for their specific application while maintaining the benefits of our conformer ensemble processing approach.
A.2 Learning over Sets
Learning commuttationinvariinnt functions over sets is fundamental problem in machine learning. with approaches broadly falling into several categories: direct permutation invariance, sorting-based based methods, and approximate invariance.
Deep Sets [33] introduced a simple yet powerful architecture where a permutation-sensitee network processes each element independently, followed by permutation- invariant aggregation operation. Set Transformers [66] extend this using attention mechanisms for more expressive set processing. DSSNet [32] generalizes this framework to handle symmetries beyond permutation invariance. They show that when working with symmetries that combine permutations with element-wise transformations, the processing can be decomposed into two parts: one that handles individual elements and another that processes information aggregated across the entire set. This framework naturally fits our molecular setting, where we need to maintain both the permutation invariance of conformer ensembles and the geometric symmetries of individual conformers. More sophisticated approaches like Janossy pooling [67] capture higher-order interactions by processing subsets of elements together. Self-attention mechanisms [59] offer another perspective, effectively comparing pairs elements through learned relationships between queries and keys.
Direct permutation invariance.
perting-basedd methods. Another strategy achieves permutation invariance by first arranging elements into a canonical order. Recent advances include methods for learning the sorting function itself [68] and approaches that sort based on learned features [69]. While these methods are computationally efficient, they face challenges training due to the discrete nature of sorting operations.
Approximate invariance. Some methods trade exact invariance for computational efficiency. These include stochastic approaches that sample random permutations [67] and adversarial training tech- niques that encourage approximate invariance [70].
Our work, while informed by these approaches, focuses on maintaining exact invariance through the principled decomposition provided by DSSNet [32]. We propose novel type-separated Mixture- of-Experts mechanism to enable selective information processing while achieving the required symmetry.
A.3 Mixture-of-Expers
When processing molecular conformers, certain atoms or structural features may be more relevant than others for specific propertics. Graph pooling approaches like SAGPool achieve this selective processing by focusing computation on important nodes through attention mechanisms. This naturally extends to Mixtture-of-Exeen (MoE) architectures that employ multiple specialized neural networks with learned routing mechanism.
The concept of MoE originated from adaptive mixtures of local experts [41], where separate networks handle different subsets of training cases. Recent developments have transformed MoEs from standalone models to components within deeper architectures [71], enabling selective computation at various scales. Modern MoE architectures have demonstrated remarkable success in large-scale models [43, 44, 46, 48, 72], achieving improved data efficiency and computational performance through sparse activation patterns.
17

---

Key design considerations of MoE include balanced expert utilization through load balancing, router design, sparse gating for computational efficiency, and appropriate capacity factors. Router loss [46] penalizes extreme routing decisions to prevent expert collapse and improve training stability, Expert capacity factors control the maximum number of tokens per expert, balancing computational efficiency with model performance [73]. While traditional MoEs use token-to-expert routing, Expert Choice routing [74] reverses this by allowing experts to select tokens, improving load balancing and computational efficiency. Recent innovations include parameter-<fficient MoE variants [75], and Soft MoE [76], which replaces discrete expert assignment with differentiable soft routing to address training instability and scaling limitations. These developments collectively enable more efficient and stable MoE architectures while preserving the benefits of specialized computation.
This MoE approach is suitable for conformer ensemble modeling as experts can specialize in different geometric patterns or conformational states. We further leverage gated aggregation mechanism which can adaptively weigh the importance of different conformers based on both local geometric features and global molecular context.
B PyTorch-like Pseudocode
PyTorchlike pseudocode is given in Algorithm $1.
C Theoretical Details
C.1 S, Squiiiivri Linear Layer Structure
Theorem 1. Consider S, equivvaraa linear layer L that also respects G G-symmetry.
If L takes as input G-equivariant features Rex3xd, then L(X); = Lo(x;)) operates on each conformer separately;
2. If L takes input G-invariant features Ruxlxd, then L be decomposed L(X)s In(x;) - L2 1 i.e. a local interaction module and global aggregation module.
Proof of Theorem /. For the first claim, we begin by noting that any linear function of equivariant features, if it respects G-symmetry, must be a G-equivariant linear function, hence the output L(X) is also G G-equivariant, and operates the 3 coordinate channels of the second dimension of L simultaneously. Next we decompose into separate components from each individual conformer:
L(X), LL Lzy(),)
(S1)
where each Lijj must be a G-equivariant function. Consider the transformation = (g),,.. .9)) E G" on input X: go X = [g] 0 ænJ. where -1,-.. :9 are independent transformations in G. From the G equuvaaaaaa constraint, we have for any g E G" that
g + Lo xx g. (Z
By fixing gi and varying gj for all j A i, we see the right hand side remains constant, while each Lig 2,2) (x;;) on the left hand side individually changes with different 9j ss. Thus the equation can only hold for any g (x,) interactions. and consequently (SI) becomes
for any j. i.e. there are no cross-conformer
L(X): = (a,(
From here we finally consider S, -quiivaiaee for a permutation p E Sn, defined by p [wa] H [<p()]== we have
po L( L(p X) (200.0(0 Li,((x(()) Vi,
which suggests by the arbitrariness of that Li, Ligs V. Writing Lo Li, finishes the proof.
18

---

Algorithm S1 Pseudocode of SPiCE in PyTorch-like style.
# Inputs:
2 batch [z, c, [tom_idx]
# (Nx1) atomic numbers for all M conformers
4 с : (Nx3) atomic coordinates for all conformers
# atom_ idx : (Nx1) index mapping each atom to its molecule ID
6 con : (Nx1) index mapping each atom to its conformer
8 --- 1) Preprocess
# build scalar & vector features and auxiliary graph attributes x_s, x_v, 3d_an aux 3nng preprocees (batch)
- 2d_aux GIN. GIN.GIN...N.... preprocesssss (batch)
12
# --- 2) Interaction block
for i in range (aum_blocks) type-0 feature x_s: Nx1xC
16
type-1 feature x_v: Nx3xC
h_s, h_v, 3d_aux 3d_gnn ffowarrd x_v, 3d_aux)
normalized type-0 feature h_s: Nx1xC normalized type-1 feature h_v: Nx3xC h_s ELN ELN_s[i] (x_s h_s)
20
b_v ELN_v (x_v h_v)
GMoE block, Eqn. (2) moe_x: NxixC, moe_v: Nx3xC # router scores: Nx [x(N_I+__E
25
27
moe_x, moe_V, scores gmoe_ [lockbl (h_s, h_v)
29
conformer set information sharing h_ bar, h b_bar__s MxC
31
b_bar scatter (moe_x, conf_idx, dim=0, reduce= 'mean') b_bar_s = GIN[i] (h_ bar, 2d_aux)
33
gated aggregation, Eqn. (10)-(11)
35
rev_idx: mapping from conformer-level back to atom ordering x_g gated_aggr [i] (h_s, h_bar_s [rev_idx])
x_g = x_g uns.uusqqeeeze (1) x_g: Nx1xC
feature update
x_s x_f
x_V = moe_V
# --- 3) Postprocess ---
convert final node-level features to molecule-level outputs out 3DCNN WDG.N. wostppocesssx..ss... (x_s, x_v, 3d_aux, stom_idx)
19

---

For the second claim, clearly the output of any G-invariant function is also G- invariant, and cannot contain any type-1 features. We similarly decompose ((X) as
I(X)= = Σ Las (zz):
(S2)
where each Lig is an G-invariant linear function. Consider E Sn again, we have po ((x) L(p X) + L(X)
p(1) p()) = L(p X); Vi
(x,)
= Σ (
j=1
(20))(0))) = Σ (k) Vi
]=1
k=1
(p())())))))) Tup (x;)] = 0, Vi,
where in the third equivalence we used k = p(j) to rewrite the summation. From the independent variabilities of æ,, each member of the above summation must take value 0, in other words
pp(0ss Lip -0) Vi.j.p.
which is equivalent to
Lp(0)(()) = Li, Vi, k.
via substitution of = p)) This means:
Li,; = Lij- = Le: Ji
Lii.ji = Liz = Lus Vi, + j1, 12 + 82i
and therefore (S2) becomes
L(X); Le(w))) + Σ In(a))
341
(Le - Lx)( Iep
Σ
x
where in the second equality used the fact that the summation of shared linear mapping of components equal to the same linear mapping of the summation of components. Taking L1 = Le Ln and L2 L,, finishes the proof.
C.2 equivvarree of Cross-Attention Output X*
Here we elaborate on the symmetry preservation of the gated aggregation operation discussed in Section 3.2.4. and prove the equivviivee of the attention output. Looking at (10) and (11), notice the input H' is equivvaiai while Ha is -nvinnarian,, therefore for a permutation E Sn.
(H)
Attnlpo o H po H)) Autnpp H", H) softmax
HWv
Vdh
(H Vd
p softmax
HW, = po Attn H+), (S3)
where the third equality holds since individual operations on each conformer preserves permutation cquivariance. This proves the overall S, S,,qquivarianne of the attention module.
20

---

Table S1: Hyperparameters for each backbone model on dataset Drugs-7.5k, and Kraken.
Dataset
Backbone
Epochs
Batch
LR
Patience
Experts
Act. Experts
Upcycle
PaiNN
2000
32
2e-4
400
8
100
0.1
lo-3
ViSNet
2000
32
1.5c-4
600
8
50
0.1
lc-3
Drugs-7.5K
ClofNet
2000
32
1.5e-4
400
8
50
0.1
le-3
Equiformer
2000
32
2c-4
400
8
50
0.1
lo-3
PaiNN
2000
16
3e-4
400
16
50
0.1
le-4
ViSNet
2000
32
4c-4
400
16
100
1.0
0.1
Kraken
ClofNet
2000
16
Ic-4
400
8
100
1,5
lc-3
Equiformer
2000
3e-4
400
16
50
1.0
le-4
D Details of Experimental Settings
Different backbones have varying architecture complexities and capacities learn molecular repre- sentations. Datasets also differ in size and complexity of the prediction task. Hyperparameter tuning for each backbone-dataset pair ensures optimal performance by balancing model capacity, data size, and task difficulty.. For example, larger datasets like Kraken may require more experts and epochs to sufficiently learn, while simpler backbones like ClofNet need fewer epochs on Drugs-7.5K, The hyperparameters include number of experts, number of activated experts, Gumbel-Softmax sampling temperature, auxiliary z-loss weight, and upcyeling epochs.
For regression tasks, each backbone model with a different aggregation strategy is trained with the same set of hyperparameters, optimized through 20 iterations of Bayesian Optimization. Specific settings are summarized Table S1 for regression datasets. For classification tasks, all experiments use the same settings PaiNN on Drugs-7.5k. classification is less sensitive to hyperparameters than regression and Drugs- 1.5gs-7.5K. 7,5K is a reliable molecular property prediction benchmark.
E Details of Datasets
We evaluate our method on four diverse molecular datasets. Their statistics is summarized in Table S2,
Drugs-7.5K. A subset of 7,500 molecules downsampled from GEOM-Drugs dataset, with three quantum mechanical properties:
lonization Potential (IP): The energy required to remove an electron from a neutral molecule (IP -Eation Eneutrat).
Electron Affinity (EA): The energy change when adding an electron (EA Encumal Ennion).
Electronegativity (x): Measuring electron attraction tendency (x -EE/ON).
Kraken. dataset of 1,552 monodentate organophosphorus(I)) ligands with DFT-computed conformer ensembles. We focus on four 3D steric descriptors (measured in Å): Sterimol B5, Sterimol L, buried Sterimol Bs, and buried Sterimol L, which are crucial for QSAR modeling in catalyst design.
Two highly imbalanced classification datasets derived from GEOM-Druge containing experimental data for SARS-CoV-2 inhibition:
CoV2 and CoV2-3CL.
CoV2-3CL: Tests specific inhibition of the SARS-CoV-2 3CL protease.
CoV2: Evaluates general SARS-CoV-2 inhibition in human cell assays.
The class imbalance summarized in Table $2b, where positive samples (hits) represent only a small fraction of the total molecules.
21

---

Table S2: Dataset statisties for all four datasets and dataset splits for COViD-related datasets.
Dataset
#Molecules
#Conformers
#Heavy atoms
#Rot. bonds
Split
CoV2-3CL
CoV2
Drugs-7.5K
7,509
54,202
30.67
7.45
Train
50 (485)
53 (3,294)
Kraken
1,552
21,287
23.70
9.05
Validation
(157)
17 (1,096)
CoV2
5,466
72,744
24.57
4.83
Test
(162)
22 (1,086)
CoV2-3CL
755
7,742
14.51
Total
(804)
92 (5,476)
(a) Molecular composition statistics. Numbers of heavy atoms and rotatable bonds ("rot. bonds") are averaged per molecule.
(b) Dataset partitioning showing number of active compounds (to-
tal compounds in parentheses)
Details of Baselines
We evaluate SPICE against two categories of baselines: ensemble learning methods with explicit set encoders and specialized conformer ensemble models.
Ensemble learning with set encoders. We implement our framework with four representative 3D backbones that guarantee E(3) SE(3) equivariance: PaiNN [37], ViSNet [58], ClofNet [13], and Equiformer [34]. For each backbone, we compare GAIN against three conformer aggregation strategies following Zhu et al. [51]:
Mean pooling: The simplest approach that computes the average of conformer cebedddnemmbbddiingg (h, ) generated by 3D GNNs:
Σ h4- n2
1
hMEAN
(S4)
DeepSets [33]: A permutation-invarian function that processes the ensemble through Multi-Layer Perceptron (MLP) ø, followed by sum pooling and another MLP pz
hps MLP, Σ (L.))
(S5)
where ML.P& transforms individual embeddings and MIP, processes the aggregated features.
This approach preserves more individual conformer information than mean pooling at the cost of additional non-linear transformations.
Self-attention [59]: Computes weighted sum of embeddings using attention scores:
HATIS Σa,bo
aa
(S6)
a, = softmax ( Wh;).
(S7)
This approach captures pairwise interactions between conformers through learned attention weights.
After obtaining the ensemble embeddings through these encoders, a linear projection head generates the final predictions.
Specialized conformer ensemble models. We also compare against two state-of-the-art models specifically designed for conformer ensemble learning:
ConfNet [60]: Extends DSSNet [32] as explicit set encoder, applying permutation-invariant operations directly to the conformer ensemble while maintaining geometric equivariance.
ConAN-FGW [31]: Introduces novel 2D-3D aggregation mechanism based on the Fused Gromov- Wasserstein Barycenter problem. It combines this with efficient conformer generation using distance geometry through RDKit, enabling joint optimization of conformer generation and property prediction.
22

---

0.50
0.8
Upeycling Fporhs
Final Validation MAE
Validation MAE
Upeycling Epuche 50
Upoyeting Spochs
0.49
0.7
0.48
0.6
0.47
0.5
0.46
100
200
300
100
500
0
50
100
(a) Epoch
(b) Upeycling Epochs
Figure $1: Impact of upcycling epochs model performance. (a) Validation loss curves for different upcycling epochs on the Drugs-7.5K dataset using ViSNet backbone for EA prediction. (b) MSE comparison of validation errors.
0.507
0.505
0.505
Total Experts
Test MAE
0.50
0.500
16
0.497
0.487
0.408
0.49
0.495
Tatt Firted
0.202
11,505
0..90
0.48
10
110
310
$107
2
(a) Number of Expert Choiers
(b) Activated Experts
Figure S2: Analysis of expert combinations model performance. Results show how varying the number of total experts (8-32) and active experts (2-8) affects prediction accuracy on the Drugs-7.5K dataset using PaiNN backbone for EA prediction.
G Detailed Ablation Studies
To better understand the key design choices in SPiCE, conduct comprehensive ablation studies examining four critical aspects: (1) the effectiveness of sparse upcycling for stabilizing carly-stage training. (2) the impact of expert granularity of TS-MoE within the architecture, and (4) the influence of router 7-loss on training stability and expert utilization. Here present analyses of the first two aspects, while full results for the rest provided in Appendix G.
model performance, (3) the optimal positioning
Sparse upeycling. Training both GNN representation and MoE components simultaneously can be challenging early stages. The sparse upcycling strategy [49] addresses this by separating the training into two phases: first training single expert optimize representation learning, then cloning into multiple experts with a newly initialized router for specialization.
As shown in Figure S1, we evaluate different upcycling epochs (0, 50, 100) on the Drugs- 7.5gs dataset using ViSNet for EA prediction. Results indicate that 50 upcycling epochs yield optimal performance, while excessive upcycling epochs can slow convergence. This confirms that appropriate upcycling improves both convergence speed and final accuracy.
Expert granularity. While finer expert granularity can increase model flexibility through more possible combinations (e.g.. (*) = 28 vs. (10) = 1820) [73], also introduces computational overhead and potential training instability. Figure S2 presents systematic study varying the number of total experts (8, 16, 32) and activated experts (2, 4, 8) on the Drugs-7.5K dataset with PaiNN backbone. The results reveal U-shaped performance curve, with optimal performance
(16 1820 combinations. This suggests sweet spot balancing model flexibility with computational efficiency for molecular conformer ensemble representation.
MoE positioning. Table S3a compares different MoE configurations on the Kraken dataset's s BurB; target using PaiNN. Our experiments examine three key design choices: conformer-wise vs. atom-wise routing, aayer-wise MoE placement, and non-linear router activation. Results show that conformer-wise routing with layer-wise MoE and softmax activation achieves the best performance, supporting design choice of treating conformers rather than atoms routing objectives.
23

---

Table S3: Ablation studies on (a) different architectural choices, (b) router z-loss weight (A), Results show validation MAE on Kraken (BurBs) and Drugs-7.5K (EA) datasets using PaiNN and ViSNet respectively.
Atom-Wiit Routing
Layer-
Jayer-Wise Wise MoE
Router Activation
À
MAE
MAE
6
0.4973
X
/
0.1645
1e
0.4960
/
0. 0.1515
0.1546
0.4888
/
/
0.1470
le
0,4985
(a) Impact of architectural choices
(b) Z-loss À
Router z-loss. Router 7-loss penalizes extreme routing decisions and helps balance expert utilization. Table S3b shows the impact of different z-loss weights (A) on validation eπrоr using the ViSNet backbone Drugs-75K. A moderate weight of 1e achieves optimal performance, suggesting that proper regularization improves model stability without overly constraining expert specialization. Larger weights, while promoting more balanced expert utilization, can lead training instability due to excessive penalization.
H Additional Experiments
BDE dataset. To showcase our method is effective and scalable to reaction-based prediction task, we conducted experiments on the BDE dataset [77]. The BDE dataset comprises 5,915 organometallic catalysts L2) with diverse ligands and metal centers, including DFT-computed binding energies for conformer ensembles of unbound and bound states. Conformers are generated via Open Babel [78] and force field optimizations, approximating global minima. Due to the high cost of DFT optimization, precise conformer ensembles are generally unknown at inference, making this realistic and challenging benchmark. The goal is to predict binding energies from either individual or ensemble conformers of the catalyst both states.
Different from the experimental settings in the main paper, this task involves two distinct sets of conformers as input, corresponding to the unbound and bound states of the catalyst. accommodate this, we employ two garameter-independeee copies of SPiCE, with their regression heads removed, to separately process each conformer set. The resulting molecular-level features are then concatenated and passed through regression head to predict the final binding energy. Table S4a presents the performance of SPICE on this dataset in comparison with the two other ensemble models Mean and DeepSets with PaiNN [37] and Equiformer [34] as backbones. It is seen that SPiCE consistently outperforms the baselines, demonstrating its effectiveness in reaction-based prediction tasks.
Invariant Type-0-only model. Our SPiCE framework is designed that can be adapted work with different geometric neural network backbones. While our method
soug-and-plas architecture
Table $4: Additional experiments on the BDE dataset and invariant models. Bold values indicate best performance.
Backbone
Model
Binding Energy
Model
B;
Mean
1.8744
PaiNN [37]
DeepSets
1.9164
Mean
0.3172
0.4258
SPICE
1.8528
DeepSets
0.2627
0.3777
Mean
1.9136
Equiformer [34]
SPiCE
DeepSets
1.9540
0.2446
0.3379
(Invariant)
SPICE
1.8978
(b) Invariant type-0-only model on Kraken dataset with DimeNet++ [79] backbone.
(a) Reaction baaecd regression task (BDE) in MAE (l).

---

can accommodate both invariant and equivariant backbones, we primarily focus equivariant 3D GNNs (PaiNN, ClofNet, Equiformer, and VisNet) due to their superior representational capabilities. Equivariant features have been shown to provide steeper learning curves, improved data efficiency, and finer angular resolution for capturing molecular structural details [15, 80, 81].
To demonstrate the versatility of SPiCE and address potential questions about backbone compat- ibility, we developed a simplified variant that operates exclusively on invariant (type-0) features, compatible with invariant-only backbones such as DimeNet++ [79]. This adaptation removes the type-1 processing branch including vector features, type-1 router, and the corresponding expert network to accommodate scalar-only representations.
We evaluate this invariant variant the Kraken dataset against strong baselines including DeepSets and mean pooling (results in Table S4b). We note that this experiment not intended as a direct comparison between invariant and equívariant approaches. Rather, it validates the effectiveness of our design a constrained setting and confirms the adaptability of our model across different architectural paradigms. Even without equivariant information, our adapted framework can outperform the invariant baselines, demonstrating that the core design remains beneficial across different geometric representations.
I Code and Dataset Availability
The implementation of this work can be found in this repository: https://github: com/ DannieS YD/SPiCE.
Limitations and Future Work
While SPiCE achieves strong performance across variety of molecular property prediction tasks, the hierarchical architecture and dependence on equivariant GNN backbones introduce additional computational overhead compared to simpler pooling methods, potentially limiting scalability in resource-constrainee environments or applications requiring real-time inference. Furthermore, al- though SPiCE facilitates cross-conformer interactions through attention-basee integration, the framework does not explicitly incorporate thermodynamic priors or statistical mechanical principles that govern conformational ensembles.
Future work could address these limitations through several directions: developing more efficient at- tention mechanisms or approximation strategies to reduce computational cost, incorporating physical distribution information, and exploring hybrid approaches that balance computational efficiency with physical grounding.
K
Raw Performance Data
The raw performance data with standard deviation of Table 1 is summarized in Table S5.
25

---

Table S5: Raw performance (mean + standard deviation) in terms of MAE (L) for regression tasks (Drugs-7.5K, Kraken) and ROC scores (f) for two classification tasks (CoV2, CoV2-3CL)
Ensemble
Drugs- 7.5gs-7.5K.. (MAE,
Kraken (MAE, 4)
CoV2
3CL
Backbone
Strategy
IP
EA
X
Bş
BurB,
BurL
ROC (1)
ROC (1)
Mean
0.5410
0.4966400336
0.2963
0.2877a0.0252
0.39504a
0.1817 0.1817
0.147220.0099
0.5722a0.0518
0.885040.1208
PaiNN [37]
DeepSets
0.539620.0544
0.509140.01
0.298220.0052
0.2225
0.361920.112
0. 4.
0.132420.0081
0.580220.0356
0.6808
Attention
0.601820.0327
0.5185
0.348810.1126
0.349620.0140
0.410910.0167
0.2123
0.150620.0002 20,002
0.417920.0357
0.6984
SPiCE
0.5281 20,0666
0.4929#0.0331
0.2792+0.00300
0.217840.0376
0.3548
0. 0.1564+00014
0. 0.129222000081
0.5910+a.0831
0.8880
Mean
.........
0.593510.0672 0.344 2000002
0..121 20:266
0.3986
0.367410.0423
0.2255
0.2327 20.1770
0.3900
0.758010.1180
96
ClofNet [13]
DeepSets
0.591220.0447
0.553320.0292
0.3153+0.0174
0.3314
0.5375
0.253220.0043
0.1983
0.620820.0354
0.7628
Attention
0.669440.2264
0.5949+0.0352
0.3578+0.0090
0.4979+0.0199
0.6118+0.0328
0.3353+0.0109
0.2502+4.0000 +0.099
0.3707 +0.0149
0.8182+001004
SPiCE
0.5747+0.0322
0.5283+n.186
0.3059
0.3193+0.0234
0.4903
0.2477
0.191
0.6730+0.0347
1.0000
Mean
0.5457.0.0349
0.493240.0125
0.2977
0.2303
0.38302a.0291
O. 10.6000000004
0. 0..1559t0.m 20.0011
0.5601 #0.0351
0.838720.099
Equiformer
DeepSets
0.540420.0247
0.4888
0.299010.0016
0.256410.0159
0.377210.000s
0. 1.178220.0120
0. 0.123420.0022
0.5125
0.7134
[4]
Attention
0.5488
0..023
0.2896+0.2247
0.3187 20.077
0.450840.0352
0. 0.167320.005
0.1425.0.0199
0.388220.0877
0.7881
SPiCE
0.531 140.084
0.4830+00453
0.28 040.0332
0.224 #0.0102
0.3456+0.0299
0. 0.1611 +0.0004
0. 0.1229 +0.021
0.5650+0.0457
0.8405
Mean
0.559310.0392
0.4927 20.0451
O. 0.286210.3347
0.281 20.0163
0.3970z0.0461
0. 00.8874,m 10.187440.000 20.0036
Ó. 11.14694..0000 20.0022
0.6035
20.03540.9623
Ó. 0.7447
ViSNet [S8]
DeepSets
0.528040.0449
0.4987+0.055
0.2846+0.0248
0.3104 20.0044
0.4113
0.1716+0.011
0.131 0.131440.0242
0.6626+0.0036
0.416044.0422
Attention
0.5593+0.0455
0.4988+n.035s
0.2944+0.0108
0.3755
0.4195+n.033
0. 0.1394 0.2384+0.0172
0.5262+n.0472
0.7158+0.131
SPiCE
0.5384
0.4538+a.0491
0.28 440.0155
0.271 0.27155.0 5 30
0.3807 +n.0399
O. 0.16 20.010
0.1277 20.0077
0.6890+n.0629
0.7195

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much results can be expected to generalize to other settings.
is fine include aspirational goals motivation long is clear that these goals are not attained by the paper.
Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitation in the Appendix.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets with few runs. In general, empirical results often depend implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to seaech-tt-tess text system might not be used reliably to provide closed captions for online lectures because fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't t acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: We provide proof based on lemmas listed in Section 2.2.
27

---

Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the puper should be numbered and cross-referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof' can either appear in the main paper or the supplemental material, but if they appear the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experi- mental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide the full experiments setting only relying on open-source data and models in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided not.
the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-sourgs models, it may be that access to the model is limited in some way (e.g.. to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer:
[Yes]
Justification:
We provide detailed experiment details in Section 4.1 and Appendix.
Guidelines:
28

---

The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https: g
Salees//ooSuubmissee oooooiiooy for more details.
While we encourage the release code and data, we understand that this might not be possible,
"No" is acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines battps://nipts cc/ cc/publ ic/ httee/CooeSuubaiceee seiisssssssiiiiiiioi cy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide detailed experiment details in Section 4.1 and Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: In Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard enror of the mean.
It is OK to report 1 1-sigma error bars, but one should state it. The authors should preferably report 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of eπrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
29

---

8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the expeni ments?
Answer: [Yes]
Justification: In Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e-g.. preliminary or failed experiments that didn't make into the paper).
Code of ethics
Does the research conducted in the puper conform, in every respect, with the NeurIPS Code of Ethics https: / http///eerripp.eeeee cc/ppubicc/thiiccssuieee........
Question:
Answer: [Yes]
Justification: [NA]
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there a special consideration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss the impacts in the Appendix.
Guidelines:
The answer NA means that there is societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deploy- ment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied par- ticular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point out. For example, an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
legitimate to point that
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
30

---

11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g. pretrained language models, image generators. or scraped datasets)?
Question:
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions access the model implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets some datasets. Their licensing guide can help determine the license of a dataset.
curated licenses for
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's
creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details of the dataset/code/model as part of their sub- missions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
31

---

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human
subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applica- ble), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, non- standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the method development in this research does not involve L.LMs as any important, original, or non-standard components.
Please refer to our LLM policy httppiiwwhggge///rrrrpre////reee should or should be described.
for what
32


Reviews:
[{"review_id": "Tth6nQnrhB", "reviewer": "Reviewer_LtiX", "summary": "The paper introduces a new architecture for conformer-based molecular property prediction. This architecture involves a Mixture of Experts for analyzing the latent features of the molecule. Care is taken to differentiate between the vector and scalar components, and it is shown that the architecture preserves the equivariance/invariance of the underlying experts (GNNs). The equivariant experts are 3D GNNs, and the invariant experts are 2D topological GNNs. Attention is used to combine the information of both types of experts.", "questions": "Is it possible that the size of the datasets used in evaluation impacted the performance of the model? At the largest, 75k molecules/560k conformers is only a moderately sized dataset, and most of the comparisons were done with a model trained on a fraction of those data.\n\nPossible confusion may arise with the similarly named SPICE dataset (https://www.nature.com/articles/s41597-022-01882-6).", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Quality: The submission takes care to explain the technical details of the architecture and all of the mathematical proofs that the architecture is invariant/equivariant in the appropriate places. The work did not really cover weaknesses of the introduced architecture. In particular, the datasets evaluated seem relatively small.\n\nClarity: The writing was generally clear, and the level of detail in section 3 was good. Sections 4 and 5 felt brief in comparison to how dense section 3 was.\n\nSignificance: The work improves the state of the art on some molecular property prediction tasks, and the details are explained well enough that this could be extended in the future. The method of this work is more general than many previous works, and so the mathematical foundation developed in this work can likely be used in later works to extend and improve the results described here.\n\nOriginality: The comparisons to other similar conformer-based and non-conformer-based models are helpful in providing context. It improves on and generalizes the previous works.", "was_revised": true, "final_justification": "As my initial rating was a 5, the discussion with the authors was mainly to confirm my original evaluation. They did address a couple of my concerns. I decided to keep my rating.", "rebuttal": "We sincerely thank you for your positive assessment and recognition of our work’s technical rigor, clarity, generalizability to future research, and improvements to molecular property prediction.\n\n> Q1. Dataset size impact on model performance\n\nThank you for raising this important consideration about data scale. We agree that evaluation on large-scale datasets will strengthen our claims. However, we would like to note that many molecular property prediction tasks are of similar or smaller scales due to experimental costs, making our evaluation practically meaningful.\n\nAdditionally, our evaluation demonstrates consistent improvements across diverse dataset sizes (1.5K to 75K molecules), and we studied the scaling of training data to model performance. As shown in Section 4.3 and Figure 2(a), SPiCE exhibits approximately linear scaling from 7.5K to 75K molecules, reducing MAE from 0.4929 to 0.4386. This suggests that SPiCE maintains stable learning characteristics regardless of data scale.\n\n> Q2. SPICE naming confusion\n\nThank you for flagging this potential confusion with the SPICE dataset from Eastman et al. (2023). We will rename our method to **SPECE** to avoid ambiguity."}, {"review_id": "2n1Hyaq7qq", "reviewer": "Reviewer_CzgV", "summary": "This paper introduces Symmetry-Preserving Conformer Ensemble networks (SPiCE), a new framework designed to enhance molecular representation learning by leveraging ensembles of molecular conformers rather than relying on single static structures. SPiCE designs an architecture integrating three core components: (1) shared conformer encoding with weight-tied 3D GNNs that preserve geometric equivariance, (2) a Geometric Mixture-of-Experts (GMoE) layer that processes scalar and vector features through specialized expert networks with type-aware selective routing, and (3) a hierarchical ensemble encoder that incorporates molecular-level context via cross-conformer attention. The model also provides theoretical properties of aggregations, such as R-equivariant, etc., showing the model is physically meaningful, symmetry-aware representations, and outperforms existing methods across a wide range of molecular property prediction tasks, from quantum mechanics to biology.", "questions": "-", "limitations": "-", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n\ni) Method Novelty\n\nThough the method leverages existing techniques, its overall design is well-designed, adapted to the molecular representation learning. There are also some interesting parts like differentiable top-k with a mixture of experts to selectively fuse 3D conformer to 2D representations.\n\nAuthors also present properties about the permutation invariance of the ensemble and geometric symmetries of individual conformers. \n\nii) Experiments:\nProvided experiments are very details where SPiCe is validated on four different architectures (PaiNN, ClofNetEquiformer, and VisNet), confirming the improved performance and generalized ability of the method.\nThere are also experiments on model scaling on the Drug-75k dataset, at 10% and 100% training rates. A detailed of ablation studies is also provided in Figure 3.\n\nLimitations Weaknesses:\n\n\nAmbiguity in the definitions of scalar and vector features: The paper defines h_{i}^{s} ​\\in R^{1\\times d} as scalar features and h_{i}^{v} \\in  R^{3 \\times d} as vector features (line 181). However, this naming can be confusing, especially when d>1, as it may not align with the conventional understanding of scalars and vectors in the molecular context. The confusion is exacerbated by the early introduction of these terms in Figure 1. The reviewer suggests the authors consider alternative, more intuitive terminology or provide clearer justification and explanation for these definitions. Similarly, the definitions of X^s and X^v (line 151), which are inputs to the message-passing network that generates H (line 180), are not clearly explained and would benefit from further clarification.\n\n\nClarity and readability of Section 3.2.2: This section contains dense notation and is challenging to follow. The reviewer recommends augmenting this part with a relevant subfigure from Figure 1 to visually support the equations and improve reader comprehension.\n\n\nLack of discussion on limitations: The paper does not adequately discuss the limitations of SPiCE. Including such a discussion would help readers better understand the potential constraints of the method and set appropriate expectations for its applicability.\n\n\nUnclear attention mechanism in Figure 1c: It is unclear why only the 3D scalar features are used as queries in the attention mechanism with 2D features. The specific role of vector features within the GMoE module should be clarified to improve the interpretability and transparency of the design choice.\n\n\nRedundant discussion on equivariance and invariance: The method section repeatedly revisits the concepts of equivariance and invariance, which, while important, may distract from the understanding of other architectural components. The reviewer suggests summarizing these theoretical properties in a dedicated subsection or theorem early in the method section, allowing the rest of the space to focus on describing the novel contributions and architectural details more clearly.", "was_revised": true, "final_justification": "After the rebuttal phase, I raised my score as most of my concerns  (similar to other reviewers questions) have been addressed and the explanations are very good.", "rebuttal": "We thank you for your detailed and constructive review. Your recognition of our method’s novelty, comprehensive experimental validation, and interesting technical contributions is very encouraging. We address your specific concerns below:\n\n> Q1. Ambiguity in scalar and vector feature definitions\n\nThank you for raising this clarity concern. Our terminology follows established geometric deep learning conventions [1] where each tensor has three distinct axes: channel ($d$), tensor-type ($l$), and tensor-component ($m$). We recognize this notation will benefit from clearer explanation. For tensor type $l$, there are $(2l+1)$ tensor components:\n* $h^s_i \\in \\mathbb{R}^{1 \\times d}$: scalar (type-0, rotationally invariant) features with 1 tensor component and $d$ channels\n* $h^v_i \\in \\mathbb{R}^{3 \\times d}$: vector (type-1, rotationally equivariant) features with 3 tensor components and $d$ channels\n\nThe tensor-type axis ($l$) is handled by separating $h^s$ and $h^v$, which are then concatenated to form $\\mathbf{X} = (\\mathbf{X}^s, \\mathbf{X}^v) \\in \\mathbb{R}^{n \\times |\\mathcal{V}| \\times 4 \\times d}$, where the 4 comes from concatenating tensor components (1+3=4). As for the definitions of $\\mathbf{X}^s$ and $\\mathbf{X}^v$:\n* For the first interaction block ($l=1$):\n  * $\\mathbf{X}^s$ (Type-0) : Initial molecular features including embeddings of atomic numbers $z$, distance-based radial basis function $\\phi_k(r)$, and topological descriptors $\\mathcal{E}$\n  * $\\mathbf{X}^v$ (Type-1) : Initial geometric features including embeddings of relative position vectors $\\boldsymbol{\\rho}$ and directional bond features\n* For subsequent interaction blocks ($l>1$), they include both the initial features above plus the processed outputs $\\mathbf{X}^s$ and $\\tilde{\\mathbf{H}}^v$ from the previous interaction block.\n\nWe will add clearer explanations of this tensor notation early in the paper and provide concrete examples to aid reader comprehension.\n\n*[1] A Hitchhiker's Guide to Geometric GNNs for 3D Atomic Systems, arXiv 2312.07511*\n\n> Q2. Section 3.2.2 clarity and readability\n\nThank you for your feedback!  We will improve readability by adding relevant subfigures from Figure 1(d) directly into the text, providing step-by-step walkthroughs of Equations (2)-(8) with intuitive explanations, and introducing the routing mechanism in a separate subsection of expert network design.\n\n> Q3. Attention mechanism clarity (Figure 1c)\n\nThank you for raising this question. We use only 3D type-0 features as queries because type-0 features are rotationally invariant. This ensures attention weights remain orientation-independent. Using type-1 features as queries would violate this invariance property since rotating the molecule would change attention scores, leading to inconsistent ensemble representations. In our model, type-1 features are processed separately through dedicated equivariant experts in the GMoE module to preserve their directional properties and geometric information. We will add this justification to Section 3.2.4 and revise Figure 1c with clearer annotations.\n\n> Q4. Redundant equivariance discussion\n\nThank you for this organizational suggestion. We will restructure Section 3 by creating a single theorem early that establishes all symmetry preservation properties, then reference it throughout rather than re-explaining concepts.\n\n> Q5. Missing limitations discussion\n\nThank you for raising this point. Due to space limitations, we included our limitations discussion in Appendix J. The key limitations include: *“While SPiCE achieves strong performance across a variety of molecular property prediction tasks, the hierarchical architecture and dependence on equivariant GNN backbones introduce additional computational overhead compared to simpler pooling methods, potentially limiting scalability in resource-constrained environments or applications requiring real-time inference. Furthermore, although SPiCE facilitates cross-conformer interactions through attention-based integration, the framework does not explicitly incorporate thermodynamic priors or statistical mechanical principles that govern conformational ensembles.”*\n\nWe will move limitations to the main paper in the final version.\n\n---\nThank you again for taking the time to review our manuscript. Please let us know if you have any additional questions."}, {"review_id": "rkoS4Xq8ge", "reviewer": "Reviewer_K5sT", "summary": "The authors address the (often-overlooked) role of 3-D atomic arrangements in molecular machine-learning tasks. They introduce SPiCE, a symmetry-preserving architecture that leverages conformer ensembles instead of collapsing them into a single structure. Across several real-world datasets, SPiCE reportedly outperforms existing approaches.", "questions": "1. Are the conformers $\\{c_i\\}$ supplied by the datasets or generated by the authors? Please clarify the protocol.\n2. Is $R = \\mathrm{SO}(3)$ or $\\mathrm{O}(3)$?\n3. Why is $r^s$ linear in $h$ while $r^v$ is quadratic?\n4. Line 196: Does $R^n$ denote $\\mathrm{SO}(n)$, $\\mathrm{O}(n)$, or something else?\n5. Line 180: $H^s$ is $N \\times d$, yet $r^s$ is $N_I$. Where does the reduction from $N$ to $N_I$ occur?\n6. What specific bottleneck does the expert-routing mechanism alleviate in this context?\n7. Line 224: Why is $\\bar{H}$ claimed to be $S_n$-invariant rather than $S_n$-equivariant?\n8. Table 1: What are the parameter counts for each model? Were they matched across methods?\n9. Fig. 3d: Performance drops for both too few and too many experts; can the authors provide intuition?\n10. How much wall-clock time or GPU hours does SPiCE add relative to its single-conformer backbone?\n\nMinor Comments\n1. Line 185: Should $\\hat{r}_i$ lie in $\\mathbb{R}^{2 \\times (N_I + N_E)}$?\n1. Fig 2: There is a loss spike near epoch 100; any explanation?\n1. Fig 3: Font size is too small; please enlarge axis labels for readability.\n\n---", "limitations": "The limitations are not discussed. It would be nice to include the Compute/accuracy tradeoff mentioned in Strengths.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 1, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths\n1. SPiCE outperforms baselines on reported benchmarks.\n1. The paper's argument, \"preserving conformer information is beneficial for accurate molecular property prediction,\" sounds convincing\n\nWeaknesses\n1. Notation and definitions. Some symbols lack formal mathematical definitions, and dimension mismatches (e.g., between H^s and r^s) create confusion. A concise notation table would help readers.\n1. Domain-specific shortcuts. The manuscript assumes prior knowledge of certain molecular-representation conventions. Brief primers or citations would benefit readers outside the subfield.\n1. Rationale for design choices. Key architectural decisions (e.g., differing functional forms for r^s vs. r^v, adoption of Mixture-of-Experts) need clearer theoretical or empirical justification.\n1. Compute/accuracy trade-off. Modeling multiple conformers inevitably increases computational cost, but the paper does not quantify this overhead or compare run-time vs. baseline methods.", "was_revised": true, "final_justification": "Resolved issues:\n- Notation issues (vector dimension etc)\n- Lack of computational cost analysis\n\nRemaining issues:\n- Validation of computational cost increase vs performance improvement", "rebuttal": "Thank you for your detailed and constructive review. Please see our responses below.\n> Q1. Dimension clarification for $H^s$ vs $r^s$\n\n- $H^s$: Type-0 features with shape $\\mathbb{R}^{n \\times |V| \\times 1 \\times d}$ (features for all nodes across conformers)\n- $r^s$: Scalar routing scores with shape $\\mathbb{R}^{N_I}$ per node ($N_I$ = number of invariant experts)\n\nThe seemingly mismatch occurs because $r^s$ operates per-node, while $H^s$ represents the full feature tensor. For node $i$, we have $h^s_i \\in \\mathbb{R}^{1 \\times d} \\rightarrow r^s_i \\in \\mathbb{R}^{N_I}$, and we collectively denote $H^s = [h^s_i]_{i=1}^{|V|}$.\n\nWe will include a comprehensive notation table in the camera-ready version.\n\n> Q2. Conformer dataset source\n\nThe conformers ${C_i}$ are supplied by the datasets, not generated by the authors. The paper closely follows the established protocols [30,51] for conformer preprocessing. For the specific datasets: Drugs-7.5K uses refined conformers from MARCEL [51], Kraken uses DFT-computed conformer ensembles [52], and CoV2 uses conformers from GEOM-Drugs [53]. We will make this clearer in the revision by explicitly stating the conformer generation protocols for each dataset.\n\n> Q3. R = SO(3) or O(3)?\n\nThe paper uses R to denote the rotation component of the geometric symmetry group G (as in lines 99-100). The choice of R depends on which geometric group is selected and which GNN backbone is used:\n- When G = SE(3) (special Euclidean group), then R = SO(3) (rotations only)\n- When G = E(3) (full Euclidean group), then R = O(3) (rotations + reflections)\n\nOur model supports both options (lines 89-90) and we note that different GNN backbones correspond to different symmetry groups, e.g., ClofNet is SE(3)-equivariant (R=SO(3)). We will clearly state which geometric group and corresponding rotation group are used for each experiment.\n\n> Q4. Why $r^s$ is linear in h while $r^v$ is quadratic?\n\nThis design choice is driven by symmetry preservation requirements. From Eqs. (2-3):\n- $r^s$ uses GCN aggregation: $r^\\text{s}(h_i^s) = softmax(\\sum_{j \\in \\mathcal{N}(i)} h_j^s W^s / \\sqrt{d_id_j} + b)$, which is linear in $h^s$\n- $r^v$ uses inner product: $r^\\text{v}(h_i^v) = softmax((h_i^v)^T h_i^v W^v)$, which is quadratic in $h^v$\n\nThe quadratic form for vector features is necessary to maintain rotational invariance (Lemma 1, part 3). The operation $(h^v_i)^T h^v_i$ performs inner products over the equivariant dimension, ensuring the routing scores remain rotationally invariant while the linear form for scalar features naturally preserves invariance.\n\n> Q5. Meaning of $R^n$\n\nFrom the context in line 196, $R^n$ refers to the product group of $n$ independent rotational transformations, where each conformer can undergo independent geometric transformations. This is clarified in line 91: \"We allow individual transformations on each conformer individually and independently, hence the overall geometric transformation $G^n$.\" So $R^n$ represents $n$ copies of the rotation group $R$ (either SO(3) or O(3)), not the matrix group SO($n$) or O($n$). Each conformer has its own geometric transformation capability.\n\n> Q6. Line 180 dimension reduction: $H^s$ is $N\\times d$, yet $r^s$ is $N_I$\n\n$H^s$ represents node features across all conformers ($N = n\\times |V|$ nodes total), while $r^s_{e,i}$ in Eq. (4) represents routing scores for the $e$-th expert and $i$-th node, with $N_I$ being the number of invariant experts. The dimensions $N$ and $N_I$ have very different meanings, and this switch occurs through the routing computation in Eqs. (2-4), where each node gets assigned scores for each of the $N_I$ invariant experts. We will clarify the dimensional flow in our revision.\n\n> Q7. Expert-routing bottleneck alleviation\n\nThe routing mechanism addressed three limitations of prior uniform ensemble methods: \n1. Uniform processing: Traditional ensemble methods treat all conformers uniformly through simple pooling, but different properties may require different conformational features (Section 3.2.2) For instance, binding affinity depends on pocket conformations while solubility depends on surface accessibility.\n2. Symmetry preservation: The type-separated routing maintains distinct symmetry properties. A unified routing would violate physical laws by mixing R-invariant (scalar) and R-equivariant (vector) features. Separate routers ensure scalar features remain invariant while vector features preserve equivariance under geometric transformations.\n3. Computational efficiency: Sparse activation (top-k selection) enables fine-grained expert specialization without proportional computational cost increases.\n\n> Q8. $\\bar{H}$ $S_n$-invariant vs $S_n$-equivariant\n\nAs in lines 223-224, $\\bar{H}$ is defined as the mean-pooled representation: $\\bar{H} = (1/n)\\sum^n_{c=1} H^s_c$. Mean pooling across conformers breaks the equivariance to conformer permutations and instead produces an invariant representation. If conformers are permuted, the mean remains the same, making $\\bar{H}$ $S_n$-invariant rather than $S_n$-equivariant.\n\n> Q9. Parameter counts and matching\n\nWe ensured fair comparison by matching backbone parameters across all methods. All backbone architectures used identical configurations (hidden dimensions, number of layers, cutoff distances, RBF functions, etc.), ensuring backbone parameters remain constant with only ensemble strategies varying.\n| Backbone | Mean (M) | DeepSets (M) | Attention (M) | SPiCE (M) | Overhead (M) |\n|--|--|--|--|--|--|\n| Equiformer | 0.18 | 0.25 | 0.26 | 0.94 | 0.75 |\n| PaiNN | 0.15 | 0.22 | 0.23 | 1.16 | 1.01 |\n| ViSNet | 1.71 | 1.78 | 1.79 | 3.38 | 1.66 |\n| ClofNet | 0.52 | 0.58 | 0.60 | 2.29 | 1.77 |\n\nNote: SPiCE values represent activated parameters (8 experts, 2 activated in GMoE)\n\nDespite parameter increase, SPiCE demonstrates strong parameter efficiency through sparse activation where only a small number of experts are active during inference, significantly reducing computational cost. The consistent performance gains across all backbones and datasets justify the overhead.\n\nWe acknowledge in Appendix J that “the hierarchical architecture and dependence on equivariant GNN backbones introduce additional computational overhead compared to simpler pooling methods, potentially limiting scalability in resource-constrained environments or applications requiring real-time inference”. Note that several established techniques can address these scalability concerns in future work. For example, linear attention models like Performer [1] can reduce complexity from $O(n^2)$ to $O(n)$.\n\nAdditionally, expert pruning [2] could further reduce computational requirements while maintaining strong performance.\nWe will include this detailed analysis in the camera-ready version.\n\n*[1] Rethinking Attention with Performers, ICLR 2021*\n\n*[2] Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models, ACL 2023*\n\n> Q10. Performance drops for both too few and too many experts\n\nThe U-shaped curve reflects the tradeoff between expert specialization and training stability.\n- Too few experts (2-4): Insufficient specialization capacity for diverse molecular patterns. Conformer ensembles contain varied geometric features (linear chains, cycles, branches, functional groups) requiring distinct computational pathways that cannot be captured with too few experts.\n- Optimal range (8-16): Provides adequate specialization while ensuring sufficient training examples per expert.\n- Too many experts (32+): Over-specialization leads to overfitting as each expert sees too few examples. The routing mechanism also becomes unstable with excessive choices, causing inconsistent expert selection.\n\n> Q11. Computational cost comparison\n\nWe provide time comparison of our model with baselines as follows.\n\n*Training Time (5000 epochs using PaiNN backbone)*\n| Method | Hardware | GPU-Hours | Overhead |\n|-|-|-|-|\n| Mean | A100×2 | 10.4h | 1.0× |\n| DeepSets | A100×2 | 10.4h | 1.0× |\n| Attention | A100×2 | 10.6h | 1.02× |\n| SPiCE | A100×4 | 34.9h | 3.4× |\n\nSPiCE requires 3.4× more GPU-hours for training (configuration: 8 experts, 2 active), primarily due to the mixture-of-experts forward passes, cross-conformer attention mechanisms, and 2D topology processing. However, inference costs remain highly similar to baseline methods, showing only minimal overhead (0.03s vs 0.02s per prediction on A100). We also note that training time can be shortened when using early stopping strategies. Despite the increased training cost, the model delivers consistent performance improvements across all tasks.\n\n> Q12. $\\tilde{r}_i$ dimensionality\n\nThe dimensionality $\\hat{r}_i \\in \\mathbb{R}^{N_I + N_E}$ is correct as written, since $\\hat{r}_i = (\\hat{r}_i^\\text{s}, \\hat{r}_i^\\text{v})$ denotes the concatenation of routing scores for both scalar experts ($N_I$) and vector experts ($N_E$), rather than stacking. Thanks to your inquiry, we will clearly state the concatenation operation in our updated manuscript. \n\n> Q13. Loss spike near epoch 100\n\nThank you for noting this. The loss spike corresponds to the expert upcycling transition phase (Section 3.2.2). During upcycling, additional experts are activated and routing weights are reinitialized, causing temporary optimization instability. This is expected behavior that typically resolves within 20-30 epochs as the routing mechanism adapts to the expanded expert capacity. We will clarify this in the main text.\n\n> Q14. Domain-specific shortcuts \n\nThank you for your valuable suggestion. We will add more comprehensive background information in the appendix, including Group Theory, Equivariance and Tensor Products, to benefit readers outside the field.\n\n> Q15. Font size\n\nWe will enlarge the labels and text in Figure 3 for improved readability in the revision.\n\n---\nThank you again for taking the time to review our paper. Please let us know if you have any further questions or suggestions."}, {"review_id": "VOr7SNoFXY", "reviewer": "Reviewer_rRzB", "summary": "The paper presents Symmetry-Preserving Conformer Ensemble networks (SPiCE), a model achieving equivariance to permutations of conformer ensemble and geometric transformations of the molecules. The model is tested using different GNN backbones and various property prediction problems. The method's main contribution is a hiearchical GNN that uses Geometric Mixture of Experts (GMoE) to create an ensemble of weight-shared equivariant 3D GNNs that model intra-conformer interactions. GMoE is an extension of Mixture of Experts, using two sets of experts to process invariant and equivariant features separately.", "questions": "1) In Table 1, what are the units of the different quantities? \n2) Can you explain the increase of the training loss around 100 epochs in Fig. 2b?\n3) How does the computational cost of SPiCE compare to the baseline GNN models? (For training and evaluation?)\n4) The Experimental setting states: \"Training terminates if the loss shows no improvement for 400 consecutive epochs.\" - Is 400 epochs without improvement really necessary to achieve good models? Could there be a more computationally efficient strategy?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "I thank the authors for the well-written paper.\n\nThe SPiCE framework is implemented so that it can handle different GNNs and the authors tested it for various models. They also compare the method to other ensemble methods. (The code seems to be built using that of Ref. [51] as mentioned by the current paper.) Table 1 shows that SPiCE often achieves the best performance with one of the backbone models; however, other ensemble-based methods often achieve performances very close in absolute values (<0.03 differences for the MAE), which correspond to 2.8-8 % improvements as discussed by the paper.\n\nThe studied properties of Table 1 are only described in the Appendix, many of them are only referred by abbreviation in the main text (e.g. EA, IP). It would help readability to write these out in the paper. Moreover, units of the quantities are missing.\nAlso, there seems to be a mistake in Table 1: it is stated that the bold value is the best model but for B_5, another value is bold, the underlined one seems to be the best.\n\nNote that from the name choice, readers might associate it with the dataset of the same name (see: Eastman, Peter, et al. \"Spice, a dataset of drug-like molecules and peptides for training machine learning potentials.\" Scientific Data 10.1 (2023): 11.).", "was_revised": true, "final_justification": "The authors answered my questions. I keep my original score which was 5.", "rebuttal": "We sincerely thank you for your detailed and constructive review. We appreciate your recognition of our well-written paper, our work’s ability to handle different GNN backbones, and our comprehensive evaluation.\n\n> Q1. Units of different quantities in Table 1\n\nThank you for raising this important clarification. The units for each property are:\n- Drugs-7.5K: IP, EA, χ in eV (electron volts)\n- Kraken: B5, L, BurB5, BurL in Å (Angstroms)\n- CoV2, CoV2-3CL: ROC scores are dimensionless (0-1 scale)\n\nWe will add these units clearly to Table 1 and define all property abbreviations in the main text. We also note the formatting error you identified in Table 1 and Page 2 and will correct these inconsistencies.\n\n> Q2. Training loss increase around 100 epochs in Fig. 2b\n\nThe loss spike corresponds to the expert upcycling transition phase (Section 3.2.2). During upcycling, additional experts are activated and routing weights are reinitialized, causing temporary optimization instability. This is expected behavior that typically resolves within 20-30 epochs as the routing mechanism adapts to the expanded expert capacity. We will clarify this in the main text.\n\n> Q3. Computational cost comparison\n\nWe provide time comparison of our model with baselines as follows.\n\n*Training Time (5000 epochs using PaiNN backbone)*\n| Method | Hardware | GPU-Hours | Overhead |\n|-----------|----------|-----------|----------|\n| Mean | A100×2 | 10.4h | 1.0× |\n| DeepSets | A100×2 | 10.4h | 1.0× |\n| Attention | A100×2 | 10.6h | 1.02× |\n| SPiCE | A100×4 | 34.9h | 3.4× |\n\nSPiCE requires 3.4× more GPU-hours for training (configuration: 8 experts, 2 active), primarily due to the mixture-of-experts forward passes, cross-conformer attention mechanisms, and 2D topology processing. However, inference costs remain highly similar to baseline methods, showing only minimal overhead (0.03s vs 0.02s per prediction on A100). We also note that training time can be shortened when using early stopping strategies. Despite the increased training cost, the model delivers consistent performance improvements across all tasks.\n\n> Q4. Early stopping strategy\n\nOur conservative 400-epoch patience was chosen to ensure complete convergence across all configurations. Preliminary analysis suggests 200-250 epochs would suffice for most cases, potentially reducing computational cost by ~40%. We will explore adaptive early stopping strategies based on dataset size and model complexity in future work.\n\n> Q5. SPICE naming confusion\n\nThank you for flagging this potential confusion with the SPICE dataset from Eastman et al. (2023). We will rename our method to SPECE to avoid ambiguity."}]

Author rebuttal:

Meta-review: The submission proposes augmenting the model input with 3D conformation ensemble samples of the input molecule for predicting molecule-level (i.e., not depending on a specific conformation of the molecule but determined by the thermodynamic distribution of the conformations) properties. The authors designed some model components (e.g., geometric mixture-of-experts, invariant and equivariant experts, inter-conformation interaction) that seem reasonable, and geometric and permutation symmetries are tracked along with the model development. Empirical results demonstrate better performance than models that do not use conformation or use a single conformation.

The authors and I appreciate the idea of leveraging detailed conformation ensemble information for predicting molecule-level properties, and the design seems reasonable and effective. Therefore, I'd recommend an acceptance considering the (one of, if not exactly) the first work that embarks this route. Nevertheless, I hope the authors could improve the paper according to reviewers' feedback, particularly the acronym, notation, presentation, and performance-cost trade-off comparison results. Moreover, I also sensed a few points that I suppose are worth discussing:
* Should the learned ensemble/molecule representation and prediction model be affected by the distribution of the provided samples? I suppose it should be (different distributions, e.g., under different temperatures, should affect ensemble/molecule-level properties), but the authors are expected to discuss how to ensure this. Particularly, in aggregating conformer-level representations, it seems that the conformers are equally weighted, but the samples are usually not following the Boltzmann/thermodynamic equilibrium distribution (e.g. in the GEOM case).
* The authors are expected to expand the definitions of the predicted properties and explain why they are ensemble/molecule-level properties.
Decision: Accept (poster)