Probing Equivariance and Symmetry Breaking in Convolutional Networks
Authors: Sharvaree Vadgama
Mohammad Mohaiminul Islam
Domas Buracas
Christian A Shewmake
Artem Moskalev
Erik J Bekkers
Venue: NeurIPS 2025 poster ()
Keywords: convolution networks
geometric deep learning
equivariance
symmetry-breaking
generative modeling
Primary area: deep_learning
TL;DR: A study on equivariant and non-equivariant convolution networks for point clouds

Abstract: In this work, we explore the trade-offs of explicit structural priors, particularly group-equivariance. We address this through theoretical analysis and a comprehensive empirical study focusing on point clouds. To enable controlled and fair comparisons, we introduce \texttt{Rapidash}, a unified group convolutional architecture that allows for different variants of equivariant and non-equivariant models. Our results suggest that more constrained equivariant models outperform less constrained alternatives when aligned with the geometry of the task, and increasing representation capacity does not fully eliminate performance gaps. We see improved performance of models with equivariance and symmetry-breaking through tasks like segmentation, regression, and generation across diverse datasets. Explicit \textit{symmetry breaking} via geometric reference frames consistently improves performance, while \textit{breaking equivariance} through geometric input features can be helpful when aligned with task geometry. Our results provide task-specific performance trends that offer a more nuanced way for model selection.
Code available at [github.com/Sharvaree/EquivarianceStudy](https://github.com/Sharvaree/EquivarianceStudy)

Probing Equivariance and Symmetry Breaking in Convolutional Networks
Sharvaree Vadgama 122,+
Mohammad Mohaiminul Islam
Domas Buracast
Christian Shewmake
Artem Moskaley
Erik J. Bekkers
LML University of Amsterdam
New Theory AI
QurAI, University of Amsterdam
UC Berkeley Indeppeddent Researcher
Abstract
In this work, we explore the trade-offs of explicit struetural priors, particularly -quivarriiii We address this through theoretical analysis and a compre- hensive empirical study focusing on point clouds. To enable controlled and fair comparisons, we introduce Rapidash, unified group convolutional architecture that allows for different variants of equivariant and non-equivariant models. Our results suggest that more constrained equivariant models outperform less con- strained alternatives when aligned with the geometry of the task, and increasing representation capacity does not fully eliminate performance gaps. We see im- proved performance of models with equivariance and symmetry-breaking through tasks like segmentation, regression, and generation across diverse datasets. Ex- plicit symmetry breaking via geometric reference frames consistently improves performance, while breaking equivariance through geometric input features can helpful when aligned with task geometry. Our results provide task-specific performance trends that offer a more nuanced way for model selection. Code available gi thub. com/ Shaarrareee/uuuvvaaaannnaaa
Introduction
There is an ongoing debate about the necessity of explicit structural priors, particularly group- equivariance. A growing line of work argues that strict equivariance may over-constrain model and limit its scalability, and that increasing model capacity and training data [Qu and Krishnapriyan, 2024] can compensate for the lack of built-in symmetry. This perspective is supported by models like AlphaFold [Abramson et al., 2024], which train equivariance by data augmentation. several recent works highlight the limitations of learning equivariance from data alone. Moskalev et al. [2023] show that learned symmetries can unreliable and rapidly degrade out out-of-dii -f-diiiiiit primary contributors layer-
contrast,
Gruver et al. [2024] demonstrates that aliasing and nonlinearities wise equivariance loss, and that data and training scale can have a greater influence than architectural design in learning equivariances.Peree Petrache and Trivedi [2023] demonstrate that even partial symmetry alignment can improve generalization. Theoretical results by Perin and Deny [2024] further highlight that learning equivariance fundamentally limited when class orbits sparse poorly separated.
Given these competing perspectives, it remains unclear under what conditions equivariance leads tangible benefits, and when unconstrained models may be preferable, highlighting the need for deeper understanding of this trade-off.
To address this debate, systematically investigate the impact of equivariant and nouiequivariant models across a range learning tasks involving geometry. We begin by formalizing five core research questions: (i) how kernel constraints affect expressivity and generalization, (ü) whether
"Corresponding author: <sharvaree. vadgama@gmail.com com.
i equal contribution
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

at test set quality within hours
Molecule Stability (5)
Equdencinnc reatfin gencratattion
86-
SUS(5) (Iniated)
SA((3)
Broaking Broakingcauhhriinnn caulvrriaoe trenjet my mymetry suppent ssymmette Aho0
Ta Onffotedi
73
76
data
203 wws.orroaaoeeeeeee
Training time (hours)
Figure 2: Comparison of sapidash model variants with different qquivariance-brakkiin on Shapenet part segmentation and genera- tion. (X%) indicates what portion of the training dataset the model was trained on. We also evaluate on aligned and rotated ver- sions of the test set.
Figure 1: We plot molecular stability and train- ing time for SF(3) and T3 equivariant models with a hidden dimension of 256 / 512(inflated) for the molecular generation task on QM9, In- flated models have representations with higher hidden dimension, hence increased capacity.
increased representation capacity can close the performance gap between more and less constrained models, (iii) how dataset size influences performance, (iv) when pose-informed symmetry breaking improves equivariant models, and (v) how different forms of equivariance breaking affect perfor- mance.
We approach these questions through both theoretical and empirical analysis. Theoretically, we show that equivariant models decompose into and provide formal results that link generalization to pose entropy and inductive bias alignment. To conduct systematic empirical analysis, introduce Rapidash, unified group convolutional architecture that enables fine-ge thee-gained control over equivariance constraints, input/output represen- tations, and symmetry- and hitivarranneebreaking mechanisms. This design supports systematic comparisons across a wide range of model variants with varying degrees of equivariance. Experi- ments on molecular property prediction and generation. 3D part segmentation and shape generation, and motion prediction show that more constrained equivariant models outperform less constrained alternatives when aligned with task geometry. Increasing representation capacity helps both strongly and weakly constrained models, but does not fully climinate performance gaps (Fig. 1). Models with stronger equivariance are more data-efficient. Explicit symmetry breaking via geometric reference frames consistently improves performance, while breaking equivariance by introducing geometric input features (e.g., coordinates as scalars) can be helpful when aligned with task geometry (Fig. 2). These results provide concrete answers to the research questions and clarify the conditions under which different degrees of equivariance are most effective. Rapidash with gquivariance-brekkie and symmetry-breakee achieves state-of-the-art performance on QM9 generation, property prediction, and CMU motion prediction tasks.
and symmetry-breaking components,
The contributions of this paper can be summarized as:
Theoretical analysis: We provide formal results showing when equivariance and symmetry breaking offer provable advantages in expressivity and generalization.
We systematically study the impact of various equivariant and uncon- strained models across multiple learning tasks on point clouds, covering molecular property prediction and generation, 3D part segmentation and shape generation, and motion predic- tion.
Empirical study:
Unified architecture: We introduce Rapidash, a scalable and modular group convolutional architecture. This design supports multiple symmetry groups, input/output variants, allowing for gguivariance-breakiin and gemmetry--rraakien options.
Rapidash achieves state-of-the-art performance on QM9 molecule generation and CMU motion prediction, relative to recent methods from literature.
The remainder of the paper is organized as follows. Section reviews relevant background. Sec. introduces theory, the research questions and Rapidash. Sec. 4 discusses related work. Experimental

---

results are presented in Sec. 5. Finally, Sec. 6 addresses each research question. summarizes key takeaways on when to prefer equivariant or unconstrained models.
2 Background
Our study of equivariant deep learning on 3D point clouds investigates the impact of varying linear layer within convolutional architectures (which alternate linear layers with non-linear operations, e.g. activations), while holding other architectural components fixed to isolate the influence of cquivariance. We consider a 3D point cloud X' = {xI,.... XN} } C R of N points with associated feature fields f X' + RC Assuming neighborhood sets (ii) define connectivity for each point x, (indexed by V {I,..... N.,, the general form of a linear layer for such feature fields is given by
[Lf((x.)
Σ xxxx,x,)f(x;) JEN())
(I)
where the kernel k(x,,,) CCCC maps point features to C' output channels.
In works such as [Cohen et al., 2019, Bekkers, 2019], constraining linear layers (1) to be equivariant is shown to yield group convolutions. For instance, translation equivariance restricts the keree to relative positions, k(x, - x,). To achieve SE(3) equivariance on 3D data, the main approaches are regular and steerable group convolutions [Weiler et al., 2021, Brandstetter et al., 2022]. Steerable methods (a.k.a. tensor field networks [Thomas et al., 2018J) embed features in group representations (definition in App. A), which restricts non-linear operations to specialized equivariant ones to preserve these properties [Weiler and Cesa, 2019]. Regular group convolutions, conversely, lift features to an extended domain (e.g., R x S3) to explicitly track group information, thereby permitting standard pointwise non-lincarities [Cohen and Welling, 2016]. As our study isolates linear layer effects by fixing conventional non-linear components, we focus exclusively on regular group convolutions.
Within this regular group convolution paradigm, operations can be defined over the full group, creating feature fields on R3 SO(3) with convolutions of the form [x,] R) = Jf k(R (x' x), R') RR) d dR' For improved efficiency, these be formulated
homogeneous spaces (definition in App. A) of the group. Our work adopts this strategy, utilizing feature fields over the resition-orientatiinn space R x S Convolutions are then given by [Lf](x, n) = Jf k(R,, 7x' x), Rn n') ff fx', n' )d''dd',, dn', with R., aligning canonical axis to E S and reduce the integration domain from SO(3) to 52 at the cost of moderate kernel symmetry constraints (e.g., axial symmetry [Bekkers et al., 2024]). A further simplification for SE(3) equivariance involves convolutions directly on R using isotropic kernels dependent only on distances, &[[|x; - x:l).. This approach eliminates the orientation axis entirely, but imposes more restrictive, non-directional kernll constraint.
3 Methodology
3.1 Expressivity and Equivariance Constraint Analysis
Our analysis of expressivity and equivariance constraints centers our model, Rapidash (detailed in App. B). Rapidash employs regular group convolutions by processing feature fields f RR x S + RC over the cosition-orientation space. This involves convolution kernels on R x S that respect the quotient space symmetries (R3 x 52 = SE(3)/S((2), a condition guaranteed to be met by conditioning message passing layers the geometric invariants derived in [Bekkers et al., 2024].
key theoretical aspect of Rapidash's 's architecture its fundamental connection to steerable tensor field networks. This relationship is established through Fourier analysis on the sphere 52, where scalar fields can be decomposed into spherical harmonic coefficients that correspond to features transforming under irreducible representations of (O(3) building blocks of steerable networks. This equivalence is formally stated as:
Proposition 3.1 (Equivalence via Fourier Transform). A regular group convolution operating on scalar fields f : R2 x Y + CRC (where Y is 5 or SO(n)) can be equivalently implemented as steerable group convolution operating fields of Fourier coefficients l: R - V, (where V, is the space of combined irreducible representations) by performing point-wise Fourier transforms F before the steerable convolution and inverse Fourier transforms Fyy after.
3

---

Remark 1. This equivalence (discussed furthrr by Bekkers et al. [2024, Appx. A.II, Brandstetter et al. [2022], and Cesa et al. [2021]] implies that Rapedash could, in principle, be reformulated as a tensor field network. Thus, our architecture inherently represents the capabilities of this widely used class of steerable networks.
This connection steerable networks also underpins its crucial universal approximation property. Building on established results for steerable GNNs [Dym and Maron, 2020] and related equivariant message passing schemes [Villar et al., 2021, Gasteiger et al., 2021], the SE(3) -equivaaqia universal approximation for Rapidash is formally stated as:
Proposition 3.2 (Universal Approximation for Rapidash). Rapi idsh, as an instance of message passing networks over R S² with message functions conditioned on the bijective invariant attributes (derived in [Bekkers et al., 2024, Thm. 11). is an -qqiivaria universal approximator This specific universality follows from [Bekkers et al., 2024, Cor. 1.1]. leveraging the sufficient expressivity of feature maps over R x S
Regardless of Rapidash' strong theoretical expressivity, practical performance must weigh computa- tional cost against actual expressivity. G-convs on extended domains like IR3 S involve feature fields of size, e.g. P x O x C per point cloud (Points x Orientations x Channels). compared to P x C for R -assed models. While matching total features (e.g., O x C in an R x S model to an enhanced C" in an IR model) might suggest a nominally similar capacity, the O-axis in the former signifies structured domain where features are correlated, not merely independent channels. The true expressive power, related to model's hypothesis space size [Elesedy and Zaidi, 2021], varies with the imposed equivariance constraints: -qquivauirr IR convolutions are least constrained, followed by -equivarvv R x 5 convolutions, and isotropic SE(3) SE(3)-equvvaris equivariant R33 convolutions being most constrained. For fair architectural comparison, we thus must consider scenarios where channel capacity (C) is maximized within practical limits. This leads to key research questions:
Research Question 1: Kernel Constraint and Expressivity Impact How do different equivariance strategies-orderee from most to least constrained as previously outlined-ipaac task performance.
Research Question 2: Capacity Scaling and Generalization Gaps If more constrained SE(3)- equivariant models outperform ((3)-models (despite T(3)'s 's larger theoretical hypothesis space), can scaling -models (e.g., via channel capucity C or training duration) close this apparent generalization gap? How do these architectures compare under nominally matched capacities?
Research Question 3: Data Scaling and Symmetry Relevance in Varied Contexts How does dataset size scaling impact the performance relativities of SE(3) versus T T (3)-constraiied models on SE(3) syymmmteyet tasks? In other contexts, such as tasks without SE(3) symmetry or low-data regimes, what is the practical generalization impact (benefit or hindrance) of imposing SE(3) constraints versus T(3) constraints?
3.2 Symmetry Breaking, and Generalization
This section analyzes generalization performance through the lens of probabilistic symmetry breaking, as introduced by Lawrence et al. [2025b] in their SymPE framework. Their work shows that incorporating informative auxiliary random variables Z (such as pose information Z E SO(3)) into stochastic model f(X, Z)-that e-g. takes a featurized point cloud X as input- can positively impact predictive inference, particularly when using jointly equivariant architectures. The ability of model to leverage such an auxiliary variable is reflected in the effective conditional distribution P(Z]X) that the model implicitly defines. We adopt this perspective to interpret how model's architectural constraints, like those in our Rapidash architecture, affect its ability to use global orientation information, satisfying joint invariance f(gX,
f(x, Z) for g SO(3).
We analyze model expressivity by considering two scenarios for the conditional distribution X) of the auxiliary pose Z E SO(()). First, if the pose Ranowa is explicitly available (e.g., for aligned datasets like ShapeNet), P(Z = 8(2 RAnowa). This provides zero-entropy pose information, enabling a model fxX RAsown) to directly leverage this specific orientation. Second, standard 50(3)- invariant models Jim((X), which do not receive explicit pose input, operate as if Z is drawn from a maximum entropy (uniform) distribution. This implies no specific orientation information is utilized, as derived Proposition D.I. The disparity in available pose information and thus in the effective entropy of Z betweeeen these scenarios directly dictates model expressivity:

---

Corollary 3.1 (Expressivity Gain from Low-Entropy Pose Information). Let the optimal invariant mapping for task, (X,, R), depend non-trivially on canonical orientation Based on the distinct informational content of 7. outlined above: (a) Standard invariant models Jinn (maximum entropy pose) lack the expressivity to represent f"; and (b) Pose-conditioned models Scond (X, R) (zero-entropy pose) can represent f"
The enhanced expressivity afforded by conditioning on pose R, as established in Corollary 3.1, fundamental for enabling symmetry breaking. By providing determinate canonical reference R. models can disambiguate features arising from exact or approximate input symmetries, thereby producing more specific and potentially less symmetric outputs than standard equivariant models. We elaborate this viewpoint in context of ShapeNet segmentation in App. D.3 and generative modeling in App. D.4. Crucially, the well-known generalization advantages of equivariant architectures [Flesedy and Zaidi, 2021] also extend to models fx ; Z) that incorporate breaking, as formalized by Lawrence et al. [2025b, Thm. 6. This general theorem confirms that structuring models f(X, 2) be jointly equivariant is critical for realizing provable generalization gains when an auxiliary variable Z is introduced (e.g., for symmetry breaking). Our Proposition D.2 details the application of this principle to setting where Z is a known, determinate pose R.
auxiliary variable Z for symmetry
Research Question 4: Empirical Benefits of Pose-Informed Symmetry Breaking Does incor- porating explicit geometric information, such as pose R, to facilitate symmetry breaking lead to improved empirical performance compared models that do not utilize such information?
3.3 Equivariance breaking
Our architecture, Rapidash, is de- signed with the flexibility to process both scalar and vector-valued features at its input and output stages. For in- stance, it can map input vectors of (e.g., initial velocities or normals) to spherical signals and predict output vec- tors ufte (e.g., displacements) by ap- propriately projecting from spherical representations (see App. B for archi- tectural specifics). This capability al- lows Rapidash naturally incorporate informative geometric inputs such as global pose, normal vectors, veloci- ties. If these inputs are treated as proper geometric objects that transform sistently under SE(3) actions, their in- clusion enhance the model's contextual understanding while preserving its overall SE(3)- equivariance.
Lifting to S?
Model
0.5
Namrub
SOUTI
Rapidash Model
Wettor Imputs
T121
Coondiaates
Neroub
Velocitoo
Syeeetry
Figure 3: Input variations to Rapidash with base space R x S and -euuivaaqua convolutions lifted S shows equivariance breaking through different inputs and symmetry breaking by using a global frame input.
This same flexibility in handling inputs also allows for controlled deviations from strict SE(3)- cquivariance within Bapidash. For example, global coordinates or other geometric data can be supplied as fixed scalar features, which do not transform canonically under SE(3) actions as true geometric vectors would (Fig.3). This presents critical trade-off: while more input data can be powerful, what is the impact if it compromises the SE((3 -equivarriee prior? As discussed in our related work (Section 4), the approximate and relaxed group equivariance approach suggests that such controlled deviations from strict symmetry can be beneficial, potentially improving training dynamics and performance. We explore this empirically with Rapidash with the mechanisms for deviating from strict equivariance detailed in Appendix C. This leads the following research question:
Research Question S: seadiaarannce-reakknn What is the impact of breaking equivariance? Does providing more information to the model at the cost of breaking equivariance improve performance?
5

---

4 Related Work
Weight sharing in convolutional networks was introduced in [LeCun et al., 2010] for images, and extended to group transformation in [Cohen and Welling, 2016] (recently formalized for MPNNs [Bekkers et al., 2024J). In the spirit of convoouution is all you need' [Cohen et al., 2019], several works emerged like [Ravanbakhsh et al., 2017, Worrall et al., 2017, Kondor and Trivedi, 2018, Bekkers et al., 2018, Weiler et al., 2018, Cohen et al., 2019, Weiler and Cesa, 2019, Bekkers, 2019, Sosnovik et al., 2019, Finzi et al., 2020] presenting different frameworks for group convolutional architectures.
Wang et al. [2022], van der Ouderaa et al. [2022], Kim et al. [2023]. Pertigkiozoglou et al. [2024] demonstrate that controlled equivariance breaking can significantly improve performance, also highlighting the benefits of relaxed group equivariance constraints. Similarly, work by Petrache and Trivedi [2023] suggests that while aligning data symmetry with architectural symmetry (strict equivariance) is ideal, models with partial or approximate equivariance can still exhibit improved generalization compared to fully non-equee -quivaarvt ones. More recent works show different ways to break symmetry, spontaneous symmetry breaking in [Xie and Smidt, 2024] and symmetry-breaking via random canonicalization in [Lawrence et al., 2025b]. For additional details on internal vs external symmetry and the relation of existing works, see App. C.
The literature presents conflicting evidence on the practical benefits of strict equivariance versus less constrained models. For example, while equivariant models with higher-order tensor representations (e.g.. E3GNN, NequlP [Batzner et al., 2022], SEGNN [Brandstetter et al., 2022]) have shown improved data efficiency and performance for tasks like molecular interatomic potentials, findings in other domains differ. Thais and Murnane [2023] reported that Lorentz equivariant models [Brehmer et atal,, 2023] offered no clear advantage over non-equivariant counterparts in particle physics. Similarly, in materials science, though (0(3)-equivaria) et al., 2024] achieved strong results, Qu and Krishnapriyan [2024] (EScAIP) demonstrated that extensively scaled non-equivariant models can be competitive or even superior. Conversely, scaling experiments by Brehmer et al. [2024] using transformers for rigid-body simulations argued in favor of equivariant networks. Such divergent outcomes highlight the challenge of drawing universal conclusions due to common variations in model architectures and datasets, a difficulty our work aims to address by introducing Rapidash unified framework for controlled comparative studies.
Graph Transformers [Liao and Smidt, 2023, Liao
5 Experiments and Results
In each table, the top section shows models with SE(3) equivariance, and the bottom with T3 equivariance. Equivariance breaking due to the input specification indicated by: ! (breaking SE(3), ! (break T3 or SO(3)), while denotes that the model is no longer equivariant. All models for equivariant tasks are trained with SO(3) augmentation. A /indicates an option is used, x, indicates it is not used, and "_" indicates the option is not available. Lastly, we add references from literature with statc-of-the-an art (SotA) performance. We emphasize that our experiments aim to demonstrate how group convolutional architecture and its variants perform to answer the proposed Research Questions, instead of having results that surpass SotA models, but we nevertheless find it important
dalio OL Instance
5.5
3.9
0.50
2031 coeivatonee I 28 souieatenss in I
0.0
8K31 17th
$2318225
0.20
07/40 190%
975
0.10
6:5
1334 10.35
731 65%
630
Skp
8:00
Figure 4: Smoothed IoU (Instance average) perfor- mance curves over time for ShapeNet part segmen- tation. The legend codes correspond to the effective equivariance and the percentage of the training dataset trained on.
Training time (hours)
Figure 5: MAE (10-3) vs training time for QM9 property (μi) prediction task.

---

to provide references from literature as they indicate our results are close to, or surpass, the state of the art, which validates the correctness of our implementation and thus the validity of our results. We list the most notable results in the tables of the main text while presenting full extended versions in the Appendix.
Molecular property prediction & generation task For predicting molecular properties and generating molccules, we use QM9 [Ramakrishnan R., 2014]. We evaluate the prediction of molec- ular properties using the MAE and compare these with EGNN [Satorras et al., 2021], J. Dimenet++ [Gasteiger et al., 2022], and SE(3)- Transformer [Fuchs et al., 2020]. For the molecule generation, we train a generative model that uses equivariant denoising layers We evaluate molecular generation on metrics from Hoogeboom et al. [2022] that include atomic stability, molecule stability, well as a new aggregate metric discovery, which is a fraction of generated samples that are jointly valid, unique, and new. We compare our models with denoising diffusion [Ho et al., 2020] model like EDM [Hoogeboom et al., 2022], PONITA [Bekkers et al., 2024]. and MuDiff [Hua et al., 2024].
Table I: Ablation results on QM9 for property prediction and generation. For prediction, we report mean absolute error (MAE) (x 10-3). For generation, we evaluate atom stability, molecule stability, and discovery.
Bapddaac internal SE,3 Equivariance Constraine
Regresion
Generation
Variation
Type
Cocrdinones Scalars
Courdinates Vectors
Edfective Equivzziance
MAE P (D)
(62)
MAB CHazz
Stah Atom
Stsh Mot
Discover
Normulizzdd Epoch
SE(3)
1741.am
22.29
none
52.22a0
22.4
9674.
72.39.1
8906.. 17
3
10:00
2 5
SO(3)
10.
10.4410:22
42.0811.84
19.6410.75
99.38
92 92.11
90.2620029
10
32 S
X
none
10.03amu7
40.21,005
18.69
99.33300s
92 72.7
91.54,0.1
Rapidazh with Insemall T2 Equivariance Constraine
22 none
22 22.11
61.974:.5
98.57.0 25.80
91.83 81662
RP
20.4
58.18
24.54.0.25
98. 20
78 78-4816m1
80 80-26,0039
Refermee mathods from literatune
EGNN DimeNeté4 8E(33-T MUDDIT
29.3
71.0
29.0
98.7
$2.0
20.7
43.5
24.6
$3.0
$1.0
$3.0
98.8
$0.9
BONN-FDM
09.3
90,7
89,5
PENITA
98.9
87.8
Tab. summarizes results on QM9. For property prediction, SE(3) (equi-vvvvv Rapidash variants consistently outperform their T' -'uuquiiar counterparts, and the performance gap does not close with extended training (Fig.5). 5) Among these SE(3) models, those utilizing the IR3S x S² base type achieve superior accuracy over R -only versions highlighting the benefits of directional input and internal directional representations- and notably reach performance levels on par with established saate-of-the-ard art methods on several regression tasks. We also find that breaking equivariance by adding scalar coordinate features further benefits both base types, likely by helping to resolve symmetric ambiguities. In molecule generation, the best results are decisively achieved by our SE(3)-equivariant Rapidash models, again favoring the R&3 X base type, which demonstrate strong molecular stability in contrast to T3 variants. Remarkably, on this generative task. these models significantly surpass existing state-of-the oofthe--att art performance. Collectively, these outcomes demonstrate that Rapidash's 's core convolutional architecture, while designed for flexibility and controlled experimentation across its variants, is itself highly effective and achieves competitive to leading results in both predictive and generative domains.
3D point cloud segmentation & generation task We evaluate models on ShapeNet3D [Chang et al., 2015] for part segmentation and generation tasks. We additionally compare our models to various SotA methods like PointNeXt [Qian et al., 2022], Deltaconv [Wiersma et al., 2022], and GeomGCNN [Srivastava and Sharma, 2021] for the part segmentation task. For the generation task, compare to LION, [Zeng et al., 2022], PVD Zhou et al. [2021], and DPM Luo and Hu [2021]]
Tab. shows our results on the ShapeNet dataset. For segmentation, especially on out-of-distribution (rotated) inputs, -E33)--quivariant Rapidash variants over IR3 x S consistently perform best, echoing our QM9 findings. Performance is further enhanced by explicit symmetry breaking (e.g.. adding global frame for a stable orientation reference) or by specific types of equivariance breaking,
7

---

Table 2: Ablation results on ShapeNet3D and CMU datasets. For ShapeNet3D, report part segmentation performance using mean Intersection over Union (mloU) on aligned and randomly rotated imputs, and 1 1-Nearest Neighbor Accuracy (1-NNA) for shape generation. For CMU motion prediction, we report mean squared error (MSE) scaled by 10-2.
Sup idasia with interal 3) Eguivriance Consuaie
StapeNet
CMU
Model Veriation
Type
Connirotes as Saaters
Cooriraass
Vectors
Narrabitelooitt
as Scelars
NormakiMoooity on Victes
Symmery Breking
Effeetiee Equierrince
(ilgged)
In
Cokmod)
Tedl
I/NNA
CD
MSB
(noata))
MSE
SE,2
32
1%
63.34.844
4.9
14.2
82
EA00
83.55:a.
12. 32.34
69.19
6.77
10
$4.45u121
13 13.53uu52
120
50/7.7
307
SE13
35.44_442
45.30ua.t
05.15a..1
SE10
85.45
27
K.57
90
2447
64.36uuee
5.0
10
90:31
65.3hasas
62.001
11
24.70..11
84.0722:1
2.21
0.222x.1
28:24
1.00
2 K
84.00
13.3b-
1K 6,00gm
Rapidwak with lntenel 23 Equivalance Coustraist
5.Monne
30.15
85.36..
31,00.119
62.29
1124
77.56
teee
81. 81.28cs
37:97"5391
5 5.0
-100
methodh lteetore ShepeNer 3D)
LION PVD DPM
51.8
PanNe GomeCNNN
87.00
89.10
Releseace tucthed litertue (CMU Motio)
IES BUNN OGENN
05.93
31.70
0.41
5 minedd with IIM parto In of 2015. oon zektiional pro pr-comppred omriicae onastins traning des.
such as treating input coordinates as fixed vector features to encode absolute spatial location, as may be expected for this non-symmetry-critice task. For 3D shape generation, which aims to produce canonically aligned objects and thus neither require full SE(()-equivariancc,, models based on the -Equi--aqvv R3 x S architecture remain highly effective. Notably, IR3 x S models that incorporate mechanisms that break equivariance achieve the top generative results. The pose- informed, symmetry- variants even slightly outperform the translation-equivarte R33 models.
Human motion prediction task We next evaluate the model variations on the CMU Human Motion Capture dataset [Gross and Shi, 2001](200 training samples), where the task is to predict motion trajectory (Fig.1 1). We list NRI [Kipf et al., 2018], EGNN [Satorras et al., 2021], CEGNN [Ruhe et al., 2023] and CSMPN [Liu et al., 2024] models from literature.
Tab. 2 reports results on the CMU motion capture dataset. We observe that SE(3) over IR3 x S with velocity vector inputs achieve the lowest MSE. Note that such models fully equivariantly map from input to output (a vector). Breaking symmetry by means of global frame further improves performance in most cases. In contrast, models operating solely over IR3 perform poorly, particularly under (out of distribution) rotations, highlighting the importance of capability to equivariantly handle directional information. Recall that the SE(3) equivariant IR' model cannot solve this task, as it is a strictly invariant model, and the task requires predicting non-trivially transforming velocity vector.
models
Representation capacity and data efficiency To study the effect of representation capacity, we scale up the number of channels in the R models to try close a potential performance gap to the R x S due to sub-optimal network capacity. As shown in Tab. 3 and App. Tab. 5, this increase has little impact on performance, suggesting that all model variants are already operating near their effective capacity limits. As an additional experiment, we also evaluate data efficiency on ShapeNet (Fig. 4) and find that models with stronger equivariance require fewer training samples to converge, indicating the advantage of equivariance in low-data settings.
8

---

Table 3: Comparing models with inflated representation capacity against regular models ShapeNet3D for the part segmentation task for aligned and randomly rotated samples.
Rapidash with SE(3) Equivariance Constraint
Normalized Epoch
Model Variation
Type
Coordinates Scalars
Coordinates
Normals
Nermals
Symmetry
Edfective
IoU T (aligned)
IoUt (rotated)
Vectoes
Scaluns
Vectors
Breaking
Equivariunce
80 60.26.a
S0.33
SE(3)
80.31 +0.15
20.20
10
83, 13.95
52. 20.7
2
X
T3
63,87
49.63 49.60.0.s7
10
R3
/
x
none
8:1.233e0.u ne
84.01 as.us
32 22.22
15.0.0
10
/
84.75 1o.on
34.07
81.18
Rapidash with Internal T3 Equivariance Constraint
R3
Ts
65.38%+0.00
65.26.2..0
32. 32.78.0.50
31-41g0 ss
10
85.82, 10.30
35.42 10.50
1
17
X
T;
85.71
32.25
10
18
x
Rome
65.5 45.52
85, +00000
31. 31.11g
32. 22.7940.54
10
86. 85.06.10
33.5010..
nome
85.16
30.62
6
Discussion
Regarding the impact of kernel constraints (RQ1), findings indicate that performance is dictated more by the appropriateness and geometric expressivity of the inductive bias than by merely minimizing group constraints, under optimized model capacities. For instance, aligned ShapeNet data (Table 2), where -quuiiaaaveee demands might seem harsh, the less constrained Tg- equivariant @ models perform comparably to, not better than, the richer -E33)_aquivariant R3 S variants. Within SE(3) models, highly constrained isotropic I kernels consistently underperform against the less constrained R x S kernels across tasks. Furthermore, property prediction, Table 1; CMU motion, Table 2), the R x S models significantly surpass the T3 I models. It underscores that well-suited inductive bias is critical.
SE(3)- tasks (QM9
Regarding capacity scaling and generalization gaps (RQ2), our findings indicate that while increased model capacity (e.g., "inflated" models in Fig. 1, Tab. and App. Tab. 5) benefits both Ty and SE(3) approaches, it does not consistently allow simpler Ty-based IR3 models to close the performance gap -equivaaarai models the richer R 52 domain. Surprisingly, the latter often achieve superior results more efficiently within given computational budget despite the higher computational complexity (Fig. 1). Moreover, performance frequently saturales with increasing capacity (Tab. 3), suggesting that models reach the limits of their architecturaall--efieee hypothesis space for geometric expressivity, rather than merely lacking parameters. Thus, the fundamental architectural choice often outweighs sheer capacity and computational resources in bridging these generalization gaps. This claim is further underpinned by additional scaling experiments on QM9, which show that the SIE(3 inductive bias maintains its significant advantage even when T3s models are trained with matched total representational capacity and comparable computational time (App. E.2).
The influence of data scaling and symmetry relevance (RQ3) highlights that (E(3)-equivarian models particularly excel in low-data regimes, showcasing strong data efficiency. seen the intrinsically small CMU motion dataset and with reduced ShapeNet training data (Fig. 4). Notably, this data-efficiency benefit of SE(3) group convolutions persists even for Shape aligned data, where strict (0 -equivariance for the specific output aligoment might seem less critical.
Concerning pose-informed symmetry breaking (RQ4), our experiments affirm its empirical benefits. Incorporating explicit geometric reference information, such as global frame, consistently improved performance across diverse tasks like ShapeNet segmentation and CMU motion prediction (Tab. 2). This aligns with the theoretical expectation (Corollary 3.1, App. D.3, and App. D.4) that providing a canonical reference frame enhances expressivity by allowing models to disambiguate inputs.
by introducing additional information (RQ5), the impact is nuanced and task-dependent, as illustrated in Fig. 2. Providing geometric information in manner that breaks strict (((()))-quvvriince (e.g., coordinates as scalar features) proved advantageous for QM9 tasks (Tab. 1) and for ShapeNet generation (Tab. 2), where it likely helped capture tesk-relevant
Finally, regarding equivariance-brenn

---

non-symmetric aspects and absolute spatial references. It also benefited ShapeNet segmentation on non-aligned samples. However, this is does not seem a universal strategy for improvement, as indiscriminate breaking or unsuitable information can be neutral or detrimental to performance.
Limitations Our empirical explorations, while extensive and in line with typical academic resources (e.g., training for Fig. was within a 9-houu budget per run, and tabular results based on up to 1000 epochs), used modest computational budget. This could limit the exhaustive exploration of scaling laws (RQ2) for simpler R--based models, although observed performance saturation suggests these specific architectures may have already approached their practical limits within our framework. Furthermore, this study's 's scope is centered on convolutional architectures applied to point cloud data; while we believe many principles discussed may generalize, an explicit investigation into other modalities or architectures, such transformers, remains future work.
7 Conclusion
In this paper, we aimed to constructively contribute to an ongoing discussion surrounding the interplay of scale and equivariance in deep learning. We divide this broad discussion into concrete research questions (RQ1-5) and then address these through exhaustive experimentation and analysis. Through the introduction of Rapidash-2 general regular group convolutional architecture enabling controlled comparisons, we have sought to provide empirical insights into these critical trade-off's with an extensive suite of nuanced experiments. We couple these results with targeted theoretical analysis of equivariance, symmetry breaking, and generalization.
Our findings offer structured understanding of when and how different equivariance strategies and symmetry considerations benefit practical model performance, thereby offering guidance for future architectural design and model selection in this domain.
Acknowledgements
Shavaree Vadgama is funded by the Hybrid Intelligence Center, a 10-year programme funded through the research programme Gravitation, which is (partly) financed by the Dutch Research Council (NWO). This publication is part of the project SIGN with file number VI. 23di. 233.220 of the research programme Vidi, which is (partly) financed by the Dutch Research Council (NWO) under the grant https://doi. org/ 10. 61686 (5886/PXQ271565
SV would like express sincere appreciation for the valuable feedback provided by Tejaswi Kasarla and Riccardo Valperga. SV is very grateful for the detailed discussion on the manuscript with members of the LCV group at NYU/Flatiron Institute. The authors gratefully acknowledge the reviewers for their detailed and constructive feedback, which helped strengthen this work.
10

---

References
Josh Abramson, Jonas Adler, Jack Dunger, Richard Evans, Tim Green, Alexander Pritzel, Olaf Ronneberger, Lindsay Willmore, Andrew Ballard, Joshua Bambrick, et al. Accurate structure prediction of biomolecular interactions with alphafold 3. Nature, pages 1 1-3, 2024.
Simon Batzner, Albert Musaelian, Lixin Sun, Mario Geiger, Jonathan P. Mailoa, Mordechai Kombluth, Nicola Molinari, Tess E. Smidt, and Boris Kozinsky. E())--quivariant graph neural networks for data-efficient and accurate interatomic potentials. Nature Communications, 13(1), May 2022. ISSN 2041-1723. doi: 10. 00.10388441447-00-2999999 841467-022-299399
URL hwtp:http//// /tsp:///dxxxd...... www.o.ooooooooooooooooooooeoeeoe 2t.g/10.10 1038/
Erik Bekkers, B-spline enns on lie groups. International Conference on Learning Representations, 2019.
Erik Bekkers, Maxime W Lafarge, Mitko Veta, Koen AJ Eppenhof, Josien PW Pluim, and Remco Duits. Roto-translation covariant convolutional networks for medical image analysis. In Medical Image Computing and Computer Assisted Intervention-MICCA. 2018: 21st International Con- ference, Granada, Spain, September 16-20, 2018, Proceedings, Part I, pages 440 448. Springer, 2018.
Erik Bekkers, Sharvaree Vadgama, Rob Hesselink, Putri A Van der Linden, and David W. Romero. Fast, expressive se(n) equivariant networks through weight-sharing in position-orientation space. In The Twelfih International Conference on Learning Representations, 2024. URL https:// openreview wwt.:eeettfoooeee http:/rrrbmm..dPPLP.ullall dddPHL.UuQurr
Johannes Brandstetter, Rob Hesselink, Elise van der Pol, Erik J Bekkers, and Max Welling. Geometric and physical quantities improve (3) equivariant message passing. In International Conference on Learning Representations, 2022.
Johann Brehmer, Pim de Haan, Sönke Behrends, and Taco Cohen. Geometric algebra transformer, 2023. URL https: / /ttp://aaavvv///a org/ abaaabaa22200 18415.
Johann Brehmer, Sönke Behrends, Pim de Haan, and Taco Cohen. Does equivariance matter scale?, 2024. URL / org//or/////eeeeeeeooeee ab////b/...../////////////////// 23179.
Gabriele Cesa, Leon Lang, and Maurice Weiler. A program to build e (n)-equivariant steerable enns. In International Conference on Learning Representations, 2021.
Angel X.
Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu. Shapenet: An information-rich 3d model repository, 2015. URL https:///axtti org/ abs/ 1512.03012
François Chollet. Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1251-1258, 2017.
Taco Cohen and Max Welling. Group equivariant convolutional networks. In International conference on machine learning, pages 2990-299099 2999. PMLR, 2016.
Taco S Cohen, Mario Geiger, and Maurice Weiler. A general theory of equivariant cnns on homoge- neous spaces. Advances in neural information processing systems, 32, 2019.
Nadav Dym and Haggai Maron. On the universality of rotation equivariant point cloud networks, In International Conference on Learning Representations, 2020.
Bryn Elesedy and Sheheryar Zaidi. Provably strict generalisation benefit for equivariant models. In International conference on machine learning, pages 2959-2969. PMLR, 2021.
Matthias Fey and Jan Eric Lenssen. Fast graph representation learning with pytorch geometric. arXiv preprint arXiv:1903 2019.

---

Mare Finzi, Samuel Stanton, Pavel Izmailov, and Andrew Gordon Wilson. Generalizing convolutional neural networks for equivariance lie groups on arbitrary continuous data. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning. volume 119 of Proceedings of Machine Learning Research, pages 3165-3176. PMLR, 13-18 Jul 2020. URL https: / /ttp//peeeeenns..//ee////e mlr. prwss/vissiee www.///f//n///// httmmii220a..............
Fabian Fuchs, Daniel E. Worrall, Volker Fischer, and Max Welling. Stansstransformmr:: 3d roto- translation equivariant attention networks, 2020. URL httts::///axiiti prrg/abs/2000 10503.
Johannes Gasteiger, Florian Becker, and Stephan Günnemann. Gemnet: Universal directional graph neural networks for molecules. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances Neural Information Processing Systems, 2021. URL https:
wwwpe/rvviee..ee/forroeeee http://orummmd-HS____1 =0ax59K-
Johannes Gasteiger, Shankari Giri, Johannes T. Margraf, and Stephan Günnemann. Fast and uncertainty-aware directional message passing for non-equilibrium molecules, 2022. URL 14115.
https://arxiv.orgg
Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural message passing for quantum chemistry. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 1263-1272. PMLR, 06-11 Aug 2017. URL www.h:tp///ppie httope///proceeddngg.
mlr. w..r.pesssvv00 1000//ggnnn gilmer17a. html.
Ralph Gross and Jianbo Shi. The cmu motion of body (mobo) database. Technical Report CMU-RI- TR-01-18, Carnegie Mellon University, Pittsburgh, PA, June 2001.
Nate Gruver, Mare Finzi, Micah Goldblum, and Andrew Gordon Wilson. The lie derivative for measuring learned equivariance, 2024. URL htttp:://aravv.orraabb///// 22t1//22...000.....22.2
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models, 2020. URL org/ :tttp::/arrdv..or///a///000000
11239.
Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant diffusion for molecule generation in 3d, 2022. URL htttps://axxiv orgg at//ab///22000 2203. 17003.
Chenqing Hua, Sitao Luan, Minkai Xu, Zhitao Ying. Jie Fu, Stefano Ermon, and Doina Precup. Mudiff: Unified diffusion for complete molecule generation. In Learning Graphs Conference, pages 33-1, PMLR, 2024.
Wenbing Huang, Jiaqi Han, Yu Rong, Tingyang Xu, Fuchun Sun, and Junzhou Huang, Equivari- ant graph mechanics networks with constraints, 2022. URL https://arxii..ort wwwpa/iwww..oo/b/////ba//////// 06442.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models, 2022. URL https://arxtii wttpii...rga/bb/////eee .0364.
Hyunsu Kim, Hyungi Lee, Hongseok Yang, and Juho Lee. Regularizing towards soft equivariance un- der mixed symmetries. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 16712-16727. PMLR, 23-29 Jul 2023. URL https: htap::://proo http://ppocccceneeeeeeeeeeee mlr www.ppresssviie /tt///22///222// html.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 1rX1v:1412.6988, 2014.
Thomas Kipf, Ethan Fetaya, Kuan-Chieh Wang, Max Welling, and Richard Zemel. Neural relational inference for interacting systems, 2018. URL httpss://arxiv.orr//aaaaae hbtg://bs/110..... 04687.
David M Knigge, David W Romero, and Erik J Bekkers. Exploiting redundancy: Separable group convolutional networks on lie groups. In International Conference on Machine Learning, pages 11359-11388 PMLR, 2022.
12

---

Risi Kondor and Shubhendu Trivedi. On the generalization of equivariance and convolution in neural networks to the action of compact groups. In International conference on machine learning, pages 2747-2755. PMLR, 2018
Risi Kondor, Zhen Lin, and Shubhendu Trivedi. Clebsch gordan nets: fully fourier space spherical convolutional neural network. Advances in Neural Information Processing Systems, 31, 2018.
Thijs P Kuipers and Erik Bekkers. Regular se (3) group convolutions for volumetric medical image analysis. In Medical Image Computing and Computer Assisted Intervention MierventtonnMICCAA 2023, arXiv preprint arXiv:2306. 13360, 2023.
Hannah Lawrence, Vasco Portilheiro, Yan Zhang, and Sékou-Oumar Kaba. Improving equivariant networks with probabílistic symmetry breaking. In The Thirteenth International Conference on Learning Representations, 2025a. URL fttt::://openrvvieee wwt.
Hannah Lawrence, Vasco Portilheiro, Yan Zhang, and Sékou-Oumar Kaba. Improving equivariant networks with probabilistic symmetry breaking, 2025b. URL pestps://arxive org/ /btpg//abb///0/.. 21985.
Yann LeCun, Koray Kavukcuoglu, and Clément Farabet. Convolutional networks and applications in vision. In Proceedings of 2010 IEEE international symposium on circuits and systems, pages 253-256. IEEE, 2010.
Yi-Lun Liao and Tess Smidt. Equiformer: Equivariant graph attention transformer for 3d atomistic graphs, 2023. URL
orgg argg/bbs/2206 1990.
Yi-Lun Liao, Brandon Wood, Abhishek Das, and Tess Smidt. Equiformerv2: Improved equivariant transformer for scaling to higher-degree representations, 2024. URL pattts://arxtti org/ abs/ 2306. 12059.
Cong Liu, David Ruhe, Floor Eijkelboom, and Patrick Forré Clifford group equivariant simplicial message passing networks. The Twelfth International Conference on Learning Representations, 2024. URL https: / /ttp:://///e/eeeeeeeeeeee nww/teruul wieiumddddZ22994NNB
Zhuang Liu, Hanzi Mao, Chao- Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1977--19866 2022.
Shitong Luo and Wei Hu. Diffusion probabilistic models for 3d point cloud generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2837-2845, 2021.
Artem Moskalev, Anna Sepliarskaia, Erik J. Bekkers, and Amold Smeulders. On genuine invariance learning without weight-tying, 2023. URL htttp:://axxi: org/ abs//b///2// ab38 03904.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
Andrea Perin and Stephane Deny. On the ability of deep networks to learn symmetries from data: A neural kernel theory, 2024. URL https://arxiv wwtperiiii..viiooeee wrgg/bss///// /2422 11521.
Stefanos Pertigkiozoglou, Evangelos Chatzipantaziss Shubhendu Trivedi, and Kostas Daniilidis. Improving equivariant model training via constraint relaxation, 2024. URL https://arxiv. htggabss24085 13242.
Mircea Petrache and Shubhendu Trivedi. under (approximate) group equivariance. In A. Oh, T. Naumann, A. Glober- son, Saenko, M. Hardt, and Levine, editors, Advances in Neural Informa- tion Processing Systems, volume 36, pages 61936-61959. Curran Associates, Inc., 2023. URL. https://proceeddnng www.eeddiinnnnnninneeeeeee neurips. cttccc/ppppa htppppaprrrr_ ppppe_fills/////pppppppppppppeee sateeeffllee//pper////// filess/paperr2222//fiee 2G5B88266681111555011122aaaa2222 Papeeereeee-P-eree--eeeereeeeeee Conference pdf.
generaiiionnnneralizzaiioono trade-offs
13

---

Charles R. Qi, Hao Su, Kaichun Mo. and Leonidas J. Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation, 2017. URL httpp/////aai..o/aaaa/////////// wwwp.bs////.......////////// 00593.
Guocheng Qian, Yuchen Li. Houwen Peng, Jinjie Mai, Hasan Abed Al Kader Hammoud, Mohamed Elhoseiny, and Bernard Ghanem. Pointnext: Revisiting pointnet++ with improved training and scaling strategies, 2022. URL http://axxvvvoooo org/ aarg/abs/22066 wwwps//2200000000...
Eric Qu and Aditi Krishnapriyan. The importance of being scalable: Improving the speed and accuracy of neural network interatomic potentials across chemical domains, 2024. URL https://arxiv.
a2/////.s/22....///// 24169.
Rupp M.and Von Lilienfeld O.A. Ramakrishnan R., Dral P.O. Quantum chemistry structures and properties of 134 kilo molecules, 2014.
Siamak Ravanbakhsh, Jeff Schneider, and Barnabás Póczos. Equivariance through parameter-sharing. In Doina Precup and Yee Whye Tch, editors, Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 2892-2901. PMLR, 06-11 Aug 2017. URL https:/ https/////cceeeess..... wttpeeeeedinnn.............e m html.
David Ruhe, Johannes Brandstetter, and Patrick Forré, Clifford group equivariant neural networks, 2023. URL pettps://artie org/ abs/2305. 11141.
Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E (n) equivariant graph neural networks. In International conference machine learning, pages
PMLR, 2021.
Ivan Sosnovik, Michat Szmaja, and Amold Smeulders. Scale-equivariant steerable networks. In International Conference Learning Representations, 2019.
Siddharth Srivastava and Gaurav Sharma. Exploiting local geometry for feature and graph construction for better 3d point cloud processing with graph neural networks, 2021. URL https://arx// org/abs /2taa///.....////////aa/.//a.aaa 2203. 15226.
Savannah Thais and Daniel Murnane. Equivariance is not all you need: Characterizing the utility of equivariant graph neural networks for particle physics tasks, 2023. URL httts:///aaxiii org/ 0tt//3311.00000
Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang, Li, Kai Kohlhoff, and Patrick Riley, Tensor field networks: Rotation-and translation-equiivarinn neural networks for 3d point clouds. arXiv preprint 180. 01/8802.02211 19, 2018.
Tycho A. van der Ouderaa, David W. Romero, and Mark van der Wilk. Relaxing equivariance constraints with non-stationary continuous filters, 2022. URL https://arxivy org/ 2204. 07178.
Soledad Villar, David W Hogg, Kate Storey-Fisher, Weichi Yao, and Ben Blum-Smithh Scalars are universal: Equivariant machine learning, structured like classical physics. Advances in Neural Information Processing Systems, 34:28848-28866 2021.
Rui Wang. Robin Walters, and Rose Yu. Approximately equivariant networks for imperfectly wttp:///rgg////////////o 22011 11969.
symmetric dynamics, 2022. URL https:
Maurice Weiler and Gabriele Cesa. General e (2)-equivariant steerable enns. Advances in neural information processing systems, 32, 2019.
Maurice Weiler, Fred Hamprecht, and Martin Storath. Learning steerable filters for rotation equivariant cnns. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition,, pages 849-858, 2018.
Maurice Weiler, Patrick Forré, Erik Verlinde, and Max Welling. Coordinate independent convolutional networks-isometry and gauge equivariant convolutions on riemannian manifolds. arXiv preprint arXiv:2106.06020, 2021.
14

---

Ruben Wiersma, Ahmad Nasikun, Elmar Eisemann, and Klaus Hildebrandt. Deltaconv: anisotropic operators for geometric deep learning on point clouds. ACM Transactions on Graphics, 41(4): 1-10, July 2022. ISSN 1557-7368. doi: 10. 10.1145/35222233533006 hRI.htpp://de doi org/ 10. 1...111//52223... 3530116..
Daniel E. Worrall, Stephan Garbin, Daniyar Turmukhambetov, and Gabriel J. Brostow. Harmonic networks: Deep translation and rotation equivariance. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), July 2017.
Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes, 2015. URL https://arx::: org/abs/ 1606.5670.
YuQing Xie and Tess Smidt. Equivariant symmetry breaking sets, 2024. URL https://arxiv. org/abs/2402. 02681.
Xiaohui Zeng, Arash Vahdat, Francis Williams, Zan Gojeic, Litany, Sanja Fidler, and Karsten Kreis. Lion: Latent point diffusion models for 3d shape generation, 2022. URL https://arxix:
org/abs/2210.00 06978.
Lingi Zhou, Yilun Du, and Jiajun Wu. 3d shape generation and completion through point-voxel diffusion, 2021. URL https::/// htttp:://axiivv/ wrt. /2104. 03670.
15

---

A Mathematical Prerequisites and Notations
Groups. A group is an algebraic structure defined by set G and a binary operator ; G x G - G. known as the group product. This structure (G.-) must satisfy four axioms: (1) closure, where Vh.geG h- g E G; (2) the existence of an identity element e E G such that e = e = 9. (3) the existence of inverse element, i.e. Vyec there exists a - G such that g y- 9 e; and (4) associativity, where Vg.h.pEG: (g h)) -p=g (h- (h.p). Going forward, group product between two elements will be denoted as g.9' G by juxtaposition, i.c., as gg'.
For Special Euclidean group SE(n), the group product between two roto-translations g=((, R) and g = (x', RR) is given by (x, R. (x', R')=(Rx' + R,,RRR) R'), and its identity element is given by e= (0.I).
Homogeneous Spaces. A group can act on spaces other than itself via group action gT' : G x X + X, where X the space on which G acts. For simplicity, the action of g E G on x X is denoted as g zz Such transformation is called a group action if it is homomorphic to G and its group product. That is, follows the group structure: (gg') x= (g' x) Vg.g g E G, E X, and e e==r. For example, consider the space of 3D positions X R e.g., atomic coordinates, acted upon by the group G=SE(3). A position p E IR is roto-translated by the action of an element (x, R) E SE(3) as (x, R. p-Rp +
group action is termed transitive every element æ E X can be reached from an arbitrary origin Zo E X through the action of some g E G, i.e., z=gzo. A space X equipped with a transitive action of G is called a homogeneous space of G. Finally, the orbit Gx == {g 1 g E G} of an element under the action of a group G represents the set of all possible transformations of x by G. For homogeneous spaces, X=G 2i for any arbitrary origin 2o E X.
Quotient spaces. The aforementioned space of 3D positions X= serves as homogeneous space G -3) as every element p can be reached by roto-translation from i.e., for every p there exists (x, R) such that p- (x, R) 0 R 0 + x-x. Note that there are several elements in (E) that transport the origin 0 to p. as any action with translation vector x-p suffices regardless of the rotation R. This is because any rotation R' E SO(3) leaves the origin unaltered.
We denote the set of all elements in G that leave origin Zu X unaltered the stabilizer subgroup Stabaa In subsequent analyses, the symbol H is used to denote the stabilizer subgroup of a chosen origin z6n in homogeneous space, i.e., H ( We further denote the left coset of H in G as H == {g h h E H). In the example of positions E X-R3 we concluded that we can associate point p with many group elements g E (3) that satisfy p=g O. In general, letting 9x be any group element s.t. x= To, then any group element in the left set gr H is also identified with the point p. Hence, any x X can be identified with left coset g and vice versa.
Left cosets g H then establish an equivalence relation ~ among transformations in G. We say that two elements g. g E G are equivalent, i.e., 9 ~g'' g', if and only if g zo=g' zu- That is, if they belong to the same coset g H. The space of left cosets is commonly referred to as the quotient space G H.
We consider feature maps f X - C as multi-channel signals over homogeneous spaces X. Here, we treat point clouds as sparse feature maps, e.g., sampled only at atomic positions. In the general continuous setting, we denote the space of feature maps over X with X. Such feature maps undergo group transformations through regular group representations p [() X + X' parameterized by g. and which transform functions/ f X via l0x (1) x
Irreducible representations and spherical harmonics Given any representation, there are often or- thogonal subspaces that do not interact with each other, making it possible to break our representation down into smaller pieces by restricting to these subspaces. Hence, it useful consider the repre- sentations that cannot be broken down. Such representations are terms irreducible representations or irreps. Given a group G. a vector space, and p: G > GT.(V) representation, a representation is irreducible if there is no nontrivial proper subspace W C V such that plw is representation of G over space W. With each irrep there an associated (harmonic) frequency The irreps of SO(3) are given by the (2l + 1) x (2l + 1) dimensional rotation matrices called Wigner-D matrices. The
central columns of these matrices comprise the set of 27 + 1 spherical harmonics y(l) 82 - R.
indexed by m -l,..,
16

---

B Rapidash: A Regular Group Convolution Approach for Flexible Equivariant Modeling
The Rapidash architecture, employed throughout our empirical study, builds upon the efficient SE(3) -equi--aguva regular group convolution framework operating on position-orientation space (R3 x 5") as introduced by PONITA [Bekkers et al., 2024]. To facilitate a comprehensive investiga- tion into the utility of equivariance and symmetry breaking, Rapidash extends this foundation by incorporating several key flexibilities: (i) versatile handling various input and output geometric quantities (e.g., scalars, vectors representing positions, normals, or pose information); (ii) enhanced scalability for large point clouds through multi-scale processing incorporating techniques like farthest point sampling; and (iii) convenient adaptability to different equivariance constraints, allowing for controlled comparisons between SE(n) and translation-onle (Tw) equivariant models. Like PONITA, Rapidash primarily adopts the regular group convolution paradigm, distinguishing from steerable G-CNNs or tensor field networks, although fundamental connections exist. This section elucidates the theoretical underpinnings of this approach.
B.1 Regular vs. Stecrable Group Convolutions
Equivariant neural networks for SE(3) are often categorized into regular or steerable (tensor field) approaches [Weiler and Cesa, 2019].
These typically operate on multi-channel scalar fields defined over a group G' or a homogeneous space X = G/H (like I3 x 8 SE(3)/SO(2) in our case). Feature fields f - R transform via the regular representation: [p()) f( (z). Convolutions are then form of template matching of kernel k(g,, æ)) with the input signal [Cohen and Welling, 2016, Bekkers, 2019]. key advantage is that woint-wise nonlinearities can be applied directly to these scalar feature maps without breaking equivariance [ef. Bekkers et al., 2024, Appx. A.
Regular Group Convolutions:
Steerable Group Convolutions / Tensor Field Networks: These operate feature fields f :R + Vp where the codomain V, is a vector space carrying a representation p of SO(n) (often
of irreducible representations, irreps). Features transform via in- duced representations, affecting both the domain and codomain: (IInd pl (x) =
p(R) (xg [Weiler et al., 2021]. Kernels k(x) must satisfy a steerability constraint k &(Rx) = Poux (x) (R-1). While this allows for exact equivariance without dis- cretizing the rotation group (if using irreps), applying nonlinearities typically requires specialized equivariant operations or transformations to a scalar basis, as standard element- wise activations on steerable features (vectors/tensors) can break equivariance [Weiler and Cesa, 2019].
B.2 Rapidash as a Regular Group Convolution and its Relation to Steerable Networks
Rapidash, like PONITA [Bekkers et al., 2024], processes feature fields f R3 x S2 + RC, That is, at each point x E R$, it maintains scalar signal fx S - C defined over the sphere of orientations S2. This aligns with the regular group convolution paradigm, kernel acts an 3 x S2, subject to symmetry constraint due to the quotient space structure Specifically, the SE(3) group convolution over RR x S is of the form
which the convolution
R3 x S2 = (E(3)
= [Lf](x, n) = // k(R,, T(xxxx' - x), Rn T "x') (xl'f(x')na) n')dx'n'',
with kernel constraint VR. E SO(2): k(Rx, Rn) = k(x, n) with R₂ a rotation around the z-axis. This symmetry constraint automatically solved when conditioning message passing layers (such as convolution layers) on the invariants outlined in [Bekkers et al., 2024, Thm. 1J. In terms of these invariants, the resulting discrete group convolution is given by
[.]](x;,n.) = Σ Ma,))((x,n))
(3)
JEV())
17

---

with the invariant wise attributes given by
n (x) x.) ny (x) - x.)l
n n;
with 1. denoting part of the vector n, orthogonal to xo, X2-
As detailed in [Bekkers et al., 2024, App A], the connection between regular and steerable convolu- tions is established through Fourier analysis the group/homogeneeus space [Kondor et al., 2018, Cesa et al., 2021]. A scalar field /x (n) over 52 (as used in Rapidash/P@MITA) can be decomposed into spherical harmonic coefficients (its Fourier transform) through spherical Fourier transform. These coefficients for different spherical harmonic degrees correspond to features transforming under irreducible representations of SO(3), which are the building blocks of steerable tensor field networks.
Proposition B.1 (Equivalence via Fourier Transform). A regular group convolution operating on scalar fields f IR" Y + RC (where Y is ST or (O())) can be equivalently implemented as a steerable group convolution operating on fields of Fourier coefficients f: R + Vp (where Vo is the space of combined irreducible representations) by performing point-wise Fourier transforms Fy before the steerable convolution and inverse Fourier transforms Fy after.
Remark This equivalence is discussed in depth by Bekkers et al. f2024, Appx. A.1], Brandstetter et al. [2022], and Cesa el al. [2021]. Consequently, Rap dasi could, in principle, be reformulated as a tensor field network by operating in the spherical harmonic (Fourier) domain.
B.3 Universal Approximation
The universal approximation capabilities of equivariant networks are crucial. For steerable tensor field networks, Dym and Maron [2020] proved universal approximation properties for equivariant graph neural networks. Building on such results, and the correspondence between regular and steerable views, it has been shown that message passing networks (which include architectures like Rapidash) conditioned on appropriate invariant attributes over cotttnttooiintttioon space (IR" t x S 1"11) are equivariant universal approximators.
Corollary B.1 (Universal Approximation for Rapidash). Rapidash, an instance of message passing networks over R3 x S² with message functions conditioned on the bijective invariant attribttes (derived in Bekkers et al. [2024, Thm. I]), is an SF(3) -qquivaaqu universal approximator.
from Remark 3. This follows from Bekkers et al. [2024, Cor. 1.1], which itself builds on universality results for steerable GNNs [Dym and Maron, 2020] and for invariant networks used to construct equivariant functions [Villar et al., 2021, Gasteiger et al., 202 J.. The key is that feature maps over TR33 x 5 are sufficiently expressive.
B.4 Advantages of the Regular Group Convolution Viewpoint for Rapidash
Rapidash adopts the regular group convolution viewpoint, working with scalar signals on discretized spherical fibers (x,, nz) where ng grid points on 5²). This offers practical advantages:
1. Simplicity of Activation Functions:
Since the features f(x, ng) at each grid point are scalars (or vectors of scalars in the channel dimension), standard element-wise nonlinear activation functions (e.g., GELU, ReLU, SiLU) can be applied directly without breaking -quuivaauvv This is because the action of E SE(() permutes these values on the fiber spatially, but the activation acts on each scalar value independently. In contrast, steerable tensor field networks require specialized equivariant nonlinearities or norm-based activations on higher-order tensors to preserve equivariance [Weiler and Cesa, 2019], which can limit expressivity or introduce computational overhead.
2. Computational Efficiency of Activations: While steerable networks can apply scalar activations by first performing an inverse Fourier transform (to get scalar fields), applying the activation, and then forward Fourier transform (back to irreps), this incurs significant computational cost at each nonlinearity [cf. Bekkers et al., 2024, Appx. A. By operating directly on scalar spherical signals, Rapidash avoids these repeated transformations. Previ- ous work on steerable group convolutions has indeed found that element- wise activation
18

---

functions applied to scalar fields (obtained via inverse Fourier transforms from steerable vector features) can be most effective [e.g,, implicitly done in some equivariant GNNs by taking norms or scalar products before activation, or as explicitly discussed for general steerable CNNs in Weiler and Cesa, 2019].
3. Conceptual Simplicity:
The regular group convolution approach, involving template matching of kernels over signals on G/H, can be more intuitive and closer to standard CNN paradigms than navigating representation theory and required for constructing steerable tensor field networks. Concepts like stride/sub-sampling and normalization layers readily transfer this setting.
tensor products often
While discretizing the sphere S introduces an approximation full SO(3) equivariance (equivari- ance up to the grid resolution), empirical results, including those for PONITA [Bekkers et al., 2024] and our findings with Rapidash, demonstrate that this not detrimental achieving state-of-theearr performance and robust generalization.
B.5 Separable group convolutions
Regular group convolutions over the full spacew SE(3) can be efficiently computed when the kernel is factorized via
(x,e(x, R) = R (x))ke
80(3)
(R) (R)/echane))
with o, the row and column indices of the "channel mixing" matrix. Then the group convolution equation can be split into three steps that are each efficient to compute: a spatial interaction layer (message passing), point-wise SO(3) convolution, and point-wise linear layer [Knigge et al., 2022, Kuipers and Bekkers, 2023]. It would result in the group convolutional counterpart of depth-wise separable convolution [Chollet, 2017], which sepa- rates convolution in two steps (spatial mixing and channel mix- ing). In particular, for our choice to the the group convolution over 3R3 x S using the pair- wise invariants of (4) the kernel is parametrized as
GELU
Lkoe (diaad)]
Figure 6: Block design with base space R3 x S2.
(ay)) = h (ag Oif Da. (a (1())aecanmn
(1) (2)
This form allows to split the convolution over several steps, which following [Bekkers et al., 2024] we adapt ConvNext [Liu et al., 2022] as a the main layer to parametrize Rapidash. Fig. shows the the steps performed in this block, relative a standard ConvNext block over position space only. Here LN denote layer norm and GELU is used as activation function.
C Sources of Equivariance Breaking
In the study of equivariant deep learning, is important to distinguish between different notions of "breaking" symmetries. Some approaches, such as those explored by Lawrence et al. [2025a] or the pose-conditioning methods analyzed in Appendix D. utilize architectures that maintain specific (joint) equivariance properties to achieve "symmetry breaking" in the output (e.g., an output sample being less symmetric than the input, or overcoming the limitations of standard invariance). This section, in contrast, focuses on mechanisms by which the strict Geequivariance of a neural network architecture itself can be compromised or intentionally relaxed, leading to a model that no longer fully adheres to the mathematical definition of G-equivariance. We categorize these into external and internal sources of equivariance breaking.
C.1 External Equivariance Breaking
External equivariance breaking occurs when an inherently G-equivariant architecture loses its equiv- ariance properties due to the way inputs are provided to network. Consider a linear layer designed to be G-equivariant (e.g., for G SE(3)). Let be vector input that transforms naturally under the group action, and define Vig = g u as its transformed version, described in global coordi- nates. When these coordinates 2g are provided, for example, as set of scalar triplets rather than
19

---

as geometric vector type that the layer is designed to process equivariantly they may be treated independently by the network without regard their collective transformation under G. In such cases, we have:
if # Fig then often [(x,)) # g [(x)) (or [(rg)) * T(xx) for invariant L.)
This means the network processes transformed inputs in a way that does not respect the group symmetry, thereby breaking the intended G-equivariance of the function L. In contrast, when inputs are specified as types that transform appropriately under the group action (e.g., as vectors for an (B)333)quuiaaiii layer that expects vector inputs), equivariance be maintained:
Lig. v) = L(v) Vg E G
(6)
Moreover, for features that are truly invariant under G (such as one-hot encodings of atom types in QM9, which invariant 50(3) rotations), the group action is trivial:
g 2 Tire
Tiny
Lig Tine) = (z
This ensures that processing these features does not break the network's 's overall desired invariance to group transformations of other, ooniinati inputs.
C.2 Internal Equivariance Breaking
Internal equivariance breaking refers to the deliberate relaxation or incorrect specification of equiv- ariance constraints within the layers themselves. Even if inputs are provided correctly, the layer operations may not fully respect the symmetry group G. Recent works have explored various approaches to controlled relaxation, including [Wang et al., 2022]:
Basis decomposition methods that mix G-equivariant and non-C--quivariant components within layer.
Learnable deviations from strict G-equivariance, often controlled by regularization terms.
Progressive relaxation or enforcement of G-equivariance constraints during the training process.
In some cases, an internally broken layer Lrooken might be expressed as a combination of a strictly G-equivariant part and a non-G-equivariant part:
Loroken (X) Lequiv (X) aopoo--quiv (X)
where a controls the degree of deviation from strict G-equivariance. Some approaches (e.g., [Wang al., 2022]) implement schemes where is annealed during training.
C.3 Interplay of Equivariance Breaking Mechanisms
In practice, both external and internal equivariance- breaking can occur, sometimes simultaneously, and their effects can interact. For example, network might employ layers with relaxed internal constraints (e.g., non-stationary filters) while also processing geometric inputs (like coordinates) in a manner that externally breaks the intended global symmetry (e.g., treating them as independent scalar features). The overall adherence of the model to a specific group symmetry then depends on the interplay of these factors. As noted by Petrache and Trivedi [2023], while aligning data symmetry with architectural symmetry (strict equivariance) ideal, models with partial or approximate equivariance can still exhibit improved generalization compared to fully con-equivariant ones.
C.4 Relevance to Architectural Choices in Our Study
While our study primarily investigates scenarios of external equivariance breaking (c.g., how pro- viding coordinate information as scalars versus vectors impacts overall SE(3) equivariance, cf. the red exclamation marks our tables), some of our architectural comparisons also touch upon concepts related to the scope of equivariance. For instance, choosing to build model with strictly Tz-equivariant layers (translation-een -equvvariiiiiviiiantttn only, Tab. 2, rows 15-18) instead of -Eeue-equivariant layers (Table 1, rows 1-4) results in model that is not (3eeuu--uaaa overall. This is an architec- tural design choice selecting a different, less encompassing symmetry group, rather than starting with an SE(3) llyee and internally relaxing its SE(3) constraints.
20

---

Our experiments demonstrate that carefully managing how equivariance is implemented or broken, whether externally through imput representation or by choosing specific architectural symmetry properties, can significantly impact model performance. especially when the underlying data exhibits only approximate symmetries or when task-relevant information is tied to a canonical orientation. This aligns with recent findings showing benefits from various approaches to relaxed modified cquivariance constraints [van der Ouderaa et al., 2022, Kim et al., 2023, Pertigkiozoglou et al., 2024]. Further theoretical analysis of how explicitly providing pose information can be seen as principled modification of standard invariance is presented in Appendix D.
C.5 Relationship to Our Work
While our study primarily focuses on external symmetry breaking (cf. the red exclamation marks in our tables), we note that the transition between canslation-equivariann layers (Table 2, rows 15-18) and roto-translation equivariant layers (Table 1, rows 1-4) could be viewed as form of internal symmetry breaking. However, we distinguish our approach from explicit internal symmetry-breakinn methods as we do not implement continuous relaxation of equivariance constraints within layers, but rather compare discrete architectural choices with different symmetry properties.
D Analysis of Symmetry Breaking and Pose-Conditioning in Equivariant Models
D.1 Standard Invariance and Effective Pose Entropy
This section formalizes the notion that a standard 5O(3)-invarian) model, when viewed within the framework of a jointly invariant function f(X,, 2). operates as if the auxiliary pose variable Z carries no specific information, i.e., it corresponds to maximum entropy distribution over poses.
Proposition D.1 (Standard -nvariine as Maximum Effective Pose Entropy). Let f(X, Z) be a function f X SO(3) Y that is jointly SO(3) -invariaai,,,,, meaning (ggX, = f(X, 2) for all E (O))). If the output of f(X, Z) is also required to be standard 50 (0(3)-invariant with respect X alone, defining a function Jino(X) such that [gX) fine (X) for all g E 5O(3), then:
1. The function f(X. 2) must be independent of the auxiliary pose variable Z. Specifically, f(X, 2) f(X, for any Z SO(3), where Id is the identity element in SO
2. Consequently, such model f(X, 7) (which produces Jine fi))(X) behaves as Z is drawn from an uninformative, maximum entropy distribution (e.g., the uniform distribution over 50(3)) The model cannot utilize any specific orientation information conveyed by Z.
Proof Sketch. To demonstrate part 1, that s(X, Z) = S(X
1. By the joint (0(3)-invariance of f. have f(X.2) f(z-'x,z-12) f(z f(1))x Id)..
2. The condition that the output of f(X is standard (O(3)-invariant with respect to X means that for any fixed second argument (like Id), the function f(,, Id) must be SO(3)-
invariant in its first argument. That is, f(gX, Id) = s(X, Id) for all E SO(3).
3. Applying this standard invaarnneee with g = - to the expression f(Z- Id). we get f(z-'X) x, Id) = f(X,Id,
4. Combining steps and 3: f(X, Z) = (1 = fd)
This establishes that f(X, Z) is independent of Z.
For part f(X, Z) is independent of Z, cannot make use of any particular value of Z to alter its output. From an informational perspective, Z provides no specific information to the model. This behavior equivalent to Z being drawn from a distribution that reflects maximal uncertainty about the pose, which for compact group like 50(3) is the uniform (Haar) measure, corresponding to maximum entropy. Thus, the model Sino (X cannot utilize any specific orientation information that might be notionally carried by Z.
D
21

---

D.2 Proof of Corollary 3.1 (Expressivity Gain from Low-Entropy Pose Information)
Corollary 3.1 states: Let the optimal invariant mapping for a task, [X( R), depend non-trivially on canonical orientation R. Based the distinct informational content of Z outlined above: (a) Standard invariant models (X (maximum effective pose entropy) lack the expressivity to represent f*; and (b) Pose-conditioned models foond R) (provided with zero-entropy pose information) can represent fr
Let G SO(3) be the group of orientations. The optimal mapping f" :. x G - ) (where X' is the space for X and y for the output) has two key properties:
1. Joint Invariance for an Invariant Task: f (" ( for g E
2. Non-trivial Dependence on R: For any given X E X, there exist RG, R,, E G such that R, o Rs but [",X, Ri) & (" R2). This implies that R is an essential input for determining the output of f", not merely a redundant pose of X that could factored out by invariance.
Proof of 1: Standard invariant models finu (X) lack the expressivity to represent f*
A standard G-invariant model fino ) is function fine X' + V. By definition, its output depends solely on X and must satisfy fine (g.X) = ( for all g E G. For any specific input X, E X. fino (Xo) produces single, uniquely determined output value, say Yin_jo-
The model (X) does not take R as an explicit imput. As established by the principle that standard invariant models operate with maximum effective entropy regarding auxiliary pose variable (see Appendix D. Proposition D. (X) cannot vary its output based on different values of for a fixed X. Its output is fixed once X is fixed.
Now, consider the target function (Xo,, RR. According to property (2) of f*, there exist distinct R1, R2 E G such that f (Xo, R2) = Yr and (Xo, R₂) = Yg', where Y & Y.z.
If (X) were represent ("X, R), then for the input Xo, it would need to output YJ when the canonical orientation is R1, and simultaneously output Y2 when the canonical orientation is Rg- However, fano (Xp) can only produce its single output Yime.o- Since Y * Yg Yime_o cannot equal both. Therefore, fino cannot represent the function f (X(,, R) due to its inability to differentiate its output based on R.
Proof of 2: Pose-conditioned models Scona(X, R) can represent f*
A pose-conditioned model [Xona(X, and R as inputs. It is designed to satisfy the same joint G-invariance the target: foond (g,, gR) foond(X (X, R).
X, RR is function fcond X' x G + y. It explicitly takes both X
The target function f*(X, R) is specific function mapping from the domain X' G to y that adheres to this joint G-invariance. We assume that the class of models from which feond( (X, R) is drawn (e.g., sufficiently large neural networks architected respect joint G-invariance) are universal approximators for continuous functions on X' x G that satisfy the given symmetry requirements.
Since Scona(X, R) takes R as an explicit input (representing zero-entropy pose information for that instance, as known and fixed), it has the architectural capacity to make its output depend on R. For given Xo. the model feond(X, (o, R) can learn to produce different outputs for different values of R. Specifically, it can learn to output f (Xo, R1) when its input is R₁, and f (Xo, R2) when its input R is R2.
Given that f",X, R) is a well-defined function of (X, R) satisfying the joint G-invariance, and food R) is a universal approximator for such functions with access to both X and R, foma R) can represent ('(X, R).
D.3 On Symmetry Breaking in ShapeNet Segmentation
Providing explicit canonical pose information R to our segmentation models acts as crucial form of symmetry breaking. This allows models to overcome limitations inherent in standard -o(3)--quivariant networks, fstd_ey(X] which operate solely on an input point cloud X. Such standard models are bound by Curie's 's Principle, meaning their output segmentation must respect
22

---

any symmetries present in X. Consequently, if X possesses exact internal symmetries (e.g., Ca-symmetric table) or even strong approximate or coarse-level symmetries (c.g.. an airplane viewed broadly as cross-shape), fssd ( (X) can struggle to distinguish parts that are (nearly) equivalent under these symmetries but are semantically distinct with respect to a canonical frame (like specific table legs or differentiating "top" from "bottom" on an airplane). These (approximate) symmetries can lead to ambiguities that (X) cannot resolve using only the information in X.
By contrast, a pose-conditioned model, frond eng (x, X, It), receives X alongside its known canonical pose reference R (e.g.. R = I for aligned ShapeNet data). The model's joint SO(3) equivariancce now applies to the effective input pair (X, R). Critically, the specific orientation R typically makes this pair (X, R) maximally asymmetric with respect to SO(3) transformations (i.e., its symmetry group G(x,za) R)) becomes trivial, { even if X itself has notable exact or approximate symmetries. Because the effective input is asymmetric, foonde (X, R) is no longer constrained by X's original symmetries and can produce segmentations that make distinctions based on R. For instance, it can learn to identify the "front-left leg" of a symmetric table or the "upper surface" of an airplane wing, as R provides the necessary external frame to dissolve these ambiguities.
This ability to generate more specific, R-conditioned segmentations by resolving ambiguities arising from X's exact approximate symmetries represents significant expressivity gain, as supported by the logic of 'Corollary: 3.1' (adapted for equivariant outputs). The model fooddeeg eggg [ (X, R) can represent optimal segmentation functions fā (X, R) that depend the canonical frame R. which f cannot. This deterministic disambiguation via R allows the model to overcome symmetry- induced limitations, distinct from, yet achieving a similar practical outcome to, the probabilistic symmetry breaking described by Lawrence et al. [2025b].
D.4 Symmetry Breaking in Generative Modeling of ShapeNet Objects
Generative modeling for ShapeNet aims map a symmetric base distribution pa(z) (e.g., (3)- invariant Gaussian noise) to the target distribution P1 (x) of canonically aligned ShapeNet objects. This target P1 (x) not 53(3)-invarinn inherently preserve the SO(3) -yymmetrry of Po (z) throughout the generation. By Curie's 's Principle, an equivariant map cannot produce a less symmetric output distribution from a more symmetric input distribution. Thus, purely SO (3)--qquivaria denoising function Do L) (or associated score, or flow [Karras et al., 2022]) would generate an (033--nnvarian distribution of shapes, failing to match the aligned ShapeNet data.
Standard -equivaaavaa generative processes would
To resolve this, symmetry must be broken. This is achieved by conditioning the denoising model on explicit pose information R. yielding De (x(xe, f, R). For generating aligned ShapeNet objects, R is fixed canonical pose (e.g.. R I). The model Dx(xe, L, R) is designed for joint SO(3)- equivariance (i.e., De(gx,, t, gR) = t,, R)).
By providing a fixed Rtarget (e.g., R I). the effective input to the jointly equivariant model becomes (x,, Rearyet). This pair can be considered maximally asymmetric (its symmetry group ((x.eee is often trivial, discussed in Appendix D.3). Consequently, the model can map potentially symmetric noise X2 towards samples Xy specifically aligned with Rearget- The fixed pose Rearger breaks the initial (033)-syyyer enabling the generation of the non-SO -invarrrnt target distribution. This use of pose I aligns with the principle from Lawrence et al. [2025b] that auxiliary information can enable equivariant networks produce outputs less symmetric than their primary inputs.
D.5
Generalization Benefits of Joint Invariance in Pose-Conditioned Models
While App. D established the expressivity advantage of condicondii invariant models foond R) over standard invariant models (X), one might consider if sufficiently complex non -onieauuvat model f (X, R), which also takes pose R as input but lacks structural symme- try constraints, could achieve similar performance. Assuming both frond (X, R) (satisfying joint invariance (gX,gRt) f(x. Rt) and Jnez(X It) have sufficient capacity to represent the opti- mal invariant solution f'(X, R), we argue that the equivariant structure of soond provides superior generalization guarantees.
23

---

This argument leverages theoretical results on the generalization benefits derived from incorporating known symmetries, such as Theorem 6.1 presented by Lawrence et al. [2025a], which extends foundational work equivariance and generalization [Flesedy and Zaidi, 2021].
Proposition D.2 (Generalization Advantage of Jointly Invariant Pose- Doss-Conditioned Models). Let X be drawn from a G-invariant distribution P(X), where G SO(3). Let R be the provided pose, treated as an auxiliary variable Z R with the equivariant conditional distribution P(Z = ö(Z R). Consider two models predicting an invariant oulput Y using imputs (X, Rt:
Soona(X, Ry): A model satisfying joint invariance, SoondggX, (igXX)gRR) gl) Scond (X, X,K, R).
"Ineq(X R): Any model using the same inpuis (X, R) that does not satisfy joint invariance.
Under suitable regularity conditions (as specified in Lawrence et al. [2025a], Thm 6.1) and for suitable risk function R(J) (XC R)), Y)] (e.g, using L2 loss), the expected risk of the non-equivariant model is greater than or equal to that of the jointly invariant model:
R(fneg) 2 R(feond)
The difference R(fneg) -ail)-R((F)) R(feomd) represents non-negative generalization gap attributable the component of faeg orthogonal to the space of jointly invariant functions.
Remark 4. This proposition follows follows from applying Theorem 6. I [Lawrence et al., 2025a] to our set- ting. The conditions are met: P(X) G-imvariant (typically assumed or achieved via augmentation). P(ZIX) 8(Z R) is equivariant, and G acts freely on Z = R E &SO33..
Proposition D.2 provides formal support for preferring the pose-conditioned equivariant (jointly invariant) architecture foond( X, R) over an unstructured, non-equivariant model fnep (X, R) that uses the same input information. By incorporating the known relationship between transformations of the input X and the pose R via the joint invariance constraint, the equivariant model leverages useful inductive bias. This bias restricts the hypothesis space to functions consistent with the underlying geometry, thereby reducing variance and improving expected generalization performance compared to less constrained model, even if both models possess sufficient capacity to represent the optimal solution.
E
Additional Results and Discussion
E.1
General experiments
Results with extra channels In this section, present additional results support the hypothesis presented in the paper. We expand ShapeNet3D table in the main paper with additional models from literature in Tab.4 and Tab.5, we show an ablation study on the ShapeNet3D dataset for the part segmentation task as well as the CMU motion capture dataset for the human motion prediction task. Here present results with two different settings of hidden features, C= 256 (gray) and C = 2048. The latter inflated model was trained to match the representation capacity of the rest of the models.
Classification task Modelnet40 dataset ModelNet40 dataset [Wu et al., 2015] contains 9,843 training and 2,468 testing meshed CAD models belonging to 40 categories. We present Modelnet40 classification results in Tab. 6.
Practical implications of the results To get insights for building models on real data, we perform experiments on ModelNet40 on the classification task. Our experiments demonstrate similar trends to those discussed in the main paper for the experiments on ShapeNet3DD, QM9, and CMU motion datasets. For non-equivariant task such as classification, we see that equivariant methods perform best (model 8 in Tab. 6), models with equivariance breaking (model 4 and 15) as well as non- equivariant models (model 19) perform almost equally well, and the performance gap is small.
Scaling experiments To show the effect of adding more training data, we perform ShapeNet3D segmentation for 60%, 80%, 100% of the data. The gap in performance is more for models with effective equivariance T3 as compared to effective equivariance SO(3) and SE(3). See Tab.
E.2 Fixed network capacity comparisons on QM9 generation and regression
This section provides a detailed investigation into the interplay of model capacity, computational efficiency, and the efficacy of different group-equivariant inductive biases on the QM9 molecular
24

---

Table 4: Comparing models with inflated representation capacity against regular models ShapeNet3D for part segmentation task with two different settings of hidden features, C 256 (gray) and C = 2048.
Hapidash with internal SE(3) Equivariance Constraint
Nomalized Egoch Time
Model Variation
Type
Coordinates as Scalass
Coordinates
Normals
Normals
Global Frame
Effective Equivariance
IoU
loUf
as Vactors
as Scalars
as Voctors
(oligned)
(totated)
R2
SEE(3)
80.31
20.00
80.33
80 60.20
10
2
R8
83.87 83.87
83, 9G 10.0e
82 62.09 20.09
49.63
~.
10
3
R3
nane
84 M.01a0.0
32 22.22..... on
10
4
R
84.75
$1.38 10.36
34.07
Rapidach with Internal T3 Equivariance Constraint
85.26
31.41
16
30.38
32.78.0 to
10
17
Ra
88.82 85.82
85. 100.77
92 20..0
38.42.0.05
~e1
10
55.51.0.c0
32.79.0.7a
18
X
none
85-5220.c0
21.11
19
Ra
85 00:00.0
16go.0
33 33.00.20.4
30.62220221
Reference methods from literature
DelaConv PointNeXt GoomGCNN
86 90
89.10
Table 5: Ablation study Human motion prediction task CMU Motion Capture dataset with inflated models and regular models with two different settings of hidden features, C = 256 (gray) and C= 2048. We evaluate models using the mean squared error (x10- metric.
Rapidash with internal SE(3) Equivariance Construint
Model
Type
Coordiaates
Coordinates
Velocity
Velocity Vectors
Global
Effective Equivariance
MSE (rotated)
Normalized Epach
MSE
as
Time
R
SE(3)
100
> 1000
>
2
R²
Tg
4.81
6.4420.12
24 34.40=0.00
25.66
~
20
3
R
nouc
5.93
6.58
>
84.54
20
5.98
no0c
5.53
> 100
19
Rapidash with Internal T3 Equivariance Constraint
15
B
To
6.53
5.99
43.50
100
16
T3
21.32,
>> 100
~ 19
6.03
77.78
6.82
69.12
18
R3
5.49
66.47
Reference mothods from literature
TFN
66.90
EGNN
31.70
CGENN
9.41
tasks, complementing the findings in Section 5. Specifically, compare three model variants that represent a spectrum of constraints on the R3 x 8 base space, keeping their total representational capacity (O C') and therefore training/inference time approximately constant:
1. None (R&3 Fully Unconstrained), Model with no explicit symmetry constraint on the kernel, k(x;, xx)) The kemnel depends on both the receiving point x; and the sending point x,, taking 6-dimensional input, representing the most unconstrained hypothesis space.
2. Ta (Translation Equivariant): Model with translation-equiivriann constraint, k(x,, x,) = k(x, - x.). The kernel depends only the relative position, taking a a- 3-dimensional input.
25

---

Table 6: Ablation study ModelNet40 for classification task using accuracy metric.
Rapidash with internal SE(3) Equivariance Constraint
Model Variation
Type
Coordinates Scalars
Coordinates Vectors
Normals Scalars
Normals
Global Frame
Effective Equivariance
Accuracy (%)
Vectors
Ra
SE(3)
71.04:00.99
R3
T3
86.73.00.12
3
R3
none
86.20
R3
nome
88.68
R x 5
SE(3)
85. 85.14420.28
R 5
X
SE
88.52=
7
R 5
SF(3)
86.0610.55
8
R 5
SE(3)
89.61 1P.6110.29
R 5
X
SO(3)
85.94
10
B 5
X
SO(3)
89.00 $0.09.1..0 10.31
11
A 5
X
SO
85 41 85.41=2.17
12
R X 5
X
SO(3)
89.13 19.13-00.18
13
R
X
T3
86.98-0.60
14
B X 5
X
87 87.25=11.07
15
B X 5
88.88
Rapi dash with Internal T3 Equivariance Constraint
16
R3
X
Tg
87 25.55222112
17
R3
X
Tg
88.51 38.51=200 20.61
18
R3
X
nome
88 $8.47 10.31
19
R3
none
89.02.0.07
Reference methods from literature
PointNet
90.7
PointNct++
93.0
DGCNN
92.6
Table 7: Ablation study on ShapeNet 3D part segmentation reporting instance mean IoU for randomly rotated dataset, comparing validation-set performance when trained different percentages of the training dataset
Rapidarh with internal SE(3) Equivariance Coastraint
Vommalized Epoch
Model Variation
Type
Coordinates
Coondinates
Normalc
Normals Vectors
Global
Effective Equivarianze
IoUl (notated)
[ol]
lol?
Scalars
R
SE(3)
85.45
85.46
85. 15
x
SD(3)
84.48
84.21
84.07
Rapiddash with T3 Equivarlance
R3
Tb
83.00 $3.00
82,50
82.03
3. SE(3) (Roto-translation Equivariant): Model utilizing the R x S base space, where the kernel is conditioned on the three SE(3) invariant attributes, as introduced [Bekkers et al., 2024], and given in Eq. 4.
E.2.1 Molecular Generation: Performance and Efficiency
The primary goal of this experiment is to assess whether the performance gap observed the main text (where SE(3) models are constrained by fewer parameters) can be closed by giving less constrained models a generous training budget and sealing up their capacity to match the computational complexity of the SE(3) models.
Performance Analysis (Fig. 7) The results confirm that equivariance critical inductive bias for the molecular generation task. The unconstrained "None" models consistently failed to produce sensible molecules despite receiving ample training time and parameters, suggesting that lack of geometric constraint leads to catastrophic performance for this task. The T3 models performed results on QM9 generation. However, peak
remarkably well, even surpassing previous state-of-ther performance is still decisively reached through the SE(3) inductive bias. This suggests that even when capacity and training time are scaled liberally for the simpler Ta model, the geometric constraint provided by SE(3) equivariance allows for significantly better generalization and performance.
26

---

Molecule Stability after 48 hours of training
None
Tn
100.00
92.66
93.59
92.95
94.99
96.00
89.24
90.00
85.90
86.40
80.00
70.37
70.00
66.29
58.19
52.38
50.00
258
512
1024
2048
4096
47.51
Total hidden dimension
40.38
Figure 7: Comparison of generative performance (Molecule Stability %) on an equal training budget (48 hours, same GPU). All models leverage positional information. The labels "None", "T3", and "SE(3)" denote equivariance to the trivial, translation, and roto-translation group, respectively.
This result shows that increasing model scale does not eliminate the performance advantage of well-aligned SE(3) inductive bias.
QM9 Generation: Molecules per second
None
Tn
SEn
100
90:788.55
82.582.00
75
70.2
28
66.82
67.367.21
58.77
45.645.964.4
50
29.03
24.724.67
25
256
512
1024
2048
4096
Total hidden dimension
Figure 8: Comparison of test-time generation speed, reported as the number of molecules generated per second. Molecules were sampled in batches of 128 over 50 denoising steps.
Efficiency Analysis (Fig. 8) The generation speed linearly decreases as the hidden dimension increases across all models. For any given hidden dimension, the speed is approximately constant regardless of the inductive bias used (None, T3, SE(3)). This indicates that the SE(3) constraint does impose significant or prohibitive computational overhead in the inference phase of the generative model compared to the less constrained alternatives. At the highest hidden dimension
27

---

tested (4096), the SE(3) model is only marginally faster (approximately 17.7%), but its moderate efficiency is vastly outweighed by the performance gain over the other models.
E.2.2 Molecular Property Prediction: Performance and Efficiency
For the property prediction task (regression), we use the Mean Absolute Error (MAE) on the property (dipole moment) of QM9.
QM9 Regression: MAE on mu target after 12 hours of training
0.125
0.100
0.092
0.075
0.050
0.040
0.041
0.038
0.033
0.022
0.022
0.024
0.025
0.017
0.015
0.013
0:013
0.015
0.000
256
512
1024
2048
4096
Total hidden dimension
Figure 9: Comparison of regression performance (MAE on μı) on an equal training budget (12 hours, same GPU). The labels "None", "T3", and "SE(3)" denote the level of symmetry constraint.
Performance Analysis (Fig, 9) The performance hierarchy clear and stable across all tested hidden dimensions: SB(3) >> T3 > None.
The SE(3) models are consistently the best performing and remain stable as capacity scales.
The critical observation is that the increased network capacity in the "None" and T3 models cannot compensate for the lack of a strong geometric inductive bias. In fact, their perfor- mance tends to degrade remain stagnant with increasing hidden dimensions, suggesting they reach the limits of their geometrically defined hypothesis space and potentially struggle with optimization or overfitting in the more expressive space.
This strongly supports the argument that the architectural choice of an appropriate inductive bias is more critical than simply scaling capacity.
Efficiency Analysis (Fig. 10) Similar the generative task, the prediction speed decreases linearly with the hidden dimension. The computational cost across all three models for given hidden dimension within the same order of magnitude. The SE(3) model only marginally slower (approx. 19.8% at the lowest hidden dimension and 19.3% faster at the highest). Considering the significant performance gap (SE(3) achieving much lower MAE), the moderute cost of the SE(3) inductive bias is justified by the large generalization guin. It is safe to conclude that the efficiency differences do not render any model variant computationally prohibitive compared to the others.
Implementation details
We implemented our models using Py PyTorch [Paszke et al., 2019], utilizing message passing and graph operations modules [Fey and Lenssen, 2019], and employed Weights and Biases for experiment tracking and logging. A pool of GPUs, including A 100,, A6000, A5000, and 1080 Ti, was utilized as computational units. To ensure consistent performance across experiments, computation times were carefully calibrated, maintaining GPU homogencity throughout.
s
28

---

QM9 regression: Samples per second
None
Tn
100.00
93.991.11
85.(85.38
73.03
75.00
68. 73
69.769.50
60.82
47.147.146.45
50.00
30.13
25.225.27
25.00
0.00
256
512
1024
2048
4096
Total hidden dimension
Comparison of test-time prediction speed, reported as the number of predictions (per sample) made per second. Predictions were made with batch size of 128.
Figure 10:
For all experiments, we use Rapidash with layers with fiber dimensions for R and 0 or 8 fiber dimensions for R S². The polynomial degree set to 2. We used the Adam optimizer [Kingma and Ba, 2014], with a Iearning rate le - 4, and with a CosineAnnealing learning rate schedule with warm-up period of 20 epochs.
All models for equivariant tasks are trained with SO(3) augmentation. For ShapeNet segmentation, we use standard augmentation (20 degree z-rotations + scaling), full SO(3) augmentation hurt in-distribution performance.
QM9 QM9 dataset Ramakrishnan R. [2014] contains up to 9 heavy atoms and 29 atoms, including hydrogens. We use the train/val/test partitions introduced Gilmer et al. [2017], which consists of 100K/18K/13K samples respectively for each partition.
ShapeNet 3D Shape 3D dataset [Chang et al., 2015] is used for part segmentation and generation tasks. ShapeNet consists of 16,881 shapes from 16 categories. Fach shape annotated with up six parts, totaling 50 parts. We use the point sampling of 2,048 points and the train/validation/test split from [Qi al., 2017]. For this task, trained we trained model variation (1-4 & 16-19) in Tab.5 and Tab.3 with two different settings of hidden features, C 256 (gray) and C = 2048. The later inflated model was trained to match representation capacity the rest of the models. For the segmentation task, we use rotated samples and compute IoU with aligned and rotated samples. All the models were trained for 500 epochs with learning rate of 5e 3 and weight decay of le 8.
For the segmentation task, we trained we trained model variation (1,2 & 6,7) in Tab.3 with two different settings of hidden features C C= 256 (gray) and C 2048. The latter inflated model was trained to match the representation capacity of the rest of the models. All the models were trained for 500 epochs with a learning rate of 5e 3 and weight decay of -
CMU Motion Prediction For the motion prediction task, we evaluate our models on the CMU Human Motion Capture dataset [Gross and Shi, 2001], consisting of 31 equally connected nodes, each representing a specific position on the human body during walking. Given node positions at a random frame, the objective is to predict node positions after 30 timesteps. As per Huang et al. [2022] we use the data of the 35th human subject for the experiment. See Figure 11 to see instance of the dataset. For this task, we trained model variation (1-4 & 16-19) in Tab. with two different settings of hidden features, C 256 (gray) and C 2048. The latter inflated model was trained to

---

40
30
40
20
30
30
Trames
Target
Predictin
Figure 11: Depiction of instance from CMU motion capture dataset.
match the representation capacity of the models (5-15). All the models were trained for 1000 epochs with learning rate 5e 3 and weight decay of 1e 8.
ModelNet40 For this task we trained model variations presented in Tab. All the models were trained for hidden features C = 256 for 120 epochs with learning rate le and weight decay of
le
Diffusion Model Unlike EDM Hoogeboom et al. [2022] that uses a DDPM-like diffusion model with a deterministic sampler, we use a stochastic sampler proposed in Karras et al. [2022]. We condition the diffusson model with feature scaling and noise scaling and combine outputs with skip connections, allowing for faster sampling. The sampler used in this work implements a stochastic differential equation with second-order connection.
Geometric task complexity Classficcatinn/Rgrression does not change with group transformation of the input, thus invariant to the transformation. Segmentation with/without global frame is invariant/cquivariant task. Dynamics (It varies with the actual dynamics (c.g. can be a simple motion equation fluid dynamics with instability terms and inputs (positions, velocity, additional parameters, thresholds etc), generally an equivariant task, and molecular generation is an equivariant task, often with certain symmetry breaking form different structures. Here is figure depicting geometric task complexity for the tasks covered in the paper.
dencise
dituue
Classification Regression
Segmentation
Dynamics
Generation
Figure 12: Progression of tasks based on geometric complexities of each task.
30

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
31

---

Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We also provide the code and the config files to reproduce all the results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
32

---

Answer: [Yes]
Justification:
The code is added in the supplementary material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/ Seaiiiicguieess/Coeeee
icyy for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: /Codeeeooiissseeeeee / //iips.coo. wttpccccpppccceeeecce..cccccccct /ute:////egiieeeeeeuuuuuiiiiii
cetpaubmmsssoonnii wswinnoolll onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
33

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that NeurIPS Code of Ethics https: : //wuurrrrs....// cc hc/pppllitt/thhiccuuddeee...
didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
34

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All the datasets used have been cited with their respective papers.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi sepprssiithood thcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
35

---

Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
36

---

Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https: http:/uuiip..cee/eeeeeeeeeeeeeee /0tpa////.//oooaaeeraa/////////0 for what should or should not be described.
37


Reviews:
[{"review_id": "UyftUkxeRm", "reviewer": "Reviewer_Bpqw", "summary": "This paper investigates the tradeoffs of utilizing various types of equivariant and unconstrained models for learning a broad range of 3D tasks. Due to the nature of the 3D tasks, the authors focus on SE(3) symmetries and introduce a scalable group convolutional architecture (called Rapidash) that allows them to control the level of equivariance and symmetry breaking applied to the models in a unified way. Using Rapidash, the paper investigates how different levels of equivariant constraints affect the performance of the models, how the size of the models impacts the various equivariant architectures, and how the performance of these architectures scales with different dataset sizes. Additionally, the authors investigate how applying a symmetry-breaking mechanism can affect the performance of the models. After defining the specific questions that they aim to answer, they perform an extensive evaluation of the different options on various 3D tasks and provide an in-depth discussion interpreting their empirical observations.", "questions": "- What is the role of augmentations when investigating the benefit of using equivariant models? Are augmentations and equivariance exclusive of each other or complementary? Additionally, when combined with the information-theoretic arguments regarding the benefits of symmetry breaking, what is the role of augmentations there?\n- Is it reasonable to expect that these results and conclusions generalize to different types of groups? What would be some necessary conditions for these results to generalize?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The topic of the paper is very timely, as understanding the benefits and the tradeoffs of utilizing equivariant networks is something that the geometric deep learning community is currently actively interested in. Additionally, the authors do a good job in the introduction of the paper of providing readers with the context surrounding the research questions this work aims to answer.\n- The proposed architecture is very flexible and can be of interest to the community even beyond the investigation performed in this work.\n- The experimental evaluation is extensive, showing how the different combinations of equivariant tasks and symmetry-breaking mechanisms can affect performance. This can be of great use to practitioners who want to decide the degrees of symmetries they want to apply to their specific task.\n\nWeaknesses:\n- While the evaluation of different degrees of equivariance is extensive, there is no discussion or comparison with the use of data augmentation, which is one of the most common ways of incorporating known symmetries into the learning process.\n- The paper focuses on symmetries that are subgroups of SE(3). It is not clear if these conclusions generalize to symmetries of groups that are fundamentally different than SE(3).", "was_revised": true, "final_justification": "This work provides an extensive investigation of the effect of equivariance which, although focused on point cloud processing tasks, offers valuable insights to the community. Additionally, during the rebuttal, the authors addressed all of the concerns I raised in my initial review. For these reasons, I maintain my rating of accept.", "rebuttal": "Dear reviewer, We thank you for the time and effort you have put into providing a review of our work. We sincerely thank you for your thoughtful and encouraging feedback. We are glad that you found the topic of the paper to be **timely and relevant** to ongoing discussions in the geometric deep learning community, particularly regarding the understanding of the benefits and trade-offs of equivariant networks. Your comments on the *flexibility* of the proposed architecture are greatly valued. We appreciate that you found our experimental evaluation **extensive**. \n\nIn the text below, we try to address your concerns and respond to the questions:\n\n##  Data Augmentation [W1]\n\nWe would like to emphasise that all model variations for tasks that are supposedly equivariant **have rotation augmentations** for training- we will explicitly mention this in the experiments section.\n\n## Equivariance beyond E3/SE3 [W2]\n\nThe reviewer raises an important point about understanding the effects of equivariance beyond SE(3) and its subgroups. We would like to first justify our choices made in the paper by mentioning that we focus on standard point clouds in the real world that have mainly E(3) symmetries and permutation symmetries. Hence, we primarily end up focusing on E(3)/SE(3) and its subgroups. \n\n## Response to Q1: Role of augmentation\n\nGenerally, models without equivariant layers are trained with augmentations. So, we train with SO(3) augmentation for all models on tasks that are equivariant. For ShapeNet segmentation, we use standard augmentation (20° z-rotations + scaling), as full SO(3) augmentation hurts in-distribution performance. This helps us understand if models improve performance with augmentation, and they do. (*This effect is less in equivariant models than in non-equivariant models.*) We don’t see equivariance and augmentations as exclusive but complementary. With regards to symmetry breaking and augmentations, the non-equivariant method generally benefits from augmentations.\n\n## Response to Q2: Generalization to other groups\n\nSee W2. The reviewer raises an important point about the effects of equivariance and symmetry breaking for groups apart from SE(3) and its subgroups. We believe this is an important question that we aim to address in future work. As far as the usefulness of equivariance goes, as long as the task is equivariant, having equivariant layers is an effective way to learn symmetries for any group. Although the performance gap between non-equivariant and different equivariant groups depends on the **complexity** of the group [1]. \n\nOnce again, we thank the reviewer for the insightful feedback. If any concerns remain from your initial evaluation, we’d be happy to clarify further.\n\nReferences:\n\n[1] *On the hardness of learning under symmetries*, Kiani et al."}, {"review_id": "wa5fHuEe64", "reviewer": "Reviewer_mQdo", "summary": "The paper introduces a unified architecture, Rapidash, which unifies non-equivariant and equivariant models, and assess the effect of models with different structural priors, including forms of symmetry/equivariance breaking, to different tasks.", "questions": "Symmetries reduce the hypothesis space to functions that respect the symmetry (hence potentially limiting train fit capacity), but allow better generalization (test data fit) if the symmetries match the underlying data distribution, with symmetry breaking/relaxations being something in between.\n\nQ1: Given the relevance of distinguishing train/test data, would it be possible to report both train and test loss. Currently it is not always clear whether MSE is on train or test data (assuming it’s the test metric for CMU), but it would be helpful if both can be provided? Adding these as an Appendix should be enough.\n\nQ2: Related to this, results of the paper are particularly strong because the representational capacity is taken into account, ensuring that symmetric models do not merely generalize better due to undercapacity. The paper mentions that saturating performance suggests that ‘the limits of their architecturally defined hypothesis space for geometric expressivity’. Although true, it seems this could be easily verified by checking if models achieve low train error.", "limitations": "- The paper is very well-written. The main point of critique is the limited assessment of the train/test error difference, which seems highly relevant given that the paper focuses on measuring generalization capabilities in the context of structural geometric priors. \n- Secondly, the evaluation and interpretation of the results could be clearer and more precise. The paper presents results in large tables, which would benefit from additional exposition and discussion.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The paper is well-written, provides an extensive study that ranges from theoretical analysis, empirical analysis, to practical architecture and empirical results on well-known data sets. The scope of this evaluation strengthens conclusions that are drawn.\n- The paper builds upon a solid geometric framework established in prior work.\n- Proposes a concrete architecture, Rapidash, that is useful to better understand and empirically evaluate geometric priors in the context of deep learning.\n- Insightful results and findings that benefit the community to better understand how models with structural geometric priors can benefit practical applications. In particular, the work offers important insights in the effect of incorporating symmetry breaking or relaxations have on predictive performance\n\nWeaknesses:\n- Although the analysis is quite extensive, one minor point of critique is that it does not seem to consider the difference between training and test error as particularly relevant. However, this distinction is crucial for understanding the difference between fitting (symmetry-restricted hypotheses) and the ability to generalize (having the right inductive bias).\n- The large result tables can be daunting at times and might benefit from clearer explanations in the text. In particular, it would be helpful to explicitly state the expected generalization behavior and explain how the results align with, or deviate from, those expectations. I recognize this may be a matter of personal preference and could also be constrained by page limits, so I won’t weigh it heavily in my overall assessment.", "was_revised": true, "final_justification": "I am satisfied with the answers by the authors and maintain my recommendation for acceptance.", "rebuttal": "Dear reviewer, we thank you for the time and effort you have put into providing a review of our work. \n\nFirstly, we sincerely thank you for your kind words and appreciation of our work. We are grateful that you found the paper to be **well-written** and recognized the extensive study spanning **theoretical analysis, empirical evaluation, and practical architecture** . We especially appreciate your recognition of our proposed architecture, Rapidash, as a *concrete and useful* contribution for understanding geometric priors in deep learning. We hope the community finds our work *insightful* as well.\n\nIn the text below, we address questions and weaknesses/limitations.\n\n## Train error  vs test error [W1, L1]\n\nWe would like to mention that for all tasks, we train with CosineAnnealing and select the best checkpoint based on validation performance. For ShapeNet, which lacks a standard validation set, we use the final model; no overfitting was observed. As we consider the validation error to decide the best model, we did not report the val (nor train) error, but only the error on the designated test set.\n\n## Readability of large tables/ Takeaways [W2, L2]\n\nWe appreciate your feedback regarding the readability of large tables. In the revised version, we will:\n\n- Add **split** (smaller) versions of the large tables in the Appendix for improved readability.\n\n- Include clearer **takeaway messages** *(as shown below)* in the discussion section to enhance accessibility:\n  * The performance gap between equivariant and non-equivariant models increases with geometric task complexity.\n  *For simpler tasks like classification and regression, *using* non-equivariant models performs somewhat similarly to equivariant models.\n  * Symmetry breaking, for example, adding positional information, can be useful and give a performance boost.\n  * For equivariant tasks like molecular generation, using equivariant models is important, as non-equivariant models perform poorly.\n\n## Response to Q1\n\nFor the CMU dataset, we report **test MSE loss** and use this to compare models. We will add a table with train and test errors for the CMU dataset in the Appendix in the updated version. \n\n## Response to Q2\n\nWe mention ‘*the limits of their architecturally defined hypothesis space for geometric expressivity’* with respect to Table 3 for inflated models vs regular models segmentation task on ShapeNet (both rotated, non-rotated samples). We indeed see that making the models larger *does not improve* test error and will report that in the updated version of the paper. \n\nOnce again, we thank the reviewer for the insightful feedback. If any concerns remain from your initial evaluation, we’d be happy to clarify further."}, {"review_id": "fHbchpCX0G", "reviewer": "Reviewer_KUZW", "summary": "While equivariant kernels ensure generalization to specified groups, this comes at the cost of potentially over constrained networks on imperfectly symmetric datasets, generally lower network capacity, and higher compute requirements for a fixed parameter count. This has led to several papers that investigate the utility of equivariance by design on disparate tasks and datasets, some of which find benefits, whereas others do not. As a result, the literature currently conflicts with itself in terms of relevance to real world datasets and a unified analysis is needed. \n\nThis submission specifically pertains to deep learning on point clouds and provides a group-equivariant framework that enables isolating various key decisions (e.g., input feature types, equivariance constraints, etc.) such that they can be studied piece-by-piece. The paper also presents some new theoretical results to clarify when and how symmetry breaking and equivariance are useful depending on the task. Lastly, experimentally, it covers a broad set of tasks and domains on toy small-scale datasets such as ShapeNet, QM9, etc.", "questions": "Please correct me if I misunderstood any central points and I would be happy to reconsider my rating. Currently, I would very likely raise my score if:\n- The paper clearly revises itself to narrow its scope to point cloud analysis and to describe which of its findings have the potential to translate across fields.\n- The empirical investigations are expanded to include at least one large-scale real-world dataset with *well tuned* baselines used as a point of comparison (with extensive augmentation and/or regularization). It is understandable if this is not doable within the time allocated to the rebuttal, but in that case, please provide a rigorous discussion regarding the choice of datasets and what we can extrapolate to real data.\n- It is clearly disambiguated which of the theoretical results are truly specific to this paper and which are inherited from previous work.", "limitations": "The discussion section includes 2 sentences on being compute bound in terms of running larger scale experiments and mentions limiting analysis to convolutional architectures. While I don't think that these are the primary limitations of this work, these are thoughtful inclusions that should be appreciated.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "## Strengths:\n- The problem space of \"*does equivariance by design matter*\" is gigantic in scope and this paper does an admirable job in trying to construct a unifying framework for empirical investigation across a wide set of tasks. \n- Section 2, background is very well done and accessible to a broad audience.\n- New theoretical results on when breaking equivariance and symmetry help in generalization (and when they do not).\n\n## Weaknesses:\n### Over-claimed scope and toy experiments:\nThe paper claims more within its scope than it actually does. The paper primarily pertains to point clouds in all of its empirical investigations (which are its core contribution), yet the framing is left as general-purpose as possible, which IMO can be misleading. As an example, [Gruver&Finzi](https://openreview.net/forum?id=JL7Va5Vy15J) show that many of the statements made here do not translate to large-scale natural image-based vision where large networks without inductive bias trained on large datasets with extensive augmentation are practically *more equivariant* than equivariant-by-design architectures. This paper should revise itself to clarify its scope and which of its findings have the potential to translate across fields.\n\nMore broadly, the paper does not get to the true tension of the field: if equivariant architectures worked at scale across tasks, they would be used at scale across tasks by the community. There are some theoretical results provided, but all of the experiments are performed on simple toy datasets such as ShapeNet and QM9 wherein it is very easy to show great results that then do not translate to real tasks. I believe that the theoretical results should be validated on at least one large-scale real-world dataset with *well tuned* baselines used as a point of comparison.\n\n### Unclear contributions:\nIt is hard to say definitively what to take away from the paper. The theoretical framework, Rapidash, is a multiscale extension of Ponita in [Bekkers, 2024](https://arxiv.org/pdf/2310.02970) (I suppose this naming scheme makes sense if you have a pokedex :), but this is never explicitly stated in the paper and it is claimed as an entirely new framework. However, many of its theoretical properties are inherited from Ponita. As a result, a significant portion of the methods section recaps properties that are originally specific to Ponita (e.g., Sec 3.1) and it is hard to isolate what is truly specific to Rapidash. Also, this raises the burden of empirical completeness to make a strong contribution, in my opinion.\n\n### Missing analysis of augmentation and regularization:\nThe paper's analysis misses key components of performance such as augmentation types and strengths and regularization via techniques such as weight decay. Augmentation is not mentioned within the paper and in practice is key to networks learning equivariance in practice ([Gruver&Finzi](https://openreview.net/forum?id=JL7Va5Vy15J)). It is plausible that the corresponding non-equivariant networks require significantly more regularization and/or augmentation strength (with more training steps) to lower-the-margin-to or even exceed their equivariant counterparts on these toy point cloud datasets.\n\nPlease detail the types, strengths, and probabilities of the augmentations and regularizations used, and whether they were held identical for both equivariant and non-equivariant networks. If they were, please perform a few experiments where the non-equivariant networks are well tuned.\n\n### Unclear relationship between the research questions\nThe research questions posed in the methods section come off as a bag of loosely-related topics that are of independent interest within the field, each of which could be its own paper. For example, the transition into Section 3.2 is very abrupt as it goes into detail about a paper by Lawrence et al 2025 proposing a probabilistic interpretation of symmetry breaking that is (a) previously unintroduced (b) hard to parse and translate to the context of this paper without being very familiar with Lawrence, et al.\n\nIn the revision, please make explicit what ties these research questions together and provide necessary background in Section 2 instead of in the Methods, which should primarily cover what is new and/or specific to this paper.\n\n### Presentation: \n- The actual methodological contribution (Rapidash) is only described in the appendix, but the main text of the paper writes at length about its properties. It is hard to follow if you don't know what the method actually is. The description should be moved into the main text and a lot of space can be made by removing unnecessary portions including L65--68 and L233--245 (which just restates the intro paragraph).\n- The paper's opening paragraph doesn't contextualize the state of the field well. It comes off as just listing papers without outlining cohesive themes across papers. The latter would be much more accessible to readers who are not following the monthly back-and-forths on this contentious topic.\n\n## Minor comments that do not impact my rating:\n- Related work of interest: In the interest of unification, Gruver & Finzi 2023 should absolutely be discussed in this paper and how its findings on images could relate to the findings on point clouds here. Not essential but potentially interesting in the context of discussing Rapidash, there are works on position-orientation deep learning in medical imaging (e.g., in vessel analysis and diffusion MRI) that could be worth briefly discussing for better contextualization.\n- L20: Please clarify that it's AlphaFold3 and not other versions of AF that did use equivariant architectures.\n- Fig2: The radar plot + seemingly random text snippets make the figure very hard to parse. Please split into two separate subfigures.\n- L218: Weight sharing in CNNs was introduced quite a bit earlier than LeCun2010 :) \n- The connections made to tensor field networks can be left at a sentence with the rest moved to the appendix. There's already too much packed into the main text of the paper as is.", "was_revised": true, "final_justification": "The rebuttal addressed many of the concerns raised in my original review and therefore I lean towards accepting the paper. My primary remaining reservations correspond to some of the unaddressed topics, including:\n1. Making broad statements regarding the practical utility of equivariant networks on point clouds based on empirical results from small-scale (and some synthetic) datasets alone. \n2. The paper requires a significant revision to reframe itself as specific to point cloud analysis and better tying the research questions together.\n\nHowever, the authors have committed to addressing 1 and 2 in the final revision. On net balance, the paper is a useful data point in this long-running debate regarding inductive biases vs. scale for atleast the specific domain of point clouds.", "rebuttal": "Dear reviewer, \n\nWe thank you for the time and effort you have put into providing a review of our work. We sincerely appreciate your thoughtful feedback. We are grateful that you find our work an **admirable job** in constructing a unifying framework. We appreciate your kind words about the background as well as the **theoretical results**.\n\nIn the text below, we try to address your concerns. We appreciate the careful suggestions that have helped improve our work and will make the changes in the updated version.\n\n## Scope of the paper\n\n- We agree with the reviewer that the focus of **our work is on point clouds**, and we will **emphasize** this in the introduction as well as in the contribution list. We will revise the wording in the paper to emphasize the point clouds aspect. Thank you for pointing us to [1], and will add this to the related work/discussion to share how some insights do not translate across fields, i.e, images to point clouds.\n- In the scope of this paper, we *do not* tackle **regularization**. We will add this to our limitations and consider this in future works to understand the role of regularization in equivariance learning.\n- We would like to point out that Rapidash architecture, unlike PONITA [1], allows for processing a larger number of points per sample. Due to various **different input options**, Rapidash is a flexible architecture that allows for detailed empirical analysis of different group equivariance as well as symmetry/equivariance breaking. In the updated version of the paper, we will add how Rapidash differs from [1] explicitly.\n\n## Connecting research questions presented in the paper\n\nThe **five core research questions** in our work focus on kernel constraints, representation capacity, dataset size, symmetry, and equivariance breaking. We think these are crucial when it comes to understanding the role of equivariance.\n\n- RQ1 addresses the role of constraining the kernel and where that helps or hurts model performance. \n\n- RQ2 and RQ3 help understand if increased model size or increased dataset size can help learn equivariance. \n\n- RQ4 and RQ5 connect to understand how to incorporate more information in data, especially if that leads to symmetry breaking or equivariance breaking, and which of these can be useful for better performance. \n\n## Presentation\n\n- We fully agree with the reviewer that the detail of Rapidash should be in the main text. Due to lack of space, the **details of the Rapidash architecture** are only discussed in the Appendix. We will incorporate the suggestion and add it to the main text of the paper.\n- We will **rewrite** parts of the introduction to better describe the related works in a way that is more focused on the theme of the work.\n- We will also provide the **necessary background** in Section 2 to help understand both the theoretical contribution and research questions cohesively.\n\n## Clarification on theoretical contributions\n\nProposition 3.2 [main text], Corollary 3.1 [main text], Proposition D.1 [Appendix], and Proposition D.2 [Appendix] are the theoretical contributions in this work, while the rest of the propositions/corollaries are presented from earlier works for completeness. \n\n## Choice of datasets\n\nWith the focus of the paper being on point clouds, we picked the most commonly benchmarked small molecule dataset **QM9** (with **generation/property prediction** task). For adding a different task, learning dynamics, we picked **CMU Motion** dataset. Given that it is a very small dataset, we wanted to understand the role of equivariance in a small data regime. For the rest, we pick the commonly used standard pointcloud datasets, i.e., **ShapeNet3D** (for **segmentation and generation** tasks) and **ModelNet40** for **classification**. These datasets collectively give a **good range** of tasks along with additional inputs *(i.e. global pose, global frame, etc )* allowing for different types of symmetry/equivariance breaking on point cloud datasets of varied size. \n\n \n\n## Regarding large-scale datasets\n\nIn response to the point raised regarding large-scale datasets, we are working towards including the **COSMO-bench** dataset [1] (comprising of 5000 point clouds, *Quijote, coarse-grained* per sample as well as a physics baseline in addition to a GNN baseline for velocity prediction tasks for ease of comparison). We are working on having *Rapidash + variants* for prediction tasks and will add the results in the updated version of the paper. If the reviewer has other suggestions, kindly let us know.\n\n## Response to minor comments\n\n- We will **update our discussion** to include [2] as well as other works in the position-orientation space.\n- We will update the AlphaFold version in the introduction and weight-sharing **citation**.\n- With regards to the figure, we will add the **split version** of Figure 2 to the Appendix.\n\n\nWe once again thank you for taking the time to review our work, and we would appreciate your reconsideration of the evaluation in light of these clarifications.  If any concerns remain from your initial evaluation, we’d be happy to clarify further. \n\nReferences:\n\n[1] *Fast, Expressive SE(n) Equivariant Networks through Weight-Sharing in Position-Orientation Space*, Bekkers et al.\n\n[2] *The Lie Derivative for Measuring Learned Equivariance*, Gruver & Finzi et al.\n\n[3] *CosmoBench: A Multiscale, Multiview, Multitask Cosmology Benchmark for Geometric Deep Learning*, Huang et al."}, {"review_id": "4MQVkiVVUj", "reviewer": "Reviewer_V9ah", "summary": "This paper introduces an architecture with flexible symmetry parametrisation based on regular group convolutions on R^3xS_2.\n\nThe paper then investigates incorporating various levels of symmetry when solving a number of tasks that may benefit from it.\n\nThe paper draws a number of conclusions about the importance of symmetry in this architecture, data scaling, model capacity scaling, and symmetry breaking.", "questions": "1. Do the authors think that adding data augmentation to the non-equivariant or reduced equivariant architectures would make a difference to the results? If so, should this be included in the ablation studies?\n2. Do the author expect these conclusion to hold across other types of equivariant architectures? Open source code exist for a number of other equivariant architectures, e.g. https://github.com/QUVA-Lab/escnn, https://github.com/vgsatorras/egnn, which could easily be adapted to the reduced symmetry and symmetry breaking settings of the paper, should comparisons to these alterative architeture types be included to make the conclusions of the paper more substantial?", "limitations": "Yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- The evaluation if reasonably well conceived, although the mix of tasks could be wider to disambiguate effects over multiple datasets to disentangle and dataset specific effects.\n- The layout of hypotheses are clear\n\nWeaknesses:\n- The authors seem to miss a key question regarding equivariant models in their analysis, eluded to in the introduction. Namely, that when equivariance is removed  from an architecture it is typically replaced with data augmentation. Without this inclusion I find the conclusions of  RQ1-3 tricky to conclude properly.\n- The authors only look at a single architecture. The conclusions seem unlikely to hold across all classes of model to me. In particular, with irregular representation convolutions vs regular representation convolutions, the additional complexity in parametrisation and runtime cost seem likely to significantly change the results of the expreiments.", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer, we sincerely thank you for the time and effort you put into providing feedback on our work. We deeply appreciate your insightful questions and respond to the concerns and questions below:\n\n## Data augmentation [W1]\n\nThank you for pointing out that we do not emphasize in the main paper that the experiments are performed with augmentations.  We will update the main paper with details on augmentations. **All models for equivariant tasks are trained with SO(3) augmentation**. For ShapeNet segmentation, we use standard augmentation (20° z-rotations + scaling), as full SO(3) augmentation hurts in-distribution performance. In the code added as supplementary material, we have config files with the augmentation parameter (set as true) for equivariant tasks, which can be used to easily reproduce the results in the paper. \n\n## Alternate architectures [W2]\n\nGenerally, for this work, we refrain from further comparison with additional model classes, as we believe our findings about equivariance effects should, *in principle, generalize across model classes since our ablations isolate **fundamental properties of *equivariance constraints***.  Only within a single architectural class can we do proper ablation studies, and thus, we can attribute performance gains/drops to changes in equivariance constraints. If we want to explore whether or not the results generalize to other architectural classes, we have to develop such a new class of models in addition to our already proposed flexible architecture (Rapidash) and redo the entire study (which includes **58 model variations across 5 tasks**). We consider this to be beyond the scope of the current paper and decide to emphasize that these results hold within the convolutional neural network class (both in $\\mathbb{R}^3$ and $\\mathbb{R}^3 \\times S^2$) paradigm. Related works like [1], answer a few questions about the role of equivariance with a transformer-based model and share similar trends as our findings in scaling models for equivariant tasks. \n\n\n\n## Response to Q1: Augmentations\n\nYes, the **presented results are with augmentations**.  We will add results without augmentations in the Appendix in our updated version. Also see W1.\n\n## Response to Q2: Alternate architectures\n\nRelated works like [1] answer a few questions about the role of equivariance with a transformer-based analysis and share **similar trends** as our results in scaling models for equivariant tasks. We believe the trends of our findings hold true across the mentioned architectures, ESCNN and EGNN, although it can be said with certainty only with a **detailed analysis** of each model class. Having extensive analysis for each available equivariant architecture/class would be very useful for the community.  We leave that as future work.\n\nWe once again thank you for taking the time to review our work. We believe the results in this paper offer relevant insights for both theory and practice *in convolutional architectures*, and we would appreciate your reconsideration of the evaluation in light of these clarifications.  If any concerns remain from your initial evaluation, we’d be happy to clarify further. \n\nReference:\n\n[1] *Does equivariance matter at scale?*, Brehmer et al."}]

Author rebuttal:

Meta-review: [Summary]

Based on the initial reviews as well as the reviewer-author discussions, my understanding of this paper's major contribution is two-fold: (i) introducing an architecture that unifies equivariant and non-equivariant models; (ii) (based on the proposed architecture,) investigating how different levels of equivariance affect model performances on various aspects.


[Strengths]

- The paper addresses an important research question in the field (timely, and large in scope).
- The paper does a good job studying this important topic, conducting extensive experiments and showing insightful analysis.
- In addition to the studies, the paper also proposes a practical framework unifying equivariant and non-equivariant models.
- The paper is well-written (in its background introduction, technical parts, and results presentation).


[Weaknesses]

It seems that the major concerns from the reviewers are not in the model design, but mostly in the experimental analysis/discussion part. Some major concerns are:
- at what scope can the statements/conclusions be drawn from the results;
- whether the key questions are properly asked/answered;
- whether the experimental designs and evaluation setups are solid enough (mostly on some details);
- and presentation of the experimental results.


[Reasons for decision]

The paper received two firm acceptances (Bpqw and mQdo), one borderline acceptance (KUZW, with remaining concerns in writing), and one borderline rejection (V9ah) in the end.

In my opinion, V9ah didn't show enough opinions or participation in the discussion to strongly suggest rejection. And given the following reasons:
- (i) the two initial acceptance reviews are firm;
- (ii) KUZW overweights the paper's contribution to the field's long-term discussions than its weaknesses;
- (iii) although there're remaining concerns on the writing (mostly on how to phrase the statements), the paper is clearly written in its technical parts (as the other reviewers commented this papaer as "well-written", and KUZW also mentioned that the background introduction was accessible to a broader audience); and the authors have committed to addressing KUZW's remaining concerns in writing in their final revision;
- (iv) for an experimental study paper on a long-existing open problem in the field, it may not be easy to answer all questions properly in one work (under a certain set of experimental designs), and I feel the remaining unclear questions are in an acceptable range;
I would like to recommend acceptance. But I hope the authors could further improve their writing based on the reviews in their final revision.


[Discussion summary]

Initially, the paper received diverging scores, with the acceptance opinions slightly stronger than the rejection opinions -- two reviewers (Bpqw and mQdo) gave accepts, and two reviewers (KUZW and V9ah) gave borderline rejects.

There weren't too many discussions. Some major discussions were with KUZW, where (i) most of KUZW's concerns were addressed, (ii) the remaining concerns were mostly on the writing (statements), and (iii) KUZW changed the score to acceptance in the end. And KUZW also mentioned that this paper can contribute to the long-running debate for inductive biases vs. scale for at least point clouds.
Decision: Accept (poster)