Adaptive Canonicalization with Application to Invariant Anisotropic Geometric Networks
Authors: Ya-Wei Eileen Lin
Ron Levie
Venue: ICLR 2026 Poster (2026)
Keywords: Equivariant machine learning
Canonicalization
Universal approximation
Classification
Graph neural networks
Spectral methods
Point cloud networks
Anisotropic networks
Primary area: learning on graphs and other geometries & topologies
TL;DR: We present adaptive canonicalization, a symmetry-preserving and continuous method with universal approximation properties for equivariant machine learning.

Abstract: Canonicalization is a widely used strategy in equivariant machine learning, enforcing symmetry in neural networks by mapping each input to a standard form. 
Yet, it often introduces discontinuities that can affect stability during training, limit generalization, and complicate universal approximation theorems. 
In this paper, we address this by introducing *adaptive canonicalization*, a general framework in which the canonicalization depends both on the input and the network.
Specifically, we present the adaptive canonicalization based on prior maximization, where the standard form of the input is chosen to maximize the predictive confidence of the network.
We prove that this construction yields continuous and symmetry-respecting models that admit universal approximation properties. 

We propose two applications of our setting: (i) resolving eigenbasis ambiguities in spectral graph neural networks, and (ii) handling rotational symmetries in point clouds. 
We empirically validate our methods on molecular and protein classification, as well as point cloud classification tasks. Our adaptive canonicalization outperforms the three other common solutions to equivariant machine learning: data augmentation, standard canonicalization, and equivariant architectures.

Published as a conference paper at ICLR 2026
ADAPTIVE CANONICALIZATION WITH APPLICATION TO INVARIANT ANISOTROPIC GEOMETRIC NETWORKS
Ya- Wei Eileen Lin and Ron Levie
1 Technical University of Munich, School of Computation, Information and Technology
2 Munich Center for Machine Learning
3 Technion Israel Institute of Technology, Faculty of Mathematics
ABSTRACT
Canonicalization is widely used strategy in equivariant machine learning, en- forcing symmetry in neural networks by mapping each input to a standard form. Yet, it often introduces discontinuities that can affect stability during training, limit generalization, and complicate universal approximation theorems. In this paper, address this by introducing adaptive canonicalization, general frame- work in which the canonicalization depends both on the input and the network. Specifically, we present the adaptive canonicalization based on prior maximiza- tion, where the standard form of the input is chosen to maximize the predictive confidence of the network. We prove that this construction yields continuous and symmetry respecttn models that admit universal approximation properties.
We propose two applications of our setting: (i) resolving eigenbasis ambiguities in spectral graph neural networks, and (ii) handling rotational symmetries in point clouds. We empirically validate cation, as well as point cloud classification tasks. Our adaptive canonicalization outperforms the three other common solutions to equivariant machine learning: data augmentation, standard canonicalization, and equivariant architectures.
methods on molecular and protein classifi-
INTRODUCTION
Equivariant machine learning (Gerken et al., 2023; Villar et al., 2021; Han et al., 2022; Keriven & Peyré, 2019) has been accentuated in geometric representation learning (Bronstein al., 2017), motivated by the need to build models that respect symmetry inherent in data. For example, permu- tation equivariance in graphs (Gilmer et al., 2017: Zaheer et al., 2017; Xu et al., 2019), translation equivariance in images (LeCun & Bengio, 1998; Cohen & Welling, 2016), and SO(3) SE(3) equivariance for 3D objects and molecules (Thomas et al., 2018; Fuchs et al., 2020; Batzner et al., 2022; Satorras et al., 2021). The symmetry is built into the method so that transforming the input induces a predictable transformation of the output. This inductive bias reduces sample complexity, curbs overfitting to arbitrary poses, and often improves robustness on distribution shifts where the same object appears in different orientation or ordering (Kondor & Trivedi, 2018; Wang et al., 2022; Park et al., 2022: Bronstein al., 2021; Bietti & Mairal, 2019; Kaba & Ravanbakhsh, 2023).
There are three principal approaches to handling symmetry in machine learning. The first involves designing equivariant architectures (Cohen & Welling, 2017: Weiler et al., 2018a; Sa; Weiler Cesa, 2019; Geiger & Smidt, 2022; Maron et al., 2019a; Lippmman et al., 2025): neural network layers are constructed commute with the symmetry. The second approach is data augmentation, where each datapoint is presented to the model at an arbitrary pose (Chen et al., 2020; Brandstetter et al., 2022b). The third strategy canonicalization (Kaba et al., 2023; Ma et al., 2023; 2024; Lim et al., 2023b;a; Mondal et al., 2023; Lawrence et al., 2025; Sareen et al., 2025; Luo et al., 2022): each input is mapped to a standard form and then processed by a non-equivariant network. Another common approach to equivariant machine learning is frame averaging (Puny et al., 2022), which averages the network's 's output over a set of input transformations.
A well-known problem in canonicalization is that in many cases it unavoidably leads to end-to-end architecture which is discontinuous with respect to the input (Dym et al., 2024; Zhang et al., 2020; Lim et al., 2023b). This inevitably leads to problems in stability during training and in generalization,

---

Published as a conference paper at ICLR 2026
Optimall Inaraformntion
xlfs... Ja
Clasiiher
Prior of clasoilier Ap
91
Prior masimiration (closs
laput
Optimal tranaformmtion
Classileer
Prior classiffer hy(
Sum D himary croes entrroiie lees
92
Prior macimization (close
Optinnal transformmtion
Classiller
Prior classiffiee Ap(We)
0p
Prior movemization (clase
Figure 1: Illustration of prior maximization adaptive canonicalization in classification. The adaptive canonicalization optimizes the transformations v, of the inputs to the classifiers V. while, during training, , are simultaneously trained w.r.t. the adaptively canonicalized inputs π T(32, W )x).
as very similar inputs can lead to very different outputs (Dym et al., 2024; Tahmasebi & Jegelka, 2025a;b). Moreover, the discontinuity of the network makes universal approximation properties less natural, as one approximates continuous symmetry preserving functions with discontinuous networks (Dym et al., 2024; Kaba et al., 2023; Wagstaff et al., 2022).
Our Contribution. In this paper, we show that the continuity problem in canonicalization can be solved if, instead of canonicalizing only as function of the input, one defines canonicalization that depends both on the input and the network. We propose such a general setting, which we call adaptive canonicalization, and show that it leads to continuous end-to-end models that respect the symmetries of the data and have universal approximation properties. Our theory does not only lead to superior theoretical properties w.r.f. standard canonicalization, but often also superior empirical performance, specifically, in molecular, protein, and point cloud classification.
We focus a specific class of adaptive canonicalizations that we call prior maximizers. To explain these methods, we offer the following illustrative example. Suppose that we would like to train a classifier of images into cats, dogs and horses. Suppose as well that each image x can appear in the dataset in any orientation, i.c., as x(a) for any a E [0, 2n] where T(a) is rotation by a. One standard approach for respecting this symmetry is to design an equivariant architecture θ which gives the same class probabilities to all rotations of the same image. i.e., (T(())z) = ((T(a')z) for any to angles a, a'. Another simple approach for improving the classifier to train a symmetryless network V and augment the training set with random rotations π T(a), ). for each input rr. Yet another standard approach is to canonicalize the input, namely, to rotate each input image x by an angle 3z that depends on x in such way that all rotated versions of the same image would have the exact same standard form, i.c., T (BB((1)):) (a))= = ग (Ba(ar))) ( Jæz for any two angles a, a'. Then, the canonicalized image T (33 )ar is plugged into a standard symmetryless neural network V, and the end-to-end architecture (1 (3)) is guaranteed to be invariant to rotations. We propose fourth approach, where the canonicalized rotation ग (33.v) depends both on the image x and on the (symmetryless) neural network V
To motivate this approach, consider a neural network which, by virtue of being symmetriless, may perform better some orientations of than others. For illustration. it is easier for humans to detect an image as a horse if its limbs point downwards. Suppose that (x) = (V) (#) = (ogg(a)) V Veae(e) (x),, (horse(z)) (x)) is a sequence of binary classifiers with values in [o, 1] each. The output of V is defined to be the class with highest probability. Suppose moreover that for each j. the network V is granted the ability to rotate x freely, and probe the output (T(())) for each a. The network then chooses the orientation a, such that (T(TT,,) )xr) is maximized. As an analogy, one can imagine an image on a piece of paper being handed at a random orientation to a person with a visual system V. To detect if there is a horse the image, the person would rotate the paper, searching for an orientation which looks like a horse. Namely, if there is an orientation a, where Whorse (T (o.). )ar is high then there a horse the image, and otherwise there none. This process would be repeated for all other classes, and eventually the image would be classified as the j. such that max, ; (a(a))) is greater than max, V (=))z) for all other j A j.. This is the process that we call prior maximization adaptive canonicalization. This process is inspired by ideas from cognitive psychology, where the human visual system is believed to learn canonical mental models of objects and to discard redundant variation due to symmetries by mentally "rotating" perceived stimuli into alignment with these canonical views (Shepard & Metzler, 1971; Cooper & Shepard, 1973; Tarr & Pinker, 1989). Our example of person rotating a sheet of paper to recognize whether it contains
2

---

Published as a conference paper at ICLR 2026
horse is directly inspired by this line of work: prior maximization adaptive canonicalization can be viewed as a neural analogue of mental rotation, where the network searches over transformations to align imputs with its learned canonical views. We note that some previous models in machine learning were also inspired by this process (Palmer, 1 Harris et al., 2001; Graf, 2006; Gomez. et al., 2008; Konkle & Oliva, 2011, I; Risko & Gilbert, 2016; Tacchetti et al., 2018; Schmidt & Stober, 2024).
We note that the inputs are adaptively canonicalized also during training. so V needs not learn to respect any symmetry on its own. In fact, can benefit from being symmetryless. For example, may search for "horse head" patterns only diagonally above where it detects "horse limbs" patterns, and rely on the prior maximization to orient horses accordingly.
We show that adaptive canonicalization leads to a continuous symmetry preserving end-to-end classifier that can approximate any symmetry preserving continuous function when V/ are non- equivariant neural networks. As an application, we propose adaptive canonicalization methods for 1) spectral graph neural networks, where the symmetry in the choice of the eigenbasis of the graph shift operator, and 2) point clouds, with rotation symmetries. We show that adaptive canonicalization in these cases outperforms both standard canonicalization and equivariant networks, as well as augmentation methods. See Figure 1 for an illustration of prior maximization.
2 RELATED WORK
Canonicalization has been studied in several forms. For example, in computer vision and geometric deep learning methods, inputs often first transformed into a standardized pose reference frame before classification (Lowe, 2004; Jaderberg et al., 2015). More recent work formalizes this as an explicit canonicalization map feeding downstream network (Lim et al., 2023b; Ma et al., 2024) as energy-based canonicalization (Kaba et al., 2023) in which one learns an energy over group elements and takes the minimizer as the canonical transformation. The latter has been further developed on symmetries defined by general Lie group actions (Shumaylov et al., 2025). Canonicalization has also been used for data aligmment (Mondal et al., 2023; Schmidt & Stober, 2025) and for test-time optimization over transformations, where one searches over group actions to select a canonical representation before downstream inference (Singhal et al., 2025; Schmidt Stober, 2024). A related line of work is frame averaging (Puny et al., 2020), which averages network's 's output over a set of group transformations, and its extension to weighted frame averaging (Dym et al., 2024), where each datapoint is equipped with probability distribution over the group and averaging is performed with respect to this measure, yielding continuity guarantees. In our work, we instead function of both the input and the network, and establish continuity
study canonicalization preserving continuous functions realized by our construction. Moreover, guarantees for symmetry-preserving our approach is not restricted symmetries defined via group actions, and allows working with more general augmentations for transforming datapoints. We refer to Appendix B for further discussion and comparison with extended related work.
ADAPTIVE CANONICALIZATION
In this section, we develop the general theory of adaptive canonicalization, and prove that it leads to continuous symmetry preserving networks with universal approximation properties.
3.1 BASIC DEFINITIONS AND BACKGROUND
The function that maps each input to the output f(z) is denoted by + f(x). The free variable of a univariate function is denoted by (-), and by (-...) for a function of two variables. For example, the function (x. y) + sin xx) exp(y) is also denoted by sin(-) exp(-)). We denote the infinity norm of
x = ( E R by x == max gas |za]] We define the infinity norm of a continuous function
f: R over a topological space IC by Il = supzex [x() If ]| f ylloo < E we say that approximates of uniformly up by 2x. When defining general metric spaces, we allow the distance between points to be co. This does not affect most of the common properties of metric spaces (see Burago al. (2001)).
The set of all subsets of a set KC, i.e., the power set, is denoted
Function Spaces. We develop the definitions of adaptive canonicalization in general locally com- pact Hausdorff spaces. Two important examples of such a space is compact metric space or V
3

---

Published as a conference paper at ICLR 2026
Definition 1. Let K. be locally compact Hausdorff space, and D N.
A function f - RD said to vanish at infinity for every > O there exists a compact set
K C. I such that [f(x) < for every K K.
The space of all continuous functions f KC - RD that vanish at infinity, with the supremun norm || b0
max (x)) denoted by Co(K,
In adaptive canonicalization, we consider families of continuous functions where the c - formulation of continuity is uniform over the whole family, as defined next.
Definition 2. Let X' and y be two metric spaces with metrics dx and dy respectively. A family F of function f X >> y called equicontinuous if for every x E and every e. O, there exists 5 such that every E X which satisfies dx(x, 2) < 8 also satisfies
V dy(f(z), <e
Universal Approximation. Universal approximation theorems (UAT) state that any continuous function over some topological space can be approximated by a neural network. In such a case, the neural networks said to be universal approximators, defined next.
Definition 3. Let K. be locally compact Hausdorff space and D E N. A set of continuous functions N(K, IR C Co(K, is said to be a universal approximator of Co(K, RP) if for every f Co(K, R)) and e > 0 there is a function E EN(C, R) such that
Vx K:: 1f(x) 0(x) <.
In the above definition, we interpret N(,, D) as space of neural networks, A UAT is hence any theorem which shows that some set of neural networks is universal approximator, Two examples of UATs are: 1) multilayer perceptrons (MLP) are universal approximators of Co,,, RP) for com- pact subset IC of the Euclidean space R² (Hornik et al., 1989; Cybenko, 1989), and 2) DeepSets (Zaheer et al., 2017) are universal approximators of continuous functions from multi-sets to R See Appendix D for more details.
3.2 ADAPTIVE CANONICALIZATION
In the general setting of adaptive canonicalization, we have domain of inputs G which need not have any structure apart for being set, e.g., the set of graphs. We consider continuous functions f: R over a "nice" domain K, e.g., = R Such functions can approximated by neural networks. We then pull-back f to be function from G to R using mapping Pj 9 + K. that depends on (is adapted to) f. Namely, we consider (-)r G - IRD. The following definitions assure that such setting leads to functions with nice properties, as we show subsequent sections.
Definition 4. Let K. be locally compact Hausdorff space, G be set, and D N. A mapping P= (-)) Co(,, RD) x G KC, (f,g) + Pf(g), is called an adaptive canonicalization if the set of functions
+ f pg g gEg) is equicontinuous (as functions Co(K, R) - D.. Here, f pr(p)) f(r((g))
Next, we define the function space that we would like to approximate using adaptive canonicalization. Definition 5. Let K. be a locally compact Hausdorff space, G be set, and D E N. Let p be an adaptive canonicalization, and let f E Co, P. The function
fo 9 D g f(p((g))
is called an adaptive canonicalized continuous function, or canonicalized function in short.
In Section 3.4, we show that for an important class of adaptive canonicalizations the set adaptive canonicalized continuous functions is exactly the set of all symmetry preserving continuous functions.
is now direct to prove the following universal approximation thcorem.
Theorem 6 (Universal approximation of adaptive canonicalized functions). Let N(K, be universal approximator of CKC, RD), and I P an adaptive canonicalized continuous function. Then, for every > 0, there exists a network 0 E A N (KC,RR. DR such that for every E
|f pp()) 0 po
4

---

Published as a conference paper at ICLR 2026
Proof. L 0. By Definition there exists 8 such that
Vy E Co(K, RP): ||S - yyllo < 6 = Vg E eg: (fo - y pg(0) <c)
By the universal approximation property, there exists a network such that |ʃ - 011o0 < Hence,
by (1), Vg E G: py(g) 8 po(g))
3.3 PRIOR MAXIMIZATION ADAPTIVE CANONICALIZATION
Prior maximization is special case of adaptive canonicalization, where Ps is chosen to maximize some prior on the output f. The maximization is done over space of transformations K2 9 + IC parameteried by i.e., maximizing the prior of (N)( w.r.t. u.
Let be metric space, and for every u E EU,
(-)(-) :4 x G IC, (u. () (g) E.C.
Suppose that ka(g) is continuous in u for every E G. We call a transformation family, trans- forming objects in into points in K. where different H = (fi..... hD), where ha R - IN for each d, be a sequence of continuous monotonic functions, that we call the ensemble of priors. We call each ha a prior. We denote H := (ha o fa)
l define different transformations. Let
For every (fi,...,
Co(K, g & and d, assume that ha o fa(((()(g)) attains maximum in U. This is the case for example when U is compact. Define
Pr(g) (0) (0
((Km.(9)
hao fa (g)) max fa( d=1 (2*)p. (2)
well
Note that P5 G (2 By abuse of notation, we also denote by Pf the mapping that returns some arbitrary sequence of points (Ia E o4. (g)) E KD for each g E G. The choice of the specific point in &%,(0) (g) does not affect the analysis. We interpret Pj function that takes input g and canonicalize it separately with respect each output channel fa, adaptively to fa.
When used for classification, we interpret each output channel fa p (g) E [o, 1] as a binary classifier, i.e., representing the probability of g being in class vs. not being class d. This multiclass classification setting is called one rest, where a standard loss is sum of D binary cross-entropies (Rifkin & Klautau, 2004; Galar et al., 2011; Allwein et al., 2000).
Definition 7. Consider the above setting. The mapping defined by (2) is called prior maximization. in addition G has metric such that for every J E Co(K, D) the family (g + (())(g)))eel equicontinuous, is called contimuous prior maximization.
In Theorem 8 we show that prior maximization is indeed adaptive canonicalization.
For example, let 2/ S3(3) be the space of 3D rotations, and & K B the set of sequences of N points in the 3D unit ball B, i.e., the space of point clouds. We consider the rotation g (g(g) of the point cloud g by u E U. Since and 2 are compact metric spaces, and (g, u) + K2 (g) is continuous, i and must be uniformly continuous. Hence, {g + fK, (g)) is equicontinuous. In fact, whenever G is compact and н continuous w.r.t. (u,g), it is automatically also uniformly continuous, so is a continuous prior maximization. See Appendix F for additional examples of continuous prior maximization.
Properties of Prior Maximization.
Theorem 8. In prior maximization, each p² Co x + K. is adaptive canonicalization.
Proof. Consider without loss of generality the case where the output dimension is L) = 1. Since the specific values of И fn do not matter, only if it is ascending or descending, without loss of in which case prior maximization maximizes directly the output of generality suppose (xx) Consider an arbitrary maximizer Pr(g) E arg max,eu f( for each foru (g)(g) with respect
E Co(K, R). The choice of the maximizer does not affect the analysis. Now, if J,y E Co(K, R)
satisfy |] vlloo < e, then also for every u E U, [f( (g)) v(K((g)) < e. uo be
maximizer of f(ra(g)) We have y(Kug (9)) > [Ne. (g)) E, s0 max (Ku (g)) > max (KU (g)) -
5

---

Published as a conference paper at ICLR 2026
Similarly, we have max, (K)( > max, (k) Together,
max NEW
f o No (g) max o Ku (9) <c
(3)
wEM
Pg(g) is Lipschitz continuous with Lipschitz constant for every g E g, and Hence, / +
therefore equicontinuous over the parameter g g.
O
This immediately gives a universal approximation theorem for prior maximization as corollary of Theorem 6. Moreover, we can show that continuous prior maximization gives functions continuous in G. This is one of the main distinctions between prior maximization and standard canonicalization.
Theorem 9. Consider a continuous prior maximization (Definition 7). Then, f o pf is continuous.
Proof. Let e > O. For every E 9 there is 5 = dg.g > O such that for every g' E 9 with d(g. g') < and every U we have (s(u)))) - ((sulg)) ) < e. Now, by the same argument as in (3), max (9) max ((Fal <c
3.4 SYMMETRY PRESERVING PRIOR MAXIMIZATION
Consider the following additional assumptions on the construction of continuous prior maximization. Suppose that the space G is disjoint union of metric spaces Gj with finite distances, i.c., 9 = U, Here, j may run on a finite or infinite index set. We define the metric d in g as follows. For 93 E Gj and 9: 9s, d(gj-9) 9:) co if j + and d(g) 9:) d (g). 9:) oc if i j. where d; is the metric in Gj. In the theory of metric spaces, the spaces Gj are called galaxies of G. This construction is useful for data which does have a uniform notion of dimension, e.g., graphs. For example, each galaxy in this case can be the space of adjacency matrices of a fixed dimension with standard matrix distance.
For each j. let Ug be a group acting continuously on Gj by Ty (ug)g, Namely, Ty Ty(Ug) : Gg Gj continuous for every u, E Uj. and for every vty E Uy and 9j E 9j we have 2z ) (u;))) Ty ((f/u,)) 1), and πz (e))))
9y- where e the identity of gy. Define U = WjHf, Namely, U the metric space with galaxies U, similarly to the construction of G. Let x be mapping that we formally call an action of on G, defined for u = u, E U4 C l and = 9و E 9 c G by x {u)g = n, (u,))) (g)) if i == j and Tuu]g= g if $
Definition 10. Consider the above setting and a continuous prior maximization Let P g + K. be continuous, and suppose that the transformation family k is of the form K., = Po (u)) We call symmetry preserving transformation family, and p a symmetry preserving prior maximization.
Note that whenever the spaces U4j are compact, the functions u + hgoo (K., (g)) of (2) are guaranteed attain maximum, even though the space '/ W, is in general not compact. Hence, the above setting with compact U, is an example of prior maximization. More generally, if the restriction of the setting to My and 9j is (continuous) prior maximization for a single j. then the setting for U/ and G is also a (continuous) prior maximization.
Definition 11. We call a function Q : G + R continuous symmetry preserving if there exists F: R in (KC IR R) such fhatfor all u u and g E G. Q(g) = F(P(T(u)(g))
When IC G = G1 and P is the identity, a symmetry preserving continuous function is a continuous function which is invariant to the action of u, i.c., the classical case in equivariant machine learning.
Properties of Symmetry Preserving Prior Maximization. We already know by Theorem 9 that fo Ps is a continuous function when is a symmetry preserving prior maximization. We next show that the set of functions of the form f o Ps exhaust the space of all continuous symmetry preserving functions.
Theorem 12. Let p be symmetry preserving adaptive canonicalization, Then,
1. Any continuous symmetry preserving function can be written as f pj for some f Co(K,
2. For any Co(KC; D), the function f pf G + RD continuous symmetry preserving.

---

Published as a conference paper at ICLR 2026
Proof. Without loss of generality, consider the case D 1 and (x)
1. For Q(g) = =(P(T(u)()))) take f= F. Then, by definition of symmetry preservation, for every U: f pr max, F(P o (v)g) = F x (u)g) Q(9).
2. For any u = u, E Uú since (T,),) is an action, for any g = 9j E Gj.
( max f(P (()))(())))))
max f(P (o,))))) fo myeū;
"y "yEly
This leads the following UAT.
Theorem 13. Consider a symmetry preserving prior maximization and let N DR be uni- versal approximator of Co(K, J.K,R" RP). Then, any continuous symmetry preserving function can be approximated uniformly by 0 po for some network 0 e &N(,R)))
4 APPLICATION OF ADAPTIVE CANONICALIZATION TO ANISOTROPIC GEOMETRIC NETWORKS
In this section, we propose two architectures based on adaptive canonicalization which can be inter- preted as anisotropic. First, a spectral graph neural network (GNN) which is sensitive to directionality within eigenspaces. Then, 3D point cloud network which is sensitive to 3D directions. In Appendix A, we give a detailed tutorial for our proposed architectures. Below, we give a concise derivation.
Basic Notations for Graphs and Vectors. We denote by No the set of nonnegative integers. We denote matrices and 2D arrays by boldface capital letters, e.g., E R NxT, We denote [N]] (1.....N) } for N E N. We denote by (::j) j) and B(j.:) :) the j thth column and row of the matrix B respectively. A graph is denoted by G = ([N], A, S), where [N] the set of N vertices, A E N is the adjacency matrix, and S E R X is an array representing the signal. Namely, S(n,:) E R.. is the feature at node n. A graph shift operator (GSO) a self-adjoint operator that respects in some sense the graph's 's connectivity, e.g., graph Laplacian or the adjacency matrix.
4.1 ANISOTROPIC NONLINEAR SPECTRAL FILTERS
Consider graphs with self-adjoint GSO L (e.g., a graph Laplacian or the adjacency matrix) and T- channel signals (over the nodes). The number of nodes N varies between graphs. Consider predefined bands bo < 61 < < bB E I. and their indicator functions P% == [bb-11bb)) R + R.! For each k E [B]. consider the space of signals X in each band [b,-1, by). namely, the range of the orthogonal projection (C). Let Mj be the dimension of Xx. We also call X₂ the k'th band. See Appendix C for details on how to plug operators into functions via functional calculus. Consider an orthogonal basis X = (Xx (t. (t); E R x M. for each band Xx. In this setting, the symmetry is the choice of the orthogonal basis within each band Xz. Namely, for X₃ NMM, and orthogonal matrix U₁ E R Mr x Mr, the bases X₁ and X, U are treated as different vectorial representations of the same linear space Xt.
In the above setting, the galaxies Gj are defined as follows. Each galaxy is indexed by j (N. M) :=
(N; M,,..., MB) where N E N is the size of the graph, and M, E No for E [B] are the dimensions of the bands, satisfying Σx M₁ < N. Each GN.M M. is the space of pairs of a signal S E RNNT xT and a sequence of B orthogonal matrices of height N and widths M with dimension 0 by 0. For two signals S, Q and orthogonal bases sequences X, Y, the metric d(X, (),))))))) (Y, Q)) is defined to satisfy
We denote the matrix
Mr ΣΣN
d(x, Y) 2 1 SS Q11
(x(:))- Y Y& is Q + X Y
k=1
where for (z,) n different galaxies is Do.
|z = En 2
By definition, the distance between any two points from two
11 lou- ) (x) the function that returns if E [bc-1, bb) and O otherwise.
7

---

Published as a conference paper at ICLR 2026
The transformation family ku apply unitary operators U₁ E (Ma) form the right each ((:, where ((M&)) is the group of orthogonal matrices in RM, XM, Namely, Uy; := I (Mx)).
We now define the operator C' that computes the spectral coefficients of the signal S with respect to the spectral basis X. Namely,
C(X. = (Cb(X B = (X(:,)) R R k==
In anisotropic nonlinear spectral filters (A-NLSF), we consider a symmetryless neural network w that operates on the space of spectral coefficients of the signal. e.g., a multilayer perceptron (MLP). To define V consistently, over Euclidean space of fixed dimension, we extend or truncate the sequence of spectral coefficients follows. Let JI,..., JB be predefined sequence of integers, and denote J = Σx Jk. We define as the mapping that takes (X, S) as input, first compute the spectral coefficients (Cx(X, k=1" and then truncates or pads with zeros each C& (X, S) to be in B
k= RJ'T. Here, the matrix 0 is padded the zero matrix 0 E R xT, Hence, the symmetryless network Y maps Xx T to RD. A more detailed derivation of the construction is given in Appendix A.3.
RJxxT, Namely, P(X,S) = (Px(X, E B
We call this architecture anisotropic for the following reason. Consider for example the grid graph with N x N vertices. Since spectral filters are based on functional calculus (see Appendix C), they are invariant to graph automorphism, and hence to rotations. This means that spectral filters treat the x and y axes equally, and any filter is isotropic in the spatial domain. On the other hand, our symmetryless network V/ can operate differently on the æ and axes, and we can implement general directional filters on images with V. In the general case, V can operate differently on Fourier modes from the same eigenspace, which we interpret as directions within the eigenspace, while standard GNNs cannot. See Appendix H.2 for more details.
4.2 ANISOTROPIC POINT CLOUD NETWORKS
Here, we present a point cloud network which is a combination of an equivariant network with adaptive canonicalization. Namely, we consider permutation invariant network Yi like DeepSet (Zaheer et al., 2017) or DGCNN (Wang et al., 2019), and to attain 3D rotation invariance in addition incorporate prior maximization. Together, the method is invariant both to permutations and rotations. We call this method anisotropic since y does not respect the rotation symmetries, and is hence sensitive to directions in the x, y, space.
We restrict the analysis to multi-sets of a fixed number of points N. Multisets are sets where repetitions of elements allowed. Here, we formally define multi-set as equivalence class of arrays up to permutation. To define this, let SN be the symmetric group of N elements, i.e., the group of permutations. Given 8 E S and X RJ let p(s) be the permutation that changes the order of the rows X according to s. We say that X ~ Y if there is E SN such that X = p(s)Y. Y. The equivalence class [X] defined also called the quotient space, is denoted by (IR Nx/ ~)) = { X E NNN. }. We identify the space (R.VJ// ~) with the space of multisets.
(Y NxJ J X}. and the space of equivalence classes,
In Appendix D.2 we show that the quotient space has natural metric. We hence take G = KC consisting of a single galaxy GNv Nx)/ ~). We moreover show in Appendix D.2 that any universal approximator of permutation invariant functions in CR xJ DR²), e.g., DeepSet (Zaheer et al., 2017), canonically gives a universal approximator of general continuous functions in (R P..).
xxJ
The symmetry in our adaptive canonicalization 3D rotations (u) E R3x3 where u is in the rotation group SO(3). Namely, we consider J -3, and rotate the rows of X E Nx3 via Xi("u=)) We take P as the identity. Note that this construction can be easily extended to multisets of arbitrary sizes, by considering the galaxies GN R Nx3/ for all N N, and UN SO(3) applied from the right on N For details on the thcoretical construction see Appendix D.2, and for details on the architecture see Appendix A.4.
8

---

Published as a conference paper at ICLR 2026
Table 1: Classification performance on the grid signal orientation task and graph classification and the second highest is in
benchmarks from TUDataset. The highest accuracy is in
TUDataset
Toy Example
MUTAG
PTC
ENZYMES
PROTEINS
MLP
50.03:0.1
79.31
63.98:20
42.17=28
75.08
77.34:15
GCN
50.01 1o1
81.63
60.22.119
43.66184
75.17
16.29
GAT
49.95
83,17
62.31
39.83+37
74.72s4.
74.01:43
GIN
50.00.01
83.29
63.25
45.69
76.02.29
79.84
ChebNet
50.12a01
82.15
64.06s11
50,42a14
74.28
76,98
FA+GIN
49.99
84.07
66.58
52.64
79.53,24
80.231
OAP+GIN
50.03,00
67.35
18.40
83.41:14
80.97.11
NLSF
50.07
84.13a15
68.17
65.94:18
82,69219
80.51
S'GNN
49.93,01
82,70
67.34
63.26g28
78.52.19
75.62
N-NLSF
99.38102
87.94.09
73.16.122
73.01
85.47:0s
4.3 ADDITIONAL APPLICATIONS OF ADAPTIVE CANONICALIZATTIN
Adaptive canonicalization can be used to model image truncation augmentations. Here, the "symmetry" corresponds different crops and prior maximization selects the crop on which the classifier is most confident (see Appendix A.5). Our formulation also accommodates various image settings with rotation symmetries and other image transformations (see Appendix F). Note that our setting also applies to pretrained networks. Finally, to broaden the applicability of our approach beyond classification, we explore using the adaptive canonicalization mechanism for point cloud segmentation (see Appendix H0).).
5 EXPERIMENTS
We evaluate the anisotropic nonlinear spectral filters (Section 4.1) on toy problems and graph classification, and test the anisotropic point cloud network (Section 4.2) on point cloud classification. The experimental details, including experimental setups and hyperparameters, are in Appendix G. Additional experiments, e.g., ablation study, are in Appendix H.
Maximization Method. We approximate the prior maximization by sampling a finite set trans- formations from a probability measure on the transformation space, evaluating the prior for all candidates, and retaining the transformation that yields the largest prior value with the one-vs-rest classification objective. We then refine the selected candidate locally by running few steps of gradient descent (see Appendix A.2).
5.1 EXPERIMENTAL EVALUATION OF ANISOTROPIC NONLINEAR SPECTRAL FILTERS
Illustrative Toy Problems: Grid Signal Orientation Task. To showcase the effectiveness of A-NLSE, we study toy classification task: grid-split channel orientation. We consider square grid on the torus, and each node has two channels. The grid is partitioned vertically into two equal disjoint halves. Channel is nonzero only the left half, and Channel 2 is only nonzero the right half. In class O, both channels are I-frequency pure harmonic signals along x, and class 1, Channel is 1 1-frequency along x and Channel is 1-frequency along y. The task is to decide if the oscillations the two channels are in the same orientation. See Appendix G for further details.
We compare A-NLSF with the following baselines: (i) MLP, (ii) GCN (Kipf & Welling, 2017), (iii) GAT (Veličković et al., 2018), (iv) GIN (Xu et al., 2019), (v) ChebNet (Defferrard et al., 2016a), (vi) NLSF (Lin et al., 2024a), and (vii) S*GNN (Geisler al., 2024). In addition, we test canonicalization baselines by combining GIN with frame averaging (FA) (Puny et al., 2022) and orthogonalized axis projection (OAP) (Ma et al., 2024). Table reports the classification results. We see that competing methods remain at chance level while A-NI.SF achieves high accuracy by adaptively resolving ambiguities, showing the advantage of adaptive canonicalization, which can internally break the rotation symmetry and use anisotropic classifiers.
Graph Classification on TUDataset. We further evaluate A-NLSF on graph classification bench- marks from TUDataset (Morris et al., 2020): MUTAG, PTC, ENZYMES, PROTEINS, and NCI1,
9

---

Published as a conference paper at ICLR 2026
and follow the experimental setup (Ma et al., 2019; Ying et al., 2018; Zhang al., 2019b) (see Appendix G). We compare with the same baselines as in the grid signal orientation tasks. Table summarizes the classification performance. Canonicalization baselines generally improve over GIN. Notably, we observe that A-NLSSF outperforms competing baselines, suggesting that our method provides more informative representations compared to fixed, precomputed canonical form or isotropic filters.
Molecular Classification on OGB Datasets. To further assess the effectiveness of A-NISF,
Table 2: Molecular and protein classification per- formance on OGB datasets.
we evaluate on large-scale molecular and pro- tein benchmarks from Open Graph Benchmark (OGB) (Hu et al., 2020): ogbg-molhiv. ogbg- molpcba, and ogbg-ppa. We compare with GCN, GIN, GatedGCN (Bresson & Laurent, 2017). PNA (Corso et al., 2020), GraphTrans (Wu et al., 2021), SAT (Chen et al., 2022), GPS (Rampášek al., 2022), SAN (Kreuzer et al., 2021), and the canonicalization method OAP+GatedGCN. The molecular classification results are reported in Table 2. We see that our method achieves consistent improvements across these datasets, leading to improved generalization in classification.
oghg-mellhiv
oghg-molpcka
ogig-ppa
AUROC
Avg. Procision
Accuracy f
GCN
0.7599200119
0.2424=00004
0.6857
CIN
a.7707
0.2703-muoed
0,7057
GatdGGCN
0,7687=20.016
0.2670=00000
0.7531
PNA
0.7905
0,2838 Euloos
GraphTrans SAT
0.2761 200
0.
GPS
0.7880
0,2907
0,801 aaotes
SAN
0.7785
0.2765=40:042
0.7802e0.us
0.2783
0,7745e0
A-NLSF
0.8019
0.2968=00002
0.8149900000
5.2 EXPERIMENTAL EVALUATION OF ANISOTROPIC POINT CLOUD NETWORKS
To evaluate adaptive canonicalization on point cloud classification, we test ModelNer40 (Wu et al., 2015). The dataset consists of 12,311 shapes from 40 categories, with 9,843 samples for training and 2,468 for testing. We build on two point cloud architectures, PointNet (Qi et al., 2017 7a) and DGCNN (Wang al., 2019), and apply adaptive canonicalization into their pipeline, denoted respectively AC-PointNet and AC-DGCNN (see Section 4.2 and Appendix A.4). Following Esteves et al. (2018); Deng et al. (2021), we perform on-the-fly rotation augmentation during training, where the dataset size remains unchanged, and for the test set, each test example is arbitrarily rotated as well. This produces data distribution where each shape appears in any orientation with the same probability. We compare with PointNet, DGCNN, equivariant networks VN-PointNet and VN-DGCNN from Deng et al. (2021), canonicalization methods CN-PointNet and CN-DGCNN from Kaba et al. (2023), and traditional augmentation baselines where the training set is statically expanded with pre-generated rotated samples, denoted PointNet-Aug and DGCNN-Aug. For further details, see Appendix G. Table 3 shows the classification performance. We observe that our method outperforms the competing baselines and it allows the model to learn both local geometric features and optimal reference alignments for each class.
Table 3: Classification re- sults on ModelNet40. Re- sults of competing meth- ods marked with * are taken from Deng et al. (2021); Luo et al. (2022); Kaba et al. (2023).
Accuracy
PointNet DGCNN
74.7"
88.6"
PointNet-Aug DGCNN-Aug
75.8.n9
89.0=10
VN.PointNet VN-DGCNN
77.2"
90.2"
CN-PoimNet CN DNCDGGNN
79.7=13"
90.0
AC: PoiniNet AC-
21.1
91.62n6
CONCLUSION
We introduce adaptive canonicalization based on prior maximization, a general framework for equivariant machine learning in which the standard form depends on both the input and the network. We prove that our method is continuous, symmetry preserving, and has universal approximation properties. We demonstrate the applicability of our theory in two main settings: resolving eigenbasis ambiguities in spectral graph neural networks, and handling rotational symmetries in point clouds.
Limitations and Future Work. Our framework is naturally suited to classification tasks, and at the current scope of the paper, we did not address regression tasks theoretically. We will extend the adaptive canonicalization to regression in future work. Another limitation of our approach that prior maximization requires solving D optimizations at runtime for the D classes. In future work, we will reduce this to a single optimization to improve efficiency.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
This work introduces a novel method for handling symmetry for equivariant machine learning, with a focus theory and with application spectral graph neural networks and point cloud networks. The experiments are conducted using simulated toy problems and public datasets, and therefore there is no concerns related to privacy, consent, or potential harm living subjects. As the data employed are technical and free from sensitive or identifiable content, the research does not raise any apparent ethical concerns. Accordingly, no additional ethical approval was required for this study.
REPRODUCIBILIIY STATEMENT
For the theoretical results, we include the main proofs in the paper and present additional analysis and illustrative examples Appendix and Appendix F. For the empirical study, the implementation details are reported in Appendix G. Our code is available at https:/ /g thub. wwt.hhu..ooo...oooooo ywwe d C.
USE OF LARGE LANGUAGE MODELS
Following the ICLR 2026 policy that requires disclosure of use of Large Language Models (LLMs), we state that an L.M was used for editing purposes, such as grammar, spelling, phrasing, and stylistic polish.
ACKNOWLEDGMENTS
We thank the anonymous reviewers for their insightful feedback. RL was supported by a grant from the United States-Israel Binational Science Foundation (BSF), Jerusalem, Israel, and the United States National Science Foundation (NSF), (NSF-BSF, grant No. 2024660), and by the Israel Science Foundation (ISF grant No. 1937/23). YEL acknowledges funding by the Alexander von Humboldt Foundation and the Munich Center for Machine Learning (MCML).
REFERENCES
P-A Absil, Robert Mahony, and Rodolphe Sepulchre. Optimization algorithms on matrix manifolds. Princeton University Press, 2008.
Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 2623 2631, 2019.
Erin L Allwein, Robert E Schapire, and Yoram Singer. Reducing multiclass to binary: A unifying approach for margin classifiers. Journal of machine learning research, 1(Dec): (1tec)::133144: 11-1411, 2000.
Brandon Anderson, Truong Son Hy, and Risi Kondor. Cormorant: Covariant molecular neural
networks. Advances in neural information processing systems, 32, 2019.
Matan Atzmon, Koki Nagano, Sanja Fidler, Sameh Khamis, and Yaron Lipman. Frame averaging for equivariant shape space learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 631-641, 2022.
László Babai and Eugene M Luks. Canonical labeling of graphs. In Proceedings of the fifteenth annual ACM symposium on Theory of computing, pp. 171-183, 1983.
Sourya Basu, Pulkit Katdare, Prasanna Sattigeri. Vijil Chenthamarakshan. Katherine Driggs- Campbell, Payel Das, and Lav R Varshney. Efficient equivariant transfer learning from pretrained models. Advances Neural Information Processing Systems, 36:4213 4224, 2023.
Simon Batzner, Albert Musaelian, Lixin Sun, Mario Geiger, Jonathan P Mailoa, Mordechai Korbluuhh Nicola Molinari, Tess E Smidt, and Boris Kozinsky. E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nature communications, 13(1):2453, 2022.
11

---

Published as a conference paper at ICLR 2026
Mikhail Belkin and Partha Niyogi. Laplacian eigenmaps for dimensionality reduction and data representation. Neural computation, 15(6):1373-1333 -139,, 2003.
Alberto Bietti and Julien Mairal. Group invariance, stability to deformations, and complexity of deep convolutional representations. Journal of Machine Learning Research, 20(25): 2019.
Deyu Bo, Chuan Shi, Lele Wang, and Renjie Liao. Specformer: Spectral graph neural networks meet transformers. In The Eleventh International Conference on Learning Representations, 2023.
Ron Bracewell and Peter B Kahn. The fourier transform and its applications. American Journal of Physics, 34(8):112-712 1966.
Amit Bracha, Thomas Dagès, and Ron Kimmel. Wormhole loss for partial shape matching. In The Tiihty-eighth Annual Conference on Neural Information Processing Systems, 2024.
Johannes Brandstetter, Rob Hesselink, Elise van der Pol, Erik Bekkers, and Max Welling. Geometric and physical quantities improve E(3) equivariant message passing. In International Conference on Learning Representations, 2022a.
Johannes Brandstetter, Max Welling, and Daniel E Worrall. Lie point symmetry data augmentation for neural solvers. International Conference on Machine Learning, pp. 22-1-2255552 PMLR, 2022b.
Xavier Bresson and Thomas Laurent. Residual 17Xiv:1711.0755: 2017.
gated graph
ConvNets.
arXiv preprint
M. M. Bronstein, J. Bruna, Y. LeCun, A. Szlam, and P. Vandergheynst, Geometric deep learning: Going beyond euclidean data. IEEE Signal Processing Magazine, 34(4):18-42 2017.
Michael M Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint arXiv:2104.13478, 2021.
Dmitri Burago, lu. D. Burago, and S. B. Ivanov. course in metric geometry Dmitri Burago, Yuri Burago, Sergei Ivanow. Graduate studies in mathematics vol. 33. American Mathematical Society, Providence, R.I, 2001. ISBN 0821821296.
Dexiong Chen, Leslie O' O'Bray, and Karsten Borgwardt. Structure-aware transformer for graph representation learning. In International conference machine learning, pp. 3469-3489. PMLR, 2022.
Shuxiao Chen, Edgar Dobriban, and Jane H Lee. A group-theoretic framework for data augmentation. Journal of Machine Learning Research, 21 21(245):1771, 2020.
Fan RK Chung. Spectral graph theory, volume 92. American Mathematical Soc., 1997.
Taco Cohen and Max Welling. Group equivariant convolutional networks. In International conference on machine learning. pp. 2990-2999, PMLR, 2016.
Taco Cohen, Maurice Weiler, Berkay Kicanaoglu, and Max Welling, Gauge equivariant convolutional networks and the icosahedral CNN. In International conference on Machine learning, pp. 1321- 1330. PMLR, 2019.
Taco S. Cohen and Max Welling. Steerable CNNs. In International Conference on Learning Representations, 2017.
Lynn A Cooper and Roger Shepard. Chronometric studies of the rotation of mental images. In Visual information processing, pp. 75-176. Elsevier, 1973.
Matthieu Cordonnier, Nicolas Keriven, Nicolas Tremblay, and Samuel Vaiter. Convergence of pressagc-passing graph neural networks with generic aggregation on large random graphs. Journal of Machine Learning Research, 25(406):1-499 2024.
Gabriele Corso, Luca Cavalleri, Dominique Beaini, Pietro Liò, and Petar Veličković. Principal neighbourhood aggregation for graph nets. Advances in neural information processing systems, 33: 13260-13271 2020.
12

---

Published as a conference paper at ICLR 2026
George Cybenko. Approximation by superpositions of sigmoidal function. Mathematics of control, signals and systems, 2(4))303-314, 1989.
Thomas Dagès, Michael Lindenbaum, and Alfred M Bruckstein. Metric convolutions: A unifying theory to adaptive image convolutions. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 13974-13984, October 2025.
Tri Dao, Albert Gu, Alexander Ratner, Virginia Smith, Chris De Sa, and Christopher Ré. A kernel theory of modern data augmentation. International conference on machine learming, pp. 1528- 1537. PMLR, 2019.
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. Advances in neural information processing systems, 29, 2016a.
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. In NeurIPS. Curran Associates Inc., 2016b. ISBN 9781510838819.
Congyue Deng, Or Litany, Yueqi Duan, Adrien Poulenard, Andrea Tagliasacchi, and Leonidas J Guibas. Vector neurons: A general framework for SO(3)-equivariant networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 12200-12209, 2021.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255, Icee, 2009.
Terrance Vries and Graham W Taylor. Improved regularization convolutional neural networks with cutout. arXiv preprint arXiv: 04552, 2017.
Weitao Du, He Zhang, Yuanqi Du, Qi Meng, Wei Chen, Nanning Zheng, Bin Shao, and Tie- Yan Liu. SE(3) equivariant graph neural networks with complete local frames. In International Conference on Machine Learning, pp. 5583-5608. PMLR, 2022.
Alexandre Duval, Simon Mathis, Chaitanya Joshi, Victor Schmidt, Santiago Miret, Fragkiskos D Malliaros, Taco Cohen, Pietro Lio, Yoshua Bengio, and Michael Bronstein. A hitchhiker's guide to geometric gnns for 3D atomic systems. arXiv preprint arXiv:2312.07511, 2023a.
Alexandre Agm Duval, Victor Schmidt, Alex Hernández-Garcdd Santiago Miret, Fragkiskos D Malliaros, Yoshua Bengio, and David Rolnick. Faenet: Frame averaging equivariant gnn for materials modeling. In International Conference on Machine Learning, pp. 9013-9033, PMLR, 2023b.
Vijay Prakash Dwivedi, Chaitanya K Joshi, Anh Tuan Luu, Thomas Laurent, Yoshua Bengio, and Xavier Bresson. Benchmarking graph neural networks. Journal of Machine Learning Research, 24 (43):1-48, 2023.
Nadav Dym, Hannah Lawrence, and Jonathan W Siegel. Equivariant frames and the impossibility of continuous canonicalization. In International Conference on Machine Learning, pp. 12228-12261 12267. PMLR, 2024.
Carlos Esteves, Christine equivariant representations with spherical cnns. In Proceedings of the european conference on computer vision (ECCV), pp. 52-68, 2018.
Ameesh Makadia, and Kostas Daniliddis. Learning SO(3)
William T Freeman, Edward H Adelson, et al. The design and use of steerable filters. IEEE Transactions on Puttern analysis and machine intelligence, 13(9):891-906, 1991.
Fabian Fuchs, Daniel Worrall, Volker Fischer, and Max Welling. transforrosss 3D roto- translation equivariant attention networks. Advances in neural information processing systems, 33: 1970-1981, 2020.
13

---

Published as a conference paper at ICLR 2026
Mikel Galar, Alberto Fernández, Edurne Barrenechea, Humberto Bustince, and Francisco Herrera. An overview of ensemble methods for binary classifiers in multi-class problems: Experimental study on one-vs-one and mee-vs-all schemes. Pattern Recognition, 44(8):1611-177, 2011.
Mario Geiger and Tess Smidt. <3nn: Euclidean neural networks. arXiv preprint wrX:v:2007000009 2022.
Simon Markus Geisler, Arthur Kosmala, Daniel Herbst, and Stephan Günnemann. Spatio-spectral graph neural networks. Advances in Neural Information Processing Systems, 37:49022 2024.
Robert Gens and Pedro Domingos. Deep symmetry networks. Advances in neural information processing systems, 27, 2014.
Jan E Gerken, Jimmy Aronsson, Oscar Carlsson, Hampus Linander, Fredrik Ohisson, Christoffer Petersson, and Daniel Persson. Geometric deep learning and equivariant neural networks. Artificial Intelligence Review, 56 12): 4605-1166, 2023.
Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In International conference on machine learning, pp- 2263-1272, Pmlr, 2017.
Pablo Gomez, Jennifer Shutter, and Jeffrey N Rouder. Memory for objects canonical and non- canonical viewpoints. Psychonomic bulletin & review, 15(5):9009944 2008
John Gower. Generalized Procrustes analysis. Psychometrika, 40(1):33-55) 1975.
Markus Graf. Coordinate transformations in object recognition. Psychological bulletin, 132(6))920,, 2006.
Jiaqi Han, Yu Rong, Tingyang Xu, and Wenbing Huang. Geometrically equivariant graph neural networks: A survey. arXiv preprint arXiv:2202.07230. 2022.
Irina M Harris, Justin Harris, and Diana Caine. Object orientation agnosia: A failure to find the axis? Journal of Cognitive Neuroscience, 1306))800-812, 2001.
Kaiming He. Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Dan Hendrycks, Norman Mu, Ekin Dogus Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshmi- narayanan. AugMix: simple data processing method improve robustness and uncertainty. In International Conference on Learning Representations, 2020.
Alex Hernández-Grarffa and Peter König. Data augmentation instead of explicit regularization. arXiv preprint 188.v:1806.03852. 2018.
Elad Hoffer, Tal Ben-Nun, Itay Hubara, Niv Giladi, Torsten Hoefler, and Daniel Soudry. Augment your batch: better training with larger batches. arXiv preprint arXiv:1901.09335. 2019.
Snir Hordan, Maya Bechler-Speicher, Gur Lifshitz, and Nadav Dym. Spectral graph neural networks are incomplete on graphs with a simple spectrum. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025.
Kurt Hornik, Maxwell Stinchcombe, and Halbert White. Multilayer feedforward networks are universal approximators. Neural networks, 255):359-366, 1989.
Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning graphs. Advances in neural information processing systems, 33:22118 18-2213,,, 2020.
Max Jaderberg, Karen Simonyan, Andrew Zisserman, et al. Spatial transformer networks. Advances in neural information processing systems, 28, 2015.
14

---

Published as a conference paper at ICLR 2026
Ian T Jolliffe and Jorge Cadima. Principal component analysis: review and recent developments. Philosophical transactions of the royal society A: Mathematical, Physical and Engineering Sci- ences, 374(2065) 2016.
Sékou-Oumar Kaba and Siamak Ravanbakhsh. Symmetry breaking and equivariant neural networks. In NeurlPS 2023 Workshop on Symmetry and Geomelry in Neural Representations, 2023.
Sékou-Oumar Kaba, Arnab Kumar Mondal, Yan Zhang, Yoshua Bengio, and Siamak Ravanbakhsh. Equivariance with learned canonicalization functions. International Conference on Machine Learning, pp. 15546 15546-15566. PMLR, 2023.
Michael Kazhdan, Thomas Funkhouser, and Szymon Rusinkiewicz. Rotation invariant spherical harmonic representation of 3D shape descriptors. In Symposium on geometry processing, volume pp. 156-164, 2003.
Nicolas Keriven and Gabriel Peyré. Universal invariant and equivariant graph neural networks. Advances in neural information processing systems, 32, 2019.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 1412. 2014.
Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, 2017.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloc Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF international conference computer vision, pp. 4015-4026, 2023.
Miltiadis Kofinas, Boris Knyazev, Yan Zhang. Yunlu Chen, Gertjan J. Burghouts, Efstratios Gavves, Cees G. M. Snock, and David W. Zhang, Graph neural networks for learning equivariant represen- tations of neural networks. In The Twelfth International Conference on Learning Representations, 2024.
Risi Kondor and Shubhendu Trivedi. On the generalization of equivariance and convolution in neural networks to the action of compact groups. In International conference on machine learning, pp. 2747-2755. PMLR, 2018
Talia Konkle and Aude Oliva. Canonical visual size for real-world objects. Journal of Experimental Psychology: human perception and performance, 37(1):23, 2011.
Devin Kreuzer, Dominique Beaini, Will Hamilton, Vincent Létourneau, and Prudencio Tossou. Rethinking graph transformers with spectral attention. Advances in Neural Information Processing Systems, 14:21618-22 1122229, 2021.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Hannah Lawrence, Vasco Portilheiro, Yan Zhang, and Sékou-Oumar Kaba. Improving equivariant networks with probabilistic symmetry breaking. In The Thirteenth International Conference on Learning Representations, 2025.
Yann LeCun and Yoshua Bengio. Convolutional networks for images, speech, and time series. The handbook of brain theory and neural networks, 1998.
Moshe Leshno, Vladimir Ya. Lin, Allan Pinkus, and Shimon Schocken. Multilayer feedforward networks with a nonpolynomial activation function can approximate any function. Neural Networks, 6(6):861-867, 1993.
Ron Levie, Federico Monti, Xavier Bresson, and Michael M Bronstein. Cayleynets: Graph con- volutional neural networks with complex rational spectral filters. IEEE Transactions on Signal Processing, 67(1):97-109, 2018.
Heng Li, Zhaopeng Cui, Shuaicheng Liu, and Ping Tan. RAGO: Recurrent graph optimizer for multiple rotation averaging. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15787-15796, 2022.
15

---

Published as a conference paper at ICLR 2026
Yi-Lun Liao and Tess Smidt. Equiformer: Equivariant graph attention transformer for 3D atomistic graphs. In The Eleventh International Conference on Learning Representations. 2023.
Derek Lim, Joshua Robinson, Stefanie Jegelka, and Haggai Maron. Expressive sign equivariant networks for spectral geometric learning. Advances in Neural Information Processing Systems, 36: 16426-16455, 2023a.
Derek Lim, Joshua David Robinson, Lingxiao Zhao, Tess Smidt, Suvrit Sra, Haggai Maron, and Stefanie Jegelka. Sign and basis invariant networks for spectral graph representation learning. In The Fleventh International Conference on Learning Representations, 2023b.
Ya-Wa Wei Bileen Lin, Ronen Talmon, and Ron Levie. Equivariant machine learning on graphs with non- linear spectral filters. In The eiirt Annual Conference on Neural Information Processing Systems, 2024a.
Yuchao Lin, Jacob Helwig, Shurui Gui, and Shuiwang Ji. Equivariance via minimal frame averaging for more symmetries and efficiency. In Forty-first International Conference on Machine Learning. 2024b.
Peter Lippmann, Gerrit Gerhartz, Roman Remme, and Fred A. Hamprecht. Beyond canonicalization: How tensorial messages improve equivariant message passing. In The Thirteenth International Conference on Learning Representations, 2025.
Yongcheng Liu, Bin Fan, Shiming Xiang, and Chunhong Pan. Relation-shape convolutional neural network for point cloud analysis. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pp. 8895-8900 8904, 2019.
David G Lowe. Distinctive image features from scale-invariant keypoints. International journal of computer vision, 60(2):91-110, 2004.
Shitong Luo, Jiahan Li, Jiaqi Guan, Yufeng Su, Chaoran Cheng, Jian Peng, and Jianzhu Ma. Equiv- ariant point cloud analysis via learning orientations for message passing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. (8932-1894) 2022.
George Ma, Yifei Wang, and Yisen Wang. Laplacian canonization: A minimalist approach to sign and basis invariant spectral embedding. Advances in Neural Information Processing Systems, 36: 11296- 17706-11333 1337, 2023.
George Ma, Yifei Wang, Derek Lim, Stefanie Jegelka, and Yisen Wang. A canonicalization perspective on invariant and equivariant learning, Advances in Neural Information Processing Systems, 37: 60936-60979, 2024.
Yao Ma. Suhang Wang, Charu C Aggarwal, and Jiliang Tang. Graph convolutional networks with eigenpooling. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 723 731, 2019.
Stephane Mallat. A theory for multiresolution signal decomposition: the wavelet representation. IEEE transactions on pattern analysis and machine intelligence, (1(7):674-693, 2002.
Haggai Maron, Heli Ben-Hamu, Nadav Shamir, and Yaron Lipman. Invariant and equivariant graph networks. In International Conference on Learning Representations, 2019a.
Haggai Maron, Ethan Fetaya, Nimrod Segol, and Yaron Lipman. On the universality of invariant networks. In International conference on machine learning, pp. 4363 4371. PMLR, 2019b.
Jonathan Masci, Davide Boscaini, Michael Bronstein, and Pierre Vandergheynst. Geodesic convo- lutional neural networks Riemannian manifolds. In Proceedings of the IEEE international conference on computer vision workshops, pp. 37-45, 2015.
Sohir Maskey, Ali Parviz, Maximilian Thiessen, Hannes Stärk, Ylli Sadikaj, and Haggai Maron. Generalized Laplacian positional encoding for graph representation learning. NeurIPS 2022 Workshop Symmetry and Geometry in Neural Reprexentations, 2022.
16

---

Published as a conference paper at ICLR 2026
Francesco Mezzadri. How to generate random matrices from the classical compact groups. arXiv preprint math-ph/0600055, 2006.
Arnab Kumar Mondal, Siba Smarak Panigrahi, Oumar Kaba, Sai Rajeswar Mudumba, and Siamak Ravanbakhsh. Equivariant adaptation of large pretrained models. Advances Neural Information Processing Systems, 36:50293-50309, 2023.
Federico Monti, Davide Boscaini, Jonathan Masci, Emanuele Rodola, Jan Svoboda, and Michael M Bronstein. Geometric deep learning on graphs and manifolds using mixture model CNNs. In Proceedings of the IEEE conference on computer vision and paltern recognition, pp. 5115-5124, 2017.
Christopher Morris, Nils M Kriege, Franka Bause, Kristian Kersting, Petra Mutzel, and Marion Neumann. TUDataset: A collection of benchmark datasets for learning with graphs. arXiv preprint arXiv:2007.08663 2020.
Alhassan Mumuni and Fuseini Mumuni. Data augmentation: comprehensive survey of modern approaches. Array, 16:100258, 2022.
Richard S Palais and Chuu-Lian Terng. A general theory of canonical forms. Transactions of the American Mathematical Sociery, 300(2):771-88) 1987.
Stephen E Palmer. Cannonical perspective and the perception of objects. Attention and performance, 9:135-151, 1981.
Jung Yeon Park, Ondrej Biza, Linfeng Zhao, Jan-Willem Van De Meent, and Robin Walters. Learning symmetric embeddings for equivariant world models. International Conference on Machine Learning, pp. 17372-17389. PMLR, 2022.
Saro Passaro and C Lawrence Zitmick. Reducing SO(3) convolutions SO(2) for efficient equivariant gnns. In International conference on machine learning, pp. 27420-27438. PMLR, 2023.
Stefanos Pertigkiozoglou, Evangelos Chatzipantazis, Shubhendu Trivedi, and Kostas Daniilidis. Improving equivariant model training via constraint relaxation. Advances in Neural Information Processing Systems, 37:83497-83522 2024.
Omri Puny, Heli Ben-Hamu, and Yaron Lipman. Global attention improves graph networks general- ization. arXiv preprint w0Xiv:2006.07846 2020.
Omri Puny, Matan Atzmon, Edward J. Smith, Ishan Misra, Aditya Grover, Heli Ben-Hamu, and Yaron Lipman. Frame averaging for invariant and equivariant network design. In International Conference Learning Representations, 2022.
Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. PointNet: Deep learning point sets for 3D classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 652-660, 2017a.
Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas Guibas. Pointnet++: Deep hierarchical feature learning point sets in metric space. Advances in neural information processing systems, 30, 2017b.
Guocheng Qian, Hasan Hammoud, Guohao Li, Ali Thabet, and Bernard Ghanem. ASSANet: An anisotropic separable set abstraction for efficient point cloud representation learning. Advances in Neural Information Processing Systems, 34:281 19-281380 20000, 2021.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. International conference on machine learning, pp. 8748-8763. PmLR, 2021.
Ladislav Rampášek, Michael Galkin, Vijay Prakash Dwivedi. Anh Tuan Luu, Guy Wolf, and Do- minique Beaini. Recipe for general, powerful, scalable graph transformer. Advances in Neural Information Processing Systems, 35:14501-14515 2022.
17

---

Published as a conference paper at ICLR 2026
Ryan Rifkin and Aldebaro Klautau, In defense of one-vs-all classification. Journal of machine learning research, 10111) 2004.
Evan F Risko and Sam J Gilbert. Cognitive offloading. Trends in cognilive sciences, 20(9):676-688, 2016.
Olinde Rodrigues. Des lois géométriques qui régissent les déplacements d' un système solide dans l'espace, de variation des coordonnées provenant de ces déplacements considérés indépen- damment des causes qui peuvent les produire. Journal de mathématiques pures et appliquées, 5: 380-440, 1840.
Eduardo Cantos-Escriche Ya- Wei Eileen Lin, and Stefanie Jegelka. LieAugmenter: Equivariant learn- ing by discovering symmetries with learnable augmentations. arXiv preprint arXiv:2506.03914 2025.
Kusha Sareen, Daniel Levy, Arnab Kumar Mondal, Sékou-Oumar Kaba, Tara Akhound-Sadegh. and Siamak Ravanbakhsh. Symmetry-aware generative modeling through learned canonicalization. In NeurIPS 2024 Workshop on Symmetry and Geometry in Neural Representations, 2025.
Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E(n) equivariant graph neural networks. In International conference machine learning, pp. 9323-9333 PMLR, 2021.
Johann Schmidt and Sebastian Stober. Tilt your head: Activating the hidden apatial-invariann of classifiers. In Forty-first International Conference Machine Learning, 2024.
Johann Schmidt and Sebastian Stober. Robust canonicalization through bootstrapped data re- alignment. arXiv preprint arXiv:2510.08178, 2025.
Kristof T Schütt, Farhad Arbabzadah, Stefan Chmiela, Klaus R Müller, and Alexandre Tkatchenko. Quantum-chemical insights from deep tensor neural networks. Nature communications, 8(1): 13890, 2017.
Claude Shannon. Communication in the presence of noise. Proceedings of the IRE, 37(1):10-21, 2006.
Ruoqi Shen, Sébastien Bubeck, and Suriya Gunasekar, Data augmentation as feature manipulation. In International conference machine learning, pp.
PMLR, 2022.
Roger N Shepard and Jacqueline Metzler. Mental rotation of three-dimensional objects. Science, 171 (3972):701-708 1971.
Ken Shoemake. Uniform random rotations. In Graphics Gems III (IBM Version), pp. 124-132. Elsevier, 1992.
Connor Shorten and Taghi M Khoshgoftaar. A survey image data augmentation for deep learning. Journal of big data, 6(1):1-48, 2019.
Zakhar Shumaylov, Peter Zaika, James Rowbottom, Ferdia Sherry, Melanic Weber, and Carola- Bibiane Schönlicb. Lie algebra canonicalization: Equivariant neural operators under arbitrary lie groups. In The Thirteenth International Conference Learning Representations, 2025.
Utkarsh Singhal, Ryan Feng, Stella Yu, and Atul Prakash. Test-time canonicalization by foundation models for robust perception. In Forty-second International Conference on Machine Learning, 2025.
Daniel Spielman. Spectral graph theory. Combinatorial scientific computing, 18(18), 2012.
Andrea Tacchetti, Leyla Isik, and Tomaso A Poggio. Invariant recognition shapes neural representa- tions of visual input. Annual review of vision science, 4(1):403-422, 2018.
Behrooz Tahmasebi and Stefanie Jegelka. Generalization bounds for canonicalization: A com- parative study with group averaging. In The Thirteenth International Conference on Learning Representations, 2025a.
18

---

Published as a conference paper at ICLR 2026
Behrooz Tahmasebi and Stefanie Jegelka. Regularity in canonicalized models: A theoretical perspec- tive. In The 28th International Conference on Artificial Intelligence and Statistics, 2025b.
Behrooz Tahmasebi, Melanie Weber, and Stefanie Jegelka. Data Augmentation: A Fourier analysis perspective. In NeurIPS 2025 Workshop on Symmetry and Geometry in Neural Representations, 2025.
Michael Tarr and Steven Pinker. Mental rotation and orientation-ddependenee in shape recognition. Cognitive psychology, 21(2):233-22 282, 1989.
Erik Henning Thiede, Truong Son Hy, and Risi Kondor, The general theory of permutation equivarant neural networks and higher order graph variational encoders. arXiv preprint arXiv:2004.0399) 2020.
Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang, Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation-and translation-equivariant neural networks for 3DD point clouds. arXiv preprint /rXiv:8022.0888 1802.082 19, 2018.
Lloyd N Trefethen and David Bau. Numerical linear algebra. SIAM, 2022.
Roy Velich, Arkadi Piven, David Bensaid, Daniel Cremers, Thomas Dagès, and Ron Kimmel. Learning eigenstructures of unstructured data manifolds. arXiv preprint arXiv:2512.01103 2025.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In International Conference on Learning Representations, 2018.
Clement Vignac, Andreas Loukas, and Pascal Frossard. Building powerful and equivariant graph neural networks with structural message-passing. Advances in neural information processing systems, 33:14143-14155, 2020.
Soledad Villar. David W Hogg, Kate Storey-Fisher, Weichi Yao, and Ben Blum-Smith Scalars are universal: Equivariant machine learning. structured like classical physics. Advances neural information processing systems, 34:28848 288663, 2021.
Ulrike Von Luxburg. A tutorial on spectral clustering. Statistics and computing, 17(4):395-416 2007.
Edward Wagstaff, Fabian B Fuchs, Martin Engelcke, Michael A Osborne, and Ingmar Posner. Universal approximation of functions on sets. Journal of Muchine Learning Reseurch, 23(151); 1-56, 2022.
Rui Wang, Robin Walters, and Rose Yu. Approximately equivariant networks for imperfectly symmetric dynamics. In International Conference on Machine Learning, pp. PMLR, 2022.
Rui Wang, Elyssa Hofgard, Han Gao, Robin Walters, and Tess Smidt. Discovering symmetry breaking in physical systems with relaxed group convolution. In Forty-first International Conference on Machine Learning, 2024.
Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. Dynamic graph CNN for learning on point clouds. ACM Transactions on Graphics (tog), 38(5): 1-12, 2019.
Maurice Weiler and Gabriele Cesa. General E(2)-equivariant steerable CNNs. Advances in neural information processing systems, 32, 2019.
Maurice Weiler, Mario Geiger, Max Welling, Wouter Boomsma, and Taco S Cohen. 3D steer- able CNNs: Learning rotationally equivariant features in volumetric data. Advances in Neural information processing systems, 31, 2018a.
Maurice Weiler, Fred A Hamprecht, and Martin Storath. Learning steerable filters for rotation equivariant CNNs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 849-858, 2018b.
19

---

Published as a conference paper at ICLR 2026
Maurice Weiler, Patrick Forré, Erik Verlinde, and Max Welling. Coordinate independent convolutional isomett and gauge equivariant convolutions on Riemannian manifolds. arXiv preprint 0rXiv:21060060200 2021.
Ruben Wiersma, Ahmad Nasikun, Elmar Eisemann, and Klaus Hildebrandt. DeltaConv: anisotropic operators for geometric deep learning on point clouds. ACM Transactions on Graphics (TOG), 41 (4):1-10, 2022.
Daniel E Worrall, Stephan Garbin, Daniyar Turmukhambetor, and Gabriel J Brostow. Harmonic networks: Deep translation and rotation equivariance. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 5028-5037, 2017.
Zhanghao Wu, Paras Jain, Matthew Wright, Azalia Mirhoseini, Joseph E Gonzalez, and lon Stoica. Representing long-range context for graph neural networks with global attention. Advances in neural information processing systems, 01133266667777 2021.
Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3D ShapeNets: deep representation for volumetric shapes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1912-1920, 2015.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In International Conference on Learning Representations, 2019.
Shuo Yang, Yijun Dong, Rachel Ward, Inderjit S Dhillon, Sujay Sanghavi, and Qi Lei. Sample efficiency of data augmentation consistency regularization. In International Conference on Artificial Intelligence and Statistics, pp. 3825-3853. PMLR, 2023.
Suorong Yang, Weikang Xiao, Mengchen Zhang, Suhan Guo, Jian Zhao, and Furao Shen. Image data augmentation for deep learning: A survey. arXiv preprint arXiv:2204.088 08010, 2022.
Li Yi, Vladimir Kim, Duygu Ceylan, I-Chao Shen, Mengyan Yan, Hao Su, Cewu Lu, Qixing Huang, Alla Sheffer, and Leonidas Guibas. A scalable active framework for region annotation in 3d shape collections. ACM Transactions on Graphics (ToG), 35(6):1-22, 2016.
Zhitao Ying, Jiaxuan You, Christopher Morris, Xiang Ren, Will Hamilton, and Jure Leskovec. Hier- archical graph representation learning with differentiable pooling, Advances in neural information processing systems, 31, 2018.
Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R Salakhutdinov, and Alexander Smola. Deep sets. Advances in neural information processing systems, 30, 2017.
Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In International Conference Learning Representations, 2018.
Yan Zhang, Jonathon Hare, and Adam Prüigel-Benntt.. Learning representations of sets through 20191.
optimized permutations. In International Conference on Learning Representations, 2019a.
Yan Zhang, Jonathon Hare, and Adam Prügel-Bennett. FSpool: Learning set representations with featurewise sort pooling. In International Conference on Learning Representations, 2020.
Zhen Zhang, Jiajun Bu, Martin Ester, Jianfeng Zhang, Chengwei Yao, Zhi Yu, and Can Wang. Hierarchical graph pooling with structure learning. arXiv preprint xrXiv:1911.0599 2019b.
Zhun Zhong, Liang Zheng, Guoliang Kang, Shaozi Li, and Yi Yang. Random erasing data augmenta- tion. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 13001-13008, 2020.
20

---

Published as a conference paper at ICLR 2026
Appendix
Tutorial for Practitioners: Applications of Adaptive Canonicalization
23
A.I One vs. Rest Classifiers
23
A.2 Random Maximization
23
A.3 Construction Details for Anisotropic Nonlinear Spectral Filters
23
A.4
Construction Details for Anisotropic Point Cloud Networks
26
A.5
Truncation Canonicalization
27
B Extended Related Work
29
B.1
Canonicalization
29
B.2 Frame Averaging
31
B.3 Equivariant Architectures
................................
32
B.4 Data Augmentation
33
C Functional Calculus and Spectral Filters
33
D Universal Approximation Theorems
34
D.1 Universal Approximation of Euclidean Fucntions
34
D.2 Universal Approximation of Multi-Sets Functions
34
E Theoretical Analysis of Randomized Prior Maximization
36
Additional Examples of Continuous Prior Maximization
36
F1
Unbounded Point Clouds and Rotations
36
F2 Continuous to Discrete Images with Rotations and Other Image Transformations
37
F3 Discrete Discrete Images with Rotations and Other Image Transformations
37
G
Experimental Details
37
G.1 Illustrative Toy Problems: Grid Signal Orientation Tasks
37
G.2 Graph Classification TUDataset
38
G.3 Molecular Classification on OGB Datasets
38
G.4 ModelNet40 Point Cloud Classification
39
H Additional Experimental Results
39
H.1 Sampling-based VS Optimization- based Implementation
39
H.2 Node-Level Anisotropic Nonlinear Filters
40
H.3 Out-of-Sample Rotation Generalization in Point Clouds
40
H.4 Ablation Studies
41
H.5 Sensitivity Analysis
H.6 Truncation Canonicalization with Pretrained Classifier
42
21

---

Published as a conference paper at ICLR 2026
H.7
Computational Requirement Comparison
42
H.8
Training Stability
43
H.9
Canonicalized Point Clouds
43
H.10
ShapeNet Part Segmentation
22

---

Published as a conference paper at ICLR 2026
A TUTORIAL FOR PRACTITIONERS: APPLICATIONS OF ADAPTIVE CANONICALIZATION
In this section, we present the construction details of the application of adaptive canonicalization to anisotropic geometric networks. We start by describing the one vs. rest classifier and our maximization method.
A.1 ONE VS. REST CLASSIFIERS
binary classifter, i.e., representing the probability of g being in class d vs. not being in class d (Rifkin Klautau, 2004; Galar al., 2011; Allwein et al., 2000). The per-class score is obtained by dd = offa - p%, (9)), where o is sigmoid function. Note that the vector (ii..... .DD is not probability measure, since each entry Úd represents a separate probability, and these probabilities do not sum to one in general. We use binary cross-entropy per class and sum over classes E yd log da
In our setting, each output channel fa o p%.(() (g) E [0. 1]
ya) log(1 - ÿa)), where
ya- 1 on the true class d², and ya for d + d"
A.2 RANDOM MAXIMIZATION
We estimate the prior maximization by searching over the transformation space (e.g., unitary trans- formations per spectral band or rotation for point clouds) and selecting the transformation that maximizes the prior on the output of the network (i.e., one vs. rest in classification). Specifically, we consider a probability measure on the space U, and draw i.i.d samples (u) K from it. The argmax of (ha fa (g)) bu, et,, of Eq. (2) is estimated as the argmax of {ha [a(Ku, (9))
For example, in anisotropic nonlinear spectral filters, draw finite pool of candidate transfor- mations from the Haar measure on M, (Mezzadri, 2006). For anisotropic point cloud networks, the search over rotations can be implemented with quaternion (Shoemake, 1992) or Rodrigues' formula (Rodrigues, 1840). For each input, we evaluate the prior objective for all candidates in the pool and keep only the maximizing orientation when computing the forward pass and gradients. In this way, the prior maximization is implemented as randomized search: we sample set of transformations. apply them in parallel, and pick the one giving the best prior value. Note that this sampling-based maximization is best thought of as one convenient implementation of our framework, and other optimization strategies over Uj could be implemented and plugged in as long as they approximately solve the same maximization problem.
In Appendix E we analyze the approximation properties of this maximization method. Specifically, we show that the randomized maximization method approximates the true maximum in high probability,
We note that prior maximization can be strengthened by locally refining the argmax sample point u/. Specifically, initialize with random transformations drawn from the probability measure, perform the sampling-basee descent to further decrease the objective locally. If 2/ is manifold embedded in Fuclidean space, one can apply the vanilla Fuclidean gradient descent and then project the obtained transformation back to U, or apply Riemannian gradient descent (Absil et al., 2008) to stay on U. We summarize the random maximization in Algorithm 1. In our experiments, we use 32 sampled transformations for anisotropic nonlinear spectral filters and use 50 sampled transformations for point clouds.
prior maximization to select a candidate, and then run a few steps gradient
A.3 CONSTRUCTION DETAILS FOR ANISOTROPIC NONLINEAR SPECTRAL FILTERS
In spectral methods for graphs, we use eigenvectors learning. However, these eigenvectors are not uniquely defined. For each eigenvector we can flip its sign and obtain another eigenvector of the same eigenvalue. Moreover, when an eigenvalue has multiplicity larger than one, any orthogonal basis of its eigenspace gives a valid set of eigenvectors. This means that the same graph lead to different eigenvectors depending numerical details. Hence, a spectral graph neural network that takes these eigenvectors as an input may give different outputs for the same graph, which affects stability and can reduce performance.
core component for graph representation
the eigensolver or
23

---

Published as a conference paper at ICLR 2026
Algorithm 1 Random maximization
Input: Input g. backbone network f, scalar prior h(r)), sampler Sample_Up for u P sampled from a probability measure over 21 number random samples K, gradient descent step @D_Step
function Random Maximization(g. f, Sample_ U, K, GD_ SD_ptep)
{ul ..... u { < Sample_Up (()
" - arg max ho (Nx ux)
"' -D__tep((")
return 2
Since the model should depend only on the graph and node features, and not on arbitrary choices of cigenvector bases, i.c., it should be invariant the choice of the eigenbasis, it is important to remove such ambiguities. In this work, we focus on graph classification tasks with D classes and study how to resolve eigenbasis ambiguities using prior maximization adaptive canonicalization.
Setup and Eigendecoomossitinn. To study this, work in the following setup. Consider graph G = ([N), S) where [N] is the set of N vertices, A E NxN is the adjacency matrix, and S NxT is array of node features (row n. is the T-dimensional feature at node n.). We consider the normalized graph Laplacian C I- D AD- as the graph shift operator (GSO) in our experiments, where D is the diagonal degree matrix with diagonal entries (d;); E RN,, where d; is the degree of node i. The cigendecomposition of L is given by L = ((C) A (G) (G)), where (G) (v:); E gN xN is an orthogonal matrix of eigenvectors as the columns (i.e., an eigenbasis) and A (G) diag(A ( ) is the diagonal matrix of eigenvalues, where 0 < \ S < AN S 2.
Band Partition and
Eigenbasis Ambiguity. We then group the spectrum into predefined bands with boundaries 6o < 61 <... <
bg contained in [0, 2]. where B N is the total number of bands. The total band [bo, ba] is chosen as a subset of [0, since the spectrum of the normalized Laplacian is guaranteed to lie in this interval?. In implementation,
dyadic
partitioning scheme. Given a decay rate 0 < r < 1, we set
by 0, by = B-k for = l,..., B - ,
1, bB = 2,
and define the k-th band as [bx-1, bx). A larger B and smaller yields narrower bands and hence finer spectral resolution. For example, taking B = and r = 0.5 gives five hands: [0, 0. 021),,, [0.125, 0.25), (0.25, 0.5), [0.5, 1) and 21.
We now make explicit the symmetry we want the model to respect. Recall that the eigendecomposition of the normalized Laplacian eigenvectors and A (G) = diag(Ar, A AN) the eigenvalues. Given the band boundaries by < < bp, we define for each band & the index set (G) == {i E [N] X; E [b bu)). and let vi()) (G) be the submatrix of V(G) whose columns are the eigenvectors (V))kez,(G) Therefore, we can write ( = [v{[') I... |v[[] For each band we denote the associated Paley- Wiener space by X (G) == span[v; i E (G)) Im(V The ambiguity we want to handle comes from changing the orthonormal basis inside each band. Any other basis of the k-th band has the form (G) - Up. where U₁ E (G)) (G) is unitary matrix. Equivalently, we can write a full orthonormal basis of IR whose columns are partitioned into subsequences corresponding to the bands, and each subsequence spans the associated Paley- Wiener space Xx (G)) We can write this condition in matrix form as: R N З (iG) = U(G), where U(G) : diag "BaaguU,..... Un) E NNNN i.e., the a block matrix with diagonal blocks Ug. The set of block-diagonal unitary matrices ( (U(G) diag(U, Un)} constitutes the band hund-preserving transformations, i.e., the
= V(G) A (G)), where (G) (v:), N collects the
Note that any other self-ndjoint GSO can be chosen in our method. For other GSOs, different bands can be chosen to match their spectral range, or to cover subset of the spectrum.
24

---

Published as a conference paper at ICLR 2026
unitary matrices that keep the Paley- Wiener spaces (or bands) invariant. The space of these matrices constitutes the symmetry space in our setting.
Spectral Coefficients and Liaee-Dimesiionll Standardization. The above construction is defined for single graph. but in our setting work with collection of graphs (i.e. a data distribution). The spectral support of the bands, that is, the intervals [b bb)). is fixed across all graphs. However, for given graph G, the k-th band, i.e, the linear span of the eigenvectors whose eigenvalues lie in [bx-1: by),, is subspace that depends on G. Moreover, different graphs can have different numbers of cigenvalues in the same interval, so the dimension of the k-th band is not constant across graphs We denote by (G) the dimension of the k-th band for the graph G.
Given graph signal S E Rgxx the band-wise spectral coefficients defined be ( S
Cx(VE S) :):==[=
R-Mx(G)xT
for k: = 1,..., B. Note that applying a unitary transformation U& E B-Ma(G)xM.(G) on the Paley- Wiener basis and then computing coefficients is equivalent to computing coefficients the original basis and multiplying by U i...
C (VV = U,) U (V( S) C.
In order to apply a task network given by a standard neural network architecture, such as an MLP. we require that all inputs lie in a common vector space of a fixed dimension, independent of the particular graph G. However, the band dimensions M₂ (G) vary across graphs. We therefore introduce a band-wise padding-or-truncant
operator. For each band k. we predefine a target size Js, which hyperparameter. For general coefficient matrix C E RMxx we define Mr. min M, J& and let C C(1 :),.) E MT denote the truncation the first Mj of C. Then, the padding-or- ortunncctittunnngggg operator is defined as PA == C₁ if M = Jk. and otherwise
C
(C) = (J2 ([J--Ma) M
R.XT
Thus, Ps pads with zeros when M < Jr. and truncates when M > JR. Applying this to each band and stacking, we obtain the exee-ddimensionl spectral representation
(V(C)(V) .))
P(V
((G)
S) = B
E J.TT
PP((Cn(V( (C) ,S))
where J = -E Js. Then, it can be fed to a task network V JJXT - RDD across all graphs.
Anisotropic Nonlinear Spectral Filters via Adaptive Canonicalization. We now resolve the band-wise eigenbasis ambiguity by prior maximization adaptive canonicalization. Consider the priors ha(z) = x. Let V : Jxx + R be a task network and we write (-)- = (P) For each class d {I...., D} and each graph-signal pair (G, S), we select band-wise unitary transforms U(d) U(d) (with U( E IR (G) (x(G) by prior maximization:
P. (VC S))
(U)(4)
U Uld)
arg max ha
Va
PP(CC
S))
This is implemented more efficiently by
P SJU ( ))
P S)UU()
(Uf())
U ( arg max ha Va U U(.)
The second implementation is more efficient, as typically > Jg. and the projections C& (v( only computed once before applying the optimization algorithm. Once the maximizing transforms
25

---

Published as a conference paper at ICLR 2026
obtained, the class-d logit is
PI(CI(V ( S)UT"a)
5d d
P SSU
and we apply sigmoid to obtain per-class probability. The training loss is the of D binary cross-entropies (one-vs-rest).
This construction seen anisotropic because Va is symmetryless, and can operate differently along different directions within the same eigenspace.
A.4 CONSTRUCTION DETAILS FOR ANISOTROPIC POINT CLOUD NETWORKS
next apply adaptive canonicalization in the spatial domain, where inputs are point clouds R represented by their (x,, V, coordinates. We focus on point-cloud classification with multiset neural networks.
Multiset Neural Networks and Orientation Sensitivity, A point cloud represented as RNx3 whose N rows points. By multiset neural network we mean mapping V RNX3 R that is invariant to permutations the N points, i.c.,
V (PX) (X) for permutation matrices (0, 1] NNN
Architectures such as DeepSet (Zaheer et al., 2017) and PointNet (Qi et al., 2017a) satisfy this permutation invariance by construction. When these models use the 3D coordinates as input features, they are not invariant to 3D rotations (Wiersma et al., 2022; Qian et al., 2021): for rotation R E (3), the logits produced for X and for its rotated version XR need not coincide. this work, such multiset architectures out-of-the-boo and address this orientation sensitivity by canonicalizing the 3D rotation of the input.
Adaptive Canonicalization over SO(3). Let V Nx3 + RR be a multiset neural network pro- ducing D class logits. In classification, D is the number of classes. We denote (XX (X)())) with V [R + R. Here, V/ is invariant to permutations of the N points, but not invariant to 3D rotations of the point cloud. The prior maximization for each class E (1...., } is performed by
R
arg max ha (Va(RR )), RESO(3)
where class-d prior is ha(x) = x. Once we found the canonical form R.-. the class scores in our anisotropic point cloud network are computed as Sa = (a(XR and are then passed through a sigmoid nonlinearity to obtain class probabilities. The training loss is the of D binary cross- entropies. Importantly, our adaptive canonicalization does not modify the underlying architecture of V, it only evaluates on rotated inputs and selects an orientation. In other words, the canonicalization setting is independent of the specific multiset neural network architecture, Below, we present three widely used multiset neural networks equipped with adaptive canonicalization.
Adaptive Canonicalization Applied to DeepSet (AC-DeepSet). DeepSet (Zaheer et al., 2017) defines permutation eermutation-invarian mappings on arrays via pointwise encoding and summation aggregation. has a universal approximation property (Wagstaff et al., 2022). Given X Nx3 with rows
X1, XN, the DeepSet has the form
N
V (XX)= = Σ
o(xi)
E R
where & and & are MLPs and & (E1....,,E) outputs D logits. Applying adaptive canonicalization, for each class d we select
N
R
= arg max &d RESO(3)
Σ
((
and then compute the class scores as 84 &d
E ((;
26

---

Published as a conference paper at ICLR 2026
Adaptive Canonicalization Applied to PointNet (AC-PointNet).
PointNet (Qi et al., 2017a)
applies a shared point encoder with symmetric pooling operator:
(X) = & (Pool((((1)) (()))...((((,,) (xx)))) E ER.
...,,
where Pool is typically max-pooling and tion selects, for each class d,
outputs D logits. Adaptive canonicaliza-
R
arg max &d (..., RESO(3)
6(xN ))),
and the class score su Ed ( RT),, R
Adaptive Canonicalization Applied to DGCNN (AC-DGCNN), DGCNN (Wang et al., 2019) builds dynamic kNN graph in feature space and applies edge convolution (EdgeConv) before global pooling and a final classifier head. Let X E R be the input shape with rows X, E R Given point features hn R layer (with h x.). DGCNN constructs dynamic kNN graph as N1 (i) kN (f) (h. J==i k), and computes the edge features for neighbors j E N (i) via
1
shared MLP gp:
(lb (h, h, ((()))) E F
The updated node feature is updated by symmetric aggregation h ((+1) Pool "ENT"()) (1) ij Stacking
such EdgeConv block yields features (h. ) DGCNN then form a global shape representation by pooling across points, given by
(concat (n.²),
1! nn(")) E Faaa
X = Pool
followed by an MLP classifier head p: F - RD:
(X) = p(g E RR,
and (X) (wa (X)( We apply adaptive canonicalization by rotating the input coordinates. For each class d E (1...., D) we select
R
arg max (a(XRT)
RESO(3)
and compute the class score as 8d = V (XR(TT) As above, we apply sigmoids to (sa) and train with the sum of D binary cross-entropies (one-vs-rest).
Remark: Canonicalization Inputs vs. Vector- Valued Representations. For PointNet and DGCNN, adaptive canonicalization can also be applied to vector-valued internal representations, e.g., using the Vector Neuron (VN) representation (Deng et al., 2021). In VN representations, features are organized as collections of 3D vectors (rather than scalar channels). This perspective provides additional fiexibility and motivation for our approach: adaptive canonicalization can be applied either (i) at the input level, by rotating X as in the main text, or (ii) at the representation level, by rotating VN feature tensor (or designated VN layer output) before the downstream classifiers. In the latter case, one canonicalizes the orientation of learned geometric representation, which can be advantageous when early layers extract more stable directions than the input itself. In both settings, the core mechanism of adaptive canonicalization is the same: we search over SO(3) to select the rotation that maximizes the chosen prior score.
A.5 TRUNCATION CANONICALIZATION
We introduce truncation-basee bbasdd prior maximization on images as an additional application of our adaptive canonicalization framework. Intuitively, many image classes are invariant under removing uninformative regions: as long as the object of interest remains in the field of view, the class label "symmetry" and apply prior
should change. We consider, in this case, the truncation maximization over this family of transformations to select a canonical truncation. We note that this transformation family
based on a group action.
Theory. We model images elements of a of G in the following way. Given N N, consider
the regular grid the unit square [0, 1]2 given by the pixels Qij [ + [ 2+1) for
27

---

Published as a conference paper at ICLR 2026
O S i,j < N. We define G to be the set of all functions g (R²) such that supp(g) C [0. 112 and 9 is piecewise constant on the grid, i.c., there exist coefficients Ui E [0. 1] such that g(z) = Vs. for all
{Qiy]: }. Since each image 9 E 9 is uniquely determined by its collection of pixel values (vt,)oss,SeN E [0, 1J.² we can identify (via an isometric isomorphism) & with with the box [0, 1] C R We consider the standard Euclidean metric and topology on [0, 1) which is hence compact space. Because G is isometrically isomorphic to [0, 1|m², the set g is also compact with respect to the (IR²) metric (and therefore also locally compact).
The truncation is parameterized by four coordinates collected in a vector u (xL, VT, RR yB) U C [0, 1J3, where we impose the constraints 0 S æ1, S &n S. 1 and 0 S yr S 1/B S 1. We consider the standard Euclidean metric in U. Since 2/ is a closed and bounded subset of [0, 1] is compact.
We now define the truncation transformation. For parameter u (ZL. Vr, ZR. yB) U, we view u as encoding the top-left and bottom-right corners of an axis-aligned rectangle: R(u) [et,; x [yT: yn] C [0, 1J2. As varies in the compact space U, this truncation window moves continuously inside the domain containing all images. Given an image g E G, we define the truncation
by
[g(=), 10.
I R(u), x E R(u).
After truncation, we include a dilation step that maps the truncated window back to the full image domain. Let Tu [0, 1,.1 R(u) be an affine map defined by
x2) = (II + (IR - =I yr + (yB - -r)))))
The function lies in 2 but not in in general. We then project it back onto the finitedimmensional subspace G of piecewise constant functions by the orthogonal projection Q L2(²) + G defined by
N
(Z))(z)
Σ 4==1
Bej 1 (x),
gig = N2 /
g(v)dy-
=
Qu
We then define Ku : 9 + 9 by
(ku(g)) Q(9 1 R(w)) TT).
We now verify that this truncation family satisfies our assumptions. By construction, 24 is compact, and the image space G is compact. In our setting we have K. G. so K. is also compact. The map
(-)(-) :u x G + IC, (u.g) + K., (g) E K. is continuous, and hence, by compactness of the product U4 G, it is uniformly continuous. This implies that [g + (Ng( is equicontinuous. Thus, the truncation transformation family satisfies the conditions for continuous canonicalization.
Implementation. We work with array images X E NxN and use a finite set of admissible truncations. Instead of allowing truncation at fractions of pixels, in practice we only allow truncation at the boundaries of the pixels. We also exclude very small windows, since they are likely to contain only a part of the object to be classified. Formally, this means that we consider a finite transformation set T and endow it with the trivial metric. That is, the distance is 1 between distinct truncations. Hence, T is compact and the truncation map is trivially continuous under this metric, so the attainment arguments for prior maximization adaptive canonicalization apply verbatim.
Concretely,
we parameterize pixel-aligned windows by a side-length s E [smin: smax] C [0.5, 1.0] and discrete top-left corner. For a given s, we set the truncation side length to M(s) [sN] and require
the window to lie inside the image domain, i.e., 1 < in < N - M(s) + 1 and 1 S jo < N 2 (s)++ +1. The corresponding axis-aligned window is ((i,j) E (1,...,,]) NN² in Sis ig + M(s) - jo S j S Jo + (s) 1). The truncation then keeps only the pixels inside the window and zeros out the rest, which yields a truncated patch C(io.io..) (X) E M(s)xx((ss) To keep the input dimension fixed, we then rescale this cropped patch back to the original resolution N x N using standard interpolation scheme. We denote this resizing operator by R. : M()) + NNN, The truncation operator is therefore defined as the composition
T ((to.o..)(X (X) = Rs Ra. (Hoodo)))))) (X)) N*N
28

---

Published as a conference paper at ICLR 2026
The transformation family used in our method is
T (T s [$min: Smax]: (4o. admissible as above),
and random truncations are obtained by sampling s and (oo.Jo) from suitable distribution under these constraints.
Truncation Canonicalization by Prior Maximization. We consider image classification with D classes. Let Vr RNNN - _2_ be a symmetryless neural network and we write (X) =V (Va(X)) For each class d E (1...., D) select the canonical truncation by
T (bo.jo.e) arg max ha(
Va(Vaal V (V((((to.oo..) (x)));
and compute the class-d score sd (a(((onnooo) (X)). The scores (sa) are passed through sigmoids obtain per-class probabilities, and training uses the sum of D binary cross-entropies (one-vs-rest).
(ta.j..e)
B EXTENDED RELATED WORK
We provide an extended discussion of related work for equivariant machine learning.
B.1 CANONICALIZATION
Canonicalization (Babai & Luks, 1983; Palais & Terng. 1987; Ma et al., 2023; 2024) is classical strategy for handling symmetry in data, especially for tasks where invariance or equivariance to group actions is desirable (Gerken et al., 2023). It preprocesses each input by mapping it standard form prior to downstream learning and inference, so that all symmetry-eqqivvllen inputs are treated identically by the subsequent model. There are two common ways for canonicalization: fixed or learned approaches.
Fixed Canonicalization. Fixed canonicalization uses deterministic and often analytic procedures to assign a unique representative to each symmetry orbit. For example, principal component analysis (PCA) (Jolliffe & Cadima, 2016) alignment canonically orients object (e.g., point cloud or molecule) by rotating it so its principal components align with the coordinate axes (Kazhdan al., 2003). Procrustes analysis (Gower, 1975) canonically orients sets of points by finding the optimal rotation, translation, and scale that minimizes squared point-to-point distances to a reference. For sets and graphs, canonicalization can be achieved by reordering nodes, atoms, or features so that isomorphic inputs share single labeling. In spectral methods and spectral graph neural networks where eigenvectors are fundamental (Kipf & Welling, 2017;
Defferrard et al., 20;6a; Von Luxburg, 2007; Belkin & Niyogi, 2003; Dwivedi et al., 2023; Maskey et al., 2022; Bracha et al., 2024; Velich et al., 2025), canonicalization of spectral decomposition (Lim et al., 2023b;a; Ma al., 2023; 2024) addresses eigenbasis ambiguity (Chung, 1997; Spielman, 2012) by processing each eigenspace independently and selecting directions by applying orthogonal or axis-based projections, typically as a graph preprocessing step. An alternative approach is eigenbasis canonicalization via the input signal, where the signal itself used to define canonical spectral representation, making the spectral transformation independent of the arbitrary choice of eigenvectors (Lin et al., 2024a; Geisler al., 2024).
representative eigenvectors
Learned Canonicalization. Learned canonicalization (Zhang et al., 2019a; Kaba et al., 2023; Luo et al., 2022) seeks to overcome the rigidity and inflexibility of fixed rules with trainable mapping that selects representative for each symmetry orbit. The canonicalizer is parameterized (typically as a neural network and trained to produce canonical forms. For example, Kaba et al. (2023) developed a neural network that learns the canonicalization transformation, which enables plug-and-pl equivariance, e.g., orthogonalizing learned features via the Cram-Schmid process (Trefethen & Bau, 2022). Their results show that the learned canonicalizers outperform fixed canonicalizers.
However, Dym et al. (2024) pointed out that regardless of whether the canonicalizations are learned or not, a continuous canonicalization does not exist for many common groups (e.g. S,, SO(d), O(d) on point clouds n 2 d). Therefore, while learned canonicalization improves empirical performance.
29

---

Published as a conference paper at ICLR 2026
it remains generally discontinuous and can induce instability, hinder generalization, limit model reliability on boundary cases, or out-of-distribution data. In contrast, our adaptive canonicalization framework learns the optimal transformation for each input by maximizing the predictive confidence of the network, resulting in a continuous and symmetry preserving mapping. We include a detailed comparison of most related canonicalization work with our method below.
B.1.1 EQUIVARIANCE WITH LEARNED CANONICALIZATIO FUNCTIONS
Recently, Kaba et al. (2023) introduced energy-based canonicalization. The central idea to learn an energy function over samples and group elements, and define the canonicalization minimizing this energy with respect to the group, given a fixed datapoint. Specifically, their energy minimiza- tion is related our prior maximization adaptive canonicalization method. However, there several important conceptual and technical differences between their approach and our adaptive canonicalization.
First, their energy is not the task neural network like in our analysis, but rather some other trainable neural network. Similarly to our approach, training s end-to-end with the task neural network can be seen as canonicalization that depends on the task network, if one considers the full end-to-end architecture consisting both of the energy minimization and the task network. However, the approach in Kaba et al. (2023) does not give continuity guarantees as opposed to our approach. Second, in their work, they consider symmetries based on group actions, while we consider more general setting of canonicalization transformations (or augmentations) that need not be based on groups. This makes our approach much more applicable across different domains. Moreover, in their framework, there is one canonical form for each datapoint and network, while in our approach, each output channel of the network defines a different canonical form of the datapoint. This allows our approach to preserve continuity. Notably, Kaba et al. (2023) do not attempt to study the continuity of the end-to-end predictor.
Another difference in Kaba et al. (2023) is that when training is initialized, the canonicalizing energy 8 is random. This leads each datapoint to be randomly transformed, so the task neural network initially has to perform well at all orientations of the data. This lead the task network to ultimately learn an "average behavior," not specializing in any special orientation but rather performing reasonably well all orientations of the data. In other words, the limited set of trainable parameters has to simultaneously specialize in many orientations, which reduces the network's 's expressive power. In contrast, in our prior maximization approach, from the begimning of training, the network only pays price for not performing well on the single best orientation per datapoint (on which the network performs the best). This encourages the network to specialize on one canonical orientation per datapoint, and not learn an average behavior. Hence, in our approach, all trainable parameters of the task network can focus on performing well only on the sole canonical orientation of each datapoint.
B. CANONICALIZATION AND DATA RE-AR ALIGNLINENTS
As discussed in Appendix B.1.I, in the energy -ased canonicalization framework of (Kaba et al., 2023), the canonicalizing energy is random at initialization. As a result, it leads each datapoint to be randomly transformed and the task neural network initially has to perform well at all orientations. This can lead the task network to ultimately learn an "average behavior," not specializing special orientation but rather performing reasonably well on all orientations of the data. To address this effect, Mondal et al. (2023) biases the canonical transformation of each datapoint to be the identity, assuming that the datapoints in the training set already have small orientation variance. On top of that, Schmidt & Stober (2025) iteratively reduces the orientation variance of the training set by iteratively reorienting datapoints that lead to a large loss. We note that these approaches are rather different from our prior maximization method, and they do not try to address the continuity problem in canonicalization.
any
1.1.3 WEIGHTED CANONICALIZATION
The ensrgy-based canonicalization (Kaba et al., 2023) was further explored and extended (Shu- maylov al., 2025) on symmetries defined by general Lie group actions. Similarly to our work, Shumaylov et al. (2025) also discusses continuity preservation, but their approach is different from ours. In the work of Shumaylov et al. (2025), they define the notion of weighted canonicalization,
30

---

Published as a conference paper at ICLR 2026
which is similar concept the weighted frame introduced by Dym et al. (2024). Here, to each data- point there is an assigned probability measure over the orbit of the datapoint. Namely, the distribution is over the space of data instead of over the group like in the weighted frames of Dym al. (2024). With respect to energy minimization, this approach is not very different from Kaba et al. (2023). The main difference is in the minimization algorithm, which minimizes over the Lie algebra instead of the Lie group. It is important to note that their work does not train the energy end-to-end with the neural network. Hence, in their work, the canonicalization depends on the whole training set, but not on the task network, which is quite different from our approach. Moreover, in their setup, one has to learn an approximation of the data distribution, which is invariant to the group action. This is a highly nontrivial approach to implement. In contrast, our prior maximization is simple and direct.
1.1.4 TEST- CANONICALIZATION
Another recent extension of canonicalization, (Singhal et al., 2025), , explores set of transformations at test time and uses the scoring functions of large pre-trained foundation models like CLIP (Radford et al., 2021) or SAM (Kirillov et al., 2023) to select the most "canonical" representation upon which downstream inference is performed. Note that the work in Singhal et al. (2025) does not involve network retraining, uses the foundation models is, and performs canonicalization entirely at inference by optimizing over transformations. While it achieves strong empirical performance, its canonicalization mapping is not guaranteed to be continuous, and in fact, continuity is not discussed. Therefore, small input changes may cause abrupt switches in the selected canonical view.
A related line of work is inverse transformation search (Schmidt & Stober, 2024), which also performs test-time optimization over transformations to exploit invariances. Their method focuses on the standard action of the special linear group on images (rotations, scalings, and shear transformations), i.e., purely group-based symmetries, and similar to Singhal et al. (2025), does not address continuity, In addition, their method does not train the model simultaneously with the canonicalization. While their energy energy-induced induced confidence is similar to our prior maximiiaaiinnnnn formulation in the classification setting, does not lead to continuity guarantee. In contrast, our work rigorously develops sufficient conditions on the canonicalizer for the canonicalized network to be continuous and have universal approximation property. Specifically, our os.-vs.-all setting a different type of energy that does lead to continuous end-to-end classifiers.
B.2 FRAME AVERAGING
Frame averaging (Puny et al., 2022) achieves equivariance to group symmetries by averaging a network's output over a set of group transformations (known as "frame"). It built on the classical group averaging operator, which guarantees symmetries by summing a function over all group ele- ments. Frame averaging has two main advantages: 1) it allows adaptation of standard non-equivalent network architectures to handle symmetry, similar to canonicalization methods, and 2) it avoids computational intractability of full group averaging, especially for large or continuous groups Recent work (Lin et al., 2024b) proposes minimal frame averaging that attains strong symmetry coverage with small frames. Domain-specific frame averaging methods (Duval et al., 2023b; Atzmon et al., 2022) show that it can be deployed in material modeling and geometric shape analysis. However, it requires a careful selection of a suitable frame. In addition, frame averaging uses a fixed set of transformations independent of the input or task, potentially leading to sub-optimal or less discriminative feature representations. In contrast, our adaptive canonicalization learns the optimal transformation for each input in a data- and network-dependent way, yielding symmetry preserving continuous functions that can improve representation quality and empirical task performance.
related work that addresses continuity is the weighted frame averaging proposed by Dym et al. (2024). They first prove that in many well-known cases, continuous canonicalization is impossible. This does not contradict our work, as in Dym et al. (2024) the canonicalization is function solely of the datapoint, and not the task network. Then, they define a variant of frame averaging, called weighted frame averaging, in which to each datapoint there is an associated probability distribution over the group, and the frame averaging is performed with respect to this measure. This construction yields continuity guarantees. However, their focus is fundamentally different from ours: they study frame averaging while focus canonicalization. Moreover, the weighted frame averaging is function only of the data, not the neural network, as opposed to our method. We next compare our framework weighted frame averaging in more detail.
31

---

Published as a conference paper at ICLR 2026
In our method, data need not come from vector space of fixed dimension (see e.g.. the application for graphs). In contrast, weighted frame averaging requires working with data that comes from vector space of a fixed dimension.
In our work, symmetries need not be based on group representations. Our notion of symmetry is called transformation family, and our "symmetries" are not even required to be invertible or based on group action. See, for example, the image truncation transformation in Appendix A.5, On the other hand, the symmetries in weighted frame averaging are required to be representations of compact groups.
In the framework of weighted frame averaging, the requirement that the weighted frame is robust is sufficient for continuity of the canonicalized function. However, this requirement is quite strong, and constructing robust frames could be challenging. contrast, we achieve continuity of the canonicalized function even though the maximizer in prior maximization need not be continuous in any sense (as a function of the datapoint). The maximizer need not even be uniquely defined. Hence, practitioners can use domains with new symmetries without having to prove any nontrivial mathematical results. In weighted frame averaging, employing the method in new setting typically requires carefully constructing a problem-specific weighted frame and proving that it is robust. It often involves nontrivial mathematical work and there is no general recipe for doing so. In our case, once an architecture and a symmetry setting are chosen, prior- mooommiaiiiztioon adaptive canonicalization is straightforward to implement and does not require additional sophisticated proofs from the practitioner. The only assumption that needs to be checked is continuity of the chosen transformations, which usually easy to verify.
method out of the box new
Robust weighted frames often require averaging the predicting network over many transfor- mations the input. In practice, our prior maximization approach works with fewer random argmax candidates. For comparison, frame averaging for rotations of n points in a 3D point cloud requires an order of n² transformations (where n = 1024 in the ModelNet40 dataset for example). while, in practice, prior maximization performs well with total of 50 random transformations.
These differences imply that framework is more flexible in terms of the data types and symmetry structures it can accommodate, while also imposing a lighter mathematical burden on practitioners who wish apply it. This gives much more freedom for future practitioners, and may lead to wider adaptation of the method.
We note that the earlier work (Basu et al., 2023) proposed a similar idea to weighted frame averaging, but did not study continuity preservation.
B.3 EQUIVARIANT ARCHITECTURES
Equivariant architectures are class of models explicitly designed to respect symmetry groups acting on the data. Formally, a network is equivariant to a group of transformations if, when the input is transformed by a symmetry group action, the output transforms via the same group action. That is, for a group G and a function f. equivariance guarantees f(g x) = g f(x) for all g E G and input x. It has been developed across images (Cohen & Welling, 2016; 2017: Kondor & Trivedi, 2018;
Worrall et al., 2017; Weiler et al., 2018b), graphs (Bronstein et al., 2017: Zaheer et al., 2017; Gilmer al., 2017; Maron al., 201 9a; Kofinas et al., 2024; Thiede et al., 2020; Vignac et al., 2020; Keriven & Peyré, 2019), molecules (Thomas et al., 2018; Brandstetter et al., 2022a; Anderson et al., 2019;
Fuchs et al., 2020; Satorras et al., 2021; Duval al., 2023a; Schütt al., 2017: Liao & Smidt, 2023;
Hordan et al., 2025; Du et al., 2022; Passaro & Zitnick, 2023), and manifolds (Masci et al., 2015;
Monti et al., 2017: Cohen et al., 2019 Weiler et al., 2021). Notably, Maron et al. (2019b) studies the universal approximation property for equivariant architectures. Notably, equivariant networks often demand group-specific designs that rely on group theory, representation theory, and tensor algebra. This can reduce flexibility and raise compute and memory costs (Pertigkiozoglou et al., 2024; Wang et al., 2024; Liao & Smidt, 2023). Making nonlinearities, pooling, and attention strictly equivariant further constrains layer choices and can increase parameter count and runtime. Moreover, imposing symmetry throughout the stack may limit the expressivity when data only approximately respect the assumed symmetry contain symmetry- breakkng noise (Wang et al., 2022; Lawrence et al., 2025). In contrast, our approach handles symmetry by learning an input- and task daskddependent canonical form through prior maximization and applying a standard backbone. By construction, this mapping is
32

---

Published as a conference paper at ICLR 2026
continuous and symmetry preserving. Our adaptive canonicalization removes heavy group- group-specific layers, reduces per-layer equivariant cost, and keeps ordinary nonlinearities and pooling.
B.4 DATA AUGMENTATION
Data augmentation (Shorten & Khoshgoftaar, 2019; Mumuni & Mumuni, 2022; Yang al., 2022) incorporates symmetry by explicitly applying symmetry transformations to training inputs, thereby encouraging the model to make predictions that are stable (or transform predictably) under those transformations. In this way, augmentation promotes invariance or equivariance by exposing the model multiple transformed views of the same input (i.e., broader coverage of the transformation orbit). In sense, augmentation changes the data distribution rather than the model. For example, if the original data consists of images of axis-aligned rectangles, then applying rotation augmentation induces a data distribution containing rectangles at general orientations. Recent work has analyzed augmentation through multiple lenses, including training dynamics, implicit regularization, group- theoretic foundations, and learnable augmentation strategies (Gens & Domingos, 2014; Chen et al., 2020; Yang et al., 2023; Hernández- García & König, 2018; Hoffer et al., 2019: Shen et al., 2022; Dao et al., 2019; Esshose al., 2025; Zhang al., 2018; Zhong et al., 2020; Veies & Taylor, 2017; Hendrycks et al., 2020; Tahmasebi et al., 2025). While both data augmentation and our adaptive canonicalization use transformations, they do so in fundamentally different ways. Augmentation adds transformed samples and implicitly trains a task network to behave consistently across them, so symmetry is encouraged indirectly through the distributional shift entailed by augmentation. Adaptive canonicalization instead selects (for each input) a canonical transformation from the same family by optimizing prior score, and then evaluates the task network on the resulting canonical representative. Thus, augmentation spreads supervision across many transformed versions, whereas canonicalization explicitly reduces transformation ambiguity by mapping each example to a preferred representative. Moreover, canonicalization can be anisotropic, selecting different canonical transformations for different decision heads, which not captured by standard augmentation strategies.
C FUNCTIONAL CALCULUS AND SPECTRAL FILTERS
In this section, we recall the theory of plugging self-adjoint operators inside functions.
Spectral Theorem. Let L be a self-adjoint operator on a finiteddimensional Hilbert space (e.g., NNN with L*). There exists a unitary matrix V and real diagonal matrix A diag(A1...... AN) such that L = VAV" The columns of V form an orthonormal eigenbasis with Le; AaS.
Functional Calculus. For any function f: + C defined on the spectrum o(C) = Ant f(C) = VS(A'', f(A) diag diag(f(A)), ((.)))...., ((Ax)).
Equivalently, we can write the spectral projections P. == uue
f(C) Σ P.
We can plug a self-adjoint matrix into a function by: 1) diagonalize L, 2) apply f to the eigenvalues, and 3) conjugate back.
Take f = 1 for a Borel set 7 C R. The indicator function (c. is an orthogonal projection, since 2 (C) and 12( (C).
Spectral Graph Filters. A graph shift operator (GSO) is a self-adjoint matrix that reflects the graph's connectivity, such as (normalized) graph Laplacian be such a GSO with eigenpairs {(),, i and V =[v...., un]. For a T-channel node signal X E IR NxT and matrix-valle valued frequency response IR + Rd XT the spectral filter
symmetrized adjacency. Let L
N
(C) = Σ
au X g9())
33

---

Published as a conference paper at ICLR 2026
applies the graph convolution theorem (Bracewell & Kahn, 1966) with d' output channel: each Fourier mode 2 is preserved in space, while channels are mixed by g in the spectral domain. the scalar case (T d' 1) with R + R. the spectral filter simply reduces to the cunctional-caleut operator acting on X:
f(a) 2. = Vf(AJV X.
Spectral graph neural networks (Defferrard al., 2016b; Kipf & Welling, 2017; Levie al., 2018) compose such filters with pointwise nonlinearities, using trainable y at each layer.
D UNIVERSAL APPROXIMATION THEOREMS
D.1 UNIVERSAL APPROXIMATION OF EUCLIDEAN FUCNTIONS
Here, we cite a classical result stating that multilayer perceptrons (MLPs) are universal approximators of functions over compact sets Euclidean spaces.
Theorem 14 (Universal Approximation Theorem (Leshno et al., 1993)). Let a R - R be continuous, non-polynomial function. Then, for every M, L E N, compact K. C RRM, continuous function f K + R.,,, and 0, there exist D N, W1 R b1 E RR and W2 E RLXD s.t.
sup [f(x)) W20 (W1 a + b.) S.. =EK
D.2 UNIVERSAL APPROXIMATION OF MULTI-SETS FUNCTIONS
Multisets are sets where repetitions elements are allowed. Formally, multiset of N elements in RJ can be defined set of pairs (x:, where x: denotes point and the number of times the point x appears in the multiset.
Standard universal approximation analysis of multi-set functions goes along the following lines. First, we represent multisets as 2D arrays X E Nxx where each row X R represents one point in the multiset. Note that the same multi-set can be represented by many arrays. In fact, two arrays X and X' represent the same multi-set if and only if one is a permutation of the other.
To formulate this property, let SN be the symmetric group of N elements, i.e., the group permu- tations of N elements. Given E S and X E IR let p(s) denote the permutation of X via By convention, permutations change the order of the N rows of X, and keeps each row intact. Now, X and X' represent the same multi-set if and only if there is a permutation s SS such that X p(s)
Now, for function R RNJ + R"
represent multi-set function, it should be invariant to permutations, i.e., for every E SN we have (()(s)X) y(X). Hence, standard UAT's of multi-set functions are formulated based on the following notion of universal approximation.
Definition 15. Let K. C NxJ be an invariant compact domain, i.e., p(s) X & K. every X E K. and & E Sx. approximator CoCC, IC,,E P) equivariant functions for every invariant function y E Co PRP) and
of invariant functions (K,, IRP) Co DRD) is called invariant universal
0 where is 0 (K,, P) such that for every X E K.
[x) y xX) <.
For example, DeeptSets are universal approximators of invariant Co(K, functions (Wagstaff et al., 2022).
In this section, we describe an alternative, but equivalent, approach model multisets of size N and their universal approximation theorems using the notion of quotient. The motivation is that our main UAT, Theorem 13, is based on the standard symmetryless notion of universal approximation, Definition 3. While it is possible to develop our adaptive canonicalization theory for functions that preserve symmetries, and obtain an analogous theorem to Theorem 13 based on invariant universal approximators, there is need for such complications. Instead, of universal approximation (Definition 3), and directly encode the symmetries in the domain using quotient spaces, as we develop next.
the standard definition
34

---

Published as a conference paper at ICLR 2026
Quotient Spaces. Let X' be topological space, and i y equivalence relation between pairs of points. The equivalence class [xr] of x E is defined to the set
[x]] (y X x y)-
Definition 16 (Quotient topology). X. The quotient set is defined to be
Let X be topological space, and ~ an equivalence relation
X/ 8/~ [[x] X}.
The quotient set is endowed with the quotient topology. The quotient topology is the finest (largest) topology making the mapping x. [xr] continuous. In other words, the open sets B (2) are those sets such that UudeBB [xr is open in X.
The mapping v X + (X/ ~), defined by v(x) = [ar]. is called the canonical projection.
Multi-Sets as Equivalence Classes and UATs. Define the equivalence relation: X ~ Y if there exists E SN such that X = Y. Now, multi-set of N elements can be defined E XXx'J - As opposed to the definition of multisets as sets of pairs (æz, i), the quotient definition automatically gives topology to the sets of multisets, namely, the quotient topology. In fact, be shown that the quotient topology is induced by the following metric.
Definition 17 (Multi-Set Metric). Given [X], YY] E (R
TRR ~) their distance is defined to be Y
d(xx), (Y) = min
X
Y
is easy to see that
[(}] [
min XX sESy
P())
One can show that the above distance is indeed metric, and that the topology induced by this metric exactly the quotient topology, i.e., d metrizes (RNx) / ~).
Theorem 18. The metric d( ((X)[ metrizes the quotient topology IR NJ
set K. NxJ is called invariant for every X and s E SN we have p(s) E K. Consider the quotient space
K// ~ (X) X E K. C R N
We now have the following proposition about the continuity of symmetric functions. Proposition 19. Let xJ be an invariant compact domain. For every continuous invariant mapping K. + R there exists a unique continuous mapping T. (K/ ~ + RRD such that
y = J v. v.
where v is the canonical projection. On the other hand, for every function E Cb(KC/ ~ ~,R"), we have that z a continuous invariant function in CK,
Let K. Nx be invariant domain. we denote
For set of continuous invariant functions N C Co(K, RR)),
(W/ ~=
= t WuIVEN BEEEN C CoCC/ ~ N,
Note that by Proposition 19
(C/G/((")))) ,,RP).
This immediately leads UAT theorem for multi-set continuous functions in which every continuous multi-set function can be approximated by a neural network.
Theorem 20. Let K. C N be an invariant compact domain, and let N(K. RD) C CoCC, R be an invariant universal approximator of Co(IC, R functioo,,,unncii universal approximator (KC/ ~
functions. Then N N RD) is
Note that the above UAT the symmetries are directly encoded in the quotient spaces, and, hence, there is need encode any symmetry in the spaces of functions. Hence, Theorem 20 is based on the standard symmetryless definition of universal approximation Definition rather than the symmetry driven construction of Definition 15. As result, directly theory of adaptive
35

---

Published as a conference paper at ICLR 2026
canonicalization multi-set functions. Specifically, use Theorem 13, where the space A. in the theorem taken as K/ ~ in our above analysis.
Now, we immediately obtain that the set of neural networks where 0 is DeepSet is a universal approximator of the space of continuous multi-set functions.
E THEORETICAL ANALYSIS OF RANDOMIZED PRIOR MAXIMIZATIOO
Consider the randomized maximization method described in Appendix A.2. To understand how well this random maximization approximates the ideal maximization over the full transformation space, we can adopt the analysis from Cordonnier et al. (2024). Their results provide tail bound for approximating a maximum over a probability space by the maximum over random i.i.d. samples.
We first recall the notion of the volume retaining probability space introduced by Cordonnier et al. (2024).
Definition 21 (Volume retaining property (Cordonnier et al., 2024)). Let X C Bd and let P be a probability measure on X. We say that the probability space (X, P) has the (ro, R)-volume retaining Lebesgue measure if there exist constants vo > and >> 0 such that for any r S To and any x E X
P(B(xrr) n X) N KAa (Bxxrr));
where dd is the d-dimensional Lebesgue measure and B(r, r) is the ball center at x with radius r.
In our case, points are randomly sampled from some canonical measure over Uj (i.e., the Haar measure), and in all of our example applications Ug has the volume retaining property.
On a volume retaining space, Cordonnier et al. (2024) prove the following concentration inequality for maxima.
Lemma 22 (Concentration inequality for volume retaining space (Cordonnier et al., 2024)). Ler (X, P) be a probability space with the (ro, voouumeee retaining property and let g X2 R be g-isishhittz For any p 2 exp(-nxrd22), f for any random variables XI...., Xn iid. probability at least it holds
P, with
Kg 2
[I(q//)) TIK
1/d
max g(,, X1) - sup g(z, (xxx,)))l- >l S Sisa
Applying this lemma to setting, g is the output of the task neural network on some class. Since typical neural networks are Lipschitz continuous (e.g., any multilayer perceptron based on a Lipschitz activation funetion), this immediately gives guarantee that our random maximization method approximates the true maximum. We plan to extend this analysis for future work.
Finally, note that when the prior maximization has e (which is random variable), this leads to an additive term e universal approximation. Namely, for any >> 0, any continuous function can be approximated by f pa up error + e instead of in Theorem 6.
F ADDITIONAL EXAMPLES OF CONTINUOUS PRIOR MAXIMIZATION
We first note that when I. is locally compact metric space, functions in Co(C,R)) must be uniformly continuous.
F.1 UNBOUNDED POINT CLOUDS AND ROTATIONS
Let 24 = (3) be the space of 3D rotations, and G = K N the set of sequences of N points in 3 i.c. the space of point clouds. Consider the La metric in G. Consider the rotation g + Ku (g) of the point cloud g by E U.
Let f E Co(g,R Next we show that f must be uniformly continuous. Let > 0. By the fact that vanishes at infinity, there exists compact domain K. C R Nx3 such that for every K. we have ff()) < e/2. By the fact that x X is compact and λ continuous, & is uniformly continuous on x X. Hence, there exists 8, > 0 such that every g, K. with d(g, g") 8. satisfy [
36

---

Published as a conference paper at ICLR 2026
Let x' be the compact space consisting of all point of distance less equal to 6, from K. There exists < 8 < 6g such that every 9.g K' with d(g.g') < 8, satisfy dg)(g), f(g)) <
Now, let g.g' satisfy d(g.g' g') 8'. If one of the point g or g' lies outside X², then both of them lie outside K, so
d(f()), & (1) [l((g'))]
Otherwise, both lie K, so f((()))(())) c. Both cases together mean that f is uniformly continuous.
As result of uniform continuity, (g (x)(( E U is equicontinuous, and this is a setting of continuous prior maximization.
In fact, this analysis shows that whenever 9 is continuous prior maximization.
K. and is continuous in (2, g). then it's corresponding
F.2 CONTINUOUS DISCRETE IMAGES WITH ROTATIONS AND OTHER IMAGE TRANSFORMATIONS
Consider the "continuous" space of images G = CR) and the discrete space IC = IR Nxx of images of x N pixels.
Let l be the unit circle. For E G and E U let π (w)g be the rotation of the image by angle u. To define the discretizing mapping P (TR2 + R consider the partition of [-1, into the N intervals
In 1-1+ 2n/N, -1 2(n + 1)/N),
= 0,.....N-1
Consider the closed linear subspace D(R²) C L2 L2 (R²2 consisting of images that are zero outside []1 and piecewise constant on the squares {Tn x Im NN Now, P is the operator that takes g E (2R2) first orthogonally projects it upon D(R?) get g'. and then returns
P(g) ( 2n/N, -1 2m Bm/N)) N E N
Define the mapping and P is non-expansive (as an orthogonal projection), Ki - NNNN is Lipschitz 1 for every Hence, {g + [(k, (9))) (g))) nez is equicontinuous.
follows: (g) = ((T(()g). By the fact that T. is an isometry for every
As result, the corresponding prior minimization continuous prior minimization.
This setting can be extended to other image deformation based on diffeomorphisms of the domain R²a parameterized by compact spaces U, For example, one can take dilations up to some uniformly bounded scale. More generally, one can consider a compact set 2 of matrices in 2x and define T (x)g(x) = ((UU) for E U4 E (R?) and x E IR?.
F.3 DISCRETE TO DISCRETE IMAGES WITH ROTATIONS AND OTHER IMAGE TRANSFORMATIONS
One can replace g by D(R2) in the above analysis. Since now D(R?) is compact, p must be continuous prior maximization.
The above examples can be naturally extended to additional image transformations, like translations and dilations. Notably, translations and dilations do not form compact group, but still they satisfy the conditions of our theory, which requires no compactness assumptions.
G EXPERIMENTAL DETAILS
In this section, we describe the experimental setups and implementation details used in Section
G.1 ILLUSTRATIVE TOY PROBLEMS: GRID SIGNAL ORIENTATION TASKS
Toy Problem and Experimental Setup. We consider a square grid on the torus with a 2-channel signal. The first channel contains sinusoidal signal aligned with the æ-axis, given by sin(2mx) (2xx
37

---

Published as a conference paper at ICLR 2026
Table 4: Datasets statistics.
Dataset
# Graphs
# Classes
Avg. # Nodes
Avg.# Edges
MUTAG
188
2
17.93
19.79
PTC
344
2
14.29
14,69
ENZYMES
600
32.63
64.14
PROTEINS
1113
39.06
72.82
NCII
4110
2
29.87
32.30
mooli
41127
25.5
27.5
ogbg-molpcha
437929
128
26.0
28.1
ogbg-ppa
158100
37
243.4
2266.1
The second channel depends on the class label: in Class 0 it is aligned with the r-axis, while in Class 1 it encodes sinusoidal signal along the y-axis, sin(2xy/ /T) Independent Gaussian noise with variance o² added to each channel. In addition, introduces an additional challenge by spatially restricting the support of the channels. The grid is vertically partitioned into two disjoint halves. The first channel is supported only on the left half. The second channel supported only on the right half. The task is to decide if the frequency at the two channels is in the same orientation. The grid size is fixed at 402, the sinusoidal period is set to T 20, and the noise level is chosen as = 0.1. We generate 1000 samples. Evaluation is carried out using 10-fold cross-validation.
Competing Methods. The competing methods include: MLP, GCN, GAT, GIN, ChebNet, NLSF, S*GNN, FA+GIN, and OAP+GIN.
Hyperparameters. We use three-layer network with hidden feature dimension chosen from (32, 64, 128) and ReI.U activation functions. The learning rate selected from {10-3 10-1, 10-9). Batch size 100. All models are implemented in PyTorch and optimized with the Adam optimizer (Kingma Ba, 2014). Experiments are conducted on an Nvidia DGX A100. The output of the GNN is then passed to an MLP.
G.2 GRAPH CLASSIFICATION ON TUDATASET
Datasets and Experimental Setup. We consider five graph classification benchmarks from TU- Dataset (Morris et al., 2020): MUTAG, PTC, ENZYMES, PROTEINS, and NCII, The dataset statistic is reported in Table 4. Following the random split protocol (Ma et al., 2019; Ying et al., 2018; Zhang al., 2019b), we partition the dataset into 80% training, 10% validation, and 10% testing. Results are averaged over 10 random splits, with mean accuracy and standard deviation reported.
Competing Baselines. We evaluate on medium-scale graph classification benchmarks from TU- Dataset, using the same set of competing methods in grid signal orientation tasks. The baselines include MLP, GCN, GAT, GIN, ChebNet, NLSF, S*NN,, FA+GIN, and OAP+GIN.
Hyperparameters. The hidden dimension is set to be 128. The models implemented using Py Torch, optimized with the Adam optimizer (Kingma & Ba, 2014). An early stopping strategy is applied, where training halts if the validation loss does not improve for 100 consecutive epochs. The hyperparameters are selected through a grid search, conducted via Optuna (Akiba et al., 2019), with with the learning rate and weight decay explored in the {le lele's le 1e-4,, }, the pooling ratio varying within [0.1, 0.9] with step 0.1, and the number of layers ranging from 2 to 9 in step size of The output representations then passed into an MLP, and predictions are obtained by optimizing cross-entropy loss function. Experiments are conducted on Nvidia DGX A100.
G.3 MOLECULAR CLASSIFICATION ON OGB DATASETS
Datasets and Experimental Setup. We evaluate on larger-scale benchmarks from the Open Graph Benchmark (OGB) dataset (Hu al., 2020) for classification tasks, including egbg-molhi ogbg- molpeba, and ogbg-ppa spbg. Dataset statistics are summarized in Table 4 The evaluation settings are followed by the OGB protocol (Hu al., 2020).
38

---

Published as a conference paper at ICLR 2026
Competing Baselines. For large-scale graph classification, include GCN, GIN, GatedGCN, PNA. GraphTrans, SAT, GPS, SAN, and the camonicalization based variant OAP+GatedGCN as the competing methods. These approaches have previously demonstrated strong performance on OGB benchmarks, and their reported results are taken from prior work".
Hyperparameters. The models are implemented in PyTorch and optimized with the Adam opti- mizer, with training capped at a maximum of 1000 epochs and controlled by an early stopping criterion. The hidden dimension is selected from the set {128, 256, 512), while the number of layers varies from 2 to 10 in steps of 1. Dropout rates are explored within the range [0. 0.1,. ..... 0.5]. the learning rate is tuned within the interval [0. [0.0001, 0.001], and the warmup set as or 10. Additionally, the batch size is chosen from {32, 64, 128, 256} and the weight decay is chosen from {10 10 10 10-6). All
hyperparameters are tuned using Optuna (Akiba et al., 2019). The experiments are conducted on an NVIDIA A100 GPU.
G.4 MODELNET40 POINT CLOUD CLASSIFICATION
Datasets and Experimental Setup. Our evaluation for point cloud classification was carried out on the ModelNet40 dataset (Wu al., 2015), which consists of 40 object categories and total of 12,311 3D models. Following prior studies (Wang et al., 2019; Deng et al., 2021), we allocated 9,843 models for training and 2,468 models for testing in the classification task. For each model, 1,024 points were uniformly sampled from its mesh surface, using only the zy² coordinates of the sampled points. We apply on-the-fly rotation augmentation during training, following Esteves et al. (2018); Deng et al. (2021), such that the dataset size remains unchanged. At test time, each example is rotated by an arbitrary rotation. Note that on-the-ffy augmentation essentially changes the training data distribution during the learning process. The purpose of comparing under rotation protocols is to assess model's invariance to rotational changes.
Competing Baselines. For point cloud classification tasks, we compare our anisotropic geometric method with point cloud approaches, including PointNet and DGCNN architectures, as well as equiv- ariant models based on the vector neuron framework, i.c., VN-PointNet and VN-DGCNN. We further include canonicalization baselines, CN- and CN-DGCNN, and traditional augmentation baselines in which the training set is expanded with pre-generated rotations (PointNet-Aug. DGCNN- Aug) with a factor of five (x5). The experimental results of PointNet, DGCNN, VN-PointNet, and VN-DGCNN are taken from Wang et al. (2019); Deng et al. (2021).
Hyperparameters. We follow the published hyperparameters and training protocol of PointNet and DGCNN. For PointNet, we uses identical channel widths to PointNet (64, 64, 64, 128, 1024). We use Adam optimizer with learning rate 0.001 and batch size 32 with weight decay 1 x 10-4 and dropout 0.3. For DGCNN, each input comprises 1,024 uniformly sampled points, and the k-NN graph uses neighborhood size t 20. DGCNN uses four EdgeConv layers (with per-layer MLPs of sizes 64, 64, 128, 256). We train with stochastic gradient descent (initial learning rate 0.1) and apply a cosine annealing schedule of 0.001. Training runs for 250 epochs with batch size of 32, and we use dropout rate of 0.5 in the fully connected layers.
H ADDITIONAL EXPERIMENTA RESULTS
In this section, we present additional results of our adaptive canonicalization, including experimental trade-offs of sampling-bur and optimization-bant construction, anisotropic nonlinear spectral filters for node-level representation, and out-of out-of-sample rotation generalization for point clouds.
H.1 SAMPLING-BASED VS OPTIMIZATION-BASHI IMPLEMENTATION
To evaluate the trade-offs between sampling- based and our sample-and (optimization- basedd) implementation, we conduct experiments on the TUDataset graph classification benchmarks. Table reports the classification performance of TUDaset under sampling- baseed and optimization-bl based adaptive caninocalization. We see that the optimization-basa implementation consistently performs
39

---

Published as a conference paper at ICLR 2026
better than the sampling-bas based one. While increasing the sampling candidates (from 1x to 5x or 10x) improves the performance, the sumpl-ann-refing strategy is more memory-efficient than massive sampling approaches. Rather than storing and evaluating hundreds of rotation matrices simultaneously, it processes a smaller working set through iterative refinement, reducing memory pressure (Li et al., 2022). In terms of computation time, the sampling-base method grows linearly with the number of candidates, while the optimization-biee based method add small overhead to the inner steps. We see that in practice, modest refinement few steps) surpasses the accuracy of large sampling budgets at a lower time, offering better accuracy-tima trade-off.
Table 5: Comparison of crmpling-basede vs. optimization- based adaptive canonicalization. Classifi- cation accuracy across TUDataset. Sampling methods use different numbers of random candidates, while the optimization approach combines sampling with local refinement via gradient descent.
MUTAG
PTC
ENZYMES
PROTEINS
NCT1
A-NLSF (sampling)
84.23=144
69.05
10.15
82.94
80.64+12
A- A-NLSF (sampling x5)
85.17
72.21
71.59:10
83,57
80.92
A NL.FF (sampling x10)
85.54
72.78
72.42
85.03g12
80.94gas
A-NLSF (optimization)
87.94s09
73.16a12
73.01a0s
85,47au6
82.01
H.2 NODE-LEVEL. ANISOTROPIC NONLINEAR FILTERS
We introduce the adaptive canonicalization applied to spectral graph neural networks for learning graph-level representation in Section 4.1 and Appendix A.3. The adaptive canonicalization can also be applied nove-level representation, where the node-level representation proceeds by mapping the input signal to the spectral domain (Mallat, 2002) in a band-wise manner with an oriented basis within each band's eigenspace, and performing a synthesis step that transforms the learned coefficients back to the node domain.
On a square grid, each x Fourier mode has a corresponding y Fourier mode of the same response. Therefore, standard spectral methods are inherently isotropic as they cannot distinguish between horizontal and vertical directional information. On the other hand, adaptive canonicalization is anisotropic and our method can learn distinct orientations. The resulting spatial operator can therefore implement any directional filter that a convolutional neural network can achieve (Shannon, 2006; LeCun & Bengio, 1998; Freeman et al., 1991; Dagès et al., 2025).
In graph-level tasks, the canonicalized node-level embeddings can serve as the intermediate represen- tation from which graph-level features are derived. Specifically, the resulting node embeddings can be aggregated through standard pooling operations to have a graph-level representation. We evaluate the node-to- to-graph construction TUDataset for graph classification tasks. The results are summarized in Table 6. We see that the node-to-graph construction achieves performance closely aligned with, and in some cases approaching, that of the direct graph-level canonicalization. We attribute the slightly worse performance to the potential pooling loss.
Table 6: Graph classification performance on TUDataset using adaptive canonicalization. Comparison between direct graph-level representations (Graph) and node-to-graph constructions Mode-to-grapphe
MUTAG
PTC
ENZYMES
PROTEINS
NCI1
Node-to graph Graph
87.02
72.14g15
71.26g1a
84.87=0x
81.64-12
87.94ano
73.16s12
73.01
85.47-06
82.01=09
H.3 OUT-OF-SAMPLE ROTATION GENERALIZATION IN POINT CLOUDS
We adopt the 2/ 23S0(3) protocol (Esteves et al., 2018: Deng et al., 2021): training with on-the-fly azimuthal rotations (z-axis) augmentation, and evaluation applies under arbitrary rotations to each test shape. In this setting, we assess out-of-sample rotation generalization by constraining training data rotations while testing on the full rotation group. The classification performance on ModelNet40 under 2/ z3) protocol is reported in Table 7. Standard PointNet and DGCNN drop sharply under this shift. Equivariant vector-neuron variants recover much of the loss, and canonicalization baselines are comparable. Our adaptive camonicalization outperforms both equivariant architecture and canonicalization baselines in both backbones.
40

---

Published as a conference paper at ICLR 2026
Table 7: Classification accuracy on ModelNet40 under z/80(3) protocol.
PointNet
DOCNN
VN-PointNet
VN-DGCNN
CN-PointNet
CN-DGCNN
AC-PointNet
AC-DGCNN
Accuracy
19.6
33.8
77.5
89.5
79.6
88.8
81.4
91.8
H.4 ABLATION STUDIES
We conduct ablation studies the spectral band partitioning and the choice of GSO for A-NLSE, as well as on the impact of different point cloud backbones in our anisotropic point cloud networks.
H.4.1 SPECTRAL BAND PARTITION
In our experiment, adopt a dyadic partitioning scheme (see Appendix A.3). In this section, conduct an ablation using a uniform partitioning of the eigenvalues with the same number of bands and report the graph classification performance in Table We see that using the dyadic partitions performs better than using the uniform partition. tion provided by dyadic bands, which could more effectively isolate hand-wise unitary actions that commute with the chosen GSO. We also note that spectral band design can be realized in more flexible and expressive ways, for example, through attention as in SpecFormer (Bo et al., 2023). Investigating such learned or adaptive band- selection strategies an important direction for future work and may further strengthen our adaptive canonicalization framework.
Table Graph classification performance using uniform and dyadic spectral band partitioning.
MUTAG
PTC
ENZYMES
PROTEINS
NCI1
Uniform
81,36+122
66.20=as
62.84=14
80.01 85,47
79.62+1.
Dyadic
87 87,94
73.16a12
73.01=08
82.01
H.4.2 GRAPH SHIFT OPERATOR
We evaluate the graph Laplacian as an alternative GSO. Table reports the graph classification performance of A-NLSF when instantiated with the graph Laplacian versus the normalized graph Laplacian. We observe that using the normalized graph Laplacian our method yields better performance than the graph Laplacian. We attribute this the properties of the normalized Laplacian: (i) the normalized Laplacian removes degree-related related scaling effects, leading to a comparable spectral domain across graphs with different degree distributions, and (ii) its eigenvalues lie in the fixed interval [0, 2). providing a controlled and interpretable frequency range, and making dyadic partitioning better aligned across different graphs.
Table 9: Graph classification performance of A-NLSF with different GSO.
MUTAG
PTC
ENZYMES
PROTEINS
NCII
Graph Laplacian
83.76210 87.94199
67.23+14
62.60-12
82.64216
78.59
Normalized graph Laplacian
73.1621.2
73.01=08
85,47
82.01
H.5 SENSITIVITY ANALYSIS
To examine the effect of different hyperparameters, we conduct hyperparameter sensitivity study covering grid size, sinusoidal period, noise level, and hidden dimension. For each hyperparameter, we swept over range of values while keeping all other settings fixed. The results of the sensitivity analysis are summarized in Figure 2. Overall, we observe that our method is reasonably robust. For grid size and sinusoidal period, performance remains stable across the tested ranges. For the noise level, small to moderate noise leads to similar performance, with a degradation only when the noise becomes large enough that it effectively corrupts the underlying structure of the data. For the hidden dimension, small dimensions impact the performance, but performance stabilizes once we enter a standard regime of model capacity.
41

---

Published as a conference paper at ICLR 2026
Figure 2: Hyperparameter sensitivity with respect to grid size, noise level, and hidden dimension.
H.5.1 POINT CLOUD BACKBONES
In order to assess the impact of the backbone choice on the perfor- mance of anisotropic point cloud networks, extended experiments to include two additional and widely used point cloud backbones, PointNet++ (Qi al., 2017b) and RSCNN (Liu al., 2019), in addition to PointNet and DGCNN reported in Table 3. We de- note the corresponding variants by AC-PoinLNet++ and AC-RSCNN. The ablation results are reported in Table 10. We see that the choice of backbone does influence the overall point cloud classification perfor- mance. However, observe that our adaptive canonicalization frame- work consistently improves the classification performance across these backbones. Moreover, when comparing methods built the same backbone (e.g., PointNet or DGCNN), our approach outperforms equivariant models, data augmentation, and standard canonicaliza- tion (see Table 3). This indicates that our method is robust across different point cloud backbones and can further benefit from stronger backbones when they are available.
Table 10: Classification re- sults on ModelNet40 for dif- ferent point cloud backbones. Results of competing meth- ods marked with are taken from Deng al. (2021); Luo et al. (2022).
Accuracy
PointNet AC-PoimiNet
74.7"
81.1107
PointNet++
83.0"
AC-PoinNNetes
8744
RSCNN
82.6
AC-RSCNN
87.6103
DGCNN
88.6"
AC-DGCNN
91.6208
H.6 TRUNCATION CANONICALIZATION WITH PRETRAINED CLASSIFIER
We introduce in Appendix A.5 an application of our adaptive canonicalization on truncation prior maximization. We now illustrate the applicability of this setup with pretrained image classifier. Specifically, we take ResNet-18 (He et al., 2016) pretrained on ImageNet (Deng et al., 2009). We freeze the backbone, and train only the classifier using the CIFAR-IO or CIFAR-100 (Krizhevsky et al., 2009) training set. The experiment is conducted with ten independent runs, and the resulting image classification performance reported in the Table 11. We see that truncation- basedd prior maximization improves classification performance over the standard vanilla baseline. This implies that our method enables the model to adaptively select a canonical truncation that enhances downstream performance. In addition, observe that the selected canonical crops tend to tightly focus on the main object while discarding background and irrelevant context. It matches the intuition behind our prior maximization: by optimizing over the truncation family, the model chooses representative transformed image that best aligns with its prior over the class. This experiment demonstrates that adaptive canonicalization framework can be instantiated with truncation symmetry and benefit from off-the-shelf pretrained models. also highlights the potential of transformation families as a practical way improve pretrained models via adaptive canonicalization.
Table 11: Image classification accuracy on CIFAR- 10 and CIFAR-100 using ResNet18 pretrained on ImageNet, with and without truncation canonicalization.
CIFAR-10
CIFAR-100
Vanilla Truncation canonicalization
72.09=10
56.9440x
74.92.06
60.38105
H.7 COMPUTATIONAL REQUIREMENT COMPARISON
Table 12 the training time per epoch with the number of parameters. We see that A-NLSF uses a similar number of parameters the other methods and fewer than the spectral method. Its computational
42

---

Published as a conference paper at ICLR 2026
requirements comparable the other methods and does not rely significantly larger training budget than the competing methods.
Table 12: Running time per epoch(s)/number of parameters.
MUTAG
PTC
INZYMES
PROTEINS
NCII
MLP
0072105K
0.10/114K
0.13/125K
0,59/129K
L01/134K
GCN
0.40116KK
0.66/120K
0.81/137K
1.9//42K
5.84/189K
GAT
062/138K
0.5771998
0.96154KK
2.34/199K
4,33/17K
GIN
0114105K
0.3271063
0.52/107K
0.44105K
1.9/121K
ChecNee
0.790185
1,25/189
L.22191K
364/217K
11. 11.51245KK
RA+GIN
0,57120K
1
L33/126K
2.55/130K
4.33/1
OAP-GIN
0.22/15KK
0,397101K
0,57/19KK
1,20/ F0K
2.36/124
0.48132K
0.39/10X
1,291155K
2,20148KK
4.37/151K
H.8 TRAINING STABILITY
To quantify the training stability of our method, we track the canonical rotations of a subset of 1500 ran- domly chosen training examples in the point cloud classification exper- iment. At each epoch, we measure the mean geodesic distance (3(3) be- tween the canonicalizations between consecutive epochs. Figure reports the mean geodesic distance between epochs. We observe that this distance decreases during the training and then remains stable, indicating that the canonical representatives stabilize with no rapid switchnn..
1.0
distance
1.0
0.0
Egoch
Figure Mean geodesic distance on SO(3) between the canonicalizations between consecutive epochs.
H.9
CANONICALIZED POINT CLOUDS
Figure 4 shows the canonicalized point clouds for the chair class in the point cloud classification experiment. We randomly select 20 examples from this class and visualize them after applying the optimal transformations. We observe that the examples in this class share similar orientation after canonicalization.
Figure 4: The canonicalized point clouds for the chair class.
43

---

Published as a conference paper at ICLR 2026
H. SHAPENET PART SEGMENTATION
To expand our experimental study on point cloud data, we further conduct experiments the ShapeNet part segmentation benchmark (Yi et al., 2016). The dataset consists of 16 shape categories anno- tated with a total of 50 parts, where each category labeled with between two and six parts. Note that our prior maximization adap- tive canonicalization method was naturally suited classification tasks. Extending it to the segmentation task is not trivial, as the seg- mentation task requires predicting a label for each point the point cloud, and applying prior maximization to each point would be com- putationally inefficient. Therefore, in order to adapt our adaptive canonicalization to the segmentation task, we consider the adap- tive canonicalization with the minimal entropy prior summing over nodes. This modification preserves the spirit of the adaptive canon- icalization while making it compatible with per-point prediction. Table 13 reports the segmentation performance. For the PointNet backbone, we see that, similar to the point cloud classification task, the entropy-based adaptive canonicalization yields advantageous segmentation performance compared to equivariant architectures and standard canonicalization baselines. For the DGCNN backbone, our method attains performance comparable equivariant architectures while outperforming existing canonicalization methods. These results demonstrate that our approach has potential beyond classification. Note that one of the main contributions of work is to construct continuous and cemmeetr--reseechee models. In the entropy prior adaptive canonicalization, the continuity property is not straightforward. We plan to investigate the continuity properties of this adapted approach in future work.
Table 13: Part segmentation performance on the ShapeNet part dataset. The metric is re- ported with the average cate- gory mean IoU Results of com- peting methods marked with are taken from Deng et al. (2021); Kaba et al. (2023).
PointNet
62.3"
DGCNN
78.6'
VN.PointNet VN-DGCNN
81.4
CN-PoinNNe CN-DGCNN
73.6-11
78.5=09"
AC-PointNer
76.0aaa
AC-DGCNN 80.9
44


Reviews:
[{"review_id": "BUBVcYsisf", "reviewer": "Reviewer_KLEU", "summary": "The paper introduces adaptive canonicalization, where the canonical map depends on both the function/network and the input: a family $\\rho_f : G \\to K$ such that models have the form $g \\mapsto f(\\rho_f(g))$. Under an equicontinuity condition on ${f \\circ \\rho_f(g)}$, the authors prove a universal approximation theorem: if a base class $\\mathcal N$ is universal on $C_0(K,\\mathbb R^D)$, then ${\\theta \\circ \\rho_\\theta : \\theta \\in \\mathcal N}$ is universal for adaptive canonicalized functions. They instantiate this via prior maximization, where a transformation family $\\kappa_u$ and monotone priors $H=(h_d)$ are used to pick a canonical representative by maximizing $h_d(f_d(\\kappa_u(g)))$ over $u$, making $\\rho_f$ depend on $f$. They show prior maximization yields continuous, symmetry-preserving models and, under a symmetry-preserving construction, that $f \\circ \\rho_f$ can represent all continuous symmetry-preserving functions, again with universal approximation guarantees. Two applications are given: (i) A-NLSF spectral GNNs that resolve eigenbasis ambiguities and allow anisotropic filters, and (ii) adaptive canonicalization for point clouds (AC-PointNet / AC-DGCNN) handling SO(3) while retaining directional information. Experiments on graph benchmarks (TUDataset, OGB) and ModelNet40 show consistent improvements over data augmentation, standard canonicalization, and equivariant architectures.", "questions": "1. In practice you maximize $h_d(f_d(\\kappa_u(g)))$ approximately. If the chosen $u$ is within $\\delta$ of the true maximum, can you bound the resulting deviation in $f\\circ\\rho_f$ or in the universal approximation error (e.g., as an additive $\\mathcal O(\\delta)$ term)?\n\n2. What specific $h_d$ do you use (logit, probability, margin)? Do different monotone choices substantially change performance or optimization stability, and do any non-monotone choices appear useful despite not fitting the theory?\n\n3. Many applications involve non-compact groups (e.g., SE(3)). Under what additional conditions (e.g., priors that decay at infinity, restricted parameter ranges) do you expect the prior maximization construction to remain continuous and symmetry preserving?\n\n4. Since $\\rho_f$ depends on the evolving network $f$, did you observe any instabilities (e.g., rapid switching of canonical representatives) during training? Have you tried freezing $\\rho_f$ after some epochs and training only the classifier/regressor, and if so, how does that affect accuracy?\n\n5. Do you foresee obstacles to applying adaptive canonicalization to other domains (e.g., SE(2) image rotations/scalings, sequence symmetries), or is it mostly an implementation effort at that point?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. **Elegant and general theory.** Adaptive canonicalization and prior maximization are simple to state yet powerful, giving continuity and universal approximation while avoiding classical discontinuity pitfalls.\n\n2. **Strong link to symmetry theory.** The symmetry-preserving construction and characterization of all continuous symmetry-preserving functions via $f\\circ\\rho_f$ provide a satisfying representation-theoretic story.\n\n3. **Practical instantiations.** A-NLSF neatly resolves spectral eigenbasis ambiguities and yields anisotropic filters; AC-PointNet/AC-DGCNN give rotation-invariant yet directionally aware point-cloud models, with solid improvements over data augmentation, standard canonicalization, and VN-based equivariants.\n\n4. **Good positioning in current literature.** The work is clearly framed against discontinuity results for canonicalization and complements other directions like probabilistic frames and probabilistic symmetry breaking.", "weaknesses": "1. **Approximate maximization not analyzed.** The theory assumes exact $\\arg\\max_u$ for prior maximization; in practice one uses discretization or iterative search. The impact of $\\delta$-suboptimal maximizers on continuity and approximation guarantees is not quantified.\n\n2. **Focus on classification tasks.** The prior design and applications are tailored to one-vs-rest classification; how to extend to regression or more structured outputs is left open.\n\n3. **Limited variety of groups in experiments.** The empirical section covers spectral band symmetries and SO(3) for point clouds; it would be interesting to see at least one example with more complex/non-compact groups (e.g., SE(3) in continuous domains), though this is not required for acceptance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VDJ64p2UTu", "reviewer": "Reviewer_dHXs", "summary": "This paper introduces new results on the continuity of canonicalization functions used for architecture agnostic equivariance. Based on this analysis, it proposes an optimization based method for canonicalization that respects continuity. The method is validated experimentally on spectral graph processing and point cloud classification.", "questions": "- An important part of the implementation that was not clear to me is how the optimization is performed for each group. Solving these optimization problems is expected to be costly in general and to introduce significant overhead. This should be discussed more transparently\n- Likewise, I didn't understand which priors are maximized in practice. Can the authors elaborate and clarify that in the main text.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- Understanding the continuity of canonicalization methods is an interesting problem given that these methods are becoming more widely used\n- The theoretical results are sound and rigorous as far as I could check\n- The experimental performance of the proposed method on the tasks presented is competitive\n- The detailed appendix is appreciated", "weaknesses": "- I think generally the paper should do a better job at discussing related works beyond the introduction and appendix\n    - In many canonicalization frameworks, the canonicalization is learned and already depends implicitly on the prediction network, since it is trained using gradients from the prediction network. We can define the combination of the canonicalization architecture, initialization, and optimization process as the \"adaptive canonicalization\", which then becomes a function of the prediction function. In that sense, I don't think \"adaptive canonicalization\" is really a novel concept.\n    - Likewise \"prior maximization adaptive canonicalization\" is closely related to the optimization canonicalization method introduced in Kaba et al. 2023. Similar ideas have also been used and proposed in subsequent papers such as Schmidt & Stober, 2024, Shumaylov et al. 2024, Singhal et al. 2025. I think it is necessary to acknowledge these works better and not overstate the novelty of these ideas\n- The framework is developed with a high degree of generality, which does not refer to group actions until section 2.4. However, in the end the only applications considered are really only group actions. I feel like the analysis should either be simplified to consider the specific case of group actions (with possibly the more general version in appendix) or applications outside group actions should be examined. In the current state, the amount of formalism seems overkill given the problem considered.\n- I think the construction for continuous canonicalization introduced in 2.3 (proved in theorem 9) is closely related to the weighted frames of Dym et al. 2024. Can the authors comment on that and discuss connections in that section of the paper?\n  - If the proposed canonicalization scheme is not a type of weighted frame, it would be great if the authors could better highlight the core ingredient that makes the scheme continuous.\n- More generally, I would appreciate if the authors could discuss their theoretical results more and try to distill them into insightful conclusions\n- I think section 3 on applications should be significantly rewritten to improve ease of understanding and reduce the amount of mathematical formalism. In its current state, this section that introduces applications will be quite difficult to understand for practitioners.\n- Minor corrections\n    - Typos:\n        - line 137: \"vanish at infinity is\" - > \"vanish at infinity if\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "q6EJuYta24", "reviewer": "Reviewer_6Rf6", "summary": "Canonicalization is widely used in equivariant network but it introduces discontinuities that cause many problems. This paper proposes an adaptive canonicalization via prior maximisation to address this problem. The paper also presents applications of the proposal method. The authors theoretically and empirically verify their results.", "questions": "Please see above.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The authors aim to address a key problem in equivariant neural networks: that canonicalization can cause problematic discontinuities. They propose a new, interesting, and well-motivated method. They present solid theory to support their claims.", "weaknesses": "My concern is mostly about the experiments. \n\nThe performance improvement in experiments on OGB looks marginal. For the experiments on OGB, gains are only 0.0114 in ogbg-molhiv, 0.0061 in ogbg-molpcba, 0.0134 in ogbg-ppa, while the standard deviations in ogbg-molhiv are 0.0132/0.0152. The standard deviations on ModelNet40 are not reported.\n\nNo ablation study is presented. For anisotropic nonlinear spectral filters, I quoted, \"our formulation partitions the spectrum into disjoint frequency bands,\" but no experiments were conducted on how to partition the spectrum. Also, \"we consider the GSO as the normalized graph Laplacian,\" and uses an MLP, without rationale or empirical validation. For anisotropic point cloud networks, the authors use backbones DeepSet and DGCNN, but not validate the choice. \n\nThe experiemtns rely on very many hyperparameters but didn't report their sensitivity. In Appendix F, the authors mentioned graid size, sinusoidal period, noise level, hidden dimension, etc.\n\nExperiments on more datasets are encouraged, such as ShapeNet.\n\nThe code is not submitted.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9G6P8sqA3i", "reviewer": "Reviewer_48pC", "summary": "The paper proposes *adaptive canonicalization*, a form of canonicalization that depends on the input and network to alleviate concerns about continuity (i.e., slight changes in the input do not change the canonical form) and about universal approximators. The paper also shows how to use this canonicalization for molecular, protein, and point cloud classification tasks.", "questions": "- No equivariant tasks (i.e., segmentation) were studied within the paper, which is one of the more relevant tasks where such a method should be evaluated.\n- One of the primary benefits of canonicalization-based methods is the adaptation of large pre-trained models in a zero-shot manner to avoid significant re-training cost. However, the experimental designs proposed in the paper require re-design and re-training of the downstream network. It would be great if the authors could add a discussion of this in the paper, along with potential ways to use adaptive canonicalization with pre-trained models.\n- Could you discuss and contrast your proposed method with the following lines of work [1, 2]?\n- How does the distribution of canonical samples look? Does it retain some structure or orientation (or specific rotations, e.g., for images, say for a horse, we get for humans, as described, with legs down), or is there no such orientation once the canonicalization is *adaptive*?\n- Can you provide the average over multiple runs in Table 3? \n- Computational requirements are not discussed, i.e., what was the training budget (time and compute requirements) for A-NLSF vs other methods? It would be unfair if the proposed method requires a significantly higher training budget.\n- How do the theoretical results in this paper compare with the results of [3]?\n\n1. Equi-Tuning: Group Equivariant Fine-Tuning of Pretrained Models. Basu et al., AAAI 2023.\n2. Efficient Equivariant Transfer Learning from Pretrained Models. Basu et al., NeurIPS 2023.\n3. On the Universality of Invariant Networks. Maron et al., ICML 2019.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "- The paper is well-structured, and the motivation is appropriately highlighted.\n- The paper constructively develops the theoretical background to prove the claims.\n- The experiments consistently show improved performance of using adaptive canonicalization for wide-variety of tasks.", "weaknesses": "I have a few minor questions, which are highlighted in the next section.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)