Tensor learning with orthogonal, Lorentz, and symplectic symmetries
Authors: Wilson G. Gregory
Josué Tonelli-Cueto
Nicholas F. Marshall
Andrew S. Lee
Soledad Villar
Venue: ICLR 2026 Poster (2026)
Keywords: equivariant machine learning
tensors
orthogonal
lorentz
symplectic
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: Learn tensors that are equivariant with respect to the orthogonal, Lorentz, and symplectic groups.

Abstract: Tensors are a fundamental data structure for many scientific contexts, such as time series analysis, materials science, and physics, among many others. Improving our ability to produce and handle tensors is essential to efficiently address problems in these domains.
In this paper, we show how to exploit the underlying symmetries of functions that map tensors to tensors. More concretely, we develop universally expressive equivariant machine learning architectures on tensors that exploit that, in many cases, these tensor functions are equivariant with respect to the diagonal action of the orthogonal, Lorentz, and/or symplectic groups.
We showcase our results on three problems coming from material science, theoretical computer science, and time series analysis. For time series, we combine our method with the increasingly popular path signatures approach, which is also invariant with respect to reparameterizations.  Our numerical experiments show that our equivariant models perform better than corresponding non-equivariant baselines.

Published as a conference paper at ICLR 2026
TENSOR LEARNING WITH ORTHOGONAL, LORENTZ, AND SYMPLECTIC S SYMMETRIES
Wilson G. Gregory
Josué Tonelli-Cueto
Dept. of Applied Mathematics and Statistics Mathematical Institute for Data Science Johns Hopkins University
Department of Quantitative Methods
CUNEF Universidad Madrid, SPAIN gosue.tonel www.olli..cutttooaaaa..eoo
Baltimore, MD 21218 wgregor 4 juu. edu
Nicholas F. Marshall Department of Mathematics
Andrew S. Lee
Department of Mathematics and Computer Science Adelphi University Garden City, NY 11530 alee2@adelphi. edu
Oregon State University Corvallis, OR 97331 marsnich& crreniichhrregostttte.....
Soledad Villar
Dept. of Applied Mathematics and Statistics Mathematical Institute for Data Science
Johns Hopkins University
Baltimore, MD 21218 Flatiron Institute
New York, NY 10010
svillar33 juu. edu
ABSTRACT
Tensors are a fundamental data structure for many scientific contexts, such as time series analysis, materials science, and physics, among many others. Improving our ability to produce and handle tensors is essential to efficiently address problems these domains. In this paper, we show how to exploit the underlying symmetries of functions that map tensors to tensors. More concretely, we develop universally expressive equivariant machine learning architectures tensors that exploit that, in many cases, these tensor functions are equivariant with respect to the diagonal action of the orthogonal, Lorentz, and/or symplectic groups. We showcase results on three problems coming from material science, theoretical computer science, and time series analysis. For time series, we combine our method with the increasingly popular path signatures approach, which is also invariant with re- spect reparameterizations. Our numerical experiments show that our equivariant models perform better than corresponding non-equivariant baselines.
INTRODUCTION
Tensors are fundamental mathematical objects that appear broad spectrum of domains (Ballard & Kolda 2025| Landsberg (012). In the natural sciences, tensor-valued data are used to express polarizations (Melrose & Stoneham 1977). permeabilities (Durlofsky 1991). and stresses (Levitas)
al. 2019), for example. Theoretical computer science studies several problems related to tensors, including factorization or decomposition of tensors (Rabanser et al., 2017). and planted tensor models
(Hopkins al.] 2016). For time series analysis, the path signature introduced in Chen [1957] 1958) transforms path data into a sequence of tensors, enabling methods to process time series data that are
invariant with respect reparameterizations (Bonnier et al. 2019 Pfeffer al. 2019 Töth 2025

---

Published as a conference paper at ICLR 2026
In physics, tensors not just multidimensional array of numbers. Tensors also have specific transformation properties under changes of coordinates, so any function of a tensor should obey certain transformation rules. These rules can be expressed as invariances equivariances with respect to group actions. In this work, we show how to parameterize machine learning models that learn tensors and implement these symmetries, and we demonstrate that imposing them improves the learning performance on a variety of problems. The relevant symmetries are given by classical Lie groups acting diagonally on tensors. The groups we study arise naturally in physics and other settings, including the orthogonal group ((d) (which typically appears in coordinate transformations), the indefinite orthogonal group O k s) (which includes the Lorentz group, a fundamental group for special relativity), and the symplectic group Sp(d) (the underlying group in much of classical and quantum mechanics).
We apply the methods to problems in time series prediction, materials science, and theoretical com- puter science. Our equivariant learned models outperform prior static methods and non-equivariant learned models in almost all cases.
Our Contributions. We provide a generic recipe to define equivariant machine learning models mapping from tensors to tensors. To this end, we give explicit parameterizations for polynomials (Sec. 3) and analytic functions with globally convergent Taylor series (Sec. (4) from tuples of tensor inputs to tensor outputs that are equivariant with respect to the orthogonal (Sec. B indefinite orthogonal
(which includes Lorentz), and symplectic groups (Sec. 4. This generalizes the existing results of
Villar et al. (202T) and leverages the tensor invariant theory (Appleby et al. 1987 Jeffreys 1973]
Roe Goodman| (209)) into a format useful for machine learning frameworks. On first reading and for those primarily interested in practical applications, we suggest focusing on Corollary D iin Section
from which our experiments follow.
we consider three disparate applications. For materials science, we use our machine learning model on tensors to learn the relationship between second-order stress and strain tensors of an Oid)-isotoopic Hookean hyperelastic material (Garanger et al.) 2024p. For time series, we focus on their representation via path signatures, which are sequences of tensors of growing order (Chen 1958, [yons et al. 2007). These tensors can be used for any downstream learning tasks. Here, we consider the problem of estimating the path signature from only a few sampled points on the path. Finally, from theoretical computer science we consider the problem of sparse vector estimation. Given a vector space R" and a random orthonormal basis of subspace Rd that contains sparse vector 2g for d < n, can we recover uoy The problem has roots in dictionary learning (Spiclman et al. 2012) and tensor PCA, it is closely related to the spiked tensor model (Montanari & Richard 2014 and has several applications (Qu et al. 2020).
In Section 5
Related work. Many recent theoretical and applied efforts have focused on the implementation of symmetries and other structural constraints in the design of machine learning models. This is the case of graph neural networks Scarselli al. 2008); Maron al. (2019), geometric deep learning Bronstein et al. 2022T;; Weiler et al. (2021). ) and Al for science Zhang el al. (2023). The goal is to design hypothesis class of functions with good inductive bias that is aligned with the theoretical framework of the physical, mathematical, or algorithmic objects it aims to represent. This includes respecting coordinate freedoms (Villar et al. 2023a), conservation laws (Alet et al. (02T), or internal symmetries (e.g. in the implicit neural representations framework (Lim et al. (023)). D). Symmetries have also been used to provide interpretability to learned data representations (Suau et al. 2023
Gupta al. 2023). Mathematically, it has been shown that imposing symmetries can improve the generalization error and sample complexity of machine learning models (Elesedy 2021b] Wang el al.
2021b Elesedy 2021a Bietti et al. 2021 Petrache & Trivedi 2024 Tahmasebi & Jegelka 2023
Huang et al. 2024).
A variety of methods can be used for implementing invariances or equivariances, including group convolutions (Cohen & Welling 2016 [017]] Wang et al., (20212)) irreducible representations (Fuchs)
et al. 2020 Kondor 2018; Weiler et al. 2018 18 Cohen al.] 2018: Weiler & Cesal 2019 Simeon
& De Fabritis 2023), constraints on optimization (Finzi et al., (021), )), canonicalization (Kaba et al.
2023), and invariant theory (Gripaios al. |02 Haddadin| [022] Villar et al., 2021 Blum-Smil &
Villar [023]] Villar et al.] [023b] Blum-SSmin et al. 2002). This work is closer to the line of research
that constructs explicit equivariant functions from invariant features.
2

---

Published as a conference paper at ICLR 2026
Closest to us is the work of Kunisky, Moore, and Wein on tensor cumulants (Kunisky al. (024);
and the works parameterizing equivariant tensor functions using Clebsch- Clebsch-Gordan methods and spherical harmonics. including e3nn (Geiger & Smidt 2022). escnn (Cesa et al. 2022). , and the recent work of Domina et al. 2025). In Kunisky et al. (2024), it is shown that -inaainnai polynomials symmetric tensors can be turned into (d)) (dd)-invarian polynomials over general tensors by symmetrizing over (d). They do not consider learning applications, tensors of different orders or parities, nor indefinite orthogonal groups or the symplectic group.
The works in Geiger& & Smidt (2022); Cesa el al. (2022); Domina et al. (2025) implement equivariant machine learning models on tensors using representation theory. To do so they decompose tensors into irreducible representations and use Schur's 's lemma to parameterize the equivariant linear maps, If the orders of the intermediate tensor layers are large enough, then these models can parameterize polynomial equivariant functions of arbitrary degree (and are universal in a Stone- Weieesttasss sense). The method we introduce here also parameterizes equivariant tensor polynomials of arbitrary degree, but does so using invariant theory results instead of irreducible representations, Our parameterization for the invariant and equivariant functions does not require the computation of the Clebsch-Gordan coefficients. The Clebsch Goredd-Grrdnn---ee based methods in Geiger & Smidt (022); ; Cesa et al. (2022); Domina et al. (225)) are specific for SO(d) and O(d) for d = 2.3, whereas our method applies to other groups as well. We remark that those methods in Theorems and 2, but they are comparable to our Corollaries I and 3 (which require the inputs to be vectors but are applicable to O(d), the Lorentz group, the symplectic group). In summary. the computational and approximation power should be equivalent, however, the parameterization is different and the mathematical techniques used to arrive the parameterization are also different.
memory efficient than our general formulation
Another related work Pearce-Crump 2023) characterizes neural networks that are (d), (O(d), and Sp(d) equivariant, but only for the case of functions whose input is a tensor power of R" and whose output is tensor power of RR.. Finally, some other works use outer products and contractions of Cartesian tensors to capture higher order interactions, such as HotPP (Wang et al. 2024), GI-Net (Gregory et al. 2025), and Vector Neurons (Deng et al., 202T). These models build higher order tensors in a point cloud or image setting, while our method exploits shortcuts depending on the type of input to build efficient models.
2 DEFINITIONS
To simplify the exposition, we start by focusing on the case of the orthogonal group before extending the result to the indefinite orthogonal and symplectic groups. We consider the orthogonal group ((d). the group of isometries of Euclidean space Idd that fix the origin. It acts on vectors and pseudovectors E Idd in the following way:
v- (M) 7 M(g)v.
(I) where g E (d), (g) E Rdxd is the standard matrix representation of g (i.e. M M(g) M(x) La where La is the identity matrix), and p E {-1, +1} is the parity of If p +1 obtain the standard (dd action on Rd vectors. If = -1 we obtain the (dd action on what in physics are known as pseudovectors. For a common pseudovector, consider rotating Ferris wheel with angular velocity whose direction is given by the right-hand rule. A reflection of the wheel, which will have (g)) = -1 in ( does not change the direction of rotation or angular velocity.
Definition 1 (kip)) -tensors). We define the space of 1 (g) -lensors to be Rdd equipped with the action (d) defined by (I) If v; is a (p..) tensor for i = 1,.... k, then a == 01 .... @ t E (Rdysk sk is a rank-1 k(p) -tensor, where p= = -I Pi and the action of ((d) the diagonal action:
g (m 2 & vn)) (g v1 @ ... @ (g v.))
This definition generalizes to higher rank -tensoos by linearity (see (5) below). The space of -tessos in d dimensions denoted To (p) We will write + or for p when it is clear; for example, Ti
is the space of pseudovectors and Ti (RR,4) +)) is the space of vectors.
Definition 2 (Einstein summation notation). Suppose that is -tensor Let (a) denote the (i1.... ... ) entry of a, where i,,.... & range from to d. The Einstein summation notation is used to represent tensor products] where repeated indices are summed over. In each product, a given
We will identify vectors with co-vectors in the usual way and will distinguish lower vs upper scripts.
3

---

Published as a conference paper at ICLR 2026
index can appear either exactly once, in which case it appears in the result, or exactly twice, in which case it is summed over and does not appear in the result.
For example, in Einstein summation notation, the product of two 2 (i.c., the matrix product ab of two x d matrices a and b) is written as
d > Sake [bles-
bl., = [als [ble, =
Using Einstein summation notation, the action of g E (()) rank-1 tensors can be extended to general tensors by linearity by expressing b E To (RR p) a linear combination of (rank-1) standard basis tensors ii...... Ci ... @ City where [el] = and [ell = 0 for i +
[g bi.i..... jbl lg (e,, Note that the action on ki(p) tensor b can be written as
@ Cj. )l...... [bI 1g enili ... [g ejele
(4)
[g 6] 1 det(M(g)) bi [M] ... (MM
(5)
= for all g E O(d). For example, this notation 2(4)-tensor has the transformation property [g-b]ag
[b),, [M{g)a& [M(g),,, which, in normal matrix notation, is written as g b == M(g) (g))
When multiple tensors are combined, and all their indices appear in the result, refer that the tensor product or outer product. When indices are summed over, we refer to that as the contraction or scalar product. We will further focus specific case of multiple tensor contractions that will refer to as a k-contraction.
Definition (Outer product tensors). Given a E Ta (IR4,
. p) and E Thr (R²,,') the outer product, denoted a & b, is a tensor in T (R4, pp') defined as [a & bli = [4] Mii
We write to denote the outer product of with itself & times and use the convention for k 0 that a 800 X b b=b b. Definition 4 (b-contraction). Given a tensor a Tek+ (Rd .)). the k-contraction of a, denoted (R*.p). (a(a), is the k' -tensor that contracts the 2k first indices follows (in Einstein summation):
(6)
For instance, if a u @ & & y & E T4+1 [Idd .p) then (a) = (u, x)) (v. y) where (u, x) denotes the standard inner product between u and æ. Since k(p) are elements of the vector space (R&) tensor addition and scalar multiplication are defined in the usual way. The final operation on tensors the permutation of the indices.
Definition 5 (Permutations of tensor indices). Given To (RR. p) and permutation o e Sk. the permutation of tensor indices by a, denoted a", is defined by
[a") k |alk--taymmn--tue
(7)
Definition 6 (Invariant and equivariant functions). We say that f Ta (RR,P) + Tw (Rd, ,') is
(dd)-invariant
f(g a) f(a, for all g O We say that f (R Th (Rd p') is -quievqauvari if
(8)
f(g a) = g f(a), for all g g Odd).
(9)
If were instead function with multiple inputs, then the same group element g would act on all inputs simultaneously.
Definition 7 (Isotropic tensors). We say that tensor a E To (R .p) -sotropc if g a = a, for g E Odd).
There are two special tensors. the Kronecker delta, and the Levi-Civita symbol. These tensors (dd)-isotropcc and, as we will show in Appendix C we can construct all (dd)-ssotropic tensors using only Kronecker deltas and Levi-Civita symbols.
4

---

Published as a conference paper at ICLR 2026
Definition 8 (Kronecker delta). The Kronecker delta, 6, is the
- satisfying [e = 1 i =ı j and 0 otherwise. When considered as matrix, it is the identity matrix Ta-
Definition 9 (Levi-Civita symbol). The Levi-Civita symbol, e, in dimension d 2 2 is the O(d)-
isotropic -t-n-tensor such that (c)i 0 if any two of the i1 :1;-.., ld are equal, [e]i,....is +1 if 81,- ; dd is an even permutation of 1, d, and [c]a.....i 1 if 81 . is an odd permutation of 1, ,............................... .....,,a d. For example, when d = 2 this is simply the matrix
:.....,,,
- b.
3 FEQUVARIAAA POLYNOMIAL FUNCTIONS
In this section, we characterize the (d))-qqqivan equivariant polynomial functions mapping multiple tensor inputs to tensor outputs. On first reading and for those primarily interested practical applications, we advise focusing Corollary 1 and Example below.
In what follows we consider functions of n input tensors of orders k; and parities p; for = 1,..., and fixed dimension d. This space is expressed by the cartesian product IT:=1 Tk. (IR (IR) (Th, (RR pi): :..,.., Tk., (Rdd P.)). ). For many practical applications all inputs are of the same type, but we write the theory in this generality to allow for functions that take, for example, L(+)-tensor positions and ((+)-tenso) stresses as inputs.
The theorem below states that every -qqi-eaavvar polynomial from tensors to tensors can be written as a combination of tensor products of the inputs with isotropic tensors followed by Einstein- summation contractions. Each term in the right-hand-side of (TO) should be viewed as combining r of the input tensors with the tensor product, then mapping them to the appropriate output with linear map. Since linear map between tensors can always be written as a tensor product followed by a sequence of contractions (Dimitrienk) 2013 Theorem 5.1), the linear map is implemented by the tensor Cē2 .e........ However, since the function is also -qquivarrar the tensor Cé.....6. must be O(d) (idd)iiotroppc. The theorem states all the tensor equivariant polynomials can be expressed this way.
Theorem 1. Let f [I: Th, (Rd .p:) - The (IRd, ,2) be an Old) -equivvrrnn polynomial function of degree at most R. Then we may write f as follows:
R
s(a an)
Σ
Ieeg (ar, ..... 20 ac, 2 CtI....t,)
(10)
12h S...eesn
where Ce.....,, is an O(d))isotropic (ke,,..,er + kk)
1k') (pe pa) tensoor with order and parity chosen
to be consistent with the output's (ke,
Σq ht, and pe,...... 11 Pt,)-
The proof of Theorem | is given in Appendix B The result is a clean theoretical characterization of ((d)-equivariant polynomial tensor functions with arbitrary order tensor inputs. However, computing large polynomials with all possible low-degree polynomials as in Example 2 in the Appendix. Alternatively, in many applications we only need function that has 1 ((+ -tnssors (i.e. vectors) as input and a k(+) -tensoe as output, and the problem takes on a form more amenable to computation.
tensors is impractical. One option is considering
The condition that Ct2 e. in Theorem 1 is (dd)-isotropic is quite restrictive. Lemma 3 in Appendix C taken from Jeffreys (1773), shows that all isotropic tensors can be constructed from the Kronecker delta 5 (Definition g) and the Levi-Civita symbol (Definition 9).
The following corollary says that when the inputs of the (dd) -quiv-rauvaa function are vectors, and the output is tensor, we can write the function as linear combination of tensor products of the input vectors and Kronecker deltas (and permutations of them), and the coefficients are scalar functions that only depend on the pairwise inner products of the input vectors. The proof is in Appendix C
Corollary 1. Let f II Ti (Rd, +) + Tx (IEd, +) be an -qquivarirai polynomial function. Then, we may write it as
Un) = Σ
Σ
Nhow ((01),))))))1)) (us, @... 8 UJe-. @ 3
1=D EES 15J.S.... zn
(11)
5

---

Published as a conference paper at ICLR 2026
where J (J1..... ................................ - are indices of the input tensors, and the function lt,o,J which depends on the tuple (t, a, is a polynomial of the inner products of the input vectors.
The second factor is permutation of the outer product of / Kronecker deltas and R 21 of the input vectors U1..... Un possibly with repeats. The first sum is over the possible numbers of Kronecker deltas O to 2 where |-] is the floor function. The second sum is over the possible permutations of the k axes, which is smaller than Se when L O due to the symmetries of g as discussed in Appendix D The third sum is over choosing - 2/. vectors from in to Uyy allowing repeated vectors. See Figure for an example. Directly evaluating the function f(r1...., U,)) defined in (II) has
computational complexity O (k'm²' (Qdn² + +dk)) )) operations, where Q is the maximum number of operations needed to evaluate the polynomials (t.a... Thus, evaluating f is only practical for small values of k'; however, since k is the rank of the output tensor, k' {1, 3, 4] already captures many cases of practical interest.
Remark Note that Corollary characterizze polynomial functions, but we allow the ot be more general (e.g. in the class of continuous or smooth functions), then we obtain parameterization of larger class of -'quiivavar functions. the experiments in Section we set the at, to be learnable multi-layer perceptrons (MLPs). We are unsure if characterization of this sort can be stated for all continuous ((d) -equivvraaa functions. However, by the Gione-Weierstreg theorem any continuous function can be approximated by polynomial function to arbitrary accuracy any fixed compact set, so constructing an architecture that can represent equivariant polynomial functions is sufficient approximately represent equivariant continuous functions (see) Yarotsky (2022)).
The following example shows how to express given equivariant polynomial in terms of invariant functions and tensors. longer example appears in Appendix B
Example 1. Let f Ti (Rd, +) To (RR, +) be an Old) equivvarraa polynomial of degree at most 2. By Theorem] we can write f in the form
f(a) to (a
00 & ca) + l (a ()) 12( (2)
(12)
where e, is an Odd)issoroopc (r + ())+-ttensor -tensor for 0, 1,2. Lemma characterizes such isotropic tensors, co 3no, Ci = is trivial and 02 is a linear combination of (892)0 for a E G4 = {01, 021,02,03) where ai (1. 4) (1, 4), (1, 4, 2)2), See Appendix D for the definition of G4 (109..
Thus the final term 12 (2 & 02) is
12 (a 22 (1002)0 82 (002)))) 31 (a, a)o 32a @ a a 8za @ a,
(13)
where the terms associated with 32 and A3 are the same due the symmetry of 82 We conclude
f(a) 30d + fi a) + Bza @
(14)
for some scalars 3o: 31, and B2.
When the input and output both symmetric -tessos,, we get another useful corollary. ---+ + T +) be an -equivvraar function. Then there exists a function f Rdxd + Raxd of diagonal matrices that is permutation equivariant such that for all A E T,2 mm (Rd +), f(A) Q (f(A)) where QAQ is the eigenvalue decomposition.
Corollary 2. Let ff Tee (Id,+)-
_
In other words, an (dd) eqqiivarra function of symmetric matrices can be reduced to a permutation- equivariant function of the eigenvalues. The proof is given in Appendix F
GENERALIZATIONS TO OTHER GROUPS
The results regarding (dd) equivvariat tensor maps from Section 3 are particular case of a more general result involving algebraic groups, such as the Lorentz and symplectic groups we cover here. We work the full generalization in Appendix where give all the details of the proofs.

---

Published as a conference paper at ICLR 2026
Tensor products of inputs
0..0
01
01
8 13
Isotropic tensors
Dot products of inputs
output
41.00
Figure 1: Illustration of the method from Corollary with 4 input vectors in E and -tesoor output. The tensor product of inputs includes all 16 possible tensor products of ordered pairs of input vectors, plus the isotropic Kronecker delta, labeled id. The coefficients at...J shown here (), the identity permutation in Sg.
Recall that (dd) is the subgroup of linear transformations preserving the Euclidean inner product. However, in some contexts, we might interested in preserving other bilinear products on Rd, such the Minkowski inner product
(u, v). u La,d_st
[Is where -=,d->=== (He
-Id-s) for d even, the symplectic product
(u, s)smmp == u. Jav
where Ja = -Tay22 Id/2
The subgroups of linear maps preserving these bilinear products give respectively the indefinite orthogonal group (which is the linear part of the Lorentz group when d and (1,3}) given by
d = [g E GL(RR) 9 Lo = 1odd-oh
(15)
and, when d is even, the symplectic group given by
Sp(d) == {g GL(R*) 1 Jag = Ja}.
(16)
For any of these groups G, consider the modules To (R". x) (Ray where x G - R an algebraic group homomorphism, where the action is given by the linear extension of
g (U1 2 (4) (g v1)
& Uk).
(17)
When G Os,, d s). with # we have four possible X: X+.+ being always equal to X+.- being the sign of the determinant of the bottom-right (d - 8) (d s) submatrix, X-.+ being the sign of the determinant of the top-left x submatrix, and X-. being the determinant of the matrix. Hence, we can represent them by (P1.P2). where p; [-1,+11, When G Sp(d), have that can only the trivial woopphomomorphiise (It follows, for instance, from the representation theory of simple Lie algebras from (Fulton & Harris 2013 Part III) and standard abelianization argument).
Additionally, we have G-equivariant contractions &f T2k+ (Rdd x) + Tw. (RR x) given by
((ad-s) (w) [a] io ... f1.... Ue a alends - 4k
(18)
and
Sp(d) (a) = [a] ia..................... Ja. [Ja]
[Ja]
(19)
Under these notations, we can state the generalization of Theorem [ as follows. Recall that an entire function is function that is analytic and whose Taylor series converges globally at any point.
Theorem 2. Let be either O(s, s) or Sp(d) and f II Th, (Rd, xx) - Tk (R. x') be G-equivariant entire function. Then we may write follows:
f(a....,an)
Σ
Σ
S e (ae, .... at, Ct Cg.m)))))
(20)
r=
1 15h55-55.5n
7

---

Published as a conference paper at ICLR 2026
dataset size
MLP baseline
MLP augmented
TFENN
Ours
5,000
1.586c-4 2.307--6
2.020e-5 2.14e-7
5.3c-5
4.057e-6 3.458e-7
20,000
4.014e-5 1.476e-6
9.365e-6 1.584e-7
3.0e-5
7.748c-7 - 2.911e-7
== 40,000
2.766c-5 8.14e-7
7.516c-6 1.457e-7
1.0c-5
3.310e-6 = 8.448e-7
Table Test error comparison on the TFENN Garanger et al. (2024) dataset, averaged over trials with standard deviation given as +0.vx. The metric the squared Frobenius norm of the difference of the predicted and target -tensren,or so lower values are better. For each row, the best value is bolded. The TFENN errors are the results reported in Garanger et al. (2024).
where Cti....ty E The (RR Xti...,,, X)) is G-isotropic tensor, i.e., a tensor in The (BR Xf1 x") invariant under the action of G; for ke = Ex=1 kr, and Xhimmer Xe.
Using the above theorem and an analogous version of Lemma that characterizes G-isotropic tensors (see Proposition 7 in Appendix Gor (Roe Goodman 2009. Theorem 5.3.3)), we can then prove the following corollary, which generalizes Corollary
Corollary 3. Let G be either O(s, d s) or Sp(d) and f: I Ti (i(R xo) To (R", Xo). with Xo the constant map to I, be G-equivariant entire function. Then we may write f as follows:
TR(
L
Un) = Σ E
E
nhow (10)) (on 8... 1...8u B0s 21 @ )
1= DESI 1SJ1S ISJIS-..Jee SJn-nnnn
(21)
where (.,-)G .) and 0g = Heddelis i O(s, d s), and (.. )G = (., eymp and Oa = -Jale,s if G Sp(d), and a,,a,, is an entire function that depends on the tuple (1, a, J) and whose inputs are all possible inner products between the input vectors and whose output is scalar.
NUMERICAL EXPERIMENTS
With the preceding theory in place,
build machine learning models to learn equivariant tensor functions. We use Corollaries I and 3 which characterize the (dd) and Lorentz-cquivariant functions from vectors to tensors, and Corolarr which characterizes -qquivvrrar functions
from symmetric (++ -tesoors to symmetric ((+) -tnssoss
Stress-Strain Tensors. We consider the problem materials science from Garanger et 2004 We can model an isotropic neo- oo-HHokeean hyperelastic material with the equation
2 (log det F)² log det F + P (tr (FT 3)
(22)
where A, μ are model parameters and a random deformation gradient. For the Cauchy-Green strain tensor C = F F, the second Kirchof stress tensor is given by
S = (22 log det C -1 c-+ pTa
(23)
Thus is a function of C. Both S and C are -tenssos,,,, and the function is -dd)-equivariant, so we parameterize this function by Corollary 2 We enforce permutation equivariance of the function of the eigenvalues by Maron et al. (2019). We compare our model to an MLP baseline, the MLP augmented dataset with random rotations, and the method from Garanger et al.] (2024) which is also equivariant. The results are shown in Ta We can see that for all dataset sizes, equivariant model performs dramatically better than the other models. See Appendix for further model and training details.
baseline trained
Path Signature. Let [0, T" Rdd be continuous path of bounded variation. The path signature S(z) is sequence of tensors 5, (x), S2(x).... where So(æ) and (x) for k > 0 is an <44)-tenson tensor defined as:
[5)())) - b [it. k. dt1 dty.
(24)
8

---

Published as a conference paper at ICLR 2026
Group
Discrete 24
MLP (same width)
MLP (same # params)
MLP augmented
Ours
O(d)
1.336
0.255 + 0.003
0.071 0.001 0.450 0.002
0.007 0.186 0.002
0.002
Lorentz 1.489
1.391 + 0.005
0.005
Table 2: Path signature test performance averaged over 3 trials with standard deviation given as +0.vxx when it is at least le-3. The metric for true truncated signature (x) and predicted truncated signature (xx
E (z), (x((())))))).........)))))))))), ((t)t))))....,,))))))
M Em 2h (z(z) (x(x(t1)....... ................................ (t((t)))..,,,(((())) where |1:1 is the Frobenius norm, so lower values are better. For each row, the best value is bolded.
where t.. de (.. TT) == {(t...... ,x)) E R... < <... <t, < T) is the k-dimensional simplex, and the integral is in the sense of a Riemmann-Stieljes integral.
de
The path signature is useful object when working with path data because it nicely encodes the properties of a path (see Lyons et al. (2007)) For example, if æ; y are regular paths, then S(x) = S if and only if x and y are the same up translation and reparameterization (Chen 1958). This result generalizes to non-regular paths (Hambly & Lyons 2010) and it's fundamental in reconstructing paths from signatures (Pfeffer et 2019 Rauscher et al. 2025). Furthermore, we can approximate any function on a path by a linear function on its path signature Chevyrev & Oberhauser (2022).
We will consider the problem of approximating the path signature from a small sample of points along the path, which if done well allows us to reconstruct the path (see for example (Pfeffer et al. 2019 ). Let P {x j0, T] >> ] } be a family of paths. Let n N be fixed and small, and let 0 03 ... in S T also be fixed. Suppose for x E P that we know x(41),.... ((()),,.,,((())))))) ((n), then the problem is to approximate the truncated signature S (z) = {2x(2), S l S M).
Our baseline for comparison is the discrete version of 24 on the points. It is not hard see that (x(t))).... (tt,)) is an O(d) equivvarat function from n input vectors to a k(-y-tensor, so we can parameterize it with Corollary [1 All the qt functions are learned as a single, shared MLP. Further, also equivariant under the Lorentz and symplectic groups, making it well-behaved in more general physical settings. We compare against three baseline MLP methods, one with the same width as our method, one with the same number of parameters, and one trained on an augmented dataset with 4 random transformations. Further architecture and training details are in Appendix L.2
The results are shown in Table 2
We generate paths for both the orthogonal group and the Lorentz group, see Appendix I.I The learned methods perform better than the fixed naive methods, and our method does the best of all. This method would be a viable first step for processing path data for any downstream learning problem.
Sparse Vector Estimation. We consider the problem of finding a planted sparse vector in a linear subspace. This problem was introduced in (Spielman et al. (2012) in the context of dictionary learning. The works Hopkins et al. (2016) and & Wein (2022) developed ofate-of-the-art methods with theoretical guarantees based on sum-of-squares (SoS). The results in this section suggest (i) the SoS methods outperform learning-bass based methods when the (strong) SoS assumptions are met, (ii) equivariant tensor learning can be used to learn models with good performance when the SoS assumptions are not met, and (iii) equivariant tensor learning outperforms standard machine learning models where no structure is imposed.
The problem is defined as follows. Let E I2 be an (upproximately) sparse vector of unit length, construct to...... 1d-1 E RR. by adding noise to v according to the procedure described in Appendix J.4 then consider S to be an n x d matrix whose columns form a random orthonormal basis of span tig, }. The goal is to recover v from S. The SoS methods consider explicit maps h: (R + Sa that take the rows of S (denoted by a .... a E Rd) and output d x d symmetric matrix. The estimator for e is the multiplication of S times the top eigenvector of h(a1...., an):
i Avee )vve(h(a
(25)
The SoS methods have theoretical guarantees under strict assumptions described in Appendix J.5 Here we learn a function h from data and we compare the learned model with SoS methods and non-equivariant learned baselines for settings that do and do not satisfy the theoretical assumptions (different sampling methods for to...,, Un described in Appendix [.3). Our method uses Corollary 1
9

---

Published as a conference paper at ICLR 2026
sampling
SoS
MLP baseline
Ours (Diag)
Ours
Random
0.610 = 0.009
0.241 0.019
0.493 + 0.005
0.938 0.002
Accept/Reeect
Diagonal
0.448 + 0.012
0.196 0.011 0.196
0.589 1 0.026
0.465 1 0.027
Identity
0.606 + 0.014
0.008
0.351 1 0.065
0.190 - 0.008
Random
0.962 = 0.002
0.242 0.006
0.917 = 0.004
0.937 = 0.002
Bernoulii-Gane Gaussian
Diagonal
0.949 = 0.005
0.205 = 0.013
0.914 = 0.006
0.463 = 0.018
Identity
0.962 = 0.002
0.196 0.009
0.908 = 0.006
0.342 = 0.043
Corrected
Random
0.412 + 0.017
0.239 + 0.012
0.372 + 0.011
0.935 = 0.002
Bernoulli Gaussian
Diagonal
0.288 = 0.018
0.206 0.003
0.550 = 0.026
0,460 0.022
Identity
0.412 = 0.011
0.198 0.005
0.239 + 0.025
0.197 = 0.011
Random
0.526 + 0.020
0.923
0.004
0.437 + 0.034
0.957 0.001
Sannmlll--ademmcheer
Diagonal
0.334 + 0.024
0.864 0.005
0.588 + 0.011
0.903 = 0.004
Identity
0.524 + 0.010
0.845 + 0.006
0.317 + 0.046
0.889 - 0.003
Table 3: Test error comparison on synthetic data averaged over trials (n 100, d = 5, 0.25)
with the standard deviation given by +O.rr.. The metric (u, 6) ranges from 0 to with 1 indicating the estimate identical to the true v. For each row, the best value is bolded. The SoS methods perform best when their assumptions are met, such as identity covariance for the noise vectors, but perform worse than our learned models when using Random or Diagonal covariances. One exception to this trend is the Bernoulli-Gand GGaussinn sampling. This is likely because in expectation the BG satisfies the sparsity requirements of SoS by a large margin (see Appendix P.3). By contrast, the Corrected GaaosslaGa has lower sparsity and the learned models perform better. Finally, all experiments, the baseline MLP generalizes poorly, despite doing well on the training data (Table 7 in Appendix). This is consistent with the claim that enforcing symmetries improves generalization performance.
that in
to learn h (see Appendix J.2 for explanation of why this problem is O(d)-equivvariat) We also include a variant which only takes the norms of each vector as input, instead of all the pairwise cross products. See Appendix I.5 for the details of all models. The results displayed in Table 3
DISCUSSION
This paper provides a full characterization of polynomial functions from multiple tensor inputs to tensor outputs that are equivariant with respect to the diagonal action by classical Lie groups, including the orthogonal group, the symplectic group, and the Lorentz, group.
Our main goal is to define equivariant machine learning models. To the best of our knowledge this is the first work that provides recipe for equivariant machine learning models for tensors at this level of generality. We apply the resulting models to time series data, stress and strain tensors, and sparse vectors. The equivariant models outperform all non-equivariant baseline models, and in the case of the sparse vector problem, the learned models can operate in settings where theoretical guarantees have yet to be developed.
Acknowledgements The authors would like thank Teresa Huang. Ben Blum-Smith, and Daniel Packer for helpful discussions. WGG and JTC are thankful to Jazz G. Suchen for useful sugges- tions regarding Appendix WGG and SV are partially supported by NSF CCF 2212457 and the NSF-Simons Research Collaboration on the Mathematical and Scientific Foundations of Deep Learn- ing (MoDL) (NSF DMS 203 1985). SV is also funded by NSF CAREER 2339682 and NSF BSF 2430292. JTC, NFM, ASL and SV are thankful to the 2022 Mathematics Research Communities program of the American Mathematical Society, particularly the MRC Conference "Data Science at the Crossroads of Analysis, Geometry, and Topology" that took place from May 29th to June 4th of 2022, for allowing them to start collaborating. JTC thanks SV, WGG and the other people at Hopkins for the wonderful working environment when writing this (and other) work(s).
Reproducibility All our code is open source, and it is available at: https://githhut com/ WilsonGreqozyy TensorPo censorPolynome als All datasets are synthetically generated by our code, or public. We also include all necessary experimental details in appendices H and
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Ferran Alet, Dylan Doblar, Allan Zhou, Josh Tenenbaum, Kenji Kawaguchi, and Chelsea Finn. Noether networks: meta-learning useful conserved quantities. Advances in Neural Information Processing Systems, 34:16344-16397, 2021.
G. Appleby, B. R. Duffy, and R. W. Ogden. On the classification of isotropic tensors. Glasgow Mathematical Journal, 185-116, 1987.
Grey Ballard and Tamara G. Kolda. Tensor decompositions for data science. Cambridge University Press, Cambridge, 2025. ISBN 178-1-.-
Alberto Bietti, Luca Venturi, and Joan Bruna. On the sample complexity of learning under geometric stability. Advances in neural information processing systems, 34:18673-18684, 2021.
Ben Blum-Smith and Soledad Villar. Machine learning and invariant theory. Notices of the American Mathematical Society, 70(8): 105)):100-1213, 2023.
Ben Blum-Smith, Ningyuan Huang. Mareo Cuturi, and Soledad Villar. A galois theorem for machine learning: Functions on symmetric matrices and point clouds via lightweight invariant features, 2025. htt.tttss//axii..coom
08097
Patric Bonnier, Patrick Kidger, Imanol Perez Arribas, Cristopher Salvi, and Terry Lyons. Deep signature transforms, 2019. httt:t:t:///ar. http:///ww..v...g/bbbbbbs/ www.:/rgggbb/..o... 1105..119000000 1909.08494.....
Michael M Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint arXiv:2104.13378, 2021.
Gabriele Cesa, Leon Lang, and Maurice Weiler. program to build EeNuvaruiaarr stcerable CNNs. In International Conference on Learning Representations, 2022. URL htttps://pperreiiiee
fnet/for htrr//orummm/ A
Kuo-Tss Chen.
Integration of paths, geometric invariants and generalized baker- hausdorff formula. Annals of Mathematics, 65(1))163-177, 1957. ISSN 0003486X, 19398980. URL
http: / /wwww//www././///////////ww jstor.sooo..oooo wttporgttaeeeeeeeoeeoe
Kuo-Tsai Chen. Integration of paths-a faithful representation of paths by noncommutative formal power series. Transactions of the American Mathematical Society, 89(2):395 1958. ISSN 00029947, 10886850. BRLhttp:: //ww: otttt.....o///taae//9999119
Ilya Chevyrev and Harald Oberhauser. Signature moments to characterize laws of stochas- tic processes. Journal of Machine Learning Research, 23(176):1-42 2022. URL http:
1466. html
Taco Cohen and Max Welling. Group equivariant convolutional networks. In Maria Florina Balcan and Kilian Q. Weinberger (eds.), Proceedings of The 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research, pp. 2990-2999, New York, New York, USA, 20-22 Jun 2016. PMLR. URL
ml press/
V48/cohenci6 6 html
Taco S. Cohen and Max Welling. Steerable CNNs. In International Conference Learning Representations, 2017.
Taco S. Cohen, Mario Geiger, Jonas Köhler, and Max Welling. Spherical CNNs. In International Conference Learning Representations, 2018. URL ptttps::/oppnrevi.eee htt.:www..e.e.eeeeeeeeeeeee forum?
1d=Hkbd5x2Rb.
David A. Cox, John Little, and Donal O' Shea. Ideals, Varieties, and Algorithms: An Introduction to Computational Algebraic Geometry and Commutative Algebra. Springer International Publishing, 2015. ISBN 9783319167213. doi: 10. 10.10077978-3331--1771111
htRl:http://// //xp doi /
To. 1001007/ 978-3-3-3333
167213
Congyue Deng, Or Litany, Yueqi Duan, Adrien Poulenard, Andrea Tagliasacchi, and Leonidas J Guibas. Vector neurons: A general framework for so equvvurrri networks. In Proceedings of the IEFF/CVF International Conference on Computer Vision, pp. 12200-122099 2021.
11

---

Published as a conference paper at ICLR 2026
Yu. I. Dimitrienko. Tensor Analysis and Nonlinear Tensor Functions. Springer Dordrecht, edition, June 2013.
M. Domina, Bigi, P. Pegolo, and M. Ceriotti. Representing spherical tensors with scalar-based machine-learning models. The Journal of Chemical Physics, 163(16):164114,, 10 2025. ISSN 0021-9606. doi: 10.1063/5.02848022 URL https:://di.
0284802
Louis J Durlofsky. Numerical calculation of equivalent grid block permeability tensors for heteroge- neous porous media. Water resources research, 27(5):699-708, 1991.
Bryn Elesedy. Provably strict generalisation benefit for invariance in kernel methods. In A. Beygelz- imer, Y. Dauphin, P. Liang, and Wortman Vaughan (eds.). Advances Neural Information Pro- cessing Systems, 2a.la.
Bryn Elesedy. Provably strict generalisation benefit for invariance in kernel methods. In M. Ran- zato, A. Beygelzimer, Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 17273-17283, Curran Associates, Inc., 2021b. URL /ttpps: http://ws.nnnunrr.n.seeeeeeeeeee cc/paper http://pppppppppeepepeeeeeeeeeee files//aper:2221/
Eile/ 1leeBBf 4tt.aaaeoo0444222255560000000000
Marc Finzi, Max Welling, and Andrew Gordon Wilson. A practical method for constructing equiv-
ariant multilayer perceptrons for arbitrary matrix groups. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning. volume 139 of Proceedings of Machine Learning Research, pp. 3318-3328 PMLR, 18-24 Jul 2021. URL
wtt.:mmm..ppp.s....m w.w.s//e/vvv............ f9/finzi22a..htm
m
Fabian Fuchs, Daniel Worrall. Volker Fischer. and Max Welling. Se(3)-transformers: 3d roto- translation equivariant attention networks. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 1970- 1981. Curran Associates, Inc., 2020. UtL. Files http:///20000aaa2222aeeeeeeae2e3 http://222.2eeeeeaaaa22222222322
www.eerris.ccpppppeee Paper pdf
William Fulton and Joe Harris. Representation theory: a first course, volume 129, Springer Science & Business Media, 2013.
Kévin Garanger, Julie Kraus, and Julian J. Rimoli. Symmetry-enforrcing neural networks with applications to constitutive modeling. Extreme Mechanics Letters, 71:102188, September 2024. ISSN 2352-4336. doi: 11.1066f.em..22241102222
URLhttp:
http:/ /xx doi. dttp:///0.dd....o/0000000.6666..
0. eml 2024. 102188
Mario Geiger and Tess Smidt. c3nn: Euclidean neural networks. arXiv preprint arXiv:2207.09453, 2022.
Wilson G. Gregory, David W. Hogg, Ben and Soledad Villar. Equivariant geometric convolutions for dynamical systems on vector and tensor images. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 38342298) 2025. doi: 10. 00.10998/stt..2224.000 URL https
Maria Teresa Arias, Kaze W. K. Wong,
Ben Gripaios, Ward Haddadin, and Christopher G Lester. Lorentz- and permutation--ivvarinns of particles. Journal of Physics A: Mathematical and Theoretical, 54(15):155201, March 2021. doi: 10. 10.118881751-8122/166 1751 0021882211/bee50
ISSN 1751-8121.
TRLhttp::/ //tp:::/ddxxro doi wtwp://.....io....110......
Sharut Gupta, Joshua Robinson, Derek Lim, Soledad Villar, and Stefanie Jegelka. Structuring repre- sentation geometry with rotationally equivariant contrastive learning. In The Twelfth International Conference on Learning Representations, 2023.
Ward Issa Jereis Haddadin. Invariant polynomials and machine learning. PhD thesis, Cambridge U., 2022.
12

---

Published as a conference paper at ICLR 2026
Ben Hambly and Terry Lyons. Uniqueness for the signature of path of bounded variation and the
reduced path group. Ann. of Math. (2), 171()):10-1167, 2010. ISSN 0003-486X, 1939-999999999 8980. doi:
10.4077aamals:200017...00 URL /tt.ppps///o....or///.00400///00
109
Dan Hendryeks and Kevin Gimpel. Gaussian error linear units (gelus), 2023. URL https:
//arxiv.org/ abs/ 606.6666000 08415
Samuel B. Hopkins, Tselil Schramm, Jonathan Shi, and David Steurer. Fast spectral algorithms
from sum-of-squares proofs: tensor decomposition and planted sparse vectors. In Proceedings
of the Forty-Fighth Annual ACM Symposium Theory Compuding. STOC 16, pp. 178-191, New York, NY, USA, 2016. Association for Computing Machinery. ISBN 4503443333
doi:
1155//297518.29777777 /RLhttss:///o.
org/10. 11tp://000..011111111111115/1//. 1ttp://1..1111289988 28972228899953
Roger A. Horn and Charles R. Johnson. Matrix Analysis. Cambridge University Press, 1990. ISBN 0521386322.
Ningyuan Huang, Ron Levie, and Soledad Villar. Approximately equivariant graph networks. Advances in Neural Information Processing Systems, 36, 2024.
Harold Jeffreys. On isotropic tensors. Mathematical Proceedings of the Cambridge Philosophical Society, 73(1):173-176, 1973. doi: 1011017/00305004100047555
Sékou-Oumar Kaba, Arnab Kumar Mondal, Yan Zhang, Yoshua Bengio, and Siamak Ravanbakhsh. Equivariance with learned canonicalization functions. In International Conference on Machine Learning, pp. -154661556. PMLR, 2023.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017.
Risi Kondor. N-body networks: covariant hierarchical neural network architecture for learning atomic potentials. CoRR, htss/1803001885 2018. URL http://arxxw..or//bb///
01588
Dmitriy Kunisky, Cristopher Moore, and Alexander Wein. Tensor Cumulants for Statistical Infer- ence on Invariant Distributions. In 2024 JEEE 65th Annual Symposiun on Foundations of Computer Science (FOCS), pp. 1007 1026, Los Alamitos, CA, USA, October 2024. IEEE Computer Soci-
ety. doi: 10.1109/FOCS/ 266 2024.00067. URL https::t////m //oi http:://do...eecccppteeeeee ieseccoppttessscc.... brtr/00.100//F03332266..2
00067
J. M. Landsberg. Tensors: geometry and applications, volume 128 of Graduate Studies in Mathe- matics. American Mathematical Society, Providence, RI, 2012. ISBN 978-0-8218-6907-9, doi: 10.1090/gsm/128 URL / http://ii..orgg///// org. /10.//g/1... 1090 gsm/ 128
Valery Levitas, Mehdi Kamrani, and Biao Feng. Tensorial stress- strain fields and large elastoplas- ticity well friction in diamond anvil cell up to 400 gpa. NPJ Computational Materials, 5(1): 94, 2019.
Derek Lim, Haggai Maron, Marc T Law, Jonathan Lorraine, and James Lucas. Graph metanetworks for processing diverse neural architectures. In The Twelfth International Conference on Learning Representations, 2023.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization, 2019. URL https /arxiv.org/abb/111.. 05101
Terry J. Lyons. Michael Caruana, and Thierry Lévy. Differential Equations Driven by Rough Paths.
Springer Berlin, Heidelberg, 2007. doi: https://doi.or//100 https///oo..og//...0//7777753550 1007/978-3-540--7225
Masud Mansuripur. An exact derivation of the thomas precession rate using the lorentz transformation.
In Spintronics XIII, volume 11470, pp. 159-158. SPIE, 2020.
Cheng Mao and Alexander S. Wein. Optimal spectral recovery of a planted vector a subspace, 2022. URL httptst///axxii.....tt http:////awxxivooo//aa////////// 15000
13

---

Published as a conference paper at ICLR 2026
Haggai Maron, Heli Ben-Hami. Nadav Shamir, and Yaron Lipman. Invariant and equivariant graph networks. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans. LA. USA. May 6-9, 2019. OpenReview.net, 2019. URLLtttss://ppeeeeeeett nwt//orumm forum? id=Syx72jc9tm
net.
DB Melrose and RJ Stoneham. The polarization tensor for a magnetized vacuum. Journal of Physies A: Mathematical and Generul, 10(7):1211, 1977.
Andrea Montanari and Emile Richard. A statistical model for tensor pea. In Proceedings of the 27th International Conference Neural Information Processing Systems Volume 2, NIPS' 14, pp- 2897-2905, Cambridge, MA, USA, 2014. MIT Press.
Edward Pearce-Crump Brauer's group equivariant neural networks. In International Conference on Machine Learning, pp. 27461 227422. PMLR, 2023.
Mircea Petrache and Shubhendu Trivedi. Approximation Appoxiimationgeenerlizaaaooo trade-offs under (approxi- mate) group equivariance. Advances in Neural Information Processing Systems, 36, 2024.
Max Pfeffer, Anna Seigal, and Bernd Sturmfels. Learning paths from signature tensors. SIAM J. Matrix Anal. Appl., 400 416, 2019. ISSN 19995-4798,109--777 doi: 10.1137/18M12123311 URL httthttss org 1137/1 1013378/1222222 8M1212331
Qing Qu, Zhihui Zhu, Xiao Li, Manolis C. Tsakiris, John Wright, and René Vidal. Finding the sparsest vectors in a subspace: Theory, algorithms, and applications, 2020. URL https:/7
06970
Stephan Rabanser, Oleksandr Shchur, and Stephan Günnemann. Introduction to tensor decompositions and their applications machine learning. arXiv preprint 1711. 2017.
Marco Rauscher, Alessandro Scagliotti, and Felipe Pagginelli Patricio. Shortest-path recovery from signature with optimal control approach. Math. Control Signals Systems, 37(2):305- 337, 2025. ISSN 01922-419,,14335556 doi: 10.1007/604998-224-0000000 URL https
7/doi. 10.
Nolan R. Wallach Roe Goodman. Symmetry, Representations, and Invariants. Springer International Publishing. 2009. ISBN 978-0-388-79852--3 doi: 10. 1011077/988-03387-79822.4 URI. https 100./000079900
Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model. IEEE transactions on neural networks, 20(1):61-80, 2008
Guillem Simeon and Gianni De Fabritiis. Tensornet: cartesian tensor representations for efficient learning of molecular potentials. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS '23, Red Hook, NY, USA, 2023. Curran Associates Inc.
Leslie N. Smith and Nicholay Topin. Super-convergence: very fast training of neural networks using large learning rates. In Tien Pham (ed.), Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications, volume 1 11006, pp. 1100612. International Society for Optics and Photonics, SPIE, 2019, doi: 10.1117/12.2520588 URL http:s://doo..org/////
1117/12. 2520589
Daniel A. Spielman, Huan Wang, and John Wright. Exact recovery of sparsely-used dictionaries. In Shie Mannor, Nathan Srebro, and Robert C. Williamson (eds.), Proceedings of the 25th Annual Conference on Learning Theory, volume 23 of Proceedings of Machine Learning Research, pp. 37.1-37.18, Edinburgh, Scotland, 25-27 Jun 2012. PMLR.
Xavier Suau, Federico Danieli, T Anderson Keller, Amoo Blaas, Chen Huang, Jason Ramapuram, Dan Busbridge, and Luca Zappella. Duet: 2d structured and approximately equivariant representations. In P Proceedings of the 40th International Conference on Machine Learning, pp. 32749-32760 2023.
Behrooz Tahmasebi and Stefanie Jegelka. The exact sample complexity gain from invariances for kernel regression. Advances in Neural Information Processing Systems, 36, 2023.
14

---

Published as a conference paper at ICLR 2026
Anh Tong. Signax: Computing signatures in jax, 2023. URL https://qithu.. com/ anh-tong/ signax
Csaba Tóth. Scalable Machine Leurning Algorithms using Puth Signatures. PhD thesis, University of Oxford, 2025. /ttp///////////..o..//////22//30
Soledad Villar, David W Hogg, Kate Storey-Fisher, Weichi Yao, and Ben Blum-Smith. Scalars are universal: Equivariant machine learning, structured like classical physics. Advances in Neural Information Processing Systems, 34:28848-28886 2021.
Soledad Villar, David W Hogg, Weichi Yao, George A Kevrekidis, and Bernhard Schölkopf. Towards fully covariant machine learning. Transactions Muchine Leurning Research, 2023a.
Soledad Villar, Weichi Yao, David W Hogg, Ben Blum-Smith, and Bianca Dumitrascu. Dimensionless machine learning: Imposing exact units equivariance. Journal of Machine Learning Research, 24 1309):1-32, 2023b.
Binghui Wang, Jinyuan Jia, Xiaoyu Cao, and Neil Zhenqiang Gong. Certified robustness of graph
neural networks against adversarial structural perturbation. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery &amp; Data Mining, KDD '21. ACM, August 202 doi: 10.11457344558834477999 URL http::tt//// /dx doi wow.ooooo/oooooooooooooooooeeoeo 10 111.1..1444445555 3467295
Junjie Wang, Yong Wang. Haoting Zhang, Ziyang Yang. Zhixin Liang, Jiuyang Shi, Hui-Tian Wang, Dingyu Xing, and Jian Sun. E(n) eqqivvaraa cartesian tensor message passing interatomic potential. Nature Communications, 15(1):7607, 2024. doi: 10. 10.1038/<44447-024-5555555
URL
Rui Wang, Robin Walters, and Rose Yu. Incorporating symmetry into deep dynamics models for improved generalization. In International Conference on Learning Representations, 2021b. URL
net
8Hx2KD
Maurice Weiler and Gabriele Cesa. General e(2)-cquivariant steerable cnns. In H. Wal- lach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (eds.), Ad- vances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL utt.ttteee//rcoccciieeeeerrrrrrr File/ F /ttp://.58111111222223001100aaa0
www.:edddiinneeeress.sss.s.c..ee httpaaaaeeeraaaaaaeaaaereeeeeeee files//aper:2019/
Maurice Weiler, Mario Geiger, Max Welling, Wouter Boomsma, and Taco Cohen. 3d steerable cnns: learning rotationally equivariant features in volumetric data. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS' 18, pp. 10402-10413, Red Hook, NY, USA, 2018. Curran Associates Inc.
Maurice Weiler, Patrick Forré, Erik Verlinde, and Max Welling. Coordinate independent convolutional networks-isometry and gauge equivariant convolutions riemannian manifolds. arXiv preprint arXiv:21066.06020, 2021.
Dmitry Yarotsky. Universal approximations of invariant maps by neural networks. Constructive Approximation, 5501):407-474 2022.
Xuan Zhang, Limei Wang. Jacob Helwig, Youzhi Luo. Cong Fu, Yaochen Xie, Meng Liu, Yuchao Lin, Zhao Xu, Keqiang Yan, Keir Adams, Maurice Weiler, Xiner Li, Tianfan Fu, Yucheng Wang, Haiyang Yu, YuQing Xie, Xiang Fu, Alex Strasser, Shenglong Xu, Yi Liu, Yuanqi Du, Alexandra Saxton, Hongyi Ling, Hannah Lawrence, Hannes Stärk, Shurui Gui, Carl Edwards, Nicholas Gao, Adriana Ladera, Tailin Wu, Elyssa F. Hofgard, Aria Mansouri Tehrani, Rui Wang, Ameya Daigavane, Montgomery Bohde, Jerry Kurtin, Qian Huang, Tuong Phung, Minkai Xu, Chaitanya K. Joshi, Simon Mathis, Kamyar Azizzadenesheli, Ada Fang, Alán Aspuru-Guzik Erik Bekkers, Michael Bronstein, Marinka Zitik,, Anima Anandkumar, Stefano Ermon, Pietro Liò, Rose Yu, Stephan Günnemann, Jure Leskovec. Heng Ji, Jimeng Sun, Regina Barzilay, Tommi Jaakkola. Connor W. Coley, Xiaoning Qian, Xiaofeng Qian, Tess Smidt, and Shuiwang Ji. Artificial intelligence for science quantum, atomistic, and continuum systems, 2023.
15

---

Published as a conference paper at ICLR 2026
A BASIC PROPERTIES OF O(d) ACTIONS ON TENSORS
In this section, we will show that the basic operations are O(d)-equivariant and linear by direct computation. We do so explicitly by performing routine computations. However, the universal property of tensor products, which we use in Appendix would give immediate proofs of these statements.
Proposition 1. The outer product is -equivvrrra bilinear map. In other words, for O(d), a' E To (RR,p) .P), D. Ter [R4 p/) and a E R. we have g (a @ b) (g a) 60 (g b). (aa + 3a') & = a(a @ b) + 3(a' @ b). and a @ (ab + -86) = a(a @ b) + 8(a @ 6"). In particular, if c E Tw [ p') is an Odd) tensor, then the function mapping
To (R" p) + Th- PP') by a+a0
(26)
an ((d) equivvaaua linear map.
Proposition 2. The k-contraction T2 (R², - Tw (R2, (def. 4) is an O(d)-equivariant linear map.
Proposition 3. For fixed a E Sg, the tensor index permutation mapping Te - Ta (R., .p) by + a is an -quivvaaiaa linear map.
Proof of Proposition g E O(d). We have
7 First, establish equivariance. Let E TR p). Tk (R", ,'), and
[g (a b)]ln...... p...ee
de)( --pe' [[a b)]es...n [M (g)lin ... (M
dot((M(g)) - det dct(M( -/ 16 [M(g)lnn [MM [M (g)li (M infaai
= =(det(M(())) - [a] [a]a,... [M (M
(de((M(()) 'tss" (Di iap [g alji...sulg blu
... [M()liianrden
Ig a ② g blirmndees where the second equality uses the fact that
det (g))
(Mt( det(
which is straightforward to verify via a case analysis over possible parameter values (i.e., p.p e {+1, -1) and det(M(g)) E {+1,-1}).
Next, we verify linearity. Let a, a' E (Rd. b Ter (Rd,,'), and a 3 E R. Then,
[(aa + Ba') & bls = I(aa + Ba Ba')).. (2) [6)
(27)
ala)......t ( Blal)i..ntt tomeeee
(28)
ala & bl Ila' @ b) bl
(29)
The linearity in the second argument follows in the same manner.
Finally, (26) follows immediately from the fact that the bilinear O(d)-equivariance.
16

---

Published as a conference paper at ICLR 2026
Proof of Proposition [g [4
To establish equivariance, let T2k+H (R", p) and let g E O(d). Then,
(30)
dct( "
|al e es..... f1. Lndh [M ... (..
(31)
det(M(gg) -
[a] ee..... 820
( ... [6] fal (M ... [M(g))ww
(32)
dct(M(()) - [alei.... $1.. s
(M) (M)
(33)
(g) (M Pan.
[M ... [M [M
(34)
[g m. me Jarrrii
(35)
14 (g a)
(36)
where the third equality uses the fact that 6 - M(g) M(
Next, establish linearity, let a, b TeR+ ( p) and let a, B R. Then,
[uaaaa + [86]] = aaa + ( dn
(37)
al in Ba + 8T0
(38)
= aalu(a))i.....) + (l)x
(39)
This completes the proof.
Proof of Proposition Fix Sp. To establish equivariance, let Te p) and g E O(d).
Then,
= det( 22 [a a (M ... (g)
[g (a Miimmn
(40)
= det(M 2t [al_--1a))- ( [M (g)lmn ... (.M
(41)
=do((M(g)) 1 [a]i 10) (0)
M ( [M
100
1000
1(6)
(42)
= [g ali, 1 1(k)
(43)
= [(g u)"l li......
(44)
(45)
where the third equality holds since we are merely reordering the M(g) components-which allowed because they are scalars.
To show linearity, let a, b E To (R(, and ar, R. We have
[(aa + (,36)"",..... = aaa + 36] (666 -00
(46)
a [a] (10p___-ppp- -
(47)
a ) + 8(b%)......
(48)
B PROOF THEOREM
The main idea of the proof of Theorem is to write out the polynomial f in a way that takes advantage of the tensor operations of Section 2 then show that each term must be O(d)--quivariant (Lemmas [). We then use group averaging argument show that Ct.... can be written as an -ss)tisotoo tensor. We state the lemmas, prove the theorem, then prove the lemmas.
Lemma 1. Let f 11.h Th, (RRd, ,p:) + Tkl (Rd p') be a polynomial map of degree It and write
f(a an) Σ fa a.). P=0
where fr IT Th. (Rd + Tx (R',p') homogeneous degree polynomial. If f is O(d)- equivariant, then each fr is Odd)-equivaaiiat.
17

---

Published as a conference paper at ICLR 2026
Lemma 2. Let fr II-==1 Th: .P) Th (IR p²) be homogeneous polynomial of degree Then, we can write fr as
Fr(a an)
Σ
fermeee ( ae,))
(49)
15h S..
where fei....,, IT The, (R. pe.) + Th (R² .p) is the composition of the map
II Thi, (IR", .p.)) + +Tii heg
R
Pe.
(50)
(at) 0o1.:..;01. (z,) + dr, 8
@ ae,
(51)
with linear map Tem Ke, (R4 II== + To (R.../)
Moreover, f. is O(d) equivvaann, then so are the fe....... Remark 2. Note that Lemma i nothing more than the decomposition of fr sum of multiho- mogeneous maps in the inputs d1,..., an.
Proof of Theorem ! Combining Lemmas D and 2
we can write f follows:
R
f(a an) =
Σ
fe (ot, (oG.....,E..
(52)
15 St..
where the fen...,, is the composition of a linear map The (R.d Ptr....e)) - Tk (R., , pp with the map (a1...., ae) + az, 8... 2 ag,. Recall that ke,,.... Eg=z ki, and pt,....e. = II Peg Moreover, by the lemmas, each fe, fe.....e, e, is ((d) -equiiarrri Hence, without loss of generality, is enough to prove the theorem in the special case
f(a an) = M(a, @ .... ar,);
(53)
where A :E ke; (Rd), II Pc.) + Thr (RR,, p') is linear.
Now, in coordinates, we can write this map as (n
= d de Siundd (ag, 8 .... @ at a "ereeeee
(54)
Consider now the tensor E Theg +h (Rd, P') given by
[c] Icli ee +v dismant She
(55)
Then we have that [f an) an)lim.. Iclir...ity Ir aas @ at, de
(56)
lae, @ ... @ at, c) comminng...... d1 Findee
(57)
[oky, (ag, 2 ae, 8 c)lin...er
(58)
after using the definition of k-contraction. Hence
f(a an) te...... (at, ② ... X az, 8 C).
(59)
Since is ((dd)-quiivaran have that for all y E O(d),
f(a an) Lkey..... (ar, 2 ... X 0,, 8 c).
(60)
To see this, argue follows:
S(a1..... (n))
(61)
=s(g (g-1 1 (1) ... g (g an))
(62)
= g f(lg 01), .... (g an))
(f -quivvaai)))rantt) (63)
g Use .r ((g 0t, 3 8 (g az,) 69
(64)
UKke (g (g ar,) 8 2 (g 1, az ) 8 c)) (Uke -quuequuia))ian)) (65)
Uer (at, 8 ... (g c)).
(66)
18

---

Published as a conference paper at ICLR 2026
Hence, by taking the expectation with respect to the Haar probability measure of O(d) and linearity of contractions, we have that
an) an)= Uke.....r (an
g. (.c))
f(a.....,
(r. 8 ... ac, @
E (EO(d)
(67)
where (g))(() is the expectation with respect the Haar probability measure of O(d). This holds because
f(a1..... an) = E gEO(d)
fai....., an)
(68)
E sEO(d)
Uk (at, 8...0 ae, 2 (g)c))
(69)
dg & ae,
E g
(70)
(EO(d)
Now, (d))(d) is an Odd)-isotropic tensor. Hence, we have shown that we can write f in the desired form.
Proof of Lemmal 7 Let & E I,, since each fr is homogeneous of degree r, we have
R
(tta,,..,,to,, Σ fa (,(ta....,ta)))
Σ (a)(
Let now g Od)), then, by equivariance f, we have
R Σ
t' fr 01 ......................... ... = Σ ==
fgg f(lo.....,auu)
(71)
==
since
B Σ
fg ....... a.) = f(t(g- - aI),...,t(f an))
(72)
f(g Lai, 9 Lu))
(73)
g f(ta)..... tax)
(74)
g Σ f f((aa......n
(75)
Σ g. fr(ai,...,
(76)
Hence, for all O(d), E R and (an...., E II Th, (Rd Pt). we have that
R 0
De lr (g T=0
(((p....,,,,) Srlg ........ a,)))
(77)
Now, the only way in which the univariate polynomial in t of degree R is identically zero is if is the zero polynomial (cf. (Cox et al.
2015 Chapter $1 Proposition 5)). Therefore for all N.
g E O(d) and (a1...., Gn) E II. Th. .P).
g. an) ........ = fr fai,...., an),
(78)
i.e., for each r, fr is -quivvaran,,
wanted show.
Proof of Lemma 2 First, we will show that if the decomposition exists, each summand is equivariant. Then, will show that the decomposition exists.
19

---

Published as a conference paper at ICLR 2026
Let in R. Then, by the linearity, we have that
fr(t1 a1 on...., an)
E
te, t. Se (att.................... ac,)
(79)
1sh SS1.S..Sis5
since
fr 01, (n
E
Ser...e (te, ob,...., te, at,)
(80)
Str
Σ
le th le,
(81)
St.5n
Now, let g O(d). Then, by the equivariance of fr, we have
Σ
t1 ... te e (gag.. .. (0.0.)))....0)))))))))
Σ
te t., 9 g. ,
(82)
since
Σ
th te, Sts a .....,)))) at.))
(83)
frl (g 01), ................................ tn (g an))
(84)
frlg la ....... ............. In an)
(85)
= g fr aI...., tn.
(86)
g.
Σ IshS..t.
th te, (at ac,)
(87)
Le le, 9 Strmae, (at) (t.)).
(88)
15 St.
Hence, for all g O(d), e R and (ai...., an) II Tki (RR p.). we have that
0= Σ te ... te bg Seamee, (at..... ai,)) - ft (g ar at, dt
18h5...5..5
(89)
Now, each of these a polynomial in fi..... tu that vanishes on R" Moreover, note that no two (e .,, give the same monomial. Hence, by (Cox et al. 2015 Chapter 1 $1 Proposition 5), all these polynomials are the zero polynomial, i.e., their coefficients are zero. this way, we conclude that for each fe....... L.. and all g E O(d), & IR and (a1,.... ...,.... .n, II Th, (L)).
Se e, (g- ......... ar.)) = g Seg..., (at, (at,...., at.)
(90)
ie., each fe,...,, is equivvvruaar
Now, show how to obtain the decomposition. Recall that f. homogeneous of degree r. Therefore each entry of f-(a1,..., an) is an homogeneous polynomial of degree r in the [ai]jt...on i.e., a linear combination of products of the form
II li
where, without loss of generality, we can assume that & S ... S ly. Hence, in coordinates, we have
[ar "bi....,ay, an) ant.....,,))).... an)li...
Σ
at a (91)
20

---

Published as a conference paper at ICLR 2026
And so, we can consider the map fes....,, given in coordinates by [fer,.er (ae,..... (e,))a.... = bo a trn
ol Is to.ne
9=1
(92)
which, by construction, is the composition of the linear map given by
b + Ari fight 1.. ".. Key
[b] "
in coordinates, and (at,...., ag,) + ar, 2 ... @ ae,. Hence the desired decomposition of fr has been obtained.
C PROOF OF COROLLARY
In this section, we will prove Corollary I using the following lemma, which we prove afterward, This lemma, originally from Pastori, follows from Jeffreys (177)).
Lemma 3 (Characterization of (d)-isotropic k(m) -tnsoors) Suppose isotropic. Then the following holds:
Te p) is O(d)-
Case p +1: Assume p +1. If l is even, then can be written in the form
Σ Oke (88)) ))
for some Oar E R.
(93)
oES²
Otherwise, if k is odd, then 0 is the zero tensor.
Case p = 1: Assume p = 1. k even and d, then e can be written the form
Σ
Bar 8 8e)
(94)
SE
for some Bgg E IR Otherwise, if is odd k < d, then 0 is the zero tensor.
Note that in both cases only a subset of the permutations a E S₃ will yield different isotropic tensors (see Appendix D for discussion).
By Theorem and Lemma 3 assume, without loss of generality, it suffices to consider the special case where consists single term
Proof of Corollary 7
S(v1,..., Un) 4r (1h & ve, (S0
(95)
for some a E Srtk' and r + k even. To simplify notation, set / = rtk'
2
Now, note that
e, @ Ci.
where fei....,edl ea} is the canonical basis of Ed Hence we get
f( vn) (ve, X x us Ø (c:, 2 Ci @ ... @ @ )%
(96)
Let's write
(ei, @ e 2 e, @ ) Cjo 8 ... @ Eju-
where (Ji,... )2x) some permutation of (in , ie) and so, by Einstein notation, we are still adding repeated indexes. Then have that
f(ui...., v,.) =
... (ve, e,,)),r4 X ..... @ Ej..
(97)
Now, freely rearrange the (vz, ez) they scalars. There three cases for each of the original indices iy: (a) e, appears an inner product twice, (b) es, appears an inner product once, or (c) e,, does not appear in inner product.
In the case (a), we will get
(ve, es) (ve, ex) (ve, ve))
21

---

Published as a conference paper at ICLR 2026
In the case (b), we will get And, in the case (c), we will get
(Ue, e,)e; = Vz.
6j 2 E = 6. Now, assume that we have a of the case (a), 3 of the case (b) and y of the case (c). By permuting the ig. which does not change the result, we can write for some permutation E S84y and some permutation J1,.... J Jr some permutation of .,,..., that
F(U1....,,h Un) et,), Cn)) (1Jza-11 Cs,) (1/20 esa))
((UJatt Matieaa 1 (1Jaata Aatt
@6i 1911 Ciasoi1
1217 8 (iu1
= (M U.2) ... (UJ20 1 U. ) (Ugaa+1 @ ..... @ 12 2 d07)²
Hence, the desired claim follows, and we finish the proof.
Proof of Lemma 3 o(d)-isotroppc tensor is always an (O())-isoroopic tensor since det ( (M(g) for all g SO(d). Now, by (Jeffreys 1973. Theorem $2) (ef. (Appleby et al. 1987 Eq. (4.)))), any (O(d)-isotropic
We will prove each case separately. However, note that no matter the value of p. an
tensor z can be written as linear combination of the form
Okerr
8
+ Bor (800 k-d ØE);
(98)
where 8 is the Kronecker delta (Definition 8). and e is the Levi-Civita symbol (Definition 9. with the convention that the coefficients and 9, are zero when the expressions 8 and 8 - do not make sense. More precisely, the a, if is odd, and the B, 0 if k- is odd.
Note that under the (O(d)-accion) we don't 't need to worry about the parity, and so both 5 and c are (Odd)-inariiat. However, for the add)acni the parity matters. Suppose y E O(d) a hyperplane reflection, and let be an Odd) -te-ooo If f is -(---tensor whose components equal T, then
f = -T.
(99)
Likewise, if is an (dd)issotoopc k(t)-tensor -tnsoor and 7 is a k(-)-tensor whose components equal T, then
7 --
(100)
Note that being isotropic depends on the parity because it affects the considered action.
Case p = +1: Let z E Ta (Rd, +) be Odd)iisotopi.. In particular, = is also we can write it using (98).
and so
Recall that (dd) is generated by all the (hyperplane) reflections. Hence, to show that 2 is an O(d)- isotropic, we need only to show that for every (hyperplane) reflection 7 E O(d),
7
2
Now by 99
2 = Be,
(101)
since 88 شية 8 e is an O(d)-isotropic
k -tnsor. Hence
Σ a (00 + B (10 8
7,2=7
(102)
Σ at (17-8)80)
+ 3 (17 ((7-8)8* 8) 87.6)
(103)
E
Σ Ole (000))=
- 30 (800 80)²
(104)
Bo (
2
(105)
22

---

Published as a conference paper at ICLR 2026
By assumption, = is O(d)-isotropic, so we conclude that Eoes, 3o (10 x0)² = 0 and z has the desired form.
reflecccioon Case = -1: We argue as above, but using that for (hyperplane) reflection 7 E O(d), we have, inside To -).
8 -8
(106)
since の is an ((d) -isotropic -teyseensor Hence, using a similar argument to the previous case, we conclude that Eoes,
600 = 0, so 2 has the desired form.
D SMALLER PARAMETERIZATION OF O(d)-ISOTROPIC TENSORS
In Lemma 3 the sum does not have be over all permutations. The reason for this that the tensors
f and 8 - +
do not have trivial stabilizer under the action of Sg. One easily the following proposition. Recall that the stabilizer of a k-tensor +T in Sy is the following subgroup:
Stabs, s. (+7) = for E S T = +
(107)
where 7 +T means that either 7 T T - Note that the laxity in the signs comes from the fact that positive summands and their negative counterparts can be combined. Proposition 4. Consider the cases:
1. If l is even, Stabs, (+004) is generated by the transpositions (3,4)) (k: 1) and all permutations of the form (i, + 1,j + 1) with i, & odd. In particular, # Stabs, (= (k/2)12*/2.
2. If k (1, (3, 4)...., (k 1, k d). all permutations of the form (i. 1,j + 1) with i,j < k d odd, and all transpositions of the form (i,j) with k - d < i, In particular,
is even, Stabs, (- 60 is generated by the transpositions
# Stabs, (+ x e) = ((k: d) 2)/2) 2
Proof. This follows from Roe Goodman 2009
Theorem 5.3.4).
Using this proposition, write any (dd) isotropic ((+)-tensor as
Σ 0g (885)
(108)
with the Og real and
Gk = fo e Sk o(1) < (3) <... ... olk 1) and for all x k
2 a(2i 1) a()))
(109)
of size
E (*/2)1247) and any Odd)-isotoop k h(-)-tensor
Σ oEHi
Bo ( 0+)
(110)
with the 3nr real and
k -- a(2 - 1) o(2))
Hy = fo e Sx : o(1) <o(3) < ... < cok - -d 1), for all i is
and for all j j> k d,o()) < <o])+1))
of size
(- - d!
23

---

Published as a conference paper at ICLR 2026
E EXAMPLE OF THEOREM 1
In this section, we give second example of Theorem 1
Example 2. Let f Ti (RR, +) x T2 +)) + T2 (2 +) be O(d)-qquivariant polynomial of degree at most 2. By Theorem 1
we can write f in the form
f(a1, 02) = Σ
Σ
(ae, .... @ ae, @ Cl
(111)
(Key,..,,, + (t))so By Lemma Cti....,e is nontrivial only when le........ + 2 is even. Recall that ke..... = Eq= k The inputs are 1 (4))tensor and -ees-r The even combinations of 1 and 2 with at most 2 terms are 0, 1, + so we have
where Ce,,..,,, is
-tensor.
a2) = 3gg + 12(22 8 02) + (2 @ a @ ()) + (4 a2 & ca).
(112)
where c22 62 are Oid)iioorooi 4(+) -tensoss and is an Odd)-isotoopi (4t)-ennor. By similar can write
reasoning to Example !
(2 & 62) 31 tr(a2)ō + 8202 + 8302
(113)
for constants B1, 32,82, B3 and
(2 @ 01 2 ()) (1 $5a1 01
(114)
for constants 84, 33 (there are only two terms due to the symmetry a @ a1). It remains to consider write
(a @ 02 cs). By Lemma 3
C3
Σ
Bo (6
(115)
TEGG
where Gol = 61/(322)) 15. In particular, we have Ga =
(1333,,,,)))
(1,3,4,,, (1, 4.5, (1,3,44,, 2) (133, (116)
(1,3,5,2,6, 6, (1, 5,4,2,6) (1,3,5, (, (133,,,,.444
However, due to the symmetry of 0g @ dg when we compute 14 (1 @ (803))) for E Go. there are only distinct terms
(a 82 c3)
+ f110222 + Praezag + 8g tr(a2) (rtr(2))a + 3s tr(az)ag + 8yaz
Biza af (117)
In summary,
f(e1, a2) = 308 + 31 tr(a2)6 8 + B202 + T + 33(aa,, Bia BB tr(aa)a2 B7 tr(a)aa ABaj
& + 83 tr
B10 + A a2
Bnajaj (118)
for some coefficients 3o. 81..... 811-
F PROOF OF COROLLARY
Before proving Corollary 2 will state and prove important lemma. Since this is the setting of symmetric matrices, we will use familiar notation and concepts from that setting rather than relying on tensor algebra.
Lemma 4. Let f Rdxd + Iddd be an (dd)-qquivarian function. Let A E Reyd then f(A) A
A ((A) they are simultaneously diagonalizable.
24

---

Published as a conference paper at ICLR 2026
Proof. Let Regi Ragg be an -dd)-qquivariant function. First we will show that for all diagonal matrices D, ((D) is also diagonal. Let D 1ggm be a diagonal matrix. Let Q; for i ...... d be the orthogonal matrices that are equal the identity matrix except [Qi -1. Since Q; Q = D, we have fD) = f(Q, D DQ;) ) Q: f(D) Q Now we note that the 8t row and column of Qi J(D) Q is equal to the negative 1th row and column of S(D), except for the diagonal elements which are equal: ((D)
= [ However, f(D) = Q; (
Thus we conclude that the off diagonal elements of ((D) must be equal to since this is true for i i= 1, ... d. Hence, (D) is diagonal when D is diagonal.
Now let A E IRdxd stym so A is diagonalizable, A = Q Q Thuss
= f(QA="QQQQTT = f(A)AQ = QAS(A)Q*= = =QQQ" (QAAT)) =Af(A))=)(A)) (119)
This concludes the proof.
Now we will prove the corollary. Note that since f is function of diagonal matrices, we could equivalently think of it as the composition of converting the diagonal matrix to vector, performing the function on the vector, then converting back to a diagonal matrix. We will follow this implementation in the code to avoid storing a large diagonal matrix.
Corollary. Let f aymm +
Tedxdd be an O(d)-equivarian function. Then there exists function
Rdxd + IR idd of diagonal matrices that is permutation equivariant such that for all e ding
diag T (Rd +), f(A) = QT where A = Q A AQ is the eigenvalue decomposition.
Proof. Let Rdyd - Rdxd be an -queivquivariant function. Let A Rdxd with eigenvalue decomposition A QAQ Then by Lemma Q also diagonalizes fA), that is QT f(A) Q is diagonal. Hence Q f(A) Q f(Q AQ) f(A) is also diagonal. Since the input and output are both diagonal,
define the function that is the same but restricted to diagonal matrices, and is therefore f Iddxd ddxd Thus f(A) F(A). Therefore Q f(A) Q = F(A) implies that diag
= = QTA)Q.
Finally, we must show that F is permutation equivariant. Let A be any diagonal matrix. Since the permutations of elements is subgroup of Odd), we have for any permutation matrix P:
S(PAP) = P P(A)
(120)
j(PAP)) = P
(121)
Thus f is permutation equivariant for any input, which completes the proof.
G GENERALIZATION TO OTHER LINEAR ALGEBRAIC GROUPS
In this section, we will show how Theorem and Corollary can be extended the indefinite orthogonal and the symplectic group as Theorem 2 and Corollary
The main idea to extend Theorem other groups to use some form of averaging. On O(d), the compactness guarantees the existence of a Haar probability measure. However, to apply the same trick over non-compact groups such as (s,, s) and Sp(d). we need to use technical machinery to imitate the averaging strategy.
First, we introduce some definitions and examples regarding complex and real linear algebraic groups. The main point will be to establish how to get compact subgroup over which to average. Basically the results will generalize to real linear algebraic groups such that their complexifications have Zariski-dense compact subgroup. For instance reductive connected complex algebraic groups satisfy this assumption. Second, we prove generalization of Theorem [or complex linear algebraic groups with Zariski-dense compact subgroup acting on complex tensors. Third, prove generalization of Theorem Zariski-dense compact subgroup. Finally, we prove Corollary
[ for real linear algebraic groups that are compact or such that their complexification has
25

---

Published as a conference paper at ICLR 2026
G.1 REDUCTIVE COMPLEX AND REAL LINEAR ALGEBRAIC GROUPS
Recall that a complex linear algebraic group is a subgroup G of GL(V), where V is a finite- dimensional complex vector space, such that G is the zero set of some set of complex polynomial functions over End(V), the set of (complex) linear maps V > V. Recall also that a rational G-module of G vector space U together with a linear action of G on U such that the map G x U (g. x) + x E U is polynomial and that G-submodule Uo of U is vector subspace Uo U such that for all g E G.g Up S Uo.
Definition 10. (Roe Goodman 2009 Def. 3.3.1) A reductive complex linear algebraic group is complex linear algebraic group G C GT.(V) such that every rational G-module U is completely reducible, i..., for every G-submodule Uo of U, there is G-submodule U1 such that U Uo + U1 and Uo nU1= =0
Example 3. Given any mimen-dimensional vector space, the classical complex groups GV( and SL(V) are reductive complex linear algebraic groups.
Example 4. Given any finite-dimensional vector space V together with symmetric non-degenerate bilinear form (., ): V V V + C, the (complex) orthogonal group
O ((V.(.,))) -)) = {g E GL(V) for all v, w E V, (g v, w) = (v, w)}
(122)
is a reductive complex linear algebraic group. We will pay special attention to the following family of complex orthogonal groups:
O d s) == tg E GL(C g Le.d-sg = [,d_-) = OCC .))
(123)
where (u, u), == u uJI, ",,d_ Note that all these groups are isomorphic, satisfying that
Lo =la..)
Il. ("
ds
illa-.)
Moreover, this is true in general: any two complex orthogonal groups are isomorphic if they are of the same order-this follows from the fact that all symmetric non-degenerate bilinear forms are equivalent over the complex numbers.
Example 5. Given any kmite-dimensionn vector space V together with anti-symmetric non- degenerate bilinear form )): : V x V - C, the (complex) symplectic group
Sym(V (., -)) = tu E GL(V) for all u, w E V, (g 0, w) = (v, w)}
(124)
is a reductive complex linear algebraic group. We will pay special attention to the following special case:
(d) = {g E GL(C") g Jag J3} Sp(C", (.,
(125)
where (u, syympp = u. Jav Note that any symplectic group of order d is isomorphic to Sy (d) because any two antisymmetric non-degenerate bilinear forms are equivalent over the complex numbers.
Example 6. The complex lincar algebraic group
H -={((
1)IE
is not reductive, since C2 is an H-module that is not completely reducible. Note that C x 0 is the only H-submodule of C2, so we cannot find a complementary H-submodule.
Recall that a subset X of a set X Zariski-dense in X if every polynomial function that vanishes in X vanishes in X, ie. if every polynomial function that does not vanish on X does not vanish in X. The following theorem allows to use the power of averaging for reductive connected complex linear algebraic groups
"To be precise, we mean that the map G x U + U is morphism of algebraic varieties. Choose basis for U and V. so that we can identify V with Cd and U with C". Then, being a morphism between algebraic varieties, just means that the map G x C C is the restriction of map Cd x C + C that be written componentwise
(p(((o))) (ux) (/)) g)*(), where each pı a polynomial in the Ii,5 and uy and each
a an integer.
Recall that this means that for all u. E V and f, a E C: (a) (u, = u), (b) for all x V. (u, ( if and only if 0, and (c) (tu + su, m0) t(u, w) s(0, m²).
26

---

Published as a conference paper at ICLR 2026
Theorem 3. (Roe Goodman 2009
Theorem 11.5.1) Let G' be reductive connected complex algebraic group. Then there exists a Zense compact subgroup K. More precisely. there is subgroup (G) of G that is Zariski-dense in G and that, with respect the usual topology is compact.
Remark 3. Note that using this compact subgroup K, we can consider expressions of the form
E
u.
HEU(G)
by taking the expectation with respect to the unique Haar probability measure of K. Now, since U(G) Zariski-dense in G, we have that the fact that for all U(G), u- (Euev(G) u- T) EueD (G) u-T implies that for all g G.
(E E u. u.T) = EE (EE((G)
".T)=
u-T. T. UEU(G)
Note that U(G is not necessarily unique. Example 7. In GL(C"), the Zariski-dense compact subgroup is the group of unitary matrices:
U(C")= (g GL(C4)) g" = ls}
where denotes the conjugate transpose. In (CC)), is the group of special unitary transformations:
SU(CH) = {g E (C*) det g= = 1}- Example In 0 s), the Zariski-dense compact subgroup is
(th Il...) O(d)
[
O(d))
ila..)
Note that when & = 0 or s = d, this the orthogonal group over the reals. Moreover, this does not follow from Theorem 3 as 0 (s, [1-]] s) is not connected.
Example 9. In (d), the Zariski-dense compact subgroup is the so-called compact symplectic group:
USp(d) = (U)
Recall that real linear algebraic group subgroup G of GL(V), where V is finite-dimensional real vector space, such that G is the zero set of some set of real polynomial functions over Rdxd Similarly, did with complex linear algebraic groups, we can talk about rational modules and about reductive real linear algebraic groups.
However, given reductive real linear algebraic group we cannot necessarily guarantee the existence of Zariski-denss compact subgroup. This means that we cannot apply the averaging trick directly, but we can do so by passing to the Zariski-dense compact subgroup of the complexification of the real linear algebraic group.
Definition 11.
Let G C GL(V) be a real linear algebraic group. The complexification GC of G is the complex linear algebraic group given by
C = {g GL(VE) for every polynomial f such that f(G) = 0, f(g) - =0
(126)
where vc On C is the complexification of V, i.e., the complex vector space obtained from V by extending scalars.
Remark 4. In essence, we complexify the underlying real algebraic variety. Group multiplication preserves its structure as complex variety as it is given by polynomial functions of the matrix entries.
Definition 12. A real linear algebraic group G is complexly averageable if it's Zariski-dent in its complexification and its complexification admits Zariski-dense compact subgroup closed under complex conjugation.
Remark Recall that the complexification of d naturally isomorphic to Cd. Example 10. The complexification of GL is GL(Cd), and the complexification of SL(Rd) is SL(C)).
*The topology inherited from the Fuclidean topology of GT.(C").
27

---

Published as a conference paper at ICLR 2026
Example 11. We have that
((s,d-s) =0==00)=s -)
and that
= Sp
Hence, both the indefinite orthogonal group and symplectic group are complexly averageable. The symplectic group is connected but the indefinite orthogonal group is not connected. However, it does have a Zariski-denss compact subgroup (see Example 8).
The following proposition shows that complexly averageable real linear algebraic groups common. Proposition 5. Let G c GL(V) be a real linear algebraic group. Then: (1) G is Zariski-dense in GC, (2) If the complexification G of G is connected and reductive, then G' is complexly averageable.
Proof. (1) Let f be complex polynomial vanishing on G. Then, we can write this polynomial as f fr + if; for some polynomials fr and fi with real coefficients. Now, since f vanishes on G, then fr and fi vanish also G G-as as otherwise there would be g E G such that either f-(g) & 0 or fg) + 0, contradicting fg) = 0. But then, by definition of G fr and fi vanish on G and so = fr + if, vanishes G Hence have just proven that complex polynomial vanishes G and only if vanishes on G i.e., we have proven that G is Zariski-dense in G
(2) This follows from Theorem 3
Example 12. Observe that the Zariski-denss compact subgroups of (f d -s) s) and Sp that have been given satisfy that they are closed under the complex conjugation.
G.2 COMPLEX EQUIVARIANT TENSOR MAPS
We will consider vector spaces which non-degenerate bilinear form has been chosen. Definition 13. A self-paired vector space (V. ( .)) is fiite-dimeensinnl vector space V together with non-degenerate bilinear form (., .) ::VV V
C.
Recall the universal property of tensor products of vector spaces, by which multilinear maps Vi X V& + W can be lifted to linear maps V1 ..... & V₁ - W. Using the universal property, we can see that from a self-paired vector space (V, (., })), we get the family
(V (),
of self- self-paired spaces of tensors, by extending by linearity the expression
(U1 ? x uk 0n @ ... @ 0x) = (vi, 02) ... (vk, 0x)-
(127)
And again, by the universal property, we get k-contraction by extending by linearity, the expression
4 (2(++)))) z s & y & vak veK
(128)
2 bb b)c.
(129)
Now, in the above setting, let G be group acting in struecuue-preserving way (V.(,, )), meaning that the action is linear and preserves (-, )), i.e., for all g E G, v, E V, (v, ē) = (g-v, g) Then, by the universal property, we get that G acts also on SVSk {-, )) by extending linearly the expression
v (g)) .... & (gvx). F Moreover, by considering all (rational Junidimensional representations X G + C* of G. we get the following family of self-paired (rational) G-modules:
g(0
(130)
Te(V, = (V& (.,
(131)
where the action by G is given by
g T (())
(132)
'Reaall that rational means that the homomorphism is given by polynomials.
28

---

Published as a conference paper at ICLR 2026
in structure-preservin way. For the sake of distinction, will denote the k-contraction
i + (V,x) - (V,
(133)
in this setting to emphasize the dependence on the group G, we will be choosing the original {+> in terms of the group. Using the universal property, we can easily see the following: Proposition 6. The following statements hold:
(a) The outer product map
(v. x (V, + Te+x)(V,xX)
is a G-equivariant bilinear map.
(b) The k-contraction of T2x+ (V,x) - Th (V, is G-equivariant linear map.
(c) For any E Sg. the tensor index permutation by Tv(..)) + Ta(V, given by U1 X @ U2 + -0 (1) 8 ... @ (k)) a G-equivariant linear map.
Finally, recall that a G-isotropic tensor of Te(V, x) a G-invariant tensor in Te(V, x). Further, recall that entire function is analytic function whose Taylor series any point has an infinite radius of convergence. We can now state the theorem.
Theorem 4. Let G C GL(V) be a reductive connected complex linear algebraic group (or more generally, complex linear algebraic group with Zariski-dense compact subgroup) acting ra- tionally on an structure-preserving way on self-paired complex vector space (., .)) and f: f: (V. + Tw (V,x), a G-equivariant entire function. Then we may write f as follows:
00
f(a f(a an) Σ
Σ
(at, ... at, Ct...ner)
(134)
where Cf E The th' (Rd, Xty...,, X') a G-isotropic tensor for key E hit and
Xf Xt,
To prove this, we proceed as in the orthogonal case: we reduce to the multihomogeneous case and then prove the result using averaging over the Zeriski-dense compact subgroup.
Lemma 5. Let G GL(V) be any subgroup acting linearly on self-paired complex vector space ())).,))) and f [ (V, > Th (V, X') an entire function. Then, we can write f as
fa an) = Σ Σ
fer... e, (oe...... ae
(135)
P= 15115 St.sn
where fe.....,, IT Thi (V, + (V.''))) is the composition of the map
II
The, (V.xe.) + TT kG; V. V. Xt.
(136)
i=1
(aft)..., ((.)) + 0zg $ ... ... 2 ae,
(137)
with linear map TE:_ kc, (V.II Xx,) Th. [R.d xx)
Moreover, for the above decomposition, if is G-equivariant, then so are the fe.....,. Remark 6. Note that we don't need to assume anything about G in the above lemma.
Proof of Theorem By Lemma 5 we can assume without loss of generality that f is of the form
f(a an) >(ae,, @ ac.)
for some non-negative integer
< 1 < l 5 and X T ke, (V Xx.)
Te (V, x') is linear.
29

---

Published as a conference paper at ICLR 2026
The above map can be written a linear combination of maps of the form
(a1..... an) + II 1=1
(aaa)))
U/2 ... U/2
where the X are linear and fj E V, due the universal property the factor (TT dr(ae, )) just
corresponds to linear map Tea ke, (V, Il Xx,) C. Moreover,
HIMeas) U/i 0....0jj. = = Einn ke, +x (at, X ... @ ag &c @ ... Cy 1 Ujj @ ... Ujo)
where the e E Th, (V, xx) are the unique tensors such that for all ae, E The (V,x))
(a(,)) (aa. G).
These e, exist, because (., >) is non-degenerate. Hence for some e E &T he, te (V, xix). we have
I (a an)) Ei ke, +x (atz
ae, @ @c).
(138)
Since and (.)+w(() are G-equivariant, we have that for all g G and a II Tk, (v)
Ik-kk (ae, .... at, @ ce) f(a1....... Qn)
(139)
f(g (g a1) . (g an))
(140)
g. f(lg 11 a1) ...,, (g an))
(141)
g 1kotk' (gg at, 3 (g 1 at (4)
(142)
lketk (at,
ae, (g ca)
(143)
Finally, G has Zariski-dense compact subgroup UG). Hence, averaging UGG), substitute e by the U(G)-isotropic tensor
E u.c
MED(G)
where the expectation is taken with respect the unique Haar probability measure of UG). But, since (G) is Zariski-dense G and the action rational, ELEU(G) u c also G-isotropic, as we wanted to show.
Proof of Lemma 5 Recall that, since f in entire, we have, by Taylor's 's theorem, that
f(a) = r!
(af(a
(144)
where a = (a....., an) E IT Th. (V. Xi) and D
( To, (V.)))))) + (V, x') is the k-multilinear map given by kth order partial derivatives of O. Now, write a₁ 0 as an abuse of notation for
a (a1, 0, 0) + +.. (0....,0.a,))
We will further this abuse of notation write a; instead of (O,..., 0, ai, .,.., 0). Now, since D f is k-multilinear and symmetric, we have that
DEJ a)) a)= r!
Σ SSAS--SSS at
D (at.
(145)
where ake & N' is the vector given by (atg #li E i and a! ay!... a.!. Note that this terms appears when we reorder (at,,..., ae,) ) so that the subindices are in order.
Summing up. write f as 135 with
St (a1,..., an) a.,.....
.6,))
(146)
30

---

Published as a conference paper at ICLR 2026
where this has the desired form by the universal property of tensor products. Now, observe that for fi..... tn E C and (a1..... an) E II==1 Tk. (V.
fe, t (n,d)....,.....
t fer..... (a1..... an)
(147)
where tuuee = t th. Hence, arguing in Lemma we have that for any g G and all (01,..., An) E [Ti=1 (V,)),
Σ
Σ
maneegg Stamen (a1,...,an)
P=O 1-15115"555
=-
E
the ftr...., Fo (g ........-- a.))) (148)
Hence, by the uniqueness of coefficients for entire functions functions that are equa we conclude that for any 9 G and all (a1....., ap) III Th, (V, Xi).
g f (.,,..., .., Qn) = fta -1,.... .nn), and so that the fey..e, are G-equivariant.
(149)
G.3 REAL EQUIVARIANT TENSOR MAPS (AND PROOF OF THEOREM 2
All the definitions in the previous subsection can be specialized to the real case. Hence we will have self-paired real vector space (V, (., .)) on which a group G acts (rationally) in structure-preserving way. Then get the family of (rational) G-modules:
Tv, == (V (.,
where X G + R' is one-dimensional (rational) group- homomorphmmo of G. Together with this family, we have the k-contractions given by
(150) which are G-equivariant linear maps. Then we get very similar theorem Theorem from which Theorem Secoorem@ooloor follows.
T2 The (V,
Theorem 5. Let G C GL((V) be either compact complexly averagable real linear algebraic group acting rationally a structure-preserviin way on a self self-paired vector space (., and f:ll= Tea (V. (i) - (V. x') a G-equivariant entire function. Then may write f as follows:
20
[aa
Σ
Σ
Le (at, 2 ... 8 ae, & Cli...er))
(151)
r=0
where Cfr...dy E Thee Xbimmet 11 Xe.
) #h (R", XBrumnfr x') is G-isotropic tensor for kis ker,,...rr Enen Ke, and
This is just a particular case of Theorem 5 both O(s, d) and Sp(d) are both real linear algebraic groups and their complexifications have Zariski-dense compact subgroup. D
Proof of Theorem 2
Proof of Theorem 5 When G compact, we can just repeat the proof for the orthogonal group. When G is a linear algebraic group such that its complexification has Zariski-dense compact subgroup, we can extend, using the same analytic expression evaluated in the complex tensors, the G-equivariant map f II T (V, x)) Tx. (V,X') complex Ge-qquivariant map fc II Th, (VC, Xx) + (VC X").
The map becomes G GC-equivariant because G is Zariski-dense
inside Ge by Proposition
But for E I Th, (V, xi). we have that
1 f(a) = 1 2 Fu(a).
(152)
by reality of the input and output. Hence, by linearity, we can change the not necessarily real Ct....., by the still G-isotropic and real 2 ct..... + c....... The latter is G-isotropic, finishing the proof.
'The statement is qualitatively different from (Cox et al. 2015 Chapter $1 Proposition 5), but its proof is similar. We only need to use that univariate entire function which vanishes in an infinite set with an accumulation point has vanish everywhere.
31

---

Published as a conference paper at ICLR 2026
G.4 PROOF OF COROLLARY
The following proposition is needed to prove the above corollary.
Proposition 7. Roe Goodman 2009, Theorem 5.3.3) Let G be either O(s, k: - s) or Sp(d) and be the corresponding non-degenerate bilinear form fixed by the usual action of on R i.e. : ( for Oss, k 8) and (-, >symp for Sp(d). The subspace of G-isotropic tensors T& (IR", Xo). where Xo the constant map to consist only of the zero tensor if is odd, and is of the form
Σ 0
0
(153)
DES
with the a, and 0g E (IR4)02 02 the only tensor such that for all o E IR ( @ 0a) v, if even.
Remark 7. Recall that 0 = رلا if G O d s) and 0g [Jke if G = Sp(d).
Remark Note that the above sum can written with less summands using the methods of Appendix D
Proof of Corollary generality, that
By Theorem Proposition 7 and linearity, we can assume, without loss of
brth 2 vr 2 00"
with 6 .... S. $ and r k' even.
Now, the proof is very similar to that of Corollary I However, note that now, we write
e, X E.s
where fes | i [d]} and (ē i E [d]} are dual basis to each other, i.e., for all i, (e,, Eg) 6, The reason we have to pick couple of bases is that the bilinear form is not necessarily an inner product.
Now, the proof becomes the same that of Corollary T but we have to be careful regarding the e; and the é, However, after making the pairings for contraction, we get four cases:
(v, ew, ē2) [vv w), where the sign depends on whether (.,,) is symmetric or antisymmetric.
2. (v, E,,e v.
3. ev,e,) tr, where the sign depends on whether (+, +) is symmetric or antisymmetric.
4. e, & ê = =0, where the sign depends on whether ( is symmetric or antisymmetric, or
Σ, ég 2 ej 0g.
Now, putting these back together as we did in the proof of Corollary ggives the desired statement.
H STRESS-STRAIN TENSOR
H.1 DATA
We use the leo-Hookean material dataset from Garanger et (024)) which can be found here: https://g3 thub. cowc 2000 and 000 samples, and validation and test datasets of 4 000 samples each. Similar to Garanger et al. (204)), we normalize the data for each model individually during training, then compare the outputs under the same scaling for an apples-to tosessst--applss comparison. For the baseline model, we shift and scale the data that the mean and standard deviation of the components are and respectively. For our equivariant model, we scale the matrices so that the mean and standard deviation of their eigenvalues are 0 and respectively.
SarNN-exmpples. We used training sets of 5 000,
32

---

Published as a conference paper at ICLR 2026
H.2 MODELS AND TRAINING
Our equivariant model takes the input matrix, performs the eigenvalue decomposition, sends the cigenvalues through a permutation equivariant network, then takes the output and uses the eigenvectors the reconstruct matrix. The permutation equivariant network has three hidden layers following Maron et al. (2019) with a width of 23. We compare against a baseline MLP model which also uses 3 hidden layers, but has a width of 32 result in roughly the same number of parameters. We also compare against the baseline MLP trained on an augmented dataset. For each data point in the original training dataset, we sample four elements uniformly from ( and use them to rotate the input and output tensors. Thus the resulting dataset is four times the size of the original. Each model
uses the GELU Hendryccks & Gimpel 2023) non linearity, which we found significantly improves the accuracy.
Each model is trained with the AdamW Loshchilov & Hutter 2019) optimizer using a cosine annealing learning rate schedule |mmth & Topin (2019) for 1500 epochs and batch size of 256. The baseline models were trained with learning rate 3e- while the equivariant model used e when the dataset had 5 000 samples and 1e-3 otherwise. The experiments were run on single RTX 6000 Ada GPU.
model
parameter count
layer width
learning rate
MLP Baseline
2 229
32
3e-3
MLP augmented
272
32
3
TFENN
2
23
Ours
2278
23
1e-3
Table 4: Parameter count and learning rate for each model. All models use three hidden layers. Numbers for TFENN are taken from Garanger et al. (2024).
I PATH SIGNATURES
1.1 DATA
For the orthogonal group experiments, we generate paths in = dimensions using degree polynomials for each coordinate in the domain E [-,, The coefficients of the polynomial are sampled uniformly from [-1, 1]. We calculate the signature tensors numerically using Signax Tong (2023) with 1000 points from the path and select
10 evenly spaced points for the input data. We truncate the signature to first, second, and third order tensors. For the Lorentz group, we use spatial dimensions and I time dimension for d 4 total dimensions. The train, validation, and test data sets each have 1024 trajectories. We normalize the data for each model individually during training, then compare the outputs under the same scaling for an apples-to-apples comparison.
1.2 MODELS AND TRAINING
Our model calculates the inner products of all pairs of vectors which are then input into an MIP with three hidden layers with a width of 32 and GeLU nonlinearitics Hendrycks & Gimpel (023)). The output of the model is the coefficients of linear combination of basis elements specified by Corollary The baseline models merely input all the vectors into an MLP and directly calculate the path signature tensors. The baseline models also have three hidden layers and GeLU nonlinearities. and the same width model has width 32 while the same params model has width 128. For the Lorentz data set, the same params model has width 1 116. This information is summarized in Table
We also compare against the baseline MLP trained on an augmented dataset where each data point has been replaced by four transformed copies of that data point. For the orthogonal group, we can sample four elements uniformly from the Haar measure of (dd) and use them to rotate the input and output tensors. Since the Lorentz group is not compact, we instead sample elements from a compact subgroup the following way. First we sample a boost vector 3 E R whose entries B3 are from truncated normal distribution with mean 0, variance 1, lower bound =1 and upper bound 1 which /
33

---

Published as a conference paper at ICLR 2026
ensures |
- 1. We then construct the pure boost transformation matrix:
7
-7,30
-718g
-7
18
1
(7
1)
(7 1)
Sz 2
(v 1) Lx
||
||
A8)
193
(7
1)
8.
1 1+ (7 1)
(7 1)
(154)
|
||
732
(7- 1)
3,
(7-1) ,,
1 (7 1)
|1
||
||
-79
7/
13 + (7
1)
88
(155)
|
where 7
See for example Mansuripur (2020) for derivation of this matrix. Next
sample an orthogonal matrix Q ~ O()) and construct the matrix R(Q)) = : 0 Q Finally we sample
B 0
B = +1 or -1 from Bernoulli random variable to make the time inversion matrix T(B) =
0 11.
Our Lorentz transformation the product all these matrices, = T(B)A R(Q) For the orthogonal group and the Lorentz group, the resulting dataset is four times the size of the original.
data set
model
paramcter count
layer width
learning rate
Baseline Same Width
4.391
32
5e-3
(d)
Baseline Same Params
42023
128
le-3
Baseline Augmented Ours
42 023
128
1e-3
41 557
32
5e-4
Baseline Same Width
196
32
5e-3
Lorentz
Baseline Same Params
41 728
116
1e-3
Baseline Augmented
41 728
116
1e-3
Ours
557
32
5e-4
Table 5: Parameter count and learning rate for each model. Since all models have the same number of hidden layers of the same width, the difference in the number of parameters is driven by the different inputs and outputs of each model.
For training we the AdamW optimizer Loshchilov & Hutter (2019) with cosine annealing learning rate schedule Smith & Topin (2019). To determine the peak learning rate, we held all other hyperparameters fixed and varied the learning rate. We used batch size of 32 and trained for 500 epochs. The experiments were run on a single RTX 6000 Ada GPU and took under 10 minutes to train per model, per trial.
J SPARSE VECTOR ESTIMATION DETAILS
J.1 PROBLEM SETTING
The methods developed in HHpkinn et al. (2016) and Mao & Wein (2022) each derive an h function and prove using sum-of-squares methods that with high probability that the solutions accurate. Their proofs hold under particular assumptions, including that for sparsity parameter & S 1/3, ||al| 2 I. and that Un and U1 UI...., Pa-1 (O.,, I.m))
In our experiments we violate some or of these assumptions and use data to find the best function for these new, unexplored settings. We sample the sparse vector using four different methods: Accept/Reject (AR), Bernoulli Bernoulli-Gaussial Gaussian (BG), Corrected
(CBG), and Bernoulli- Rademacher (BR). Only the Accept/Reject method explicitly satisfies || 2 1. the others only satisfy this condition in expectation. Additionally, sample vI...., Ud-1 ~ (o,,, Σ) where Σ can be the identity, a non-identity diagonal covariance, or a random covariance from a Wishart distribution.
34

---

Published as a conference paper at ICLR 2026
J.2 EQUIVARIANCE SPARSE VECTOR RECOVERY
In this section, we show sufficient condition for the Odd)-invariance of sparse vector estimation. We start with lemma the equivariance of finding eigenvector.
Lemma Let be 2(4)-tensor and let g O If u is an eigenvector for eigenvalue of M(g)b b M T, then Mg)) T a is an eigenvector for eigenvalue À of
Proof. Let be a 2(+)-tenson let g O(d), and let X, u be an eigenvalue, eigenvector pair of M
(M(g)bM())) Au b(M w) X(M(g) u)
Thus (g) T u an eigenvector for eigenvalue À of
Proposition 8. Let S Raxd with rows a. Rd that o; are column vectors. We define the action of (dd) S for all g E O(d) (M), and therefore (g) for the rows. Let Rdxd symmetric such that f(S) S Xve (h(a),( ..,)..)) where
f Raax + R h (Rd)* Aver() (-) returns normalized eigenvector for the top eigenvalue of the input symmetric matrix. If h is O(d) -quivvaarr,,,,,,, then f is ((d))invariant
Proof. Let S,h, and be defined above. Suppose that h is O(d) -quivvvrirr Suppose Avee (M) M(a....... (.a (o)) = then by lemma 6 up sign flip, we have:
Aveo (9) h(a........ a.) M()) = M(9) M(g) u M(g) (voo((()........ ......,......................... an))
(156)
Thus,
f(g. 5) = () Avoo (11g- 01 a1,...... an))
(157)
= (g. () Avec (g -a(a,....,, an))
(158)
MM) Aved (M) h(al, an) (9))
(159)
= =S M ((g) (ve
(160)
S Avec(hh(a)........
(161)
= f(S).
(162)
This completes the proof.
J.3
DATA
First we generate the sparse vectors with one of the following sampling procedures for sparsity
8 < 1/3.
Accept/Reject (A/R). A random vector vo (N In)) is sampled and normalized unit & length. We accept if [|vo 2 en and otherwise reject Note that the sparsity of up is not explicitly imposed, but the 4-norm condition suggests that is approximately sparse. The 4-norm condition of sparsity is used in Hopkins et al. (2016).
Bernoulli-Gaussian (BG)
This sampling procedure, considered in Mao & Wein 20022) defines Vo
as
[Pols =
with probability 1 1-E
(163)
[vo]i (O )
with probability E.
Note that under this sampling procedure E El/0o 3 cm.
Corrected Bernoulli Bernoulli-Gaussian (CBG)
consider a modified version of the Bermoulli-Gaussiee that replaces the values set to exactly 0 in the Hamoulli-Gaussiaa distribution with values sampled from a Gaussian with small variance. Under this distribution we have El[vollz and E||
[vo]s ~. 0, 1-c -C-Vi (1u-))(1--))
with probability 1-
(1-eJm
(164)
[wols ~ ~N (1-)
with probability 8.
35

---

Published as a conference paper at ICLR 2026
(BR)
This sampling procedure, studied in Mao & Wein (2022). defines db
as
0
with probability 1 with probability
[ool:
(165)
Ven
with probability 2-
Under this distribution we have Bilvoll2 and E || 2
J. en
Since the BG, CBG, and BR distributions have E llvo ll2 1, we also normalize these vectors unit lg length after generating them.
Proposition 9. Let dg be Bernoulli-Gaussiie vector. Then IE [|vol|]] and IE
||b0ll
3
am
Proof. Let (0, 1] and let vo be a Bernoulli Gaussian sparse vector. Thus
" EE
E 1
E
(166)
Thus, we need to find the 2nd moment of an entry of [vo]s. which we will do by first calculating its moment generating function. If Z is a Bernoulli-Gaussian random variable, then Z XY where X and Y are random variables with X ~ Bern(e) and ~N,0 H.) Then
E[ex)(LXY)] = [[X[exp(LXXY)xi]
(167)
E[exp((XY))X = P) 0) + [[exp([XY]]x= []) - etx
(168)
(169)
(1 -) =) + =|E[exp(tY]]]
(170)
Since [()))([V)] is the moment generating function of Y, a Gaussian random variable, we can see
that the 2nd moment of 7 is the 2nd moment of Y multiplied by e. Then
E [ool
= - Am
= -
(171)
EN
iml
Now, for the sparsity condition, we have
11
n
3
3
E |1oll =
(172)
8n
En
en
This follows because our previous analysis shows that the 4th moment of an entry of [vo] is 3 3 322)2 This completes the proof.
Proposition 10. Let vo be Corrected Bernoulli-Gaussaan vector. Then ||oll2 = and
Proof. Let E (0. 3] and let up E I be a Corrected Bernoulli Gaausiiin sparse vector. Thus
E 1
I
Σ E [val]]
(173)
Thus, we need to find the 2nd moment of an entry of [vo]:. which we will do by first calculating its moment-generatin function. If Z is a Corrected Bernulli-GGussian random variable, then Z = XY (1-X)W where X, Y, and W are random variables with X ~ Bern(s), Y ~ (O, on ) and W ~ ~V - where g v (1 - e) 3e). Then
etx
E(exp(1(XY + (1 (1- X)W)) E[E(exp(((XY + X)W))XX]
(174)
E[exp[tW]] H|X =] = 0) + Elexp(ty = 1JP(X = 1)
(175)
(1 - E(xxp)(( cB(c
(176)
36

---

Published as a conference paper at ICLR 2026
Since [[exp[[W]] is the moment generating function of W and B[exp(tY)) is the moment generat- ing function of Y, we can immediately get the moments of Then
Σ E [
E E+g 9 + (1 - -E)
1--- n11
E
q atq-l---9 =1. (177)
For the sparsity condition, we the same result above but now for the 4th moment
1olll
E
[uol]]]
(178)
n
9
2)
(1 -5)
1 9
(179)
NE
71 - 8)
35
(1 3E)
(180)
n²e
(181)
This completes the proof.
Proposition 11. Let uo be Gernouli-Raddmmachee vector. Then E [|0011]]] and E |100l1
Proof. Let (0, 1] and let vo be a Bernoulli-Rademacher sparse vector. Thus
E
000
2
E
(182)
[uoo?
(183)
(0)
(
2
E
1
(184)
VEn
2
v
en
(185)
(186)
We also have
4
E
=
EE
[vo]
(187)
(188)
_(1
-1
(0)
(189)
Ven
2
Ven
(190)
pn2
i=1
1
(191)
en
This completes the proof.
37

---

Published as a conference paper at ICLR 2026
J.4 CONSTRUCTION OF OBSERVED SUBSPACE
Next, we generate the noise vectors that determine the rest of the subspace. For a particular experiment trial, we generate covariance matrix that is either the identity matrix, diagonal matrix, random symmetric positive definite matrix. The diagonal matrix has diagonal entries [-],, ~ Unif 1 $ For the random covariance matrix, first we generate an n x n matrix M with entries [M (O. 1), then set Σ = M M + 0.00001 In, We then generate the noise vectors
~ (0, Σ).
the following algorithm to get random orthonormal basis of span ..... Vd-1 }. Let B be the matrix with columns Do. ..... Ud-1, and sample an orthogonal matrix from ((d). Multiply B and take the Q-R factorization, then Q is random orthonormal basis of span (No, -.. Pa-1). We prove this below with the additional assumption that the to....,, Ud-1 are linearly independent, which is reasonable given that d < and we are generating these vectors randomly.
Finally,
Proposition 12. Let n 2 d, and let B be the n x d matrix with Vo. ..... fd-1 as the columns. Assume that co..... Vd-1 are linearly independent, so rank B d. Let O be a x d orthogonal matrix and Q R B be Q-R factorization of B O. Then the columns of Q form an orthonormal basis of span .o.............................. 1-
Proof. The Q-R factorization gives that the columns of Q are orthonormal. Thus we just have to show that span to...........................t.. 1d-1] = span ( Qu...... Qa-13
Let spanfvo.... vd-1). ). so for some og...., add1 we have anto ad-1Vd-1- Let a E Rd be the vector of these coefficients, and then we have,
a = BB BOO a - QROT a Qá Thus â E Rdd is a vector of coefficients, so a E span span|Qo... ..., Qd-1J. Therefore, span spanfeo..... fe...... va-13 S
(192)
Now let b span (u......... Qd-1). J. so for some B we have b = + 31-111 Let IRd be vector of the coefficients So...., ..-11-111 Now, since rank B = rank BO = d. so in the Q-R factorization, the upper triangular R has positive diagonal entries, so it is invertible Horn
Johnson (990), Theorem 2.1.14). Then we have,
b= Q QB= S BB3.
(193)
Thus 3 E Rd is a vector of coefficients, so E span co.... va-1). Therefore, span Qo. ..., Qa-13 S span(ro..... 2d-1} which completes the proof.
J.5 MODELS
The h function given in Hopkins et al. (2016) is
d ||a:112 n )
)
h(a,....,a..)
(194)
1=1
and given in Mao & Wein 2022) is
d- a.a
3 - 1l
h(a an)
[a:112
(195)
Note that equations 194 and 195) are -(d)--qquivrran and a special case of Corollary 1 since they define sum of outer products of the inputs with coefficients that are polynomial functions of inner products of the inputs.
In comparison to these fixed methods, we propose two machine Cearningbassed models defined using the results of Section 3
The first model parametrizes
Gi.j ((ar am) amd)).mm)) [aja] (a,a) + to,a)) + ((1ae, am mmm)).mmm)))
)))
h(a......a)) =
(196)
38

---

Published as a conference paper at ICLR 2026
The Diag variant of our model only uses the norms of a, imput and a:a, the basis elements to be more comparable to (194) and (195):
e= )aaT + +qr e
"
Ma an)
(197)
where Gi.s 9s, and ai are Ond)-invariant scalar functions. The form of equation 196) follows from Corollary [ as shown in Appendix [.6] By averaging the general form of a matrix valued O(d)-
equivariant function with its transpose, we obtain the form of any O(d)--quivariant polynomial function that outputs symmetric matrix. Equation (197) follows the scheme of (I96) but only includes inner and outer products of the same vectors to be more directly comparable to (194) and (195). Corollary I specifies that qi.j di, and yr should be polynomials, but will approximate them with dense neural networks. The networks themselves are multi-layer perceptrons (MIP) with hidden layers, width of 128, and ReLU activation functions.
To demonstrate the benefits of equivariance, we also implement non-equivariant baseline model (BL) which takes as input the nd components of and outputs the d (5) components of a symmetric x d matrix. This implemented as a multi-layer pereeptron with 2 hidden layers, width of 128, and ReLU activation functions.
J.6 DERIVATION OF (96)
In the following, we derive the general form of ((d))eqquvaarann function h (Rd)) + Sa stated in (196) from Corollary D
First, we use Corollary to write the arbitrary form of an Odd)-qqiivarian function g (Rd)" Rdxd that takes values in the space of d x d matrices that are not necessarily symmetric. Given the general form of g, follows that
h = プ (g+
(198)
is the general form of an -quivvaaia function h (R + Sa-
In the notation of Corollary From Corollary with follows that y can be written in the form
we seek an
function g (Ti(R,, (R))))))) + T2(R",
Σ 0 "5..
Σ
at (((v, ())))))))) (ny 8 ... 2 CJ2 2. 8 888)
(199)
Expanding the sum of the t and 1 terms, we have
Σ
a0. (((v, 1=1) (oy, @ v2z)"
rES2 IShSJsn
+ Σ N1a (()))))))))))) 8%. (200)
oES²
The set of permutation S2 consists of (1,2) and (2, 1). Using the fact that (u @ v).1)) = u @ v.
(u @ (221)) = 8 u, and 80 = for all a E S2, we can write the above expression as
g v.,) Σ
9o.J (((2), ())) (uy, @ (y,) + + (((,, oy))
(201)
J1=1 1==
where the double sum over J1 and J2 accounts for both the sum over S J/ and the sum over the permutations in Sz. Next, we swap to standard matrix and vector notation as well as more simple indices make the equations clearer for readers who are primarily interested in the application. Thus & v = uw! 5 + Id and J1, J2, i,j become i, f, m, and we have
a a
g(u vn) Σ> tio (llae, am am))), m a,a + q (('ae, am (m.)).mm))) Ia
(202)
im1 jml
Finally, setting h = [g T) gives the desired form of h stated in (96)..
39

---

Published as a conference paper at ICLR 2026
J.7 TRAINING DETAILS
The train dataset had 5 000 vectors and the validation and test datasets had 500 vectors. For training models, used 1 (é, un)² the loss function. We used the Adam optimizer Kingma & Ba (0177) with an exponential decay rate of 0.999. We used a batch size of 100 and trained until the validation error had not improved for 20 epochs. See Table 6 for the learning rate and number of parameters for each model. We did small exploration to find these hyper-parameteen
These nyper-parameters seemed to work well, but it is always possible that better ones could be found with more exploration.
model
parameter count
learning rate
Baseline
99 087
1e-3
Ours (Diag)
58 981
5e-4
Ours
1331
3e-4
Table 6: Parameter count and learning rate for each model. Since all models have the same number of hidden layers of the same width, the difference in the number parameters is driven by the different inputs and outputs of each model.
The experiments were run on single RTX 6000 Ada GPU and took 18 hours.
sampling
SOS-I
SOS-II
MLP
SVH-Diag
SVH
Random
0.610 0.011
0.610 0.001
0.647 = 0.177
0.768 0.045
0.966 0.001
A/R
Diagonal
0.444 = 0.012
0,444 0.012
0.561 + 0.262
0.698 + 0.034
0.755 + 0.057
Identity
0.611 = 0.002
0.611 0.002
0.494 0.285
0.622 = 0.201
0.647 0.289
Random
0.963 0.001
0.963 0.001
0.783 = 0.090
0.970 0.003
0.965 0.002
BG
Diagonal
0.949 = 0.002
0.949 0.002
0.672 + 0.260
0.974 + 0.004
0.775 0.078
Identity
0.963 + 0,000
0.963 0.000
0,681 = 0.241
0.966 + 0.004
0.999 + 0.001
Random
0.409 0.005
0.409 0.005
0.836 + 0.149
0.490 0.089
0.965 0.002
CBG
Diagonal
0,292 0.005
0.292 0.005
0.835 1 0.150
0.597 + 0.027
0.722 0.013
Identity
0.418 + 0.006
0.418
0.006
0.558 1 0.216
0.368 1 0.119
0.750 0.288
Random
0.523 = 0.006
0.523 + 0.006
0.975 = 0.005
0.669 = 0.150
0.970 + 0.002
BR
Diagonal
0.340 0.010
0.340 0.010
0.943 1 0.008
0.701 1 0.041
0.913 0.002
Identity
0.526 0.005
0.526 0.005
0.949 0.006
0.570 0.199
0.898 0.001
Table 7: Train error comparison of different methods under different sampling schemes for up and different covariances for Ui,..., P4-1- The metric is (Vn, i)2, which ranges from 0 to 1 with values closer 1, meaning that the vectors are closer. For each row, the best value is bolded. For these experiments, = 100, d = 5, e 0.25, and the results were averaged over 5 trials with the standard deviation given by +0.ræx.
40


Reviews:
[{"review_id": "7BTmQpFIU1", "reviewer": "Reviewer_RRnU", "summary": "The paper provides a general mathematical characterization and construction of tensor-to-tensor functions equivariant to orthogonal, Lorentz, and symplectic groups. It gives explicit parameterizations, extends invariant-theory results, and demonstrates the approach in three domains (materials science, path signatures, sparse vector estimation), with consistent improvements over non-equivariant baselines.", "questions": "How does the proposed method scale computationally compared to established equivariant architectures (e.g., E3NN)?\n\nCan you demonstrate results on a real dataset involving Lorentz or symplectic symmetry (e.g., physical simulation or robotics)?\n\nIs there a formal or empirical ablation isolating the gain from the new isotropic-tensor formulation vs. standard equivariant parameterizations?\n\nHow large are the MLPs used for $q_{t,σ,J}$ (remark 1)? Are they a computational bottleneck?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**Originality**\n\nExtending equivariance theory to Lorentz and symplectic groups in a unified tensor framework is new and meaningful.\n\n**Quality**\n\nThe theoretical development seems mathematically rigorous, consistent, and well-supported by references in invariant theory.\n\nEmpirical results, though synthetic or semi-synthetic, convincingly show the advantage of symmetry-aware architectures across different domains.\n\n**Clarity**\n\nDespite mathematical density, definitions and proofs are clearly stated, and appendices support reproducibility.\n\n**Significance**\n\nThe proposed general framework could influence how equivariance is implemented for high-order tensor data—a major area of emerging interest in scientific ML.\n\nPotentially impacts ML applications in materials modeling, physics simulation, and tensor decomposition problems.\n\nProvides a foundation for future architectures extending GNN and geometric deep learning paradigms to more general symmetry groups.", "weaknesses": "All data are synthetic or toy (the stress–strain task uses the Garanger et al. (2024) dataset, which if I understood correctly is still a simulated material data), no large-scale, real-world or high-dimensional experiments.\n\nNo complexity or scalability discussion, crucial for practical adoption.\n\nComparative evaluation is narrow. Competes mostly with MLPs and one prior symmetry-enforcing method, no comparison to modern equivariant neural architectures (E3NN, LieConv, TFN). Without these, it’s hard to judge whether improvements are due to the new theory or simply to enforcing symmetry at all.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yeCOJ0CtPx", "reviewer": "Reviewer_A7mP", "summary": "This paper studies the representation of equivariant mappings between tensors to tensors under various symmetry groups. In particular, they characterize equivariant polynomials for general rank input and output tensors under $O(d)$, $O(s, d-s)$ and $Sp(d)$ symmetries. In the case of vectors and symmetric matrices (ie, rank 1 and rank 2 tensors) under $O(d)$ symmetry, their general characterizations specialize to those known in the literature (eg. Villar, et al. 2021). They derive an extension of their results to algebraic groups. Finally, they report empirical results using their characterization compared to existing models on predicting stress-strain relationships, path signature prediction and sparse vector estimation.", "questions": "* Do you think there is a way to additionally incorporate permutation equivariance (eg. when dealing with sets of vectors, not sequences) into your characterizations?\n* Do you have thoughts on parametrizing $SO(d)$-equivariant polynomials, such as those considered in Villar, et al. 2021 for vectors (where now cross-products come into play)?\n* Do you have thoughts on how one can predict $O(d)$-isotropic rank-$k$ tensors?\n* Can you add an experiment with existing E3NN baselines such as NequIP for the $O(d)$ equivariance experiments? For example, you could take a look at the e3tools repository: https://github.com/prescient-design/e3tools/blob/main/examples/models/conv.py for an existing PyTorch implementation. My understanding is that those models are performing pairwise tensor products but are explicitly permutation equivariant as well.\n* Do you perform data augmentation for your MLP baseline, to approximately obtain G-equivariance? If not, can you add those results?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "I am a big fan of this work. The paper is very well-written, and generalizes several existing results. While the most general constructions are not practically implementable, the authors derive specializations in specific practical settings which are practical to implement.\nI think the field will benefit from having these general characterizations in one place.", "weaknesses": "The empirical comparisons could be strengthened (see comments below).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Q2W4GZhEa7", "reviewer": "Reviewer_jDLE", "summary": "This paper develops equivariant neural network architectures for tensor-valued data under orthogonal, Lorentz, and symplecticsymmetries. The method constructs outputs as linear combinations of outer products of inputs and isotropic tensors, with coefficients given by learnable scalar functions $q_{t,\\sigma,J}$. These scalar functions are implemented as MLPs acting on the set of pairwise invariant inner products between input vectors, ensuring equivariance under the chosen group without relying on Clebsch–Gordan tensor products. The framework generalizes previous O(d) methods to indefinite orthogonal and symplectic groups, and experiments on stress–strain tensors, path signatures, and sparse recovery demonstrate flexibility and improved inductive bias.\n\nOverall, this is a strong and well-motivated contribution that provides a clean, general, and computationally efficient formalism for building equivariant networks under multiple symmetry groups. The theoretical construction is sound and practically valuable, but the manuscript would benefit from clearer explanations of indexing, permutation handling, and the role of the $q_{t,\\sigma,J}$ functions.", "questions": "1. How are the $q_{t,\\sigma,J}$ implemented in code, e.g. shared across $\\sigma$ or instantiated per isotropy class?  \n2. Does Eq. (11) enforce any permutation equivariance among input vectors, or is equivariance only with respect to the underlying group?  \n3. For the neo-Hookean example, what precise symmetry assumptions on the rank-4 tensor (e.g., $ijkl=klij$) are encoded?  \n4. How does the proposed construction’s expressive span compare to that of CG-based O(3) models?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "* Provides a unified theoretical construction of equivariant tensor functions across orthogonal, Lorentz, and symplectic groups.  \n* Avoids Clebsch–Gordan contractions by using invariant scalar functions and tensor assembly, leading to simpler implementations.  \n* The mathematical framework (Theorem 1, Corollaries 1–3) is general and connects to isotropic tensor theory.  \n* Demonstrates practical versatility across physics, geometry, and learning tasks.  \n* Empirical results show strong improvements over non-equivariant baselines.", "weaknesses": "1. **Notation clarity:** The paper should explicitly distinguish permutation indices (e.g., $i,j,k,l$) from “rotational” or group indices (e.g., $a,b,c,d$). Figure 1 blurs this distinction, making it hard to track which quantities are equivariant to which symmetries.  \n\n2. **Parameterization of $q_{t,\\sigma,J}$:**  \n   It would be helpful to more clearly explain how the MLPs are used in practice on the collection of inner product of input vectors. I understand that each $q_{t,\\sigma,J}$ outputs a scalar coefficient that modulates a tensor built from outer products and isotropic components, but I'm not sure how the MLP handles the permutation indices on the inner product \"tensor\". \n\n3. **Permutation handling:**  \n   Related to above, my understanding is that the $q_{t,\\sigma,J}$ are not permutation-equivariant functions; they are invariant with respect to the rotational/Lorentz/symplectic group but labeled by specific index tuples $(t,\\sigma,J)$.  \n   In theory, Corollary 1 sums over all permutations $\\sigma \\in S_{k'}$, but in practice the implementation collapses equivalent $\\sigma$ into isotropy classes (Appendix D), so only a small subset of unique tensor structures is instantiated. It would be good to clarify what is a helpful an analytical indexing device vs. how things are practically implemented.\n\n4. **Isotropic tensors:**  \n   Clarify whether isotropy is defined only with respect to the group action or also includes permutation symmetries (e.g., $a_{ij}=a_{ji}$).  \n   For rank-4 tensors relevant to the neo-Hookean example, an explicit enumeration of independent isotropic components under partial or full permutation symmetry would help readers connect this to classical elasticity theory.  \n\n5. **Related work:**  \n   It would be good to discuss Cartesian-tensor–based equivariant methods that similarly use inner/outer product constructions to add to the context of your methods:\n  * https://www.nature.com/articles/s41467-024-51886-6\n  * https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9711441 (which while for a only the rank 1 case, does generalize to higher dimensions)\n  * perhaps https://dl.acm.org/doi/10.5555/3666122.3667745\n\n6. **Comparison to irrep-based models:**  \n   A short discussion on expressivity, i.e., whether the span of inner/outer-product constructions matches the space of irreducible-tensor interactions for O(3), would contextualize how complete the representation is relative to Clebsch–Gordan–based methods.  \n\n7. **Minor:** Corollary 1 refers to Figure 3, but this appears to correspond to Figure 1.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HCAXd0cc5d", "reviewer": "Reviewer_ikq1", "summary": "The paper claims to provide a full characterisation of G-equivariant polynomial functions from tuples of tensors to tensor outputs, where the group G is a classical Lie group. The authors apply their characterisation in a number of numerical experiments where the input data is a tuple of vectors.", "questions": "Beyond the questions asked in the Weaknesses section:\n\n- The authors note that their Theorem 1 is impractical for computing purposes. Could they clarify further why that is, and if they have thought of any ways to mitigate this so that the theorem in all its glory could be employed in neural networks?\n- I might have missed this but where are the non-linearities included in the characterisation/paper?\n\n\nOther minor questions/points:\n- Should there be a standard deviation reported for their model in Table 2 for O(d)?\n- I think in their Example 1 equation (13), there should be a delta after beta_1<a, a> - do the authors agree?\n- In line 253 I think they should refer directly to (107) instead of Appendix D for the definition of G_4 - I originally thought it was a typo for S_4.\n- I think the authors in their Definition section should introduce the plus/minus notation where they write e.g 2_(+), just so that it is 100% clear to everyone. There is a lot going on and since it is used a lot (in place of p = +1, -1), I think the notation should be introduced explicitly.\n- Anything else they can do (e.g in the Appendix) to make their examples even clearer (say, by providing more steps) would be useful in helping the reader to understand all of the concepts that are introduced in Section 2.", "limitations": "", "rating": 10, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "- This is a very strong paper, both in its theoretical contributions and practical results. I commend the authors for such a well-written, densely packed presentation. I felt like I learned a lot as I got into the weeds while reading it.\n- To the best of my understanding, I can see that the authors have characterised O(d)-equivariant polynomial functions from tuples of tensors to a tensor output, and have also considered the particular case where the input is a tuple of tensors instead (so that this can be used in their experiments for practical reasons, as the authors state in the paper). I can also see that they have extended this to O(s, d-s) and Sp(d) equivariance for entire functions too. (I say best of my understanding as I haven't gone through every single line of the proofs but I have gone through a decent amount of it + with my background in this area I can see that it is clearly correct.)\n- I particularly enjoyed the authors explaining complex equations in words immediately after they are introduced (e.g after Corollary 1). I will be stealing this idea for my own papers in the future.\n- The results are clearly significant and deserve top billing at ICLR.", "weaknesses": "It is hard to come up with too many weaknesses, but I think the authors should consider the following:\n\n- I think the claim that the authors make in lines 49-50 is too strong (this is repeated in lines 475-476). \"General\" sounds like it could be used for any group, which I don't think is what the authors are claiming. \n- The theory looks at the generic O(d) group action, allowing p to be +1 or -1, which is extended to other classical Lie groups. However the practical experiments only consider the O(d) case for the +1 case, which (to the uninitiated) might make the theory somewhat \"overkill\". Are there any practical uses of their theory in the -1 case?\n- Whilst the related work is pretty comprehensive, I think they are missing a reference to the potential related work [1] which looks at characterising O(d) and Sp(d) equivariant functions between tensors, exactly of the form they have considered when the input tuple is of length 1. Could the authors perhaps comment on the differences between the two papers beyond this?\n\n[1] Pearce-Crump, E. - Brauer's Group Equivariant Neural Networks (ICML 2023).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)