Lorentz Local Canonicalization: How to make any Network Lorentz-Equivariant
Authors: Jonas Spinner
Luigi Favaro
Peter Lippmann
Sebastian Pitz
Gerrit Gerhartz
Tilman Plehn
Fred A. Hamprecht
Venue: NeurIPS 2025 poster ()
Keywords: Geometric deep learning
equivariance
Lorentz symmetry
Transformer
high-energy physics
particle physics
Primary area: machine_learning_for_sciences
TL;DR: Make any backbone architecture Lorentz-equivariant with minimal computational cost, with applications in high-energy physics

Abstract: Lorentz-equivariant neural networks are becoming the leading architectures for  high-energy physics.
Current implementations rely on specialized layers, limiting architectural choices. We introduce Lorentz Local Canonicalization (LLoCa), a general framework that renders any backbone network exactly Lorentz-equivariant. Using equivariantly predicted local reference frames,
we construct LLoCa-transformers and graph networks.
We adapt a recent approach for geometric message passing to the non-compact Lorentz group, allowing propagation of space-time tensorial features.
Data augmentation emerges from LLoCa as a special choice of reference frame.
Our models achieve competitive and state-of-the-art accuracy on relevant particle physics tasks, while being $4\times$ faster and using $10\times$ fewer FLOPs.

Lorentz Local Canonicalization: How to Make Any Network Lorentz-Bquivariant
Jonas Spinner"
Luigi 2
Peter Lippmann"
Sebastian Pitz
Gerrit Gerhartz²
Tilman Plehn Fred A. Hamprecht
ITP, Heidelberg University, Germany, ²CP3, UCLouvain, Belgium,
SWR, Heidelberg University, Germany
wps.nerrthhpyys...m..m uni www.umiiiieeeeeee www.uneeeeeeeege.....oeeeeeeeeee
luigi favaro@uclouuaiin
peter. p www..11ppmaaniiir. uni www.eeeeeeee.....e..ee de
Abstract
Lorentz-equivariant neural networks are becoming the leading architectures for high-energy physics. Current implementations rely specialized layers, limiting architectural choices. We introduce Lorentz. Local Canonicalization (LLoCa), a general framework that renders any backbone network exactly Lorentz-cquivariant Using equivariantly predicted local reference frames, we construct LLoCa- transformers and graph networks. We adapt a recent approach for geometric mes- sage passing to the non-compact Lorentz group, allowing propagation of space-time tensorial features. Data augmentation emerges from LLoCa as special choice of reference frame. Our models achieve competitive and state-of-the-ee art accuracy on relevant particle physies tasks, while being faster and using 10x fewer FL.Oss.
Introduction
Many significant recent discoveries in the natural sciences are enhanced by Machine Learning (ML) [2, 14, 23, 46]. particular, High Energy Physics (HEP) experiments profit from ML vast amount of data is available [12, 22, 25J. For instance, the Large Hadron Collider (LHC) at CERN collects data rates which are unmatched in the natural sciences, seeking explain the most fundamental building blocks of nature [13].
The collision between highly-energetic particles produces multitude of scattered particles. The physical laws which govern the dynamics of these particles respect Lorentz symmetry, the symmetry group of special relativity. Incorporating the Lorentz symmetry neural networks, the form of Lorentz equivariance, has proven to be critical for physics tasks which require data efficiency and high accuracy [7, 21, 39, 42]. However, existing Lorentz-equu -quiiii-euuv architectures often rely on task-specift building blocks which impede the general applicability of Lorentz- eeuevarrquuva networks the field. Other more versatile equivariant architectures impose prohibitive computational costs, limiting usability at scale, and increasing the energy footprint from training. In this work, we address these limitations by introducing a method that guarantees exact, but also partial and approximate, Lorentz equivariance with minimal additional computational costs. Our approach is agnostic to the architecture of the model, enabling the adaptation of existing graph neural networks (GNNs) and transformers to Lorentz-equivariant neural networks. We make the following contributions:
'Equal contribution
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Particle
Frames-Net
Transform local reference frames
Any backbone
Equivariant
architecture
output
(Eo.Po)"
pa
Le Le
Lapo Ve
Trenfurnc Particletiet
Regression
L
Classification
P1
p2
Lg
LiP1
MLP
Par
Generation
D2p2
Figure I: Lorentz Local Canonicalization (LLooa) for making any architecture Lorentz-equivariant. The input consists of a set of particles each associated with an energy and momentum (and possibly other particle features). The particle features are transformed into learned local reference frames, turning them into Lorentz- invariant local features. The local features then be processed by any backbone architecture produce exactly Lorentz-equivariant outputs for variety of possible tasks. Without including additional domain-specific priors, our approach elevates the performance of domain- -again- models, such as a vanilla transformer, to the SOTA in the field (sce c.g. Fig. 3).
To the best of our knowledge, we introduce the first (local) canonicalization framework for Lorentz-equivariant deep learning, novel approach that does not rely specialized layers to achieve internal representations space-time tensors. We present novel approach for Lorentz-oquivariantt prediction of local reference frames and adapt recently proposed ap- proach [30] for geometric message passing to the non-compact Lorentz group. In particular, we propose variant of scaled dot-product attention based on the Minkowski product that leverages efficient off-the-shelf attention implementations.
Our framework is readily applicable to any non-equivariant backbone architectures. It is easy to integrate and can significantly improve the training and inference times over Lorentz- equivariant architectures that use specialized layers.
In several experiments, we demonstrate the efficacy of exact Lorentz equivariance by achiev- ing state-of-the-arr or competitive results using powerful Lorentz-equivarian transformer and by making several domain -eeefff- networks Lorentz-equivariant
Our framework allows for a fair comparison between data augmentation and exact Lorentz equivariance. Our experiments support the superiority of Lorentz-equivariant models when ample training data is available in HEP, while augtenttgmtttio achieves competitive accuracy when training data is scarce.
Unlike prior equivariant models, our approach incurs only a moderate computational over- head, with 10-50% increase in FLOPPs and a 60-100% increase in training time compared to non-equivariant baselines. Compared to other SOTA Lorentz-equivariant architectures, our models train 4x faster and use 10x fewer FLOPs. Our implementation of LLoCa is publicly available on https: hhtp///eiieeeeeeeeeeeeee////laae and experiments can be reproduced with https http//gtttttttt/t com www.///wweeeeeeeeeeeemmee hhtp////eeeebbeeeeeeeee////aaaea http///llaaeeeeeeeeeeeeeeeeeee
Background
Lorentz group. The theory of special relativity [19, 35] is built on two postulates: (i) the laws of physics take the same form every inertial reference frame, and (ii) the speed of light e identical for all observers. Together they imply that every inertial observer must agree on a single scalar quantity, the spacetime interval As = At2 A Adopting natural units (c = 1), we formalize it with the Minkowski product
(x, v)
x v00 - x'y¹
x² x²y²
(1)
The Minkowski product acts on column four-vectors x = -xx =
R Rd,, which can be decomposed into a temporal part 20 and a spatial part (x' 2² T/. Equation (1) can be compactly written as x gy using the Minkowski metric g = diag -1, 11. 11 11) [34]. Hence, frame-to- to-fame trans- formations can be represented as matrices A E R4x4 that preserve the Minkowski product, i.e. that fulfill AT A = g. The inverse transformation is given by A-1 = gATz. Lorentz transformations act on four-vectors x as x > x' = Ax. The collection of all Lorentz transformations constitutes the Lorentz group O(1, In the following we will focus the special orthochronous Lorentz

---

group SO (0,(1, 3), which emerges subgroup from the constraints that det A = and Aoo > 0.
convenient parametrization of Lorentz transformation A E SO* $0*(1,3) is obtained via polar decomposition, which factors the transformation into purely spatial rotation I and boost B
-78 13 1 (7B
A = RB
1 0' 0
R
-1B
?
38"
7- (1 (1--9)-111
(2)
In this decomposition, R leaves the time component a four-vector untouched while the submatrix R E RR33 rotates its spatial coordinates. Meanwhile, B performs a hyperbolic "rotation" that mixes time with a chosen spatial direction determined by the dimensionless velocity vector 3. The Lorentz factor = (1- )1--3)-1// encodes the amount of time-dilation and length-contraction introduced by the boost. Every four-vector = (p", .),, with positive norm m = ||pl| = ((,p)) 1)/, defines such boost (p) with velocity 3 pppo-
Group representations. To characterize how transformations of a group G act on elements of vector space V. we use the notion of a group representation. A group representation is map p that assigns an invertible matrix p(g) E GL(V) to each group element such that p(9192) = p(g (((2) holds for any pair of group elements 91 92 G. In this expression, 9192 is the group product and p(g))((g)) the matrix product. For instance, four-vectur vectors transform under the d-mimensional vector representation of the Lorentz group, x'' Az, or in components, x'" Ev A",," ,²x" Higher-order tensor representations follow the pattern
fl (p(A))))
Σ "1 ... A for...
(3)
Mi...
A tensor with n indices is said to have order each index runs over four spacetime directions, so the associated representation acts on a - di-dimensional space.
Equivariance. Geometric deep learning takes advantage of the symmetries already present in problem instead of forcing the model to "rediscover" them from data [11. 15]. An operation is equivariant to a symmetry group G when the group action commutes with the function, i.c. hig- x) = g h(x) for any g E Invariance of h emerges as a special case of equivariance if the output invariant under group actions, hig xr) h(r).
High-energy physics. most steps HEP analysis pipeline, the data takes the form of a set of particles. Each particle is characterized by its discrete particle type as well as its energy E and three- momentum p which make up the four-momentum p (E, p) 8 R2, Four-moments transform in the vector representation of the Lorentz group. Examples for particle types are fundamental particles like electrons, photons, quarks and gluons as well as reconstructed objects like "jets" [40]. In addition the particle mass m ((p. )//2, particles might be described by extra scalar information such as the electric charge depending on the application. Particle data is typically processed with permutation- equivariant architectures such as graph networks and transformers, with each particle corresponding to one node or token in fully connected graph.
Lorentz symmetry breaking. Although the underlying dynamics of high-energy physics respect the unbroken Lorentz group SO* (0+(1, (1, the experimental environment in HEP introduces explicit symmetry breaking. The proton beams as well the detector geometry single out a preferred spatial axis, breaking the symmetry of rotations around axes orthogonal to the beam direction. Further, reconstruction algorithms use the transverse momentum (p² + + to define jets, a quantity which only invariant under rotations around the beam axis. As result, many collider observables retain at most residual S0(2) symmetry of rotations about the beam axis. In particular, the classification score in Sec. 5.1 is only S0(2) (ou))--iuiaaiian,, while the regression target in See. 5.2 is fully Lorentz- cquivariant.
Nevertheless, Lorentz-equivariant networks outperform SO(2) invariant networks in tasks with
"'Special" and "orthochronous" means that spatial temporal reflections are not included in the group. "More generally, a polar decomposition describes the factorization of square matrix into a hermitian and unitary part.
3

---

partial Lorentz symmetry breaking [7. 21, 39, 42]. This is achieved through a flexible symmetry breaking strategy that preserves corentz-equivarian hidden representations. Typically, fixed reference four-vectors such as the global time direction (1,0,00 0, and beam axis (0, 0, O, 1) are provided as additional inputs, restricting equivariance to transformations that leave these vectors invariant.
Related work
Lorentz-equivariant architectures. Neural networks with exact Lorentz equivariance have emerged as powerful tool for HEP analyses. A basic approach achieves Lorentz invariance by pro- jecting the particle cloud onto all pairwise Minkowski inner products and processing the results with an MLP [10, 42]. The lack of permutation equivariance in this design is overcome by PELICAN [7], which uses layers that capture all commuttaiionequivaarte mappings edge features. In order to predict non-scalar quantities such as four-vectors, existing architectures use internal features that transform in Lorentz group representations. LorentzNet [21] uses scalar and vector channels, while CGENN [39] and I--GTTr [9, 42] adopt geometric algebra representations, including antisymmetric second-order tensors. CGENN and LorentzNet message passing, whereas L-GATr relies self- attention. Unlike these models, LLoCa supports arbitrary representations with fewer architectural constraints and improved efficiency.
Equivariance by local canonicalization An alternative approach to achieve equivariance is to transform the input data into a canonical reference frame. An arbitrary backbone network then acts on the canonicalized input, before a final transformation back to the initial reference frame provides the equivariant output. Such canonicalization can be achieved in two different ways: a) via global canonicalization [24], which uses global reference frame for the whole point cloud, which is not used in our method, or b) via local canonicalization, where each point in the point cloud is equipped with its own reference frame. In contrast to global canonicalization, local canonicalization facilitates that similar local substructures will yield similar local features. Several methods have been proposed to find global canonical orientation for 3D point cloud data [6, 28, 49, 50], local canonicalization for the compact group of 3D rotations and reflections [17, 31, 47]. However, to the best of our knowledge, there exist no equivalents for the non-compact Lorentz group. As shown in [30], properly transforming tensorial objects between different local reference frames during message passing substantially increases the expressivity of the architecture. We extend their solution to the Lorentz group. While the mentioned approaches use one global reference frame one local reference frame per node, it is possible and in some cases advantageous to extend the framework of canonicalization to multiple different reference frames (per entity), as presented in the form of frame averaging in [18, 29, 36.. Extending these methods to our framework could be a promising direction for future research.
Scaling of exact equivariance vs. data augmentation. Recent work has begun to probe approxi- mate symmetries in neural networks [27] and to question whether exact equivariance improves data scaling laws of neural networks [8, 30]. Building on local canonicalization, our framework enables a controlled comparison: it allows us evaluate exactly Lorentz-equivariant model and equally engineered, non-equivariant counterpart trained with data augmentation on equal footing. We conduct several experiments to investigate how both types of models scale with the amount of training data,
Methods
The general idea of our framework is the following (cf. Fig. I): for every particle, or node, we predict local reference frame which transforms equivariantly under general Lorentz transformations. Then, we express the particle features in the predicted local reference frames, therefore transforming them into Lorentz-invarian features which can be processed by any backbone network. In the end, we obtain Lorentz-equivariann prediction by transforming the space-time space-time objects back to the global reference frame. In its minimal version, only local scalar information exchanged between nodes, constraint that significantly limits the expressivity of the architecture. We instead propose to group particle features into tensor representations to enable the exchange of tensorial space-time messages between different reference frames, extending the method proposed by Lippmann, Gerhartz et al. [20, 30].

---

4.1 Local reference frames
local reference frame is constructed from a of four four-vectors up U1, u2 us ("vierbein") that form an orthonormal basis in Minkowski space, i.e. they satisfy the condition
Σ Un " Un Juv Gab
The transformation behavior of foutovectors u, Aug implies that the local reference frames L transform as L + LA if constructed from row vectors in the following way"
u
u AAT g
A g
L
L'=
L'=
LA
(5)
A
Ug
ug A²
In the final step, we insert two Minkowski metrics as / = allowing us to identify the inverse transformation A A-1 gA g. With this relation, the orthonormality condition (4) becomes LgL" the defining property of Lorentz transformations. In Sec. 4.5, we explicitly construct local reference frames that satisfy both conditions introduced above. When working with sets particles, separate local reference frame can be assigned to each particle, see Fig. 1.
4.2 Local canonicalization
Local four-vectors vT. La are defined by transforming global four-vectors x into the local reference frame Thanks to the transformation rule + LA -1 these local vectors remain invariant under global Lorentz transformations A, i.e. Fı. La x'y LA-IAr La LL. This property readily generalizes to general Lorentz tensors f (cf. Eq. (3)), which transform as f - ((A)..
For features in the local reference frame fL f find
fr - - -
(6)
The local particle features fo can be processed with any backbone architecture without violating their Lorentz invariance, cf. Fig. 1. Finally, output features y in the global reference frame extracted using the inverse transformation = p(L ")f.. These output features y are equivariant under Lorentz transformations
- 5/ dL L p(AL -()) p(()g
y
4.3 Tensorial messages between local reference frames
Consider a system of N particles, where a mes- sage is sent from particle particle i. The message mj.., in the local reference frame Lj is Lorentz-invariant, the invariant particle features f2.- To commu- nicate this message to particle in reference frame L,, we apply the reference frame trans- formation matrix L,L,"' which gives the trans- formed message:
Global frame
Input S3 p(Lz)
Output p.pdd p(L.
it is constructed from
fi, mj.k,
+ mj.L. uud
S
Local frame Lj
Local frame Li
Particle
Particle
Figure 2: Tensorial message passing in LLoCa. Lorentz-invariant local messages transformed be- tween local reference frames.
mj.L. = "),,,,
(8)
The message representation p is hyperparam- eter that can be chosen according to the problem. Our implementation supports general tensor repre- sentations following Eq. (3), though in practice, we find equal mix of scalar and vector representa- tions sufficient for our applications. Regardless of the chosen representation, the received message miL, remains invariant since the transformation matrix L, is invariant:
I.I.,' A L I I,A-
'The object u² ut gi called the covector of the vector u, with the transformation behavior " u' A1
5

---

This formalism can be easily integrated into any message passing paradigm. Writing & and v for unconstrained neural networks and + for a permutation-inve invarriant aggregation operation, the updated features for particle i can be written as:
pupdated eb (fr. f
(pLL, ( m
(10)
We extend this framework proposed by Lippmann, Gerhartz al. [30] to the Lorentz group, see Fig.
Tensorial scaled dot-product attention. A particularly prominent example of message passing scaled dot-product attention [44]. We obtain scaled dot-product attention with tensorial message passing as a special case of Eq. (10)
pupdated Li
E
softmax (LL.
((L:L1z)) UL uLD
(11)
The objects qL,, kr, UL are the d-dimensional queries, keys and values, each computed in the respective local reference frame from the respective local node features Jt,, The softmax normalizes the attention weights over all sending nodes. The Minkowski product (+.-) reduces to the Fuclidean product if the queries, keys and values are assigned to scalar representations p. For higher-order representations, we find that Minkowski attention outperforms Euclidean attention, see App. F. Since the Minkowski product is Lorentz-invariant
(qL. ((LL, )),,) = (p(L, ((LL,))L,,, ((L, (L,,," kk,)) = (oLL""),, ((T,))
(12)
That is, keys and queries can simply transformed into the global frame reference, in which all Minkowski products can be efficiently evaluated. This facilitates the use of highly optimized scaled dot-product attention implementations [16].
4.4 Relation to data augmentation
Architectures with local canonicalization achieve Lorentz equivariance if the local reference frames satisfy Eq. (5), imposing strong constraints on their construction. Non-equivariant networks emerge as a special case where the local reference frames are the identity, emerges special case, where one global reference frame is drawn from a given distribution of Lorentz transformations. Implementing non-equivariant networks and data augmentation as special cases equivariant networks enables fair comparison between these three choices.
1. Data augmentation also
4.5 Constructing local reference frames
We now describe how to construct local reference frames for set of particles, the typical data type in high-energy physics. Each local reference frame L has to satisfy the Lorentz group condition L" gL = and transform as L + L' LA -1 under Lorentz transformations A. The local reference frames are constructed in two steps: first we use simple Lorentz-equivarian architecture predict three four-vectors for each particle. The four-vectors are then used to construct the local reference frames L following a deterministic algorithm.
In system of N particles, each particle is described by its four- mum-mmmnntum Pe and additional scalar attributes 8;, such the particle type. For each particle i, the three four-vectors are predicted as
U Σ softmax (pe(ss, 8j, (p))
pi +Pj |p + p,ll +
for 0, 1,
(13)
This operation is Lorentz-equivariant because constructs four-vectors as a linear combina- tion of four-vectors with scalar coefficients [45]. The function yp is MLP with three out- put channels that operates on Lorentz sealars. Empirically, we find that a network &p that significantly smaller than the backbone network is already sufficient, sure positive and normalized weights, a softmax is applied across the weights of all sending rescaled by their norm. This constrains the resulting vec-
App. D. To
nodes and the four-momenta tors Ug, to have positive inner products (Vi,A- Ui,k) > 0, given that the input particles have positive inner products and positive energy ph > 0. We observe that the softmax operation significantly improves numerical stability during the subsequent orthonormalization procedure.

---

Indeed, large boosts B may lead to numerical insta- bilities. However, although the particles p; are highly boosted, we find that the predicted vectors vi,A typi- cally are not, resulting in stable training. Appendix D.1 holds a detailed discussion on the numerical stability of the local frame prediction. In tasks where symmetry breaking effects reduce full Lorentz symmetry to sub- group, we incorporate reference particles (cf. Sec. 2) which are needed only for the local frame prediction.
Algorithm 1 Local reference frames via polar decomposition
Require: un, U1 22
TR1 with of Avs.
(vo, un) >>>>60 0
Ensure: L gL = g and L + L' LA-1
1: B t ((vo) using Eq. (2)
2: we Bug for k - 1,2
3: प1. U2 Ug t GranSchmidd((23, 1722)
4: R t- -ū) (2, 0z)²
For each particle i, the local reference frame L3 is constructed from the three four-vectors U/ k = 0, 1.2 using polar decomposition, cf. Eq. (2). For readability. we omit the particle index from here on. As outlined in Alg. 1, the boost (1)) is built from do. while the rotation R is derived from V and U2 These two vectors are first transformed with the same boost and then orthonormalized via the Gram-Schmidt algorithm. Alg. I is equivalent to Gram-Schmidt algorithm in Minkowski space; see App. C for details and a proof that satisfies the transformation rule L + LA- The above construction contains equivariantly predicted rotations for (33))c)rvvarrant architectures
0
5: R -
Ö R
6: I. c RB
special case with to (1, O, O, 0), though this restricted variant yields inferior performance, see App. E. In App. we describe how the LLoCa framework can be extended to the Poincaré group and (
The SchmidS orthogonalization requires linear independence of 11 and 02- In practice, find this holds reliably for systems with N 2 3 particles as carefully address the numerical stability of the local frames prediction (see App. D.1). For systems with N < 3 particles, it is not possible construct equivariant local frames from the information contained in the input set of particles. We handle this very special case by sampling the missing spatial directions vł and vi (ef. Alg. 1). randomly from an (3)-innaaria distribution, the cost of formally breaking Lorentz- equivariance to SO(3) or SO -Quu)--quivariance. In situations with partial Lorentz symmetry breaking, Sec. 2, the reference particles encoding the time and beam direction can lift the number of input vectors p: to 3 or higher.
Limitations.
5 Experiments
We now demonstrate the effectiveness of Lorentz Local Canonicalization (LLoCa) for a range of different architectures two relevant tasks in HEP. We start from the classification of jets, "jet tagging". Then, we present extensive studies on QFT amplitude regression.
5.1 Jet tagging
Jet tagging is the identification of the mother particle which initiated the spray of hadrons, "jet", from a set of reconstructed particles. This task plays a key role in HEP workflows, where even marginal gains in classification performance lead to purer datasets ultimately saving experimental resources. For this application the big-data regime is most interesting, because simulating training data is cheap. Therefore, we the JetClass dataset [38], modern benchmark dataset that contains 125M jets divided into 10 classes.
The ParticleNet [37] and ParT [38] networks are established jet tagging architectures in the high- energy physics community. ParticleNet uses dynamic graph convolutions, and ParT a transformer with class attention [43] and learnable attention bias based on Lorentz-invariant edge attributes. We use the LLoCa framework to make both architectures Lorentz-equivariant with minimal adaptations to the official implementation, see Tab. 1. Further, we compare against a vanilla transformer and its LLoCa adaptation (cf. Sec. 4.3), see App. D for details.
Without any hyperparameter tuning, the Lorentz-equivarian outperform their non-equivariant counterparts, at the cost of some extra training time and FLOPs (see Tab. 1). Interestingly, LLoCA can elevate the performance of the domain domain-agnostic transformer to the performance of the domain-specifie ParT architecture. This indicates that LooCa provides an effective inductive bias, without the need for specialized architectural designs.
models based on LLoCa consistently
7

---

Table LLoCa consistently improves the performance of non-equivariant architectures. For classification the curve (AUC) as well as training time on a GPU and FLOPs for a forward pass with batch size for the JetClass dataset from [38]. For our models, accuracy and AUC metrics are significant vanilla transformer competitive accuracy (* indicates Lorentz equivariance). For more metrics, see Tab. 5.
on JetClass dataset we compare accuracy, area
the last digit. LLoCa improves domain domain-specifee architectures and elevates
Model
Accuracy (1)
AUC (1)
Time
FLOPs
PFN [26]
0.772
0,9714
3h
3M
P-CNN [41]
0,809
0.9789
3h
12M
LorentzNet [21]
0.847
0,9856
64h
676M
MIParT-L [48]
0.861
0.9878
43h
225M
L-GAT [9]
0.866
0.9885
166h
2060M
ParticleNet [37]
0.844
0.9849
25h
413M
LLoCa-ParticleNett*
0.845
0.9852
43h
517M
ParT [38]
0.861
0,9877
33h
211M
LLoCa-ParT*
0,864
0.9882
66h
315M
Transformer
0.855
0.9867
15h
210M
LLoCa- "
0.864
0.9882
31h
301M
10a3
MSE on log amplitudes
Model
MSEx10 (L)
FLOPs
Time
10-3
MLP-I 42]
137.0 + 2
0.1M
0.4h
10-6
L-GATr [42]
1.8
0.2
528.0M
8.3h
GNN
10.5 1
0.2
20.7M
0.9h
10-7
MLPJ
LLoCa-GNN
5.0
0.2
22.3M
1.5h
GAT
Transformer
8.3
0.3
14.9M
1.3h
GNN
10-3
LLoCa-Transfonee Transformer
1.2
0.2
16.3M
2.1h
MonCa
Transformer
10-9
LLoCa-Transt
Z 2
Z
Figure 3: LLoCa surpasses SOTA performance while being 4x faster. The collisions of two particles produces single Z boson and n 2, 3, gluons (x-ticks Z + ng). LLoCa significantly improves the prediction of interaction amplitudes of the non-equivariant GNN and transformer. Left: Our TroCa-Transformer achieves state-of oheee-of-hhaatt art results over all four multiplicities. Right: Accuracy and compute for Z + 4g. The Craaa--Trasformrrr uses a tenth
FLOPs and a fourth of training time relative to the second most accurate model. Uncertainties are standard deviations over three runs. See App. D for more details.
5.2
QFT amplitude regression
In collisions of fundamental particles, the occurrence of specific final state is governed by probability. This probability is a function of the four-momenta of the incoming and outgoing particles, well their particle types. Using the framework of Quantum Field Theory (QFT), this probability or "amplitude" can greatly speed up this process, since the factorial growth of possible interactions with increasing particle number often makes exact evaluation unfcasible. The exact Lorentz invariance of amplitudes makes them prime candidate for LLoCa surrogate model.
calculate perturbative expansion. Neural surrogates [3-5, 10, 32, 33, 42]
We follow [42] and benchmark our Lorentz Local Canonicalization (LLoCa) approach on the process 1,..., additional gluons from quark-
qq Z + ng. the production of Z boson with antiquark pair. The surrogates are trained for each value of n separately. We evaluate the neural surrogates in terms of the Mean Squared Error (MSE) between the predictions and the ground truth of the logarithmic amplitudes. Architectures and training details are described in App. D.
LLoCa allows us upgrade any non-equivariant baseline and directly study the benefits of built-in

---

MLPJ
MLRI GAT:
109
GAT
109
MSE on log. amplitudes
GNN
MSE on log amplitudes
Transformer
10-1
DA-GNN
10
DA-Transt.
LoCa GNN
JlaCa-Transf.
10
10
10
10
10-4
10
10-5
Z 7+48
10-5
X+48
10a
10"
10
10
10
10
10a
10
102
10
10"
10
Number of training samples
Number of training samples
Figure Data efficiency LLoCa networks. Lorentz-equivariant amplitude regression Z + 4g production is more data efficient in the large data regime for both graph networks (left) and transformers (right). Perhaps surprisingly, data augmentation (DA) outperforms equivariance for small datasets.
Lorentz equivariance. To that end, we construct LLoCa adaptations of baseline graph neural net- work (GNN) and a vanilla transformer as described in Sec. 4.3. For comparison, we consider a simple MLP actant corentz-invarian improves the accuracy of non-equivariant architectures, see Fig. 3. Crucially, without including addi- tional domain sppecff priors, our -ranssfoooormm achieves state-of sate-of-thee-rt- the-art-performant outper- forming the significantly more expensive L-GATr architecture over the entire range of multiplicities.
(MLP-I) [10] and the performant L-GATr [42]. LLoCa consistently
Message representations. Beyond the im- provements obtained through exact Lorentz equivariance, we have identified the tensorial message passing in LLoCa (cf. Sec. 4.3) as crucial factor for expressive message passing based on local canonicalization. We compare the performance of the I.LoCa-rranffommer with different message representations in the 16-dimensional attention head, see Tab. 2. We compare 16 scalars against four 4-dimensional vectors, a single 16-dimensional second-order tensor representations as well as our default of eight scalars combined with two four-vectors. All tensorial representations significantly out- perform the scalar message passing. For reference, we include the non-equivariant transformer and the same model with global canonicalization (one shared learned reference frame to every particle).
Table 2: Effectiveness of tensorial messages. LLoCa- Transformer with different hidden tensor representations on Z + 4g QFT amplitude regression.
Method
MSE 100 5)
Non-equivariant
8.3
+ 0.5
Global canonical.
4.4
1.0
LLoCa (16 scalars)
40
+ 4
LLoCa (single 2-tensor)
2.0
0.4
LLoCa (4 vectors)
1.4 + 0.2
LLoCa scalars, 2 vectors)
1.0
0.1
Lorentz equivariance at scale. Lorentz equivariance introduces strong inductive bias in neural networks, typically at the cost of extra compute. Data augmentation provides cheap alternative which aims at learning approximate Lorentz equivariance directly from data. LLoCCa includes data augmentation as a special case, allowing us to directly compare exact Lorentz equivariance (LLoCa) with data augmentation (DA) using the same backbone architecture and training parameters. For this purpose, we focus on the most complicated process Z + 4g and train surrogate models with different fractions of the training dataset, see Fig. Augmented trainings outperform the non- equivariant baseline for small training datasets due to the symmetry group information encoded in data augmentations, but the two approaches agree for large dataset sizes where model expressivity becomes the limiting factor. The foreetzzeqqivariin models surpass the competition in this big-data regime due to their increased expressivity at fixed parameter count. Interestingly, we observe that trainings with data augmentation outperform even the Lorentz-equivarian models for small training dataset sizes. We include extensive ablation studies on the method used for local canonicalization, tensorial message representations, and data augmentation in App. E.

---

5.3 Computational efficiency
While Lorentz-equivariin architectures offer valuable inductive biases, they often come with sig- nificant computational overhead compared to the corresponding non-equivariant architectures. To evaluate this trade-off, we report both nominal FLOPs which capture architectural complexity and empirical training time, which also reflects implementation efficiency, in Tab. 1 and Fig. 3. Our LLooa networks introduce a modest increase in computational cost, with FL.OPs rising by 10-50% and training time by 60-100%, depending on the task and architecture. The computational overhead of LLLCCa networks is split about evenly between (i) the initial prediction of local frames and (ii) the name-to-frame transformations in tensorial message passing. A more efficient implementation is likely to further reduce both sources of training-time overhead within LLoCa networks, In contrast, the state-of-the-art L-GATr architecture [42] incurs significantly higher costs, with 10x increase in FLOPs and 4x increase in training time relative to our LLoCa- Transformer, see App. E.2.
Conclusion
Particle physics provides ample data and studies systems that exhibit Lorentz symmetry. Neural net- works that respect Lorentz symmetry have emerged as essential tools in accurately analyzing the data and modeling the underlying physics. However, existing Lorentz-cquivariant architectures introduce large computational costs and rely on specialized building blocks, which limits the architectural de- sign space and hinders the transfer of deep learning progress from other modalities. We address these issues by introducing Lorentz Local Canonicalization (LLoCa), novel framework that can make any off-the- architecture Lorentz-equiiariant. In several experiments, we have demonstrated the effectiveness of LLoCa by achieving stateoof-the-ar results even with domain -agnosii back- bone architectures. Furthermore, our approach significantly improves established domain-specifie but non-equivariant architectures, without any hyperparameter tuning. While the LLoCa framework introduces computational overhead, our LLoCa models are still significantly faster at training and inference than other SOTA -ventz-equivariint networks. We hope that LLoCa facilitates the design of novel Lorentz-equivariant architectures and helps to bridge innovations between machine learning in particle physics and other domains.
Acknowledgements
The authors thank Víctor Bresó for generating the amplitude regression datasets. They are also grateful to Huilin Qu for assistance with the JetClass dataset and for support with the ParticleNet and ParT implementations. Further thanks go to Johann Brehmer, Víctor Bresó, Jesse Thaler, and Henning Bahl for valuable discussions and insightful feedback on the manuscript.
The authors acknowledge support by the state of Research Foundation (DFG) through the grants INST 35/1597-1 FUGG and INST 39/1232-1 FUGG. Computational resources have been provided by the supercomputing facilities of the Université catholique de Louvain (CISM/UCL) and the Consortium des Equipements de Calcul Intensif en Fédération Wallonie Bruxelles (CÉCI) funded by the Fond de la Recherche Scientifique de Belgique (F.R.S.-FRSS) under convention 2.5020. 11 and by the Walloon Region. The present research benefited from computational resources made available Lucia, the Tier-1 supercomputer of the Walloon Region, infrastructure funded by the Walloon Region under the grant agreement n 19100277.
through bwHPC and the German
J.S. and T.P. are supported by the Wurttee-rrrtteeeee-----ii Stiftunn through the program Internationale Spitzenforschung, project Uncertainties - Teaching Al its Limits Forschungsgemeennccaatt (DFG, German Research Foundation) under grant 396021762 TRR 257 Particle Physics Phenomenology after the Higgs Discovery. J.S. is funded by the Carl-Zeiss-Se Zeiss-Stiftung through the project Model-Based Al: Physical Models and Deep Learning for Imaging and Cancer Treatment. L.F. is supported by the Fonds de Recherche Scientifique FNRS under Grant No 4.4503.16. 16. P.L. and F.A.H. acknowledge funding by Deutsche Forschungsgemeinschtt (DFG, German Research Foundation) Projektnummer 240245660 SFB 1129.
the Deutsche
This work is supported by Deutsche Forschungsgemeinscaatt (DFG) under Germany's Excellence Strategy EXC-2181/1 390900948 (the Heidelberg STRUCTURES Excellence Cluster).
10

---

References
[1] Johan Alwall, Michel Herquet, Fabio Maltoni, Olivier Mattelaer, and Tim Stelzer. MadGraph 5: Going Beyond. JHEP, 06:128, 2011. doi: 10.1007/JHEP0662001)) (Cited on page 18)
[2] ATLAS collaboration. Observation of new particle in the search for the Standard Model Higgs boson with the ATLAS detector at the LHC. Physics Letters B, 716(1):1-29, 2012. (Cited on page I)
[3] Joseph production at hadron colliders using amplitude neural networks. Journal of High Energy Physics, 2021(8):1-30, 2021. (Cited page 8)
Simon Badger, and Ryan Moodie. Optimising simulations for diphoton
[4] Simon Badger and Joseph Bullock. Using neural networks for efficient evaluation of high multiplicity scattering amplitudes. Journal of High Energy Physics, 2020(6): 2020. (Not cited.)
[5] Simon Badger, Anja Butter, Michel Luchmann, Sebastian Pitz. and Tilman Plchn. Loop amplitudes from precision networks. SciPost Physics Core, 6(2):034, 2023. (Cited on page
[6] Justin Baker, Shih-Hsin Wang, Tommaso de Fereex, and Bao Wang. An explicit frame con- struction for normalizing 3D point clouds. In Forty-first International Conference Machine Learning, 2024. (Cited on page 4)
[7] Alexander Bogatskiy, Timothy Hoffman, David W. Miller, and Jan T. Offermann. Pelican: Permutation equivariant and lorentz invariant or covariant aggregator network for particle physics, 2022. (Cited pages and 4)
[8] Johann Brehmer, Sönke Behrends, Pim de Haan, and Taco Cohen. Does equivariance matter at scale? arXiv preprint arXiv:2410.23: 2024. (Cited on page 4)
[9] Johann Brehmer, Víctor Bresó, Pim de Haan, Tilman Plchn, Huilin Qu, Jonas Spinner, and Jesse Thaler. A lorentz-equivariiat transformer for all of the lhe. arXiv preprint arXiv:2411.00444 2024. (Cited on pages 4, 8, and 22)
[10] Víctor Bresó, Gudrun Heinrich, Vitaly Magerya, and Anton Olsson. Interpolating amplitudes, 2024. URL https: //tpx///vvvv/////////////////av/ org/ abs /2412. 241.//9922......... (Cited pages 4, 8, and 9)
[11] Michael M Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković, Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint arXiv:2104.13478, 2021. (Cited on page 3)
[12] Anja Butter, Tilman Plehn, Steffen Schumann, Simon Badger, Sascha Caron, Kyle Cranmer, Francesco Armando Di Bello, Etienne Dreyer, Stefano Forte, Sanmay Ganguly, et al. Machine learning and LHC event generation. SciPost Physics, 14(4):079, 20233 (Cited page 1)
[13] CERN Data Centre. Key facts and figures, 2021. web. cern. ch/sited testee/deffane http://eeeeeeeeeee/////eeeeeette pdf. (Cited on page 1)
[14] CMS collaboration. Observation of a new boson at mass of 125 GeV with the CMS experiment the LHC. Physics Letters B, 716(1):30-61, 2012. (Cited on page 1)
[15] Taco Cohen. Equivariant Convolutional Networks. PhD thesis, University Amsterdam, 2021.
(Cited page.)) 3)
[16] Tri Dao, Dan Fu. Stefano Ermon, Atri Rudra, and Christopher Ré. FlashAttention: Fast and memory-efficient exact attention with IO-awareness. Advances in Neural Information Processing Systems, 35:16344-16359 2022. (Cited on page 6)
[17] Yuanqi Du, Limei Wang, Dieqiao Feng, Guifeng Wang, Shuiwang Ji, Carla P Gomes, Zhi-Ming Ma, al. A new perspective on building efficient and expressive 3D equivariant graph neural networks. Advances in Neural Information Processing Systems, 36, 2024. (Cited on page
11

---

[18] Nadav Dym, Hannah Lawrence, and Jonathan W Siegel. Equivariant frames and the impossibil- ity of continuous canonicalization. arXiv preprint arXiv24402.16077, 2024. (Cited on page 4)
[19] Albert Einstein. Zur Elektrodynamik bewegter Körper. Annalen der Physik, 4, 1905. (Cited on page 2)
[20] Gerrit Gerhartz, Peter Lippmann, and Fred Hamprecht. Equivariance by local canonicalization:
A matter of representation. arXiv preprint arXiv:2509.26490 2025. (Cited on page 4)
[21] Shiqi Gong, Qi Meng, Jue Zhang, Huilin Qu, Congqiao Li, Sitian Qian, Weitao Du, Zhi- Ming Ma, and Tie-Yan Liu. An efficient lorentz equivariant graph neural network for jet tagging. Journal of High Energy Physies, 2022(7), July 2022. ISSN 1029-8479, doi: 10. 1007/jhep07(2022)050. URL http: doi 1007 pages I, 4, and 8)
(2022) 030. (Cited on
[22] Dan Guest, Kyle Crammer, and Daniel Whiteson. Deep learning and its application to LHC physics. Annual Review of Nuclear and Particle Science, 68:161-181, 2018. (Cited page 1)
[23] John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ron- neberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, al. Highly accurate protein structure prediction with alphafold. nature, 596(7873):583-58,, 2021. (Cited on page 1)
[24] Sékou-Oumar Kaba, Arnab Kumar Mondal, Yan Zhang, Yoshua Bengio, and Siamak Ravan- bakhsh. Equivariance with learned canonicalization functions. In International Conference Machine Leamiin, pages 15546-15566. PMLR, 2023. (Cited on page 4)
[25] Gregor Kasieczka, Tilman Plehn, Anja Butter, Kyle Cranmer, Dipsikha Debnath, Barry M Dillon, Malcolm Fairbairn, Darius A Faroughy. Wojtek Fedorko, Christophe Gay, et al. The machine learning landscape of top taggers. SciPost Physics, 7(1):014, 2019. (Cited on page
[26] Patrick T Komiske, Eric M Metodiev, and Jesse Thaler. Energy flow networks: deep sets for particle jets. Journal of High Energy Physics, 2019(1):1-466, 2019. (Cited on pages 8 and 22)
[27] Marcel F Langer, Sergey N Pozdnyakov, and Michele Ceriotti. Probing the effects of broken symmetries in machine learning. arXiv preprint arXiv:2406.17747 2024. (Cited on page 4)
[28] Feiran Li, Kent Fujiwara, Fumio Okura, and Yasuyuki Matsushita. A closer look at rotation- invariant deep point cloud analysis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 16218-16227, 2021. (Cited on page 4)
[29] Yuchao Lin, Jacob Helwig, Shurui Gui, and Shuiwang Ji. Equivariance via minimal frame aver- aging for more symmetries and efficiency. In International Conference on Machine Learning. pages 30042-30079. PMLR, 2024. (Cited on page
[30] Peter Lippmann, Gerrit Gerhartz, Roman Remme, and Fred A Hamprecht. Beyond canoni- calization: How tensorial messages improve equivariant message passing. In The Thirteenth International Conference on Learning Representations, 2025. (Cited on pages 2, 4, 6, and 14)
[31] Shitong Luo, Jiahan Li, Jiaqi Guan, Yufeng Su. Chaoran Cheng, Jian Peng, and Jianzhu Ma. Equivariant point cloud analysis via learning orientations for message passing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18932-18941, 2022. (Cited on page 4)
[32] Maître and Truong. One-loop matrix element emulation with factorisation awareness. Journal of High Energy Physics, 2023 2023. (Cited on page 8)
[33] Daniel Maître and Henry Truong. A factorisation-awan matrix element emulator. Journal of High Energy Physics, (021(11):1-24 2021. (Cited on page 8)
[34] Hermann Minkowski. Die Grundgleichungen für die elektromagnetischen Vorgänge in bewegten Körpern. Nachrichten von der Gesellschaft der Wissenschaften zu Göttingen, Mathematisch- Physikalische Klasse, 1908:53-111, 1908. (Cited page 2)
12

---

[35] Henri Poincaré. Sur la dynamique de I' I'électron. Circolo Matematico di Palermo, 1906. (Cited on page 2)
[36] Omri Puny, Matan Atzmon, Heli Ben-Hamu, Ishan Misra, Aditya Grover, Edward J Smith, and Yaron Lipman. Frame averaging for invariant and equivariant network design. arXiv preprint arXiv:2110.033336, 2021. (Cited on page 4)
[37] Huilin Qu and Loukas Gouskos. Jet tagging via particle clouds. Physical Review D, 101(5), March 2020. ISSN 2470-0029.
doi: 10.1 103pphysredd.. 1et..1011111111 http:///d/d doi. org/10. 1tt.:1110/PPPPPP 110///yyseee....... 101.056013 (Cited on pages 7, 8, and 22)
[38] Huilin Qu, Congqiao Li, and Sitian Qian. Particle transformer for jet tagging, 2024. (Cited on pages 7, 8, 20, 22, and 23)
[39] David Ruhe, Johannes Brandstetter, and Patrick Forré. Clifford group equivariant neural networks. In Advances in Neural Information Processing Systems, volume 37, 2023. (Cited on pages and 4)
[40] Gavin P. Salam. Towards Jetography. Eur. Phys. J. C, 67:667--6866 2010. doi: 10.1 10.1 140/epje/ s 1000520110-111--- (Cited page 3)
[41] Albert M Sirunyan al. Identification of heavy, energetic, hadronically decaying particles using machine-learning techniques. JINST, (5(06):P0005,, 2020. doi: 00.1088/1748-0220 06/P06005. (Cited on pages 8 and 22)
[42] Jonas Spinner, Victor Bresó, Pim De Haan, Tilman Plehn, Jesse Thaler, and Johann Brehmer. Lorentz-equivarian geometric algebra transformers for high-energ, physics. In Advances in Neural Information Processing Systems, volume 37, 2024. URL https::////// //axxiv org/ /bbs/ 2405. 14806. (Cited on pages 1, 4, 8, 9, 10, 17, 18, 22, and 23)
[43] Hugo Touvron, Matthieu Cord, Alexandre Sablayrolles, Gabriel Synnaeve, and Hervé Jégou. Going deeper with image transformers. In Proceedings of the IEEE/CVF international confer- ence computer vision, pages 32 42, 2021. (Cited on page
[44] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Eukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Information Processing Systems, 30, 2017. (Cited on page 6)
[45] Soledad Villar, David W Hogg, Kate Storey-Fisher, Weichi Yao, and Ben Blum-Smith. Scalars are universal: Equivariant machine learning, structured like classical physics. Advances in Neural Information Processing Systems, 34:28848 228863, 2021. (Cited on page 6)
[46] Hanchen Wang, Tianfan Fu, Yuanqi Du, Wenhao Gao, Kexin Huang, Ziming Liu, Payal Chandak, Shengchao Liu, Peter Van Katwyk, Andreea Deac, et al. Scientific discovery in the age of artificial intelligence. Nature, 620(7972):47-60) 2023, (Cited on page 1)
[47] Xiyuan Wang and Muhan Zhang. Graph neural network with local frame for molccular potential energy surface. In Learning Graphs Conference, pages 19-1. PMLR, 2022. (Cited on page
[48] Yifan Wu, Kun Wang, Congqiao Li, Huilin Qu, and Jingya Zhu. Jet tagging with more- interaction particle transformer*. Chin. Phys. C, 49(1):0131 10, 2025. doi: 1088//616
1 1137/ad7f3d. (Cited on pages 8 and 22)
[49] Chen Zhao, Jiaqi Yang, Xin Xiong, Angfan Zhu, Zhiguo Cao, and Xin Li. Rotation invariant point cloud analysis: Where local geometry meets global topology. Pattern Recognition, 127: 108626, 2022. (Cited on page 4)
[50] Yongheng Zhao, Tolga Birdal, Jan Eric Lenssen, Emanuele Menegatti, Leonidas Guibas, and Federico Tombari. Quaternion equivariant capsule networks for 3D point clouds. In European conference on computer vision, pages 1-19. Springer, 2020. (Cited on page 4)
13

---

A Tensor representation
The tensor representation is introduced in Eq. (3). The tensor representation is a group representation of the Lorentz group ((1,3) 3) and is used in this paper to define the transformation behavior of tensorial space-time features when transformed or out of local reference frames.
Proof representation property. The representation property follows from the fact that the 4 x Lorentz transformation matrices itself form a representation. For any A1, SO (1, 3) we have
(o(A)))(A))))) = A1 A1"1 A2 in A2 Vn. T = (A112) (M un = (A(A
(14)
Representation hidden features. The input and output representations are determined by the input data and the prediction task. Contrarily, the internal representations are chosen as hyperparameters. In our framework, we choose direct sum of tensor representations, which again forms representation. feature f' will transform under Pr - #t @ ... @ Pv. with a block diagonal matrix:
(A(A)
(r(Aff=
(15)
(k(A))
So that the composition scalar, vectorial and tensorial features can be chosen freely. The feature dimension of f will be given by dim(f) dim(pi) + dim(px).
B Extension of LLoCa to O d) and the Poincaré group
First, note that LLoCa does not cover the full Lorentz group including parity and time reversal, but only the cullecco subgroup denoted by SO* (1,3). An extension to 0(1,3) requires to also predict extra parity and time reversal transformations, similar to what has been done in [30] for extending local canonicalization from SO(3) to O(3). Afterwards, extending LLoCa O d) is straightforward. Using Eq. (13), one may predict d many + 1-dimensional vectors. Then, a trivial extension of Alg. with d-dimensional to predict orthonormal local frames for canonicalization w.r.t. O(1, d). Furthermore, the tensorial message passing based the tensor representations defined by Eq. (3) be extended directly to the case of O(1,)) d).
-rham- algorithm for the spatial rotation can be used
In addition, invariance w.r.t. translations could be obtained in the LLoCa framework simply by operating only on differences of four-vectors. all our experiments, the input is provided in momentum space where translation equivariance/iivrriance is not preferable. However, an extension to the Poincaré group is possible by modifying Eq. (13) to
U. softmax(pp(s: 8y, || - x -1,,-x11)))00 (z))))( - =2,),
(16)
where we assume to work in position space with four-vectors xy. One possible challenge may to ensure that at least one of the predicted U has positive norm from which one construct the boost B, cf. Alg.
C Background information on the construction of local reference frames
C.1 Gram-Schmidt algorithm in Minkowski space
In this section we discuss how to construct reference frames that satisfy the transformation behaviour L > LA 11 of Eq. (5). This can be achieved via Tram-SShhiie orthonormalization algorithm adapted for Minkowski vectors. Similar to Alg. 1, this algorithm starts from three Lorentz vectors up, U1 02 and returns local reference frame L. While, in Alg. 1, L is decomposed into rotation and boost, here we construct I. directly from a set of four orthonormal vectors, as illustrated in Eq. (5).
Starting from three Lorentz vectors Vo, U1, 12, apply the following algorithm to obtain four four-
14

---

vectors i1, u2, Uy that satisfy Eq. (4)
norma (v) = ||a|| +
ug form4(vo):
(1), uo)
01 norm
21 uo
(wo: uo
(22 uo)
(22, u1)
02 norm
1/2 do
11
(uo: uo)
(u1: u1)
Σ K.P,o.
gete Eupax My ui
(17)
We 10-15 and the absolute value of the Lorentz inner product ||>||| = (x 2) The last line uses the Minkowski index notation to construct U3 using the totally antisymmetric tensor Gurper the extension of the cross product to Minkowski space. This tensor defined 60123 1, and flips sign under permutation of any pair of indices and is zero otherwise.
We identified two advantages to the polar decomposition approach (PD) described in Alg. compared to the Gram-Schmide algorithm in Minkowski space (GS") described in Eq. 17 above. First, the PD) has the conceptional advantage of explicitly separating the boost and rotation. This allows to easily study properties of the boost and rotation parts independently, to identify the special cases of a pure boost and a pure rotation, and, necessary, to constrain the magnitude of the boost vector vo. Second, we find that the PD approach of boosting into reference frame typical for the particle set and then orthonormalising two vectors in that frame is numerically more stable than the GS* approach of directly orthonormalising set of three vectors, although the approaches are formally equivalent, see App. C.2.
When constructing the local reference frame from this orthonormal set of vectors u0, U1 U2 U3. it is important to additionally transform them into covectors u where up denote the vectors constructed in the algorithm above.
C.2 Proof that polar decomposition via Alg. 1 is equivalent to Gram-Schmidt algorithm in Minkowski space
Let us first consider the Gram-Schmidt algorithm in Minkowski space (denoted by GS*). As for Alg. 1 we start from the of four-vectors dg, k 1, predicted by Eq. (13), assuming (No, vo) > 0. Now, we use the Gram-Schmidt algorithm in Minkowski space described in Eq. (17), to orthonormalize the three vectors: )==, 1,2,3 GS (vo, vi, U2). Then, the transformation matrix
constructed from these orthonormal vectors
GG
g
(18)
9 Ug g
Equation (18) has the correct transformation behavior L {ue} h==0 1.2.3 are also four-vectors due to the Lorentz--quivaaianne of GS4 According the po- lar decomposition, we can decompose a proper Lorentz transformation into a pure rotation At and general boost B. By realizing that
LA-1 shown in Eq. (5) since
L= L = 6 R
(7g
7
-18")
-((r 178-7
(19)
we can identify the boost part of a Lorentz transformation from the first row, i.e. we consider I RGSBCs
BGS_ = -Bă -0o=-uu/uu
-To/ug) = B(wn)-
(20)
15

---

Also note that uo = / (vo. ub) due to the design of GS*, therefore B(uo) B(vo). Consequently, the pure rotation RGS given by
((n))
RGS = (BSS)
w (VD))
(21)
u (v))
ug (gggB(v))
We now show that the local frame ID obtained from polar decomposition (PD) via Alg. is equal to L S Starting from the same set of four-vectors Uk2 k 0, 1, 2, first construct the rest frame boost (())). Since (uo) = B(w)). the boosts in polar decomposition and Minkowski Gram-Schmidt are equal, BPD BGS, To prove that the rotation parts agree well, i.c. that RPD = RCS, we first show that the set of GS vectors obtained from the spatial parts of ( o1 and ((vg) embedded in set of four-vectors with zero energy component, is equal to the set given by the last three orthonormal vectors obtained from GS of (o ((v)) 61, and (1)) U2 i.e.
{(0, (B(r)) (( U2 k (GS*(((V))) B(0)) ( U2) k=,,, Indeed,
140 = norm (B(w))u) = (1,
U1 = norm, (v1 - (o), 0)) = =00, norms(v1))
U2 norma(U2 (v2, 0) (0, ūh (122 xh)) norma (0, (T2 ūT)
= (0, norma - ti (72 ūi))
wy
Σ gMr
Cupor uf un²
Σ g 9tth
ckosj 2 uiub Σ
gut uhosfutu
u.p.o.x
1f Enis EORE un = [11 E2 if 1
(22)
k
Note that (wo, un) = and (u:, uz) = - for i 1, 2. Therefore, the first component of the GS* vectors other than un is always zero while the spatial components are equal to those obtained from GS (cf. Eq. (25) below). Since GS is Lorentz-equivariant, we have GS (B(r))
((o))oo vo. B(vo)u1, B(ug) (22))))) = ( GS (uo U1, 02) (e (ux 1, Taken together, we have thus shown that the pure rotation RPD is the Lorentz transformation which its rows contains the covectors corresponding to namely:
the orthonormal four-vectors
4 B(vo)
T B (1))
" B(vo)'
u gB(())
RPD =
RGS
(23)
us B(vo)) 9
up (v0)
up B(uo)"
Ug (Bv)
where in the last step we have inserted I4 = 99 and then identified B(vo)-1= gB(w)) (T. We indeed find that also the rotational part of the two local reference frames is the same which concludes the proof.
Since LD = LG, the correct transformation behavior L + LA of the local reference frame based on Alg. immediately follows from the correct transformation behavior of GS
D Experimental details
D.1 Constructing local reference frames
Here we give additional details on the local reference frame construction presented in Sec. 4.5. We use double precision for precision-critical operations minimize sources equivariance violation. In particular, we use double precision for all operations in the vector prediction (13) and the polar decomposition Algorithm I, except for the neural network spa (8i, 83, (pt.P))) which is evaluated in single precision.
16

---

Equivariant vector prediction. We use the same architecture for the network 9i in all experiments, an MLP with 2 layers and 128 hidden channels. We find that normalizing the result of Eq. (13) improves numerical stability. The modified relation reads
T Σ
1/ Us
(24)
where ||a|| = / (a, a) and i I...N runs over the N particles in the set. Note that //,x 2 0 and lp: + pyl 2 by construction.
3D Gram-Schmidt orthonormalization. Here we describe the operation GramSchmidt((ij, u72)
(or GS (w), (2)) in Algorithm in more detail
W
normg(12)
| +
T norma (wi),
12 normy (182 ūn (162 (1))
T3 = ū, x U2-
(25)
We use 10-15 and denotes the cross product. The resulting vectors ūn ū2 ig are orthonormal, iee they satisfy it ūz dzg.
Numerical stability of local frame prediction. Most particles at the LHC can be considered massless as the typical energy scale is much larger than the mass of single particles. When operating on four- nour-momenta using single precision, the particle mass value is often modified due numerical underflows, leading to particles with zero mass in some cases. To avoid downstream instabilities, enforce minimal particle mass m. by increasing the energy of all input particles pt as E' /mL + E hence m² m m We use m. 10-5 and m. 5 10-3 for the amplitude regression and tagging experiments, respectively.
Furthermore, we have carefully designed Eq. (13) to avoid vectors v1o0 with small or negative norm which would cause instabilities due large boosts. The reason that local frames with large boosts B problematic is that they multiply the latent features in each message passing step, possibly leading to exploding gradients. Such large boosts are caused by vectors 15,0 constructed in Eq. (13) that have small but positive norm, because they yield large values for in Eq. (2). These small-norm vectors v1,0 can be caused by the typically small-norm particle 4-momenta p; in Eq. (13) either if all 4-momenta p; point in similar direction, or if the coefficient softmax((x )) is small for all except particle.
Even bigger problems could occur if the vectors Us,0 had zero or negative norm, because the boost matrix B in Eq. (2) is ill-defined due to f2 > 1. In practice, such pathological vectors 0; do not occur with our implementation.
Moreover, we have identified several techniques to avoid large boosts already at initialization, where such instabilities most likely to occur. First, we include softmax operation in Eq. (13) to prevent any negative-norm vectors in the process. Second, we use regulator masses to enforce a lower limit on the norms of the input particles pę. Third, multiply by p; Pý instead of simply p; in Eq. (13), because + Pj typically is not strongly boosted anymore.
Lastly, the (3( Fram--Shhmid orthonormalization used in Alg. I requires two linearly independent vectors. We ensure that the two vectors Wi, U2 are not collinear nor null by calculating the cross- product between the two vectors. Concretely, if || X T22l < Ecallimears we replace Wi and W2 by wi Coollincar 82 and T2 + Eoollincar og. respectively. where Đ1.2 are random normal directions, i.c. 5, (O, 1). In experiments, we Gaollinear 10-16 and we observe that, besides handful of exceptions at initialization, the predicted vectors are never regularized. This indicates that our orthonormalization procedure is well under control.
D.2 QFT amplitude regression
Dataset. The amplitude regression datasets are publicly available on https: org/ records/16793011. They are generated in complete analogy to Spinner et al. [42]. The Monte
17

---

Carlo generator MadGraph [1] is used to generate particle configurations for a given process following the phase-space distribution predicted in QFT. Each data point contains the kinematics of the two colliding quarks, the intermediate Z. bosons, and the additional gluons. The events have unit weight and, therefore, closely simulate the distributions which can be observed in the experiments. The amplitude is re-evaluated using a standalone MadGraph package and stored with the particle kinematics. Physics-motivated cuts are needed to avoid divergent regions. We apply a cut on the transverse momentum, pr > 20 GeV, and on the angular distance between gluons, AR >> 0.4.
The + {1. 2, g datasets contain 10M events each, while we generate 100M events for the more challenging Z + 4g dataset to complete our scaling studies. For validation and testing, we use the same independent dataset as in [42], from which we use 100k events for validation and 500k events for the final evaluation. The amplitudes A are preprocessed with a logarithmic transformation followed by standardization,
A' A'= log log A
(26)
olog A
The input four- four-momenta are rescaled by the standard deviation of the entire dataset. Finally, we remove the alignment along the 7-axis in two steps. First, boost the inputs in the reference frame of the sum of the two incoming particles, also called center-of-mass frame. Then, we apply a random general Lorentz transformation.
Models. Our GNN and LLoCa-GNN use standardized four-momenta and one-hot encoded particle types inputs. For LLoCa-GNN, the four-momenta are transformed into their local frames. Both networks share the same graph neural network backbone. The GNN consists of three edge convolution blocks. In each convolution, the message passing network an MLP with three hidden layers with 128 hidden channels. The message between nodes is constructed from the hidden node representations and one additional edge feature defined as the Minkowski product of the two particle's four-momenta. The aggregation of the messages is done with a simple summation. For the LLoCa- NNN, we use 64 channels each for scalar and vector representations, i.e. 64 scalars and 16 vectors. The GNN and the LLoCa-GNN have 2.5 10t parameters and 2.7 10 parameters, respectively.
Our Transformer and LLoCa-Transformer networks use the same inputs as the GNN and LLoCa- GNN described above. These features are encoded with a linear layer in a latent representation with 128 channels. Each transformer block performs a multi-head self-attention with eight heads followed by a fully-connected MLP with two layers and GELU nonlincarities. Similar to the LLoCa- GNN, the latent representation of the LLoCa-Transformer is divided into 50% scalar and 50% vector features. We stack eight transformer blocks, totaling 10 parameters for both networks. The LLoCa- Transformer has 2 x 10 additional parameters.
The other morentz-equivarri modify the number of hidden multivector channels 20, such that the number of parameters of all our transformer models is roughly 10,, matching the CLoCa-Transformrr. The inputs of the MIP-I baseline are all the possible Lorentz-invariant features. The neural networks consists of five hidden layers with 128 hidden channels each, summing up to 5.5 x 10 learnable parameters.
baseline, L-GATr, is taken from its official repository We only
Training. All models are trained with batch size of 1024 and using the Adam optimizer with 3 = [0.99.0. 0.999] to optimize the network weights for 2 10 iterations. We use Py PoTccsss ReduceLROnPlatean learning rate scheduler to decrease the learning rate by a factor 0.3 if no im- provements in the validation loss are observed in the last 20 validations. We validate our models every 100 iterations; if 10 iterations are less than 50 epochs then we validate every 50 epochs. We use the same settings for the L-GATr baseline after finding improved accuracy over the original I.- GATr training parameters. The models are trained on single A100 GPU.
Data augmentation. In our framework, to train with data augmentation we have to sample global reference frames that are elements of the respected symmetry group. We follow the polar decom- position discussed in App. C to define global frames. We sample uniformly distributed rotations represented as quaternions. While it is fcasible to uniformly sample rotation matrices on the 3-sphere, the non-compactness of the Lorentz group makes it impossible to uniformly sample boost matrices. We therefore sample each component of the boost velocity from a Gaussian distribution A (0.00.0.1)
'nttps://github wtw.:tttbb..co///eeeebbrrrgg hww.eedeewerreeeeeeeeeeeee hww.h/hpmlllaeeee /1gat
18

---

truncated at three standard deviations. The mean and standard deviation of the Gaussian is estimated based on the training dataset. Finally, we define the global reference frames as the combination of a boost to the center-of- center-of-mass mass frame of the two incoming particles and the random Lorentz transforma- tion described above.
Timing and FLOPs. We evaluate the total training time and the FLOPs per forward pass for the amplitude regression networks. Training times are measured within our code environment on an H100 GPU, excluding overhead from validation and testing. FLOPs per forward pass are computed for Z + 4g events using PyTorch's torch. utils. fwoollloooooo counter. wwwnter...ooooomme......... FlopCounterMode. The results for both training time and FLOPPs are summarized in 3.
D.3 Jet tagging
Dataset. We use the JetClass tagging dataset of Qu et al. [38]. The data samples are organized as point clouds, simulated in the CMS experiment environment at detector level. For details on the simulation process, see Qu et al. [38]. The number of particles per jet varies between and 128, typically averaging 30-50 depending on the jet type. Each particle is described by its four- momentumn,, six features encoding particle type, and four additional variables capturing trajectory displacement. The dataset comprises 10 equally represented classes and is split into 100 million events for training. 20 million for testing, and million for validation. Accuracy and AUC are used as global evaluation metrics, and class specifce background rejection rates at fixed signal efficiency are reported in Tab. 5.
Models. Our implementations of ALoCa-Particless official versions of ParticleNet and ParT." We apply minimal modifications to the original codebases. In LLoCa-ParticleNet, we transform the sender's local features from its own local frame that of the receiver, and perform the k-nearest-neighhor
and LLoCa-ParT are based on the latest
search using four-momenta expressed in each node's s local frame. In L.LoCaParT,, we similarly evaluate edge features used to compute the attention bias based on four-momenta in local frames. Additionally, we implement tensorial message passing in the particle attention blocks, as described in Sec. 4.3, while retaining standard scalar message passing for class attention. Unlike the official ParT implementation, which uses automatic mixed precision, our LLoCa-Part ParT implementation employs single precision throughout to avoid potential equivariance violations caused by numerical precision limitations.
Our baseline transformer is simplified variant of ParT, maintaining a similar overall network architecture. Whereas ParT aggregates particle information using class attention, our baseline instead employs mean aggregation. Furthermore, the learnable attention bias present ParT is omitted in our design, leading to significant gain in evaluation time. Consistent with ParT,, we use 128 hidden channels and expand the number of hidden units in each attention block's MLP by a factor of four. While ParT includes eight particle attention blocks followed by two class attention blocks, our baseline transformer consists of ten standard attention blocks, followed by mean aggregation.
The inputs to all our models are the particle four-momenta P; combined with the 17 particle- wise features described Tab. Following Eq. (13), the Frames-Net & receives pairwise inner products based on the four moun-mamenta| pt combined with the 17 particle-wise features in Tab. 3 as scalars sg. Note that the kinematics features are not invariant under Lorentz transformations, we discuss this aspect below in more detail. For the backbone network, the kinematic features An, Ań, log pr. log E, log / log E,j,, AR R are all expressed in the respective local refer- ence frames. The 6 particle identification features and the 4 trajectory displacement features are treated like scalars, i.c. they are not transformed when moving to the local reference frames. The backbone network input is then constructed from these combined 17 features.
For all models, we break Lorentz equivariance by including additional symmetry-breatee inputs in two ways. First, the 7 kinematic features listed in Tab. 3 are only invariant under the unbroken SO(2) subgroup of rotations around the beam axis, but included in the list of Lorentz scalars s, that serves as input to the Frames-Net. Second, we append additional three "reference particles" to the particle set that serves as input for the local reference frame prediction described Sec. 4.5. These reference particles are the time direction (1, O. O. 0) as well as the beam direction and its counterpart (1, 0, 0, +1). Together, they define the time and beam directions as two reference directions which will
Available at htttp:://zend:: wwtp:ooorreeeeeeeeeeeeeeoere
under a CC-BY 4.0 license.
Available http://tttt...ccooo//////////ee/ wwtp:/eeeeeeeeeeeeeeeeeeeeeeeeee
19

---

Category
Variable
Definition
Anj
difference in pseudorapidity between the particle and the jet axis difference in azimuthal angle between the particle and the axis logarithm of the particle's transverse momentum pr logarithm of the particle's energy
AΦ
log pT
Kinematics
log
log Pr
logarithm of the particle's pr relative to the jet pr logarithm of the particle's energy relative to the jet energy
log E(jt
A AR
angular separation between the particle and the jet axis (A()))) + (Ad)2)
charge
electric charge the particle if the particle is electron (/pid =11)
Electron
Muon
the particle a muon (pid|=13)
Partiele identification
Photon
if the particle a photon (pid=22)
CH
the particle a charged hadron (/pid/=211 or 321 or 2212)
NH
the particle neutral hadron (pid|=133 1330=1300m or 2112 0)
tanh
hyperbolic tangent of the transverse impact parameter value hyperbolic tangent of the longitudinal impact parameter value error of the measured transverse impact parameter error of the measured longitudinal impact parameter
tanh d
Trajectory displacement
Odo
Od.
Table Input features for the JetClass dataset [38]. Kinetic features in the global frame are used symmetry breaking during prediction of the local frames.
stay fixed for all inputs, effectively breaking the Korentz-equivariane, down to the transformations that keep these two directions unchanged. We note that two of the three reference vectors would already be sufficient, and also already one of the two ways of Lorentz symmetry breaking would be sufficient principle. The effect of symmetry-breaking inputs is discussed in App.
Training. For both LLoCa-ParT and CoCa-PaaticleNet,, we adopt the same training hyperparame- ters as the official implementations, without any additional tuning. Specifically, models are trained for 1,000,000 iterations with a batch size of 512, using the Ranger optimizer as implemented in the official ParT and ParticleNet repositories. constant learning rate is applied for the first 700,000 iterations, followed by exponential decay. We use a learning rate of 0.001 for all networks.
Our Transformer and LLoCa-Transformer models are trained with the same number of iterations, batch size, and initial learning rate as L.LoCaPPaTT, but use the Adam optimizer in conjunction with cosine annealing learning rate schedule.
Timing and FLOPs. To compare the training costs of the different tagging networks, we eval- uate both training time and FLOPs per forward pass. Training times measured within our code environment on an H100 GPU, excluding overhead from data loading, validation, and test- ing. FLOOPs per forward pass computed for events containing 50 particles using Py PyTorch's torch. utils. flop__ooooooooooo cecppprccttr...ee www.tter...oooouttteee..ee...ee. Floppoouttteeroddee... The results for both training time and FLOPs summarized in Tab.
E
Additional results
In this section, we present additional results for the QFT amplitude regression and jet tagging experiments.
E.1 QFT amplitude regression
More Lorentz-equivariant architectures. Following the guiding principle our contribution, we present other architectures that are turned into Lorentz-eeuuvvarian via local canonicalization. We study the same graph network without including the Lorentz-invariant edge features (pr, P,) arguments for the main network. The other baseline consider is simple MLP with four-
20

---

10
MLPP
GNN w/o
100
DA-GNN w/o
DA-MLP
LLoCa-GNN w/o inv.
MSE on log amplitudes
LLoCa-MLP
MSE on log amplitudes
100
10-
10
10
10
10
10-2.
Z+
Z 4g
10a
10
10a
100
10?
10
10a
10a
10a
10
107
10a
Number of training samples
Number of training samples
Figure 5: Comparison between LLoCa models and their non-equivariant counterparts. We consider simple MLP (left) and graph network that does not additional Lorentz invariant edge features (right). The global frames for the data augmentation (DA) results are sampled as described in App.
momenta as inputs. We do not include non-trivial frame-to- framee transformations in Eq. (10), effectively allowing only scalar messages. The architecture still Lorentz-equivariant due to Sec. 4.2. In Fig. 5 repeat the same scaling study as presented in Sec. 5. Although these two models are not competitive with the models presented in Sec. 5, we observe significant improvements from introducing Lorentz-equivaaiance.
Frames-Net regularization. Besides the message representations discussed in Tab. 2, the main design choice of LLoCa is the architecture of the Frames-Net yp described in Sec. 4.5. We consider on the Z + 4g dataset for different training dataset
several choices and train sizes, Fig. First, find that the default Frames-Net L"Learnable LLoCa") tends to overfit larger training dataset sizes than the main network. We tackle this by using a dropout rate of 0.2 when less than 10a training events are available ("Learnable LLoCa + Dropout"). We do not use dropout for larger training dataset sizes because we find that it affects the network performance. A similar effect can be achieved by decreasing the size of the Frames-Net ("Learnable small LLoCa"), where we use 16 instead of 128 hidden channels. However, we find that decreasing the network size slightly degrades performance in the large-data regime. We find that Frames-Nee with parameters fixed after initialization ("Fixed LLoCa") still outperforms the eoneequivaiant counterpart. When little training data available, a fixed Frames-Net combined with dropout rate of 0.2 ("Fixed LLoCa + Dropout") even achieves the best performance.
Constructing local reference frames. Our default approach for constructing local reference frames based polar decomposition is equivalent direct Gram-Schmidt algorithm in Minkowski space and removing the boost yields the special case of local reference frames for (3)eqquuiraii architectures, see Sec. 4.5 and App. C. We put these observations to the test and train LLoCa- Transformers on different training dataset sizes of the Z + 4g dataset, see Fig. First, we find that a direct Gram-Schmidt algorithm Minkowski space (GS*) shows very similar performance to our default polar decomposition approach (PD). The special
of (0(33)equivarianee obtained by only including the rotation part (GS$) yields significantly worse performance due to the reduced symmetry group. All approaches show a very similar scaling behaviour with the training dataset size.
Attention inner product. For the scaled dro-product attention discussed in Eq. (11), the Minkowski product to project keys onto queries. We find that this design choice significantly outperforms the naive choice of Euclidean attention, Tab. 4.
E.2 Jet tagging
21

---

Non-equivariant
Transformer (rans-rrrnf
100
100
F-Net small
F-Net fixed
MSE on log amplitudes
MSE on log amplitudes
10
F-Net dropout
10
Nett fixed + dropout
Neter default
10-2 2
10-2
10-3
10
10
10
10 5
Transformer 4g
10-5
Z 4g
10a
10a
10a
10
10?
10a
10a
10a
10a
10
10
10a
Number of training samples
Number of training samples
Figure 6: Design choices when constructing local reference frames. Left: Effect of the network architecture used for the Frames-Net yp We "Learnable LLoCa" in all other experiments, and "Learnable LLoCa + Dropout" when less than 10t training events are available. Right: Comparison different orthonormalization schemes. We use "PD" in all other experiments.
Hf + b5
Hf cc
H!
H - 4g Rejo0%
H - lurq'
- bigg'
f blv
W - qq'
Z - Rejoa5%
Rejo0%
Rejno%
Rejo0%
Rejgo%
Rejon56
Rejog.550
Rej
PEN 126)
PartickeNer [37]
2475
10526
347
285
ParT [38]
4149
32787
543
4012
MIParT-L 148]
10753
4202
31250
542
4012
L-GAT+ [9]
12987
4819
128
2311
6116
47619
20408
588
432
Transformer (ours)
10753
3333
116
1369
4630
24390
17857
415
334
LLoCa Trandd (curs)
11628
9651
125
2037
5618
39216
17241
548
410
PC (curs)
41667
LL.oCl-PurriceNNe (ouns)
2833
403
Table 5: Background rejection rates 1/eB for the JetClass dataset [38]. For each class, Reivs represents the inverse signal fraction at fixed background rejection rate N%. Lorentz-equivarian methods are marked with asterisk See Tab. 1 for the global accuracy and AUC.
More evaluation metrics. In addition the global evalu- ation metrics reported in Tab. 1, we report class-wise back- ground rejection rates in Tab. 5.
Table Benefit of Minkowski at- tention compared to Euclidean atten- tion. We show results for the LLoCa- Transformer on the full Z + Ag dataset.
Effect of Lorentz symmetry breaking. We follow the ap- proach of Ref. [9, 42] of using fully Lorentz-equivarian mod- els and breaking their symmetry instead of considering models that are only equivariant under the SO(2) symmetry of rota- tions around the beam direction. Note that the baseline models ParticleNet, ParT, MIParT and our Transformer are all SO(2)- invariant, because their inputs are invariant under SO(2) trans- formations.
Attention
MSE (x 10-5)
Euclidean Minkowski
2.5 0.4
1.0 + 0.1
As described in App. D,
two independent methods to break the Lorentz symmetry down to the (2) subgroup of rotations around the beam direction. They are (a) including features in the list of Lorentz-invarian inputs s; in Eq. (13) that are only invariant under the unbroken subgroup (non-invariant scalars, or NIS), and (b) including reference vectors (RV) pointing in the directions orthogonal the unbroken subgroup additional particles. The additional NIS and RV inputs are described in more detail in App. D. In Tab. 6 we study the effect of this design choice.
To study the impact of Lorentz symmetry breaking, we directly compare the cases of "No symmetry breaking" with only "RV", only "NIS", and our default of including both "RV & NIS The Lorentz- equivariant model that does not include any source of symmetry breaking is significantly worse and
22

---

Symmetry breaking
All classes
Hf + 56
H - cc
H +
H - 4g
H + lugd'
- baq"
t bly
W + ag'
Z -
Accuncc
AUC
Reigots
Re1g0%
Reigo7
Re1go%
Reigg%
Reigom
Rejgo 55%
Nem-oquivariant
17857
S003)-qui.
37037
symmetry breaking
RV
0.861
0.9877
11494
4444
1923
5223
34453
17391
300
NIS
019881
4630
40817
17544
NIS (default)
0.864
39216
Table Impact symmetry breaking for jet tagging, rejection L/ for different breakking approaches. We compare the effect of symmetry breaking with reference vectors (RV) and non-in non-invariant scalars (NIS) with the case of no symmetry breaking as well as non- equivariant and (O(3) -qquivaaaaari model (which also uses symmetry breaking with RV &V&))
accuracy, AUC, and background
Table 7:
LLoCa significantly lifts the performance of non-equivariant backbone architectures even when using the same computational resourees. See Tab. for more basclines.
Network
Training time
Accuracy
AUC
Transformer
15h
0,855
0.9867
Transforrr (0.5x iterations)
15
0.861
0.9877
Transformer (2x iterations)
31 h
0.855
0.9868
L-GATr (0.2x iterations)
33 h
0.855
0.9869
ParT
33
0,861
0.9877
LLoCa-Tramee Transformer
31 0.864
0.9882
only marginally better than the non-equivariant approach, because it constrains the model to assign an equal tagging score to input jets with different physical meaning. The models trained with symmetry breaking through the "Rv", "NIS" and "RV & NIS" approaches all have a sufficient amount of symmetry breaking included, but still yield slightly different results, We also compare to a SO(3)- equivariant model using the same reference vectors and non-invariant scalars to break the symmetry down the unbroken subgroup rotations around the beam direction.
Efficiency of LLoCa taggers at equal computational cost. As demonstrated in Tab, 1, LLoCa taggers consistently improve the performance of backbone networks such as vanilla transformers, at the cost of increased computational cost. In Tab. 7, we present additional results for networks trained with equal computational cost. The main baselines are ParT [38], widely used by experimental collaborations, and the recently proposed L-GATr [42], which improves upon ParT but requires 4x longer training (166h). Our most compute-efficient LLoCa tagger the LLooC--ransformer It almost matches L-GATr's accuracy yet trains in only 31h (vs 33h for ParT), delivering extra performance at lower cost one of the key results of this paper.
We also matched GPU hours by training the plain transformer for 2x iterations and the LLoCa- Transformer for 0.5x iterations. Again, we find that the -aoCC--rrasforme consistently achieves improvements at fixed training time (see Tab. 7).
23

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Both theoretical and practical claims listed in abstract and introduction are adequately addressed in the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims
made in the paper.
The abstract and/or introduction should clearly state the claims made, including the
contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss LLoCa's shortcomings in the main paper in Sections 4.5 and 5.3.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-speciffiation, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofss
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: Proofs for all theoretical results can be found either in the main paper or the appendix, including the full set of assumptions.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
24

---

All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclu- sions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: In App. D,, we list the details of our used architectures and the training setup. Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the partieular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. 5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Our code is open-public, the github link is in the introduction. The datasets used in the classification and generation experiment are already public, the regression dataset is linked in Appendix D.
Guidelines:
The answer NA means that paper does not include experiments requiring code. Please see the NeurIPS code and data submission guidelines (https://nips cc/
mouuides/Codeeumiisse
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not
25

---

reproducible, they including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: /nips. cc/ wctppblcc//cccccccccccccccccclcl guwiidgguss//eee Sutps//coeeeeeieieeeeeee ssssnnolicyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why. At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyperpa- rameters, how they were chosen, type of optimizer, etc.) necessary understand the results? Answer: [Yes]
Justification: the experiment section of the main text and in App. D, we list the details of our architectures and the training setup.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We provide an estimate of epistemic uncertainty through the variance between multiple independent results in the regression and generation experiments. We do not report uncertainties for the classification experiment due to limited computational resources.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula,
call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
26

---

We report the compute resources used in our experiments together with execution times in the main paper as well as App. D. Our experiments are of sufficiently small scale that they should be reproducible in typical academic environment. Guidelines:
Justification:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental rums well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: httees:///euuieeeeeeeeeeeeeeeeee /tt.rrrurpp..pp httpppplll/cciictuiiaieieeaeeeee [Yes]
Answer:
Justification: We fully conform the Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction). 10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Since this work is specific to high-energy physics, we expect societal impact through the effect that our architecture has on research in that field. We do not anticipate any particular risk of negative socictal impact. Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal
impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA]
Justification:
We are not aware of any such risks.
27

---

Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We cite the existing assets we use in the paper. Licenses are mentioned in Appendix D.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi wwwode.e........... com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information not available online, the authors are encouraged to reach out to the asset's ' creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Code and new datasets are both publicly available and properly documented, the links are in the manuscript.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Including this information in the supplemental material is fine, but if the main contribu-
28

---

tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
A According to the NeurIPS Code of Ethics, workers involved in data collection, curation,
or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/teview based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing research with human subjects. Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution. For initial submissions, applicable), such the institution conducting the review.
not include any information that would break anonymity (if
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non- standard component of core methods in this research? Note that the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: The paper does not make use of LLMs beyond writing, editing, or formatting purposes.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components. Please refer to LLM policy (https: : http//eerrppp.eeeee wwt.c//eeeeeeccccccc/ccccccccaae /20.///L///////121// /LLM) for what should or should not be described.
29


Reviews:
[{"review_id": "s6U9IA3ooZ", "reviewer": "Reviewer_sk9Q", "summary": "This paper introduces LLoCa, a framework that makes any neural network design Lorentz-invariant for use in high-energy physics. The main idea is to use equivariantly predicted local reference frames to turn particle properties into Lorentz-invariant representations. \n\nThese representations can then be processed by traditional architectures like transformers or graph networks before being transformed back to produce equivariant outputs. \n\nThe authors incorporate the non-compact Lorentz group into geometric message passing and develop an attention mechanism based on the Minkowski product. \n\nLLoCa is four times faster and uses 5 to 100 times fewer FLOPs than other Lorentz-equivariant methods, and it can even make vanilla transformers competitive without modifying their architecture for a specific domain. \n\nThe approach also enables fair comparison between precise equivariance and data augmentation, showing that exact Lorentz equivariance performs better with large amounts of training data.", "questions": "You mention that large boosts B may lead to numerical instabilities during orthonormalization, but that predicted vectors vi,k are typically not highly boosted. Can you provide a more rigorous analysis of when this stability breaks down? \n\nYour baselines primarily compare against established methods. How does LLoCa perform against more recent equivariant architectures?\n\nProvide more detailed algorithmic descriptions and pseudocode for the local reference frame construction, particularly the numerical procedures for ensuring stability.", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "**Strengths:** The core idea of using equivariantly predicted local reference frames to achieve Lorentz equivariance is both theoretically solid and practically valuable, as it separates equivariance from architectural limitations. The authors provide strong theoretical foundations with thorough proofs, demonstrate the framework's versatility by successfully applying it to various architectures (transformers, graph networks, domain-specific models), and present impressive empirical results that consistently outperform existing methods. The approach is also practically beneficial for large-scale applications because it substantially boosts computational efficiency—training is four times faster, and FLOPs are reduced by 5 to 100 times compared to specialized Lorentz-equivariant designs. \n\nFurthermore, incorporating geometric message passing for the non-compact Lorentz group and developing Minkowski product-based attention are technically sophisticated contributions that enhance the field's theoretical understanding.\n\nI have no more concerns.", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the thorough and constructive review. We are happy to hear that you appreciate the theoretical soundness and practical value of our approach. Thanks as well for the questions and criticisms, which we address in the following.\n\n### 1 Large boosts and numerical stability\n\n> You mention that large boosts B may lead to numerical instabilities during orthonormalization, but that predicted vectors $v_{i,k}$ are typically not highly boosted. Can you provide a more rigorous analysis of when this stability breaks down?\n\nLocal frames with large boosts $B$ are problematic, because they multiply the latent features in each message passing step, possibly leading to exploding gradients. Such large boosts are caused by vectors $v_{i,0}$ constructed in Eq. (13) that have small but positive norm, because they yield large values for $\\gamma$ in Eq. (2). These small-norm vectors $v_{i,0}$ can be caused by the typically small-norm particle 4-momenta $p_i$ in Eq. (13) either if all 4-momenta $p_i$ point in a similar direction, or if the coefficient $\\mathrm{softmax}(\\varphi_k (\\dots))$ is small for all except one particle.\n\nEven bigger problems occur if the vectors $v_{i,0}$ have zero or negative norm, because the boost matrix $B$ in Eq. (2) is ill-defined due to $\\vec \\beta^2 \\ge 1$. In practice, such pathological vectors $v_{i,0}$ do not occur with our implementation.\n\nWe have identified several techniques to avoid large boosts already at initialization, where such instabilities are most likely to occur. First, we include a softmax operation in Eq. (13) to prevent any negative-norm vectors in the process. Second, we use regulator masses to enforce a lower limit on the norms of the input particles $p_i$. Third, we multiply by $p_i+p_j$ instead of simply $p_i$ in Eq. (13), because $p_i+p_j$ typically is not strongly boosted anymore.\n\n### 2 Comparison against recent Lorentz-equivariant architectures\n\n> Your baselines primarily compare against established methods. How does LLoCa perform against more recent equivariant architectures?\n\nThroughout the paper we benchmark LLoCa against L-GATr [1], the most recent Lorentz-equivariant architecture. For jet tagging, L-GATr is the only Lorentz-equivariant architecture with public results on JetClass besides our LLoCa networks. We conducted an additional experiment by retraining LorentzNet [2] based on its official implementation, and using the same training hyperparameters as for LLoCa‑ParticleNet. LorentzNet matches LLoCa‑ParticleNet in accuracy but is ~30% slower. The transformers (L‑GATr and LLoCa‑Transformer) clearly outperform the graph networks, underscoring the scalability advantage of transformers.\n\nNetwork | Time | Accuracy | AUC\n--- | --- | --- | ---\nLLoCa-Transformer | 33h | 0.864 | 0.9882\nL-GATr | 180h | 0.866 | 0.9885\nLLoCa-ParticleNet | 41h | 0.848 | 0.9857\nLorentzNet (new) | 57h | 0.847 | 0.9856\n\n### 3 Numerical details of local frame construction\n\n> Provide more detailed algorithmic descriptions and pseudocode for the local reference frame construction, particularly the numerical procedures for ensuring stability.\n\nThanks for the suggestion. We will add a detailed algorithmic description in an appendix of the revised version of the paper. We will also publish our code upon publication, allowing the community to profit from our implementation of the local frame construction.\n\nFirst, most particles at the LHC can be considered massless as the typical energy scale is much larger than the mass of single particles. When operating on four-momenta using single precision, the particle mass value is often modified due to numerical underflows, leading to particles with zero mass in some cases. To avoid downstream instabilities, we enforce a minimal particle mass $m_\\epsilon$ by increasing the energy of all input particles $p_i$ as $E' = \\sqrt{m_\\epsilon^2 + E^2}$, hence $m'^2 = m_\\epsilon^2 + m^2$. In our experiments, we use $m_\\epsilon=10^{-5}$ and $m_\\epsilon=5\\cdot 10^{-3}$ for the amplitude regression and tagging experiments, respectively.\n\nSecond, we carefully design Eq. (13) to avoid vectors $v_{i,0}$ with small or negative norm which would cause instabilities due to large boosts. We discuss our design choices in the answer to the first question.\n\nThird, the $\\mathrm{SO(3)}$ Gram-Schmidt orthonormalization used in Alg.1 requires two linearly independent vectors. We ensure that the two vectors $\\vec w_1$, $\\vec w_2$ are not collinear by calculating the distance between the two vectors. Concretely, if $(\\vec v_1-\\vec v_2)^2 < \\epsilon_\\text{collinear}^2$, we keep $\\vec v_1$ and replace $\\vec v_2$ by $\\vec v_2 + \\vec \\epsilon$ with a random direction $\\vec \\epsilon$. In our experiments, we set $\\epsilon_\\text{collinear}=10^{-4}$ and we observe that, besides a handful of exceptions at initialization, the predicted vectors are never regularized. This indicates that our orthonormalization procedure is well under control.\n\nWe hope we were able to address your questions and look forward to discussing further. \n\n**References**\n\n- [1] J. Spinner et al, \"Lorentz-Equivariant Geometric Algebra Transformers for High-Energy Physics\", NeurIPS 2024\n- [2] S. Gong et al, \"An efficient Lorentz equivariant graph neural network for jet tagging\", JHEP 2022\n- [3] A. Bogatskiy et al, \"Explainable equivariant neural networks for particle physics: PELICAN\", JHEP 2023\n- [4] D. Ruhe et al, \"Clifford Group Equivariant Neural Networks\", NeurIPS 2023"}, {"review_id": "0npj1hlBSq", "reviewer": "Reviewer_79Y8", "summary": "This paper introduces LLoCa, which can enable Lorentz-equivariant adaptation of arbitrary backbone networks (e.g., Transformers). By predicting local reference frames to transform input data into Lorentz-invariant features, the framework allows backbone networks to process these features and produce equivariant outputs. It addresses the computational inefficiency and architectural inflexibility of existing methods. It achieves state-of-the-art (SOTA) performance in high-energy physics tasks such as Jet tagging and QFT amplitude regression, with significantly improved computational efficiency.", "questions": "(1)\tFor two-particle systems, could virtual particles or preprocessing be introduced?\n(2)\tCan the framework be modified to support computations for other non-compact groups (e.g., the Poincaré group)?", "limitations": "Yes (Section 4.5 explicitly states the particle-count constraint)", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Quality：\n1) Rigorous methodology: Polar decomposition for Li (Algorithm 1) achieves strict equivariance, backbone-agnostic compatibility, resolving computational challenges of non-compact groups. \n2) Comprehensive experimental design for high-energy physics, including 10-class Jet tagging and QFT amplitude regression, validated on 125M particle jet data. \n\nClarity：\nTheoretical derivations are precise: Eqs.1–12 formally define Lorentz group representations, canonicalization, and message passing. Ablation studies reveal the necessity of mixed tensor representations.\n\nSignificance:\nThe framework solves a critical bottleneck in HEP machine learning: it enables Lorentz-equivariant adaptation of general-purpose backbone networks, providing a tool for analyzing the massive datasets from the Large Hadron Collider (LHC).\n\nOriginality:\nNovel contributions: Local canonicalization unifies data augmentation and exact equivariance and introduces an innovative Minkowski inner-product attention mechanism (Eq. 12).\n\nWeakness：\n(1)\tParticle-count constraint:  It requires ≥3 particles systems for stable Li prediction (Section 4.5), making it unsuitable for two-particle systems.\n(2)\tGeneralizability: Experiments are limited to HEP tasks; potential applications in other domains remain untested.", "was_revised": true, "final_justification": "My concerns about this work are largely alleviated. I will keep my score 5 unchanged.", "rebuttal": "Thank you for the detailed review of our submission. We are glad that you value the clarity and the rigor of our work. Thanks as well for the questions and criticisms, which we address in the following.\n\n### 1 Local frame prediction for $<3$ particles\n\n> It requires ≥3 particles systems for stable $L_i$ prediction (Section 4.5), making it unsuitable for two-particle systems. [...] For two-particle systems, could virtual particles or preprocessing be introduced?\n\nThe local frame construction described in Sec. 4.5 relies on the equivariant prediction of three linearly independent 4-vectors $v_{i,k}$. In the case of one or two particles, there are at most two independent directions, the 4-momenta $p_1$ and $p_2$. In this very special case, it is not possible to construct equivariant local frames based on the information contained in the input point cloud. Linear combinations of the 4-momenta, like virtual particles, are not linearly independent and can therefore not solve the problem. In our code, we handle this special case by sampling the missing directions randomly from a $\\mathrm{SO}(3)$-invariant distribution, at the cost of formally breaking Lorentz-equivariance to $\\mathrm{SO}(3)$-equivariance.\n\nIn situations with partial Lorentz symmetry breaking, the reference particles encoding the time and beam direction can lift the number of input vectors $p_i$ to 3 or higher. This happens in our jet tagging experiment, where 0.001% of the jets consist of only one or two particles.\n\nMachine learning tasks in high-energy physics where Lorentz symmetry is relevant typically feature large numbers of particles. We therefore think that the constraint to $\\ge 3$ particle systems is no problem in practice.\n\n### 2 Experiments beyond HEP tasks\n\n> Experiments are limited to HEP tasks; potential applications in other domains remain untested.\n\nThe experiments intentionally focus on high‑energy physics, where precise benchmarks and strong Lorentz‑equivariant baselines allow us to show that local canonicalization with tensorial messages scales to demanding real‑world tasks. We agree that cross‑domain validation would be valuable and have added a broader evaluation to our future‑work plan.\n\n### 3 Extension of our framework to other non-compact groups\n\n> Can the framework be modified to support computations for other non-compact groups (e.g., the Poincaré group)?\n\nIn addition to Lorentz-equivariance, invariance w.r.t.~translations is typically obtained by using only differences of four-vectors. In all our experiments, the input is provided in momentum space where translation equivariance/invariance is not preferable. However, the extension to the Poincaré group is possible by modifying Eq. (13) to \n\n$$v_{i,k} = \\sum_{j=1}^N\\mathrm{softmax}(\\varphi_k(s_i, s_j, \\| x_i - x_j \\|))(x_i - x_j) ,$$\n\nwhere we assume to work in position space with four-vectors $x_i$. One possible challenge may be to ensure that at least one of the predicted $v_{i,k}$ has a positive norm from which one can construct the boost $B$, cf. Alg. 1.\n\nFurther, it is possible to directly extend our framework (including the tensorial message passing) to the group of $\\mathrm O(1,d)$. We refer to the answer to reviewer VHeN for a broader discussion.\n\nWe hope we were able to address your questions and look forward to discussing further."}, {"review_id": "dRGtCEe46k", "reviewer": "Reviewer_VHeN", "summary": "The paper introduces **Lorentz Local Canonicalization (LLoCa)**, a general framework that converts any backbone, e.g., graph network, transformer, or MLP, into an *exactly* Lorentz-equivariant model without bespoke equivariant layers. The authors use a simple Lorentz-equivariant architecture to predict three four-vectors for each particle. The four-vectors are then used to construct the local reference frames following a deterministic orthogonalization algorithm. The authors extend recent tensorial message-passing ideas to the non-compact Lorentz group and derive a Minkowski dot-product attention that reuses efficient vanilla attention code. The approach incurs only a moderate computational overhead, with a 10–30% increase in FLOPs and a 30–110% increase in training time compared to non-equivariant baselines. Compared to other SOTA Lorentz-equivariant architectures, the models train up to 4x faster and use 5–100x fewer FLOPs.", "questions": "1. [2] prove that, for a certain domain, no *continuous* canonicalization map exists for the compact orthogonal group. Is the vierbein predicted by LLoCa a continuous function of the input four-vectors and is the overall architecture of LLoCa continuous w.r.t. the input?\n2. Can the canonicalization procedure be extended from $O(1,3)$ to the general family $O(1,d)$ as considered in [3]? It is recommended for a brief discussion over this generalization.", "limitations": "The authors have discussed the limitations in line 226 and 317. It is recommended for the authors to write them into one section and give possible future solutions for them.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strengths**:\n\n1. LLoCa generalizes the canonicalization strategy of [1] from compact groups to the non-compact Lorentz group and can be grafted onto any backbone (graph network, transformer, MLP) without bespoke equivariant layers, widening the design space beyond previous Lorentz-specific networks.\n2. Relative to the corresponding non-equivariant model, LLoCa adds only 10–30% FLOPs and 30–110% training time, yet remains 4x faster and 5–100x lighter than L-GATr, the prior best equivariant architecture.\n\n\n\n**Weaknesses**:\n\n1. The result accuracy is not the SOTA in the JetClass dataset, contradicting the claim in the abstract. Also, the performance lift over the chosen backbone is marginal, while training time nearly doubles. Results obtained under the equal training time of the backbone and the backbone with the proposed technique would clarify whether the trade-off is warranted.\n2. Canonicalization has been comprehensively explored in [1–4] as a special case of frame averaging [5]; especially, Lorentz canonicalization via generalized Gram–Schmidt for Minkowski metric is already discussed in [3], possibly contradicting the first contribution claim the authors make. The paper should state precisely in the main text how LLoCa diverges from, or improves upon, these methods to justify its novelty.\n\n[1] Kaba, Sékou-Oumar, et al. \"Equivariance with learned canonicalization functions.\" International Conference on Machine Learning. PMLR, 2023.\n\n[2] Dym, Nadav, Hannah Lawrence, and Jonathan W. Siegel. \"Equivariant frames and the impossibility of continuous canonicalization.\" arXiv preprint arXiv:2402.16077 (2024).\n\n[3] Lin, Yuchao, et al. \"Equivariance via minimal frame averaging for more symmetries and efficiency.\" arXiv preprint arXiv:2406.07598 (2024).\n\n[4] Ma, George, et al. \"A canonicalization perspective on invariant and equivariant learning.\" arXiv preprint arXiv:2405.18378 (2024).\n\n[5] Puny, Omri, et al. \"Frame averaging for invariant and equivariant network design.\" arXiv preprint arXiv:2110.03336 (2021).", "was_revised": true, "final_justification": "The rebuttal addressed all of my questions. The work is clear and the empirical result is strong, though its theoretical novelty is moderate.", "rebuttal": "Thank you for the thorough and constructive review. We are happy to hear that you appreciate the flexibility and efficiency of our approach. Thanks as well for the questions and criticisms, which we address one by one.\n\n### 1 SOTA on JetClass dataset\n\n> The result accuracy is not the SOTA on the JetClass dataset\n\nWe all agree that given the same training and test data, overall accuracy depends not only on the architecture but also on the many concomitant design choices / hyperparameters ranging from number of channels to choice of optimizer to training schedule. Our focus here was the architecture, and it is for this reason that LLoCa-ParT, Transformer, and LLoCa-Transformer models all use the default ParT settings [7]. This uniform configuration was selected to ensure a fair and consistent comparison between LLoCa architectures and their non-equivariant baselines.\n\nIn contrast, the L-GATr results reported in [6] and cited in our original Table 1 rely on carefully tuned hyperparameters. For an apples-to-apples comparison, we have retrained L-GATr with ParT settings and found the following:\n\nNetwork | Training setup | Accuracy | AUC\n--- | --- | --- | ---\nL-GATr | L-GATr [6] | 0.866 | 0.9885\nL-GATr | ParT [7] (new) | 0.864 | 0.9882\nLLoCa-Transformer | ParT [2] | 0.864 | 0.9882\n\nWe are confident that LLoCa-Transformer accuracy could be further increased with hyperparameter tuning, but this was not our primary aim here and also not feasible within the rebuttal period. We will amend the abstract to state that we achieve \"competitive and SOTA results\".\n\n### 2 Performance lift from LLoCa\n\n> Also, the performance lift over the chosen backbone is marginal, while training time nearly doubles. Results obtained under the equal training time of the backbone and the backbone with the proposed technique would clarify whether the trade-off is warranted.\n\nAlthough LLoCa improves accuracy/AUC only in the third decimal place, the gain is nonetheless meaningful. A more tangible metric is the class‑wise background‑rejection rate in Table 5: doubling this rate effectively doubles the usable data when the analysis is background‑limited. For instance, for the $H\\to c\\bar c$, $H\\to 4q$ and $t\\to bq\\bar q'$ classes LLoCa-Transformer increases the background rejection rates by 40% or more compared to the baseline Transformer. Neural jet taggers are used in almost every LHC analysis, turning already smaller performance gains into real-world money savings because the LHC has to run for less time to collect the same statistical power.\n\nLLoCa adds full Lorentz equivariance to lightweight backbones like a vanilla Transformer. The main baselines are ParT, widely used by experimental collaborations, and the recently proposed L‑GATr, which improves upon ParT but requires $5\\times$ longer training (180h). Our LLoCa‑Transformer almost matches L‑GATr’s accuracy yet trains in only 33h (vs. 38h for ParT), delivering extra performance at lower cost - one of the paper’s key results. For LLoCa‑ParT and LLoCa‑ParticleNet, however, we agree that the incremental gain may not justify the additional effort.\n\nWe also matched GPU hours by training the plain transformer for $2\\times$ iterations and the LLoCa-Transformer for $0.5\\times$ iterations. Again, we find that the LLoCa-Transformer consistently achieves improvements at fixed training time.\n\nNetwork | Training time | Accuracy | AUC\n--- | --- | --- | ---\nTransformer | 17h | 0.855 | 0.9867\nLLoCa-Transformer (0.5x iterations) | 17h | 0.861 | 0.9877\nTransformer (2x iterations) | 34h | 0.855 | 0.9868\nParT | 38h | 0.861 | 0.9877\nLLoCa-Transformer | 33h | 0.864 | 0.9882\n\n### 3 Relation of LLoCa to existing canonicalization approaches\n\n> Canonicalization has been comprehensively explored in [1–4] as a special case of frame averaging [5]; especially, Lorentz canonicalization via generalized Gram–Schmidt for Minkowski metric is already discussed in [3], possibly contradicting the first contribution claim the authors make. The paper should state precisely in the main text how LLoCa diverges from, or improves upon, these methods to justify its novelty.\n\nIn [1], the authors propose a learned canonicalization function that maps the entire input sample to a single canonical orientation. The key distinction of these 'global canonicalization' methods to our 'local canonicalization' is that LLoCa does not canonicalize the whole sample into one orientation, but instead predicts a unique local coordinate frame for each particle or node in the sample. The methods in [2–5] likewise only study global canonicalization functions. Our LLoCa method is a strict generalization of global canonicalization. In our experiments, we find that local canonicalization outperforms global canonicalization, see Table 2.\n\nFurthermore, the frame‑averaging techniques described in [2,4,5] achieve equivariance by rotating the global input into several canonical poses, passing each through the network, and averaging the resulting outputs. In principle, our method of using just a single frame per particle can be extended to frame-averaging methods that use multiple reference frames. We view this as a direction for future work.\n\nFinally, although [3] implements a canonicalization approach in Minkowski space to construct Lorentz‑equivariant networks, it remains a global technique. To the best of our knowledge, LLoCa is the first framework to realize local canonicalization for exact Lorentz equivariance. Importantly, our experiments demonstrate that local canonicalization without tensorial message passing does not work for the reasons given in Sec. 4.3 and [8], see Tab. 2. Therefore, our generalization of tensorial messages [8] to the Lorentz group provides a key ingredient in making local canonicalization a practically useful method for realizing Lorentz equivariance.\n\nWe will expand the related work section of the revised manuscript to include a more in-depth discussion about the difference between local and global canonicalization, and add the possibility of further work in the direction of frame averaging. For clarity, we will remove the brackets around the word \"(local)\" in the first contribution in the revised manuscript.\n\n\n### 4 Continuity of the LLoCa framework\n\n> Is the vierbein predicted by LLoCa a continuous function of the input four-vectors and is the overall architecture of LLoCa continuous w.r.t. the input?\n\nIn [2] it is proven that in $\\mathrm{SO}(d)$ and in the special case $\\mathrm{SO}(3)$ it is not possible to find a continuous canonicalization as a function of the input vectors. As $\\mathrm{SO}(3)$ is a subgroup of $\\mathrm{SO}(1,3)$, our local frame prediction can in general not be a continuous function of the input four-vectors. But in practice, we do not find this to be a problem. For a detailed answer about the numerics of the learned local frames, please take a look at our answer to Reviewer 1.\n\n[2] and [9] argue that with frame averaging of specific weighted frames, the discontinuity of the function can be mitigated. This method can in principle be used in our framework to create a smoother prediction of local frames and is a promising avenue for future research.\n\n\n### 5 Extension of LLoCa to $\\mathrm O(1,d)$\n\n> Can the canonicalization procedure be extended from $\\mathrm O(1,3)$ to the general family $\\mathrm O(1,d)$ as considered in [3]?\n\nYes, our method can be extended to the general family $\\mathrm O(1,d)$. Note that LLoCa does not cover the full Lorentz group including parity and time reversal, but only the fully-connected subgroup denoted by $\\mathrm{SO}^+(1,3)$. Extending LLoCa to $\\mathrm O(1,3)$ requires to also predict extra parity and time reversal transformations, see [8] for how this can be done for $\\mathrm O(3)$. Extending LLoCa to $\\mathrm{SO}^+(1,d)$ is straight-forward. Using Eq. (13), one may predict $d$ many $d+1$-dimensional vectors. Then, a trivial extension of Algorithm 1 with $d$-dimensional Gram-Schmidt algorithm for the spatial rotation can be used to predict orthonormal local frames for canonicalization w.r.t. $\\mathrm{SO}^+(1,d)$. Furthermore, the tensorial message passing based on the tensor representations defined by Eq. (3) can be extended directly to the case of $\\mathrm O(1,d)$.\n\n### 6 Limitations and future solutions\n\n> The authors have discussed the limitations in line 226 and 317. It is recommended for the authors to write them into one section and give possible future solutions for them.\n\nFollowing the suggestion of the reviewer, we will include a section in our revised manuscript that combines the limitations of our framework with suggestions for possible future solutions, similar to the answers provided here in the author response.\n\nWe hope we were able to address your questions and look forward to discussing further.\n\n**References**\n\n- [6] J. Spinner et al, \"Lorentz-Equivariant Geometric Algebra Transformers for High-Energy Physics\", NeurIPS 2024\n- [7] H. Qu et al, \"Particle Transformer for Jet Tagging\", ICML 2022\n- [8] P. Lippmann et al, \"Beyond Canonicalization: How Tensorial Messages Improve Equivariant Message Passing\", ICML 2025\n- [9] S. Pozdnyakov and M. Ceriotti, \"Smooth, exact rotational symmetrization for deep learning on point clouds\", NeurIPS 2023"}]

Author rebuttal:

Meta-review: This paper introduces Lorentz Local Canonicalization (LLoCa), a general framework that makes arbitrary backbone networks exactly Lorentz-equivariant by predicting local reference frames. The approach enables the construction of Lorentz-equivariant transformers and graph networks, extends geometric message passing to the Lorentz group, and naturally incorporates data augmentation as a frame choice. Empirical results show state-of-the-art accuracy on particle physics tasks with improved efficiency (faster inference and fewer FLOPs). Overall, the paper presents a novel, flexible, and effective method for advancing Lorentz-equivariant architectures in high-energy physics.

During the review and rebuttal period, all reviewers held a positive score for this paper, recognizing its novelty and technical contribution.
Therefore, I recommend the acceptance of this paper.
Decision: Accept (poster)