Explicit Discovery of Nonlinear Symmetries from Dynamic Data
Authors: Lexiang Hu
Yikang Li
Zhouchen Lin
Venue: ICML 2025 poster (2025)
Keywords: symmetry discovery
Lie theory
equivariant neural network
Primary area: deep_learning
TL;DR: A pipeline for explicitly discovering nonlinear symmetries from dynamic data

Abstract: Symmetry is widely applied in problems such as the design of equivariant networks and the discovery of governing equations, but in complex scenarios, it is not known in advance. Most previous symmetry discovery methods are limited to linear symmetries, and recent attempts to discover nonlinear symmetries fail to explicitly get the Lie algebra subspace. In this paper, we propose LieNLSD, which is, to our knowledge, the first method capable of determining the number of infinitesimal generators with nonlinear terms and their explicit expressions. We specify a function library for the infinitesimal group action and aim to solve for its coefficient matrix, proving that its prolongation formula for differential equations, which governs dynamic data, is also linear with respect to the coefficient matrix. By substituting the central differences of the data and the Jacobian matrix of the trained neural network into the infinitesimal criterion, we get a system of linear equations for the coefficient matrix, which can then be solved using SVD. On top quark tagging and a series of dynamic systems, LieNLSD shows qualitative advantages over existing methods and improves the long rollout accuracy of neural PDE solvers by over $20\\%$ while applying to guide data augmentation. Code and data are available at [https://github.com/hulx2002/LieNLSD](https://github.com/hulx2002/LieNLSD).

Explicit Discovery of Nonlinear Symmetries from Dynamic Data
Lexiang Hu? Yikang Li'
Zhouchen Lin 123
Abstract
2000; Bluman & Anco, 2008; Bluman, 2010). Equivariant networks embed symmetries into their structure, achieving better performance on specific tasks (Zaheer et al., 2017; Weiler et al., 201 80;;;; Kondor & Trivedi, 2018; Wang et al., 2021; Finzi et al., 2021; Satorras et al., 2021). Furthermore, symmetries can guide the discovery of governing equations from data (Yang et al., 2024b). However, these methods all require prior knowledge symmetries, which is challeng- ing when dealing with complex tasks and messy datasets. Therefore, automatically discovering symmetries from data has become an important topic.
Symmetry is widely applied in problems such as the design of equivariant networks and the dis- covery of governing equations, but in complex scenarios, not known in advance. Most pre- vious symmetry discovery methods are limited to linear symmetries, and recent attempts to dis- cover nonlinear symmetries fail to explicitly get the Lie algebra subspace. In this paper, we pro- pose LieNLSD, which is, to our knowledge, the first method capable of determining the number of infinitesimal generators with nonlinear terms and their explicit expressions. We specify function library for the infinitesimal group action and aim to solve for its coefficient matrix, proving that its prolongation formula for differential equations, which governs dynamic data, is also linear with respect to the coefficient matrix. By substituting the central differences of the data and the Jaco- bian matrix of the trained neural network into the infinitesimal criterion, we get system of linear equations for the coefficient matrix, which can then be solved using SVD. On top quark tagging and a series of dynamic systems, LieNLSD shows qualitative advantages over existing methods and improves the long rollout accuracy of neural PDE solvers by over 20% while applying guide data augmentation. Code and data are available http://ggthbbboomm/uuxx001111.s
Most existing symmetry discovery methods can successfully identify linear symmetries et al., 2021: Dehmamy et al., 2021; Moskalev al., 2022: Yang al., 2023). However, they are unable to discover nonlinear symmetries, which are common in dynamical systems. For example, in the case of the wave equation We = Use + Uyg. these methods can only find Lorentz symmetry and scaling symmetry, while missing translation symmetry and special conformal symmetry.
observation space (Zhou
Recent works have attempted explore nonlinear sym- metries. LaLiGAN (Yang al., 2024a) uses an encoder- decoder architecture to map the observation space to a la- tent space, making nonlinear symmetries in the observation space appear as linear symmetries in the latent space. How- ever, since the encoder and decoder are black boxes, they cannot explicitly discover the nonlincar symmetries in the observation space. Ko et al. (2024) and Shaw et (2024) propose methods to directly find non-affine symmetries from observed data. Nevertheless, Ko al. (2024) use MLP to model the infinitesimal generators, which suffers from the same interpretability issues as LaLiGAN, while Shaw et al. (2024) are limited to one-parameter group symmetries.
1. Introduction
From traditional mathematical physics to deep learning, symmetry plays an important role. In differential equations, symmetries can assist in integration by reducing the order (Olver, 1993; McLachlan, 1995; Ibragimov, 1999; Hydon,
In this paper, we propose the Lie algebra based NonLinear Symmetry Discovery method (LieNLSD), which can deter- mine the number of infinitesimal generators and explicitly provide their expressions containing nonlinear terms. Our setting assumes that the dataset originates from dynamic system governed by differential equations, with common forms including first-order dynamic systems u, = f(u') and second-order dynamical systems utr = f(u'), where u' represents and its spatial derivatives. This is a general set- ting, as static systems governed by algebraic equations are
'State Key Lab of General AI, School of Intelligence Sci- ence and Technology. Peking University "Institute for Artificial Intelligence, Peking University 'Pazhou Laboratory (Huangpu), Guangzhou, Guangdong, China. Correspondence Zhouchen Lin <zlin Siiin@kuu.cdu.c... edu.cn".
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Explicit Discovery of Nonlinear Symmetries from Dynamic Data
also included (i.e., zero-order dynamic systems). We spec- ify function library @ for the infinitesimal group action, which may include nonlinear terms. Then the infinitesimal group action can be expressed in the form of W Our ultimate goal is to solve for the coefficient matrix W of o, thereby explicitly obtaining the expression for the infinitesi- mal group action.
element v E g can be associated with a Lie group element 9 E G through the exponential map exp + G, i.e., g = exp(v). Representing the space of the Lie algebra with a basis (V1, "y....,,r,,, in the neighborhood of the identity element, we have = exp (Ei e'v;), where == (e", .e..,,..,,.") ,,) E IK' are the coefficients. The group G acts on a vector space X = =R" via a group action G X + X. Correspondingly, the infinitesimal group action of E gg x E X is defined as:
We first prove that the prolonged infinitesimal group action is linear with respect to vee (W", and thus the infinitesimal criterion for the symmetry group of differential equations is also linear with respect to vec (.). We then estimate deriva- tives from the dataset using the central difference method and obtain the Jacobian matrix of differential equations via automatic differentiation from the trained neural net- work. Substituting these into the infinitesimal criterion, we derive system of linear equations for vee(W), whose solu- tion space is found based on Singular Value Decomposition (SVD). Finally, we apply the Linearized A Alternating Direc- tion Method with Adaptive Penalty (LADMAP) (Lin et al., 2011) to sparsify the infinitesimal generators for intuitive and interpretable results.
d
v(v))22 = de c=U
(eexp(ev) V.
(1)
We abbreviate (gg, I) and (v() as g r and v, respectively, when the context is clear. We refer to the infinitesimal group actions of the Lie algebra basis as infinitesimal generators.
This is a more general definition that encompasses the nonlinear case. Previous works focusing on linear sym- metries (Moskalev et al., 2022; Desai et al., 2022; Yang et al., 2023) commonly use the group representation px G + GL(n) to characterize group transformations. i.e. Vg E G, E X : x) -x (g)x. The correspond- ing Lie algebra representation døx g + gl(n) satisfies Vv g px (x(exp(v)) = exp(dex (v)). Then, the re- lationship between infinitesimal group actions and Lie algebra representations is:
In summary, our contributions are follows: (1) we pro- pose LieNLSD, novel pipeline for discovering nonlinear symmetries from dynamic data, which is the first method ca- pable of determining the number of nonlinear infinitesimal generators and explicitly solving for their expressions to (2) we prove that if the infinitesimal group action is linear with respect to the coefficient matrix W, then its prolongation formula is also linear with respect to
our knowledge;
WegrreX:
v(v)]2 = dvx V.
(2)
A concrete example of group actions and infinitesimal group actions is provided in Appendix C.I.
W; (3) we construct and solve a system of linear equations for the coefficient matrix vec(W') of the infinitesimal group action based on the infinitesimal criterion; (4) we apply LADMAP to the sparsification of infinitesimal generators to enhance the intuitiveness and interpretability of the results; (5) the experimental results on top quark tagging and series of dynamic systems confirm the qualitative advantages of LieNLSD over existing methods; (6) we apply LieNLSD to guide data augmentation for neural PDE solvers, improving their long rollout accuracy by over 20%.
Symmetries of differential equations. Before defining
the symmetries of differential equations, we first explain how groups act functions. Let ʃ X + U be a function, with its graph defined I'y {(z, f fxx) x E X} C X x U. Suppose a Lie group G acts on X x U. Then, the transform of y by g E is given by g g-r, g (z, u): (z, u) E ry We call a function f X + U the transform of f by g. denoted as f f=g g. f. if its graph satisfies r g y- Note that f may not always exist, but in this paper, we only consider cases where it does. A concrete example is provided in Appendix C.2.
{(ē, û)
2. Preliminary
Before discussing related works and methods, we first briefly introduce some preliminary knowledge of Lie groups and differential equations. For readers who are not familiar with the relevant theory, we strongly recommend referring to Appendix A or the textbook (Olver, 1993) for a detailed version, and Appendix C for concrete examples.
The solution of a differential equation is expressed in the form of a function. Intuitively, the symmetries of differen- tial equation describe how one solution can be transformed into another. Formally, let y be system of differential equations, with the independent and dependent variable spaces denoted by X and U, respectively. Suppose that a Lie group G acts on X x U. We call G the symmetry group of L if, for any solution f X - U of y, and Vgg E G. the transformed function g f X + U is another solution .f
Group actions and infinitesimal group actions. A Lie group G is a mathematical object that is both a smooth manifold and a group. The Lie algebra g serves as the tangent space at the identity of the Lie group. A Lie algebra
2

---

Explicit Discovery of Nonlinear Symmetries from Dynamic Data
Table 1. Comparison of LicNI.SD and other symmetry discovery methods.
Applicability
L-conv
LieGG
LieGAN
LaLiGAN
et.
Shaw et al.
LieNLSD (Ours)
Nonlinear?
Explicit?
VX
X
X
Determine Lie algebra subspace dimension?
X
X
V X
X
Discover Lie algebra subspace?
V
Prolongation. To further study the symmetries of differ- ential equations, we need to "prolong" the group action on the space of independent and dependent variables to the space of derivatives. Given a function f X - U, where X R and U = R it has P:Pk- q- (P+k-1) distinct 8* (7())
strong restrictions on the type of group symmetries-ther require prior knowledge of the parameterized form of group actions or are limited to subgroups of given group.
Subsequent works use Lie group and Lie algebra representa- tions to characterize group symmetries. L-conv (Dehmamy et al., 2021) proposes the Lie algebra convolutional net- work, which serves as a building block for constructing group-equivariant architectures and learns the Lie algebra basis. LieGG (Moskalev et al., 2022) constructs a polariza- tion matrix based on the trained network and training data, extracting the Lie algebra basis from it. SymmetryGAN (Desai et al., 2022) and LieGAN (Yang et al., 2023) use gen- erative adversarial training to align data distributions before and after transformations, where the generator generates group transformations from the Lie algebra basis. As shown in Equation (2), Lie group and Lie algebra representations can only describe linear group actions, which limits these methods to discovering linear symmetries.
order derivatives u = 0,fx(x)) (1.... 9). J = (Jı. (Ji...., Jk), and Ji denote the space of all k-th order derivatives as Uk = PR and the space of all derivatives up to order n as U(m) =
Dwiidw O.0t where
.P). We
U x U1 x ... Un R where p(n) (ztn) Then, the n-th prolongation of f. pr(") f: : X + U()), is defined as pr(") fxx) = u()), where u-= = (x).
Based on the above concepts, the n-th order system of dif- ferential equations J can be formalized as Fxx, u m()))= = 0, where F: Xx x (m)) + R². A smooth function f X - U is a solution of S if it satisfies F(z, fr fx)) = 0.
Below, we explain how to prolong the group action on XU to X x (())) Let be a Lie group acting on the space of independent and dependent variables X x U. Given a point (xo, u6 (") E X x Ul)), suppose that a smooth function f: X U satisfies wi") = (r) f(x)). Then, the n-th prolongation of g E G at the point (zo, uu " is defined as pr(")g g- (zo, un (n)) (zo, up " where (ão, ūo) = g (zo, uo) and af"" = pr (ão))
Recently, some works have explored the discovery of nonlin- ear group symmetries. LaLiGAN (Yang et al., 2024a) learns a mapping from the observation space to a latent space and extends the LieGAN approach to discover linear group symmetries in the latent space. Ko et al. (2024) model the infinitesimal generators using an MLP and optimize the va- lidity score of the data transformed through ODE integration. Shaw et al. (2024) propose an efficient method for detecting non-affine group symmetries. However, LaLiGAN and Ko et al. (2024) fail to explicitly provide the infinitesimal group actions on the observation space. Shaw et al. (2024) are limited to discovering a single Lie algebra element rather than the subspace it belongs to. Furthermore, these meth- ods cannot accurately determine the number of infinitesimal generators, i.e., the dimension of the Lie algebra subspace.
Note that by the chain rule, the definition of pr depends only on (zo, uo ((")) and is independent of the choice of ʃ. Similarly, the n-th prolongation of v E at the point (z, u()) E X x (m) is defined as:
pr(")
d
{ (prl") (exp(ee))) (z,, u u(n))).V...
(2,u(n))
de c=
(3)
We compare our LieNLSD with other symmetry discovery methods in Table 1. To our knowledge, LicNLSD is the first method capable of accurately determining the number of nonlinear infinitesimal generators and explicitly solving for their expressions.
Concrete examples of prolongation are provided in Ap- pendix C.3.
3. Related Work
Symmetry discovery. Some previous works have at- tempted to discover symmetries from data. Early methods parameterize group symmetries as part of network training (Benton et al., 2020; Zhou et al., 2021; Romero & Lohit, 2022; van der Ouderaa et al., 2024). However, they have
Note that from implementation perspective, the types of PDE symmetries discovered by LieNLSD (ours) and LaLiGAN (Yang et al., 2024a) differ. Taking X = RR,--= R as an example (e.g. u(æ, y) represents a planar image), the symmetries found by LieNLSD act pointwise on X x
3

---

Explicit Discovery of Nonlinear Symmetries from Dynamic Data
infinlsiinal v ())
Thoren
prolonged infisitaiinal action prlti- (J)c
speetyy freetin Strery
Jole, "eleaa"" -
Ourpt
symmeny group G difermntial equations F66, (6) -0
42-9
Infisiasinal oritrion pr"N))/l,,
number of intintttirall generators
(vne( =0
-ve
Gool
epplct ecpressions of ganerators (Vi)fa)
2
((2 1 1
dffeworr?
prolonged pr ((26) a (3))
Inpur
Figure I. Pipeline of LieNLSD
T= symmetries are commonly referred to in the literature as "Lie point symmetries". On the other hand, the symmetries found by LaLiGAN are defined over the entire discretized field. Specifically, if u is a field on a 100 x 100 grid, then LaLiSAN's symmetries act on x (see the reaction-diffusion dataset the original paper of LaLiGAN for details). For PDEs, the setting of Lie point symmetries (which act pointwise on both coordinates and the field) is more common, as the search space is significantly reduced compared to symmetries defined over the entire discretized field, and the physical interpretation is more intuitive.
case where p 1 and q 1, we can specify the function library to include terms up to the second order, ( u) = [1,0, 2. u, x², u², xu]] T_ Clearly, v is linear with re- spect to W, i... V = Oo(a, 2 vov (W), where Oo(2, u) = To study the symmetries of differential equations, we are inter- ested in whether pr(*)v remains linear with respect to W. In fact, the following theorem will provide the answer.
..... diag diag|(@ar, ,.... (ax u) E (R(+t))x((*+0))
Theorem 4.1. Let G be Lie group acting on X U Rº R. with its corresponding Lie algebra g. Assume that the infinitesimal group action v E g takes the following form:
Applications of symmetry. Symmetry is widely applied in problems such as the design of equivariant networks and the discovery of governing equations. We summarize related works in Appendix D.
= WO(I, V
(4)
W W2
W T ((r,))- V
(5)
a
a
Σ ( T We
+ Σ O(r, u) W W
Or
p+a Ou
4. Method
a=1
(6)
We formalize the problem as follows: given dynamic data governed by differential equations, we aim to determine the number of infinitesimal generators and their explicit expres- sions. In Section 4.1, we provide a flexible function library for the infinitesimal group action and present the prolonga- tion formula in Theorem 4.1, which linear with respect to the coefficient matrix W. In Section 4.2, we introduce the infinitesimal criterion for the symmetry group of differential equations in Theorem 4.2. It is combined with central dif- ferences of the data and automatic differentiation from the trained neural network to construct a system of linear equa- tions for veo(W), whose solution space corresponds to the infinitesimal generators. In Section 4.3, we summarize the overall algorithm. In Section 4.4, we sparsify the infinitesi- mal generators for better intuitiveness and interpretability. We present the pipeline of LieNLSDD in Figure 1.
where W E (R(+t)xx and O(r, u) E RR"x1 Then, the n-th prolongation of is:
pr
So u())
(7)
Oug
where J = (i jk). with je E {L....,p) p) and k {1, (l,...,m)} n). The coefficients de are given by:
& (z, u ) = ΣΣ W..
W p+a
EIIC
(8)
where u3, = difference, and DJ represents the total derivative.
On Br
g Ow' D.t J I denotes the set
The proof of Theorem 4.1 is provided in Appendix B.2. Note the distinction between total derivatives and partial derivatives. For a smooth function P(z, ())). they are OP
4.1. Prolongation Formula of Infinitesimal Group Actions
To explicitly obtain the infinitesimal group action on X x U = Rº x RJ, we define v = W u)- V. Here, O u) E Rxx is a predefined function library, and W = [W1, W Wptel e x( is a co- efficient matrix to be determined. For example, in the
+ - E,, u9,, Bug- related by D,P ax
OP Equations (7)
and (8) indicate that pr (r) v remains linear with respect to W, thus it can be expressed as:
pr = On (1) vec .
(9)

---

Explicit Discovery of Nonlinear Symmetries from Dynamic Data
where On (2; u()) E IR. ([(+t)"""x(((t""" We provide concrete example of the construction of on in Ap- pendix C.4.
Theorem 4.2. Suppose T:
Fu ())) =0, V v= 1,....,
(10)
is system of differential equations, where F X x U(()) + Rt' is of full rank (the Jacobian matrix Jr(r, ul!))
Algorithm 1 LieNLSD
aFF, aFg E R2x(ptap'')) "()) is of rank whenever F(x, u()) 0). A Lie group G acts on X x U, with its corresponding Lie algebra g. If E Eg:
Input: Dataset D = {(1[1) =li)))= prolongation order n, function library θ : X x - I x1 sample size M,
Oz 8uy
thresholds C2.
Output: Number of infinitesimal generators d, explicit expressions of infinitesimal generators {v:)%=1-
pr(") (m)))] = 0, v= (11)
Execute:
whenever F(x, ul 0, then G is a symmetry group of L.
Estimate the derivatives of u with respect to 2 using the central difference method, resulting in the prolonged dataset pr (n = [(=[d, (n") [1))
Substituting Equation (9) into Equation (11), we find that the infinitesimal criterion is also linear with respect to W:
Separate the variables of pr(") = Sin U Saut into input and output features (for example, for first-order dynamic system, S.n (w'} and Sout {u,)), and train neural
Jr(r, u (2) )
) (2r, u m(") vee()) = 0, (12)
network to fit the mapping f Sin Sout. Construct O, X x (m))) (e++0 "x)xx())))()))))>>)))))))))))))) from O by Equations (o (9).
whenever Fla, u (())) = 0. As shown in Equations (7) to (9), O, can be computed from θ, which is manually specified. Therefore, as long as JF and the data samples of (I, (*)) are available, W can be solved for.
Apply automatic differentiation to the trained neural net-
work, obtaining Jp : X x Utn)_ + (xx((=-p(
if Mi- < 6 then (p+q)-r
In practice, we use the central difference method to esti- mate arbitrary-order derivatives of with respect to from the dataset D = [ [ obtaining the prolonged dataset (r) D = [xx[].
Compute C E IR (((A(=)))((+++)))) by Equation (13). Perform SVD on C, the number of singular values less than 62 is d, and the corresponding right singular vector matrix is Q E IR ()))
i==- We then separate the variables in pr(") into input and output features, fitting the mapping of this supervised learning problem using a neural network. For example, first-order dynamic system is governed by a PDE of the form 14, f(u') (Kantamneni et al., 2024), where w' represents the set of u and its spatial derivatives. In this case, the differential equation can be expressed as F(I, u(²) = f(u') de = O, and neural network is used to fit f. The Jacobian matrix Je. can then be obtained by applying automatic differentiation to the trained neural network.
[] u (m)) []]
else
Initialize CTC 10
for = 1 to M do
Ci - CT CTCCC
(a(() (i).
end for
Perform SVD on CTC C, the number of singular values less than o is d, and the corresponding right singular vector matrix is Q E (x)) end if
Equation (12) holds for each data sample in (r We sample M points from it for symmetry discovery, resulting in the following system of linear equations:
for i 1 to d do Q(: reshape( + q,r). W. - W.O(x, x, u). V.
end for
(JrOw) (n) (T)
(L)))
Return d, {v.
(m))
Cvoc(W)=
vcc(W) = 0.
(J+On) (m) [M))
4.2. Explicit Discovery of Infinitesimal Group Actions
(13)
Perform Singular Value Decomposition (SVD) on C: Cvec(W)= - U
Our goal is to determine the coefficient matrix W of the in- finitesimal group action v WO(2,, u)- V. and we have al- ready derived the formula for its prolongation (r Theo- rem 2.31 in the textbook (Olver, 1993) relates the symmetry group of differential equation to its prolonged infinitesimal group action, which is restated as follows.
Σ
0
Q vec(W) = 0. Then, the null
space of C' is the solution space of vec(W)), with the general solution being voc(W) Q3. In other words, the column vectors of Q E r)) form a basis for the space of veo(W), and B E Rdx1 is the coordinate vector. Overall,
5

---

Explicit Discovery of Nonlinear Symmetries from Dynamic Data
the number of zero singular values of C represents the num- ber of infinitesimal generators, and the corresponding right singular vectors are their coefficient matrices. Therefore, compared with recent methods for discovering nonlinear symmetries (Yang et al., 2024a; Ko et al., 2024; Shaw et al., 2024), our approach can mathematically determine the di- mension of the Lie algebra subspace and explicitly reveal the infinitesimal generators.
original problem is transformed into:
min | 1,1. R,
s.t. RRR-I, - ZQQR.
(15)
As shown in Algorithm 2, during the iterative process, we alternately update R and Z. Here, the 2-norm of a matrix Alla = mmaz (A) is the largest singular value, and the oo- norm of matrix || so = max; E, |4tyl is the maximum row sum. The detailed derivation is provided in Appendix G.
When the hyperparameter M > r, for efficiency, we compute C.C E R(((+t))))x(( ((--))-r) instead of C E Specifically, CTC Ei C,T.,. C₁, where C, = (JpO,)(x(1), u (1). Note that if C UEVT, then C = VV²VT. Therefore, performing SVD on CCC will give us the singular values and right singular vectors of C' that we are interested in.
Algorithm 2 Basis sparsification
Input: Basis matrix Q E (R(((=))-)xxd
Output: Sparse basis matrix Q" E ()(
Initialize é1 > 0, 62 > 0, Bmar > 8o > 0, Po 2 1, JR 11Q1 7 1, Ro Ld. Zo QRo. Ao
0 ((p+g))))x and l +- 0. repeat
4.3. The Overall Algorithm
Update RA+1 UVT, where UEVT is the SVD of (' (e(He--2))))
We now summarize the LieNLSD method Algorithm LieNLSD consists of two main stages: neural network train- ing and symmetry discovery, which are decoupled from each other. In other words, the symmetry discovery procedure is plug-and-pe aad--ndd-ppa play for a trained neural network. The time and space complexity analysis of LieNLSD is provided in Appendix E. Although LieNLSD starts from the setting of nonlinear symmetries and dynamic data, it can still handle cases of linear/affine symmetries and static data (governed by the arithmetic equation F(x)= = 0), which are discussed in Appendix F.
R₂
BanR
Update Zk+1 S((nz))-1 7x + Act -x+)
BLUZ
where Sx(x) sgn(x) m c, is the soft
thresholding operator.
Update A8+1 = A2 + 3a(Q - Zx+1).
Bg max ( Ra+1 - Rxlloo, vTZ 1
ZR lx < E2 then
Set p = Pv.
else
Set p 1. end if
Update Bk+1 = min min(3mar. p8x). 1: - k: + 1.
4.4. Basis Sparsification
In Algorithm 1, SVD guarantees the orthogonality of the basis, but it does not ensure its sparsity. For example, if the ground truth basis vectors are v [1,00, 1, 0) and 2
until |QQ - ZZlloo < El and 36-1 max(V/Rlll4 /RRl
R2-1llos /Tz Zx-1lloo)= S 62
Return QRy.
2 (o. 1,T we might solve for -1,,,1,,,,1] and 2 1,, -1] Although they span the same subspace and are both orthogonal, the latter lacks intuitiveness and inter- pretability. Therefore, we aim to find an orthogonal trans- formation that makes the transformed basis as sparse as possible, which can be formalized as:
5. Experiment
In this section, we evaluate LieNLSD on top quark tagging (Section 5.2) and a series of dynamic systems (Section 5.3). The selection of the quantitative metric and baseline is dis- cussed in Section 5.1. We demonstrate the application of LieNLSD in guiding data augmentation for neural PDE solvers in Section 5.4. The additional experiments are pro- vided in Appendix I.
min [QQRI s.t. RR R
(14)
where Q E ((((+)))r)xd R E Rkd and the (1, n)- of matrix ||A En, - |Aay is the sum of the absolute val- ues of all its elements. Note that due to the non-smoothness of the objective function, gradient- trasient-based optimization meth- ods such as SGD (Robbins & Monro, 1951), Adam (Kingma, 2014), etc., cannot work. We use the Linearized Alternating Direction Method with Adaptive Penalty (LADMAP) (Lin et al., 2011) to solve this constrained optimization problem. By introducing an auxiliary variable Z E R. (((+)))))xd the
5.1. Quantitative Metric: Grassmann Distance
LieNLSD uses the subspace where the Lie algebra resides to represent symmetries. Therefore, we choose the Grass- mann distance, which measures the difference between the computed subspace and the ground truth subspace, as a quantitative metric. Let Q1, Q2 E ma xd be the orthogonal bases of two d-dimensional subspaces in R", Perform SVD
6

---

Explicit Discovery of Nonlinear Symmetries from Dynamic Data
on Qi Q2 to obtain the singular values {o The Grass- mann distance between the two subspaces is then defined as
is M 100. For LieGAN, we set the dimension of the Lie algebra basis to 7, using an MLP with 2 hidden layers and hidden dimension 512 the discriminator, which is the same setting as in the original paper (Yang et al., 2023), More implementation details are provided in Appendix H.1.
(Q(Q Q2) = / E/ ff. where 0g arccos d; are the principal angles.
As shown in Table I, other methods for discovering nonlin- ear symmetries (Yang et al., 2024a; Ko et al., 2024; Shaw et al., 2024) cannot explicitly provide the subspace where the Lie algebra resides, rendering this quantitative metric inapplicable. We extract linear infinitesimal generators from LieNLSD for comparison with LieGAN (Yang et al., 2023), the state-of-the-ard method for explicitly discovering Lie al- gebra basis. Before calculating the Grassmann distance, we perform QR decomposition on all basis matrices to ensure orthogonality and normalization. Furthermore, we will com- pare the number of parameters in LieNLSD and LieGAN. The parameter overhead of LieNLSD is mainly focused on the neural network used to fit the mapping, while that of LieGAN mainly concentrated in the discriminator.
We present the visualization result of LieNLSD on top quark tagging in Figure (in Appendix H.I). LieNLSD obtains 7 nearly zero singular values, which indicates that the number of infinitesimal generators is 7. The corresponding explicit expressions are shown in Table 3. These constitute Lorentz symmetry and scaling symmetry, where V1, V3, V5 repre- sent spatial rotations, V2. V4: Ve represent Lorentz boosts, and V7 represents scaling transformations. As shown in Table 2, even for linear symmetry discovery, LieNL.SD out- performs LieGAN with fewer parameters. Additionally, LieNLSD can automatically determine the dimension of the Lie algebra subspace, whereas LieGAN requires manual specification.
We quantitatively compare LieNLSD with LieGAN on all experiments in this section in Table 2. The randomness of LieNLSD mainly comes from the selection of sample points for symmetry discovery, while that of LieGAN mainly arises from the random seed setting.
5.3. Nonlinear Symmetry Discovery
We next evaluate the ability of LieNLSD to capture non- linear symmetries on dynamic data governed by Burgers' equation, the wave equation, and the Schrödinger equation. We generate several initial conditions by randomly sampling the coefficients of the Fourier series. Then, we estimate the spatial derivatives at each point using central differences and numerically integrate the trajectories corresponding to these initial conditions using the fourth-order Runge- Runge-Kutta method (RK4). By selecting time and space steps for sam- pling, we obtain a discrete dataset.
Table Quantitative comparison of LieNLSD and LieGAN on all experiments in this section. The Grassmann distance is presented in the format of mean + std over three runs.
Dataset
Model
Grassmann distance (4)
Parameters
Top quark tagging
LieNLSD
(9.20 1.83) 10-2
97K
For LieNLSD, the function library is specified as up to second-order terms. We set the prolongation order to n = 2, and use the central difference method estimate all derivatives up to the second order. We train MLP with 3 hidden layers and hidden dimension 200 to fit the mapping f(((2) for first-order dynamic systems, or du ((((2)) for second-order dynamic systems. The sample size for symmetry discovery is AM = 100. More implementation details of dataset generation and symmetry discovery are provided in Appendix H.2.
LieGAN
(2.51 0.41) 10-1
321K
LicNLSD
(1.26
0.200)
10
81K
Burgers' equation
LieGAN
1.58 + 0.05
265K
Li
0.01))).01)
10
82K
Wave equation
LieGAN
2.36 0.15
206K
Li
(8.62 :: 1.31) 10 2
83K
Schrödinger equation
LieGAN
0.05
266K
5.2. Linear Symmetry Discovery
Top quark tagging. We first evaluate the ability of LieNLSD to discover linear symmetries on top quark tag- ging (Kasieczka et al., 2019). The task is to classify hadronic tops from QCD backgrounds. Its input consists of the four- momenta //¹ = (pl: .P. 1² p?) E Rd of several jet con- stituents produced in an event, and the output is the event label (1 for top quark decay, 0 for other events).
Burgers' equation. Burgers' equation describes the convection-diffusion phenomenon, which is widely applied in areas such as fluid mechanics, nonlinear acoustics, gas dy- namics, and traffic flow. Its potential form is u, Uzz + ug.
We present the visualization result of LieNLSD Burgers' equation in Figure 3 (in Appendix H.2). LieNLSD ob- tains 6 nearly zero singular values, which indicates that the number of infinitesimal generators is 6. The corresponding explicit expressions are shown in Table 3. The group actions they generate for the symmetry group are 91 (1, x. u) =
This is static system, so we set the prolongation order to n = O. To discover linear symmetries, the function library is specified as Opp")= = Ipº, p' .p?: ppp1 E R4x1, We use an MLP with 3 hidden layers and hidden dimension 200 to fit the mapping. The sample size for symmetry discovery
7

---

Explicit Discovery of Nonlinear Symmetries from Dynamic Data
Table 3. Infinitesimal generators found on all experiments in this section by LieNL.SD.
Ditanct
Top quakk legzing
Bungers equztion
Wave equation
Schnidinger equation
1 1y
61
-
5
0 Fu
052
+ 5V
0
16'
V4
16
x²)
211
20
Ganmaon
V.
0%
+
YL
20
Viz 26
Oyt
8y
Via D6
Via
l- 1-let. .92 (t, (c2't, Ex u), 93 (t, u) = (t,z + 2ct, u - cx et),
t x u 1 In(11 - 4ct) cx2 )x²) (t, u) le 1-tett
We present the visualization result of LieNLSD the Schrödinger equation in Figure 5 (in Appendix H.2). LieNLSD obtains 6 nearly zero singular values, which in- dicates that the number of infinitesimal generators is 6. The corresponding explicit expressions are shown in Ta- ble 3. Then, u = f(1, x) is a solution to the Schrödinger equation, we can obtain several derived solutions through these infinitesimal generators. Specifically, V1 represents scaling i1 = e' f xx)), V22 represents rotation in the complex plane U12 = Rf x), V3 represents space ro- tation U3 = ft,, R 'x), Va represents time translation un = f(t - C, x), and vs. V. represent space translation U556 = f(1, - +), where E SO(2) and e E IR..
2
91 u, = (t,z,u+c), gs (t,x, u) = (t x. u). and 9o (1, u) = (1, u), where 9i = exp(ev.). The prac- tical meaning that, if uf = f(t x)
is a solution Burgers'
equation, then 211 = [1447t 1- 1 In(1 4ct) T440 U2 f(e-21,,, e xx), 11g f(t, x - 2el) - ex E
04 I((,z) + e, us= = 1(| - e, ([--,.)) and us- = 1(1, - c)
are also solutions.
Wave equation. The wave equation describes how waves propagate through various mediums. Its form in two- dimensional space is Uee = Uga + "yy, where u(t, x. y) is the displacement or, more generally, the conserved quantity at time/ / and position (x.
LieNLSD has already demonstrated qualitative advantages over LieGAN, such as the discovery of nonlinear symme- tries and the determination of Lie algebra subspace dimen- sion. Furthermore, we extract the linear infinitesimal gen- erators found by LieNLSD on the aforementioned three dynamic datasets and quantitatively compare their accuracy with the results from LieGAN. The discriminator of Lic- GAN is set as an MLP with 2 hidden layers and hidden dimension 512, which is the same setting as in the original paper (Yang et al., 2023). As shown in Table 2, although LieNLSD expands the function library θ to include up to decond-order terms, while LieGAN's symmetry discovery is limited to a linear search space, LieNL.SD still achieves more accurate linear symmetries with fewer parameters.
We present the visualization result of LieNLSD on the wave equation in Figure (in Appendix H.2). LieNLSD ob- tains 20 nearly zero singular values, which indicates that the number of infinitesimal generators is 20, The corre- sponding explicit expressions are shown in Table 3. There are 10 infinitesimal generators that constitute the confor- mal symmetry, where Vs, Vg, V12 represent Lorentz trans- formations, Vis, V19, V20 represent translations, Vi0 rep- resents scaling transformations, and V1, V2, Ve represent special conformal transformations. Additionally, V13 gener- ates the group action exp(evis))t, 2, y, u) (t, xr y, e' e'u), and V3, V4, V5. V7, V11, V14; V15, ViG, V17 can be unified as Va = att, x, y) Du. where a is a solution to the wave equa- tion au = Ogx + Gyy-- This implies that if u = f(, v) is a solution to the wave equation, then U13 = f v) and v. = ft,, x. y) + caa(t, x. y) are also solutions.
5.4. Application: Guiding Data Augmentation for Neural PDE Solvers
We further apply the symmetries identified by LicNLSD to Lie Point Symmetry Data Augmentation (LPSDA) (Brand- stetter et al., 2022) to improve the long rollout accuracy of the Fourier Neural Operator (FNO) (Li et al., 2021), type of neural PDE solver. Due to the technical limitation of LPSDA to oie-dimensi cases, we evaluate it on Burg- ers' equation, the heat equation (see Appendix 1.1), and the KdV equation (see Appendix 1.2). The implementation details remain consistent with the original paper (Brandstet-
Schrödinger equation. The Schrödinger equation de- scribes the evolution of the quantum state of microscopic particles over time. Its parameterization on plane in terms of real (u) and imaginary (w) components is expressed as:
Ut = -0.5(U2z + Vyy)) + vu² + v3,
(16)
F1 0.5(Mz + Uyu) - u 3
8

---

Explicit Discovery of Nonlinear Symmetries from Dynamic Data
Table Comparison of long rollout test NMSE for FNO without data augmentation, FNO with I.PSDA based on Ko al. (2024), FNO with LPSDA based on LieNLSD, and FNO with LPSDA based on ground truth (GT). The results are presented in the format of mean = std over three runs with different random seeds.
Dataset
FNO + (
FNO + Ko et al.
FNO + LieNLSD
FNO + GT
Burgers' equation
(2.33 + 1.07)
10-4
(1.93 + 0.75) 10-4
(1.80 + 0.56) x 10-1
(1.75 + 0.37) 10-4
Heat equation
(1.07 + 0.10) 10-1
(7.33 + 1.07) x 10-2
(5.99 +: 0.04) 10-2
(6.01 + 0.20) x 10
KdV equation
0.10) x 10-1
x 10-1
(1.42 = 0.16) x 10-1
(1.47 0.02) x 10-1
ter et al., 2022). As shown in Table 4, compared to FNO without data augmentation, FNO with LPSDA based on
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential socictal consequences of our work, none which we feel must be specifically highlighted here.
LieNLSD improves accuracy by over 20%, which is com- parable to FNO with LPSDA based ground truth (GT). This further validates the effectiveness of LieNLSD results.
Additionally, we compare the long rollout accuracy of FNO with LPSDA based on LicNLSD and FNO with LPSDA based on Ko et al. (2024). For Ko et al. (2024), although they cannot obtain explicit expressions for the infinitesimal generators, feeding a given point into their trained MIP still yields the specific values of the infinitesimal generators at that point, thereby guiding data augmentation. The im- plementation details are consistent with the original paper. As shown in Table 4, Ko et al. (2024) can improve the ac- curacy of FNO, but not as significantly as our method. In addition to quantitative advantages, Ko et al. (2024) require the explicit expression of the PDE to compute the validity score (see Section 4.2 of their original paper), whereas our LieNLSD does not rely on this prior knowledge.
References
Akhound- Sadegh, T., Perreault-Levasseunr L., Brandstetter, J., Welling, M., and Ravanbakhsh, S. Lie point symmetry and physics-informed networks. Advances in Neural Information Processing Systems, 0444246844248 2023.
Arora, S., Bihlo, A., and Valiquette, F. Invariant physics- informed neural networks for ordinary differential equa- tions. Journal of Machine Learning Research, 25(233): 1-24, 2024.
Bekkers, J., Lafarge, M. W., Veta, M., Eppenhof, K. A., Pluim, P., and Duits, R. Roto-translation covari- ant convolutional networks for medical image analysis. In Medical Image Computing and Computer Assisted Intervention 2018: 21st International Confer- ence, Granada, Spain, September 16-20, 2018, Proceed- ings, Part 1, pp. 440-448. Springer, 2018.
6. Conclusion
To our knowledge, we propose the first pipeline that can determine the number of infinitesimal generators and explic- itly provide their expressions containing nonlinear terms. The selection of the function library θ is more flexible com- pared with the Lie algebra representation. This is why the infinitesimal group action solved by LieNLSD can include more nonlinear terms. After training the neural network, we solve for the coefficient matrix W of the infinitesimal group action using mathematical methods. This is why LieNLSD can obtain the entire Lie algebra subspace and determine its dimension. In the future, we expect to use the symmetries identified by LieNLSD to guide the discovery of conserved quantities or differential equations.
Benton, G., Finzi, M., Izmailov, P., and Wilson, A. G. Learn- ing invariances in neural networks from training data. Ad- vances in Neural Information Processing Systems, 33: 17605-17616, 2020.
Bertolini, M., Le, T., and Clevert, D.-A. Generative model- ing on Lie groups via Euclidean generalized score match- ing. arXiv preprint arXiv:2502.02513. 2025.
Bluman, G. and Anco, S. Symmetry and integration methods for differential equations, volume 154. Springer Science & Business Media, 2008
Acknowledgements
Bluman, G. W. Applications of symmetry methods to partial differential equations. Springer, 2010.
Z. Lin was supported by National Key R&D Program of China (2022ZD0160300) and the NSF China (No. 62276004).
Brandstetter, J., Welling, M., and Worrall, D. E. Lie point
symmetry data augmentation for neural PDE solvers. In International Conference on Machine Learning, pp. 2241- 2256. PMLR, 2022.
9

---

Explicit Discovery of Nonlinear Symmetries from Dynamic Data
Champion, K., Lusch, B., Kutz, J. N., and Brunton, S. L. Data-driven discovery of coordinates and governing equa- tions. Proceedings of the National Academy of Sciences, 116(45):22445-2244 2019.
Guan, Y., Brunton, S. L.,, and Novosselov, I. Sparse nonlin- ear models of chaotic electroconvection. Royal Society Open Science, 8(8):202367, 2021.
Hu, L., Li, Y., and Lin, Z- Symmetry discovery for different data types. Neural Networks, pp. 107481, 2025.
Cohen, T. and Welling, M. Group equivariant convolu- tional networks. In International Conference on Machine Learning, pp. 2990-2999. PMLR, 2016.
Hydon, P. E. Symmetry methods for differential equations: beginner's guide. Number 22, Cambridge University Press, 2000.
Cohen. T. S. and Welling, M. Steerable CNNs. In Interna- tional Conference on Learning Representations, 2017.
Ibragimov, N. K. Elementary Lie group analysis and or- dinary differential equations, volume 197. Wiley New York, 1999.
Dehmamy, N., Walters, R., Liu, Y., Wang, D., and Yu, R. Automatic symmetry discovery with Lie algebra convolu- tional network. Advances in Neural Information Process- ing Systems, 34:2503-2515, 2021.
Kantamneni, S., Liu, Z., and Tegmark, M. OptPDE: Discov- ering novel integrable systems via Al-human collabora- tion. arXiv preprint arXiv:2405.044: 2024.
Desai, K., Nachman, B., and Thaler, J. Symmetry discovery with deep learning. Physical Review D, 105(9):096031, 2022.
Kasieczka, G., Plehn, T., Thompson, J., and Russel, M. Top quark tagging reference dataset. (No Title), 2019.
Kingma, D. P. Adam: A method for stochastic optimization.
Esteves, C., Allen-Blanchette. C., Makadia, A., and Dani- ilidis, K. Learning SO(3) equivariant representations with spherical CNNs. In Proceedings of the European Confer- ence on Computer Vision (ECCV), pp. 52 68, 2018.
arXiv preprint arXiv:1412, 6980, 2014.
Ko, G., Kim, H., and Lee, J. Learning infinitesimal genera- tors of continuous symmetries from data. arXiv preprint arXiv:2410.21853, 2024.
Esteves, C., Sud, A., Luo, Z., Daniilidis, K., and Makadia, A. Cross-domain 3D equivariant image embeddings. In International Conference Machine Learning, pp. 1812- 1822. PMLR, 2019a.
Kondor, R. and Trivedi, S. On the generalization of equivari- ance and convolution in neural networks to the action of compact groups In International Conference on Machine Learning, pp. 2747-27777 2755. PMLR, 2018.
Esteves, C.,, Xu, Y., Allen-Blanchetter C., and Danillidis, K. Equivariant multi-view networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019b
Lagrave, P.-Y. and Tron, E. Equivariant neural networks and differential invariants theory for solving partial differen- tial equations. In Physical Sciences Forum, volume 5, pp. 13. MDPI, 2022.
pp. 1568-1577, 2019b.
Esteves, C., Makadia, A., and Danilidis, K. Spin-weighted spherical CNNs. Advances in Neural Information Pro- cexsing Systems, 33:8614-8622 2020.
Lengyel, A., Strafforello. O., Bruintjes, R.-J,, Gielisse, A., and van Gemert, J. Color equivariant convolutional net- works. Advances in Neural Information Processing Sys- tems, 36, 2024.
Finzi, M., Stanton, S., Izmailov, P., and Wilson, A. G. Gen- eralizing convolutional neural networks for equivariance to Lie groups on arbitrary continuous data. In Interna- tional Conference on Machine Learning, pp. 3165-3176. PMLR, 2020.
Lenssen, J. E.,, Fey, M., and Libuschewski, P. Group equiv- ariant capsule networks. Advances in Neural Information Processing Systems, 31, 2018.
Li, J., Yang. Z., Liu, H., and Cai, D. Deep rotation equivari- ant network. Neurocomputing. 290:26-33. 33, 2018.
Finzi, M., Welling, M., and Wilson, A. G. A practical method for constructing equivariant multilayer percep- trons for arbitrary matrix groups. In International Con- ference on Machine Learning, pp. 3318-3328. PMLR, 2021.
Li, X., Wang, S., and Cai, Y. Tutorial: Complexity analysis of singular value decomposition and its variants. arXiv preprint arXiv: 12085, 2019.
Li, Y., Pang, Y., and Shan, B. tion for learning the solution operator of linear differential equations. In 2022 IEEE Sth International Conference on Cloud Computing and Intelligent Systems (CCIS), pp. 543-547. IEFE, 2022.
data augmenta-
Graham, S., Epstein, D., and Rajpoot, N. Dense steerable filter CNNs for exploiting rotational symmetry in histol- ogy images. IEEE Transactions on Medical Imaging, 39 (12):4124 4136, 2020.
10

---

Explicit Discovery of Nonlinear Symmetries from Dynamic Data
Li, Y., Qiu, Y., Chen, Y., He, L., and Lin, Z. Affine equivari- ant networks based on differential invariants. In Proceed- ings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 5546-5556, 2024.
Romero, D. W. and Lohit, S. Learning partial equivariances from data. Advances in Neural Information Processing Systems, 35:36466-36478, 2022.
Satorras, V. G., Hoogeboom, E.,, and Welling, M. E(n) equivariant graph neural networks. In International Con- ference on Machine Learning, pp. 2021.
Li, Y., Qiu, Y., Chen, Y., and Lin, Z. Affine steerable equivariant layer for canonicalization of neural networks. In The Thirteenth International Conference on Learning Representations, 2025.
PMLR,
Shaw, B., Magner, A., and Moon, K. Symmetry discovery beyond affine transformations. Advances in Neural Infor- mation Processing Systems, 37:1 12889-1129188 2024.
Li, Z., Kovachki, N. B., Azizzadenesheli. K., Bhattacharya, K., Stuart, A., Anandkumar, A., et al. Fourier neural operator for parametric partial differential equations. In International Conference on Learning Representations, 2021.
Shumaylov, Z., Zaika, P., Rowbottom, J., Sherry, F., Weber, M., and Schönlieb, C.-B. Lie algebra canonicalization: Equivariant neural operators under arbitrary Lie groups. arXiv preprint arXiv:2410.02698, 2024.
Lin, Z., Liu, R., and Su, Z. Linearized alternating direction method with adaptive penalty for low-rank representation. Advances in Neural Information Processing Systems, 24, 2011.
Sosnovik, I,, Moskalev, A., and Smeulders, A. How to transform kernels for scale-convolutions. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1092-1097, 2021.
Loiseau, J.-C. and Brunton, S. L. Constrained sparse Galerkin regression. Journal of Fluid Mechanics, 838: 02- 2018.
van der Ouderaa, T., Immer, A., and van der Wilk, M. Learn- ing layer-wise equivariances automatically using gradi- ents. Advances in Neural Information Processing Systems, 36, 2024.
MacDonald, L. E., Ramasinghe, S., and Lucey, S. Enabling equivariance for arbitrary Lie groups. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1183-8192. 2022.
Wang. A. X., Shumaylov, Z., Zaika, P., Sherry, F, and Schönlieb, C.-B. Generalized Lie symmetries in physics-informed neural operators. arXiv preprint arXiv:2502.00373, 2025.
McLachlan, R.
On the numerical integration of ordinary differential equations by symmetric composition methods. SIAM Journal on Scientific Computing, 16()):111-168 1995.
Wang, D., Walters, R., Zhu, X., and Platt, R. Equivariant Q learning in spatial action spaces. In Conference on Robot Learning, pp. 1713-1723, PMLR, 2022a.
Moskalev, A., Sepliarskaia, A., Sosnovik, I., and Smeul- ders, A. LieGG: Studying learned Lie group generators. Advances in Neural Information Processing Systems, 35: 25212-252233 2022.
Wang. R., Walters, R., and Yu, R. Incorporating symmetry into deep dynamics models for improved generalization. In International Conference on Learning Representations, 2021.
Naderi, H., Goli, L., and Kasaei, S. Scale equivariant CNNs with scale steerable filters. In 2020 International Confer- ence on Machine Vision and Image Processing (MVIP),
Wang. R., Walters, R., and Yu, R. Approximately equivari- ant networks for imperfectly symmetric dynamics. In In- ternational Conference on Machine Learning, pp. 23078- 23091. PMLR, 2022b.
pp. 1-5. IEEE, 2020.
Weiler, M. and Cesa, G. General E(2)-equivariant steer- able CNNs. Advances in Neural Information Processing Systems, 32, 2019.
Olver, P. J. Applications of Lie groups to differential equa- tions, volume 107. Springer Science & Business Media, 1993.
Weiler, M., Geiger, M., Welling, M., Boomsma, W., and Cohen, T. S. 3D steerable CNNs: Learning rotation- ally equivariant features in volumetric data. Advances in Neural Information Processing Systems, 31, 2018a.
Robbins, H. and Monro, S. A stochastic approximation method. The annals of mathematical statistics, pp. 400- 407, 1951.
Romero, D.,, Bekkers, E., Tomezak, J., and Hoogendoorn, M. Attentive group equivariant convolutional networks. In International Conference on Machine Learning, pp- PMLR, 2020.
Weiler, M., Hamprecht, F. A., and Storath, M. Learning steerable filters for rotation equivariant CNNs. In Pro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 449, 2018b.

---

Explicit Discovery of Nonlinear Symmetries from Dynamic Data
Winkels, M. and Cohen, S. Pulmonary nodule detection in CT scans with equivariant CNNs. Medical image analysis, 55:15-26, 2019.
Zhu, Y., Chen, T., Kong, L.,, Theodorou, E., and Tao, M. Trivialized momentum facilitates diffusion generative modeling Lie groups. In The Thirteenth International Conference on Learning Representations, 2025.
Worrall, D. and Brostow, G. CubeNet: Equivariance to 3D rotation and translation. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 567-584, 2018.
Worrall, D. and Welling, M. Deep scale-spaces: Equivari- ance over scale, Advances in Neural Information Process- ing Systems, 32, 2019.
Worrall, D. E., Garbin, S. J., Turmukhambetov, D., and Brostow, G. J. Harmonic networks: Deep translation and rotation equivariance. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5028-22280 1028-5037, 2017.
Xie, X., Zhou, P., Li, H., Lin, Z., and Yan, S. Adan: Adap- tive Nesterov momentum algorithm for faster optimizing deep models. IEEE Transactions Pattern Analysis and Machine Intelligence, 2024.
Yang, J.. Walters, R., Dehmamy, N., and Yu, R. Generative adversarial symmetry discovery. In International Con- ference on Machine Learning, pp. 39488-39508, PMLR, 2023.
Yang. J., Dehmamy, N., Walters. R., and Yu, R. Latent space symmetry discovery. In International Conference on Machine Learning, pp. 56047-56070. PMLR, 2024a.
Yang. J., Rao, W., Dehmamy, N., Walters, R., and Yu, R. Symmetry-informee governing equation discovery, arXiv preprint arXiv:2405.1 16756, 2024b.
Zaheer, M., Kottur, S., Ravanbakhsh, S., Poczos, B., Salakhutdinov, R. R., and Smola, A. J. Deep sets. Ad- vances in Neural Information Processing Systems, 30, 2017.
Zhang, Z.-Y.. Zhang, H., Zhang, L.-S., and Guo. L.-L. En- forcing continuous symmetries in physics-informed neu- ral network for solving forward and inverse problems of partial differential equations. Journal of Computational Physics, 492: 12415, 2023.
Zhou, A., Knowles, T., and Finn, C. Meta-learning symme- tries by reparameterization. In International Conference on Learning Representations, 2021.
Zhu, W., Qiu, Q. Calderbank, R., Sapiro, G., and Cheng, X. Scaling-translatinn translation -tauslatioo-euuivariiatt networks with decom- posed convolutional filters. Journal of machine learning research, 23(6)) 2022.
12

---

Explicit Discovery of Nonlinear Symmetries from Dynamic Data
A. Preliminary
In this section, we provide an in-depth introduction to some preliminary knowledge Lie groups and differential equations. For more details, please refer to the textbook (Olver, 1993).
A.1. Group Actions and Infinitesimal Group Actions
A Lie group G a mathematical object that is both a smooth manifold and a group. The Lie algebra g serves as the tangent space the identity of the Lie group. A Lie algebra element v E g can be associated with Lie group element g E G through the exponential map exp: : g + G, i.c., 9 = exp(v). Representing the space of the Lie algebra with basis (V1, V2,.... Vr we have g exp (E== vvv,), where
(c) 2 ,...... (") R" are the coefficients. The group G' acts on vector space X = _ via group action V x X + X. Correspondingly, we can define the infinitesimal group action of on X.
Definition A.1. Let G be a Lie group with its corresponding Lie algebra The group action of G on a vector space X is given by : G x + X. Then, the infinitesimal group action of v E g at E X is:
d
(vv)l,
((exp(ev), V. de e=0
(17)
We abbreviate (gg, x) and (vv) as g. x and v, respectively, when the context is clear. We refer to the infinitesimal group actions of the Lie algebra basis as infinitesimal generators.
This a more general definition that encompasses the nonlinear case. Previous works focusing on linear symmetries (Moskalev et al., 2022; Desai al., 2022; Yang et al., 2023) commonly use the group representation px G + GL(n) to characterize group transformations, i.e., Vy E G, X (g, x) px (g)x. The corresponding Lie algebra representation dpx + gl(n) satisfies Vv E px (exp(v))) = exp(dpx(v)). Below, we demonstrate the relationship between infinitesimal group actions and Lie algebra representations.
Proposition A.2. Let G be a Lie group with its corresponding Lie algebra g. Suppose the group action of G on vector space X is linear:
Vg,,x) = px (g)x. E G
(18)
Then, we have:
((v),, E dpx V.
(19)
The proof of Proposition A.2 and concrete example of group actions and infinitesimal group actions are provided in Appendices B.1 and C.I.
A.2. Symmetries of Differential Equations
Before defining the symmetries of differential equations, we first explain how groups act on functions. A concrete example provided in Appendix C.2.
Definition A.3. Let f X + U be function, with its graph defined as F = [(2, f()) : c x U. Suppose
Lie group G acts on X x U. Then, the transform of y by G' is given by g r ((ē, ū) = g (x,,) (1, u) E T
We call function f X U the transform of f by g, denoted as f- g f. if its graph satisfies F g Ty.
Note that f may not always exist, but in this paper, only consider cases where it does. The solution of differential equation is expressed in the form of a function. Intuitively, the symmetries of a differential equation describe how one solution can be transformed into another.
Definition A.4. Let I be a system of differential equations, with the independent and dependent variable spaces denoted by X and U, respectively. Suppose that a Lie group G' acts on X x U. We call the symmetry group of / if, for any solution f X of yf and Vg E G, the transformed function g f X - U another solution of s
A.3. Prolongation
To further study the symmetries of differential equations, we need to "prolong" the group action on the space of independent and dependent variables to the space of derivatives. Given a function f X - U, where X Rp and U TR.
13

---

Explicit Discovery of Nonlinear Symmetries from Dynamic Data
8 y (a) has Pk = ( distinct k-th order derivatives uÿ = (x) = 8g 02 8x where o E {I,.... 1), (i....,, )k), and ji [I....,P). We denote the space of all k-th order derivatives as U = P and the space of all derivatives up to order as ((m) U x U1 ....U, Un = grp wheee p(*) (rtt". Then, we define the prolongation of f.
Definition A.5. Let X U be smooth function. Then the n-th prolongation of S, pr X Utm), defined as:
pr (1) f(z) = u u = 0))
(20)
Based the above concepts, the n-th order system of differential equations I can be formalized as:
Fala, u (*)) = 0, V v=
(21)
where F: X U(x) + Rd. smooth function f X U is a solution of s if it satisfies F(x, sx))
Below, we explain how prolong the group action X x U to X U(m).
Definition A.6. Let G be a Lie group acting on the space of independent and dependent variables X U. Given
point (zo: [m)) E X x (r,), suppose that smooth function ʃ X - U satisfies up = =p((0) (xo)) Then, the n-th
(n) prolongation of g E G at the point (zg. =n")) defined as:
pr(" g- (ro, up ((())) = (To, )
(22)
where
(zo, ūo) = g (xo: uo);
u6 = pr (g-)) (āo).
(23)
Note that by the chain rule, the definition of pr g depends only on (.p, up (n)) and is independent of the choice of f. Similarly, we can prolong the infinitesimal group action on x U X U()).
Definition A.7. Let G be Lie group acting on X x U, with its corresponding Lie algebra g. Then, the n-th prolongation of v E g at the point (z, (())) E x (n)) is defined as:
pr(")y
d de
{prim)) f (exp(ee)) (x, u())) V.
(24)
(2,a(n))
Concrete examples of prolongation are provided in Appendix C.3.
B. Proof
B.1. Proof of Proposition A.2
Proof. Vv E g, E X, we have:
((v)l, d
de V(exp(ev))x).
(25)
c=0
d
de --0 px(exp(cv))r V
(26)
d (xp(dxx(ev)lz
(27)
de c=0
d exp(cdpx(v))
(28)
de
dv)xvvxx V.
(29)
14

---

Explicit Discovery of Nonlinear Symmetries from Dynamic Data
B.2. Proof of Theorem 4.1
Before proving Theorem 4.1, we first introduce Theorem 2.36 from the textbook (Olver, 1993) lemma.
Lemma B.1. Let G be a Lie group acting on X U R R2, with its corresponding Lie algebra g. Denote the infinitesimal group action of E g as:
(r,
u
Se (x u)
(30)
Oxt
Oua
1=1
a=l
Then, the n-th prolongation of v is:
0 Vy (5 m())
pr +
(31)
Oug
where J (Jk)...,Jk). with je 1...., p pand k I....,n. n. The coefficients 0% are given by:
(( DD &a
(32)
where ut 0
Ou2 Dzt
ghtt n 0z Ox B and D. represents the total derivative.
Next, we prove Theorem 4.1 follows.
Proof. By Lemma B. I and the multivariable derivative formula, we have:
(x) =D,
e @T pta
EoTwat Wow
(33)
DDO Wpta
p+a
Σ
u9 D D Σ e STw, W.
(34)
im TCJ
UF W; + D,, pta..
(35)
C. Example
In Appendices C. to C.3, we present several key examples from the textbook (Olver, 1993) to help readers intuitively understand the preliminaries. In Appendix C.4, we provide an example of the construction of θ, in Equation (9).
C.1. Example of Group Actions and Infinitesimal Group Actions
Example C.I. Consider the 2D rotation group G =SO(2), with its group action given by:
(ee, (z. v)) (I cos yin sin e + y cos cos.).
(36)
Then, according to Equation (1) (or Definition A. its infinitesimal group action is:
( = - Oz Oy
+
x
(37)
Its Lie algebra representation is:
00 -1
dpx(v)
(38)
1 0
Thus, Equation (2) (or Proposition A.2) is satisfied.
15

---

Explicit Discovery of Nonlinear Symmetries from Dynamic Data
C.2. Example of Group Actions on Functions
Example C.2. Let X R and U -R. The 2D rotation group G SO(2) acts X U:
e: (z, u) = (Ir cos e sin e, x sin + u cosc).
(39)
Let f: X - U be a linear function u f(x: = oz + b. The points on the graph of f transform as follows:
(ā, 8) (a cos (ar + b) sin e, sin (ar + b) c)s
(40)
To solve ū = [2), we first solve for inversely (assuming cos e asin e + 0):
+ bin
I
(41)
COS e sin
Then we obtain the transformed function f = 6 =:
= f(ī) sin + a CoS c
b
I
(42)
COS ε sin E
CO8 a asin
C.3. Examples of Prolongation
We first provide an example of the prolongation of a function.
Example C.3. Let X R22 with coordinates (1r, y), and U R with coordinate Then U3 R.2 has coordinates (Mxs thy
U2 3 has coordinates (Maz May: Myy), and (22)) U x U1 x U2 = Rº has coordinates u (2) (u: U2y Uyi Unass Uays Uyy
Consider the function u = far y). Then its second prolongation u = pr fIr y) is:
0f 0f f²f 0. 0² Ox dy 0x2 Ordy' 0yy²
(u;; Ugy Vyy Mans Ugys Uyy)
(43)
We next provide example of the prolongation of group action.
Example C.4. Let X IR and U R. The 2D rotation group SO(() acts on x U as shown in Equation (39). Given point (z", n. ( E X U()), we choose function
U as:
f(x) = uh. + (u u 2)
(44)
which satisfies:
f(x)==i) f( w..
(45)
From Equation (42), the transform of f by is (assuming cose u sin + 0):
f(i) = f(i) sin e u cos c T +
④ 2000
(46)
cose u sin e
cos e 2 sin
Let (r())) (000 n0 (o",",,u)) (50, n0, 7.). Then, have:
sin e + n cos 6
a = ((0))
(47)
cos u sin
Thus, the first prolongation (0()) X (11) is:
sin e + uy COS
pr ( (x, u, uz) ( - u usin e, sin e + u cos
(48)
cos - My sin
We finally provide an example of the prolongation of an infinitesimal group action.
16

---

Explicit Discovery of Nonlinear Symmetries from Dynamic Data
Example C.S. Let X = R and U R. The 2D rotation group G -SO(2) acts on X x U as shown in Equation (39). Its corresponding infinitesimal group action is:
0
0
- +
(49)
dx
8u
As shown in Equation (48), have:
pr O (exp(ev)((z, u,
x cOS usi e, sin E + cose,
sin + Ug COS
(50)
c U2 sin
Then, according to Equation (3) (or Definition A.7), we obtain the first prolongation of v:
0
a
(r -1 dr
+ x au
+ (1+ u²))
(51)
Zuz
C.4. Example of the Construction of O
Example C.6. Consider the case where X = R and U = R. In this case, W = [W1, [W] E R2xr and v = ou- According to Equations (7) and (8), we have pr(")y = v + o=t (x. u(l)) Du."
(xx, u) W1 0 (z )) u,D, W1 D.o This can be rewritten as:
dz + (x, u) TW a
a
where
TT
Wi [2
eT D
v
(52)
D = -1(x, u (vec)))(WW) V.
(53)
If we specify the function library (zr,u) [1 u, x², u,,,,,,, au]] T, then its total derivative is computed as D2O
[0, 1, Uz 2x, 2u + zuz)]
D. Applications of Symmetry
One application of symmetry is the design of equivariant networks, which embed group symmetries into the network structure such that when the input undergoes transformation, the output will undergo corresponding transformation. Cohen & Welling (2016) propose the method of group convolutions, which has been extended to different types of groups (Sosnovik et al., 2021; Worrall & Welling, 2019; Zhu et al., 2022; Naderi et al., 2020; Finzi et al., 2020; MacDonald et al., 2022; Li et al., 2024) and action spaces (Lenssen et al., 2018: Li et al., 2018; Bekkers et al., 2018; Romero et al., 2020; Worrall & Brostow, 2018; Winkels & Cohen, 2019; Esteves et al., 2018; 2019a;b). At the same time, some works construct partial equivariant networks (Wang et al., 2022b) and color equivariant networks (Lengyel et al., 2024) based on the framework of group convolutions. Later, Cohen & Welling (2017) propose the method of steerable convolutions, which is generalized by Weiler & Cesa (2019) an equivariant convolution framework for the general E(2) group. It has also been applied to different kinds of groups and action spaces (Worrall al., 2017; Graham et al., 2020; Wang et al., 2022a; Esteves et al., 2020; Li et al., 2025).
Another application of symmetry is the discovery of governing equations (Loiseau & Brunton, 2018: Guan et al., 2021; Yang et al., 2024b), which helps reduce the search space of equations and improve accuracy. In neural PDE solvers, recent works have successfully incorporated symmetries into Physics-Informel Informed Neural Networks (PINNs) (Arora et al., 2024; Lagrave Tron, 2022: Shumaylov al., 2024: Zhang al., 2023; Wang al., 2025; Sachhond-Sddeeee et al., 2023) or performed data augmentation based PDE symmetries (Li al., 2022; Brandstetter et al., 2022), leading to significant performance improvements. Furthermore, symmetry concepts have enabled notable advances in generative modeling, with a series of works making outstanding contributions to extending diffusion models from Fuclidean space to Lie groups (Bertolini et al., 2025; Zhu et al., 2025). However, these works require prior knowledge of symmetries, and incorrect symmetries can have negative effects. Therefore, symmetry discovery from data has become an important topic. We expect to combine our data-driven symmetry discovery approach with these works in the future, eliminating the need for prior knowledge of the symmetry group to guide their processes.
17

---

Explicit Discovery of Nonlinear Symmetries from Dynamic Data
E. Time and Space Complexity Analysis of LieNLSD
The computational cost of the symmetry discovery procedure is concentrated in the SVD, which has time and space M-f
complexities of (mm min(m, n)) and (mm) for matrix m=*x (Li al., 2019). When (p+g) E, the time and space complexities of performing SVD C ((M are ( + q)r) and ((Mi(p + g)r). Otherwise, performing SVD on C C E (((++t)))))(((+))>>>>>> results in time and space complexities of (((pt)))r") and (({++)))r)) In summary, the time and space complexities of the symmetry discovery procedure are OO + q)r minmii,, (p g)r) and O((p + g)r min(Ml, (p + g)r)), respectively.
F. Extension of LieNLSD to Linear/Affine Symmetries and Static Data
Linear/aftine symmetry discovery. LieNLSD can discover a broader range of nonlinear symmetries compared with group representation-basee methods (Dehmamy et al., 2021; Moskalev et al., 2022; Yang et al., 2023), thanks to the flexibility in the choice of the function library ((x, u). If we want to restrict LieNLSD to discovering linear symmetries, we can set (x, u) = [ar, u]] [(++))x1, where the coefficient matrix W corresponds to the Lie algebra representation according to Equation (2). Furthermore, for aftine symmetry discovery, we can include a constant term in the function library: (x, u) = [1, x, u]" E R(I+++))x1.
Symmetry discovery from static data. Relative to the dynamic data governed by the differential equation F(z, u(")) O. refer to the data governed by the arithmetic equation F(x) = 0 as static data. The infinitesimal criterion for the arithmetic equation, similar to Equation (11), is v(F(z)) whenever Fzz) O. To apply LieNLSD for symmetry discovery from static data, we simply set the prolongation order n 0. In this case, the procedure no longer needs to estimate derivatives but directly uses the original dataset.
G. Detailed Derivation of Basis Sparsification
We use LADMAP (Lin al., 2011) to solve the constrained optimization problem presented in Equation (15). The augmented Lagrangian function is constructed as:
C(R, A) = || QR Z) 8
-1QR-2 2
(54)
where A E ((((+)))))x the Lagrange multiplier, is the penalty parameter, and the Frobenius norm of a matrix ||A||F Eis A.y. During the iteration process, we alternately update R and Z.
We first update R with Z fixed, as shown in Equation (8) of Lin et al. (2011):
BEUR 2
(Ax + BeRQRR Zx))
R 1 arg min R=1
R
(55)
BUUR
QT (Aa + Bi (QH - 2.)) then have:
Let R. Rg
R arg min
|| R1
(56)
R-
arg min Ir((R rt) TR) R?) RTR=1
(57)
arg min tr(R" R.) 2t R) + tr(R R)
(58)
R. R=1
arg min d - 2tr(R d-2tr(RR R) tr( R) (ZTR=)
(59)
arg max tr(R R).
(60)
R R-
Let T T the SVD of i = R₁
Q (An Ba (QB 2,))
then:
Bann
RE+1 = arg max tr UEVT) R R=1
(61)
= arg max (((TRTUU) R R=1
(62)
18

---

Explicit Discovery of Nonlinear Symmetries from Dynamic Data
Let = U, then = RVVVRTU I. We have:
R&+1 arg max tr(SE) R R=1
(63)
arg max
Skoz.
(64)
R=1
The orthogonality of S and the non-negativity of the singular values imply that S S 1 and o; 2 0. Therefore, when Sa 1, i.e., when S V R U I, the expression Σi=1 Sgo, reaches its maximum. Due to the orthogonality of U and V, we have R UVT Thus, obtain the update formula for R:
R&+1 = UVT.
(65)
We then update Z with R. fixed, as shown in Equation (9) of Lin et al. (2011):
3k7Z
As + Ba (R [k}
Zx+1 arg min 1211,,1 +
Z Zx
(66)
2
F
Let 7 = Zx Ax+ (Q++1108 Zz) then:
8.nz
3x7Z
xx+1 = arg min ||
+
1 7
(67)
2
2
SkA2
arg min 2
Zzgl
+
Σ (Zig - Zzz)².
(68)
2
i
This problem can be optimized element-wise:
(Zk-1)9 arg min |2| Bu71% (z 7zn7²
(69)
Let f() = |2| + Danz (= - 2. When > 0, f²) = 1 + B=11 7zj)) = 0 implies 2 7/j 1
1
where
3znz
Zij BRn= When = < 0, (z) -1 -1+ f= - Z.j) = 0 implies 7.i + Benz where 7.jj < 3,1z
1
When 0,
if it is the optimal solution, then 0 E af(()) = [-11 - Bz Zigj B1 Zijl. and in this case, B.nx S 7.j S summary:
In
B.n
[Zij-
1
Zij Banz
Banz
(2R+1)s3 =
Zij + Janz :
Zij < Benz'
(70)
0,
DAnz < Zij S Jenz
Thus, we obtain the update formula for Z:
Zu+1 S(3,n2)-1
(71)
A& + Ba(QR Zk)
= -((Bnz))-
Zk +
(72)
where S.(r)) = sgn(x) max([=] -, 0) the soft thresholding operator.
H. Implementation Detail and Visualization Result
H.1. Linear Symmetry Discovery
Top quark tagging. In this task, we observe the four-momenta 224 E I of the 20 jet constituents with the highest transverse momentum pr, and predict the event label (1 for top quark decay, 0 for other events). We configure an MLP with 3 hidden layers, setting the input dimension to 80, the hidden dimension to 200, and the output dimension to 1. The activation function is ReLU. This is a binary classification task, so we apply the Sigmoid function to the output and use BCE as the loss function. For training, we set the batch size to 256 and use the Adan optimizer (Xie et al., 2024) with learning
19

---

Explicit Discovery of Nonlinear Symmetries from Dynamic Data
rate of 10-3. For symmetry discovery, share a group action, which allows treat these 20 channels as single channel in the subsequent analysis. This is static system, we set the prolongation order to 0. With the function library set to O(p*) (p", pp,pp, p²] [R
Theorem 3 in LieSD (Hu al., 2025) ensure that the 20 four-momenta
Theorem 4.1 gives Gn in Equation (9) as:
0
O
0
9
T
0
E 4
(73)
0
0
0
0
0
0
We sample 100 points from the training set to construct C' R100x16 in Equation (13) (in practice, we construct CTC ReeRl6e6 16)16). For quantitative analysis, we conduct multiple experiments with different sampled points report error bars. As shown in Figure 2, consider the singular values smaller than 62 10 as the effective information of the symmetry group, where the singular values drop sharply to nearly zero. For basis sparsification, set 61 10 and 10-1, while keeping the remaining hyperparameters consistent with the original paper (Lin et al., 2011).
Figure 2. Visualization result of symmetry discovery on top quark tagging by LieNLSD. The first subplot shows the last 8 singular values. The other subplots display the infinitesimal generators corresponding the nearly-zero singular values after sparsification.
H.2. Nonlinear Symmetry Discovery
Burgers' equation. For dataset generation, we first uniformly sample N₂ spatial points x E R.N, on the interval [-4.54 4]
By sampling the coefficients of the Fourier series f(x) so + Σ (a,, cos 2n52 + b sin 2n3 from Gaussian distribution, we obtain Nics initial conditions uo R Niea N=, We then uniformly sample N₂ time points E I on the interval [0, T and numerically integrate the trajectories u E R.N.. N.N.= using the fourth-order Runge-Kutta method (RK4), which, along with t and æ, forms the discrete dataset. In practice, we set L = 20, N₂ 100, N2 10, Noss 10, 2, and N₂ = 1000.
For LieNLSD, we set the prolongation order to n 2, use the central difference method to estimate the derivatives Ue, Ur, Maz, Ut.. We configure an MLP with 3 hidden layers to fit the mapping u, f(u, Uz Uax), setting the input dimension the hidden dimension 200, and the output dimension 1. The activation function is Sigmoid. For training, set the batch size to 256 and use the Adan optimizer (Xie et al., 2024) with learning rate of 10-3 For symmetry discovery, we specify the function library up to second-order terms as O u) [1,4, u, 12 x² u2 Lx, tu, avi]" 110x1 The infinitesimal group action v has the form:
W a + +0tt,x,u) Tw,
+ xtt, Wa à Ou'
v = O(t, v,t, u) W1 at
(74)
az
20

---

Explicit Discovery of Nonlinear Symmetries from Dynamic Data
where W
[11 W2, W3] T E RR According Theorem 4.1, the second prolongation of is:
0
1r -ow,,
W33 0u
+-u,D,,,T, uzD, DDO 0
Our
+ (, 6" T + 2u D2 WW (u Dan θ + 2u D₂ D, + +Dzx +D2 Wal
Juzz
0
+ W- - - W; + D W3)
Ou,
+...
(75)
where D,O = 1, 0, ue, 21, 0, 2 v, u + bue Ius D2O 0, 1, ux 0, 2x, 2ulli i, bug u + ru,] and D O [0, O, O, Myz. 0, 2, (u + ull.)) 0, Uugz 20 + Iugn We omit the irrelevant terms here because we assume the form of the differential equation Flu, U2s Ugzs Ue) f(u, [u, uz Uxs) ur O, which only depends on u, Ugy Uzei Ug. More specifically, for example, since have OF
the row corresponding to a in O2 of Equation (12) will be ignored. Buer
0012
Equation (75) gives O₂ in Equation (9) as:
θ D D T D,O
Ue D - (,, θ D.O Du
Uy O D θ + 2u D,O
O₂
R 1x30
(76)
-,
We sample 100 points from the training set to construct C R 100x30 in Equation (13) (in practice, we construct CTC 300030). For quantitative analysis, we conduct multiple experiments with different sampled points to report error bars. As shown in Figure 3, we consider the singular values smaller than 62 0.5 as the effective information of the symmetry group, where the singular values drop sharply nearly zero. For basis sparsification, set Ei 10-4 and cg 10-4, while keeping the remaining hyperparameters consistent with the original paper (Lin et al., 2011).
214
Figure Visualization result symmetry discovery Burgers' equation by LieNLSD. The first subplot shows the last singular values. The other six subplots display the infinitesimal generators corresponding to the serrly-zero singular values after sparsification.
Wave equation. For dataset generation, we first rewrite the equation in its first-order form:
Uc U, U/ Uxx Uyg-
(77)
uniformly sample N2 spatial points on the interval [- [- 2 with coordinates T R and y E R By sampling the coefficients of the Fourier series f(x,y) Nr (amn cos 2Mar cos 2nny + bmm cos L sin 2nny Cmn sin 2mnaa cos 2nay dmn sin 2maz sin 2nsT
We
an
+
from Gaussian distribution, we obtain Nic. initial conditions ug E IR Ni.. X and Up E R Mic xN2 We then uniformly sample N₂ time points N.N. the interval [0, T], and numerically integrate the trajectories E N.N. XN using the fourth-order Runge-Kull Kutta method (RK4). We sample every points in space to obtain the discrete dataset t R N: 2 IR (N))000 . E (N=/00)) ū E Nex.N110 In practice, we set 20, N₂ = 100, Ny = 3, Nics 10, 2, and N, 1000.
R N. N. and
21

---

Explicit Discovery of Nonlinear Symmetries from Dynamic Data
For LieNLSD, we set the prolongation order to n = 2, and the central difference method estimate the deriva- tives We, Mar My; Utt, Uaz: Myy: Mtz, Mtys May- We configure
MLP with 3 hidden layers to fit the mapping Vee f(,, Uz Mys Mayy Myys Uzy), setting the input dimension to 6, the hidden dimension to 200, and the output dimension to 1. The activation function is Sigmoid. For training, we set the batch size to 256 and use the Adan optimizer (Xie et al., 2024) with a learning rate of 10-3. For symmetry discovery, we specify the function library up to second-order terms as (t, F, y, u) = [1, 1 x, y. u, t2, x², ..y2 u², tæ, by, tu, ay, æu, yu]" E 111x1 The infinitesimal group action v has the form:
a +0(t,z,,,u) W3 dy T
a
v = V. W1 Ot
+ ((t. W2 dr
+ at,,,,v,u u) Wa Ou'
(78)
where W- = [Wi, 22, W3, Wa]J E R4x15, According to Theorem 4.1, the second prolongation of v is:
pr v - W4 ru
0 + (-u, W1 12 W2 - u,0 W3 + + W4)
Ou. θ
+ + Wi un D. θ W2
W3 + D2 Wa) (W))
Buy
+ + Te + 20 D, W (u, Dan θ 2u D₂ W
(unD O + 2uz D θ W + DRI θ Wal
Ougx
(- T + 2u, D, 0
W TW
(u, by O
T
+ 2uny M, D,OT)Ws
θ
(uyD θT + 2uy D2 θ W3 + Dwy θ W[] Juyy
(u,DDa θT + u , θ + D,0 T TW; (y OT + tuZoD + 2
0
aay T + Uzy D, TT Myy D.
D 0 W3 + D θ W4]
Juzy
+ (W + 2u |W, OT. Zu 0 W)2
(uyDuO + D,0 DW's + D Wal Oun
+...
(79)
where
D20 = [0, 1, 0, u. 2t, 2u4, x, x, + tut O, xu,, yue] T
D₂O = [0, 0,2, 0, 2wu,, t, tu,, y, u + Nu, yu
D,0 = [0,
0, tuy, x. Tuy; u + yuyl
D&O 0,0, W..
um.), 0, 0, 2u + tuee, O, xuse, yuus] TT
(80)
Dza θ = [0,0,0,0, u.n. 0,2,0,2(u2 + wuzz). 0, O, tuge, 0, 2ug + Iuga yuzæ]
Day θ = 00,0,0,0, " ,y, 2,0,2, + uwyy); 0, O, buy, O, xuly: 2u, + yugg
D Day θ 0,,0,0,0 May. 2,0,0,2(u,,4 + ulzy), 0, O, tuxy: 1, My Dulyy Us + ylayl T.
It gives O2 in Equation (9) as:
0 DDD.
DDe,eT
DT
D,OT
n D
- D,OT
D,OT
D
02-
Or. + 2,.
Du, + 20 D.0
Dua + 20 (u, DTT + 20 D,OT)
e
E R
(w. + 2n D
- Dw. + 20 D,0
OT
a + D, + D,OT) -(u + 2,
Do, + D,0 + t. DT)
Da,, May D, + "ng D,OT)
Da + 2,,
-u, + +,y
D&0
(81)
We sample M = 100 points from the training set to construct C' E R 100x60 in Equation (13) (in practice, we construct CT E 200x60). For quantitative analysis, we conduct multiple experiments with different sampled points to report error bars, As shown in Figure 4, we consider the singular values smaller than c2 = 1 as the effective information of the symmetry
22

---

Explicit Discovery of Nonlinear Symmetries from Dynamic Data
group, where the singular values drop sharply nearly zero. For basis sparsification, set 61 10 and e2 10-3
while keeping the remaining hyperparameters consistent with the original paper (Lin al., 2011).
FT 137-
Figure 4. Visualization result of symmetry discovery on the equation by LieND.DD. The first subplot shows the last singular values. The other twenty subplots display the infinitesimal generators corresponding to the nearly-zero singular values after sparsification.
Schrödinger equation. For dataset generation, we follow the same approach as the wave equation to obtain the time series t RV: grid coordinates z, R N² and trajectories
R Ni.. N.x,,, We sample every 10 points in space to get the discrete dataset RN,,, , _ (1)) - R /100 ū RN (NNx(1= T Niex x(N For the parameters, we set L 20, N₂ 100, N2 2, Nics 10, T 2, and N, 1000.
For LieNLSD, we set the prolongation order to 2, and use the central difference method to estimate the derivatives Me, Uy, Maws Myys Uta, May: May: Ut, Uz Vy: Uaay Gyy: Utas Cty, Vay. We configure an MLP with 3 hidden layers to fit the mapping (u,, vy) = f(u, Uzz My: thary Uyy. Mays v2 Up, Ug. Uzx, Uyp- U2y). setting the input dimension to 12, the hidden dimen- sion to 200, and the output dimension to 2. The activation function Sigmoid. For training, we set the batch size to 256 and use the Adan optimizer (Xie al., 2024) with learning rate of 10- 3 For symmetry discovery, we specify the function library up second-order terms @(,, y/s u, v) [1,4, U, u, v, x² y² u² v² txx, by, tu, bv, ay, zu, 2o, yu, you uo] 22ix1 The infinitesimal group action has the form:
Ott, u, u) T Wi se Ott, W. & + O(L,,,,,, TW by 2(,,,,,,u,)) ou Ot,, W, ye
(82)
where W
[1, W2, Ws. W, W3]T e p5x21
Similarly to the wave equation, Theorem 4.1 gives O₂ in Equation (9) as:
DD -.D,eT
DDe
D,D
D,eT
D,e e D.
D,D (u, 2a D.0
D
( + D,6
D
0
D,, + D,,,
Dw + D,0
Du + Du D 0
D
0
( θ- D,e e u. D.0
-i, + D, D.e Da,
D, + D.e
D
0
eT
02
D,
D.T
-0D,00T
D,eT
D2,
- D2eT D..,D
D,0T
(w. D,0
Dv.
D,, + 26, D,eT)
D
(v,,,, 2ny D,0
D.,0 + 2 D,0
(,, + 20 D,OT
D
(v. AT D,6 D,0
D,e D.TT)
(n, 2. D,0 + D.0
D2,0T
-,e
0,
D,eT
D,9
0
D
D.OT
D,OT
D,0
(83)
We sample M = 100 points from the training set to construct C 100xx005 in Equation (13). For quantitative analysis, we conduct multiple experiments with different sampled points to report error bars. As shown in Figure 5, we consider the singular values smaller than 62 2 as the effective information of the symmetry group, where the singular values
23

---

Explicit Discovery of Nonlinear Symmetries from Dynamic Data
drop sharply nearly zero. For basis sparsification, we 61 10 and e2 10-3, while keeping the remaining hyperparameters consistent with the original paper (Lin et al., 2011).
Figure Visualization result symmetry discovery Schrödinger equation by LieNLSD, The first subplot shows the 7 singular values. The other six subplots display the infinitesimal generators corresponding to the nearly-zero singular values after sparsification.
I. Additional Experiment
We supplement experiments on dynamic data governed by the heat equation, the Korteweg- De Vries (KdV) equation, and reaction-diffusion system. The quantitative comparison between LieNLSD and LieGAN on additional experiments provided in Table 5.
Table Quantitative comparison of LieNLSD and LieGAN on additional experiments.
The Grassmann distance is presented in the format
of mean 1 std over three
Dataset
Model
Grassmann distance (4)
Parameters
LieNLSD (7.39 1.04) x 10
81K
Heat equation
LieGAN
2.59 + 0.04
265K
LieNLSD
(5.24 + 1.55) x 10
82K
KdV equation
LieGAN
1.56 +: 0.00
265K
LieNLSD (1.24 + 0.18) x 10
83K
Reaction-diffusion system
LieGAN 1.11 + 0.11
266K
Table 6. Infinitesimal generators found on additional experiments by LieNLSD.
Dataset
Heat equation
KdV equation
Reaction-diffusion system
9
9
0
(2f +
x²
Ou
21
Øt
+
Ox
31
0t
2u
Ou
Vi
Ou
+
Or
0
0
0
0
ò
0
Generators
Vg 21
8r
211
Ou
V4 = x
Ou'
V;
On
V2
Or
On
-1
Or
Oy
0
0
0
0
0
0
Ve
Ou'
0x
V8
8f
Øt
Or
Ht'
0r
V5
I.I. Heat Equation
The heat equation describes the evolution the temperature distribution over time in given medium. Its one-dimensional form is:
0/ Uzz:
(84)
24

---

Explicit Discovery of Nonlinear Symmetries from Dynamic Data
where x) is the temperature time / and position z. The experimental setup is consistent with Burgers' equation.
Figure Visualization result symmetry discovery on the heat equation by .SD. The first subplot shows last singular values. The other cight subplots display the intinitesimal generators corresponding to the nearly-zero singular values after sparsification.
We present the visualization result of LieNLSD on the heat equation in Figure 6. LieNLSD obtains 8 nearly zero singular values, which indicates that the number of infinitesimal generators is 8. The corresponding explicit expressions are shown in Table 6. The group actions they generate for the symmetry group are (t, a, u) (t, a, u + e(2t + x²)), exp(ev2) (t, I, u) = (e² e'x, u). exp(ev3) x, u) (f, 2et, ue = exp(ev,) u) = (t,z,u+en) exp(cv,)(tt u) = (t, x, u), (t, u) = (f, x. u-) (tp(ev>)(t, u) = (t, x u), and (x,, (t + 6, x, w). The practical meaning is that, u = ftt x) is a solution to the heat equation, then i1 = f(t, + c(t e exte't f(t, 2ct), 44 = f(t, x) + cr, us = e ft, us = (tt, +c = f(t,
U2 = fe ((t,-,t, e * and Us f(t e, x) are also solutions.
1.2. Korteweg-De Vries (KdV) Equation
The KdV equation describes the propagation of solitary waves on shallow water surfaces. Its standard form is:
Uazz
uig
0,
(85)
where u(t, x) the wave profile at time and position æ. The experimental setup is consistent with Burgers' equation.
Figure 7. Visualization result of symmetry discovery on the KdV equation by LieNLSD. The first subplot shows the last 5 singular values. The other four subplots display the intinitesimal generators corresponding to the nearly-zero singular values after sparsification.
We present the visualization result of LicNLSD on the KdV equation in Figure 7. LieNLSD obtains 4 nearly zero singular values, which indicates that the number infinitesimal generators is 4. The corresponding explicit expressions are shown Table 6. Then, if = [t,, æ) is a solution to the KdV equation, we can obtain several derived solutions through these infinitesimal generators. Specifically, V1 represents scaling #n = =2* f( exx, V2 represents Galilean boost U2- = ft,, et) + Va represents time translation u3 f(t e, 2), and v4 represents space translation u4 f(t, x - E).
I.3. Defftiinn System
reaction-diffusion system describes the behavior of chemical substances or biological patterns that undergo both reaction and diffusion. We consider X-w reaction-diffusitn system (Champion et al., 2019) governed by:
Wt (1 (w² + x²) 3(u² x²) r2)x + di (uga + Ugy): (u² + v² (1 - (u² + "²)) ) + dz (Urr Ugy):
(86)
with d = 0.1, d2 0.1, and
1. The experimental setup consistent with the Schrödinger equation.
25

---

Explicit Discovery of Nonlinear Symmetries from Dynamic Data
Figure 8. Visualization result of symmetry discovery on reaction-diffusion system by LieNLSD. The first subplot shows the last 6 singular values. The other five subplots display the infinitesimal generators corresponding the nearly-zero singular values after sparsification.
We present the visualization result LieNLSD on reaction-diffusion model Figure 8. Although LaLiGAN discovers SO(2) symmetry in the latent space of this system, cannot explicitly provide the group action in the observation space, LieNLSD takes the independent variables into account and directly discovers the SO(2) symmetry in the observation space. Specifically, LieNLSD obtains 5 nearly zero singular values, which indicates that the number of infinitesimal generators is 5. The corresponding explicit expressions are shown in Table 6. Among these, Vi represents rotation in the complex space, V2 represents space rotation, represents time translation, and V1, represent space translation.
26


Reviews:
[{"review_id": "7faMmEPoWV", "reviewer": "Reviewer_pziR", "summary": "In this paper, the authors proposed LieNLSD, a method for explicitly discovering nonlinear Lie group symmetries that are not represented by $\\mathrm{GL}(n)$, from data governed by a PDE $ F(x, u^{(n)}) = 0$. The proposed method parameterizes a nonlinear infinitesimal group action $\\mathbf{v}$ acting on $(x, u)$ using a symbolic regression technique $\\mathbf{v} = W \\Theta(x, u) \\cdot \\nabla$, where $\\Theta(x, u)$ is a predefined symbolic library and $W$ is a learnable matrix. The method then constructs the prolonged version of this group action, which satisfies $\\mathrm{pr}^{(n)} \\mathbf{v} = \\Theta_n(x, u^{(n)})\\mathrm{vec}(W) \\cdot \\nabla$. Utilizing the fact that the prolonged group action must satisfy $ \\mathrm{pr}^{(n)} \\mathbf{v} [F(x, u^{(n)})] = 0$ whenever the PDE $F(x, u^{(n)}) = 0$ exhibits symmetry under this group, the authors derive a natural learning criteria $J_F(x, u^{(n)}) \\Theta_n (x, u^{(n)}) \\mathrm{vec}(W) = 0$.\n\nTo find Jacobians $J_F$ from data pairs $(x[i], u[i])$ originating from an unknown PDE, the authors first construct derivatives $u^{(n)}[i]$ using the central difference method, then approximate the PDE through a neural network $f$, assuming a form of $u_t$ (or $u_{tt}$ or $0$) $= f(u’) \\implies F(x, u^{(n)}) = f(u’) – u_t = 0$. Using the Jacobian approximated via automatic differentiation of the neural network $f$, the authors solve $J_F(x, u^{(n)}) \\Theta_n (x, u^{(n)}) \\mathrm{vec}(W) = C \\mathrm{vec}(W) = 0$ through SVD of $C$ (which means that the method automatically identifies the Lie algebra dimension based on the number of zero singular values) , with additional sparsifications. \n\nThe authors benchmark their proposed approach with top quark tagging and PDE datasets, and showcase the use-case of the extracted symmetries to enhance the performance of neural PDE solvers.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your careful reading and valuable feedback! Below we will address each of your concerns point by point.\n\n**Methods And Evaluation Criteria**\n\n(1) See the Methods And Evaluation Criteria section of the Rebuttal to Reviewer 1vcx.\n\n(2) We conduct ablation experiments using a limited $\\Theta$ (including only linear and constant terms) and a large $\\Theta$ (additionally including sine and exponential terms) to evaluate the robustness of LieNLSD with respect to the choice of function library. The quantitative comparison results (Grassmann distance) with LieGAN are presented below.\n\n|Dataset|LieNLSD (limited $\\Theta$)|LieNLSD|LieNLSD (large $\\Theta$)|LieGAN|\n|-|-|-|-|-|\n|Burgers' equation|$(1.30\\pm0.23)\\times10^{-2}$|$(1.26\\pm0.20)\\times10^{-2}$|$(1.06\\pm0.14)\\times10^{-2}$|$1.58\\pm0.05$|\n|Wave equation|$(1.41\\pm0.01)\\times10^{-2}$|$(1.40\\pm0.01)\\times10^{-2}$|$(1.43\\pm0.03)\\times10^{-2}$|$2.36\\pm0.15$|\n|Schrodinger equation|$(3.13\\pm0.91)\\times10^{-2}$|$(8.62\\pm1.31)\\times10^{-2}$|$(8.74\\pm0.65)\\times10^{-2}$|$2.22\\pm0.05$|\n|Heat equation|$(3.08\\pm1.29)\\times10^{-3}$|$(7.39\\pm1.04)\\times10^{-4}$|$(1.31\\pm0.51)\\times10^{-3}$|$2.59\\pm0.04$|\n|KdV equation|$(9.66\\pm3.39)\\times10^{-3}$|$(5.24\\pm1.55)\\times10^{-3}$|$(1.12\\pm0.41)\\times10^{-2}$|$1.56\\pm0.00$|\n|Reaction-diffusion system|$(8.23\\pm1.13)\\times10^{-2}$|$(1.24\\pm0.18)\\times10^{-1}$|$(1.24\\pm0.13)\\times10^{-1}$|$1.11\\pm0.11$|\n\nFor large $\\Theta$, although we misspecify several terms, LieNLSD can still obtain accurate results within a large search space. For limited $\\Theta$, LieNLSD may indeed miss some generators, but for all generators whose terms are fully included in $\\Theta$, it can still accurately discover them (as shown in the table above, LieNLSD with $\\Theta$ containing only linear and constant terms can correctly identify all linear symmetries).\n\n**Experimental Designs Or Analyses**\n\nAlthough the Grassmann distance is not applicable to the method of implicit symmetry discovery, we additionally compare the long rollout test NMSE of FNO with LPSDA based on LieNLSD and FNO with LPSDA based on Ko et al. [1]. For Ko et al., while they cannot obtain explicit expressions for the infinitesimal generators, feeding a given point into their trained MLP still yields the specific values of the infinitesimal generators at that point, thereby guiding data augmentation. Note that for PDEs, the types of symmetries found by LaLiGAN and our method differ (details can be found in the Claims and Evidence section of the Rebuttal to Reviewer 1vcx), making it impossible to establish a unified metric. The experimental results are presented below. Ko et al. can improve the accuracy of FNO, but not as significantly as our method. In addition to quantitative advantages, Ko et al. require the explicit expression of the PDE to compute the validity score (see Section 4.2 of their original paper), whereas our LieNLSD does not rely on this prior knowledge.\n\n|Dataset|FNO+$\\emptyset$|FNO+LieNLSD|FNO+GT|FNO+Ko et al.|\n|-|-|-|-|-|\n|Burgers' equation|$(2.33\\pm1.07)\\times10^{-4}$|$(1.80\\pm0.56)\\times10^{-4}$|$\\mathbf{(1.75\\pm0.37)\\times10^{-4}}$|$(1.83\\pm0.25)\\times10^{-4}$|\n|Heat equation|$(1.07\\pm0.10)\\times10^{-1}$|$\\mathbf{(5.99\\pm0.04)\\times10^{-2}}$|$(6.01\\pm0.20)\\times10^{-2}$|$(1.04\\pm0.14)\\times10^{-1}$|\n|KdV equation|$(1.74\\pm0.10)\\times10^{-1}$|$\\mathbf{(1.42\\pm0.16)\\times10^{-1}}$|$(1.47\\pm0.02)\\times10^{-1}$|$(1.58\\pm0.14)\\times10^{-1}$|\n\n**Other Comments Or Suggestions**\n\n$\\Theta(x,u)\\in\\mathbb{R}^{r\\times 1}$, then $\\Theta(x,u)^\\top\\in\\mathbb{R}^{1\\times r}$. By repeating it $(p+q)$ times and arranging them diagonally, we obtain $diag[\\Theta(x,u)^\\top,\\dots,\\Theta(x,u)^\\top]\\in \\mathbb{R}^{(p+q)\\times((p+q) \\cdot r)}$. We have double-checked that it is not a typo, but thank you for your reminder!\n\n**Questions For Authors**\n\nAs mentioned in the Experimental Designs Or Analyses section of the Rebuttal, the accuracy of FNO with LPSDA can serve as one metric. In fact, the accuracy of many recent symmetry-informed neural PDE solvers (detailed in the Relation To Broader Scientific Literature section of the Rebuttal to Reviewer LaJV) can also be used to measure the quality of symmetry. Essentially, these use the performance on downstream tasks after applying symmetry as a metric.\n\nAdditionally, following the approach of LieGAN and LaLiGAN, a discriminator can be employed to quantify the distributional differences between the original and transformed data. These metrics do require some training overhead, but we believe this is nearly unavoidable in scenarios where the PDE is entirely unknown, as evaluation can only be conducted based on the dataset.\n\n**Reference**\n\n[1] Ko, Gyeonghoon, Hyunsu Kim, and Juho Lee. \"Learning Infinitesimal Generators of Continuous Symmetries from Data.\" arXiv preprint arXiv:2410.21853 (2024).", "claims_and_evidence": "The authors' claims regarding the advancements of their proposed method are clearly summarized in Table 1. The algorithm design is carefully structured to support the authors' objectives. Additionally, the claims are empirically validated through experimental results. For instance, the proposed method accurately identifies nonlinear symmetries from data across three different PDEs. Furthermore, its capability to automatically estimate the dimension of the Lie algebra is demonstrated using the top quark dataset, in contrast to LieGAN, which treats the Lie algebra dimension as a hyperparameter.\n\nHowever, there are some concerns that need to be addressed to further support the authors' claims, which will be elaborated in the Method and Experimental Designs sections.", "methods_and_evaluation_criteria": "This paper presents a rigorously justified method based on a solid mathematical formulation for nonlinear symmetry discovery. Leveraging Lie group theory, the proposed method is technically sound and principled. However, there are two potential concerns regarding the proposed method:\n\n- The Jacobian $J_F$ is approximated by training a neural network surrogate to learn the PDE directly from observed data. Consequently, the accuracy of the identified symmetry depends on the performance of this neural surrogate. Therefore, the robustness of this approach is uncertain due to two primary sources of noise: (1) numerical derivative estimation based on the central difference scheme, and (2) approximation errors inherent in the neural network itself.\n\n- The symbolic library $\\Theta$ should be predefined based on prior knowledge of the problem. The automatic selection of the Lie algebra dimension and the explicit construction of the Lie algebra basis come at the cost of manually selecting the symbolic library. A library that is too limited may fail to capture accurate symmetries, while an excessively large library increases both the search space and computational complexity. It can be particularly problematic when there is no information about the governing equation at all.\n\nTherefore, I believe the authors should discuss these aspects and conduct ablation studies to assess the robustness of the proposed method, particularly its sensitivity to noise and dependence on library selection.\n\n\nFor evaluation, the authors employ the Grassmann distance to measure similarity between the ground truth and the discovered Lie algebra subspace. While this metric is theoretically principled, its use inherently limits comparison with other nonlinear symmetry discovery methods that do not explicitly produce a subspace representation.", "theoretical_claims": "This paper is not a theoretical study but primarily presents an algorithm for discovering nonlinear symmetry groups. The algorithm is constructed based on two theorems: Theorem 4.1 and Theorem 4.2. While Theorem 4.2 is drawn from a textbook, the proof of Theorem 4.1 is provided in the paper. I briefly reviewed the proof of Theorem 4.1 and found no significant flaws.", "experimental_designs_or_analyses": "The authors benchmark the proposed method using a top quark dataset and three PDEs, comparing it against the linear symmetry discovery model (LieGAN). Additionally, the nonlinear symmetry extracted by the proposed method is utilized to enhance the accuracy of the neural Fourier operator (FNO) by augmenting its training data with Lie point symmetries. The results demonstrate that the discovered symmetry can significantly improve the performance of FNO, achieving accuracy nearly comparable to augmentation with ground-truth symmetries. This finding is highly promising.\n\nHowever, there is a concern that although the proposed method and benchmarks emphasize nonlinear symmetry, the authors do not provide comparisons with other nonlinear approaches, such as LaLiGAN and the method by Ko et al. The authors state that this omission is due to their choice of Grassmann distance as the evaluation metric, but this reasoning alone does not fully justify the absence of comparisons with these established nonlinear methods. If possible, I strongly recommend that the authors perform additional comparisons with these existing approaches using alternative, suitable metrics.", "relation_to_broader_scientific_literature": "This paper is highly relevant for identifying symmetries in PDEs, a fundamental task across various scientific fields.", "essential_references_not_discussed": "The authors provide a comprehensive list of references and discuss related work appropriately.", "other_strengths_and_weaknesses": "The paper is well-motivated, and the proposed method is technically sound. Furthermore, it is clearly written and easy to follow, particularly given the depth of the mathematical content. However, I believe the authors should address the concerns I raised in the Method and Experimental Design sections for further clarity.", "other_comments_or_suggestions": "A small typo in line 199: $\\mathrm{diag}[\\Theta(x, u)^{T}, \\dots, \\Theta(x, u)^{T}] \\in \\mathbb{R}^{((p+q)\\cdot r) \\times ((p+q)\\cdot r)}$.", "questions_for_authors": "Please refer to the Method and Experimental Designs sections for major questions. Additionally, a minor question to consider: \n\nConsider a real-world challenging scenario in which the governing PDE is entirely unknown, introducing uncertainties in both Jacobian estimation and library selection. Under such circumstances, is there an auxiliary metric or criterion available to evaluate the physical plausibility of the estimated symmetry?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UUc3NLnbJC", "reviewer": "Reviewer_1vcx", "summary": "The paper introduces a novel data-driven PDE symmetry discovery. In contrast to previous arts based on end-to-end symmetry discovery (Ko et al., Yang et al.,), the proposed methods applies a post-hoc nullspace analysis on the learned PDE operator to discover the subspace of infinitesimal symmetry generators that correspond to the Lie algebra basis. This strategy offers significant benefits that was impossible before: \n1. The dimension of the symmetry is not a predefined hyperparameter but a discovered quantity\n2. The generator's explicit differential operator representation could be discovered.\n\nFurthermore, while not highlighted by the authors, the training is much simpler and easier as the symmetry discovery is separated from the governing equation learning. This is a clear advantage over previous arts that leveraged unstable adversarial objective (Yang et al.) or several hyperparameter-sensitive regularizers (Ko et al.).", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your careful reading and valuable feedback! Below we will address each of your concerns point by point.\n\n**Claims And Evidence**\n\nNote that in our paper, the symmetry is defined on $X\\times U$ (see the \"Symmetries of differential equations\" paragraph in Section 2, lines 82-88, column 2). Therefore, the symmetries we find can be nonlinear with respect to both the coordinates $X$ and the field $U$.\n\nHowever, from an implementation perspective, the types of PDE symmetries discovered by LieNLSD (ours) and LaLiGAN differ (perhaps this is what you are concerned about?). Taking $X=R^2,U=R$ as an example (e.g. $u(x,y)$ represents a planar image), the symmetries found by LieNLSD act pointwise on $X\\times U=R^3$. Such symmetries are commonly referred to in the literature as \"Lie point symmetries.\" On the other hand, the symmetries found by LaLiGAN are defined over the entire discretized field. Specifically, if $u$ is a field on a $100\\times100$ grid, then LaLiGAN's symmetries act on $R^{100\\times100}$ (see the Reaction-diffusion dataset in the original paper of LaLiGAN for details). For PDEs, the setting of Lie point symmetries (which act pointwise on both coordinates and the field) is more common, as the search space is significantly reduced compared to symmetries defined over the entire discretized field, and the physical interpretation is more intuitive.\n\nWe will add the above comparison of the two types of symmetries to the related work section. Thank you!\n\n**Methods And Evaluation Criteria**\n\n(1) Noisy samples\n\nWe add multiplicative noise $u'=u(1+\\epsilon)$ to the variables in the training dataset to evaluate the robustness of LieNLSD against noise, where $\\epsilon\\sim N(0,\\sigma^2)$. The quantitative comparison results (Grassmann distance) between LieNLSD and LieGAN for different noise levels $\\sigma$ are as follows. Although the performance of LieNLSD declines as the noise level increases, its accuracy remains higher than that of LieGAN, which shows robustness to noise.\n\n|Dataset|LieNLSD|LieGAN|\n|-|-|-|\n|Top tagging ($\\sigma=0$)|$\\mathbf{(9.20\\pm1.83)\\times10^{-2}}$|$(2.51\\pm0.41)\\times10^{-1}$|\n|Top tagging ($\\sigma=0.05$)|$\\mathbf{(3.75\\pm0.40)\\times10^{-1}}$|$2.27\\pm0.01$|\n|Top tagging ($\\sigma=0.1$)|$\\mathbf{(8.61\\pm0.55)\\times10^{-1}}$|$2.39\\pm0.02$|\n|Burgers eqn ($\\sigma=0$)|$\\mathbf{(1.26\\pm0.20)\\times10^{-2}}$|$1.58\\pm0.05$|\n|Burgers eqn ($\\sigma=0.05$)|$\\mathbf{(1.54\\pm0.25)\\times10^{-1}}$|$1.34\\pm0.45$|\n|Burgers eqn ($\\sigma=0.1$)|$\\mathbf{(3.30\\pm0.73)\\times10^{-1}}$|$1.66\\pm0.02$|\n|Wave eqn ($\\sigma=0$)|$\\mathbf{(1.40\\pm0.01)\\times10^{-2}}$|$2.36\\pm0.15$|\n|Wave eqn ($\\sigma=0.05$)|$\\mathbf{(3.80\\pm0.26)\\times10^{-2}}$|$2.24\\pm0.05$|\n|Wave eqn ($\\sigma=0.1$)|$\\mathbf{(1.55\\pm0.64)\\times10^{-1}}$|$2.24\\pm0.26$|\n|Schrodinger eqn ($\\sigma=0$)|$\\mathbf{(8.62\\pm1.31)\\times10^{-2}}$|$2.22\\pm0.05$|\n|Schrodinger eqn ($\\sigma=0.05$)|$\\mathbf{1.89\\pm0.20}$|$2.48\\pm0.28$|\n|Schrodinger eqn ($\\sigma=0.1$)|$\\mathbf{2.14\\pm0.03}$|$2.46\\pm0.16$|\n\n(2) Insufficient samples\n\nTo evaluate the error impact of estimating high-order derivatives using central differences on low-resolution grid points (insufficient samples), we downsample the discrete points in the training dataset with a sampling interval of $s$. For the low-resolution dataset, we employ higher-precision central differences to estimate the derivatives. For example, we replace the first-order derivative calculation $u_x=[u(x+h)-u(x-h)]/(2h)$ (with a truncation error of $O(h^2)$) with $u_x=[-u(x+2h)+8u(x+h)-8u(x-h)+u(x-2h)]/(12h)$ (with a truncation error of $O(h^4)$). The results of the ablation study (Grassmann distance) are presented below.\n\n|Dataset|LieNLSD|LieGAN|\n|-|-|-|\n|Burgers eqn (s=1)|$\\mathbf{(1.26\\pm0.20)\\times10^{-2}}$|$1.58\\pm0.05$|\n|Burgers eqn (s=3)|$\\mathbf{(2.56\\pm0.48)\\times10^{-1}}$|$1.31\\pm0.45$|\n|Burgers eqn (s=5)|$\\mathbf{(6.84\\pm0.86)\\times10^{-1}}$|$1.70\\pm0.07$|\n|Wave eqn (s=1)|$\\mathbf{(1.40\\pm0.01)\\times10^{-2}}$|$2.36\\pm0.15$|\n|Wave eqn (s=3)|$\\mathbf{(1.26\\pm0.01)\\times10^{-2}}$|$2.39\\pm0.05$|\n|Wave eqn (s=5)|$\\mathbf{(1.95\\pm0.02)\\times10^{-2}}$|$2.37\\pm0.12$|\n|Schrodinger eqn (s=1)|$\\mathbf{(8.62\\pm1.31)\\times10^{-2}}$|$2.22\\pm0.05$|\n|Schrodinger eqn (s=3)|$\\mathbf{2.08\\pm0.07}$|$2.55\\pm0.19$|\n|Schrodinger eqn (s=5)|$\\mathbf{2.18\\pm0.01}$|$2.52\\pm0.21$|\n\nAs the resolution decreases, LieNLSD still performs better than LieGAN. In fact, works on PDE symmetry discovery can hardly bypass the limitation that numerical derivatives on low-resolution data tend to have large errors. For example, when calculating the validity score, Ko et al. [1] also needed to compute numerical derivatives for the transformed data to evaluate whether it satisfies the PDE (see Section 4.2 of the original paper [1] for details). However, higher-precision difference methods can be introduced to mitigate the impact of low resolution.\n\n**Reference**\n\n[1] Ko et al. \"Learning Infinitesimal Generators of Continuous Symmetries from Data.\"", "claims_and_evidence": "The paper's core claim is that the explicit representation of the PDE symmetry generators could be discovered with a post-hoc nullspace analysis on the Jacobians of the prolonged PDE operator as in Eq.13. This claim is theoretically and empirically well-supported.\n\nOne major concern is the potential confusion or overstatement regarding the term 'nonlinear symmetry.' While the symmetry operators are nonlinear with regard to the coordinate variables, the PDE and the symmetry operators are linear w.r.t. the field. The proposed method **cannot discover generators that act nonlinearly** on the field as LaLiGAN does. The authors should clarify this in Table 1 to avoid overstatement.", "methods_and_evaluation_criteria": "The method has been tested on four scientific PDE. The method could discover the correct infinitesimal generator in their explicit form. The accuracy of the discovered subspace is measured with the Grassman distance, which is also an important contribution. The accuracy of the discovered generator subspace is significantly better than LieGAN, which is natural because the proposed method employs exact SVD-based subspace discovery instead of inaccurate feed-forward discovery. \n\nHowever, it is unclear how the proposed approach would behave when the sample point is not enough or the measurement is noisy. As the proposed method relies on higher order derivatives, it is expected that the model is vulnerable to noise or low-resolution measurements. Since LieGAN does not rely on higher order derivatives, the authors should also compare the proposed method against LieGAN with **noisy and/or insufficient samples**.", "theoretical_claims": "The theoretical claims appear to be valid.", "experimental_designs_or_analyses": "This has already been discussed in *Methods And Evaluation Criteria*.", "relation_to_broader_scientific_literature": "As mentioned in the paper, the discovered symmetry could be used to augment the data for neural PDE solvers. It could also be used to prune the redundant generators that are discovered by approaches that require predefined number of generators like LieGAN.", "essential_references_not_discussed": "Relevant works are appropriately cited.", "other_strengths_and_weaknesses": "1. The proposed method requires predefined function libarary.\n2. The proposed method cannot be used for arbitrarily high order prolongations.\n3. Higher order derivatives may not be available or accurate in real-world scenarios.", "other_comments_or_suggestions": "As mentioned earlier, the statement regarding nonlinear symmetry needs further clarification, because both the PDE and the group action are linear.", "questions_for_authors": "I am willing to raise my score once my two major concerns are addressed.\n1. Clarification regarding the term *nonlinear symmetry.*\n2. Validation under noisy or insufficient number of sample points.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "opBMTxZxQQ", "reviewer": "Reviewer_LaJV", "summary": "This paper introduces LieNLSD, a novel method for discovering nonlinear symmetries from trajectory data. It addresses the limitations of previous methods that primarily focus on linear symmetries. LieNLSD aims to determine the number of infinitesimal generators and their explicit expressions. The method involves specifying a function library for coefficients of the lie algebra, showing its prolongation formula's linearity with respect to the library coefficient matrix, and solving a system of linear equations derived from the infinitesimal criterion using SVD. The authors also apply LADMAP for sparsification of infinitesimal generators. The method demonstrates strong results on  various dynamical systems, and improves the accuracy of neural PDE solvers when using discovered symmetries for data augmentation.\n\n## update after rebuttal\nMy concerns were addressed and with the new ablations, I raised my score.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your careful reading and valuable feedback! Below we will address each of your concerns point by point.\n\n**Methods And Evaluation Criteria**\n\n(1) See point (1) in Methods And Evaluation Criteria section of Rebuttal to Reviewer 1vcx.\n\n(2) According to the procedure in Section 2.4 of the textbook [1], given the expression of the PDE, we can mathematically derive its true infinitesimal generators:\n\nBurgers: $\\partial_x,\\partial_t,\\partial_u,x\\partial_x+2t\\partial_t,2t\\partial_x-x\\partial_u,4tx\\partial_x+4t^2\\partial_t-(x^2+2t)\\partial_u,\\alpha(x,t)e^{-u}\\partial_u$, where $\\alpha_t=\\alpha_{xx}$\n\nWave: $v_1=\\partial_x,v_2=\\partial_y,v_3=\\partial_t,v_4=-y\\partial_x+x\\partial_y,v_5=t\\partial_x+x\\partial_t,v_6=t\\partial_y+y\\partial_t,v_7=x\\partial_x+y\\partial_y+t\\partial_t,v_8=(x^2-y^2+t^2)\\partial_x+2xy\\partial_y+2xt\\partial_t-xu\\partial_u,$$v_9=2xy\\partial_x+(y^2-x^2+t^2)\\partial_y+2yt\\partial_t-yu\\partial_u,v_{10}=2xt\\partial_x+2yt\\partial_y+(x^2+y^2+t^2)\\partial_t-tu\\partial_u,v_{11}=u\\partial_u,v_\\alpha=\\alpha(x,y,t)\\partial_u$, where $\\alpha_{tt}=\\alpha_{xx}+\\alpha_{yy}$\n\nSchrodinger:\n$\\partial_x,\\partial_y,\\partial_t,-y\\partial_x+x\\partial_y,-v\\partial_u+u\\partial_v,-2t\\partial_t-x\\partial_x-y\\partial_y+u\\partial_u+v\\partial_v$\n\nThen, it is easy to verify that the infinitesimal generators discovered by LieNLSD, as shown in Table 3, are all correct.\n\n(3) See point (2) in Methods And Evaluation Criteria section of Rebuttal to Reviewer pziR.\n\n(4) See point (2) in Methods And Evaluation Criteria section of Rebuttal to Reviewer 1vcx.\n\n**Relation To Broader Scientific Literature**\n\nThank you for your addition! We will include the following content in Related Work section:\n\nSome recent works have introduced symmetry into PINNs [2,3,4,7,8,9] or performed data augmentation based on PDE symmetries [5,6], significantly improving the performance of neural PDE solvers. We expect to combine our data-driven symmetry discovery approach with these works in the future to explore methods for PDE solving without requiring prior knowledge of symmetries.\n\n**Questions For Authors**\n\n(1) True symmetries of the wave equation are provided in point (2) in Methods And Evaluation Criteria section of the Rebuttal (calculation process see [1]). As mentioned in Related Work section (Lines 128-130, Column 2), existing works on linear symmetry discovery use Lie algebra representations to characterize symmetries. Then, according to Eqn (2), they can only discover linear generators of the form $\\phi(v)=d\\rho_X(v)x\\cdot\\nabla$. For the wave equation, $v_1,v_2,v_3$ (translation symmetry) and $v_8,v_9,v_{10}$ (special conformal symmetry) do not conform to this form, meaning these symmetries cannot, in principle, be discovered by such methods.\n\n(2) We will revise the sentence to \"Then the infinitesimal group action can be expressed in the form of $W\\Theta$. Our ultimate goal ...\".\n\n(3) We will revise the sentence to \"... with a basis $\\\\{v_1,v_2,\\dots,v_r\\\\}$, in the neighborhood of the identity element, we have ...\".\n\n(4) In Chapter 1.4 of the textbook [1], Eqn (1.46) defines the infinitesimal group action as $\\psi(v)=\\frac{d}{d\\epsilon}|_{\\epsilon=0}\\Psi(\\exp(\\epsilon v),x)$. However, $\\psi(v)$ is usually written in the form $-y\\partial_x+x\\partial_y=(-y,x)\\cdot\\nabla$ rather than (-y,x). In Chapter 1.3 of the textbook, the paragraph immediately following Eqn (1.4) also mentions that readers can regard $\\partial_x$ as \"place holders\" or a special \"basis\". To avoid confusion, we add the partial differential operator $\\nabla$ to the infinitesimal group action, as shown in Eqns (1) and (2) of our paper.\n\n(5) We will make the revision. Thank you!\n\n(6) For a 1st-order dynamic system, the PDE can be written as $F(x,u^{(n)})=f(u')-u_t=0$ (Line 238, column 1). By treating u' as the input and $u_t$ as the output, we can train a neural network to approximate f, thereby obtaining F.\n\n(7) Due to page limitations, the experimental results of the KdV equation are provided in Table 6 of Appendix I rather than in the main text.\n\n(8) As mentioned in the response to Question 6, we indirectly obtain F by training a neural network to fit f. The training details are provided in Appendix H.\n\n(9) From methods, the original paper [6] only uses the 1D KdV equation as a worked example to discuss the implementation process of data augmentation. The trigonometric interpolation they employ becomes more complex in higher-dimensional cases. From experiments, they only test on 1D evolution equations as mentioned in Section 4.1 of their paper. Moreover, their code implementation is also limited to 1D cases.\n\n**Reference**\n\n[1] Olver, Peter J. Applications of Lie groups to differential equations.\n\n[2] arxiv.org/abs/2310.17053\n\n[3] www.mdpi.com/2673-9984/5/1/13\n\n[4] arxiv.org/pdf/2410.02698\n\n[5] arxiv.org/pdf/2212.04100v1\n\n[6] proceedings.mlr.press/v162/brandstetter22a.html\n\n[7] arxiv.org/abs/2206.09299\n\n[8] arxiv.org/pdf/2502.00373\n\n[9] arxiv.org/abs/2311.04293", "claims_and_evidence": "Most of the claims made by the paper appear to be substantiated by empirical evidence, while theoretical results appear to be well explained. However, limitations of the method remain completely unexplored and it is not clear whether the results were cherry-picked.", "methods_and_evaluation_criteria": "The problems considered in this paper appear to be relevant and good examples for the probem considered. However, the settings consdiered here appear to be quite limited. To be precise, while the method here appears to be quite good, SVD and library based methods for symbolic discovery have been known to be susceptible to a multituted of issues, and particualrly the following questions need to be addressed (at least through ablations): \n\n* How sensitive is the method to noise in the observations? \n* Currently, Table 3 presents the discovered bases, but not the true bases - making it unclear if they are accurate. \n* What happens when there is a dictionary (library) misspecification? \n* What is the effect of number of samples of the trajectory on the discoverability?", "theoretical_claims": "The theoretical claims appear to intuitively be correct, but I have not looked at the exact proofs.", "experimental_designs_or_analyses": "Apart from the issues listed above, the analysis appears to be sound.", "relation_to_broader_scientific_literature": "There appear to be a few recent missed references on the usage of lie symmetries derived through the algebra in the context of neural PDE solvers, beyond augmentation. I list these below: \n\nhttps://arxiv.org/abs/2310.17053\nhttps://www.mdpi.com/2673-9984/5/1/13\nhttps://arxiv.org/pdf/2410.02698\nhttps://arxiv.org/pdf/2212.04100v1\nhttps://arxiv.org/abs/2206.09299\nhttps://arxiv.org/pdf/2502.00373\nhttps://arxiv.org/abs/2311.04293", "essential_references_not_discussed": "n/a", "other_strengths_and_weaknesses": "see above.", "other_comments_or_suggestions": "see below.", "questions_for_authors": "* Line 23 column 2 - Please provide a reference for why that is the case.\n* Line 55 column 1 - both W and \\Theta are introduced for the first time. It would be beneficialy for the paper to provide a short describption of the method at the beginnign of the paper, where these can be introduced.\n* Line 107 - this makes it seem that it is always possible to represent a group element as an exponential, but the exponential map may not be surjective.\n* Where does the notation for eqns 1 and 2 come from? This seems to be a bit different to Olver.\n* Line 227 \"can be solved *for*.\"\n* Line 236 column 2 - the mapping f is meant to be F?\n* Why is KdV not included in Table 3 if it is used for point symmetry augmentation?\n* How are the networks F trained?\n* Line 402 column 2 - you claim that LPSDA is only applicable to 1-dim cases, but that does not appear to be the case in the paper considered. Could you elaborate what you mean by this?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZFWVASGVCb", "reviewer": "Reviewer_TKxK", "summary": "The paper proposes a method for detecting non-linear symmetries in data. In a nutshell, the authors solve this by first defining a base for the Lie algebra space (which is the space of infinitesimal generators), then solve the matrix equation of which of the linear combinations of these generators align with the \"derivative of the data\". The system the authors needs to solve remains linear because they are working at the level of the infinitesimal generators.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your careful reading and valuable feedback! Below we will address each of your concerns point by point.\n\n**Theoretical Claims**\n\nTaking the discovery of linear symmetries in the Heat equation $u_t=u_{xx}$ as an example. We specify the function library as $\\Theta=[t,x,u]^T\\in R^{3\\times1}$. Then, the infinitesimal group action takes the form\n$$\nv=W\\Theta\\cdot\\nabla=\\Theta^T W_1\\partial_t+\\Theta^T W_2\\partial_x+\\Theta^T W_3\\partial_u,\n$$\nwhere $W=[W_1,W_2,W_3]\\in R^{3\\times 3}$. Our ultimate goal is to solve for W.\n\nThe algorithm first constructs $\\Theta_n$. From Equations (7) and (8), we obtain $pr^{(2)}v=\\phi^t\\frac{\\partial}{\\partial u_t}+\\phi^{xx}\\frac{\\partial}{\\partial u_{xx}}+\\dots$ (here, for simplicity, we omit irrelevant terms), where $\\phi^t=-u_tD_t\\Theta^T W_1-u_xD_t\\Theta^T W_2+D_t\\Theta^T W_3,\\phi^{xx}=-(u_tD_{xx}\\Theta^T+2u_{tx}D_x\\Theta^T)W_1-(u_xD_{xx}\\Theta^T+2u_{xx}D_x\\Theta^T)W_2+D_{xx}\\Theta^T W_3$. We rewrite it as\n$$\npr^{(2)}v=\n\\begin{bmatrix}\n-u_tD_t\\Theta^T&-u_xD_t\\Theta^T&D_t\\Theta^T\\\\\\\\\n-(u_tD_{xx}\\Theta^T+2u_{tx}D_x\\Theta^T)&-(u_xD_{xx}\\Theta^T+2u_{xx}D_x\\Theta^T)&D_{xx}\\Theta^T\n\\end{bmatrix}\nvec(W)\\cdot\\nabla=\\Theta_2vec(W)\\cdot\\nabla\n$$\nTherefore, we get $\\Theta_2$ in Equation (9). Substituting $D_t\\Theta=[1,0,u_t]^T,D_x\\Theta=[0,1,u_x]^T,D_{xx}\\Theta=[0,0,u_{xx}]^T,u_t=u_{xx}$, we have\n$$\n\\Theta_2=\n\\begin{bmatrix}\n-u_{xx}&0&-u_{xx}^2&-u_x&0&-u_xu_{xx}&1&0&u_{xx}\\\\\\\\\n0&-2u_{tx}&-(u_{xx}^2+2u_{tx}u_x)&0&-2u_{xx}&-3u_xu_{xx}&0&0&u_{xx}\n\\end{bmatrix}\n$$\nThe PDE can be expressed as $F=u_{xx}-u_t=0$, so its Jacobian matrix with respect to $u_t,u_{xx}$ is $J_F=[-1,1]$. Substituting into Equation (12), we have $[u_{xx},-2u_{tx},-2u_{tx}u_x,u_x,-2u_{xx},-2u_xu_{xx},-1,0,0]vec(W)=0$.\n\nComparing the coefficients, we obtain the basis of vec(W) as $[2,0,0,0,1,0,0,0,0]^T,[0,0,0,0,0,0,0,1,0]^T,[0,0,0,0,0,0,0,0,1]^T$. Substituting into $v=W\\Theta\\cdot\\nabla$, we get the infinitesimal generators $v_1=2t\\partial_t+x\\partial_x,v_2=x\\partial_u,v_3=u\\partial_u$. These correspond to the linear generators $v_2,v_4,v_5$ shown in Table 6 (in Appendix I).\n\n**Relation To Broader Scientific Literature**\n\nThank you for your addition! We will include the following content in the Related Work section.\n\nA series of recent works have made outstanding contributions to extending diffusion models from Euclidean space to Lie groups [1,2,3]. Since our method can directly discover the infinitesimal generators of Lie groups from data, we look forward to future integration with these works, eliminating the need for prior knowledge of the group to guide their processes.\n\n**Questions For Authors**\n\nBoth of these scenarios involve discovering linear symmetries from static systems. By setting the prolongation order n=0 and specifying the function library $\\Theta(x)=x$ as linear terms, LieNLSD can cover them (see Appendix F for details). The core issue is how to define F in Algorithm 1 and estimate $J_F$, which we will discuss in detail below.\n\n(1) For a two-dimensional manifold in $R^3$, its implicit equation can be written as $F(x)=0,x\\in R^3$, but it may not have an explicit equation (e.g., S2). Therefore, we use the $k$-NN to estimate the Jacobian matrix $J_F$. Specifically, we fit a tangent plane $J_F(x-x_0)=0$ to the manifold using the k neighboring points around the given point $x_0$, and assign a Gaussian kernel weight $w_i=\\exp(-\\\\|x_i-x_0\\\\|^2/2)$ to each point to reduce the influence of more distant points.\n\nIn practice, we set the number of neighboring points to k=5. The experimental results are shown below. We successfully discover the generators of SO(3).\n\nSingular value: $[57.4,36.9,36.6,36.5,36.2,36.1,5.1,4.6,4.2]$\n\nThe Lie algebra basis (W) corresponding to the 3 smallest singular values:\n$$\n\\begin{bmatrix}\n7.58e^{-4}&5.58e^{-4}&-0.706\\\\\\\\\n1.10e^{-3}&9.63e^{-4}&8.15\\times 10^{-4}\\\\\\\\\n0.708&-1.07e^{-4}&-7.22e^{-4}\n\\end{bmatrix},\n\\begin{bmatrix}\n-1.28e^{-3}&0.706&-1.89e^{-5}\\\\\\\\\n-0.709&5.09e^{-4}&-1.71e^{-3}\\\\\\\\\n5.31e^{-4}&2.08e^{-3}&3.47e^{-4}\n\\end{bmatrix},\n\\begin{bmatrix}\n-1.18e^{-3}&2.27e^{-4}&-5.77e^{-5}\\\\\\\\\n-3.56e^{-3}&1.17e^{-4}&0.704\\\\\\\\\n-9.67e^{-4}&-0.710&1.21e^{-3}\n\\end{bmatrix}\n$$\n(2) In this case, the group transformation acts on the coordinates $x\\in R^2$, so we need to compute the derivative of the classifier with respect to the coordinates $D_xF$. We train a CNN on MNIST, which takes grayscale images I(x) as input. Automatic differentiation yields $D_IF$. We then estimate  $D_xI$ using central differences on the image and apply the chain rule to obtain $D_xF=D_IF\\cdot D_xI$. The experimental results are shown below, which demonstrate that we successfully discover the SO(2) symmetry.\n\nSingular value: $[48.8,46.8,41.2,22.0]$\n\nThe Lie algebra basis (W) corresponding to the smallest singular value:\n$$\n\\begin{bmatrix}\n-2.90e^{-3}&-0.680\\\\\\\\\n0.733&-1.05e^{-2}\n\\end{bmatrix}\n$$\n**Reference**\n\n[1] arxiv.org/abs/2502.02513\n\n[2] arxiv.org/abs/2405.16381\n\n[3] arxiv.org/abs/2205.13699", "claims_and_evidence": "The authors claim that LieNLSD is, to the best of their knowledge, \"... the first method capable of determining the number of infinitesimal generators with nonlinear terms and their explicit expressions.\"\n\nI am not familiar enough with the literature to determine whether this is true or not.", "methods_and_evaluation_criteria": "The experiments the authors propose are interesting, but I would be interested to understand if the method can work in other cases. See the Question section below.", "theoretical_claims": "I think the theoretical claims are well-defined and, to the degree I checked the math, correct. \n\nI would be interested in seeing, however, a complete (toy?) example, where all the steps can be followed \"by hand\". This is very helpful when discussing implementation of theoretical constructs. I see that the authors have some parts of it in the appendix (SO(2)). Maybe for that case the authors can provide a whole setting and showing how their framework applies explicitly?", "experimental_designs_or_analyses": "The experiments are interesting, but I am not sure how standard they are in the field. See the Question section below.", "relation_to_broader_scientific_literature": "If my interpretation of the claim of the paper is correct, namely \"a method for detecting non-linear symmetries in data\", I see very interesting potential application of the method applied to generative modelling. There is a recent body of work that designs score-based generative modeling using Lie algebra (also these works and other could be included in the related work section):\n\n- Bertolini et al, \"Generative Modeling on Lie Groups via Euclidean Generalized Score Matching\"\n- Zhu et al, \"Trivialized Momentum Facilitates Diffusion Generative Modeling on Lie Groups\"\n- Kim et al, \"Maximum likelihood training of implicit nonlinear diffusion model\"", "essential_references_not_discussed": "See above", "other_strengths_and_weaknesses": "See below.", "other_comments_or_suggestions": "See below.", "questions_for_authors": "My main question is related to the applicability domain of the methods. The experiments look very specifically designed to me. So, to make things concrete, could you apply your method to the following scenarios:\n- SO(3) in R^3: I take data on S2 but presented in Cartesian coordinates (using the embedding S^2 \\subset R^3), Can you discover the (local) generator of  SO(3) just from the data?\n- if you take the rotated MNIST dataset, where each MNIST digit is randomly rotated (SO(2) symmetry). Can you discover SO(2) just from the data?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper proposes a method for detecting non-linear symmetries in data. The authors solve this by first defining a base for the Lie algebra space (which is the space of infinitesimal generators), then solve the matrix equation of which of the linear combinations of these generators align with the "derivative of the data". The system the authors needs to solve remains linear because they are working at the level of the infinitesimal generators. The discovered symmetry could be used to augment the data for neural PDE solvers. It could also be used to prune the redundant generators that are discovered by approaches that require predefined number of generators like LieGAN. The methods are demonstrated on important PDEs.
Decision: Accept (poster)