Symmetric Space Learning for Combinatorial Generalization
Authors: Jaehyoung Jeong
Hee-Jun Jung
Kangil Kim
Venue: ICLR 2026 Poster (2026)
Keywords: Generative Model
Generalization
Combinatorial Generalization
Machine Learning
Manifold Learning
Representation Learning
Primary area: generative models

Abstract: Combinatorial generalization (CG)—generalizing to unseen combinations of known semantic factors—remains a grand challenge in machine learning. 
While symmetry-based methods are promising, they learn from observed data and thus fail at what we term $\textbf{symmetry generalization}$: extending learned symmetries to novel data. 
We tackle this by proposing a novel framework that endows the latent space with the structure of a $\textbf{symmetric space}$, a class of manifolds whose geometric properties provide a principled way to extend these symmetries. 
Our method operates in two steps: first, it imposes this structure by learning the underlying algebraic properties via the $\textbf{Cartan decomposition}$ of a learnable Lie algebra. 
Second, it uses $\textbf{geodesic symmetry}$ as a powerful self-supervisory signal to ensure this learned structure extrapolates from observed samples to unseen ones. 
A detailed analysis on a synthetic dataset validates our geometric claims, and experiments on standard CG benchmarks show our method significantly outperforms existing approaches.

Published as a conference paper at ICLR 2026
S SYMMETRIC SPACE LEARNING FOR COMBINATORIAL GENERALIZATION
Jachyoung Jeong, Hee-Jun Jung, Kangil Kim Department of Al Convergence
Gwangju Institute of Science and Technology
(jaehyoung98, Junghee jun 8 @mm. gist. kr, kangil kim. 01 01@gmaill com
ABSTRACT
to unseen combinations known semantic cestors-reminng fundamental challenge in machine learning. While symmetry-based methods are promising, they learn from observed data and thus fail at what we term symmetry generalization: extending learned symmetries to novel data. We address this by proposing novel framework that endows the latent space with the structure of symmetric space. This class of manifolds provides principled geometric foundation for extending learned symmetries. Our method operates in two steps: first, imposes this structure by learning the un- derlying algebraic properties via the Cartan decomposition of learnable Lie algebra. Second, it uses geodesic symmetry as a powerful self-supervisory signal to ensure this learned structure extrapolates from observed samples to unseen ones. A detailed analysis on a synthetic dataset validates our geometric claims, and ex- periments standard CG benchmarks show method significantly outperforms existing approaches.
Combinatorial generalization
INTRODUCTION
Generalizing model to unobserved combinations of semantic factors is a critical challenge in achieving human-like generaliza- tion (Fodor & Pylyshyn 1988). In representation learning, this problem is known as Combinatorial Generalization (CG), where the goal is capture the semantic data structure within latent repre- sentations (Vankov & Bowers 2019). Despite its importance, most existing approaches often fail achieve this effectively (Schott al. 2021).
Symmetry, which captures transformations that leave an object's identity invariant, is cornerstone for addressing CG. For instance, Higgins et al. (2022) demonstrated that symmetry learning effec- tively captures structural information, while Hwang et al. (2023) showed direct benefits for CG. Nonetheless, a critical limitation persists: these methods learn symmetries exclusively from observed data, hindering their ability to generalize these symmetrics to unseen combinations.
Figure 1: Since trained group actions should be closed on the observed region, they can- not affect the unseen region.
Exploiting geometric information offers a promising avenue for such generalization. Geometric Deep Learning has emerged powerful paradigm for understanding the dynamics of representation learning (Bengio et al.) 2013 Bronstein et al. 2021). Specifically, the geometry of latent manifolds can provide deep insights into semantie factors and data relationships (Shao et al. 2018 Choi et al.) 2021). However, integrating these geometric insights with symmetry learning has been limited, often relying on restrictive assumptions, such predefined Lie groups fully observed data, leaving critical gap for CG.
"Corresponding author.

---

Published as a conference paper at ICLR 2026
We summarize our primary contributions to the field of Combinatorial Generalization (CG) follows:
1. Problem Formulation: We formally identify the core limitation of existing symmetry- based methods by defining the Symmetry Generalization challenge: extending learned symmetries from partially observed data to unobserved regions.
2. Novel Framework: We propose CartanFM, a novel framework that embeds the latent representation of transformations with the rigorous geometric structure of Symmetric Space, offering principled solution to the challenge.
3. Technical Mechanisms: We introduce two novel, complementary losses to realize this structure:
The Cartan Loss (LCartan). which enforces the local algebraic structure via Cartan Decomposition on learnable Lie algebra.
The Geodesic Symmetry Consistency (GSC) Loss (LGsc), a supf-suuery signal that leverages geodesic symmetries to explicitly extrapolate the learned structure to unseen regions of the data manifold.
BACKGROUND
manifold is topological space that is locally Euclidean. This concept is foundational machine learning, motivated by the manifold hypothesis, which posits that high-dimensions data often lie on or near a low-dimensional manifold embedded within the ambient space (Narayanan & Mitter 2010).
Many such manifolds exhibit rich symmetries that can be formally described using the theory of groups. A Lie group is a particularly suitable framework, as it differentiable manifold that also group with smooth operations. A canonical example is the general linear group GL(n), the set of all invertible n X n matrices.
The interaction between group and manifold is formalized by the notion of a homogeneous spuce, which serves as a cornersoone of our method.
Definition 2.1 (Homogeneous Space) Riemannian manifold M is a homogeneous space if there exists a Lie group G that acts transitively on M.
In homogeneous space, any point can be mapped any other point via an element of G, implying that the manifold is geometrically uniform. This property allows for a coordinate-fre analysis the manifold's structure through its symmetries (do Carmo 1992).
While homogeneous spaces offer a general framework, our work focuses on a more structured subclass known symmetric spaces. These spaces possess additional symmetries, such point reflections, and can be defined from several equivalent perspectives (Helgason 200T).
Definition 2.2 (Symmetric Space via Geodesic Symmetry) Let M be a Riemannian manifold. The geodesic symmetry Sp point p E MM is map that reverses geodesics passing through Specifically, for a geodesic (t) with (0) = p. it satisfies sp(r((t) >-t)). The manifold M is symmetric space if. for every point p E M, this map Sp can be extended to global isometry.
An equivalent algebraic characterization defines symmetric space as special type of coset space.
Definition 2.3 (Symmetric Space via Lie Groups) Let G be connected Lie group and K be closed subgr subgroup. The coset space G/K K is a symmetric space if there exists an involutive automor- phism G G such that Gg C K G", where Go (g Gla(g) g} and Gg is the identity component G G
This group-level algebraic structure induces a corresponding structure at the Lie algebra level, giving rise to the third equivalent definition via the Cartan decomposition.
Definition 2.4 (Symmetric Space via Cartan Decomposition) Let g be the Lie algebra of G. A symmetric space structure on G/K corresponds to a Cartan decomposition of its Lie algebra, g t p. where t is the Lie algebra of the subgroup K. This decomposition is induced by the
2

---

Published as a conference paper at ICLR 2026
differential of a the identity, involutive automorphism 0 on g. Here, and p the eigenspaces of for eigenvalues +1 and -1, respectivv and they satisfy the Lie bracket relations:
[t,t] S
[t.p] P. and [p.p] S..
(1)
The pair (g. is called a symmetrie pair.
METHOD
3.1 MOTIVATION: SYMMETRY GENERALIZATION
Limitations of Existing Symmetry Learning Symmetry-basee machine learning methods aim to capture data variations by learning underlying symmetrics, often framed through equivariant group actions. While effective, these methods face critical limitation when symmetries are learned exclusively from subset of the data. For instance, approaches like Hwang et al.] (2023) learn group actions from observed training samples generate novel data. However, the learned symmetries confined to the training data, restricting generalization to unseen samples. We term this challenge symmetry generalization, which formalize follows:
The Symmetry Generalization Challenge
Let X be the complete data space and Xois C X be the observed subset. Let Gots be a symmetry group learned exclusively from actions within Xobs- The generalization failure occurs because for an unseen sample Tnew E X Xobs. there is generally no transformation g E Gobs and sample Tobs E Xohs such that Toos Knew-
This limitation is not inherent in group theory itself, but in methods that infer symmetriee from incomplete data. Overcoming this gap requires framework capable of extending learned symmetries beyond the observed samples, motivating our use of geometric structures.
Exploiting Geometric Information From a geometric perspective, the data manifold encodes the semantic structure of both observed and unobserved samples. A promising solution to the symmetry generalization challenge is consider symmetries that act on the entire data manifold, not just on the training set. Imposing the structure of a homogeneous space provides principled way to extend locally observed symmetries, as its transitive property guarantees a path from any observed point to any unobserved point. However, this is impractical for general problems because the global group actions required for such navigation are unknown.
To circumvent this, we propose enforcing the properties of symmetric space- special class of homogeneous spaces with additional geometric structure. symmetric space is endowed with geodesic symmetry (a reflection) every point (Definition 2.2 This reflection provides concrete, locally defined operation for generalization that does not require knowledge of the whole global group. Specifically, by reflecting observed point through another chosen local origin, we can generate a plausible sample in an unseen region. By training our model to be consistent with this reflection property, we directly extend the learned symmetry structure beyond the training data.
3.2 SYMMETRIC SPACE LEARNING VIA LIE ALGEBRA
To model the data manifold M as symmetric space G/K, K, face two primary challenges. First, the algebraic structure, defined by the symmetry group G and stabilizer K, is unknown for general datasets. Second, any structure learned solely from observed data must be able to generalize unseen regions. Our method addresses these two challenges with two corresponding components for learning and generalizing the symmetric space structure.
Learning the Symmetric Pair via Cartan Loss We begin by identifying the latent space of our feature encoder with the tangent space T.M at a chosen origin o. In a symmetric space, this tangent space corresponds to the subspace p from the Cartan decomposition each data point æ, the encoder thus outputs a latent vector 2g E Crucially, our model also learns the bases for the subspaces t and p themselves, which parameterized sets of matrices. To
e (Definition 2.4 For
3

---

Published as a conference paper at ICLR 2026
Inwert
Cartan
Summutry
Ck.
B+
B
X1
+
GSC
Bp
(-(),))))
(R,)
CFM
(0.1))
Flow Matching
(xx P.L)
N (0,1)
Flow Matching
vxxx
P,t))
One step Generation
(a) CFM Loss and Cartan Loss
(b) Geodesic Symmetry Consistency Loss
cartan-
Cartan
Generating Symmetric
-P
P
Space via Cartan Decomposition
Xo
LCFM
:Mtchhing vector fields for training data :Ettending G/ to unseen region
G/K
LGSC
X1
Lasc
Unseen region
Ground Doc Maniail
X1
CCF
GGodesic
TTagent vector
(c) Geodesic Symmetry by Lie algebra negation.
Figure 2: Overview of proposed CartanFM framework and its underlying geometric principle. (a) Main Training Path: The model learns to reconstruct data conditioned on a Lie algebra element P with CFM Loss LCFM. The learnable bases that constitute P are simultaneously regularized by the Cartan Loss (CCaman) to form a valid algebraic structure. (b) Generalization Path: A self-supervisory loop leverages Geodesic Symmetry Consistency (Cusc) train the encoder. This ensures that synthetically generated unseen sample, when encoded, is consistent with the negated representation of its observed counterpart. (c) Geometric Intuition: In the learned symmetric space, geodesic (P - P) in the tangent space, providing principled bridge between the observed and unseen data regions.
symmetry corresponds to simple negation
ensure these learnable bases form valid symmetric pair (g, t), we introduce the Cartan Loss, which imposes the required Lie bracket relations:
LCartan == L Ct t) [(te, p) C(pp.p.
(2)
where .(..) measures the projection error of the Liee bracket's output onto the target subspace.
Generalizing Learned Structures via Geodesic Symmetry Consistency The algebraic structure learned via the Cartan Loss only reflects the observed data. To promote generalization, we leverage key property of symmetric spaces: the geodesic symmetry at the origin corresponds to negation in the tangent space p. Specifically, if data point 2ı is represented by the tangent vector p E p. its so(r1) is defined by -p We operationalize this geometric prior
symmetric counterpart through cycle-consistency objective we term Geodesic Symmetry Consistency (GSC), with the following procedure:
1. Encode an observed sample Tobs into its latent vector; Pobs = Encoder(<rota) E E.p.
2. Generate an unseen "candidate" sample by decoding the negated vector: Feand Decoder(-Potan
3. Encode candidate sample back into the latent space: Poand = Encoder(sranna).
4. Optimize the GSC Loss to enforce consistency between the re-encoded and the negated original vectors:
LGsC Enres TeetsnXou Xotn [llieand + Pota 12]
4

---

Published as a conference paper at ICLR 2026
This objective encourages the model to map geometrically symmetric points in the data space to algebraically opposite points in the latent space, thereby extending the learned structure to unseen regions.
3.3 IMPLEMENTATION DETAILS
Overview This section outlines the practical implementation of our symmetric space learning framework. We integrate our proposed architectural components and loss functions into state-of- the-art generative model by conditioning its underlying vector field. As illustrated in Figure our model consists of three core components:
A Lie Algebra Encoder (d): Maps input data point to the coefficients (ct, cp) for the learnable Lic algebra bases.
Learnable Lie Algebra Bases (Be, Bp): set of matrices that form the bases for the subspaces and respectively. The Cartan Loss regularizes these.
A Conditional Flow Matching Model: Generates a vector field v(a;, 1, P) that condi- tioned the tangent vector P p, which is constructed from the encoder's output.
These components are trained jointly. The generative capability is learned via the standard Conditional Flow Matching loss (Ccrm). At the same time, the geometric structure is enforced by our two proposed losses, the Cartan Loss (Eq. 2 and the GSC Loss (Eq. 3p. The following sections provide detailed explanations of each component.
Backbone Generative Model We build our method upon Flow Matching (Lipman et al. 2022 recent class of generative models that learn a probability flow from a noise distribution to the data distribution. This flow is defined by neurally parameterized vector field v( f, c) and is sampled by solving an ordinary differential equation (ODE). Inspired by DiffAE (Preechakul et al.) (022). which adapts diffusion models for representation learning, employ an autoencoder architecture. A dedicated encoder network learns mapping from a data sample x to a Lie algebra element P E which serves as the condition for the vector field, i.e., v(x, P).
Cartan Loss Our encoder consists of two main parts: standard VAE-style encoder and set of learnable basis matrices. For given input, the encoder outputs coefficient vectors Cr Tummm and Cp E R These coefficients form linear combinations of the learnable bases {K,} and {P,} to produce Lie algebra elements
Σ; Ce,:K E f and = E, Cp.j E p. While directly conditions the generative process, the bases for both subspaces must satisfy the Lie bracket relations from Eq. _
For computational efficiency, we apply these constraints directly to the basis elements. This is enforced via the Cartan Loss, which encourages orthogonality between the relevant subspaces:
Ccartan
Σ (.G0
[K, K2]
P.
Σ R.
K, Py] Kil
[[ -
(4)
Geodesic Symmetry Consistency To implement the GSC objective described in Eq. efficiently. we bypass the computationally expensive complete decoding process of the Flow Matching model by using one-step approximation. The core idea is to generate synthetic sample corresponding the negated latent vector and ensure re-encoded representation aligns with - P. We approximate this by taking single step from the original data point zg along the vector field conditioned on -P. The resulting GSC Loss is:
Lasc E [ncoderr(ro + (1- miin) v(x0, t 0, -P;0)) P11
(5)
where miin is hyperparameter defined in Lipman al, (2022). This objective pushes the encoder to be consistent with the geodesic symmetry defined in the latent space.
Full Objective Function Our final training objective is weighted sum of the generative loss and our architectural losses:
L = LCFM + 3 LKI. + ACartan LCartan + Aaso Lasc + e Lhasis
(6)
where:
5

---

Published as a conference paper at ICLR 2026
CCFM is the conditional flow matching loss for the generative task (Lipman al. 2022)
CKL. is the standard Kullback-Leibler divergence from the VAE-style encoder.
CCartan and Lasc are proposed losses for enforcing the symmetric space structure and encouraging generalization, respectively.
Lhasis = E; 1/Il 1/ h is regularization term on the learnable basis elements {K,} and {Py} to prevent them from collapsing to zero.
3, A.arrtan ÀGSO, and e are hyperparameters that balance each term's contribution.
4
RELATED WORK
Symmetry and group theory are cornerstones of modern representation learning (Higgins et al. 2022). particularly for achieving disentangled repre- sentations (Higgins et al. 2018 Various studies have demonstrated that enforcing group structures, such as orthogonality, enhances disentanglement (Cha & Thiyagalingam, 2023) Yang et al., 2022). This principle extends to Lie groups, which have been used to develop equivariant networks like G-CNNs on homogeneous spaces (Cohen et al. 2019) and their more general Lie algebra-based successors, L-CNNs (Dehmamy | 2021).. Other works also leverage Lie groups to improve disentanglement (Zhu et al. 2021 Tonnaer et al. 2020) or to decouple object representations from group actions (Keurti et al. 2023). ) However, common limitation of these "top-down" approaches is their reliance on predefined group structures, which restricts their ability to generalize from partially observed symmetries. In contrast, framework learns the underlying algebraic structure of symmetric space from data, enabling generalization to unobserved symmetry transformations.
Representation Learning
Geometry in Generative Models A geometric perspective offers powerful tools for improving generative models. Manifold learning, for instance, has been shown to enhance disentanglement in latent spaces (Fumero et al. 2021, Falorsi et al. 2018) and to model hierarchical data structures using hyperbolic geometry (Mathieu et al.) 2019). for generalization tasks, such as improving Furthermore, geometric priors have proven effective robustness (Ng et al.) 2020; Vural & Guillemot 2016). While these studies highlight the benefits of incorporating geometry, takes this a step further. We do not just learn an arbitrary manifold; we impose the rich structure of a symmetric space, which provides principled framework for solving the symmetry generalization problem.
work
Symmetry Discovery Finding the intrinsic symmetry structure is an important task across various domains, particularly in scientific data analysis. The Symmetry Discovery task aims autonomously identify the latent structure of invariance and equivariance, which is explicitly unknown a priori. One efficient approach leverages Generative Adversarial Networks (GANs)
find Lie algebra
bases that describe data distributions under various transformations (Yang et al. 2023b a). Another promising approach utilizes infinitesimal generators, which are typically found using tools like Ordinary Differential Equations (ODEs) or self-supervised learning techniques (Shaw et al. 2024 Ko al., 2024. Allingham al.) 22024] Hu et al.) (205)). While these methods efficiently discover explicit symmetry structures using Lie algebras or vector fields, our approach focuses on how the model can extend learned symmetries rather than merely representing them. Moreover, given our crucial problem setting of partial observation (c.g., g., an incomplete manifold), there is guarantee that such discovery methods can work properly without an explicit, global inductive bias. Thus, we propose our method to directly solve the symmetry generalization problem.
Combinatorial Generalization Combinatorial generalization (CG) remains a critical challenge in machine learning (Vankov & Bowers 2019). Early work suggested that disentangled representations could be a solution (Montero et al. 22220) but subsequent studies revealed their frequent failure to generalize to unseen combinations of factors (Montero et al. (022) More recent approaches have focused on identifying sufficient conditions for CG CGGWWal (Wiedemer et al. 2023) or designing architectures
that explicitly model group actions (Hwang et al., 2023). Our work contributes to this line of research by proposing novel synthesis: integrate latent geometry and group actions through the framework of symmetric spaces. This approach directly tackles the limitations prior methods by addressing the fundamental problem of symmetry generalization.

---

Published as a conference paper at ICLR 2026
332
6.
480
431
-67
-079
-100
118805659002p am Lo 1994 00:0-2a0nt000
-3.3
16-5
www.ae........................ C2 ans 140 10.5 2w
(a) Training Data (270' arc)
(b) Test Data (Ground Truth)
(c) Cartan FM GSC
140
062
042
-027
-1.9
ars Lo0 L00,
an 1.0 1020000000
1:0
(d) Vanilla VAE
(e) Cartan VAE
Cartan FM w/ GSC
Figure 3: Manifold Reconstruction Results on the sphere point cloud. (a) The distribution of
training data, consisting of partial 270 arc. (b) The ground truth distribution for the unseen 90º are test data. (c-f) Reconstruction results for the unseen test data from each model. Our full method (c) is the only one that successfully generalizes the unseen region and reconstructs the complete sphere.
EXPERIMENTS
5.1 COMPREHENSIVE ANALYSIS VIA 3D SPHERE SHAPE MANIFOLD RECONSTRUCTION
Experimental Setup As visualized in Figure 3 we partition a sphere point cloud to cre- ate a challenging generalization task. The training set consists of an incomplete 270-
degree (Fig. Ba), while the test set is the entirely unobserved 90-degree (Fig. 3b). We compare four models based on the standard PointNet architecture (Qi et al. 2017):
Vanilla VAE: A standard VAE baseline.
VAE + Cartan module: A VAE equipped with our Cartan Loss, but without the generative flow model.
Table I: Sphere Manifold Reconstruc-
tion Error (Chamfer distance)
Flow Matching + Cartan Module: Our full gen- crative model, but without the GSC Loss, serving key ablation baseline.
Method
Error (L)
Vanilla VAE
0.0601
Cartan VAE
0.5763
Cartan FM
0.0068
Flow Matching + Cartan Module GSC Loss: Our full proposed model.
Cartan FM GSC
0.0061
We evaluate reconstruction quality using the Chamfer dis- tance (Fan et al. 2016). a standard metric for comparing point clouds. Full implementation details available in Appendix C.I
The quantitative results in Table [show a clear progression across the model configurations. The Vanilla VAE baseline struggles reconstruct the unseen test data, only managing to generate points near the observed boundary. The failure of the VAE + Cartan module underscores that the algebraic structure requires a compatible generative process. This result validates the design choice of using Flow Matching, as it operates the tangent vector level and is thus ideally suited to realize the transformations defined by our learned Lie algebra. Accordingly, the Flow Matching + Cartan Module model achieves significant performance improvement. Finally, adding the GSC Loss to create our full model, the Flow Matching + Cartan Module + GSC Loss, yields the best reconstruction performance by large margin. This quantitative superiority is mirrored in the qualitative results (Figure 33. which visually demonstrate that our full model is uniquely capable of reconstructing the complete spherical manifold.
Results
7

---

Published as a conference paper at ICLR 2026
Table 2: Mean Squared Error([) in Combinatorial Generalization, Bold indicates the best perfor- mance.
R2E
R2R
Symmetry
Model
Casel
Case2
Case3
Casel
Case2
Case3
VAE
7.25
8.08
14.96
28.81
34.64
28.90
3-VAE (3
12.29
13.91
24.23
31.02
35.74
60.31
dSprites
3-VAE (3 4)
22.54
20.13
31.18
32.38
40.80
61.43
3-VAE (3 8)
31.02
27.83
39.65
39.76
55.30
67.03
GAGA
17.17
17.56
25.69
28.07
36.96
44.71
CLGVAE
72.40
76.49
133.15
132.83
218.15
236.28
MAGANet
13.30
10.63
19.49
115.46
108.96
23.35
CartanFM (Ours)
2.21
1.10
2.49
7.02
9.57
5.87
VAE
10.93
13.82
12,40
21.07
37.65
13.65
3D Shapes
8-VAE (3
12.70
17.34
18.35
33.82
41.42
17.71
3-VAE (8 4)
22.17
22.21
22.81
46.05
49.06
22.45
3-VAE (3 8)
21.11
48.52
46.55
99.73
63.85
44.57
CLGVAE
15.54
71.13
86.60
75.73
38.63
15.46
MAGANet
16.79
16.66
19.60
23.69
37.79
18.87
CartanFM (ours)
5.20
6.96
5.99
8.12
33.87
6.68
VAE
7,80
12.69
6,51
6.29
6.91
9.35
3-VAE (3
20.03
72.03
7.49
15.68
20.40
13.02
MP13D
x
3-VAE (3
30.95
73.41
8.35
17.05
20.61
13.36
3-VAE (3
33.85
73.95
10.02
18.35
22.87
15.22
MAGANet
10.97
21.56
6.56
8.12
7.71
8.42
CartanFM (ours)
1.18
1.99
0.51
0.51
0.76
0.50
5.2 PERFORMANCE ON COMBINATORIAL GENERALIZATION BENCHMARKS
Datasets and Protocol We evaluate our method on two standard benchmarks for combinatorial generalization, dSprites (Matthey al. 2017) and 3D Shapes (Burgess & Kim 2018). as well as a more complex benchmark, MPI3D Gondal et al. 2019). Following the protocol Montero et al. (2020), we create training and test splits for each dataset under two challenging settings: Recombination ter-tt----eeee to-Elements (R2E) and Recombination toe-ttt-RReeeee (R2R). To ensure robust evaluation and minimize bias from any single data split, we generate and test three distinct of excluded factor combinations for both R2E and R2R settings. Further details are in Appendix
Baselines and Training Details We compare method against VAE-based basclines from two categories: (1) models that do not explicitly model symmetries, VAE (Kingma & Welling 2013 and 3-VAE (Higgins al., 2016), Geometry Aware Generative Autoencoder (GAGA) (Sun al. 2025
and (2) symmetry-aware models, Commutative Lie Group VAE (CLGVAE) (Zhu et al. 2021 and MAGANet (Hwang et al. 2023). All models were trained for 100 epochs using an Adam optimizer with a learning rate of 5e-4. For 3-VAE, we report the best performance for each 3 E {2, 4, 8}. For MAGANet, we conducted grid search determine the optimal hyperparameters, as were unable to reproduce the reported performance from the original paper. To ensure a fair comparison, all models were trained and evaluated using the Mean Squared Error (MSE) loss.
presents the quantitative MSE results for all tasks. Our method, CartanFM, demonstrates superior performance across all three datasets. On dSprites, CartanFM consistently outperforms all baselines in both R2E and R2R settings. The improvement is particularly pronounced in the more challenging R2R setting. On 3D Shapes, which contains more complex factors, CartanFM again establishes a clear and significant advantage, outperforming all baselines across every case. On MPI3D, the most complex benchmark, CartanFM achieves state-of-the the-art results by large margin in all settings. This substantial performance gain larger, more varied dataset suggests that our method effectively leverages more data to learn the underlying symmetric space structure better.
Quantitative Analysis
8

---

Published as a conference paper at ICLR 2026
Case1
Case2
Case3
Case1
Case2
Case3
Case1
Case2
Case3
Ground-
Ground-
Ground
Ours
Ours
MAGANet
MAGANet
MAGANet
VAE
WAE
VAE
PVAEIB-2)
PVAE(P-2)
PVAE(B-2)
(VAE(=-)
(VAE(p=4)
PVAE(
DVAE()=B)
(VAE(p=a)
(VAEO)=0)
(a) dSprites
(b) 3D Shapes
(c) MPI3D
Figure 4: Generated Images in Recombination to--ange (R2R) Setting
Table 3: Ablation study of our core components. Bold indicates the best performance. In the 'Components' columns, denotes the inclusion and denotes the exclusion of the corresponding loss term.
Components
R2E
R2R
CCartan
LGSC
Casel
Case2
Case3
Case
Case2
Case3
dSittes
1.87
1.89
29.06
23.70
10.40
1.64
21.50
57.38
18.52
40.34
V
1.31
1.09
2.94
13.11
47.43
9.23
V
2.21
1.10
2.49
7.02
9.57
5.87
Shapes
x
9.70
9.25
9.20
20.12
33.09
8.97
6.00
9.32
12.50
9.42
32.61
10.96
5.96
8.30
8.85
8.96
33.16
12.11
33.87
6.68
Qualitative Analysis Figure 4a Figure 4b and [eggure/c/shoy the generated images for the R2R setting across different datasets and models. For VAEs, the generated outputs often appear blurry or distorted blobs, particularly at higher values of B. By contrast, our method effectively captures critical factors, such as distinct object shapes, where baseline models struggle to generalize unseen cases. For example, shown in Fig. 4a our method successfully generates heart shapes, whereas other methods produce squares, ellipses, or amorphous blobs. Similarly, as illustrated in Fig. 4bb VAEs generate cylinders instead of cubes, indicating failure capture the underlying factors of variation. Furthermore, as shown in Fig. 4c our method captures the object's ss shape, color, and background more accurately than other methods. Our approach demonstrates strong generalization capabilities, effectively generating unseen combinations while preserving geometric and semantic fidelity.
Ablation Study We conduct an ablation study to analyze the individual contributions of our two main components: the Cartan Loss (CCurtan) for inducing the symmetric space structure, and the Geodesic Symmetry Consistency (GSC) Loss for extending it. We evaluate four model configurations: the Flow Matching backbone alone, the backbone with each of our two losses applied individually, and our full model combining both.

---

Published as a conference paper at ICLR 2026
As summarized in Table the backbone model without our components struggles in most cases. Adding either the Cartan Loss or the GSC Loss individually yields significant performance gains, demonstrating that both are effective for improving generalization. The GSC Loss alone provides particularly substantial boost, highlighting the importance of extending the learned structure. Our full model, which combines both losses, achieves the hest or highly competitive performance in nearly all scenarios. This result confirms that the two components are complementary and that their synergy is crucial for the robust performance of CartanFM.
This study addressed the limitations of trained symmetries in generalizing to unseen data for combi- natorial generalization. We proposed a novel method for learning symmetric space structure on the data manifold and extending it to unseen data via Lie algebra properties and geodesic symmetry consistency, which facilitates the generalization of trained symmetries. An in-depth analysis of synthetic sphere manifold dataset validates our geometric claims and the effectiveness of our approach. Furthermore, experiments on widely used benchmarks, including dSprites, 3D Shapes, and MPI3D, corroborated that our method significantly outperforms existing approaches. Our study is the first to establish the utility of integrating manifold and symmetry learning to enhance combinatorial generalization. This contribution opens up promising directions for future research, including the exploration of diverse sampling strategies tailored to specific data characteristics and the extension of the approach to broader range of generalization tasks beyond combinatorial generalization.
6 CONCLUSION
This study addressed the limitations of trained symmetries in generalizing to unseen data for combi- natorial generalization. We proposed novel method for learning symmetric space structure the data manifold and extending it to unseen data via Lie algebra properties and geodesic symmetry consistency, which facilitates the generalization of trained symmetries. An in-depth analysis of a synthetic sphere manifold dataset validates our geometric claims and the effectiveness of our approach. Furthermore, experiments on widely used benchmarks, including dSprites, 3D Shapes, and MPI3D, corroborated that our method significantly outperforms existing approaches. Our study is the first establish the utility of integrating manifold and symmetry learning to enhance combinatorial generalization. This contribution opens up promising directions for future research, including the exploration of diverse sampling strategies tailored specific data characteristics and the extension of the approach to a broader range of generalization tasks beyond combinatorial generalization.
ACKNOWLEDGMENTS
This research was supported by Culture, Sports and Tourism R&D Program through the Korea Creative Content Agency grant funded by the Ministry of Culture, Sports and Tourism in 2022 (Project Name: Development of service robot and contents supporting children's reading activities based on artificial intelligence, Project Number: R2022060001, Contribution Rate: 33%). This work was supported by Institute of Information & communications Technology Planning & Evaluation (IIT)) grant funded by the Korea government (MSIT) (No. 2019-0-01842, Artificial Intelligence Graduate School Program (GIST), Contribution Rate: 10%). This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2022R1 1 A2C201 2054, Development of AI for Canonicalized Expression of Trained Hypotheses by Resolving Ambiguity in Various Relation Levels of Representation Learning, Contribution Rate: 56%).
REFERENCES
Allingham, Bruno Mlodozeniec, Padhy, Javier Antor'an, David Krueger, Richard E. Turner, Eric T. Nalisnick, and Jos'e Miguel Hern andez-Lobato. A generative model of symmetry transfor- mations. In Neural Information Processing Systems, 2024. doi: 104485500axXx..2433.11444
Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 1788)1179-1188 2013.
Michael M. Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković, Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges, May 2021.
10

---

Published as a conference paper at ICLR 2026
Chris Burgess and Hyunjik Kim. 3d shapes dataset. http:://gihibkooommmeeppmi/aaaas 2018.
Christopher Burgess, Irina Higgins, Arka Pal. Loic Matthey, Nick Watters, Guillaume Desjardins, and Alexander Lerchner. Understanding disentangling in S /\betaS- VAE, 2018. URL https:
//arxiv.oxq/aaa.
Jachoon Cha and Jeyan Thiyagalingam. Orthogonality-en latent space in autoencoders: An approach learning disentangled representations. In Proceedings of the 40th International Con- ference on Machine Learning, pp. 3913-3948. PMLR, 2023. URL
mlr. vv/22//222//// cha23b. html
Ricky T. Q. Chen. torchdi ffeq.
torchdiffeq. 2018.
URL httpps///gtthbb..cm/rr..g......
Jaewoong Choi, Junho Lee, Changyeon Yoon, Jung Ho Park, Geonho Hwang, and Myungjoo Kang.
Do not escape from the manifold: Discovering the local coordinates the latent space of GANs. 2021. URL httptttss///peeeeviiii wwwp://.wwwweeetoetemee.m d= 4yV RO
Taco S Cohen, Mario Geiger, and Maurice Weiler. A general theory of equivariant CNNs on homogeneous spaces. In Advances in Neural Information Processing Systems, volume 32. Cur- ran Associates, Inc., 2019. URL httttppsss/ppcccceiiiiiiioo www.wnnn..nnnnnuueeeeeeeee.....e neurips wtt.ccc/ppaaa cctppppr//011/////////ppp
.html
Nima Dehmamy, Robin Walters, Yanchen Liu, Dashun Wang, and Rose Yu. Automatic symmetry discovery with lie algebra convolutional network. Advances in Neural Information Processing Systems, 34:2503-255 2515, 2021.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-8794, 2021.
M.P. do Carmo. Riemannian Geometry. Mathematics (Boston. Mass.). Birkhääser, 1992. ISBN 978- 3-7643-3490-1. httpp::/:///o http///www..ooolooooooooooo/o//a http/bookk22dduxxxoqAAAAAAA
co. wwobooks/kkkkcooooooooooookkk
Luca Falorsi, Pim De Haan, Tim R Davidson, Nicola De Cao, Maurice Weiler, Patrick Forré, and Taco S Cohen. Explorations in homeomorphic variational auto-encoding. arXiv preprint arXiv: 04000,00499...0 2018.
Haoqiang Fan, Hao Su, and L. Guibas. A point set generation network for 3d object reconstruction from single image. In Computer Vision and Pattern Recognition, 2016. doi: 10.1109/CVPR. 2017.264.
Jerry A Fodor and Zenon W Pylyshyn. Connectionism and cognitive architecture: critical analysis. Cognition, 28 -2):31111, 1988.
Marco Fumero, Luca Cosmo, Simone Melzi, and Emanuele Rodolà. Learning disentangled repre- sentations via product manifold projection. In International conference on machine learning, pp. 3530-3540. PMLR, 2021.
Muhammad Waleed Gondal, Manuel Wuthrich, Djordje Miladinovic, Francesco Locatello, Mar- tin Breidt, Valentin Volchkov, Joel Akpo, Olivier Bachem, Bernhard Schölkopf, and Stefan Bauer. On the transfer of inductive bias from simulation to the real world: new disentan- glement dataset. H. Wallach, Larochelle, A. Beygelzimer, F. d'Alché-Ban Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran As- sociates, Inc., 2019, URL
neurips. http://s......aooccooa////////// (07.44000.1111111444402213331113
.dd
S. Helgason. Differential Geometry. Lie Groups, and Symmetric Spaces. Crm Proceedings & Lecture Notes. American Mathematical Society, 2001. ISBN 978-0-8218-2844 URL https
7/books. - goog le kr. wtt.kkkbooookssissokk AWAAQQAJ
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick. Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with a constrained variational framework. In International conference learning representations, 2016.

---

Published as a conference paper at ICLR 2026
Irina Higgins, David Amos, David Pfau, Sebastien Racaniere, Loic Matthey, Danilo Rezende, and Alexander Lerehner. Towards a definition of disentangled representations, 2018. URL
http: org/ htarg//bs//e 1812111111 02230
Irina Higgins, Sébastien Racanière, and Danilo Rezende. Symmetry-based representations for artificial and biological general intelligence. Frontiers in Computational Neuroscience, 16:836498, 2022.
Lexiang Hu, Yikang Li, and Zhouchen Lin. Explicit discovery of nonlinear symmetries from dynamic data. 2025.
Geonho Hwang, Jaewoong Choi, Hyunsoo Cho, and Myungjoo Kang. MAGANet: Achieving combinatorial generalization by modeling group action. In Proceedings of the 40th Inter- national Conference on Machine Learning, pp- 14237-14248. PMLR, 2023. URL https
//proceedings... mlr www.llppres///ee wttppppe/////////////333333 html
Hamza Keurti, Hsiao-Ru Pan, Michel Besserve, Benjamin F Grewe, and Bernhard Schölkopf. Homomorphism autoencoder-leamiing group structured representations from observed transitions. In International Conference on Machine Learning, pp. 16 16190-16215. PMLR, 2023.
Diederik P Kingma and Max Welling. 13126613 14, 2013.
Auto-encoding variational bayes.
arXiv preprint
Durk P Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible 1xl convolu- tions. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Gar- nett (eds.), Advances in Neural Information Processing Systems, volume 31. Curran Asso- ciates, Inc., 2018. httppp:://oooceeeiiss..eerpppppp cwtp:ppppppppppppppppppppppppppe files/paper) 2018/fi 2wt.4//0000020000001001101111121
pdff
Gyconghoon Ko, Hyunsu Kim, and Juho Lee. Learning infinitesimal generators of continuous symmetries from data. In Neural Information Processing Systems, 2024. doi: 10.48550/arXiv. 2410.21853.
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2022.
Emile Mathieu, Charline Le Lan, Chris Maddison, Ryota Tomioka, and Yee Whye Teh. Contin- uous hierarchical representations with poincaré variational auto-encoders. Advances in neural information processing systems, 32, 2019.
Loic Matthey, Irina Higgins, Demis Hassabis, and Alexander Lerchner. dsprites: Disentanglement 2017.
testing sprites dataset. http:///tthab.cco//eeeemnn//s/pp
Milton Montero, Jeffrey Bowers, Rui Ponte Costa, Casimir Ludwig, and Gaurav Malhotra. Lost in
latent space: Examining failures of disentangled models at combinatorial generalisation. 35:10136 10149, 2022. URL httpp:://prooeeeeinsn..e.eeeeree
2022 0022hh///71a88002222222555555550 -wttssssttaaattttttt-aaatt Confeerectcooeeeeeeeee. html
Milton Llera Montero, Casimir JH Ludwig, Rui Ponte Costa, Gaurav Malhotra, and Jeffrre Bowers.
The role of disentanglement generalisation. 2020. URL /ttl://tpp///peeeeeeeeew... net/
[Forumii=ggbH 1KUVy]
Hariharan Narayanan and Sanjoy Mitter. Sample complexity of testing the manifold hypothesis. Advances in neural information processing systems, 23, 2010.
Nathan Ng. Kyunghyun Cho, and Marzyeh Ghassemi. Ssmba: Self-supervised manifold based data augmentation for improving out-of-domain robustness. In 2020 Conference Empirical Methods in Natural Language Processing, EMNLP 2020, pp. 1268-1283. Association for Computational Linguistics (ACL), 2020.
Konpat Preechakul. Nattanat Chatthee, Suttisak Wizadwongsa, and Supasorn Suwajanakom. Dif- fusion autoencoders: Toward a meaningful and decodable representation. In Proceedings of the IEFF/CVF conference on computer vision and pattern recognition, pp. 10619-10629, 2022.
12

---

Published as a conference paper at ICLR 2026
Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 652-660, 2017.
Lukas Schott, Julius Von Kügelgen, Frederik Träuble, Peter Vincent Gehler, Chris Russell, Matthias Bethge, Bernhard Schölkopf, Francesco Locatello, and Wieland Brendel. Visual representation learning does not generalize strongly within the same domain. 2021. URL net wttp:/wwwwweeeeoteeeeeeeeeme www.paidddROPPPaaa.g..
https::/openrey/
Hang Shao, Abhishek Kumar, and P. Thomas Fletcher. The riemannian geometry of deep generative models. pp. 315-323, 2018. URL
thecvf. com conteotnttt evpr 2018 workshops w10 htt00/htm//aaaaa The Riemannian Geometry CVPR 2018 paper... html
Ben Shaw, A. Magner, and Kevin R. Moon. Symmetry discovery beyond affine transformations. In Neural Information Processing Systems, 2024. doi: (4..5500arxiv.2246.0366666
Xingzhi Sun, Danqi Liao, Kincaid MacDonald, Yanlei Zhang, Chen Liu, Guillaume Huguet, Guy Wolf, Ian Adelstein, Tim GJ Rudner, and Smita Krishnaswamy. Geometry-avee generative autoencoders for warped riemannian metric learning and generative modeling on data manifolds. In International Conference on Artificial Intelligence and Statistics. PMLR, 2025.
Lock Tonnaer, Luis A Pérez Rey, Vlado Menkovski, Mike Holenderski, and Jacobus W Porte- gies. Quantifying and learning linear symmetry-basee disentanglement. arXiv preprint arXiv:2011.06070 2020.
Ivan I. Vankov and Jeffrey Bowers. Training neural networks encode symbols enables com- binatorial generalization. 375(1791)220190399 2019. doi: 10. (tt9////t.2009..00... URL
Elif Vural and Christine Guillemot. Out-of-sample generalizations for supervised manifold learning
for classification. IEEE Transactions Image Processing, 25(3):1410-1424 2016.
Thaddäus Wiedemer, Prasanna Mayilvahanan, Matthias Bethge, and Wieland Brendel. Compositional
generalization from first principles. 2023. URL
net. forum?
LqoQluJmSx
Jianwei Yang. Nima Dehmamy, R. Walters, and Rose Yu. Latent space symmetry discovery. In International Conference Machine Learning, 2023a. doi: 10.48550/axx...33000000
Jianwei Yang. R. Walters, Nima Dehmamy, and Rose Yu. Generative adversarial symmetry discovery.
In International Conference on Machine Learning, 2023b. doi: 10448550/arXiw.2332.0033
Tao Yang, Xuanchi Ren, Yuwang Wang, Wenjun Zeng. and Nanning Zheng. Towards building
group-based unsupervised representation disentanglement framework. 2022. URL https
/ /ooenrevewww net /ttp://www......om.omooooooooooo
Xinqi Zhu, Chang Xu, and Dacheng Tao. Commutative lie group VAE for disentanglement learning.
In Proceedings of the 38th International Conference on Machine Learning, pp. 12924 12934. PMLR, 2021. URL https:tt//// httpss://ppccceciing. mlr www..r..reesvvv3e1 21111/////2211
zhu216 html
13

---

Published as a conference paper at ICLR 2026
A DETAIL ON THEORETICAL BACKGROUND
Group Action Group (G, *) is a mathematical structure which is tuple of a set G and binary operation closed the set. The group should satisfy the following axioms:
1. (associativity) (b c) (a b)
2. (identity element) there exists E G such that e
3. (inverse element) there exists a- G such that a a a-- =a=e
for every a, c E G. The group plays as representation of symmetry. The group action on set X of group G map G. X X which satisfies the following axioms:
1. (identity) f(e,x))
2. (compatibility) flg s(h, x²) f(gh, x)
for an identity E G and every g. E G and every E X. We can decompose natural phenomena into objects and symmetries of those via group action.
Definition A.I Let G be group and X be G-space. The action is said be transitive if there exists y E G such that * x for any v. y E X.
This means that every point x E X can be translated into any point X with an action g E G.
Geometry In geometry and topology, manifold topological space that locally resembles Euclidean space at every point. More formally, a smooth manifold, which is type of manifold, can be defined as follows (do Carmo 1992).
Definition A.2 A smooth (or differentiable) manifold of dimension is M and family of injective mappings Da Ua C R2 >> M of open sets Ua Tf
TR. into M such that
(aza(Ua))M))
2. for any pair 3 with Ta (Ua) n (Up) W + the x and 3 (W) open sets in I² and the mappings 23 are differentiable.
3. The family {(U,,, { is maximal relative to the conditions (1) and (2).
Every point on a manifold has a tangent space, which is the vector space tangent to the manifold.
Definition A.3 Let M be differentiable manifold. A differentiable function (-e,e) M is called a (disferentiable) curve in M. Suppose that a0)) = p E M, and let D be the set of functions on M that are differentiable at p. The tangent vector the curve a at 0 a function a' RR given by
d(f a
d'())
.fe D. 1=U
(7)
dt
A tangent vector at is the tangent vector at t 0 of some curve a (-c,c) + M with (: The set of all tangent vectors to M at p will be indicated by the tangent space T.MM
Informally, neighborhood of a point p on a manifold is open subset of M that contains p.
B ARCHITECTURES
B.1 BASELINES
To implement 3-VAE (Higgins al. 2016), used the structure introduced in (Burgess al. 2018). The encoder consists four convolutional layers with 32 channels, two fully connected layers with 256 nodes, and fully connected layer with d nodes, where d is the latent vector dimension. The
14

---

Published as a conference paper at ICLR 2026
decoder consists of transpose of the encoder structure. ReLU activation is used for each layer, except for the last layer of the encoder and decoder.
To implement MAGANet (Hwang al. 2023). follow the proposed architecture. The encoder for modeling the group actions was the same as that of the VAE encoder architecture. The decoder consists of linear layer without bias to apply the group action, well as the GLOW model (Kingma Dhariwal 2018). The GLOW model comprises three flow modules, each consisting three flow blocks and squeeze layer. Each flow block comprised ActNorm, x 1 convolution without LU decomposition, and an additive coupling layer. MAGANet incorporates three primary loss functions train the VAE and flow-based components.
Lrecon
(2) x1). (F((,,, 2(2,x)));;
(8)
Lrecon latent
(9)
Lasse
Lreeon + BKI.LKI. + Brenon latent Creron latent:
(10)
where lp denotes the loss image space, I represents the L1 norm, E is the encoder, and the decoder. The hyperparameters 3KL and Precoulaaentt control the weighting of KL divergence and latent reconstruction losses, respectively.
B.2 PROPOSED METHOD
We utilize the Lie algebra encoder, inspired by Zhu et al. (2021). However, we do not employ commutativity, as most symmetric spaces are not commutative, and the influence commutativity on combinatorial generalization is not our research focus. The flow matching module consists of simple UNet architecture and conditioning by the Lie algebra using AdaGN, as proposed in Dhariwal & Nichol 2021).
C EXPERIMENTS DETAILS
C.I EXPERIMENT ON SPHERE SYNTHETIC DATA
Dataset and Common Setting The data point cloud comprises 15,000 points of the training data and 3,000 points of the test data. Every model was trained for 100 epochs with a learning rate of 1e-4. Each model has three latent dimensions; Cartan models have two Bp and a Bt.
Model Architecture Every PointNet block consists of layers point-wise MLPs and Leaky ReL.Us between the layers. Moreover, for flow matching, the AdaGN layer conditioning is performed with Lie algebra after each block.
Hyperparameter 3 for the VAE encoder is set to 0.001 to prevent the model from collapsing. ACartan is set to 0.1 and in Equation i set to 0.001. The number of groups in AdaGN is set to 8.
C.2 EXPERIMENT ON BENCHMARKS OF COMBINATORIAL GENERALIZATION
Dataset Setting For split dSprites dataset Matthey al. 2017 in Recombiatiio-to-Eleennts setting, we except following combinations:
1. shape=ellipse, scale=0.5, 120" < orientation < 240", 0.6 < 0.6 <
2. scale=0.5, orientation=0". x S 0.25, S 0.25,
3. shape=heart, orientation=0". 0.5 < 0.5 < y-
In the Recombinatioonto-RRngg setting, we except the following combinations:
1. shape=heart, 0.5 < x,
2. shape=square, 0.5 2,
3. shape=ellipse, 3 <scale, y < 0.5.
15

---

Published as a conference paper at ICLR 2026
For 3D Shapes dataset Burgess & Kim (0188 in Recombinationn-o--llaaee setting, except following combinations:
1. floor-hue>> 0.5, wall-hue 0.5, object-hue>> 0.5, scale=7, shape=cube, orientation=O",
2. floor-hues 0.5, wall-hues 0.5, object-hues 0.5, scale=7, shape=cylinder, orientation=0",
3. floor-hues 0.5, wall-hue>> 0.5, object-hue=0, scale=0, shape=(sphere, cube), orientation=-30%
In the Recombination to- Ra-ga setting, we except the following combinations:
1. 0 <floor-hue< 1, 0 <wall-hues 1, object-hue>> 0.5, 0 <scale 1, shape=oblong.
30 Sorientation 30"
2. 0 fflor-hhues 1, 0 <wall-hue< 1, 0 Sobject-hue> 1, scale 2, shape=sphere, -300 Sorientations 30"
3. floor-hue< 0.5, 0 <wall-hues 1, 0 Sobject-hues 1, 0 <scales 8, shape=cylinder, -30 Sorientation 0%
C.3 MODEL ARCHITECTURES
C.3.1 BASELINE MODELS
3-VAE For the 3- 3-VAE baseline (Higgins et al. 2016), we adopt the standard convolutional archi- tecture from Burgess et al. 2018).
Encoder: Consists of four convolutional layers (32 channels, kernel size 4, stride 2), followed by two fully-connected layers (256 nodes each) that output a latent vector of dimension d.
Decoder: Symmetrically mirrors the encoder architecture using transposed convolutional layers.
Activation: ReLU activation used for all layers except for the output layers of the encoder and the decoder.
MAGANet For MAGANet (Hwang et al., 2023), the encoder for learning the group action is identical to the 3-VAE encoder. The decoder is composed of linear layer (without bias) apply the group action and a GLOW model (Kingma & Dhariwal (018)) for the generative process. The GLOW model comprises three flow modules, each consisting of three blocks that utilize ActNorm, 1 x convolution, and an additive coupling layer. The model is trained with the following primary loss functions:
Lrecon 11D(D(E(211 22) x1),
(11)
Crecon Jatent (B(x,, D(z, (),, 2),
(12)
Lase Lrecon + LKI + Creeon where lp is the image reconstruction loss, and is the Ta norm.
(13)
C.3.2 PROPOSED METHOD (CARTANFM)
Our model's architecture consists of a Lie algebra encoder (detailed in Section 3.3 and conditional generative model.
The generative model a simple UNet architecture, where the conditioning on the Lie algebra element P is performed using Adaptive Group Normalization (AdaGN) (Dhariwal & Nichol 202I) in each block. The channel dimensions are structured as follows:
Generative Model
Input Channels: 1 (dSprites, MP13D) or 3 (3D Shapes).
Downsampling Path Channels: [C, 64, 128, 256), where the input channel size.
Middle Block Channels: [256].
Upsampling Path Channels (with skip connections): [256 + 256, 128 128, 64 + 64, C
CJ.
16

---

Published as a conference paper at ICLR 2026
ODE Solver For the decoding process, the standard Euler method, basic ODE solver implemented in the torchdi ffeq library (Chen 2018
Hyperparameter Hyperparameters of MAGANet had been searched by grid and Bayesian search in the range of 9KL, PLR E [O, 10000. We find that is the best value for hyperparameters.
The proposed model has the following hyperparameter settings:
ACartan: AgsC 1.0.
0.001,
3 0.01,
Group number of AdaGN is 8.
We set the number of basis elements for p to 10, consistent with the latent dimension of the baseline models, and the number of basis elements for to 5.
Computing Resource We conducted experiments on a local server equipped with NVIDIA graphies cards, including the RTX 2080 Ti, RTX 3090, RTX A6000, and RTX A100. Each run requires approximately 6000MiB of VRAM and takes about 30 hours. These requirements may vary depending on the dataset, split settings, and GPU used.
C.4 ADDITIONAL GENERATED SAMPLES
Every image consists of ground truth (odd columns) and reconstructions (even columns).
D ADDITIONAL EXPERIMENTS
D.1 LIEGAN ON SPIERRE DATASET
We conducted additional experiments of LieGAN (Yang et al., 2023a) on 3D Sphere dataset of subsection 5.1 Experiments details are as follows:
Training dataset composed of 15000 points on upper sphere of 135 degree.
Lie algebra basis of model is randomly initialized.
Discriminator consists of PointNet block and linear layers.
Sampled result image be found in 10 Because of difference between VAE based model and GANs, we sample transformed points by sampled 100 transformations and inspect whether test region be generated by such transformations. We observe that test region is not generated perfectly by such transformation. Moreover, learned basis are not form skew-symmetric.
D.2 VERIFYING GEODESIC SYMMETRY
We visualize pairs of basis vectors (lines ending arrows) and their negated vectors (lines ending in circles). This alignment is consistent with the Lie algobra- definition of geodesic symmetry, implying that points on the generated geodesics are effectively reflected through the origin to their antipodal position including test region.
17

---

Published as a conference paper at ICLR 2026
dSprites R2F Case
(b) dSprites R2E Case2
(c) dSprites R2E Case3
(d) dSprites R2R Casel
(e) dSprites R2R Case2
dSprites R2R Case3
Figure 5: Generated Images dSprites
18

---

Published as a conference paper at ICLR 2026
(a) 3D Shapes R2F Case1
(b) 3D Shapes R2F Case2
(c) 3D Shapes R2E Case3
(d) 3D Shapes R2R Casel
(e) 3D Shapes R2R Case2
(l) Shapes R2R Case3
Figure 6: Generated Images 3D Shapes
19

---

Published as a conference paper at ICLR 2026
3
3
3
(a) MP13D R2F Casel
(h) MPI3D R2F Case2
J
(c) MPI3D R2E Case3
(d) MPI3D R2R Casel
I
6
(e) MPI3D R2R Case2
(i) MPI3D R2R Case3
Figure 7: Generated Images MPI3D
20

---

Published as a conference paper at ICLR 2026
Figure 9: Visualization of Geodesic Symmetries (p only)
Figure 8: Visualization of Geodesic Symmetries
All Transformeed Points
Full Sphere Referance
Sahaer Retweee twiSSS3
1.
-21
-1
-18
-10
Figure 10: Visualization of Sphere Construction of LieGAN
21

---

Published as a conference paper at ICLR 2026
-Bt
2.0
-B,
B,
-B, B
15
Br + 8
B, B
1.0
0.0
-0.5
-1.0
-2.0
-2.0
-15
-1.0
-0.5
2.0
Figure 11: Visualization of Geodesic Symmetry on Tangent Space. Each basis vector (lines ending in arrows) is reflected to its negation (lines ending in circles) by geodesic symmetry.
22


Reviews:
[{"review_id": "Y74GqPAm26", "reviewer": "Reviewer_WXcp", "summary": "This paper addresses a key challenge in symmetry learning—how to generalize from training data to unseen data. It introduces the Cartan loss and the Geodesic Symmetry Consistency loss, which are designed to enforce fundamental properties of Lie algebras and enhance generalization capabilities, respectively. The learnable components include a Lie Algebra Encoder, which maps input data to coefficients of Lie algebra bases; learnable Lie algebra bases; and a conditional flow matching model, which generates a vector field based on the Lie algebra and data points. Across a series of combinatorial generalization benchmarks, the proposed method consistently outperforms existing baselines.\n\nGiven the theoretical rigor and innovation, I am inclined to accept it.", "questions": "- I notice that Definition 2.2 is defined for a one-parameter group action ($t \\in \\mathbb{R}$). So how does it guide the design of the GSC loss in the multi-parameter case?\n\n- Does this method have potential for application in high-dimensional data—such as in the embedding spaces of images and text?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- To my knowledge, none of the existing symmetry discovery methods take into account the fundamental property of Lie algebras—the relationship defined by the Lie bracket. The proposed CartanFM in this paper rigorously incorporates this aspect, resulting in a more reasonable learned Lie algebra space, which I consider one of the highlights of this work.\n\n- Most symmetry discovery approaches opt to directly learn either the basis of the Lie algebra or the vector field. The former is often limited to relatively simple linear symmetries, while the latter, due to the complexity of vector fields, generally lacks interpretability. CartanFM’s flow matching module innovatively addresses both limitations. Specifically, it establishes a mapping between the Lie algebra and the vector field, enabling us to explicitly solve for the Lie algebra basis while also obtaining its specific action form on data points.\n\n- The generalization capability on unseen data provides potential for scientific discovery and the learning of structured representation spaces.", "weaknesses": "- Due to the involvement of learning Lie algebra bases, a brief discussion and comparison of the related symmetry discovery work (https://arxiv.org/abs/2310.00105, https://arxiv.org/abs/2410.21853, https://arxiv.org/abs/2403.01946, https://arxiv.org/abs/2510.01855) is necessary.\n\n- It is better to list the contributions point by point in detail in Section 1.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2Z1YXBqlom", "reviewer": "Reviewer_iY5v", "summary": "In this paper, the authors propose a method for addressing the issue of symmetry generalization. The method operates under the assumption of symmetric spaces, using Cartan decomposition and learning a Lie algebra. The geodesic symmetry property of symmetric spaces is used to extend/extrapolate symmetries to unobserved samples in the complete data space--that is, the space in which the observed data lives. The authors also conduct experiments in which their method appears to outperform baselines.", "questions": "1. It seems that the perceived issue of symmetry generalization is being solved primarily by making restrictive assumptions about the structure of the symmetry group. Generally, restrictive assumptions can make problems more tractable, but I am concerned that the limitation to transitive group actions is too restrictive. After all, the action of rotations in the plane about the origin is not a transitive group action: this seems like one of the simplest possible symmetries to consider. Can the authors speak to this?\n\n2. The notion that there are points in the complete space that are not mapped to via the observed group acting on the observed data seems to be a model assumption, which assumption seems equally valid to the assumption that training data defines the manifold and/or distribution of data. It seems that the notion that test data is not only out of distribution, but also that it aligns with the symmetric space model, is a model assumption that is not motivated by information obtainable from the training data: in fact, methods which rely on the training data are prone to inherit the generalization challenge spoken of. Are these correct statements? If so, can the authors speak to (or reiterate) justification for these model assumptions?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. This paper is built upon a strong mathematical foundation.\n\n2. The experimental results demonstrate the advantages of the proposed methodology compared with the chosen baselines.\n\n3. This paper demonstrates potential for acceptance, provided that the perceived weaknesses can be addressed.", "weaknesses": "1. *The issue of symmetry generalization is not properly motivated.* The authors cite a single paper from which they draw the conclusion that current symmetry-based machine learning methods are limited in that learned symmetries do not extend beyond observed samples. In light of previous work that is not considered in this paper, the statement on the symmetry generalization challenge is actually not true. The authors are encouraged to discuss their statement in light of additional recent work on symmetry discovery, particularly for the following sources: \"Generative Adversarial Symmetry Discovery\" (Yang et al., 2023), \"Learning Infinitesimal Generators of Continuous Symmetries from Data\" (Ko et al., 2024), and \"Symmetry Discovery Beyond Affine Transformations\" (Shaw et al., 2024). On the other hand, the authors may be able to justify the exclusion of certain sources. But as written, the symmetry generalization statement does not appear to be reflective of recent work.\n\n2. *Additional methods to compare with.* From an experimental standpoint, it seems there are other geometry-inspired autoencoders that should be compared with. One such method is \"Geometry Regularized Autoencoders\" (Duque et al., 2023), and another is \"Geometry-aware generative autoencoders for warped riemannian metric learning and generative modeling on data manifolds\" (Sun et al., 2024).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2e27h9J9zN", "reviewer": "Reviewer_8EBC", "summary": "This paper explores a method for so-called combinatorial generalization. The idea is to assume that the data lies on a manifold $M$ which has the form of a symmetric space, i.e. has the structure $G/K$, where $K \\subseteq G$ are Lie groups. $G$ and $K$ are thereby unknown. The authors utilize that such a structure induces a decomposition of the Lie algebra $\\mathfrak{g}$ of $G$ into spaces $\\mathfrak{l}$ and $\\mathfrak{p}$ corresponding to the Lie algebra of the subgroup $K$ and the tangent space of the manifold $M$. Flipping a vector $P\\in \\mathfrak{p}$ thereby corresponds to changing direction of a geodesic through a chosen origin on $M$. \n\nThe authors use facts about the structure of $\\mathfrak{l}$ and $\\mathfrak{p}$ ($[\\mathfrak{l},\\mathfrak{l}]\\subseteq \\mathfrak{l}, [\\mathfrak{p},\\mathfrak{l}]\\subseteq \\mathfrak{p}, [\\mathfrak{p},\\mathfrak{p}]\\subseteq \\mathfrak{l}]$), and the fact that flipping the $P$-vector corresponds to changing time-direction in geodesics to design loss functions for a learning framework. This is subsequently incorporated into a flow-matching model to produce state-of-the art results on several datasets.", "questions": "See weaknesses. \n\nAlso, the symmetry techniques do not bring any increase in performance on the R2R task on the 3D Shapes dataset. Can the authors comment on why they think this is?", "limitations": "", "rating": 2, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "The framework the authors propose is grounded in mathematical theory and is elegant.The models have been tested on multiple dataset, and fare well on all of them.", "weaknesses": "Let me start by saying that I have little experience on the problem of combinatorial generalization, so I am very open to the authors correcting me/ critiquing my assessment.\n\nWith this said, I think that the main weakness of this paper in that it suffers in clarity and reproducibility. This is the main reason for my rating.\n\n1. The 3D sphere shape manifold reconstruction data is not properly described. It is not describe how the training and data set are generated. In fact, it is to me not clear whether the point cloud in (a) is the entire dataset or if (a) is a point in the training dataset -- same for (b). Given the description of the dataset in the appendix, I am inclined to believe that (a) is the entire data set, and that each point in the cloud is a training example, but how is then a PointNet used to process the data point?\n\nIt is not clear what the generation process of the points in (a) is, and how the ground truth (b) is fed into the models. If I understand e.g. the dSprites datasets correctly, the net is given a set of latent parameters it has not seen before (shape,scale, orientation, position-x,position-y) and its task is to generate an image corresponding to the latent parameters. What are the latent parameters here? \n\nIt should also be noted that the authors have provided no code.\n\n2. I do not understand entirely understand to which extent the GSC-loss is encouraging the decoder to be geodesic-symmetric. In fact, as formulated in 3.2, the loss 'only' encourages the encoder E and decoder D to satisfy $E\\circ D\\approx \\mathrm{id}$ on the flipped versions of the latent vectors of the datasets. Can the authors provide any experimential insight to whether this is actually what happens in practice? This is particularly interesting given that the ablation studies point to the GSC-loss being important.\n\n In my opinion, providing some type of experimental evidence here is crucial. If the GSC-loss is not actually encouraging the decoder to be geodesic-symmetric, the theoretical considerations of the paper are not providing and explaination to the increased performance of the model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)