Equivariance by Contrast: Identifiable Equivariant Embeddings from Unlabeled Finite Group Actions
Authors: Tobias Schmidt
Steffen Schneider
Matthias Bethge
Venue: NeurIPS 2025 poster ()
Keywords: equivariance
identifiability
representation learning
group theory
Primary area: general_machine_learning

Abstract: We propose Equivariance by Contrast (EbC) to learn equivariant embeddings from observation pairs $(\mathbf{y}, g \cdot \mathbf{y})$, where $g$ is drawn from a finite group acting on the data. Our method jointly learns a latent space and a group representation in which group actions correspond to invertible linear maps—without relying on group-specific inductive biases. We validate our approach on the infinite dSprites dataset with structured transformations defined by the finite group $G:= (R_m \times \mathbb{Z}_n \times \mathbb{Z}_n)$, combining discrete rotations and periodic translations. The resulting embeddings exhibit high-fidelity equivariance, with group operations faithfully reproduced in latent space. On synthetic data, we further validate the approach on the non-abelian orthogonal group $O(n)$ and the general linear group $GL(n)$. We also provide a theoretical proof for identifiability. While broad evaluation across diverse group types on real-world data remains future work, our results constitute the first successful demonstration of general-purpose encoder-only equivariant learning from group action observations alone, including non-trivial non-abelian groups and a product group motivated by modeling affine equivariances in computer vision.

Equivariance by Contrast: Identifiable Equivariant Embeddings from Unlabeled Finite Group Actions
Tobias Schmidt Steffenn S 2.3+ and Matthias Bethge Institute of Computational Biology, Helmholtz Munich Munich Center for Machine Learning (MCML) Tübingen Al Center
Abstract
We propose Equivariance by Contrast (EbC) learn equivariant embeddings from observation pairs (y.9 y), where is drawn from finite group acting the data. Our method jointly learns a latent space and group representation in which group actions correspond to invertible linear maps-without relying group-specific inductive biases. We validate our approach on the infinite dSprites dataset with structured transformations defined by the finite group G == (Ru, x Z, x Z,,), ) combining discrete rotations and periodic translations. The resulting embeddings exhibit high-fidelity equivariance, with group operations faithfully reproduced in latent space. On synthetic data, we further validate the approach on the non- abelian orthogonal group (nn) and the general linear group GL(n). We also provide a theoretical proof for identifiability. While broad evaluation across diverse group types real-world data remains future work, our results constitute the first successful demonstration of general-purpose encoder-only equivariant learning from group action observations alone, including non-trivial non-ahelian groups and a product group motivated by modeling affine equivariances in computer vision.
1 Introduction
In many real-world inference problems, the relationship between obser- vations is governed by structured transformations. The same sample may be observed before and after an "action" has been applied. In computer vision, an object may be observed in the form of an image before and after rotations, translations, other types of transformations have been applied 11 32.. In biology, large-scale single-cell transcriptomic datasets in- creasingly contain observations of cells before and after perturbations in the form of gene knockouts [17] or pharmacological intervention [41]. In neuroscience, neural activity reflects changing brain states under sensory input or behavioral output ing the data lies not only in modeling individual observations but also in capturing the structured relationships between them.
R(g)
x
y
y
db
(5). In all these cases, the key to understand-
R'
Figure 1: Commutative
This motivates the goal of learning equivariant embeddings. In this em- bedding space, actions are represented by linear transformations. To address this challenge in theoretically grounded way, adopt a per- spective rooted nonlinear Independent Component Analysis (ICA) and group theory. Suppose that each observation Y arises from latent representation nonlinear mixing function i.e., y f(ær). Nonlinear ICA aims to invert this process: to learn an encoder f that recovers the latent structure from the observed data.
diagram showing data and model for EbC.
X through an unknown, injective
(Co-corresponding authors: www.:/hwwniieeeeeeeeootooti-eeee
matthias. @bethgclab.org.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Ground Truth Group Action
Observed Sampies
Encoder
Implicit Group Representation
Φ
45 3+
X(X,X")
g G
R(X,X")
min |X XR
Figure 2: Overview of the approach. Left to right: One batch of observed sample consists of n + -1 paired samples ((y v0)= where yf are related to y; via an unkown group action such that y = yss An encoder @ maps these observations into latent space {(é., 2)) { = where samples
used to estimate representation R of the group.
The nonlinear ICA problem becomes tractable by additional assumptions about the structure of the data-generaiie generatinn process [18] 19 21 22 56).. Here, we leverage the property that many datasets do not consist of isolated samples but of pairs (y, y'), where y' is a transformed version of Group theory provides a formalism to describe these transformations as the elements of group G.
Each group element E G can act on a latent æ E X, yielding a transformed latent gz. This operation known group action. In observation space, introduce the shorthand y == 9 y denote the respective relation between y = {gæx and f(x). In the latent space, these group actions have a particularly simple form: they correspond linear maps. Formally, group representation is a homomorphism R G + GL(X) which maps each group element to an invertible matrix acting on the vector space X. Thus, while the transformation g y may have complicated non-linear effects in observation space, in suitable latent space it can be reduced to linear relationship of the form:
x' = ga: = R(ggx.
(1)
Our goal to infer this relation directly from pairs (y.g g y) of observable data and learn an encoder Y + X' and representation R' of the group such that
Φ(g y) = R' (g)o(y).
(2)
and It' : + GL(X') is representation of G (potentially different from R) on the vector space X'. Learning such representations directly from data difficult. A growing body of work has approached this problem by leveraging pairs (y.g y) even when the specific group element g is unknown with varying constraints: CARE (15) restricts itself to orthogonal representations on the hypersphere. STL [52] allows nonlinear equivariant relations in latent space, and the neural fourier transform [NFT; 28 requires to learn a generative model of the data.
[15 28 52
Here, we propose Equivariance by Contrast (EbC), to the best of our knowledge the first encoder-only method that learns general linear group representations from group action observations with formal identifiability guarantee. In & [ implicit group representation via contrastive learning, without generative modeling or group-specific architectural biases. In $ we show that EbC recovers the true latent space and the underlying group representation up linear transformation. In 6 we evaluate EbC on synthetic and structured vision datasets, including finite product groups G' == [Rm x En, x E,,) and non-abelian groups such as (n) and (n). EbC achieves high-fidelity equivariant embeddings across diverse settings.
we present the algorithm which jointly learns the encoder & and an
2 Learning group-equivariant representations with contrastive learning
we assume that data come in the form of batches, which are grouped into n + 1 pairs undergoing the same action g. This form of data is common in various scientific fields, for example, in neuroscience and biology. Pairs can also be derived from time-series data under the assumption that nearby points are governed by a shared action ["slowness prior": 26)].
Figure 2 outlines our approach: Similar to previous work 15 28 36 52
The intuition behind our objective function depicted in Figure The model has access to a set of mixed samples related via the group action (Y and Y'), along with query sample y. The objective

---

is to infer the group action from the examples Y and Y', apply it to the query, and select the correct answer y among a set of options that include the correct answer alongside negative samples y" E S. In our case, the set S contains negative samples randomly selected from the dataset, which cover all potential mismatches (different content, different group action, etc.) alongside the positive sample. We use contrastive learning to encode this objective in the likelihood
exp -
1 (y: Y, Y') 2130))13)
Po(y' y. Y', En Ey exp U y. Y, X¹) - (1)))
(3)
The shorthand us denotes the operation of inferring the linear representation of the group element, R(((Y), (YY),, and then applying it to the feature vector of the reference sample d(y):
(y, Y, Y') R( (Y) Φl ()) R x')
(4)
min RcGL(d)
|
X 1
(5)
To find the optimal feature encoder o, optimize the likelihood across all pairs of samples and uniformly sampled negative examples,
min [(ø] -Ey.y Y,Y', [log p y. Y. N.Y.Y,YS s))
(6)
which is related to the InfoNCE loss with the additional structure required for group learning.
Separating content and style In many cases, we want to produce embedding spaces in which we can separate what is transformed from how transformed. Intuitively, we consider these aspects of the latent representation to encode content and style. The content is the part of the representation that is expected be invariant w.r.t. the group action g. whereas the style is expected to be equivariant w.r.t. the group action. In practice, it is therefore useful to split the vector space induced by & into an equivariant and invariant part. Algorithmically. this is done by imposing additional structure on the matrix in Eq. (5), constraining the minimization across matrices of the form diag(GT.,. Im) C GL(m + n). The representation we learn then has the form
Rm (X, X'X
R,, Xx, X')
O 0nxm
O Omxn
I
This results in an encoder & with a n-dimensional equivariant subspace, and m-dimensional invariant subspace. We discuss the theoretical properties of this parametrization below. Note the conceptual similarity to subspace contrastive learning discussed in [44. The difference here is that we do not use multiple instances of the contrastive loss but a single contrastive loss that directly learns structured feature space.
3 Equivariance by Contrast is identifiable
By formalizing assumptions about the lata-generating process, can derive identifiability guarantees for the algorithm described in the previous section.
Dataset. We define the dataset in terms of the underlying data-generating process: Let E V C Rd be a vector describing the ground truth latent components, let g be an element of a group G, and let f: V + RD be an injective map. Assume that for each group element g, we have at least M pairs of samples (yy, 9¹) with
y; f(x.).
3í ((R(g)x))
1, M
(8)
where R G + GL R) representation of G on V. If R is structured accordingly (Eq. 7 can be decomposed into an equivariant style and an invariant content, which is denoted as e further below in the experimental sections.
We model the group representation using the non-parametric approach outlined in E Assume that for each group element g E G, are given two matrices
Implicit group representations.
X, X' E Md M > d, where the row vectors (I,, x)) are related via the group element as x¹ = ga;, i E {I,..., 3. As shorthand, we write X' gX. Then, the expression
R(X,X') = min REGL(d)
|X - X
R X X (X
(x X')
(9)
3

---

?
S (y' E ES)
Y
Figure 3: Infuitive explanation of the training objective. Given pairs of samples (Y, Y') connected by an action infer the most likely action, and apply it to novel sample y with different content. The training objective is contrastive, and aims to select the correct matching sample y' from selection of positive/negative samples S.
a representation of G with R(g) RxX,gX) for each E G. In practice, we do not have access to (X, X') directly, but to a nonlinear projection of these points via the mixing function f, denoted as (f(X), f(X') (Y, Y'). We map these mixed samples to a feature space using learnable encoder Φ RD Rd and insert the resulting matrices into Eq. 9 Our goal is to optimize & such that R()(Y), & (YY)) becomes a representation of G. An advantage of this approach is that both the feature space and the group representation are fully defined via the feature encoder Φ).
Equivariance by Contrast learns group representations. Our theory builds on the canonical discriminative form discussed by Roeder al. 40]. which requires conditions on the diversity of the dataset to be fulfilled. Specifically, the latents and feature spaces of á need to "sufficiently vary" across the points in S. addition, the vectors of differences ( Φ(yB) with YA y columns of S need to span a d-dimensional feature space. The conditions are discussed in detail in Appendix This gives the following main results for our model:
Theorem 1 (Identitiability of the Group Representation; informal), Assume that Ps Pr and let the dataset satisfy the diversity conditions mentioned above. Define h == do Then, for all points æ in the support of the dataset:
(a) We recover the original vector space h(x) Lx, up to an ambiguity GL(d).
(b) We recover a representation of the group, R(h(X), h(gX)) = LR(g)
Proof sketch. With minor modification, Eq. 3 follows the canonical discriminative form for which Roeder al. [40] proved linear identifiability. Due the use of the more flexible Euclidean loss, we instead obtain affine identifiabiliyy for h(x) = Lx + b. The second condition requires Rő(x) LRx and implies b O. From these two conditions and the specific definition of our model, we can derive the results (a) and (b). The full proof is given in Appendix A
Corollary (Equivariance). We have V, R(g) representation of the group in V we have h: + W where we define h = Φo and R' (g) = R(((X), h(gX)) the representation on W. Then, we have
h(ga) gh(#))
(10)
Proof. The result follows from Theorem For all we have h(gæ) gh(x). We insert the representation g on V on the LHS, and on W on the RHS, to obtain h(R(g)æ) = (())) We insert Thm. [ to obtain (R(g))= x= R' (g) Lx. From here we obtain LR(g) - R'(g) which we showed in Thm. Tb.
4 Experiment Setup
We validate our proposed model on set of diverse datasets with different underlying groups.
Synthetic Group Datasets We first consider three types of synthetic datasets following the data- generating process Eq. S matching the assumptions required for Theorem I Group elements taken from a subgroup of special orthogonal group SO(n), the orthogonal group O(n) and the general linear group GL) We start by sampling random group elements g ~ (a(g) from G E (8O(n), On)) (m))) We sample at least (with d) random vectors z S" from
the unit sphere and sample a content component C from finite set of vectors C Sd From

---

(R(g), { c) we generate Y = {v y DT, where y, = f(z;, c) and y f(R(g) c). We repeat this sampling procedure until we reach the desired dataset size. The nonlinear injective mixing function f is parameterized by a random 3-layer MIP [8] followed by a random linear map to 50-dimensional space. For the full dataset we sample a maximum of 1000 matrices R(g) and a total of 1M pairs (y, y²). Unless stated otherwise, we choose n = 3, d = 3, |C|
100. In Appendix C. we show additional results for n E 3, 5, 7, 9 and in Appendix C.2 we report results on data efficiency.
According to the assumptions of Theorem [ the relationship ( in the data-generating process does not include noise. But we show a variation of this experiment In Appendix C.3 that includes noise in the application of the group action.
Infinite dSprites We leverage infinite dSprites [8 idSprites], an extension of dSprites [3] for validation on more diverse visual data. We subsample all images to a resolution of 64 x 64 The dataset allows for rich variations in the content and ensures that the resulting objects do not have any symmetries. By default, we use the same number of varying factors as in dSprites, namely 3 random shapes of 6 different sizes. For our purposes, we consider the combination of shape and size to be the content. Each of these objects may then be oriented in 40 different ways or have one of 32 32 x or y position translations. We consider the orientations and translations to be our groups of interest and sample the data such that we obtain closed groups. By default, we sample the dataset such that any paired sample (y.3) 3²) undergoes a combination of these three types of transformations.
Training Protocol We use a three layer MLP with 512 hidden units for ø. We fit the implicit representation R(Y, Y") using the gels least squares solver in PyTorch. By default, we use 84 sample pairs for idSprites and 12 for the synthetic data in Y. Each batch has 1024 positive and 16k negative samples. We train the model for 20k with Adam (25) with learning rate 10-3 We perform 80/10/10 train/valid/test splits. Metries are computed out-of-sample, by fitting on one part of the valid split, and reporting on the other. Unless stated otherwise, standard deviations and confidence intervals are reported across three dataset and three model seeds. More details in Appendix B
Baselines We leverage standard InfoNCE [37] training as a baseline (amounting to setting R = I) similar to SimCLR (2) We also consider dynamics contrastive learning (30] using a linear dynamical system (LDS) or a switching linear dynamical system (SLDS). We use the same training protocol for the baselines as we do for our own EbC model (same encoder, batch size, training iterations, optimizer, and data loader).
Metries Our metrics aim to quantify the representation of the content, the quality of the group representation, and the linear identifiability of the latent space. To measure the quality of the representation of the content we fit a Logistic Regression (W. b) and predict the class label j K of content cy:
Ac,(C, == TopK-Acc(a(Wh(()) + b) YY)
a()) == softmax() (-)
(11)
Likewise, to quantify the linear identifiability of the embedding space (Theorem Ta), we measure:
R²
max R ())(z))
(12)
LEGL(N)
To measure the quality of the group representation we introduce two different metrics. First we introduce an R² based metric which we derive from Theorem Ib:
R²(G)) == R Rh( Rx), L == arg max |X h h(X) 1E
(13)
To measure the group representation without access to the ground truth latent representation æ, we introduce an Accuracy over the KNN lookup of the transformed input vector y':
Acc(G, K)= TopK-Acc(KNN(Y))Y)
kNN(y) == arg max jEJ
(|o(s) Ro(v))?
(14)
To compute this metric, we sample 20k random pairs y, y' to solve a 20k-class classification problem. We compute the accuracy vols-validation across every of the 20k samples.
The metries defined above can be split into two categories: Rxxx) and R²(G) require access to the ground truth latents and hence can only be computed on synthetic toy datasets for which we have full knowledge about the data-generating process. In practice, this not the case. Therefore, we defined a second group of metrics, including Acc(GG Go K) and Acc(C'), that are applicable in the case where there is no access to the ground truth latents. We refer to appendix C.4 for a study on the relationship of R² (G) and Ace (G, K), which suggests we can use Acc( G. K) proxy of R2 (G).
5

---

Table 1: Overview. We vary the group G used for the data-generating process, and consider a non-linear mixing through a neural network ("non-linear") as well as the infinite dSprites dataset. We report the R2 on the equivariant part of the embedding, and the Accuracy (in %) on the invariant part of the embedding for identifying the content information.
Group (G)
SOa
O
GLa
Rm. x Zn x Zn
Obs. (f)
non-linear
non-linear
non-linear
indSprites
Metric (x)
R²)
Acc(C)
R (x)
(G)
Acc(C)
R (x))
R²)
Acc(C)
Acc(Gi 5)
Acc(C)
InfoNCE 0.0
0.0
98.9
0.0
0.0
99.1 so.s
0.1 10.15
1.00
-8.5
0.36
99.97 10:0
+LDS 0.0
0.0
99.0.as:
0.0
0.0
99.0.o.s
0.1 10:00
1.0
98.4 :m
0.31 10:0
99. s0
+SLDS 0.0
0.0
s0.0
98.9 .0.7
0.0
0.0
98.7 10.05
0.2
0.0 a.0
97.7
0.28 szoz
99.81 sou27
EbC (lin.) 70.9.am0
54.1
20.2
70.8
19.8 54.0
59.7 28.10
39.8 a:8.0
22.3
FbC 99.7 10.2 99.7 10.25
99.1 10.70
99.8
10.0
99.7 10.0
99.2 mo
99.8 10.05
98.5 00.7
99.91
10.0s
74.04...1
5 Empirical Results
We apply EbC on a variety of group learning settings, summarized in Table | EbC is effective at recovering group structure both in setting where we have precise access of the latent space (for verification), and scales to benchmarking datasets used in objective-centric learning. Across all baselines, only EbC is able to recover the structure with high fidelity: We reach an R² of > 99% in recovering the ground truth latent space on synthetic data, validating Thm Mp and substantially outperforming linear baseline (60-70%) This result verifies that indeed h(xx)
Læ up to linear indeterminacy in practice. We verify the group structure through an R² between the projected sample and the ground-truth for synthetic data with perfect recovery of >98% (Thm [Tb). [T)) As verification, we compare to existing contrastive learning baselines which are well established at learning invariant representations. i.e., the content. Indeed, all considered baselines are effective at recovering the content information with accuracies typically >98% On synthetic data, EbC matches this performance (>98%); on idSprites we encounter trade-off and accuracy drops to 75%.
EbC learns a representation of (Rm x Zn x Znd from image data. We next evaluate EbC on idSprites. An optimal embedding space for the Rm x In x Σn, is 3-torus, with the three individual groups represented along the three circular coordinates. Fig 4 depicts this (assumed) underlying structure of the latent space (which is not enforced in the real dataset or during training), along with the reconstructed embedding space of EbC. Note that the dimensions are related to each other; in Fig. 4 we show the dependency based on the y-, x- coordinate for a fixed angle. Qualitatively, this embedding space was stable also under variations of the content (shape and size of the object).
EbC learns faithful representations of content and style under diverse conditions. Next, we perform a fine-grained quantitative evaluation of the embedding space. In particular, we vary the properties of the indSprites dataset by varying the number of options for the content (shape and size) and the number of variations to learn from (translation and rotation) while keeping the overall dataset fixed. In Figure we outline these options, reporting both the classification accuracy of the content, and the quality of the representation using our kNN metric. EbC recovers the group structure with high Top-5 accuracy close 100% when performing an 1-over-20k kNN lookup for range of number of contents. However, we notice that the content classification declines from around 80% to 20%. We report qualitative results of the KNN prediction in Appendix C.4
EbC is robust to over- parmeearamttiioaatoonn In Figure 41, we vary the output dimension of the encoder (divided into 2 dimensions for the content, and 4-7 for the group). We observe that the group structure preserves a high predictive kNN above 99% and a stable content classification performance above 80%. However, this requires a sufficient number of samples for estimating the implicit representation: For 4x the group dimensionality is required when single transitions are observed, and 8x is required in the case of compound actions. Note that in practice, it is possible to even inform model hyperparameters on this metric, as it is available without any knowledge of the underlying ground truth structure of the data.
EbC identifies group representations of Onn), GL(n), and 50(n). Next, we investigate the applicability to more general group structures. For this, we generate additional synthetic data from more complex groups. SO(n) is close to our example on idSprites, which extended by O(n) and GL(n, R) as fairly general and challenging example (Figure 5) Here we report results for n = 3.

---

B
X
C
D
Acc(G,
Acc (G.5)
Acc (C)
1.00
1.00
0.75
0.50
com
0.25
0.25
com (4)
com (8)
0.00
0.00
com 112)
ind (12)
104
102
102
10a
10a
10a
classes
output dim
output dim
Figure 4: EbC learns faithful representations of group actions. A, possible ground-truth latent space for idSprites. Depicted 2n Z,,- B, the actual embedding obtained by EbC, plotted for fixed value of the orientation angle and colored in two views for variations in and direction. C, evaluation of representation quality and content accuracy across an increasing amount of classes. D, impact of observing the single groups (ind.) vs. a combination at each step in training (com.) across different output dimensions. Number brackets is the number samples in per output dim.
(G(
R²(x)
Acc (G, 1)
Acc (C)
1.0
1.0
1.0
1.0
0.5
0.5
0.5
0.5
0.0
0.0
0.0
0.0
L
3
4
2
12345
4
ELCIMUM
Figure 5: Verification on simulated data. A, traversals through the ground-truth latent space of our synthetic datasets. We manually constructed two orthogonal transformations Qz Q,, perpendicular to each other to obtain sphere. B, recovered latent space after non-linear demixing and optimal linear alignment. C, comparisons of EbC (green) against linear baseline (gray) and an InfoNCE baseline (black) across an increasing number of mixing layers.
in Appendix C.1 we show additional results for higher dimensions. The complexity of the problem is additionally ly varied by generating increasingly non-linear datasets. This is expected to eventually degrade performance when a fixed-size dataset is used. We vary the number of mixing layers from n=2 higher dimensions.
EbC is able to recover both a suitable vector space and an implicit group representation on these datasets with high fidelity. Figure shows the ground truth latents, and Figure shows qualitative impression how the spherical latent space recovered after unmixing. Quantatively, the R2(x) metric in Fig. 5 confirms the quality of this mapping up n=4 mixing layers before obtain degradation. Likewise, forward prediction through the group representation obtains high R2(G) up to 4 mixing layers, substantially outperforming the linear baseline. A relevant metric in practice is Ac)(G) measuring forward prediction through kNN based metric. comparision to the InfoNCE baseline, perform comparatively in identifying the class content across all group types (Acc(C)).
6 Further Analysis, Ablations, and Modeling Choices
Following experimental validation of our model and applicability to image data, we are interested in analyzing the empirical behavior of the loss function in more detail. In practice, an important
7

---

R² (x)
R² (G)
Acc (G, 1)
Acc (C)
1.0
1.0
1.0
1.0
0.5
GLin)
0.5
0.5
0.5
50in
0.0
0.0
0.0
0.0
1
2
3
4
1
2
3
5
1
2
5
6
1
2
4
5
6
dim (group)
dim (group)
dim (group)
dim (group)
R² (x)
R (G)
Acc (G, 1)
Acc (C)
1.0
1.0
1.0
1.0
0.5
0.5
0.5
0.5
0.0
0.0
0.0
0.0
1 2
1
1
dim (content)
dim (content)
dim (content)
dim (content)
Figure 6: Robustness to model misspecification. In the ground-truth process, we allocate dimensions to coordinates affected by the group action, and dimensions to to content. Note, R² metrics reported here not observable on real data, while Ace(G) is observable and Acc(C) is observable if labeled test set is given.
hyperparameter is the parameterization of the embedding space, as well as the setup of the loss function. Further validation and ablation experiments can also be found in Appendix C
Identification of the correct latent dimensionality practice, the true dimensionality the latent vector space is not known, or even ambiguous. Algorithms for learning group structure should be ideally (1) robust to model misspecification and (2) provide a clear hyperparameter validation protocol for choosing the correct dimensionality. In Fig. [6, report the properties of EbC under model misspecification. Both for the content and groups, we consider a 3d ground truth latent space, resulting in total of six latent factors. In Figure [ second row, we show performance variations as we specify the wrong content dimension. Except for a degenerative case for d=1, we obtain close to perfect recovery of both group structure and content.
In the first row Figure 6 we misspecify the group dimensionality, which is more critical. We see a clear peak in 1-NN lookup performance (Acc(G)) as well as saturation for R2(x)/R2(G) prediction performance for d>33, Very crucially, the practically observable metric for hyperparameter selection, Acc(G), shows clear peak at the correct dimensionality (d=3), making it prime target for model selection (which we indeed did the experiments showed so far, cf. the discussion in Appendix B). As before, with the correctly selected dimension, obtain recovery of the latent space and the representation of the group.
performance in both
Analysis of optimal model parameters and hyperparameter selection With our fixed selection protocol, we finally consider model choices for the implementation of the algorithm outlined Sec. In particular, the choice of negatives for the set S is relevant for satisfying the diversity conditions required for Theorem 1. Furthermore, we can extend the loss function to consider the property of an inverse element, i.e., that both the action R X X') and R(X', X) are valid representations. Note, we do not explicitly force the constraint that the respective matrices should be inverses of each other, making our approach still potentially applicable to structures not fulfilling all group axioms.
Table 2 depicts the different choices for both symmetric loss and different negative distributions. On simpler groups, SO(n) and O(n), when our input vectors are normalized, the co-domain of the action will still be the sphere, making less crucial to consider both y and y as negative samples. In contrast, we move to GL(n), it is crucial to use both samples for training to avoid performance drop. On O(n), GL(n), a symmetric loss additionally improves the performance.
7 Limitations
Accuracy trade-off for many classes While EbC is able to convincingly estimate both the latent space and the group actions, could construct failure case in idSprites when class dimensions

---

Table 2: Model variations across different loss functions, negative samples, and groups.
Sym.
S
SO,
On
Gl
Acc(G, 1)
Acc(G, 5)
Acc(G, 1)
Acc(G, 5)
Acc(G, 1)
Acc(G, 5)
x
both
56.23 22.02
77.38 :24.3
86.53
89.75 21.85
93.14 1
99,98
X
28.
421.98 a21.88
83.65
85.49 21.39
53,23 288.79
99.91 so.1e
99, 19.98 s0.01
X
y
73.44
79.52
87.08
95,91 $1.48
96.83
99.98
V
both
94.70
86.95
85,59 28.04
99.83 a0.11
99.98 so.01
99, 99,93 s0.16
/
y
58,55
85.54.254
85.70
87.77 26.78
99.97
99.95 s0.10
y
94.44
84.31 26.99
83.61 :7.88
99.84.0.c0
99.89 20.25
99.85 s0.95
increases to multiple hundreds or more. While we continue to estimate the group representation, the classification performance of the content decreases these cases, even if the dimensionality of the latent space is matched to our baseline. A possible explanation is that EbC needs to implicitly learn a prototype of each class to estimate the coordinates affected by the group action, which might be harder to embed in the network than classification objective alone.
Style vs. content subspaces identifiability Our proposed prior separate content and style also has limitations. A clear separation can only be expected if the group dimensionality is chosen to be minimal. While we demonstrated that such minimal dimensionality can be computed on real- world datasets, it comes with the additional computational burden of hyperparameter estimation, which might be prohibitive in practice. Future work might consider improved versions of our content/style subspace prior and provide theoretical guarantees on the optimal separation. Within the scope of this work, we showed a practical way to estimate the hyperparameters using the Acc(G) metric in the analysis section.
Real-world data and baselines Although we are the first show that identifiable group repre- sentation learning from unlabeled observational data is feasible at all, our empirical analysis can be substantially extended. Firstly, on selected previous setups, more baseline comparisons could be conducted which is here limited due to the lack of extensive benchmarks. It will be particulary interesting to see how the algorithm scales to to full-scale image datasets like 3DIdent [56]. However, we do report additional experiments on real-world data in Appendix C.5
Scaling properties, empirical vs. theoretical Our theoretical result mandates the of d 1 examples for estimation of the embedding space, and currently does not make a claim how convergence is influenced by limited data. In practice we observed that substantially more than + 1 samples are required (about 6-8x gave good empirical results). In this light, our theoretical results can be extended include bounds on the behavior with limited samples, although challenging and not every common in existing identifiability work. On the other hand, replacing our out-of-the-box least squares estimator by more stable and more adapted algorithms might further close the gap the theoretical optimum. We report additional results on data efficiency in Appendix C.2
8 Related Work
Learning equivariant representations Early work in equivariant representation learning focused on designing neural network architectures that are explicitly equivariant predefined class of transformations 33 10
42 (53) These methods were powerful but require expert knowledge to design an inductive bias specific to particular groups Inspired by the success of invariant self- supervised learning (I-SSL) methods 13 37 [4], a new paradigm emerged that sought to learn cquivariant representations directly from data. The first wave of SSL-based methods assumed full knowledge of the group actions. Latent representations were learned either via encoder-only frameworks (23
27
31
39 Equivariance was encouraged either by predicting the parameters of the group action or enforced by explicitly The most recent advances tackle the more challenging problem of learning equivariance without explicit knowledge of the group approaches 28 35 36
frameworks 44] 12 38 or via autoencoder-bast
modeling the effects of group actions in latent space [2 23 24
38
39
actions. Both encoder-only approaches [15] 15 [2) and autoencoder-
have been proposed. The central innovation of these methods is that the representation of group actions R(g) is learned directly from data pairs (y, g y. For example, CARE (5] is an encoder-only,
contrastive learning based approach with implicit R(g) O(n). STL 52 also uses encoder-only

---

approach but enforces non-linear equivariant relationships ø(g y) - (( NFT 128 adopts an auto-encoder framework with explicit linear group representations R(g) E GL(m). While these methods mark important progress, they each leave open critical challenges: some are restricted to special classes of group representations, some require generative modeling of the input space, and none provide formal identifiability guarantees of the applied algorithm. While Koyama et al. [28] are the first to show that the problem itself is identifiable (i.e., there exists non-trivial G-equivariant map Y + X unique up to G-isomorphisms of the embedding space), they do not provide identifiability guarantees for the learning method itself.
Identifiable nonlinear ICA In nonlinear ICA, observed variables y assumed to be generated from latent variables via an unknown nonlinear "mixing" function f. The goal is to recover f f-,, or equivalently, estimate the latent variables æ from the observed variables y. Hyvärinen and Pajunen [20] showed that in contrast to linear ICA, the nonlinear ICA problem is, in general, unidentifiable. Subsequent work has established that identifiability can be recovered by making additional assumptions about the data-generating process [[8] 19 21.. More recently, this theory has
been connected to contrastive learning [40 [56). demonstrating that: (a) the choice of the conditional distribution of positive samples encodes the assumptions about the data-generating process, and (b) optimizing the InfoNCE loss [6 37 recovers solution to the nonlinear ICA problem. The identified representation is unique up to an indeterminacy that is directly determined by the assumed conditional distribution. Building these advances, Laiz et al. [30] proposed DCL, which identifies latent representations under the assumption of linear or switching linear dynamical systems. Since G-equivariant representations also rely sequential data, there exists a natural connection between DCL and the equivariant representation learning methods discussed above. Our work leverages this connection, combining the identifiability guarantees of nonlinear ICA with the goals of equivariant representation learning.
linear relationship between (x, xx) or (æx, 12+1)) in
Conclusion
We proposed the first identifiable learning algorithm for group representations from unlabeled data without relying on generative models. We demonstrated, theoretically and empirically, that latent spaces of datasets with underlying group structure can be estimated from observational data, Crucially, the assumptions of our algorithms match datasets encountered in engineering and science.
The supervisory signal merely requires knowledge that "the same" action is applied to a collection of samples. We confirmed that under these settings and mild overall assumptions variety the dataset, few of such pairs are sufficient to estimate the underlying ground truth latent space in which the group actions are faithfully represented applicability of this approach especially in computer vision (e.g., for -guur--rroud segmentation), robotics (for learning affordances and planning), and biology and biomedicine (for modeling the effects of longitudinal data and perturbation effects), among other fields.
sufficient
linear transformation. We see wide
Acknowledgments and Disclosure of Funding
We thank Susanne Keller for in-depth discussions about the model setup and related work. We thank Luisa Eck for discussions on the theory. We thank the anonymous reviewers and the area chair at NeurlPS 2025 for their constructive feedback improve our manuscript. This work was supported by a Google PhD fellowship to StS. This work was supported by the German Research Foundation (DFG): SFB 1233, Robust Vision: Inference Principles and Neural Mechanisms, project number: 276693517 and by Open Philanthropy Foundation funded by the Good Ventures Foundation. MB is a member of the Machine Learning Cluster of Excellence, funded by the Deutsche Forschungsgemminschatff (DFG, German Research Foundation) under Germany's 's Excellence Strategy EXC number 2064/1 Project number 390727645. This work was supported by the Helmholtz Association's Initiative and Networking Fund on the HAICORE@N KIT and HAICORE FZJ partitions.
Author contributions
TS: Methodology, Software, Investigation, Writing-Editing StS Conceptualization, Methodology, Formal analysis, Writing-Origine Draft, MB: Conceptualization, Writing-Editingg
10

---

References
[1] James U. Allingham, Bruno K. Mlodozeniec, Shreyas Padhy, Javier Antorán, David Krueger, Richard E. Turner, Eric Nalisnick, and José M. Hernández-Lobato. A Generative Model of Symmetry Transformations. Advances in Neural Information Processing Systems, 37:91091- 91130, December 2024. URL /ttpsttps///pppoeeedinng..ooeee
httpp//aaww.eerereppppppppppppep html
[2] Ting Chen, Simon Korbblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597-1607. PMLR, 2020.
[3] Taco
Cohen and M. Welling. works. ArXiv, February 2016.
Group Equivariant Convolutional Net-
URL http:///wwww..eeatttcccccccorrrr org// faf Bttaafoesssssccccaaaaaclscooaaa.
hatp:r/rroorpooiiiaaaaaaaoooollr -heeraaaeeeeeeteteoroooeeeooeeaa Welling
[4] Rumen Dangovski, Li Jing, Charlotte Loh, Seungwook Han, Akash Srivastava, Brian Cheung. Pulkit Agrawal, and Marin Soljacic. Equivariant Self-Supervised Learning: Encouraging Equivariance Representations. International Conference on Learning Representations, October 2021. URL htRL/htttps://peeeeviiee nwt nett/forum/ddggKLAE...
[5] Nima Dehmamy, Robin Walters, Yanchen Liu, Dashun Wang. and Rose Yu. Auto- matic Symmetry Discovery with Lie Algebra Convolutional Network. In Advances Neural Information Processing Systems, volume 34, pages 2503-2515. Curran As- sociates, Inc., 2021. URL https: https::/pproceedinggs neurips cc/ http//ppe/2222//s///// AbbS-AAttttttttt..... html
[6] Emily L Denton and vighnesh Birodkar. Unsupervised learning of disentangled representations from video. In I. Guyon, U. Von Luxburg, Bengio, H. Wallach, R. Fergus, S. Vishwanathan. and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL 2017/file/2d2ca eedf
http://prooeeddnnnsee..e.rrg www.neurii....
pddf
[7] Alexandre Devillers and Mathieu Lefort. EquiMod: An Equivariance Module to Improve Visual Instance Discrimination. In The Eleventh International Conference on Learning Representations, September 2022. URL
net
[8] Sebastian Dziadzio, Çağatay Yildız, Gido M van de Ven, Tomasz Trzciáski, Tinne Tuytelaars, and Matthias Bethge. Infinite dsprites for disentangled continual learning: Separating memory edits from generalization. arXiv preprint 1rXiv:2312.16733 2023.
[9] Cian Eastwood, Julius Kügelgen, Linus Ericsson, Diane Bouchacourt, Pascal Vincent, Mark Ibrahim, and Bernhard Schölkopf. Self- sepf-Super Disentanglement by Leveraging Structure in Data Augmentations. In Causal Representation Learning Workshop NeurlPS 2023, October 2023. URL / net Let/forumiiddoolaaat
[10] Marc Finzi, S. Stanton, Pavel Izmailov, and A. Wilson. Generalizing Convolu- tional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data. ArXiv, February 2020. URL https://wwel semanticscholes www.cholaar.rg/pppee.... org/paper/ Neurall Networrs for co-orr-o-Finiiii Finzi SStaittntttttnnnno 45344400666444000555400000062663
[11] Sanket Gandhi, Atul, Samanyu Mahajan, Vishal Sharma, Rushil Gupta, Arnab Kumar Mondal, and Parag Singla. Learning disentangled representation in object-centric models for visual dynamics prediction via transformers. CoRR, abs/2407.03216,, 2024. Uttpttts: org/
10. 48550/arXiv 2407. 2407.03216
[12] Quentin Garrido, Laurent Najman, and Yann LeCun. Self-supervised learning of split invariant equivariant representations. In Proceedings of the 40th International Conference Machine Learning, volume 202 of ICML 23, pages 10975-10996, Honolulu, Hawaii, USA, July 2023. JoLR.

---

[13] Jean-Bastier Grill, Florian Strub, Florent Altché, Corentin Tallec. Pierre H. Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, Rémi Munos, and Michal Valko. Bootstrap your own latent new approach to self-supervised learning. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS '20, pages 11271-21288 Red Hook, NY, USA, December 2020, Curran Associates Inc. ISBN 978-1-71382-95444
[14] Andres D Grosmark and György Buzsáki. Diversity in neural firing dynamics supports both rigid and learned hippocampal sequences. Science, (51(6) 1400::::4440000 1440-14444444 1443, 2016.
[15] Sharut Gupta, Joshua Robinson, Derek Lim, Soledad Villar, and Stefanie Jegelka, Struc- turing Representation Geometry with Rotationally Equivariant Contrastive Learning. In The Twelfth International Conference on Learning Representations, October 2023. URL
https://openrerv/// httpreeeiwe..eeeefoormmm//taaaae
[16] Michael Gutmann and Aapo Hyvärinen. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statisticcs, pages 297-304. JMLR Workshop and Confer- ence Proceedings, March 2010. URL http:tttss////pceddddaa mmr.innnsmm.......mmmmmmmmmmmmmm www.pppess//ee prress/9//gs hwtg9/gutmannnn. 10a
html
[17] Lukas Heumos, Anna C Schaar, Christopher Lance, Anastasia Litinetskaya, Felix Drost, Luke Zappia, Malte D Lücken, Daniel C Strobl, Juan Henao, Fabiola Curion, et al. Best practices for single-cell analysis across modalities. Nature Reviews Genetics, 24(8):550- 572- 2023.
[18] Aapo Hyvarinen and Hiroshi Morioka. Unsupervised feature extraction by dime-contrastiog learning and nonlinear ica. Advances in neural information processing systems, 29, 2016.
[19] Aapo Hyvarinen and Hiroshi Morioka. Nonlinear ica of temporally dependent stationary sources. In Artificial Intelligence and Statistics, pages 460-469, PMLR, 2017.
[20] Aapo Hyvärinen and Petteri Pajunen. Nonlinear independent component analysis: Existence and uniqueness results. Neural Networks: The Official Journal of the International Neural Network Society, 12(3):29--439 April 1999. ISSN 1879-2782, doi: 10. 10.1016// (0.111///0893-6080000
40-3.
[21] Aapo Hyvarinen, Hiroaki Sasaki, and Richard Turner. Nonlinear ica using auxiliary variables and generalized contrastive learning. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 859-868. PMLR, 2019.
[22] Aapo Hyvärinen, Ilyes Khemakhem, and Hiroshi Morioka. Nonlinear independent com- ponent analysis for principled disentanglement in unsupervised deep learning. Patterns, (10):100844, October 2023. ISSN 26663899. doi: 10. 10.10166/..atter222233 100844. URL
httts:://inkingg wwwpengghu...e.eeelee..e. elsevier com/ wtw.:iier...oretrreeeeeeeee http:///ww....ooeoe////////////.
[23] Yinzhu Jin, Aman Shrivastava, and T. Fletcher. Learning Group Actions on Latent Repre- sentations. Advances in Neural Information Processing Systems, 17:127273-127221 Decem-
ber 2024. URL http:////pprooeddinnnnnoooeieses nwwrips. http://iinnnnunuuuieeeeepa/ppppp
/004///ar//222/hhh html
[24] Hamza Keurti, Hsiao-Ru Pan, Michel Besserve, Benjamin F Grewe, and Bernhard Schölkopf. Homomorphism autoencoder-leaminng group structured representations from observed transi- tions. In International Conference on Machine Learning, pages 16190-16215. PMLR, 2023.
[25] Diederik P Kingma. Adam: A method for stochastic optimization. arXiv preprint
1rXiv:1412.6980. 2014.
[26] David A. Klindt, Lukas Schott, Yash Sharma, Ivan Ustyuzhaninov, Wieland Brendel, Matthias Bethge, and Dylan Paiton. Towards nonlinear disentanglement in natural data with temporal sparse coding. International Conference on Learning Representations, 2021. URL hhtpps
//openreview net
12

---

[27] R. Kondor and Shubhendu Trivedi. On the Generalization of Equivari- ance and Convolution in Neural Networks to the Action of Compact Groups. ArXiv, February 2018 URL https://ww semanticscholar org/paper/ Sewpraaooeoe-uuuuaaaaaeeeeeeee in- Lenedddi--eural--ee Kondor-Trivedi
On- teetthe-Geeeeee
[28]
Masanori Koyama, Kenji Fukumizu, Kohei Hayashi, and Takeru Miyato. Neural Fourier Trans- form: General Approach to Equivariant Representation Learning. The Twelfth International Conference on Learning Representations, October 2023. URL ftttps://opereeete net./
#orum2id==@0CAA8ixt1
[29] Lucas Laird, Circe Hsu, Asilata Bapat, and Robin Walters. MatrixNet: Learn- ing over symmetry groups using learned group representations. Advances in Neural Information Processing Systems, 37:32512-32535, December URL
2024.
https: https://prrcceedingss http://oooeeddss..eeeeerrrrppppp ctt.pps.cc/paae files/llaaaee 391 35777551111000e117ca00007444-4-3 Conforence. wttee//ppppr222///aaaaee
[30]
Rodrigo González Laiz, Tobias Schmidt, and Steffen Schneider. Self-supervised contrastive learning performs non-linear system identification. In The Thirteenth International Con- ference on Learning Reprexentations, 2025. URL https::/ net/ nww/ffreufffiiee-eooo
ONFWFIuZBI]
[31] Yann LeCun and Yoshua Bengio. Convolutional networks for images, speech, and time series. In Michael A. Arbib, editor, Handbook of Brain Theory and Neural Networks, page 3361. MIT Press, 1995.
[32] Dominik Lorenz, Leonard Bereska, Timo Milbich, and Bjor Ommer. Unsupervised part-based disentangling of object shape and appearance. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
[33] Loic Matthey, Irina Higgins, Demis Hassabis. and Alexander Lerehner. dsprites: Disentangle- ment testing sprites dataset. http:///ddbbbbcoo////eemm//////p
2017.
[34] Francesco Mezzadri. How to generate random matrices from the classical compact groups. 2007. CtRLhhtts:////et /tttps://ariiv. www.aaiwww/....aaa.......e.ooore wwtp:/maathhhh/0000a000
[35] Thomas W. Mitchel, Michael Taylor, and Vincent Sitzmann. Neural Isometries: Taming Trans- formations for Equivariant ML. Advances in Neural Information Processing Systems, 37:7311- 7338, December 2024. URL http:::
nww. http:cccccppppeeeeeeeeeeeeeeeeee 2024/haa http:///20092fteeeaaaaafaaaeaaa3
html
[36] Takeru Miyato, Masanori Koyama, and Kenji Fukumizu. Unsupervised Learning of Equivariant Structure from Sequences. In Advances in Neural Information Processing Systems, October 2022. URL htt.:ttst//ppeeeeiiiieeee
[37] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint 1rXiv:1807.0377 03744, 2018
[38] Jung Yeon Park, Ondrej Biza, Linfeng Zhao, Jan- Willem Van De Meent, and Robin Walters. Learning Symmetric Embeddings for Equivariant World Models. In Proceedings of the 39th International Conference on Machine Learning, pages 17389. PMLR, June 2022. URL
https://proceedingg.uu mlr press/vv622/e https///v12ppaaka22.aa html
[39] Guo-Jun Qi, Liheng Zhang. Feng Lin, and Xiao Wang. Learning Generalized Transformation Equivariant Representations Via AutoEncoding Transformations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(4):2045-200 2057, April 2022. ISSN 1939-3539. doi: 10. 0009TTPAML.202233022988 URL https:hts::// pttttp::/iieexplorree leee. arg./occument/921922
[40] Geoffrey Roeder, Luke Metz, and Durk Kingma. On linear identifiability of Iearned rep- resentations. In International Conference on Machine Learning, pages 9030-9039, PMLR, 2021.
[41] Anastasiia V Sadybekov and Vsevolod Katritch. Computational approaches streamlining drug discovery. Nature, 61667958))673-665, 2023.
13

---

[42] Victor Garcia Satorras, Emiel Hoogeboom, and M. Welling. E(n) Equivariant Graph
Neural Networks. ArXiv, February 2021. URL https://ww http://www.smaniicsscccccccho./e
org prgg/aape//((n) Equivariant-
Coouual-Nettinn Netaaaaattettss---rrrrr------ooe Ba985555777860000006777766666633
[43] Steffen Schneider, Jin Hwa Lee, and Mackenzie Weygandt Mathis. Learnable latent embeddings for joint behavioural and neural analysis. Nature, 617(7960):3603368, 2023.
[44] Steffen Schneider, Rodrigo González Laiz, Anastasiia Filippova, Markus Frey, and Macken- zie Weygandt Mathis. Time-series attribution maps with regularized contrastive learning. arXiv preprint arXiv:2502. 12977 2025.
[45] Anne E Urai Brent Doiron, Andrew M Leifer, and Anne Churchland. Large-scale neural recordings call for new insights to link brain and behavior. Nature neuroscience, 15(1): 2022.
[46] Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weekesser, Jonathan Bright, Stéfan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern, Eric Larson, C Carey, ilhan Polat, Yu Feng, Eric W. Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, Ian Henriksen, E. A. Quintero. Charles R. Harris, Anne M. Archibald, António H. Ribeiro, Fabian Pedregosa, Paul van Mulbregt, and SciPy 1.0 Contributors. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods, 17:261-272, 2020. doi: 10. 1038//41152222 1522201--921-6--222
[47] Julius Von Kügelgen, Yash Sharma, Luigi Gresele, Wieland Brendel, Bernhard Schölkopf, Michel Besserve, and Francesco Locatello. Self-supervised learning with data augmentations provably isolates content from style. Advances in neural information processing systems, 34: 16451-16467, 2021.
[48] Robin Winter, Marco Bertolini, Tuan Le, Frank Noé, and Djork-Amé Clevert. Unsupervised learning of group invariant and equivariant representations. Advances in Neural Information Processing Systems, 35:31942-3188 31956, 2022.
[49] Tete Xiao, Xiaolong Wang, Alexei A. Efros, and Trevor Darrell. What Should Not Contrastive in Contrastive Learning. In International Conference on Learning Representations, 2021. URL
httpp:://ppenrviiee.eeet www.neeeoooomnddd//eaa.a...
[50] Jianke Yang, Robin Walters, Nima Dehmamy, and Rose Yu. Generative Adversarial Symmetry Discovery. In Proceedings of the 40th International Conference on Machine Learming, pages 39488-39508. PMLR, July 2023. Uttpttts:://////t hhttpss:/prooceeinggat mll. .wresspee////.e prress/v022/ v2////202//aaa/ yang23n html
[51] Thomas Yerxa, Jenelle Feather, Eero Simoncelli, and Sue Chung. Contrastive- Equivariant Self-Supervised Learning Improves Alignment with Primate Visual Area
IT. Advances in Neural Information Processing Systems, 37:96045-96070, Decem- ber 2024. URL
html
[52] Jaemyung Yu. Jachyun Choi, Dong-Jae Lee. HycongGwon Hong, and Junmo Kim. Self-supervised Transformation Learning for Equivariant Representations. Advances in Neural Information Processing Systems. 37:83068 -30900, December 2024. URL https::/proceeddingg neurips. cc/ httppsscccppppprpeeeeeeee httpc/pppee.eepepapppeeeeeeeeeee wtwpee_fileee/apr//222222/ ppppp///22//aash//// 97707770994400133001111205555553 Abstrest-ttttttt-C-caoat Conference html
[53] Manzil Zaheer, Satwik Kottur, Siamak Ravanbhakhsh, Barnabás Póczos, Ruslan Salakhutdinov, and Alexander J Smola. Deep sets. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS' 17, page 3394-3404 Red Hook, NY, USA, 2017.
Curran Associates Inc. ISBN 9781510860964.
[54] Jure Zbontar, Jing, Ishan Misra, Yann LeCun, and Stephane Deny. Barlow Twins: Self- Supervised Learning via Redundancy Reduction. In Proceedings of the 38th International Conference on Machine Learning, pages 12310-12320. PMLR, July 2021. URI, https //proceedings. mlr, Tteess/v13//bbottr220 html
14

---

[55] Ding Zhou and Xue-Xin Xin Wei. Learning identifiable and interpretable latent models of high- dimensional neural activity using pi-VAE. In H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 7234-7247. Curran Associates, Inc., 2020. URL http:://pppoeeedngggs.... newriis............... ec/ Paper.ddf.
paper files/ httpee//apprrr//////////////////
[56] Roland S. Zimmermann, Yash Sharma, Steffen Schneider, Matthias Bethge, and Wieland Brendel. Contrastive learning inverts the data generating process. Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 12979-12299 12990. PMLR, 2021.
15

---

Supplementary Material
Proof Theorem
Extended canonical discriminative form
18
A.2
Properties of the implicit group representations
19
A.3 Proof of Theorem
19
B
Additional Experimental Details
21
B.1
Experimental protocol
................................
21
Hyperparameter Selection
................................
22
B.3
Third-Party License Information
................................
22
B.4
Compule Resources
................................
22
C
Additional Experimental Results
23
C.I
Recovery of group structure for n > 3.
23
C.2
Data efficiency
................................
23
Robustness to noise
................................
25
Infinite dSprites
................................
26
C.5
Application real-world data
29
D Additional Literature Discussion and Related Work
33
NeurIPS Paper Checklist
35
16

---

A Proof of Theorem 1
In this chapter, we provide proof for Theorem | in the main paper. We assume the following data generating process:
Definition 1 (Data generating process). Let æ V C R be a vector describing the ground truth latent components, let be an element of a group G, and let f V + IR_ be an injective map. Assume that for each group element g. we have least M pairs of samples (y fj)) with
y, f(x,),
y - (((())
i= 1, i...,.., M
(15)
where R: G GL(d, R) is representation of G on V.
If we think of æ containing an invariant part (the content), the theory by Von Kügelgen et al. 47 already covers the case for identifying the content component of æ when considering the group actions as augmentations. We will therefore not explicitly discuss the discovery of the invariant part of the embedding, but focus our following statements on learning equivariant representations.
We extend the canonical discriminative form introduced by Roeder et al. 40 to arrive at the form
exp (u(x) v (v) + a(() (a() (())
Pu, x. S) Egres exp (u(x) v(y') + a(x) + ((y)))
(16)
The functions u, v Rd + Rd a, B Rd + R can have shared underlying structure (e.g., are parameterized by the same neural network) and are not necessarily independent. We extend the diversity conditions [40]:
Definition 2 (Diversity Conditions, adapted from Roeder et al. 40).. Assume dataset of tuples (xx y. 9) and let Pdsta( y, S) be distribution over this (possibly discrete) dataset. We then require the following properties:
For any æ; S we can find at least N pairs of points (y., yB) with Paata YA yA. > 0 and Pdata (z, yB, S) > 0 such that the finite differences v3y -vyy') are linearly independent.
2. For any y. we can find at least N pairs of points a, a and their sets of negative samples SA, Sn such that E SA 0 Sn is in the negative sets for both examples, and the finite differences u(x))) uu(x))) are linearly independent.
Both assumptions need to be assured through design of the sampling procedure and initialization of the networks. In particular, the first condition requires that sufficient variation present among negative examples. Similar to Roeder et al. [40) since v is randomly initialized neural network, expected that the variability is met as long as variability in the ground truth factors is given. The second condition requires that negative samples sufficiently often co-occur with enough variation in the reference examples. In addition, this condition requires R be full-rank, which ensure by adding at least d samples for estimating a x d matrix.
We can now re-state the theorem and discuss the proof strategy. Please note that we relate the embedding space according to the commutative diagram in Figure 1: We consider the relations between samples in the original vector space V vs. the recovered factor space, which is the co-domain V. We can state:
of first applying the mixing and then the de-mixing functions, h = o h:
Theorem (Identifiability of the Group Representation). Assume the following:
(1) f is an injective mixing function, E G is group element according to Def. 3.
(2) The model Φ IRD + V satisfies the diversity conditions in Def. 4.
(3) is representation of G, and R is the implicit representation of the group (Eq. 2) such that
R(g) for pairs of transformed samples (X,9X) and group actions
G.
Define h & of Then, for matching conditional distributions pa = Pr-1, for all points and group actions in the dataset:
(a) We recover the original vector space h(x)) Læ, up to ambiguity L GL(d).
(b) We recover a representation of the group, R(h(X),, h(gX)) LR1
17

---

As outlined in the main paper, the proof proceeds as follows: First, we extend the canonical dis- criminative form of Roeder al. 40 towards more general setting, giving affine instead of linear identifiability (AA.T). Then, we show that our implicit group representation admits ($A.2). Finally, we leverage these results to arrive at the indeterminacies reported in the theorem A.33
A.1 Extended canonical discriminative form
The following proposition is only a slight modification from Eq. in Roeder et al. The key difference is the introduction of the scalar potential functions a and 8. making the canonical discriminative form more flexible. In particular, now admits a mean squared error loss function.
Under these assumptions, we can state:
Theorem (Generalized Canonical Discriminative Form). Let u, V R - a, 3 R. R be functions satisfying Def. 2 In particular, u. v, a, 3 can have shared underlying structure and are not necessarily independent. Then, the probabilistic model of the form
exp (u(xr) v(y) + a(x) + B(y))
Pu. x. Eyes
Eyes exp (u(x) v(y') a(x) + B(y'))
(17)
with y E S is identifiable up to affine indeterminacy, ie., for two models (u', ,',''','''' ,',3' and (u" v², of, ,8)) and their respective distributions (p',2" "y',""", we have for all a, y with Paata (z. y, S) > 0,
[u'(2) (x) c
p'=p"
(18)
(1) Bo" (y) +
for two invertible d matrices A and B and vectors c, d E Rd.
Proof. The proof technique adopts the strategy from Roeder et al. 40) for the indeterminacy of u. and re-applies this the indeterminacy of u. The approach considers finite differences on the log probabilities for p.p" with
log logplylx, u(x) vly) + a(x) + B(y) 2(x S)
(19)
= =xx) (yy) + a(x, S) + (
(20)
Part 1: Affine identifiability of u. For any æ in the data distribution, and corresponding S, select two points YA. yB E S. For the points, it holds that
log p(g A lz, S) log [z, S) = log p" (y^ S) - log (yn| S)
(21)
Let AVAB == (yA) v vy AB = 3(yA) - ((yB) to obtain
Au A + ABAB = (Avin) u" (a) + A
(22)
By assumption (1), we can find such pairs with linearly independent AUAB, Av, which lets us re-write the equation using the full-rank matrices L, L
L (()) + c (L")Tu" (x)
(23)
u(x)) =
(c" (c") cc
(24)
which requires the map from u to u to be affine, concluding the first part of the proof.
Part 2: Affine identifiability of v. For a given y. find a HB, SA. SB such that y SA n S and note that SA = SB is possible but not required. Then we consider the finite differences:
log pby a. AA) logp 5B) = log ( ( SA) - log pogp"t (y|æB,S))
(25)
Let AU := u< u(xB), Ad == a( S) a S) to obtain
Au)Bo(y) + AdaB (Au T 00 (g) A
(26)
Stacking multiple conditions gives
L TT + (L*) 00' (y) +e-
(27)
( = (L* n' (1) (c' - c)
(28)
which requires the map from to v² to be affine, concluding the proof.
0
18

---

A.2 Properties of the implicit group representations
We next consider properties of the implicit group representation defined in Sec.
2 given by
x(X.X)= min REGI(())
|X' 1X'-X''I
1 +
R(x,
= (X X) (x X').
(29)
Lemma (Equivariance of implicit group representations). Let X,X n. Define R: Rmxxn x Rmxx _ RRxx as
X E R m
m 2 n have rank
R X') arg min |X
XL
(30)
and let f(X) = XAT + 1m b be an invertible affine transform applied to each row of X, then we have
R(f(X), = A
(31)
Proof.
The solution to the least squares problem can be re-written using the normal equations,
R(X, X') arg min L
||X
XL 1 + X X) x
x'
(32)
Let X XAT and X X'AT. Then, inserting the definition of gives
= arg min
||
X + 1. -XL T 1 b 11
(33)
L
arg min X XL T 11
(34)
which is equivalent to solving the normal equations
(x*)) ((((()) X
(35)
Substituting back in terms of X and A:
AX XA f(X))) A
(36)
A is invertible by assumption and it follows
x x A (((()) - x
(37)
Note that TTX X E R2x has full rank by the assumption on X. Then, we get
A (((((X)f(XX)) A- (x - 4x x(X
(38)
Taking the transpose and re-arranging yields
f(F((), = A x X
(39)
concluding the proof.
A.3 Proof of Theorem 1
Proof. We have pa Pr-1. After rewriting Eq. 3 in terms of x, X, X' instead of y f(x) Y f(X), Y' = f(X)), our model follows the generalized canonical discriminative form (Proposition) with the following parametrization for the data likelihood pr-1:
(x, X, X') = R(X, X x = R(g)æ
(40)
4² (ga) = gar
(41)
To specify the model likelihood we introduce the shorthand h =- e o f which maps samples from the ground truth latent space the recovered latent space (e.g., the composition of the data generating process and feature encoder) and obtain
u (x, X, X') R(b(X), v'(') h(x")
(42)
(43)
19

---

By assumption Ps = Pr-1. From Proposition 2 then follows that
u' ax, x x') = Au" (a. X,X') d'(x') = Bvn (x') + d.
X') + c.
(44)
(45)
where A, B E GL.(d) and c, d E IRd Inserting model and data generating process yields
h(o") Bæ' + d
(46)
((h(X),h(X))
A
(47)
Since h is affine, we insert the first into the second equation and invoke Lemma to arrive at
X (Bx + +d) AR(X, X')x +e
(48)
and re-arranging yields
BR(X, x BR = X +
(49)
(B - A F = c BR X, d'
(50)
This is an equation the form Ua = u. Assuming that we observe x such that matrix collecting all has full rank, i.c. all latent dimensions vary, it follows that U 0, 0, hence
B A),(XX x') =0
(51)
(52)
and inserting R(X,X') -R(g) gives
(B A)R(gg) =0
(53)
= BR(g)d
(54)
Since R(g) has full rank, from the first equation it follows that A = B. For the second equation, since the left hand side is independent of g and all matrices have full rank, can only admit the trivial solution where c 0. follows that
h(x) Bx E
(55)
R(((X), b(x)) = X')
(56)
for an invertible matrix B, which concludes the proof.
20

---

B Additional Experimental Details
In this section, we provide further details on the algorithm and other experimental details. As an extension of this section, we would like to point to our figure and results repository httpe//ab.coomddmmmmcaam---aiiia
Schmidt) as well as to our code repository as the reference for all experimental details. The figure repository contains the code to plot the figures, but more importantly, also contains all the raw results and all training and data generation parameters of all model runs that went into the respective figures. Each of those parameters can easily be traced back to their definition in the main code base repository.
B.1 Experimental protocol
Here we describe the default dataset generation and training protocol of our experiments first, and then provide additional details for any deviation from these default parameters that may exist in any of the figures.
B.1.1 Dataset Generation
Synthetic Group Data We consider three types of synthetic datasets following the data generating process of Theorem [ Group elements are taken from a subgroup of the special orthogonal group (n),,, the orthogonal group (n) and the general linear group (L)::
1. We sample random group elements R(g) pG(g) from G E (S0(m), O(n), GL())).
2. We sample m (with m > n) random vectors x: E S" from the n-dimensional unit sphere.
3. We sample the content component c E C from a finite set of vectors C' c Sd.
4. From (R(g), {m e) we generate Y (y: where y; = f c) and y [(Rgg)æ; c).
5. We repeat this sampling procedure until we reach the desired dataset size.
To sample from the (n) and (O(n) group, we sample from the Haar distribution making use of the SciPy (46] package. To sample from GL(n) we implement a rejection sampling procedure. We first generate a random E R=*a matrix with values 0t.y from a standard normal distribution. Then we compute the determinant (At( & the "identity error" Lerr |I AA-1F and we reject A if [A)] < 0.2 or (et(A) > 10 or leer > 10 10-3, otherwise we accept. The nonlinear injective mixing function f parameterized by random 3-layer MLP and sampled via rejection sampling to fulfill the injectivity assumption as is done in [18 30). The MLP is followed by a random linear map 50-dimensional space. For the full dataset, we sample a maximum of 1000 matrices R(g) and total of 1M pairs (y, y'), ). Unless stated otherwise, we choose n = 3, d = 3, |C| = 100.
Infinite dSprites We leverage infinite dSprites g idSprites], extension of dSprites 33 for validation on more diverse visual data. infinite dSprites an extension of the original dSprites dataset, which allows generate variations of dSprites dataset. We subsample all images 64 x 64. The dataset allows rich variations of the content and ensures that the resulting objects do not have any symmetries. By default we use the same number of varying factors as in dSprites, namely random shapes of 6 different sizes. For our purposes we consider the combination of shape and size the content. Each of these objects may then be oriented in 40 different ways or have one of 32 x x or y position translations. We consider the orientations and translations to be our groups of interest and sample the data such that we get closed groups By default, we sample the dataset such that any paired sample (y, y/) is undergoing a combination of these three types of transformations.
resolution of
B.1.2 Dataset Split
Before training on the synthetic group dataset, we perform a hold-out split in terms of the group actions 9, into training, validation, and test dataset such that we get a 80/10/10 split of the group actions. For idSprites, we randomly sample approx. 20k group actions during training and sample another approx. 90k group actions for validation and test respectively. When computing metrics that require to fit another model, such as a KNN, Linear Regression or Logistic Regression model, we split the validation test dataset again 50/50 and report the metric on the hold-out set.
21

---

B.1.3 Model Training
Estimating R(g) To estimate R(X, X') use k n additional samples where is the assumed group dimensionality of the embedding space and is the samples per action factor. We fit R(X, x') via linear least squares (PyTorch) 's' "gels" solver). By default, we do not compute gradients through the linear least squares estimation to save on compute time. By default for idSprites we k 12 and for the synthetic group dataset we use k = 4.
Encoder Model For the feature encoder é we of 512 for idSprites and 128 for the synthetic group dataset. The output dimension the MLP is where is the assumed group dimensionality of the embedding space and d the assumed content dimensionality. Before passing the idSprites images to the MLP we scale them to [-1, and flatten the image. By default for idSprites we assume group dimensionality dimensionality 2. For the synthetic group dataset we use the same dimensionalities n. and d as are used for the data generation.
MLP with three layers and hidden unit size
and content
Loss Function & Optimizer As loss we minimize the negative mean likelihood (Eq. 3 and for
each sample compute the loss in both directions L(y. y') and L(y', y)). As negatives S. sample uniformly from all available samples in the dataset. We use 1024 positive pairs and 16k (2 negative samples in each batch and train each model for 20k iterations. We train the encoder via gradient descent using the Adam optimizer 25 with a learning rate of x 10
Error bounds For our main results, compute error bounds or standard deviation across our metrics, we generate each dataset with three different seeds and fit each model three times with different seeds during training. For variations and ablation experiments extending main results, we only fit a single model across the three different dataset seeds. We refer to the Figure & Results repository for details on the exact number of seeds for every experiment.
B.2 Hyperparameter Selection
We use the Aco(G, 1) metric for any and all hyperparameter selection. since this is an observable metric and good proxy for the identifiability metries R2 (G) (see Figure () and, by extension, for R² (x)). Additionally, when we perform explicit hyperparameter selection and do not present results from experimental variations test set instead of the metrics from the validation set, which were used for hyperparameter selection. Otherwise, show the metrics the validation set. Again, we refer to the Figure and Results repository for more details.
ablation studies, display the metrics computed the
B.3 Third-Party License Information
We used the infinite dSprites (idSprites; () dataset available httpp://gitthb..cmm///bb// idsprites in the pip-installable version v1.0.1 (MIT License).
B.4 Compute Resources
Experiments were carried on a compute cluster with A100 cards with 40Gb VRAM. On each card, we ran 2-6 experiments simultaneously, depending on the dataset size. The run time for individual experiments trained for 20k steps varied between approximately 10 minutes and one hour depending on the experiment configuration. the most important factors that impact the train time being batch sizes, specifically the number of negative samples, and the k n number of samples used to fit the linear least squares estimator.
22

---

Additional Experimental Results
C.1 Recovery of group structure for >> 3.
We extend results from the paper towards higher dimensions, and consider SO(n), O(n), (L( for n = 3, 5, 7,9. Results are depicted in Fig. [7] We mirror the main paper results with close to perfect reconstruction scores both for the latent space and the group representation for SO(n), (n)), but notice a drop in performance for GL(nn as dimensionality increases beyond 5. In this case, we recover the performance for 7 if we allow to backpropagate through the least-squares solver. As the dimension grows to n 9, the model is no longer able to discover the full group structure, likely due to mismatch between the complexity of GT((9) and the size of the dataset/variation present.
(²(x)
R²(G)
Acc(G, 1)
Acc(C)
1.0
GL(M)
0.0
3
5
7
9
3
5
7
9
3
5
7
9
3
5
7
9
1.0
(U)
0.5
3
5
7
9
3
50(m)
0.5
0.0
5
9
3
5
7
9
3
data dim (group)
data dim (group)
data dim (group)
data dim (group)
w/ gradients
w/o gradients
Figure 7: Higher Latent Dimensions for Synthetic Group Data. Comparison of EbC across an increasing number of true latent dimensions of the synthetic group dataset. We show results for all group types GL(n). O(n), (O(n) with n being on the x axis and fixed content dimension d = 5. Additionally, indicate whether we compute gradients through the linear least squares estimator or not.
C.2 Data efficiency
We analyze the data efficiency of our method by varying three key hyperparameters while keeping all other settings identical to those for 3 in Table I of the main paper. We test across SO(3), (3)). and G3) by:
1. Varying the total dataset size.
2. Varying the number of negative samples used in the contrastive loss.
3. Varying the number of samples per action (k) used for the linear least-squares fit of R(g).
Discussion (Dataset Size): The results in Table show that our method is highly robust to smaller dataset sizes. Even with 20x less data (50k 1000k), the identifiability of the equivariant representa- tion (R2 (2)(x)) and R2 R²) remains nearly perfect, staying above 99.6% for all groups. The primary effect of smaller datasets moderate drop in accuracy for the invariant part of the embedding (Ace(C)).
23

---

Table 3: Data efficiency: Reduced dataset size. We evaluate EbC trained on datasets ranging from 50k to 1M samples (1000k). The 1000k setting replicates the main paper's 's results. We report mean and standard deviation over runs for n2 =
Dataset Size
1000k
100k
500k
50k
Group Type
Metric
SO(n)
R²(x)
99.81 10.4
99.79 s0.09
99.75 s0.22
99.80=0.00
RTG)
99.75
99.71 10.07
99,66 s0.28
99.72.0.06
Acc(C)
99.1910.c9
97.30.2
98.97
95.25
O(n)
R²(x)
99.80sa.s
99.81 11 =o.0s
99.79 s0.12
99.73-0.16
R4GG)
99.73
+9.74
99.72 r0:14
99.65=0.1
Acc(C)
99.06.0a7
97.90
98.92 10.78
94.57=1.60
GL(n)
R
99.83.0.0
99.82 20.03
99.82
20.00
99.82.0.08
RTG))
99.72 10.05
99.71 +0.05
99.71
10
99.68-88 -0.08
Acc(C)
98.42 10:79
96.42.1.2
98.37 10.68
92.02 -9.55
Table 4: Data efficiency: Reduced number of negative samples. We vary the number of negative samples per batch from 1024 16384. The 16384 setting replicates the results of table [ We report mean and standard deviation over runs for n = 3.
Neg. sample hatch size
1024
2048
4096
8192
16384
Group Type
Metric
SO(a)
R²(x)
99.51 s0.28
99.66.0.09
99.48
99.77
99.81 s0.04
R(G)
99,28
99.52
99.30s091
99.67 s0.04
99.75 10.04
Acc(C)
98.23
98.78
98.60
99.13.0.e
99.19 10.09
O(n)
R²(x)
99.55
99.63-00.1
99.62.0.8
99.75
99.80
R2(G)
99.37 s0.10
99.49 -0.19
99.49.0.s
99.64.s.1
99.73 +0.06
Acc(C)
98.49 s0.81
98.90=0.0
98.97 s0.57
99.09 s0.82
99.06 s0.87
GL(n)
R'()
99.70 J0.04
99.72.00.0
99.7610.0
99.79 so.05
99.83 10.02
RTG))
99.53 +0.07
99.56-0.0
99.61 +0:10
99.65 +0.0s
99.72 +2.00..0
Acc(C)
98.92 +0.53
99.09
99.1 +0.55
98.76=0as
98.42 Ho.72
Discussion (Negatives): As shown in Table 4 reducing the number of negative samples has a minimal effect performance. While there slight, consistent downward trend in the equivariant metrics (x), R (G) as negatives decrease, the changes are very small and performance remains high (> 99.2%)
with 16x fewer negatives.
Table 5: Data efficiency: Reduced number of samples per action (k) for fitting R(gg). We vary from the theoretical minimum standard deviation over 5 runs for = 3.
up 3n Table uses 12 (4n). We report mean and
Samples per Action
Group Type Metric
SO(n)
R(0)
99.71
6.86
89.12.33.00
99.85.0.01
10.83:0000
99.83
99.77.0011
RR(G)
8.77+858
99.99 67
88. $8.166533.0
99.63-0006
99.66-0.13
99.70-0.07
99.66-0.16
Acc(C)
20.20.0.32
51.17
97.47
99.40.0.47
99.30.0.74
99.07:0.4
99.10:0.00
OKn)
RP(s)
19.65-0.10
4.68-5.56
99.77-0.09
99. 99.84-0.00
99. 09.88+0005
99.23
99.39 x1.91
R(G)
118
:0.03:.000
99 :0.099:035
99.59
99.67 99.67:1.11
98.95
Acc(C)
99.00-0.22
54.32+12.08
98.88-0at
99.18-0.55
99.13-0.50
97.58+4.55
97.94-3.08
GL(B)
RF(x)
98 v8.46.1.13
5.56:9.54
89.63 86
99.63.0.00
99.76.0.09
99.80
99.80s0.07
RR(G)
0.28+2447
-0.070028
77.00 es
93.25-1.40
98. 93.93+0 92
99. 99.48++0.0
99.56+0018
Acc(C)
35.38
98.36.0.28
98.56.0.82
Discussion (Samples per Action): Table conffirm that the theoretical minimum of k: = n 3 samples insufficient in practice. This expected, as k n samples must form a full-rank system latent space to uniquely identify R(g)) which unlikely to occur consistently. Using = also leads to instability. However, performance rapidly recovers. With k = 6 (2n). R²2(G) is > 99% for (0(3)O(3) and > 93% for GT. Using & (3n) yields near-perfect identifiability (> 99.5%) for R² (G) across all groups, demonstrating practical applicability with modest number of samples.

---

C.3 Robustness to noise
To study the effect of noise, adjust data-generating process. We introduce Gaussian noise term 6 ~ (0, o²) to the group action relationship in the latent space: x' = R(g:)æ + c. We test this for (3), varying the noise standard deviation from 0.0 0.1 and the number of samples per action (k) from 3 to 12. All other parameters follow the setup of Table
Table 6: Robustness to noise for GL(3). We add Gaussian noise ((,0 o²) to the latent transformation x' = R(g)æ x We vary the noise standard deviation a and the number of samples per action (3 = n, = 2n, = 3n, 12 = 4n) used to fit R(g). We report mean and standard deviation over runs.
Group Type
GL(n)
Metric Samples/Action
R²(x)
RT(G)
Acc(C)
Noise Std.
0.0e+00
98.46 11.13
-0.28 10:47
99. +0.58
10.03
93,25 11
98. 70 20.57
99.80 20.07
99,56
98.37 =0.51
12
99.83 s0.02
99.72
98.42 -0.72
1.0e-05
97.89
-0.30
98.89
99.62 20.05
93,24
98.68 .0.78
99.82
99.65 .0.07
98.60
12
99.82 10.03
99.70 20.00
98.48 .0.61
1.0c-04
98. 21.17
-28.47
99.00 20.36
99.64 20.07
93.78 32.07
98.73 20.68
99.83
99.65
98.45 a0.47
99.83 20.04
99.71 =0.07
98.60 s0.60.0.80
1.0c-03
98.58 60.88
-50.29 2127.95
98. 98.98.. u0.51
99.58 20.11
93.33
98.51 20.67
99.81 10.05
99.61 10.09
98.58 20.48
99.81 #0.04
99.68 =0.08
98.50
1.0c-02
88.58 $30.68
-1.02 11.99
98.98 10.00
99.58
93.05
98.61 =0.70
99.81 20.04
99.59.0.0
98.46 s0.8
12
99.83
99.69
98.54
1.0e-01
94.56 #11.34
-27.93 255.48
98.97 s0.37
99.58 s0.05
91.76.028
99.04 20.56
99.77 20.04
96.98 =0:44
99.05 +0.40
12
99.82 s0.05
97.98 =0:15
98.94 -0.74
Discussion (Noise): The results in T show that the (z) and Acc(C) metries are highly robust to noise, remaining stable even at a 0.1. As expected, noise primarily affects the identifiability of the group representation (R2 (G)). This effect is strongly dependent the number of samples (k) used for the least-squares fit. With & = 6 (2n), R²(G) drops from 93.25% (no noise) to 91.76% (o 0.1). However, increasing the samples completely mitigates this. With A (3n), (G) only drops from 99.56% to 96.98%. With & = 12 (4n), the performance remains excellent, dropping from 99.72% 97. 97.98%. This demonstrates that the noise in the least-squares problem can be effectively averaged out by using more sample pairs, confirming the method's 's robustness.
25

---

C.4 Infinite dSprites
Linear Regression R?(G) vs Acc(G, 1):
Linear Regression R vs Acc(G, 1):
R² 0.9363, p p=1.3652e-43
Rt=0.9779, p=6.6379e-34
1.0
Fitted Line
32.0
16.0
Sample per Action Pactor
0.98
Fitted Line
16.0
Sample per Action Factor
0.8
8.0
R1G
0.6
0.96
4.0
0.4
3.0
0.95
3.0
Group
0.94
Group
GL(n)
GL(n)
2.0
77
Q(n)
0.93
0(n)
SO(n)
0.92
50(n)
1.6
0.00
0.25
0.50
0.75
1.00
0.7
0.8
0.9
Acc(G, 1)
Acc(G, 1)
Figure 8: Ace(G, 1) observable proxy for R GG). Comparison of Top-1 NN-Lookup Ace(G, 1) and group action identifiability metric R2 (G) across EbC models trained on synthetic group datasets with varying samples per action factor k. Left: Results for the selected dataset configurations (n = 5, d = 5, |C| 1000). Right: Filtered results for Acc(G,1 > 0.6.
The Acc(G) metric is suitable for practical hyperparameter selection. In the main paper, we described the R2(G) metric measure the quality of the group representation with respect the ground truth group representation and latent factors. On practical experiments, this metric is not observable. Instead, infinite dSprites but also for hyperparameter selection the toy dataset, consider the Acc(G, n) metric which performs a kNN lookup only in the reconstructed embedding space. Intuitively, this metric uses the computed embedding, and a measure of self-consistency of the representation. As validation for this choice. we varied the number of samples per action (one hyperparameter in the fitting procedure) and visualize the correleation between the observable Ace(G, 1) metric and the unobservable, but desired, R2(G) metric in Figure 8 On the full hyperparameter sweep, the metrics are significantly correlated with R2=93.6 (p<le- 10; Figure left). When treating unsuccessful hyperparameter configurations outliers, i.e., discarding models with Acc(G, 1) < 0.6, this correlation gets even more clear at R2-97.8 (p<le-10; Figure right).
Hyperparameter selection indicates suitable group dimension on idSprites We now leverage the Ace(G, metric to perform hyperparameter selection on the idSprites dataset. To provide full picture, Figure shows 10 variants of the metrics with differently strict criteria, considering top-1 to top-10 accuracies during the NN-lookup.
For 18 and 144 classes, we observe a clear increase in performance in all metrics at d=4 for the least conservative Acc(G, 10) metric, however, the most conservative Ace(G, metric still indicates subpar performance at this dimensionality (below 50%). All metrics start saturating around a dimensionality of d=6, which would allow embedding of Rn, Zn Zn into 3 circular dimensions.
Figure TO shows example transitions at this hyperparameter point, which show close-to-perfed overparameterize the model in terms of the group
recovery of the group structure. However, dimensions, the Acc(Gi,) metrics improve slightly while the qualitative evaluation shows a noticeable decline in embedding and representation quality (Figure IT]. In contrast to the results from Figure 8 this suggests a mismatch of the Aco(G, metric and the R2 R²(G) metric which is unknown for this dataset.
We hypothesize that this is because the Acc(G,) -) metrics a) are not able measure the group action prediction exclusively, but instead also measure misclassification of the content, and b) don't measure the degree of the prediction error. Instead any type of classification error is counted with the same weight, no matter if the prediction was off by small or large amount.
26

---

# #classes=18
# #classes=144
1.00
dim Conttnngyz
Accuracy
0.75
Acc(C)
0.50
Acc(G, 1)
0.25
Acc(G, 2)
0.00
Acc(G, 3)
Acc(G, 4)
1.00
dim Cooneenn
Acc(G, 5)
Acc(G, 6)
Accuracy
0.75
Acc(G,
Acc(G, 8)
0.25
Acc(G, 9)
Acc(G, 10)
0.00
2
5 9
2 5 9
dim (group)
dim (group)
Figure 9: Embedding Dimension Hyperparameter Sweep for idSprites dataset. Comparison of EbC for the idSprites dataset across embedding dimensions for the group E [2, 10) and for the content d [2.9] showing observable Top-K NN-lookup metric Acc(Ge G, k) for k [1, 10) as well as unobservable content classification accuracy Acc(C). We show the comparison across two configurations the idSprites dataset. Left: original dSprites configuration, Right: 16x #shapes, 0.5 x #scales #rotations #translationsX #translationsy
27

---

PACE
JAAC
PACE
JNAC
PACZA
PAC2
pert
part
por
por
peor
por
p4
par
pert 34
aeation
iittion
ventor
Panson in
rosee
ronnan is
venoe
roM as
man.e. 22
reseree
mansin XX
nenve
Top-1 Nearest Neighbor Lookup based on EbC with 6D group dimensions. For the idSprites dataset configuration with |C| 18, we sample random reference image y and generate sequence of transformed images y (targets) according to the different types of possible group actions. We estimate R from the embeddings, compute Ræ and perform top-1 NN lookup across all embeddings to get a prediction in image space. In the three blocks images we show action traces across posX translation (top), posY translations (middle), and rotations (bottom). Within each block we show the reference image (first row), the target images (second row), the top-1 NN image prediction (third row). We color the images in the third row according the Manhattan distance between the associated true latent factors of the target and predicted image.
Figure 10:
peaco
20012
2001-4
pHRt'S
PRNT 3
2000 LD
PORK
200 S4
part 25
par
par 22
FORP 24
perizE
peak 28
pest
Dap
20072
part
pwr 3
p0n II
12
pan 14
per 10
patf 16
per 20
DEF 22
per 24
22
ree'
rESt 2
relutien
Wulles?
mesion to
rntaties 28
vtalin 23
vtabbe 23
wakke Intator 21
Matioon 31
Iiteion32 30
restion 39
Figure 11: Top-1 Nearest Neighbor Lookup based EbC with 8D group dimensions. Just like Figure 1u but for EbC with n =
28

---

C,5 Application to real-world data
Here, we demonstrate how EbC be used on real-world time series dataset to find group equivariant embedding spaces. We apply EbC to the rat hippocampus dataset recorded by Grosmark and Buzsáki [14.. We use the data as preprocessed by Zhou and Wei [55]. In this dataset, neural activity from CA1 neurons in hippocampus of multiple rats was recorded via implanted silicon probes. During the recordings of each session, the rats moved on 1.6m long linear track. Neural activity, the position on the track, and the movement direction of the rat was recorded (see Figure [2A). We apply EbC to the data of the following three rats: "Achilles", "Buddy" and "Gatsby".
Using EbC for data analysis requires to specify how positive pairs are sampled and how positive pairs grouped together into the same group action. This is conceptually close to CEBRA 43 where auxiliary variables are used to resample the dataset to encode particular neuroscientific hypotheses. As in CEBRA, we leverage the behavioral variables of position and movement direction, but discretize the position variable with a bin size of lcm.
We propose three different sampling strategies, depicted in Figure 12B:
1. Group=Position: We consider all possible pairs of data points to be positive pairs and group positive pairs based on the difference in position regardless of the movement direction.
2. Group=Position & Direction: We consider all possible pairs data points to be positive pairs and group positive pairs based on the difference in position and difference movement direction.
3. Group=Position, Content=Direction: We consider only those pairs of data points where the movement direction does not change and we group positive pairs based on the difference in position.
We compare our method to and closely related to EbC in the sense that is also encoder-only representation learning method with identifiability guarantees. We compare both against CEBRA- REEBA--Beavior using only the position variable and the variant using both position and movement direction as label information. We follow the same dataset split into train, validation and test used in (43) and leverage consistency across runs for selecting the best hyperparameters for each model. To match the "offsett0-mmdell" used by CEBRA as best as possible, we leverage a time-delay embedding for EbC with receptive field 10.
44 which a state-of-the-ar method on this dataset
For the CEBRA models, we run hyperparameter sweep over latent dimensions of d E {6, 8, 12, 32) and train for 10k steps. For the remaining parameters, we use the default pa- rameters specified via CEBRA's sklearn API. In particular, this means using the cosine distance in the loss (restricting embeddings to the hypersphere), using the "offset10-modee" (CNN based model with embeddings normalized to the hypersphere), and "time offset"=10 specifying how positive pairs are related in terms of the time offset between them.
For EbC, we run hyperparameter sweep across group (2 or 3) and content dimensions (0, 1, or 4), as well as over the number of samples per action for which we run experiments with (k 4d, k = 8d). We use a three layer MLP with 128 hidden units per layer for the encoder. We compute the gradients through the linear regression modeling fitting as done in Appendix C.I As for CEBRA, we train for 10k steps.
We compute the following three types of metrics:
1. Decoding metrics:
We follow CEBRA's decoding setup using a KNN (with K=3) both for decoding the position and direction from the embeddings.
2. Consistency metrics: We the consistency metrics across runs defined by CEBRA to measure how well different embedding spaces can be aligned to each other. This metric fits affine transform to map from embedding space another and measures the R² score of the prediction. We use this on 5 runs of the same model with different seeds and compute the consistency between every resulting embedding space with every other embedding space of those runs.
3. Linear Group Action (Position): To measure how well a given embedding space is linearly structured with regard to the position variable, we collect all embedding points related by a specific difference in the position variable (here: 5cm) and fit a linear model for which
29

---

we measure the R² score of the prediction. To make this metric fairer, we fit two separate models for pairs based on the direction label. The metric value we report is the average of these two individual R2 scores.
Group-Pastion Nontent=Nore
Group-Positen & Diraction
Group-Rosition Content=Dircction
Content=None
Postlon Dccoding
Direction Decoding
(Deha Somo
IDeita 50om]
Somi
(Dsta 50cm)
CEBRA
CESRA pleadd
CEBRA pos+dir
USC Grrgenar11
E&C Gepos
G-pos,
Ebc. poeede di (CIn)
ESC Gepos dir-
Hedian Decoding Broo (am) buddy
0.00
0.25
0.50
50
75 Accuracy %
100
0.0
1.0
0.0
0.5
RP
1.0
0.5
R
0.5
R
schilles
gatsty
achilles
gatsty
Gropp.Potter Cortari-cere
Group Poctoe Giwction Costortakine
Grop-PPsttion Certart-Croclln
For 9Ach
Rot "Gatctty"
RoX 6000
To9t
Figure 12: Application to neural time series data. (A) We used recordings from the hippocampus of rats running on 1.6m long linear track. This yields two time series: neural activity y, and behavior (position P: and direction de). We use EbC to encode the neural activity with the behavior labels acting as labels determining the positive pairs and thereby the group structure. (B) Illustration of the different sampling strategies employed with EbC. (C) KNN-Decoding performance on the test set for decoding the position and direction label. We compare EbC to CEBRA and include label shuffle control (D) Measure of linearity (for train and test set) of the embedding spaces conditioned on the difference in the position variable. Reporting the 1t2 score linear regression models trained on predicting the next point in latent space for pairs of data where the difference in position is 5cm. We also report the R² of using the same 5cm model on predicting 50cm ahead, i.e. 10 forward predictions in latent space. (E) EbC embeddings of the neural activity of the rat "Buddy" with different variations on how the behavior labels are used to determine positive pairs (F) EbC & CEBRA A-rsRA-Behaen Behavior Embeddings rats Achilles, Gatsby and Buddy. Showing EbC model with group=position and content=direction, and CEBRA model with behavior labels position & direction.
EbC reliably encodes position and direction. Similar to CEBRA-behavion EbC produces latent spaces from which both the position and movement direction of the rats can be decoded (Figure 12C). For position decoding, EbC and CEBRA-
achieve comparable test performance: median errors of approximately 6cm for "Achilles", =12-14 em (EbC) and x1 cm (CEBRA-behavion) for "Gatsby", and x13-15 em (EbC) and 122 cm (CEBRA-behaving for "Buddy". In contrast, models trained with shuffled labels yield errors of Rs40-50 cm, confirming that both methods leverage meaningful -euavo-be relationships for decoding.
For direction decoding, the EbC variant trained with direction as part of the content performs on par with CEBRA-behavin As expected, the EbC model trained with both position and direction as group variables performs worse- parsc-pprtyy for "beady" this formulation enforces invariance to direction. Consistently, EbC models using only position the group variable (no content) perform close to the shuffle baseline for direction, reflecting the intended invariance.
30

---

EbC recovers latent linear group actions. EbC reveals highly linear relationships in the latent space with respect to position (Figure[[Pp.. At a local scale cm position differences), the recovered embeddings exhibit near- wear-perfect linear structure across all three rats. Notably, a linear model fitted only on 5 em transitions generalizes well to more global scale: predicting 50 cm ahead (10 steps) while retaining most of its predictive performance. In contrast, CEBRA--bhaviin does not consistently capture such linear group structure. At the 5 cm scale, performance is comparable for "Achilles" but weaker for "Buddy" and "Gatsby". At 50 cm, the linear trend largely disappears for "Buddy" and "Gatsby"; for "Achilles", the Rt2 drops below 75%, whereas EbC remains at =080-10%. However, for both methods, linear models fitted on the train embeddings do not systematically generalize to the test embeddings, potentially due to the limited data size. For "Achilles" " which has roughly twice as many neurons recorded in other sessions, predictive performance generalizes to the test set for EbC.
Sampling strategy induces structured representations. As described above, EbC allows explicit control over the structure of the latent space through the sampling strategy defining positive pairs. For "Buddy", using Group= Position (direction ignored) yields embeddings that are equivariant to position and invariant to direction (Figure I2 When Group = Position=Directionn the latent space organizes position
approximately circular linear trajectory, while direction is encoded as orthogonal movement towards or away from the center of this "position circle". When grouping by position difference while holding direction fixed within group (Group Position, Content Directioon, EbC separates the two behavioral variables: direction is encoded along the content dimensions, while the group (style) dimensions capture the linear equivariant structure of position.
Limitations and interpretation of model assumptions. Both A-behvibe and EbC provide identifibillity guarantees (recovering the latent space up to a linear/affine transformation) under specific assumptions about the data-generating process. Violations of these assumptions break identifiability, and thus each method can be interpreted as testing a specific hypothesis the structure (Pos+Dir), nearby latent points correspond
of the neural-behavioral mapping. For to samples that are close in time, position, and direction, with latent trajectories constrained to the hypersphere. For EbC, pairs of samples are related through general linear group action in Euclidean space, where the group action is defined by the sampling strategy (as described above).
Three key differences follow: (1) CEBRA A -hhhavior incorporates temporal proximity into the hypoth- esis; (2) CEBRA- constrains samples to be close in latent space, whereas EbC enforces the existence of linear relations between groups of pairs; and (3) in our experiments, CEBRA-behavior restricts representations to the hypersphere, while EbC does not. In this sense, EbC imposes a stronger structural hypothesis on the data, particularly through the assumption of linear group actions.
Model Consistency
CLORA pos
CEBRA pos+dir
EBC
G-pos, C-dir
EaC G-pos
E&C G-pas (Ctrl.1
EbC Gmpas, C-dir (Corl.)
FbG G-pocedir (Chrl)
achilles
buddy
gatsby
Figure 13: Consistency on the test set across model runs measured via the It2 as defined by Schneider al. 4
Consistency across runs. Within this hypothesis- view, we assess whether embeddings from multiple runs (different random seeds) of the same model are linearly related, as predicted by the identifiability guarantees. For both EbC and CEBRA-behavior, we test whether a linear map exists that aligns embeddings across runs.
31

---

As shown in Figure Posttion-DDirectio) on "Achilles" (=97-100%) For "Buddy", EbC reaches =70-75%, and for "Gatsby", x.00-6%% In contrast, CEBRA- CEeBRA-beiie A-eehvor achieves higher average consistency across rats: x=85-330 (Achilles), -8% (Buddy), and =79-8449 84% (Gatsby), although for Achilles, EbC exceeds CFBRA-behavior.
13
the highest consistency is observed for EbC (Group = Position and Group
Drou-Pas ContottNen
Droup-Pox Group-Pox Gi
Figure 14: Embeddings of EbC label shuflle control experiments. Shows the same experimental setup as EbC experiments in Figure 12 but with the behavior labels randomly permuted before training.
importance of held-out evaluation. The shuffled-label control highlights the necessity of train/(val)/test splits and cautions against interpreting structure from train embeddings alone. Even with randomized behavior labels, EbC imposes its specified group structure the training latent space, producing structured embeddings across all rats (Figure [4]. However, this structure differs from the non-shuffled embeddings its general geometry and does not generalize to the test set when the neural-behavione relationship is destroyed. When interpreting embeddings, hence important to consider that the mere linear arrangement of points in the embedding is consistently visible in the shuffled embeddings, but additional geometry in the non-shuffled embedding (e.g. alignment of positional information across directions; eircular structure for some of the rats) is visible and non-trivial. On top. the observed structure in the non-shuffled embeddings transfers to the test set as an additional control to rule out the possibility overfitting.
Shuffle control:
32

---

D Additional Literature Discussion and Related Work
Equivariant representation learning has been a prominent topic in machine learning research from different angles. Below we denote observed data as y Y. with y' = y short-hand for the transformed data. Embeddings of the observed data are denoted as 2; x' E X, where X is a vector space. Representations of the group are denoted as R: G + GL(n, X). and x' = 9 x R(g)x. We focus on methods for learning vector embeddings x of the data y, which are equivariant and/or invariant to group actions g. In contrast, orthogonal related work focuses on learning representations of the group given X, Y 29|50, Another rich area of literature focuses on neural network architectures which are invariant or equivariant specific predefined groups 3 5
10 27
Methods. Invariance to all possible group actions (augmentations) may hinder downstream tasks (e.g. color-invariance for flower-type classification). Xiao et al. 44 propose to use a contrastive learning framework to learn representation space in which every subspace is invariant to only one type of augmentation. Eastwood et al. [9] extend this by introducing second entropy loss to encourage subspaces to become disentangled.
Equivariant and Invariant Representation with known group action (explicitly observing g in a parameterized form). Several encoder- based algorithms building on contrastive learning exist: E-SSI. [4] embeds reference sample of and multiple transformed samples y 9i y into a latent space at, ar which is split into an invariant and equivariant subspace. Invariant parts are learned via SimCLR (2) and equivariant parts through an auxiliary task that predicts the parametrized group action 9 from x EquiMod (7] achieves equivariance by embedding pairs of y. 9 y by explicitly modeling the group action in latent space via a neural network uw g) á' and minimizing the distance between é' and x'. Park et al. (38] additionally parameterize the encoder and latent group action prediction model as G-equivariant neural networks. Garrido et al. 12 propose variant using non-contrastive SSL losses.
Beyond contrastive learning, several approaches leverage (variational) autoencoders. Qi et al. [3 en- code pairs of y. y into latent space and decode the group action g from the concatenated embeddings ar xF. Jin et al. (23] embed y into latent space, model the latent space group action as x'' = R(g) ar and decode y' from the predicted x' assuming full knowledge of R(g). Keurti et al. 24 use a linear prediction x' = R(g)= a where R(g) is predicted by a neural network from the observed and parametrized g, essentially an auto-encoding variant of Garrido et al. [12].
Equivariant& & Invariant Representation with unknown group (implicitly observing g) Al- though the aforementioned approaches share conceptual similarity their goal of learning invariant and equivariant embeddings of data by modeling linear relations in latent space, EbC does not assume knowledge of the parametrized form of group actions g. Instead of observing information about g directly, a second class of methods assumes that two or more pairs of data share the same underlying action, y: 9 yss This is key assumption we also require for EbC.
Encoder-only methods include CARE [5], a contrastive learning framework to learn invariant an locally equivariant representations. CARE encodes two pairs of observations (y1.g - yi) and (y2.9 9- Y22 with the same group action 9 into a latent space such that the embeddings are related by the same matrix Rg E (dd) through æ = R2 and x² = R2 CARE achieves this by introducing an additional loss term to the InfoNCE loss, which maximizes the similarity between the dot products 2 a and x x/y. Instead of applying this to subspace of the embedding space, they apply both the invariant loss term of InfoNCE and their new equivariant loss term to the full embedding space and use a weighting factor to control the trade-off between invariant and equivariant representations. Yerxa et al. [51] propose a variation of CARE in which the embedding space is split into an invariant and equivariant subspace. Group actions are encoded as a = 21 + by and ary = x2 + by- STL. (22) learns representations of (a) y, y such that x. x' are equivariant to the group action and (b) the group action g itself, again from two pairs of data. Similar to EquiMod 77, the latent group action is parameterized by neural network. However, instead of assuming knowledge about g, they use a learned representation Rg as the second input to the neural network: a' = uyy (2',, R2,) where R. is predicted by another network (2 x'e) R. from the latent representations of the second observed pair (y2. g y2). To learn the correct representation of Rg.. EquiMod is extended by a third loss term that maximizes the similarity of Re (1, x1) and Rg (2 2.)). Like CARE, they don't 't learn separate subspaces and instead use weighting factors for the invariant and equivariant loss terms to control the trade-off between invariant and equivariant representations in X.
33

---

the irreducible The problem can also be approached with auto-encoding approaches: Winter et al. [48] learn embeddings of y and g which the factorize y = g y into an representation of y which is invariant to and representation Ry (g) that represents the action in the data space Y, such that y' Ry (g) (())) with being the decoder.
The Unsupervised Neural Fourier Transform (U-NFT) 288 36 is an auto-encoder framework with a linear group action model in latent space. For their learning setup, Koyama et (28) assume access (2) N.
Di y one sequence to multiple sequences of data points (y == (yb : ... Vr J === with y
for each implicitly observed, but unknown group action di E from the predicted # which in turn is predicted from a R(g))a)
autoencoder reconstructs Vi+1
(3) The representation from these examples is estimated using least squares and post-hoc basis transformation on the set of R(g:) find a block diagonal representation (g:) to facilitate disentanglement
components of R(g.). Mitchel et al. [35] propose variation of NFT, where the learning setup is restricted directly produce block-diagonal representation R(g:), ), avoiding the need for a post-hoc basis transformation. However, to achieve this, they also restrict R(g:) to be orthogonal.
Winter al. 488 and Allingham al. [ only recover invariant representation and model the group action separately in the data space. All other the methods in this section try to solve the same general task of learning equivariant and invariant representations of the data without explicit knowledge of the underlying group actions. CARE (Gupta et al., 2023) is the closest encoder-only the hypersphere, which imposes
(contrastive) approach to EbC. but constrains the embedding additional structure on the learned representation, while EbC can learn different topologies (c.g.. torus in Fig. 4, hypersphere in Fig. 5). In terms of function and data requirements, U-NFT (Miyato al., 2022; Koyama et al., 2023) is the conceptually closest work, but requires to learn a full generative model of the data.
34

---

E
NeurIPS Paper Checklist
1.
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our key theoretical claim is identifiability, which we proof through Theorem 1. Our empirical claim is validated on synthetic datasets as well as infinite dSprites.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings. It is fine to include aspirational goals as motivation long it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Limitations are discussed in Section 7
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
35

---

The full proof for Theorem given in Appendix The proof for Corollary is given directly in the main paper. The full definition of assumptions stated in the Appendix, and informally outlined in the main paper.
Justification:
[]]
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: The key components of the algorithm are outlined in Section 2 Experimental details are given in Section Additional experimental details are provided Appendix B
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to a hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
36

---

reproducible, they Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
The code for our paper is available at https///gihub..co//ynmaat
inference/cbe During the phase of double-blind review, we provided anonymized version the codebase. For additional pointers, see Appendix B
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.c ec/
public/guides/
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https for more details.
7 //nips. hhaps..ccpubllic/uuidess
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: The most important details are outlined in Section The full details are given in Appendix B
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: details.
We run all experiments across dataset and model seeds, see Section for
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emro bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
37

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Appendix B contains a list of compute resources used for the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
The paper conforms with the NeurIPS Code of Ethics in every regard.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: As noted in the guidelines, our method is general-purposs. approach for esti- mating the effect of actions/intervention from observable data. There is broad applicability of such an algorithm, but no specific societal impacts that are particularly tied to the paper.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
38

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We report results small-scale synthetic datasets.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Appendix Bottinns all relevant license information.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
39

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
datasets.papersstt
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do not release new assets.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details of the dataset/code/mmodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
40

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM for what should or should not be described.
41


Reviews:
[{"review_id": "T59i8W56nk", "reviewer": "Reviewer_zh3E", "summary": "The paper proposes a method for learning representation of a group from data through contrastive learning approach with theoretical guarantees and verifies on synthetic and 2D image datasets.", "questions": "- Figure 2: The encoder in Caption is f, and there is another encoder \\phi after the samples, which is a bit confusing. What is Q in the Figure 2?\n- I have less background in neuroscience and biology, and I’d appreciate some concrete examples on this.", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n\n- The paper is very well written, with clear motivation and exposition.\n- Figure 3 provides an intuitive visual explanation of the proposed method, making the main idea easy to understand.\n- Theoretical results (e.g., Theorem 1) demonstrate identifiability of the group action and latent space under mild assumptions, which is uncommon for self-supervised equivariant learning methods.\n- The approach is simple and efficient, relying on contrastive learning with an encoder-only pipeline and on-the-fly least-squares fitting, avoiding the complexity of generative models.\n- Strong empirical results: Experiments cover a diverse set of group structures (e.g., SO(3), O(3), GL(3), and product groups), demonstrating the generality of the method. Achieves near-perfect latent reconstruction and group prediction accuracy, outperforming relevant baselines on synthetic data.\n\n**Weaknesses:**\n\n- Data efficiency is not systematically studied; there is no analysis on how performance scales with dataset size, batch size, or the number of negative samples, which is especially relevant given the theoretical assumptions on required samples. I would suggest including experiments sweeping dataset size and batch/negative counts to measure convergence and data efficiency.\n- Comparison to baselines is somewhat limited; recent related approaches such as Park et al. (ICML 2022), Homomorphism Auto-Encoder (Keurti et al., ICML 2023), and other group-equivariant learning methods are not included or discussed in detail.\n- The key assumption (that all n+1 samples in a mini-batch share the same, unknown group element) is not tested or validated on real-world datasets; it is unclear how well the method handles weaker or approximate group structure in practical settings.\n- Application to real-world data is limited; all experiments are on synthetic datasets (e.g., Infinite-dSprites), with no demonstration on realistic or noisy sequence data.\n\n**Additional References Suggested:**\n\n- Park et al., “Learning Symmetric Embeddings for Equivariant World Models,” ICML 2022.\n- Keurti et al., “Homomorphism Auto-Encoder,” ICML 2023.\n- Winter et al., “Learning Invariant and Equivariant Representations in Unsupervised Group Settings,” 2022.", "was_revised": true, "final_justification": "I read the author response and other reviews. I would like to acknowledge the very detailed response of the authors to my concerns and questions. The response addresses most of my questions, and I think my comment on e.g., data efficiency experiment should not be expected to be fully addressed during the short rebuttal period. I still hold my view that this paper should be accepted to the conference and am happy to defend. I encourage the authors to incorporate the new materials and further strengthen in the paper revision.", "rebuttal": "We would like to thank the reviewer for their positive assessment and very helpful and constructive feedback. \n\nBefore addressing the specific points, we'd like to highlight that on synthetic group data our emprically results also extend beyond $n=3$ to $n \\in [5,7,9]$, see Appendix C, Fig. 7.\n\n### Re: Data efficiency\nWe very much agree with your proposal. We have run new experiments to analyze data efficiency by varying dataset size, number of negative samples, and samples per action (for fitting $R(g)$), keeping all other parameters identical to our s in Table 1. We will add the full results to the appendix.\n\nNote: The following results also exist for SO(n), O(n), but GL(n) has performed strictly worse. Happy to provide SO(n), O(n) results upon request.  \nNote 2: We have marked the hyperparameter setting corresponding to the replication of Table 1 in each of the following result tables with **\n\n**Table 1.1: EbC on reduced dataset size, $n=3$.**\n|       | Dataset Size    | 50k    | 100k   | 500k   | 1000k **   |\n|-|-|-|-|-|-|\n| **Group Type** | **Metric**              |  |  |  |   |\n| GL(n) | R²(x)              | 99.8±0.0 | 99.8±0.0 | 99.8±0.0 | 99.8±0.0  |\n| GL(n) | R²(G)              | 99.7±0.1 | 99.7±0.0 | 99.7±0.0 | 99.7±0.0  |\n| GL(n) | Acc\\(C\\)             | 91.9±0.9 | 97.2±0.8 | 98.5±0.6 | 98.5±0.7|\n\n\n**Summary:** The results show that our method is highly robust to smaller dataset sizes. Even with **20x less data**, the identifiability of the equivariant representation remains nearly perfect ($R^2(x)$ and $R^2(G)$ stay above 99.7%). The primary effect of smaller datasets is a moderate drop in accuracy for the invariant part of the embedding ($Acc(C)$).\n\n**Table 1.2: EbC on reduced number of negatives in a batch, $n=3$.**\n|       | Batch Sizes (Negatives)   | 1024     | 2048     | 4096     | 8192     | 16384**    |\n|-|-|-|-|-|-|-|\n| Group Type | Metric                         |  |  |  | \n| GL(n) | R²(x)                         | 99.7±0.0 | 99.7±0.1 | 99.8±0.1 | 99.8±0.1 | 99.8±0.0 |\n| GL(n) | R²(G)                         | 99.5±0.1 | 99.6±0.1 | 99.6±0.1 | 99.6±0.1 | 99.7±0.0 |\n| GL(n) | Acc\\(C\\)                        | 98.9±0.6 | 99.1±0.6 | 99.1±0.5 | 98.8±0.6 | 98.5±0.7 |\n\n\n\n**Summary:** Reducing the number of negative samples has a minimal effect on performance. While there is a slight downward trend in the metrics, the changes are very small and often close to the standard deviation across runs.\n\n**Table 1.3: EbC on reduced number of samples per action for fitting the linear least squares, $n=3$.**\n|       | Samples Per Action  | 3           | 4          | 5         | 6        | 7        | 8        | 9        | 12**       | \n|-|-|-|-|-|-|-|-|-|-|\n| Group Type | Metric                    | |  |\n| GL(n) | R²(x)                    | 98.5±1.1    | 5.5±10.0   | 89.6±28.8 | 99.6±0.0 | 99.8±0.1 | 99.8±0.0 | 99.8±0.1 | 99.8±0.0 | \n| GL(n) | R²(G)                    | -1.4±1.7    | -7.3±21.7  | 76.9±28.8 | 93.1±1.7 | 98.9±0.9 | 99.5±0.2 | 99.5±0.2 | 99.7±0.0 | \n| GL(n) | Acc\\(C\\)                   | 99.1±0.3    | 35.4±27.1  | 98.4±0.3  | 98.7±0.6 | 98.4±0.8 | 98.6±0.8 | 98.4±0.5 | 98.5±0.7 | \n\n\n**Summary:** These results confirm that the theoretical minimum of $n$ samples is insufficient in practice. This is expected, as achieving the theoretical limit would require the sampled pairs to form a full-rank system after being encoded into latent space, which is unlikely to occure consistently throughout the model training. However, performance recovers to $>99\\%$ with a modest number of samples (e.g., >$2n$), demonstrating practical applicability.\n\n### Re: Comparison to baselines\nWe appreciate this point and have added an extended related work section to the paper (see the **full discussion in the response to reviewer s1fj**). The primary distinction between our method and the suggested works (**Park et al. '22**, **Keurti et al. '23**) is that **our method does not require ground-truth knowledge about group actions $g$**.\n\n* **Park et al. (ICML 2022):** This method requires known group actions $g$ as input to its latent action model and requires the network architecture itself to be G-equivariant, presupposing knowledge of G.\n* **Keurti et al. (ICML 2023):** This work uses an Autoencoder framework and also requires known parametrized $g$ to learn a matrix representation $\\hat{R}(g)$. While its goal of finding a linear representation is similar, it relies on a stronger supervisory signal and the overhead of a decoder.\n\nIn contrast, our approach only requires the weak assumption that sets of sample pairs underwent the *same unknown* transformation.\n\n### Re: Weak or approximate group structure & **Validation on real-world dataset**\nWe agree that these are crucial points and have addressed them both theoretically and empirically.\n\n**Theoretically**, our identifiability proof (see Appendix) does not rely on a strict group structure. The core assumption about the data generating process that is used in the proof is that pairs of latent points $(x, x')$ are related by a linear map, $x' = A_ix$. While we frame this in the context of group representations ($A_i = R(g_i)$), the proof does not require the set of matrices $\\{A_i\\}$ to form a group. This inherent flexibility suggests our method is suited for data with approximate or learned symmetries as well.\n\nTo **empirically** validate this, we applied our method to a **real-world neuroscience dataset** (Grosmark & Buzsáki, Science 2016).\n**Data:** We used recordings from the hippocampus of rats running on a **1.6m** long linear track. This yields two time series: neural activity $\\{y_t\\}$ and behavior (position $p_t$ and direction $d_t$).\n**Setup:** We formed pairs $(y_t, y_{t+k})$ and clustered them based on the change in behavior $\\Delta p_{t,k} = p_{t+k} - p_t$. Each cluster of pairs represents an approximate, unlabeled transformation of the same type (action). We then applied our method (EbC) to learn a behavior-equivariant latent space from the neural data. We compare against **CEBRA-Behavior** (Schneider et al., Nature 2023), a state-of-the-art method for this task, using the same data splits and evaluation protocol.\n**Evaluation:** We decode position (k-NN Regressor) and direction (Logistic Regression) from the learned embeddings.\n\n**Table 2.1: Position Decoding (Median Absolute Error in cm)**\n|Method|Rat Name|Train|Val|Test|\n|-|-|-|-|-|\n|CEBRA-Behavior|achilles|01.67±0.04|06.93±0.42|05.57±0.51|\n|CEBRA-Behavior|buddy|06.13±0.17|12.41±0.66|12.40±0.63|\n|CEBRA-Behavior|gatsby|06.68±0.14|11.65±1.09|11.40±0.30|\n|EbC|achilles|00.83±0.06|04.74±0.35|05.76±0.47|\n|EbC|buddy|01.24±0.10|14.04±0.70|16.02±1.55|\n|EbC|gatsby|00.89±0.02|08.76±0.24|14.28±0.87|\n            \n\n\n**Table 2.2: Direction Decoding (Accuracy %)**\n|Method|RatName|Train|Val|Test|\n|-|-|-|-|-|\n|CEBRA-Behavior|achilles|60.15±0.32|55.95±0.98|59.62±0.79|\n|CEBRA-Behavior|buddy|60.26±0.25|69.30±0.81|58.33±1.36|\n|CEBRA-Behavior|gatsby|62.78±0.64|56.94±1.81|65.96±0.77|\n|EbC|achilles|92.17±0.66|88.62±1.77|82.20±0.50|\n|EbC|buddy|85.40±3.38|84.74±4.57|79.06±5.28|\n|EbC|gatsby|83.62±2.70|84.58±1.24|79.18±0.68|\n\n\n**Results:** Our method (EbC) performs close to / on par with CEBRA-Behavior for position decoding (test set relative error of 3.5-8.9% for EbC vs. 3.5-7.7% for CEBRA). Crucially, EbC **significantly outperforms the baseline in decoding the direction** of movement, achieving 79-82% accuracy on the test set compared to CEBRA's 58-69%. This successful application demonstrates that EbC can learn meaningful representations from noisy, real-world data with only approximate symmetries.\n\nFinally, we also ran a systematic study on the effect of noise within the data-generating process. We refer to our reply to bGkb for these results. \n\n### **Answers to Questions**\n\n#### Q1 - Figure 2\nWe apologize for the confusion caused by the typo and notation. We will correct Figure 2 in the revision. To clarify:\n* $f$: Represents the unknown data-generating function, an injective map from a true latent space $x$ to the observed samples $y$.\n* $\\phi$: The **learnable encoder** that maps observed samples $y$ to an embedding space. \n* In the caption, it says \"encoder $f$\", which is a very unfortunate typo and should read \"encoder $\\phi$\"\n* $Q$: This was a typo and should be $\\hat{R}(g)$. It represents the **estimated linear group representation** that is computed on-the-fly via linear least-squares within each mini-batch.\n\n#### Q2 - Example applications\nWe are happy to provide more concrete examples. Our framework is broadly applicable to any \"intervention\" setting where one observes a system's state before and after a common transformation is applied to a set of samples.\n* **Medicine:** One could study the effects of various drugs by observing patient physiology before and after treatment. The model could learn a \"drug-effect-equivariant\" representation, where the transformation corresponds to the physiological change induced by a specific drug.\n* **Cell Biology:** In single-cell RNA analysis, researchers measure gene counts, perform an intervention like a gene knockout, and measure again. Our method could identify representations equivariant to the effects of specific gene edits without prior knowledge of those effects.\n* **Time-Series with Auxilary variable:** As demonstrated in our new experiment, the framework can be creatively applied to time-series data where an auxiliary variable (like position) allows for grouping time-steps that represent a similar, unknown evolution of the system."}, {"review_id": "npmPcn6083", "reviewer": "Reviewer_4zKK", "summary": "This paper considers the learning of equivariance by unsupervised/contrastive means.  The key mathematical result is on identifiability in this setting, by extending results of Roeder et al. a little bit.  The algorithmic approach follows the mathematical development and is demonstrated to work in one exemplary setting.", "questions": "Clarifying the differences between the present work and other past work on learning group-theoretic structure would be very helpful in clarifying the contributions here.  For example [Dehmamy et al., \"Automatic symmetry discovery with lie algebra convolutional network,\" NeurIPS 2021], [Yang, et al., \"Generative adversarial symmetry discovery,\" ICML 2023], or [Yu et al., \"Information lattice learning,\" JAIR 2023].  See also references therein and thereto.  Can you do that?  It would help with your claim about \"group representation learning from unlabeled observational data is feasible at all\".", "limitations": "yes, they have addressed limitations.  no discussion of potential negative societal impact.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "STRONG: The problem of symmetry discovery is important in numerous settings, and the findings here for such a problem are intriguing.  Note that the setting is where there are batches of samples under the same transformation.  The paper hits all the points of formulation, theory, algorithm, and basic experimental demonstration.\n\n\nWEAK: The seemingly non-native English phrasing, e.g. \"Group theory allows to study this structure\" -> \"Group theory allows studying this structure\" could be improved. \n\nEven though Thm 1 is the main result, the technical diversity conditions are omitted in the main text: to this reviewer, this feels like a significant shortcoming that is easily rectified by moving them up from Appendix A.\n\nThe mathematical advance over Roeder et al [14] seems marginal.\n\nI am not an expert in the motivating examples, but the batched data needed for this approach seems like a very strong assumption.", "was_revised": true, "final_justification": "revised up, given the clarifications w.r.t. past literature", "rebuttal": "We thank the reviewer for recognizing the importance of the problem and for their valuable feedback. We are happy to clarify the points raised.\n\nRegarding the English phrasing, we appreciate you pointing this out and will perform a thorough proofread for the camera-ready version.\n\n---\n### Re: Placement of Diversity Conditions\n\nThank you for raising this concern, we agree that the theory section would benefit from highlighting the full set of assumptions more prominently and concisely. in the revised version of the main text.\n\nWe would like to point out thought, that the diversity condition has been (informally) introduced in the main text (lines 107-111) right before stating the Theorem 1. We will move the consice but complete deftinition of all conditions from Appendix A into the main theory section to make Theorem 1 and its assumptions clearer.\n\n\n---\n### Re: Mathematical Advance Over Roeder et al. [14]\n\nOur Theorem 1 shows that group representations can be learned via the implicit represention in Eq. 2 and provides an identifiability proof for jointly learning the latent space *and* the group action using EbC. This is orthogonal to the key result in Roeder et al. [14].\n\nWe believe that you refer to the fact that as part of our proof, we indeed leverage prior results from Roeder et al. [14], specifically we extend the canonical discriminative form towards Euclidean spaces (Theorem 5, see appendix) and use this form as part of our proof. This extension is indeed marginal, but a technical detail of our proof strategy.\n\nDoes this clarify the concern? We would be happy to discuss more details.\n\n---\n### Re: Batched Data Assumption\n\nWe appreciate the sentiment and are happy to elaborate on this:\n\n* **Common Assumption:** All of the recent related works known to us, which tackle this unsupervised problem, such as CARE (Gupta et al., 2023), STL (Yu et al., 2024), and U-NFT (Koyama et al., 2023), all require observing multiple pairs of data transformed by the same unknown action. Our data requirement is therefore in line with the current state of the art for this specific problem setting.\n\n* **Practical Scenarios:** This data structure naturally arises in many scientific and systems identification domains where one observes a system before and after an intervention/pertubation/transformation. For example:\n    * **Genomics (CRISPR Screens):** Measuring gene expression in cells before and after a specific gene knockout is applied to a batch of them.\n    * **Structural Biology:** Observing a protein's structure before and after a specific ligand binds.\n    * **Neuroscience (fMRI):** Recording brain activity at rest and then again after presenting a specific stimulus to a subject.\n\n\nFor the most straight-forward application of EbC this requires discrete actions. \n\nHowever, as we demonstrate in our **new real-world experiment (see response to Reviewer zh3E)**, our method can be easily adapted to work effectively even with continuous auxiliary variables by discretizing the action space, showing its flexibility beyond these examples.\n\n---\n### Re: Clarifying Contributions vs. Other Past Work\n\nThank you for suggesting these references. They allow us to better position our work. For a broader  and more detailed comparison, we **highly recommend** checking out **our extended related work discussion in the response to Reviewer s1fj**.\n\n* **Dehmamy et al. (NeurIPS 2021):** This work on Lie algebra convolutional networks is an excellent example of building **equivariant network architectures**. This line of work is complementary to ours. Such methods use prior knowledge of a symmetry group to build a specific inductive bias into the model. In contrast, our work aims to **discover** the unknown symmetry transformation from data without pre-specifying the network architecture to be equivariant to a specific type of groups.\n\n* **Yang et al. (ICML 2023):** This work also learns a matrix representation $R(g)$ of a group action. However, a key difference is that their representation acts directly on the *observed data* (i.e., $y' = R(g)y$). Our method solves a different problem: learning a latent representation $\\phi(y)$ and a group action that applies in that *latent space*. This allows our method to handle complex, high-dimensional observations where the corresponding group action in data space may not be linear.\n\n* **Yu et al. (JAIR 2023):** Thank you for this reference. Our understanding of \"Information Lattice Learning\" is that it learns transformations between data points in the *data space*, without an explicit connection to group theory or the goal of recovering a linear group representation. The \"information lattice\" provides a different, non-algebraic structure for understanding these relationships. We therefore see the goals and technical approaches as distinct from ours."}, {"review_id": "b2vC5tUDbe", "reviewer": "Reviewer_bGkb", "summary": "The authors present an algorithm to learn a representation of the group acting on a dataset as well as the latent space of the dataset. The author assume that the data $x_i $is only observable through a transformation $y_i = f(x_i)$. The authors address the problem by learning a mapping $\\phi \\approx f^{-1}$ (up to a linear transformation on x) and a matrix $R$ being a representation of the group action. They use a likelihood model as objective function to find $\\phi$ and $R$. $\\phi$ is represented with a neural network.", "questions": "- The model of Eq. (2) assumes that the same group action (i.e., the same matrix R) is applied to the complete dataset X. Why do the authors make this assumption? It would seem more realistic that different actions are applied on different data points.\n- Why using the terms \"content\" and \"style\" in section 2?\n- The authors do not detail how they deal with different (1000) group actions in their experiments. Indeed, the method on section 2 is only described in the case where a single group action is applied on the whole dataset.\n- In the section \"Metrics\", the quantities of Eq. (8)-(10) use $h$, which should be unknown since $h=\\phi\\cdot f$  and $f$ is unknown. How are these quantities computed in practice?", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "# Strengths\n- The introduction is well-written.\n- The concepts of section 2 are introduced with enough clarity to be understood.\n\n# Weaknesses\n- The proof of Theorem 1 is given in Appendix A.3 while the main text presents an \"informal\" version if the theorem. Unfortunately, the appendix does not present a formal version of the theorem. \n- The empirical evidence of Theorem 1 and Corollary 2 on a synthetic dataset, without noise, in section 4 is not convincingly explained.\n- In a context where one seeks to learn the group acting on a dataset, it does not seem realistic to assume that both $Y$ and $Y'$ are known.\n-In the subsection \"Metrics\", $\\hat{h}$ is not defined.\n- Experiments on $SO(n), O(n), GL(n)$, are limited to $n=3$.", "was_revised": true, "final_justification": "I thank the authors for answering my concerns. I have raised my rating accordingly.\n\nThe authors have carefully and satisfyingly answered the points I made in my review.\n\nThe clarifications made by the authors in their nine responses should be included in the final version.", "rebuttal": "We thank the reviewer for their feedback. We appreciate the opportunity to clarify the points raised and answer open questions.\n\n---\n### Re: Formal Statement of Theorem 1\n\nYou are correct that the manuscript would benefit from a more formal statement of the theorem in the appendix. We will update the appendix accordingly. Here is the full version we will include:\n\n**Theorem 1 (Identifiability of the Group Representation):**\nLet $\\phi$ be a model parameterizing the distribution $p_\\phi$ (Eq. 3), and let the model satisfy the diversity conditions in Def. 4. Let $f$ be an injective mixing function, and $g \\in G$ be a group element according to Def. 3. Let $R$ be the representation of $G$, and $\\hat R$ be the implicit representation of the group according to Eq (2) such that $\\hat R(X, gX) = R(g)$ for pairs of transformed samples $(X, gX)$ and group actions $g \\in G$.\n\nThen, for matching conditional distributions $p_\\phi = p_{f^{-1}}$, the following holds for all points $x$ and group actions $g$ the dataset:\n1. We recover the original vector space $\\phi(f(x)) = L x$, up to an ambiguity $L \\in GL(d)$.\n2. We recover a representation of group in the dataset, $\\hat R(f(X), f(gX)) = L R(g) L^{-1}$.\n\n---\n### Re: Empirical Evidence on Synthetic Data (Noise)\n\nWe believe this is a potential misunderstanding. Our empirical validation exactly matches the assumptions of Theorem 1 and Corollary 2.\n\nTo clarify the potential source of this misunderstanding. We indeed do require the assumption of the Diversity Condition (Appendix A, Def. 4). But the Diversity Condition doesn't not imply the presence of noise in the datasets. Instead it requires that there are enough group actions $g_i \\in G$ with sufficient variation such that this leads to the required diversity in the set of positive and negative pairs.\n\nNonetheless we do see the value in systematically studying the effect of noise on the data generating process.\n\nTo facilitate this, we adjusted our data generating process to include a gaussian noise term in the group action relationship in latent space:\n$x' = R(g_i) x + \\epsilon$ with $\\epsilon \\sim N(\\mu=0, \\sigma^2)$\nWe set $\\sigma^2 \\in [0.0, 10^{-5}, 10^{-4}, 10^{-3}, 10^{-2}, 10^{-1}]$ and additionally run with a different amount of samples per action ($2n$, $3n$, $4n$, $5n$) for $GL(n)$ and $n=3$.\nThe samples per action are the number of pairs $(x, x')$ we use to fit $\\hat{R}(g_i)$ for any given $g_i$.\n\nExcept for the described update to the group action relationship we follow the setup of Table 1 described in our paper.\n\n**Table 3.1: EbC on noisy data**\n|||Acc\\(C\\)|R²(G)|R²(x)|\n|-|-|-|-|-|\n|Samples Per Action|Noise Std.|||\n|6|0e+00|98.7±0.6|93.1±1.7|99.6±0.0|\n|6|1e-05|98.7±0.8|93.2±1.3|99.6±0.1|\n|6|1e-04|98.7±0.7|93.7±2.2|99.6±0.1|\n|6|1e-03|98.6±0.6|93.3±1.9|99.6±0.1|\n|6|1e-02|98.7±0.7|93.0±1.4|99.6±0.0|\n|6|1e-01|99.1±0.5|91.6±0.4|99.6±0.1|\n|9|0e+00|98.4±0.5|99.5±0.2|99.8±0.1|\n|9|1e-05|98.7±0.6|99.6±0.1|99.8±0.0|\n|9|1e-04|98.6±0.5|99.6±0.1|99.8±0.0|\n|9|1e-03|98.6±0.6|99.6±0.1|99.8±0.1|\n|9|1e-02|98.5±0.8|99.6±0.1|99.8±0.0|\n|9|1e-01|99.1±0.4|96.9±0.4|99.8±0.0|\n|12|0e+00|98.5±0.7|99.7±0.0|99.8±0.0|\n|12|1e-05|98.6±0.6|99.7±0.1|99.8±0.0|\n|12|1e-04|98.7±0.6|99.7±0.1|99.8±0.0|\n|12|1e-03|98.5±0.7|99.7±0.1|99.8±0.0|\n|12|1e-02|98.6±0.8|99.7±0.1|99.8±0.0|\n|12|1e-01|99.0±0.7|97.9±0.2|99.8±0.0|\n|15|0e+00|98.7±0.5|99.7±0.0|99.8±0.0|\n|15|1e-05|98.5±0.5|99.7±0.0|99.8±0.0|\n|15|1e-04|98.5±0.5|99.7±0.0|99.8±0.0|\n|15|1e-03|98.5±0.6|99.7±0.0|99.8±0.0|\n|15|1e-02|98.6±0.7|99.7±0.0|99.8±0.0|\n|15|1e-01|99.0±0.5|98.3±0.2|99.8±0.1\n\n**Summary:**\nIntroducing Gaussian noise results in a drop in performance in terms of identifying an GL(n)-equivariant representation. This can be observed via the $R^2(G)$ metric, which reduces from ~93% to ~91% for EbC models with $2n$ sample pairs per action used for fitting $\\hat{R}(G)$. But as we would expect for this noisy setting, increasing the number of samples per action seems to rectify this problem. As we increast to $3n$ samples per action, we get ~99.5% $R^2(G)$ for the noiseless setting and any noisy settings with $\\sigma^2 <= 0.01$, while the largest amount of noise tested ($\\sigma^2 <= 0.1$) results in ~96.9%.\nFinally, setting samples per action to $5n$, we get ~98.3% for the largest amount of noise tested and ~99.8% $R^2(G)$ for the less noisy settings.\n\n---\n### Re: Assumption of Known Data Pairs $(Y, Y')$\n\nWe appreciate you raising this concern, which Reviewer 4zKK raised as well.\n\nWe'd like to refer you to our answer there.\nBut for your convenience, TLDR: This is a standard assumption all recent related work tackeling the same problem setting is making. See CARE (Gupta et al., 2023), STL (Yu et al., 2024), and U-NFT (Koyama et al., 2023). For a more detailed discussion on related work, we refer to our response to Reviewer s1fj.\n\n---\n### Re: Definition of $\\hat{h}$\n\nWe apologize for this oversight. What we mean here is $h$, not $\\hat{h}$.\n\n---\n### Re: Experiments Limited to $n = 3$\n\nWe'd like to refer you to the appendix where we already show results for $SO(n), O(n), GL(n)$ with $n \\in [3,5,7,9]$.\n\n---\n### Re: Question on a Single Group Action\n\nWe feel, there may be a missunderstanding. Of course we don't make the assumption that there exists only a single group action (=same matrix $R(g)$) that is applied to the complete dataset.\n\nTo clarify:\nThe observed variables are $Y, Y'$ not $X$. $X$ instead are the embeddings $X = \\phi(Y)$.\nThe data generating process used for the theory and empirically validation on synthethic data is defined via Eq.(1).\nThe Eq.(2) on the other hand, simply describes how, given latent representations $X \\in \\mathbb{R}^{M \\times d}$ and $X' \\in \\mathbb{R}^{M \\times d}$ of $M$ observed sample pairs $Y, Y'$ we can compute an estimated representation $\\hat{R}(X, X')=\\hat{R}(g_i)$ of the group action $g_i$ these sample pairs share.\n\nIndeed for our experiments on the synthethic group dataset, we always generate 1000 random $R(g_i)$ from the respective group ($O(n), SO(n), GL(n)$).\nAnd in the case of dSprites, the $g_i = g_i^{(T)} \\circ g_i^{(R)}$ observed during training are actually compositions of translations $g_i^{(T)}$ and rotations $g_i^{(R)}$ group actions.\n\n---\n### Re: Question on \"Content\" and \"Style\"\n\nWith these terms we wanted to intutively refer to the unchanging components, the invariant part of the representations and the changing components so the equivariant part of the representations. Imagine the objects (content) of an image and the color, rotation, and scale (style) of these objects.\nHowever, if these terms are confusing or easily interpreted in a misleading way, we're happy to discuss alternatives.\n\n---\n### Re: Question on Dealing with 1000 Group Actions\n\nAs referenced to earlier, the way we do this is defined via Eq.(2) and by extension Eq. (3-5).\nTo clarify $Y, Y'$ or their embeddings $X, X'$ are **not** the whole dataset. Instead these represent a group of paired samples for which the pairs are all related by the same group action $g_i$. Think of them instead as $Y_i, Y_i'$.\nNote that Eq.(5) defines these are sampled from $p_{data}$, which represent the actual distribution over the full dataset.\n\n---\n### Re: Question on Computing Metrics\n\nYes, this absolutely correct $f$ is unknown. However, $f(x)$ is not, because $f(x) = y$ represent the observed variable. See Eq. (1). On the other hand $x$ in this case are the ground truth latents, which usually are unknown. This is why we required a purely synthethic data setting to exactly verify the theoretical claims empirically.\n\nIn practice for real-world datasets or benchmark datasets like DSprites where the ground truth latents are unknown these metrics can not be computed.\nThis is why we introduce the Acc(G, k) and Acc(C, k) as proxy metrics."}, {"review_id": "gCJEJwwgCq", "reviewer": "Reviewer_s1fj", "summary": "The paper presents a framework for learning embeddings that are equivariant with respect to the action of a finite group on the data. This is achieved by leveraging observed pairs consisting of a data point and its transformation under a fixed group element.", "questions": "The current state of the paper makes it very difficult to evaluate the work effectively. I understand the time and content constraints of the rebuttal phase; however, I believe it is both appropriate and necessary to request that the authors revise Section 2 - which outlines the proposed problem and methodology - as this revision is essential for enabling a meaningful re-evaluation of the paper. \n\nIt is important to note that this does not imply that only Section 2 requires revision. Rather, gaining a clearer understanding of the proposed method would facilitate a more informed and constructive assessment of the rest of the paper.", "limitations": ".", "rating": 2, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 1, "clarity": 1, "significance": 1, "originality": 2, "strengths_and_weaknesses": "**Strengths.** The supplementary material is provided and includes both the theoretical proofs and additional experimental details for the conducted studies. However, at this stage, I am unable to offer a substantive evaluation of the strengths of the work, due to the concerns outlined in the Weaknesses section below.\n\n**Weaknesses.** \n\n*1.* The paper contains numerous typographical errors, unclear notations, and ambiguous conceptual explanations, which significantly reduce the overall clarity and hinder the readability of the work. It is essential that the authors carefully address these issues in a revised version. Below, I list several examples from the first two sections - though this list is not exhaustive - and many more issues are readily apparent in the experimental sections.\n\n- Line 5-7: The notation $R_m, \\mathbb{Z}_n$ are not clear. What are periodic translations? \n- Line 9-10: non-Abelian -> nonabelian or non-abelian\n- Line 10: generalized linear group -> general linear group\n- Line 18: allows to study -> allows us to study\n- Line 19-22: Only when a group acts on a vector space (of finite dimensions), we have the group representation.\n- The caption of Figure 2 is non-informative since it has too many typos in crucial equations. What does \"y'_i = gy\", or \"x_i = R(g)x_i\" mean? \n- Line 21, 62, 64, etc.: \\mapsto -> \\rightarrow\n- Line 34: questions -> question\n- Line 38: perturbations[4] -> perturbations [4] \n- Line 58-59: \"which are grouped 59 into n + 1 pairs undergoing\":  What does \"pairs\" mean here?\n- Line 66: Information of  -> Information about\n- Line 66: what is \"u\"? Is it \"c\"?\n- Line 70: process Assume -> process. Assume\n- Line 77: the second \\phi(y_i') should be \\phi'(y_i')\n- Line 79: What is R(\\Phi, \\Phi')? The paper only defines R(g) or \\hat{R}.\n- Equation (3): What is S here?\n- Line 84: where used -> where we use\n- Line 97: What is Eq. (X)?\n- Line 98: GL should not be in italic.\n- Line 99: a -> an\n- Line 112: Does \"Assume that p_\\phi = p_{id}\" mean you assume that \\phi is the identity map?\n- Line 114: I am not sure why the original vector relates to the equation right after.\n\n*2.* The discussion of related work is insufficiently thorough and lacks critical engagement with relevant prior literature.\n\n*3.* The experiments are conducted solely on the dSprites dataset. Given that the authors reference a wide range of real-world applications - including biology, neuroscience, drug discovery, and computer vision - the paper should include at least one task or dataset from these domains to substantiate these claims.", "was_revised": true, "final_justification": "I remain concerned about the numerous typographical errors and the lack of clarity in several parts of the manuscript. Many of the clarifications provided during the rebuttal phase should be integrated directly into the paper to enhance its readability and comprehensiveness. These unresolved issues continue to detract from the overall presentation. Although I have revisited my evaluation in light of the improvements, my recommendation still leans toward rejection at this point.", "rebuttal": "Dear reviewer, thanks a lot for your critical assessment of our work which raised several important points.\n\n**1. Typos and Questions.**\n\n> L 5-7, periodic translations:\nWe mean translations with periodic boundaries.\n\n> Figure 2 caption; What does $y'_i = gy$, or $x_i = R(g)x_i$ mean?\n\nApologies for the confusion, the new caption reads: $n+1$ samples are transformed using the same group action $g$, yielding samples $y$ and $y'$ in observation space. An encoder $\\phi$ maps these samples into latent space ($\\hat x, \\hat x'$), where $n$ samples are used to estimate a linear representation $R$ of the action. The minimizer of the contrastive loss ensures that $\\hat x_i = R \\hat x_i$.\n\n> L 58-59: What does \"pairs\" mean here?\n\nConsider two data points with latent representation $(x,x')$ such that $x'=gx$; we observe $(f(x),f'(x))$. We leverage $n+1$ such pairs to jointly discover the latent space and a representation of the group. Note, this is directly defined in the definition following the sentence.\n\n> L 66: u/c\n\nTypo: we mean $c$ as used in Eq. 1 -- apologies.\n\n> L 79: What is $R(\\Phi, \\Phi')$?\n\nTypo: we mean $\\hat R(\\Phi, \\Phi')$, as defined in Eq. 2.\n\n> Eq (3): What is $S$?\n\nNegative examples as implicitly described in L. 89; we added a sentence. These are uniformly drawn from the dataset.\n\n> L 97: Eq. (X)?\n\nEq. 4\n\n> Line 112: Does \"Assume that $p_\\phi = p_{id}$\" mean you assume that $\\phi$ is the identity map?\n\nYes, exactly, but this is a typo that slipped through from an earlier formulation. Given Eq 3, this should read \"Assume that $p_{\\phi} = p_{f^{-1}}$ [...]\". Our goal is that if $\\phi \\circ f$ recovers a feature space in which we match the ground truth distribution, $\\phi \\circ f$ becomes linear (see Thm 1 in response to Reviewer bGkb).\n\n> Line 114: I am not sure why the original vector relates to the equation right after.\n\nThe original vector $x$ is mapped to observed data $f(x)$ and then mapped back to a vector $\\hat x = \\phi(f(x))$. EbC assures that $\\hat x = Lx$, even though $f$ can be an arbitrary non-linear function (as long as it is bijective).\n\n**To your question about Section 2**, we would be happy to discuss more specifics of the method. We believe that beyond the typos, the level of detail in Sec. 2 is self-sufficient. Could you let us know which part of the Methods need clarification?\n\n**2. Related work**\n\nWe agree, and substantially extended our review of related work. See below.\n\n**3. Real-world data**\n\nGreat suggestion. We added a real-world use case from neuroscience, where a rat is running on a linear track and hippocampal place cells are recorded. Please see our reply to reviewer zh3E.\n\n### Extended discussion of related work\n\nEquivariant representation learning has been a prominent topic in ML research from different angles. Below we denote observed data as $y \\in Y$, with $y'=g \\circ y$ as short-hand for the transformed data. Embeddings of the data are denoted as $x, x' \\in X$, where $X$ is a vector space. They are related via the representation $R: G \\rightarrow GL(n, X)$ via $x' = g \\cdot x = R(g) x$. We focus on methods for learning vector embeddings $x$ of the data $y$, which are equivariant and/or invariant to group actions $g$. In contrast, orthogonal related work focuses on learning representations of the group given $X$, $Y$ (Yang et al., 2023; Laird et al., 2024). Another rich area of literature focuses on neural network architectures which are invariant/ equivariant to specific predefined groups (Cohen et al., 2016; Satorras et al., 2021; Kondor et al., 2018; Finzi et al., 2020; Dehmamy et al., 2021). \n\n**Non-Invariant Methods.** Invariance to all possible group actions may hinder downstream tasks (e.g. color-invariance for flower-type classification). Xiao et al. (2021) propose to use a contrastive learning framework to learn invariant subspaces to different types of augmentation. Eastwood et al. (2023) extend this by introducing a second entropy loss to encourage disentangled subspaces.\n\n**Equivariant & Invariant Representation with known group action $g$ (explicitly observing g in a parameterized form).** E-SSL (Dangovski et al., 2021) uses contrastive learning to embed a reference sample of $y$ and multiple transformed samples $y_i'=g_i \\circ y$ into a latent space $x$, $x_i'$, which is split into an invariant and equivariant subspace. Invariant parts are learned via SimCLR (Chen et al., 2020) and equivariant parts through an auxiliary task that predicts the parametrized group action $g_i$ from $x_i'$. EquiMod (Devillers et al., 2022) achieves equivariance by embedding pairs of $y$, $g \\circ y$ by modeling the group action via a neural network $u_\\psi(x, g) = \\hat{x}'$ to approximate $x'$. Park et al. (2022) leverage G-equivariant neural networks. Garrido et al. (2023) propose a variant using non-contrastive SSL losses.\n\nAuto-encoder approaches form an alternative: Qi et al. (2022) encode pairs $y, y'$ into latent space and decode $g$ from concatenated embeddings $x, x'$. Jin et al. (2024) embed $y$ into latent space, model the group action as $x' = R(g) x$ and decode $y'$ from the predicted $x'$, assuming full knowledge of $R(g)$. Keurti et al. (2023) use a linear prediction $x' = R(g)x$ where $R(g)$ is predicted by a neural network from the observed and parametrized $g$, essentially an auto-encoding variant of Garrido et al., 2023.\n    \n**Equivariant \\& Invariant Representation with *unknown* group $g$ (implicitly observing $g$)** Although the aforementioned approaches share conceptual similarity in their goal of learning invariant and equivariant embeddings of data by modeling linear relations in latent space, EbC does not assume knowledge of the parametrized form of group actions $g$. Instead of observing information about $g$ directly, a second class of methods assumes that two or more pairs of data share the same underlying action, $y_i$, $g \\circ y_i$. This is a key assumption we also require for EbC.\n\nEncoder-only methods include CARE (Gupta et al., 2023), a contrastive learning framework to learn invariant and locally equivariant representations. CARE encodes two pairs of observations $(y_1, g \\circ y_1)$ and $(y_2, g \\circ y_2)$ with the same action $g$ such that the embeddings are related by the same matrix $R_g \\in O(d)$ via $x_1' = R_g x_1$ and $x_2' = R_g x_2$. CARE extends the InfoNCE loss with an equivariant term based on $x_1^T x_1'$ and $x_2^T x_2'$. The invariant loss term of InfoNCE and the new equivariant loss are weighted and applied to the full embedding space. Yerxa et al. (2024) propose a variation of CARE in which the embedding space is split into an invariant and equivariant subspace group actions are encoded as $x_1' = x_1 + b_g$ and $x_2' = x_2 + b_g$. STL (Yu et al., 2024), learns representations of $y, y'$ such that $x, x'$ are equivariant to the group action g and to learn a representation of $g$ itself, again from two pairs of data. Similarly to EquiMod, the group action is parameterized by a neural network. However, instead of assuming knowledge about g, they use a learned representation $R_g$ as the second input to the neural network: $x_1' = u_\\psi(x_1, R_g)$ where $R_g$ is predicted by another network $R_\\theta(x_2, x_2')=R_g$ from the latent representations of the second observed pair $(y_2, g \\circ y_2)$. To learn the correct representation of $R_g$, EquiMod is extended by a third loss term that maximizes the similarity of $R_\\theta(x_1, x_1')$ and $R_\\theta(x_2, x_2')$. Like CARE, they do not learn separate subspaces and instead use weighting factors for the invariant and equivariant loss terms on a joint latent space.\n\nThe problem can also be approached with auto-encoding approaches: Winter et al. (2022) learn embeddings of $y$ and $g$ in which the factorize $y'= g \\circ y$ into an representation $\\hat{x}$ of $y$ which is invariant to $g$ and a representation $R_Y(g)$ that represents the action $g$ in the data space $Y$, such that $y' = R_Y(g) \\delta(\\hat{x})$ with $\\delta$ being the decoder. \n\nThe Unsupervised Neural Fourier Transform (U-NFT; Koyama et al., 2023; Miyato et al., 2022) is an auto-encoder framework with a linear group action model in latent space.  For their learning setup, Koyama et al. assume access to N sequences of data points $(y^0, ..., y^i, ..., y^N)$ with $y^i = (y^i_0, ..., y^i_T)$   with $y^i_{t+1} = g_i \\circ y^i_t$, one sequence for each implicitly observed, but unknown group action $g_i \\in G$. An autoencoder reconstructs $y^i_{t+1}$ from the predicted $\\hat{x}^i_{t+1}$, which in turn is predicted from $\\hat{x}^i_{t+1} = R(g_i) x^i_{t}$. The representation from these examples is estimated using least squares and a post-hoc basis transformation on the set of $R(g_i)$ to find a block diagonal representation $B(g_i)$ to facilitate disentanglement of the irreducible components of $R(g_i)$. Mitchel et al. (2024) propose a variation of NFT, where the learning setup is restricted to directly produce a block-diagonal representation $R(g_i)$, avoiding the need for a post-hoc basis transformation. However, to achieve this, they also restrict $R(g_i)$ to be orthogonal.\n               \nWinter et al. (2022) and Allingham et al. (2024) only recover an invariant representation and model the group action separately in the data space. All other the methods in this section try to solve the same general task of learning equivariant and invariant representations of the data without explicit knowledge of the underlying group actions. CARE (Gupta et al., 2023) is the closest encoder-only (contrastive) approach to EbC, but constrains the embedding to the hypersphere, which imposes additional structure on the learned representation, while EbC can learn different topologies (e.g., torus in Fig. 4, hypersphere in Fig. 5). In terms of function and data requirements, U-NFT (Miyato et al., 2022; Koyama et al., 2023) is the conceptually closest work, but requires to learn a full generative model of the data."}]

Author rebuttal:

Meta-review: The paper introduces a contrastive learning framework for group representation learning with theoretical guarantees of identifiability, supported by synthetic and 2D image experiments. Reviewers consistently highlight the clarity of the writing, strong motivation, and intuitive visuals. The reviewers noted the simplicity of the method: encoder-only with least-squares fitting, avoiding generative models, and also for achieving strong empirical results across diverse groups (SO(3), O(3), GL(3), product groups), often outperforming baselines. Theoretical contributions, especially Theorem 1, are noted as uncommon in equivariant self-supervised learning, though some reviewers find the novelty over prior work (e.g., Roeder et al.) marginal. Weaknesses center on missing details or limitations: (1) data efficiency and scaling behavior are untested, leaving unclear how performance depends on dataset size, batch size, or negative samples; (2) comparison to recent related baselines (e.g., Park et al. 2022, Keurti et al. 2023, Dehmamy et al. 2021, Yang et al. 2023, Yu et al. 2023) is incomplete; (3) the strong assumption that all samples in a batch share the same group action is unrealistic and unvalidated on real data; (4) experiments are restricted to synthetic datasets, without noisy or realistic applications. 

Despite these critiques, the overall consensus is positive: the theoretical guarantees, simplicity of the framework, and strong synthetic performance outweigh the weaknesses, warranting acceptance. One reviewer had concerns about notation and clarity, but on following the responses and reading the paper, I find the critique to be outsized -- should be easily fixable. Given all these considerations, I would recommend acceptance. The authors are encouraged to incorporate various points that have come up during the discussion, and improve discussion of prior work.
Decision: Accept (poster)