Structuring Representations using Geometric Invariants
Anonymous Author(s)
AffiliationAddress email
Abstract
Different geometries can be defined by their invariants. For example, distancesand angles are preserved in Euclidean and Conformal geometries, respectively.
Wepropose to learn equivariant representations using loss functions that enforce thecorresponding geometric invariants.
This approach requires no prior knowledgeabout the correspondence of potentially complex transformations in the input spaceand the symmetry group.
We consider geometric invariants for different Lie groupssuch as the Euclidean, Orthogonal and Unitary groups and additionally show thefeasibility of learning disentangled and decomposed representations.
We providefavorable qualitative and quantitative results on downstream tasks, including worldmodeling and reinforcement learning.
Introduction
Sample efficient representation learning is a critical open challenge in deep learning for AI.
Whenwe have prior information about transformations that are relevant to a particular domain, buildingrepresentations that are aware of these transformations can lead to better sample efficiency andgeneralization.
One way to use such symmetry priors is to make the network invariant to the giventransformations.
A generalization of this idea is called equivariance, where transforming the inputtransforms the output in a specific way.
An equivariant network that makes good predictions for aparticular input also generalizes to all input transformations, making symmetry a useful prior.
While recent years have witnessed a range of exciting equivariant deep models, there are severallimitations.
First, most equivariant networks constrain the network architecture, which sometimesrequires specialized implementations.
Moreover, transformations considered by the existing methodsare often assumed to be linear in both input and representation space.
This is the case for architecturesdesigned for finite permutation groups and continuous Lie groups.
Approaches that go beyond lineartransformations in the input space often assume access to group information – i.e. , the group memberthat transforms one input to another is known.
This paper introduces a simple approach that addressesall of these limitations.
We are inspired by the connection between various geometries and group theory as characterizedby Klein’s Erlangen program [32].
According to this view, geometries are concerned with invariantquantities under certain transformations.
For example, Euclidean geometry is concerned with thelength, angle, and parallelism of lines, among others, because Euclidean transformations preservethese.
However, moving to the more general and less structured Affine geometry, notions of distanceand angle are no longer relevant, while parallelism remains an invariant of the geometry.
Thecorresponding symmetry groups are examples of Lie groups that have a subgroup relation, E ( n ) <
Aff ( n ) , thereby enabling the groups to characterize a hierarchy (or lattice) of different geometries.
We propose to induce a geometry on the embedding and make it equivariant to a given groupby enforcing its defining invariant.
For example, distance is the defining invariant for Euclidean geometry, which means all distance preserving transformations are Euclidean.
Therefore, to enforceequivariance to the Euclidean group, it is sufficient to ensure that the embedding of any two datapoints has the same distance before and after the same transformation of the inputs; see Figure 1.
While this approach uses the defining action of different Lie groups in the embedding space, thesame group can have a non-linear and unknown action on the input space.
In the pendulum exampleof Figure 1, the group E ( 3 ) acts on the value of each pixel of the input image using an unknownand non-linear action.
Moreover, this approach does not require the pairing of group members withtransformations, a piece of information that is often unavailable.
In the rest of the paper, we arrive at the idea above from a different path: after reviewing related worksin Section 2 and providing a background in Section 3, Section 4 observes that equivariance, in itsgeneral form, can be a weak inductive bias.
This is because having an injective code is sufficient forequivariance to “any” transformation group.
However, in this manifestation of equivariance, the groupaction on the embedding can be highly non-linear.
Since the simplicity of the action on the embeddingseems essential for equivariance to become a useful learning bias, Section 5 proposes to regularize thegroup action on the code to make it “simple”.
This symmetry regularization (SymReg) objective isgroup-dependent and the essence of our approach.
Enforcing geometric invariants in the latent spaceis proposed as a symmetry regularization.
While we focus on equivariant representation learningthrough self-supervision, in principle, supervised tasks can also benefit from the proposed SymReg.
An important benefit of a symmetry-based representation is its ability to produce disentangledrepresentations through group decomposition [28].
Section 6 studies disentanglement using SymReg.
Section 7 presents a range of experiments to understand its behavior and puts it in the context ofcomparable baselines.
Related Works
Finding effective priors and objectives for deep representation learning is an integral part of the questfor AI [3].
Among these priors, learning equivariant deep representations has been the subject ofmany works over the past decade.
Many recent efforts in this direction have focused on the design ofequivariant maps [53, 14, 45, 34, 16, 24, 52, 20, 6] where the “linear” action of the group on the datais known.
Due to this constraint, the application of these models has been focused on fixed geometricdata such as images [36], sets [56, 43], graphs [39, 33], spherical data and the (special) orthogonalgroup [15, 1, 48, 23], the Euclidean group [50, 9, 25] or other physically motivated groups such asthe Lorentz [4] or Poincare group [52], among others.
In the present work, the group action is unknown and possibly non-linear.
Therefore, in contrast tomany prior works, linear representation theory plays no significant role.
Our setup is closer to thebody of work on generative representation learning [7, 12, 40], in which the (linear) transformationis applied to the latent space [44, 54, 35, 37, 17, 22].
Among these generative coding methods,transforming autoencoder [29] is a closely related early work which, in addition to equivariance,seeks to represent the part-
Since we consider learning equivariant representations through self-supervision, exciting recentprogress in this area is also quite relevant [26, 41, 10, 51, 27, 57, 21].
While the use of transformations is prominent in these works, in many settings, the objective encourages invariance to certaintransformations, making such models useful for invariant downstream tasks such as classification.
Similar to many of these methods, we also use transformed pairs to learn a representation, with thedistinction of learning an equivariant representation.
An exception is the recent work of Dangovskiet al.
[18], which learns an equivariant representation by separating the invariant embedding from thepose, where the relative pose is learned through supervision.
Therefore, in that work, in contrast toours, one needs to know the transformation that maps one input to another.
When considering the
Euclidean group, SymReg preserves distances in the embedding space under non-linear transformations of the input.
This embedding should not be confused with isometric embedding [49], where theobjective is to maintain the pairwise distances between points in the input and the embedding space.
Background on Symmetry Transformations
We can think of transformations as a set of bijective maps on a domain X .
Since these maps arecomposable, we can identify their compositional structure using an abstract group G .
For thisreason, such transformations are called group actions.
To formally define transformation groups,we first define an abstract group.
A group G is a set equipped with a binary operation, such thatthe set is closed under the operation gg ′ ∈
G ∀ g,g ′ ∈ G , every g ∈ G has a unique inverse suchthat gg − 1 = e , where e is the identity element of the group, and the group operations are associative( gg ′ )
g ′′ =
g ( g ′ g ′′ ) .
A G -action on a set X is defined by a function t ∶ G × X → X , which can be thought of as abijective transformation parameterized by g ∈ G .
In order to maintain the group structure, the actionshould satisfy the following two properties: (1) action of the identity is the identity transformationt ( e,x ) =
x ; (2) composition of two actions is equal to the action of the composition of group elementst ( g,t
( g ′ x ))
=
t ( gg ′ ,x ) .
The action t is faithful to G if transformations of X using each g ∈ G areunique – i.e. , ∀ g,
g ′ ∃ x ∈ X s.t. t ( g,x ) ≠ t ( g ′ ,x ) .
If a G -action is defined on a set X , we call X a
G -set.
Many groups are defined using their defining action; for example, the SO ( 3 ) is the group ofrotations in 3D space.
While this defining action is a linear transformation, the same group can actnon-linearly on R n using the action t ∶ SO ( 3 ) × R n → R n .
Equivariance is Cheap, Actions Matter
A symmetry-based representation or embedding is a function f
∶ X → Z such that both X and Z are
G -sets, and furthermore f “knows about” G -actions, in the sense that transformations of the inputusing t X have the same effect as transformations of the output using some action t Z :
[Equation]
The following claim shows that despite many efforts in designing equivariant networks, simplyasking for the representation to be equivariant is not a strong inductive bias, and we argue that theaction matters.
Put another way, the strong performance of existing equivariant networks should beattributed to the fact that the group action on the embedding space is simple (linear).
Proposition 4.1.
Given a transformation group t X ∶ G × X → X , the function f ∶ X
→
Z is anequivariant representation if ∀ g ∈ G,x,x ′ ∈ X
[Equation]
That is, two embeddings are identical iff they are identical for all transformations.
The proof is in the appendix.
The condition above is satisfied by all injective functions, indicatingthat many functions are equivariant to any group.
Corollary 4.2.
Any injective function f
∶ X → Z is equivariant to any transformation groupt X ∶ G × X → X , if we define G action on the embedding space as
[Equation]
The ramification of the results above in what follows is two-fold:1.
While injectivity ensures equivariance, the group action on the embedding as shown in Equation (3)can become highly non-linear.
Intuitively, this action recovers x = f − 1 ( z ) , applies the group actionx ′
=
t X ( x ) in the input domains and maps back to the embedding space f ( x ′ ) to ensure equivariance.
In the following, we push t Z towards a simple linear G -action through optimization of f .
Thisobjective can be interpreted as a symmetry regularization or a symmetry prior (SymReg).2.
Although Corollary 4.2 uses injectivity of f for the entire X , we only need this for the datamanifold.
In practice, one could enforce injectivity on the training dataset D using a decoder,architectural choices such as momentum encoder [27], or loss functions defined on the training data,such as a hinge loss [26] L hinge ( f,D ) =
∑ x,x ′ ≠ x ∈ D max ( ϵ − ∥ f
( x ) − f ( x ′ )
∥ , 0 ) or other losses thatmonotonically decrease with distance, such as 1 ∥ f ( x )− f ( x ′ )∥ , or its logarithm − log (∥ f
( x ) − f ( x ′ )∥) .
In experiments we use the logarithmic barrier function.
Symmetry Regularization Objectives
In learning equivariant representations, we often do not know the abstract group G and how ittransforms our data, t X .
We assume that one can pick a reasonable abstract group G that “contains”the ground truth abstract group acting on the data – i.e. , G action on the input may not be faithful.
Our goal is to learn an f ∶ X
→ Z that is equivariant w.r.t.
the actions t X ,t Z , where t X ∶ X × G → Xis unknown and t Z is some (simple) G -action on Z of our choosing.
More Informed but Less Practical Setting.
In the most informed case, the dataset also containsinformation about which group member g ∈ G can be used to transform x to x ′ – that is the dataset consists of triples ( x,g,x t = t X ( g,x )) .
By having access to this information we can regularize the embedding using the following loss function: L informed G ( f,D ) =
∑ ( x,g,x t )∈ D ℓ ( f ( x t )− t Z ( g,f ( x ))) , where ℓis an appropriate loss function such as the square loss.
At its minimum, we have f ( x t ) =
t Z ( g,f ( x ))or f ( t X ( g,x ))
=
t Z ( g,f ( x )) , enforcing equivariance condition of Equation (1).
However, even ifthe optimal value is not reach, due to its injectivity, f is still G -equivariant, and the the objectiveabove is regularizing the G action on the code.
This informed setup is used in equivariant contrastivelearning of [18].
The assumption of having access to g is realistic when we know the action t X , sothat we can generate ( x,g,x t ) triplets.
Fortunately, using group invariants, we may still learn anequivariant embedding, even if we do not have the group information tied to the dataset.
Euclidean Group.
The defining action of the Euclidean group E ( n ) is the set of transformationsthat preserve the Euclidean distance between any two points in R n , a.k.a.
isometries.
These transformations are compositions of translations, rotations, and reflections.
Since for the real domain, all Euclidean isometries are linear and belong to E ( n ) , we can enforce the group structure on the embedding by ensuring that distances between the embeddings before and after any transformation match.
For this, we need the dataset D to be a set of pairs of pairs (( x,x t = t X
( g,x )) , ( x ′ ,x ′ t =
t X ( g,x ′ ))) ,where x,x ′ are transformed using the same unknown group member g .
Distance-preservation lossbelow combined with injection loss are sufficient to produce an E ( n ) -regularized
embedding:
[Equation]
[Table]
[Equation]
For example, in the standard RL setup, where we have access to triplets ( s,a,s ′ ) , we can easily form
D by unrolling an episode and collecting two different state transitions corresponding to a particularaction.
In practice, with a finite actions, we can efficiently generate this dataset by keeping separatebuffer for each action where we store state transitions for that action and sample from that buffer totrain the embedding function f .
Orthogonal and Unitary Groups.
The defining action of the orthogonal group O ( n ) preservesthe inner product between two vectors.
The analogous group in the complex domain is the unitarygroup, which preserves the complex inner product.
Our symmetry-regularization objective enforcesthis invariant: L O ( n
[Equation]
x ′ t )) .
For the unitary group one additionally needs to embed to complex domain Z = C n , where the onlydifference is in the definition of the inner product.
Conformal Group.
The invariant of conformal geometry is the angle.
In a Euclidean embedding,conformal transformations include a combination of translation, rotation, dilation, and inversionwith respect to an n − 1 -sphere.
To enforce this group structure we need a triplets of inputs, beforeand after a transformation (( x,x t ) , ( x ′ ,x ′ t ) , ( x ′′ ,x ′′ t )) , so that we can calculate the angle in theembedding.
Conformal SymReg objective, which preserves angles, imposes a weaker constraint onthe embedding than the distance preservation of the Euclidean group – since the latter implies theformer.
Moreover, it has an additional benefit that, compared to L E ( n ) , the loss cannot be minimizedby simply shrinking the embedding.
Therefore in practice, the injection enforcing losses of Section 4is not as crucial when using conformal symmetry regularization.
Other Groups and Geometries.
Many other classical lie groups are equipped with defininginvariants that are sometimes more expensive to compute.
Special Linear group preserves volumeand orientation; Lorentz and Poincare group are the analogs of the orthogonal and Euclidean group inthe Minkowski space, therefore equipped with similar invariants; the Symplectic group preserves abilinear form.
Moreover, we can also use SymReg for finite groups; see Appendix C.
Decomposing the Representation
Higgins et al.
[28] suggested a notion of disentangled representation based on decomposition of theabstract group into a direct product form G =
G 1 × ...
× G k .
There are two approaches to learningsuch decomposed representation using SymReg, depending on whether or not we can perform certaintypes of transformations in isolation.
For example, an RL agent may transform its environmentthrough actions like moving a single limb that can be performed in isolation.
In this case, we call thedecomposition active to contrast it with the passive case where the action of different subgroups isalways mixed in our dataset.
Active Decomposition.
Let G =
{( g 1 ,...,g k ) ∈ G 1 × ...
× G k } be a product group, where G i ≅{
( e,...,e,g i ,e,...,e ) ∈ G } can be identified with a normal subgroup of G .
In active decomposition,sub-groups can act in isolation and therefore we have k types of tuples in our dataset D 1 ,...,D k ⊂ D .
Each subset D
i is associated with actions of a subgroup G i using t X (( e,...,e,g i ,e...,e ) , ⋅)) ,g i ∈
G i .
In this setting, the representation f ∶ X → Z = Z 1 × ... × Z k can be thought of as k separatefunctions where f i ∶ X
→
Z i is equivariant to G i -action and invariant to all G j ,j ≠ i actions.
Thisgives the following objective
[Equation]
equivariance to G
[Equation]
invariance to G j for j [Equation] i
[Equation] (5) where L inv. G ( f,D ) enforces invariance of f to G -transformations in D – e.g., by penalizing∥ f ( x ) − f ( t X ( g,x ))∥ .
Passive Decomposition.
When we have no control over transformations, and we are simplygiven the data, it is still possible to use an abstract group that has a product form.
Here again,f ∶ X → Z = Z 1 × ...
× Z k , but the loss function is simply enforced on each block separately –i.e. , L passive G ( f,D )
=
∑ ki = 1 L G i ( f i ,D ) , where L G i ( f i ,D ) is a SymReg objective from Section 5.
Experiments
We conducted many experiments to qualitatively study the representation learned by SymReg andits ability to produce a disentangled representation, and quantitatively compare it against simplebaselines in representation learning and downstream RL tasks.
For details of architecture and training,see Appendix G.
7.1 Qualitative Analysis
In this section, we visualize the representation learned for two examples from the Gym environment[5], including the pendulum and the mountain car (see Appendix D), followed by an experimentinvolving a rotating object where we know the ideal embedding is the SO ( 3 ) manifold.
Finally,
Figure 4 visualizes a conformal embedding for double-bump world.
In most cases we also visualize a
Variational AutoEncoder (VAE) [30] embedding for comparison.
Our objective here is to visuallydemonstrate the behavior of SymReg and its remarkable ability to learn an embedding informed bythe non-linear transformation of the input.
7.1.1 The Pendulum
[Figure] For this experiment, the input x is two consecutive frames of the pendulumthat have been grayscaled and downsampled to 32×32 pixels.
The action spaceis a range of torques that can be applied to the base of the pendulum.
Weuse the action to transform the data.
We use the objective of Equation (4) tolearn an E ( 3 ) -equivariant representation.
To efficiently estimate L E ( n ) , weuse a mini-batch that consists of 64 randomly sampled observations from theenvironment and their transformations via three randomly sampled actions( 4 × 64 samples in total).
Once the embedding f ( x ) is produced, the pairwisedistance between all 64 2 pairs is calculated, and the mini-batch loss penalizesthe change in these distances between any pair of transformations in the mini-batch.
Therefore, usingthis mini-batching procedure, the complexity of calculating the loss grows quadratically with boththe number of independent samples and transformations.
The model learns to parameterize the embedding using the angle and the angular momentum ofthe pendulum from the input data; see Figure 1.For comparison, Figure 3 shows the embeddinglearned using a VAE from the same dataset.
We perform a similar experiment in the Mountain Car Environment and report the results in Appendix D.
7.1.2 Rotating Chair
We consider a 3D chair from ModelNet40 [55] and transform it through the action of the group SO ( 3 ) .
The group action on the input is the 2D projection into a 48 × 48 image after the 3D rotationof the chair.
While the group of interest is SO ( 3 ) , we use SymReg loss of Equation (4).
In choosingthe abstract group, we often only need to ensure that the group is large enough to contain the groundtruth as a subgroup. Although this results in a stronger symmetry regularization on the embedding,a G -equivariant embedding is equivariant to any H ≤ G . This means that, for example, an E ( 2 )equivariant embedding can be useful for its finite subgroups such as dihedral or cyclic groups.
Weembed the chair in R 16 using SymReg 1 and visualize the latent by rotating the chair along threeorthogonal axes and projecting the latent codes into a 2D space.
Figure 2 shows three circular latenttraversals of SymReg embedding corresponding to rotation around each axis, which is consistent with the structure of SO ( 3 ) manifold.
The process of learning SO ( 3 ) manifold is a challenging task,and previous works assumed that the group member corresponding to each transformation is given[44, 2].
In contrast, we only use the observations corresponding to similar actions during trainingand not the group members themselves.
As we see later, this is critical in settings such as RL, wheregroup information is unavailable.
We could not produce a similar latent traversal for VAE due tocollapse when rotating around some axes.
7.1.3 Conformal Embedding for Double-Bump World
[Figure]
The double-bump world consists of a rectangular bump signal and a triangular bump signal,both cyclically shifted and superimposed.
Thesetransformation are given by a pair ( ∆ 1 , ∆ 2 )which cyclically shifts the rectangular bump by∆ 1 and the triangular bump by ∆ 2 .
In our experiments, the signal length is 64, and the lengthof the bump is 16.
SymReg embeds to a 4dimensional conformal space in this example.
Figure 4(right) shows the random projection ofthe embedding, where the colors change as the triangle bump moves.
The figure suggests that SymRegcan successfully learn to represent a data point using the location of two bumps.
For comparison, weshow the embedding found by VAE and SimCLR [10].
7.2 Experiments on Active and Passive Decomposition
In this section, we first contrast active and passive decomposition in their ability to disentangle thetwo bumps in the double bump world.
We observe that while both can decompose the embedding intoa product form SO ( 2 ) × SO ( 2 ) , only active decomposition leads to disentanglement.
Section 7.2.2applies active decomposition to a more complex setting of ego-motion, where SymReg can decomposethe representation of the agent’s state into location and orientation.
7.2.1 Decomposition of the Double-Bump World
Here, we compare the active and passive decomposition for the same double-bump world.
While theground truth is SO ( 2 )× SO ( 2 ) , SymReg uses the larger group E ( 2 )× E ( 2 ) .
In the active case, eachsubgroup moves one of the bumps, and the loss of Equation (5) is used to learn an embedding foreach subgroup.
In the passive case, both bumps move randomly.
Figure 5 compares the decomposed embedding found in each case.
While in both cases, the SO ( 2 ) × SO ( 2 ) torus is decomposed into aproduct of circles, only the active case successfully disentangles the two bumps.
Note that the color ofeach point is based on the location of the triangle bump.
Our results agree with Caselles-Dupré et al.[8] who claim that learning a disentangled representation requires interaction with the environment;see also [42, 40].
However, we note that while the disentangling of the bump movements does nothappen in the passive case, we can still successfully “decompose” the embedding.
7.2.2 Active Decomposition for Ego-Motion
We used a modified version of the single-room environment of MiniWorld [13] forthis experiment.
The agent is standing in a 3D room containing eight differentlycolored boxes around the walls.
A map of the room can be seen in Figure 7.
Anobservation consists of a first-person view of the room, downsampled to 32 × 32pixels.
The agent can rotate left/right or move forward/backward.
We learn an E ( 2 ) × E ( 2 ) equivariant embedding using the active decomposition objectiveof Equation (5).
Each mini-batch consists of 64 random observations and the resultof applying all four actions in those states ( 4 × 64 samples in total).
Figure 6 visualizes the embedding of the input in two sub-figures, where the more prominent figureshows the embedding corresponding to the rotation action, and the more petite figure (bottom right)shows the embedding corresponding to forward-backward movement.
The first figure also shows thefirst-person view when the agent rotates while standing in the middle of the room.
The correspondingmarkers collapse around the center of the second embedding, demonstrating an intuitive embeddingparameterized by rotation angle and location.
Walking straight across the room also produces theexpected behavior of traversing the second manifold while the rotation angle, for the most part,remains fixed (not shown).
7.3 Quantitave Evaluation in Downstream Tasks
7.3.1 World Modelling
[Table]
We select the Atari games Pong and Space Invaders as our environments for the world modeling experiments.
These environments were previously used by Kipf et al.
[31] to evaluate the
Contrastive Structured World Model (C-SWM).
We train the encoder using Euclidean SymRegof Equation (4), freeze it, and then learn a MultiLayer Perceptron (MLP) based transition function in the latent space.
Following Kipf et al.[31], we report Hits at Rank 1 (H@1) and Mean Reciprocal Rank (MRR), which are invariant tothe embedding scale.
These evaluation metricsmeasure the relative closeness of the followingstate’s representation predicted by the transition model and the representation of the observed nextstate.
We use a set of reference state representations to measure the relative closeness (embeddingrandom observations from the experience buffer).
Table 1 reports these measures and shows thata simple transition model learned on top of our embedding outperforms C-SWM in both games.
Other reported baselines use an AutoEncodcer (AE) and a Variational AutoEncoder (VAE) to learnembeddings.
7.3.2 Reinforcement Learning
[Table]
Next, we consider three Mujocoenvironments:
InvertedPendulum, Reacher, and Swimmer from OpenAI
Gym [5] and learn directly from theimage observations.
We compare ourmodel with Auto-Encoder (AE) and
Self-supervised Learning (SSL) basedbaselines.
While AE learns to reconstruct the image observations ofthe states, SSL learns to inject invariance (IN-SSL) or equivariance (EQSSL) to agent actions.
Given a triplet( s,a,s ) , IN-SSL maximizes the likelihood of f ( s ) and f ( s ′ ) being similar(SimCLR [11]).
EQ-SSL of Dangovski et al.
[19], in this context, additionally predicts the actionthat lead to the state-transition.
We introduce two variations of each model.
In the first variation,the low-dimensional embedding is used as a substitute for the high-dimensional input data withoutfurther adjustment (-decoupled).
The second variation allows for fine-tuning during the reinforcementlearning stage (-fine-tuned).
We use random policy to collect trajectories for the pre-training and use Proximal Policy Optimization (PPO) [47] algorithm for the downstream RL task.
To evaluate thedata-efficiency of these models, we report the average reward collected over 10 episodes in the first100,000 steps for Reacher and Swimmer and 30,000 steps for Inverted Pendulum in Table 2 (since Inverted Pendulum generally learns faster, we took a fewer number of steps.)
We see that, out of all the representation learning methods, learned representations of SymReg mostadequately capture the structure of the environment in Inverted Pendulum since the RL agent justtrained on the fixed representation (SymReg-decoupled) outperforms all of them, including vanilla PPO.
In Reacher, SymReg, along with other non-generative models, performs poorly compared tothe AE.
We believe that this is because the representation is focused on transformations caused bythe agent’s actions while details that can be valuable from the reward’s perspective — in this case,the small object that the Reacher should reach - are ignored.
This observation points to a limitationof all non-generative approaches that fine-tuning can resolve.
To further verify this, we combined SymReg with a Decoder (Dec-SymReg) and noticed a significant improvement in the performanceof the decoupled variation.
In Swimmer, again we see that learning the agent’s transformations isnot enough to get all the reward information as the background movement decides how far the agenthas swum.
Indeed, allowing the encoder to fine-tune allows the representations to reflect the rewardinformation and improve performance.
Conclusion
We proposed to learn equivariant representations by enforcing geometric invariants associated withthe given group.
We demonstrate this to be a simple, intuitive, and yet effective approach forrepresentation learning.
However, our approach also has several limitations that we hope to address infuture work.
Most importantly, a recipe for creating SymReg losses for general Lie groups is currentlymissing.
This is because we rely on known invariants for important Lie groups or geometries.
In thefuture, we would like to understand data characteristics that motivate the choice of one Lie group overothers.
Practical use of SymReg for finite groups is also a topic we would like to explore.
We wouldalso like to investigate further the best choice of objectives or mechanism for preventing collapsewhen enforcing geometric invariants.