Original: Extended Abstract Track
Revised: Extended Abstract Track

Original: Equivariant Representations for Non-Free Group Actions
Revised: Equivariant Representations for Non-Free Group Actions

Original: Editors: List of editors’ names
Revised: Editors: Sophia Sanborn, Christian Shewmake, Simone Azeglio, Arianna Di Bernardo, Nina Miolane

Original: Abstract
Revised: Abstract

Original: We introduce a method for learning representations that are equivariant with respect to general group actions over data.
Revised: We introduce a method for learning representations that are equivariant with respect to general group actions over data.

Original: Differently from existing equivariant representation learners, our method is suitable for actions that are not free
Revised: Differently from existing equivariant representation learners, our method is suitable for actions that are not free

Original: i e., that stabilize data via nontrivial symmetries.
Revised: i e., that stabilize data via nontrivial symmetries.

Original: Our method is grounded in the orbit-stabilizer theorem from group theory, which guarantees that an ideal learner infers an isomorphic representation.
Revised: Our method is grounded in the orbit-stabilizer theorem from group theory, which guarantees that an ideal learner infers an isomorphic representation.

Original: Finally, we provide an empirical investigation on image datasets with rotational symmetries and show that taking stabilizers into account improves the quality of the representations.
Revised: Finally, we provide an empirical investigation on image datasets with rotational symmetries and show that taking stabilizers into account improves the quality of the representations.

Original: 1. Introduction
Revised: 1. Introduction

Original: The problem of incorporating symmetries into representations defines a fundamental challenge and has been considered in a number of recent works (Quessard et al. ; Higgins et al. ; Cohen and Welling (2014); Tonnaer et al. (2022); Ahuja et al. (2021)).
Revised: The problem of incorporating symmetries into representations defines a fundamental challenge and has been considered in a number of recent works (Quessard et al., 2020; Higgins et al., 2022; Cohen and Welling, 2014; Tonnaer et al., 2022; Ahuja et al., 2021).

Original: The overall aim is to design representations which preserve symmetries – a property known as equivariance .
Revised: The overall aim is to design representations which preserve symmetries – a property known as equivariance .

Original: This is because preservation of symmetries leads to the extraction of geometric and semantic structure in data, which can be exploited for reasoning, efficiency and generalization (Bengio et al.
Revised: This is because preservation of symmetries leads to the extraction of geometric and semantic structure in data, which can be exploited for reasoning, efficiency and generalization (Bengio et al., 2013).

Original: As an example, the challenge of disentangling semantic factors of variations has been rephrased in terms of equivariant representations (Higgins et al. ; Caselles-Dupr´e et al.
Revised: As an example, the challenge of disentangling semantic factors of variations has been rephrased in terms of equivariant representations (Higgins et al., 2018; Caselles-Dupr´e et al., 2019).

Original: The majority of literature relies on the assumption that the group of symmetries acts freely on data (Marchetti et al. i.e., that no datapoint is stabilized by (nontrivial) symmetries.
Revised: The majority of literature relies on the assumption that the group of symmetries acts freely on data (Marchetti et al., 2022) i.e., that no datapoint is stabilized by (nontrivial) symmetries.

Original: This avoids the need to model stabilizers, which are unknown subgroups of the symmetry group considered.
Revised: This avoids the need to model stabilizers, which are unknown subgroups of the symmetry group considered.

Original: However, non-free group actions arise in several practical scenarios.
Revised: However, non-free group actions arise in several practical scenarios.

Original: This happens for example when considering images of  objects acted upon by the rotation group via change of orientation.
Revised: This happens for example when considering images of Extended Abstract Track objects acted upon by the rotation group via change of orientation.

Original: Such objects might be symmetrical, resulting in rotations leaving the image (almost) identical and consequently ambiguous in its orientation (see Figure 1).
Revised: Such objects might be symmetrical, resulting in rotations leaving the image (almost) identical and consequently ambiguous in its orientation (see Figure 1).

Original: In this work we propose a method for learning equivariant representation for general and potentially non-free group actions.
Revised: In this work we propose a method for learning equivariant representation for general and potentially non-free group actions.

Original: Based on the orbit-stabilizer theorem from group theory, we design a model that outputs cosets of the stabilizer subgroup.
Revised: Based on the orbit-stabilizer theorem from group theory, we design a model that outputs cosets of the stabilizer subgroup.

Original: The representation learner optimizes an equivariance loss based on supervision from symmetries alone.
Revised: The representation learner optimizes an equivariance loss based on supervision from symmetries alone.

Original: The above-mentioned theoretical results guarantee that a learner infers representations that are isomorphic to the original dataset.
Revised: The above-mentioned theoretical results guarantee that a learner infers representations that are isomorphic to the original dataset.

Original: Extended Abstract Track

Original: 2. Group Theory Background
Revised: 2. Group Theory Background

Original: Let G be the group of symmetries with multiplication denoted by ( g, h ) → gh and identity denoted by 1 ∈ G .
Revised: Let G be the group of symmetries with multiplication denoted by ( g, h ) → gh and identity denoted by 1 ∈ G .

Original: Suppose that G acts on a set X via ( g, x ) → g · x .
Revised: Suppose that G acts on a set X via ( g, x ) → g · x .

Original: The action defines a set of orbits X /G given by the equivalence classes of the relation x ∼ y
Revised: The action defines a set of orbits X /G given by the equivalence classes of the relation x ∼ y

Original: iff y = g · x for some g ∈ G .
Revised: iff y = g · x for some g ∈ G .

Original: For each x ∈ X , the stabilizer subgroup is defined as G x = { g ∈ G | g · x = x } .
Revised: For each x ∈ X , the stabilizer subgroup is defined as G x = { g ∈ G | g · x = x } .

Original: Stabilizers are conjugate as x varies in its orbit, and by abuse of notation we refer to the conjugacy class G O for O ∈ X /G .
Revised: Stabilizers are conjugate as x varies in its orbit, and by abuse of notation we refer to the conjugacy class G O for O ∈ X /G .

Original: The action is said to be free if G O = { 1 } for every O .
Revised: The action is said to be free if G O = { 1 } for every O .

Original: Recall that a map φ : X → Z between sets acted upon by G is said to be equivariant if φ ( g · x )
Revised: Recall that a map φ : X → Z between sets acted upon by G is said to be equivariant if φ ( g · x )

Original: = g · φ ( x ) for every x ∈ X and g ∈ G .
Revised: = g · φ ( x ) for every x ∈ X and g ∈ G .

Original: An equivariant bijection is referred to as isomorphism .
Revised: An equivariant bijection is referred to as isomorphism .

Original: The following is the fundamental result on group actions (Rotman (2012)).
Revised: The following is the fundamental result on group actions (Rotman, 2012).

Original: Theorem 1 (Orbit-Stabilizer)
Revised: Theorem 1 (Orbit-Stabilizer)

Original: Each orbit O is isomorphic to the set of (left) cosets
Revised: Each orbit O is isomorphic to the set of (left) cosets

Original: G/G O = { gG O | g ∈ G } (on which G acts by left multiplication).
Revised: G/G O = { gG O | g ∈ G } .

Original: In other words, there is an isomorphism:
Revised: In other words, there is an isomorphism:

Original: [Equation]
Revised: [Equation]

Original: (cid:97)

Original: [Equation]

Original: where 2 G denotes the power-set of G on which G acts by left multiplication g · A = { ga |
Revised: where 2 G denotes the power-set of G on which G acts by left multiplication g · A = { ga |

Original: a ∈
Revised: a ∈

Original: A } .
Revised: A } .

Original: Moreover, any equivariant map φ : X → (cid:96) O ∈X /G
Revised: Moreover, any equivariant map φ : X → (cid:96) O ∈X /G

Original: G/G O which induces a bijection on orbits is an isomorphism.
Revised: G/G O which induces a bijection on orbits is an isomorphism.

Original: 3. Equivariant Representation Learning
Revised: 3. Equivariant Representation Learning

Original: Our goal is to design an equivariant representation learner based on Theorem 1.
Revised: Our goal is to design an equivariant representation learner based on Theorem 1.

Original: We aim to train a model φ : X → Z with a latent space Z on a loss encouraging equivariance.
Revised: We aim to train a model φ : X → Z with a latent space Z on a loss encouraging equivariance.

Original: We assume that G is known a priori but its action on X is not.
Revised: While we assume that G is known a priori, its action on X is not and has to be conveyed through data.

Original: Theoretically the ideal choice for Z is given by  the right-hand side of the isomorphism in Equation 1.
Revised: The ideal choice for Z is given by (cid:96) O ∈X /G G/G O since the latter is the isomorphic to X (Theorem 1).

Original: In other words φ has to output cosets of stabilizers.
Revised: In other words, φ ideally outputs cosets of stabilizers of the inputs.

Original: However, thecore challenge is that the stabilizers are unknown a priori since they depend on the group action.
Revised: However, the stabilizers are unknown a priori since they depend on the group action.

Original: In order to circumvent the modeling of stabilizers and their cosets, we appeal to the following simple result (a proof is provided in the Appendix):
Revised: In order to circumvent the modeling of stabilizers and their cosets, we appeal to the following simple result (a proof is provided in the Appendix):

Original: Proposition 2 Let φ : X → 2 G be an equivariant map.
Revised: Proposition 2 Let φ : X → 2 G be an equivariant map.

Original: Then for each x ∈ X from an orbit O , φ ( x ) contains a coset of (a conjugate of) G O .
Revised: Then for each x ∈ X from an orbit O , φ ( x ) contains a coset of (a conjugate of) G O .

Revised: Extended Abstract Track

Original: Proposition 2 enables φ to output subsets of G instead of cosets of stabilizers.
Revised: Proposition 2 enables φ to output subsets of G instead of cosets of stabilizers.

Original: As long as those subsets are minimal w.r.t.
Revised: As long as those subsets are minimal w.r.t.

Original: to inclusion, they will coincide with the desired cosets.
Revised: to inclusion, they will coincide with the desired cosets.

Original: Based on this, we define the latent space as Z =
Revised: Based on this, we define the latent space as Z =

Original: Z G × Z O and implement the map φ as a pair of neural networks φ G : X → Z G , φ O : X → Z O .
Revised: Z G × Z O and implement the map φ as a pair of neural networks φ G : X → Z G , φ O : X → Z O .

Original: The component Z G represents cosets of stabilizers while Z O represents orbits.
Revised: The component Z G represents cosets of stabilizers while Z O represents orbits.

Original: Since the output space of a neural network is a finite-dimensional vector space, we assume that G is a linear Lie group ,i.e. G is a manifold of matrices, and that the stabilizers of the action are finite.
Revised: Since the output space of a neural network is a finite-dimensional vector space, we assume that G is a linear Lie group, i.e. G is a manifold of matrices, and that the stabilizers of the action are finite.

Original: The model φ G first outputs N elements ( φ 1 G ( x ) , · · · , φ NG ( x )) =
Revised: The model φ G first outputs N elements ( φ 1 G ( x ) , · · · , φ NG ( x )) =

Original: φ G ( x ) in the matrix Lie algebra g that are converted to G by the exponential map exp : g → G .
Revised: φ G ( x ) in the matrix Lie algebra g that are converted to G by the exponential map exp : g → G .

Original: The hyper-parameter N ideally should be chosen to be larger than the cardinality of the stabilizers.
Revised: The hyper-parameter N ideally should be chosen to be larger than the cardinality of the stabilizers.

Original: On the other hand, the output of φ O consists of a vector of arbitrary dimensionality.
Revised: On the other hand, the output of φ O consists of a vector of arbitrary dimensionality.

Original: The only requirement is that the output space of φ O should have enough capacity to contain X /G .
Revised: The only requirement is that the output space of φ O should have enough capacity to contain X /G .

Original: Extended Abstract Track

Original: Our dataset D consists of samples from the  group action, meaning that datapoints are triplets ( x, g, y ) ∈
Revised: Our dataset D consists of samples from the (unknown) group action, meaning that datapoints are triplets ( x, g, y ) ∈

Original: X × G × X with y = g · x . Given a datapoint ( x, g, y ) ∈
Revised: X × G × X with y = g · x . Given a datapoint ( x, g, y ) ∈

Original: D the learner φ G optimizes the equivariance loss over its parameters:
Revised: D the learner φ G optimizes the equivariance loss over its parameters:

Original: [Equation]
Revised: [Equation]

Original: where d is a (semi) metric for sets.
Revised: where d is a (semi) metric for sets.

Original: We opt for the (asymmetric) Chamfer distance d ( A, B )
Revised: We opt for the (asymmetric) Chamfer distance d ( A, B )

Original: = 1 | A | (cid:80) a ∈
Revised: = 1 | A | (cid:80) a ∈

Original: A min b ∈ B d G ( a, b ) because of its differentiability properties.
Revised: A min b ∈ B d G ( a, b ) because of its differentiability properties.

Original: Here d G is a metric on
Revised: Here d G is a metric on

Original: G and is typically set as the (squared) Euclidean one for G = R n and as the (squared)
Revised: G and is typically set as the (squared) Euclidean one for G = R n and as the (squared)

Original: Frobenius one for G = SO( n ).
Revised: Frobenius one for G = SO( n ).

Original: As previously discussed, we wish φ G ( x ) , when seen as a set, to be minimal in cardinality.
Revised: As previously discussed we wish φ G ( x ), when seen as a set, to be minimal in cardinality.

Original: To this end we add the following regularization term which measures the discreteentropy discouraging discrete entropy:
Revised: To this end we add the following regularization term measuring the discrete entropy:

Original: [Equation]
Revised: [Equation]

Original: where λ is a small weight set canonically to 0 .
Revised: where λ is a small weight set canonically to 0 .

Original: On the other hand, since orbits are invariant to the group action φ
Revised: On the other hand, since orbits are invariant to the group action φ

Original: O optimizes a contrastive loss .
Revised: O optimizes a contrastive loss .

Original: We opt for the popular
Revised: We opt for the popular

Original: InfoNCE loss from the literature (Chen et al. i.e.,
Revised: InfoNCE loss from the literature (Chen et al., 2020):

Original: [Equation]
Revised: [Equation]

Original: where x ′ is marginalized from D .
Revised: where x ′ is marginalized from D .

Original: As customary for the InfoNCE loss, we normalize the output of φ O and set d O ( a, b ) = − cos( ∠ ab ) = − a · b .
Revised: As customary for the InfoNCE loss, we normalize the output of φ O and set d O ( a, b ) = − cos( ∠ ab ) = − a · b .

Original: The denominator of Equationencourages injectivity of φ O as required by the second part of Theorem 1.
Revised: The second summand of L O encourages injectivity of φ O as and prevents orbits from collapsing in the representation. The Orbit-Stabilizer Theorem guarantees that an ideal learner achieves isomorphic representations in the following sense. If the L G ( x, g, y ) and the first summand of L O ( x, y ) vanish for every ( x, g, y ) then φ is equivariant. If moreover the regularizations ( (cid:101) L G and the second summand of L O ) are at a minimum then φ G ( x ) coincides with a coset of G O for every x ∈ O

Original: This together with Proposition 2 guarantees that an ideal learner infers an isomorphic representation.
Revised: (Proposition 2) and φ O is injective. Theorem 1 implies then that the representation is isomorphic (on its image) as desired.

Original: 4. Experiments
Revised: 4. Experiments

Original: We test an implementation of the neural networks φ G and φ O on the following two datasets consisting ofimages with a resolution of 64 × 64 pixels subject to non-free group actions:
Revised: We test an implementation of the neural networks φ G and φ O on the following four datasets consisting of 64 × 64 images subject to non-free group actions:

Original:  Rotating Arrows: each image depicts a radial configuration of ν ∈ { 4 , 5 , 6 } arrows pointing outside. The group of two-dimensional rotations G = SO(2) acts by rotating the configuration.
Revised: Extended Abstract Track • Rotating Arrows: images of radial configurations of ν ∈ { 4 , 5 , 6 } arrows rotated by G = SO(2).

Original: The number of arrows ν determines the orbit and the stabilizer. The latter is the cyclic group C ν ⊆ G of order ν .
Revised: The number of arrows ν determines the orbit with stabilizer the cyclic group C ν ⊆ G of order ν .

Original:  Double Arrows: each image consists of two radial configurations of arrows withand 3 arrows respectively placed in two corners of the image. The group of symmetries is a torus G = SO(2) × SO(2).
Revised: • Double Arrows: images of two radial configurations of 2 and 3 arrows respectively rotated by G = SO(2) × SO(2).

Original: The action produces a single orbit, i.e. it is transitive , and the stabilizer is a product of cyclic groups C 2 × C 3 .
Revised: The action produces a single orbit, i.e. it is transitive , and the stabilizer is a product of cyclic groups C 2 × C 3 .

Original: Tetrahedron: this dataset consists of  images of a monochromatic tetrahedronin different orientations (Murphy et al. The group of symmetries is G = SO(3) with a single orbit andthe stabilizeris the alternating group
Revised: • Chair: images of a monochromatic chair from ModelNet40 (Wu et al., 2015) rotated by G = SO(2) along a vertical axis with a single orbit with stabilizer the cyclic group C 4 ⊆ G of order 4. • Tetrahedron: images of a monochromatic tetrahedron (Murphy et al., 2021) rotated by G = SO(3) with a single orbit and stabilizer the alternating group

Original: A 4 of order 12.
Revised: A 4 of order 12.

Original: We compare our model with the baseline in which the model φ G produces a single output i.e., N = 1 and φ G ( x ) = φ 1 G ( x ) ∈ G .
Revised: We compare our model with the baseline where φ G produces a single output i.e., N = 1.

Original: In other words, the latent space is  Z = 
Revised: The latent space is thus Z = G

Original: G ×
Revised: ×

Original: Z O , on which G acts freely.
Revised: Z O , on which G acts freely.

Original: Thisbaseline is similar to what has been proposed in previous work assuming free group actions (Quessard et al., 2020; Caselles-Dupr´e et al., 2019; Marchetti et al., 2022; Tonnaer et al., 2022).
Revised: This is similar to what has been proposed in previous work (Caselles-Dupr´e et al., 2019; Marchetti et al., 2022; Tonnaer et al., 2022).

Original: Extended Abstract Track

Original: The models are compared based on two evaluation metrics.
Revised: The models are compared based on two evaluation metrics.

Original: The first is the equivariance loss, see Equation 3, on a test set.
Revised: First, the equivariance loss (Equation 3) on a test set.

Original: The second is the reconstruction loss  on a test set of a decoder ψ : Z → X trained jointly with φ .The loss is given by E
Revised: Second, the reconstruction loss (pixel-wise cross-entropy) on a test set of a decoder ψ : Z → X trained jointly with φ .

Original: x [ d bc ( x, ψ ( φ ( x )) ] where d bc is the pixel-wise binary cross-entropy.

Original: Quantitative results are presented in Table 1 while qualitative visualizations are shown in Figure 2.
Revised: Quantitative results are presented in Table 1 while qualitative visualizations are shown in Figure 2.

Revised: As can be seen, our model correctly infers the stabilizers (the cyclic subgroups of SO(2) and the alternating subgroup of SO(3)) by overlapping components of φ G ( x ) and distributing them geometrically.

Original: As can be seen, our model achievesequivariance and reconstruction errors that are significantly smaller than the baseline.
Revised: Moreover, our model achieves significantly lower scores than the baseline.

Original: The latter is not able to capture the stabilizers in its latent space, leading to representations of poor quality and loss of information.
Revised: The latter is not able to capture the stabilizers in its latent space, leading to representations of poor quality and loss of information.

Original: [Table]
Revised: [Table]

Original: [Figure]

Original: In the double arrows, the stabilizer C 2 × C 3 ⊆

Original: SO(2) × SO(2) is visualized on a torus seen as an identified a square.

Original: For the tetrahedron, the stabilizer A 4 is visualized as rotational vectors in the projective space RP 3 ≃ SO(3).

Original: 5. Conclusion and Future Work
Revised: 5. Conclusions and Future Work

Original: In this work we introduced a method for learning equivariant representations for general and potentially non-free group actions.
Revised: In this work we introduced a method for learning equivariant representations for general and potentially non-free group actions.

Original: We discussed the theoretical foundations and empirically investigated the method on images with rotational symmetries.
Revised: We discussed the theoretical foundations and empirically investigated the method on images with rotational symmetries [Table] [Figure]

Original: Our model relies on the assumptions that the stabilizers are finite.
Revised: Our model relies on the assumptions that the stabilizers are finite.

Original: However, nondiscrete stabilizer subgroups sometimes occur, for example in the case of symmetrical objects such as a cone or a cylinder.
Revised: However, non-discrete stabilizer subgroups sometimes occur, for example in the case of symmetrical objects such as a cone or a cylinder.

Original: An interesting future direction is designing an equivariant representation learner suitable for group actions with non-discrete stabilizers.
Revised: An interesting future direction is designing an equivariant representation learner suitable for group actions with non-discrete stabilizers and the evaluation of our method on more complex datasets. 6. Acknowledgements

Revised: The second and third named authors thank the Swedish Research Council, the Knut and

Revised: Alice Wallenberg Foundation and the European Research Council (ERC-BIRD-884807) for their support.

Revised: This work has also received funding from the NWO-TTW Programme “Efficient Deep Learning” (EDL) P16-25.

Original: Extended Abstract Track
Revised: Extended Abstract Track

Original:  Appendix 6.1. Proofs of Theoretical Results
Revised: 7. Appendix 7.1. Proofs of Theoretical Results

Original: Proposition 6.1: Let φ : X → 2 G be an equivariant map.
Revised: Proposition 3 Let φ : X → 2 G be an equivariant map.

Original: Then for each x ∈ X from an orbit O , φ ( x ) contains a coset of (a conjugate of) G O .
Revised: Then for each x ∈ X from an orbit O , φ ( x ) contains a coset of (a conjugate of) G O .

Original: Proof Pick x ∈ X .
Revised: Proof Pick x ∈ X .

Original: Then for every g ∈ G x it holds that φ ( x ) =
Revised: Then for every g ∈ G x it holds that φ ( x ) =

Original: φ ( g · x ) = g · φ ( x ).
Revised: φ ( g · x ) = g · φ ( x ).

Original: In other words G x h = hh − 1 G x h ⊆ φ ( x ) for each h ∈ φ ( x ).
Revised: In other words G x h = hh − 1 G x h ⊆ φ ( x ) for each h ∈ φ ( x ).

Original: Since h − 1 G x h is conjugate to G x the thesis follows.
Revised: Since h − 1 G x h is conjugate to G x the thesis follows.

Original: 6.2. Training Details
Revised: 7.2. Training Details

Original: We implement the neural networks φ G and φ O with a backbone ResNet18 (He et al.
Revised: We implement the neural networks φ G and φ O with a backbone ResNet18 (He et al., 2016).

Original: For a datapoint x ∈ X , the network implements multiple heads to produce embeddings (cid:0) φ 1 G ( x ) , · · · , φ NG ( x ) (cid:1) with φ iG ( x ) ∈ G .
Revised: For a datapoint x ∈ X , the network implements multiple heads to produce embeddings (cid:0) φ 1 G ( x ) , · · · , φ N G ( x ) (cid:1) with φ iG ( x ) ∈ G .

Original: The output dimension of φ O is set to 3.
Revised: The output dimension of φ O is set to 3.

Original: We train each experiment for 50 epochs  using the AdamW optimizer (Loshchilov and Hutter (2019)) with a learning rate of 10 − 4 and batches of 16 triplets ( x, g, y ) ∈ D .
Revised: We train the model for 50 epochs (except for the chair datasaet which requires a longer training ofepochs) using the AdamW optimizer (Loshchilov and Hutter, 2019) with a learning rate of 10 − 4 and batches of 16 triplets ( x, g, y ) ∈ D .

Original: The rotating arrows  dataset consists of 5000 datapoints per orbit while the double arrows and the tetrahedron datasets consist of 20000 datapoints.
Revised: The rotating arrows and chair dataset consists of 5000 datapoints per orbit while the double arrows and the tetrahedron datasets consist of 20000 datapoints.

Original: For all datasets the test set consists of a random 10% split.
Revised: For all datasets the test set consists of a random 10% split.

Original: 6.3. Reconstructions
Revised: 7.3. Reconstructions

Original: We present in Figure 3 some examples of images reconstructed by a decoder ψ trained jointly with the encoder φ .
Revised: We present in Figure 3 some examples of images reconstructed by a decoder ψ trained jointly with the encoder φ .

Original: The baseline model is not capable of clearly reconstructing the images compared to our model.
Revised: The baseline model is not capable of clearly reconstructing the images compared to our model.

Original: As seen from the quantitative results in Table 1, the baseline model is not capable of optimizing the equivariance loss from Equation 2.
Revised: As seen from the quantitative results in Table 1, the baseline model is not capable of optimizing the equivariance loss from Equation 2.

Original: This provides a hint that the encoder φ might not be converging to a stable representation.
Revised: This provides a hint that the encoder φ might not be converging to a stable representation.

Original: Consequently, the decoder is incapable of consistently reconstructing the data which results in a higher reconstruction loss.
Revised: Consequently, the decoder is incapable of consistently reconstructing the data which results in a higher reconstruction loss.

Original: [Figure]
Revised: [Figure]