Original: E XPLICIT HOMOGRAPHY ESTIMATION IMPROVES CONTRASTIVE SELF - SUPERVISED LEARNING
Revised: E XPLICIT HOMOGRAPHY ESTIMATION IMPROVES CONTRASTIVE SELF - SUPERVISED LEARNING

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: The typical contrastive self-supervised algorithm uses a similarity measure in latent space as the supervision signal by contrasting positive and negative images directly or indirectly.
Revised: The typical contrastive self-supervised algorithm uses a similarity measure in latent space as the supervision signal by contrasting positive and negative images directly or indirectly.

Original: Although the utility of self-supervised algorithms has improved recently, there are still bottlenecks hindering their widespread use, such as the compute needed.
Revised: Although the utility of self-supervised algorithms has improved recently, there are still bottlenecks hindering their widespread use, such as the compute needed.

Original: In this paper, we propose a module that serves as an additional objective in the self-supervised contrastive learning paradigm.
Revised: In this paper, we propose a module that serves as an additional objective in the self-supervised contrastive learning paradigm.

Original: We show how the inclusion of this module to regress the parameters of an afﬁne transformation or homography, in addition to the original contrastive objective, improves both performance and learning speed.
Revised: We show how the inclusion of this module to regress the parameters of an afﬁne transformation or homography, in addition to the original contrastive objective, improves both performance and learning speed.

Original: Importantly, we ensure that this module does not enforce invariance to the various components of the afﬁne transform, as this is not always ideal.
Revised: Importantly, we ensure that this module does not enforce invariance to the various components of the afﬁne transform, as this is not always ideal.

Original: We demonstrate the effectiveness of the additional objective on two recent, popular self-supervised algorithms.
Revised: We demonstrate the effectiveness of the additional objective on two recent, popular self-supervised algorithms.

Original: We perform an extensive experimental analysis of the proposed method and show an improvement in performance for all considered datasets.
Revised: We perform an extensive experimental analysis of the proposed method and show an improvement in performance for all considered datasets.

Original: Further, we ﬁnd that although both the general homography and afﬁne transformation are sufﬁcient to improve performance and convergence, the afﬁne transformation performs better in all cases.
Revised: Further, we ﬁnd that although both the general homography and afﬁne transformation are sufﬁcient to improve performance and convergence, the afﬁne transformation performs better in all cases.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: There is an ever-increasing pool of data, particularly unstructured data such as images, text, video, and audio.
Revised: There is an ever-increasing pool of data, particularly unstructured data such as images, text, video, and audio.

Original: The vast majority of this data is unlabelled.
Revised: The vast majority of this data is unlabelled.

Original: The process of labelling is time-consuming, labour-intensive, and expensive.
Revised: The process of labelling is time-consuming, labour-intensive, and expensive.

Original: Such an environment makes algorithms that can leverage fully unlabelled data particularly useful and important.
Revised: Such an environment makes algorithms that can leverage fully unlabelled data particularly useful and important.

Original: Such algorithms fall within the realm of unsupervised learning.
Revised: Such algorithms fall within the realm of unsupervised learning.

Original: A particular subset of unsupervised learning is known as Self-Supervised Learning (SSL).
Revised: A particular subset of unsupervised learning is known as Self-Supervised Learning (SSL).

Original: SSL is a paradigm in which the data itself provides a supervision signal to the algorithm.
Revised: SSL is a paradigm in which the data itself provides a supervision signal to the algorithm.

Original: Somewhat related is another core area of research known as transfer learning (Wang et al., 2020).
Revised: Somewhat related is another core area of research known as transfer learning (Wang et al., 2020).

Original: In the context of computer vision, this means being able to pre-train an encoder network ofﬂine on a large, varietal dataset, followed by domain-speciﬁc ﬁne-tuning on the bespoke task at hand.
Revised: In the context of computer vision, this means being able to pre-train an encoder network ofﬂine on a large, varietal dataset, followed by domain-speciﬁc ﬁne-tuning on the bespoke task at hand.

Original: The state-of-the-art for many transfer learning applications remains dominated by supervised learning techniques (Tan et al., 2020;
Revised: The state-of-the-art for many transfer learning applications remains dominated by supervised learning techniques (Tan et al., 2020;

Original: Martinez et al., 2019; Donahue et al., 2014; Girshick et al., 2014), in which models are pre-trained on a large labelled dataset.
Revised: Martinez et al., 2019; Donahue et al., 2014; Girshick et al., 2014), in which models are pre-trained on a large labelled dataset.

Original: However, self-supervised learning techniques have more recently come to the fore as potential alternatives that perform similarly on downstream tasks, while requiring no labelled data.
Revised: However, self-supervised learning techniques have more recently come to the fore as potential alternatives that perform similarly on downstream tasks, while requiring no labelled data.

Original: Most selfsupervised techniques create a supervision signal from the data itself in one of two ways.
Revised: Most selfsupervised techniques create a supervision signal from the data itself in one of two ways.

Original: The one approach are techniques that deﬁne a pre-text task beforehand that a neural network is trained to solve, such as inpainting (Pathak et al., 2016) or a jigsaw puzzle (Noroozi & Favaro, 2016).
Revised: The one approach are techniques that deﬁne a pre-text task beforehand that a neural network is trained to solve, such as inpainting (Pathak et al., 2016) or a jigsaw puzzle (Noroozi & Favaro, 2016).

Original: In this way, the pre-text task is a kind of proxy that, if solved, should produce reasonable representations for downstream visual tasks such as image or video recognition, object detection, or semantic segmentation.
Revised: In this way, the pre-text task is a kind of proxy that, if solved, should produce reasonable representations for downstream visual tasks such as image or video recognition, object detection, or semantic segmentation.

Original: The other approach is a class of techniques known as contrastive methods (Chen et al., 2020a;
Revised: The other approach is a class of techniques known as contrastive methods (Chen et al., 2020a;

Original: He et al., 2019; Chen et al., 2020b).
Revised: He et al., 2019; Chen et al., 2020b).

Original: These methods minimise the distance (or maximise the similarity) between the latent representations of two augmented views of the same input image, while simultaneously maximising the distance between negative pairs.
Revised: These methods minimise the distance (or maximise the similarity) between the latent representations of two augmented views of the same input image, while simultaneously maximising the distance between negative pairs.

Original: In this way, these methods enforce consistency regularisation (Sohn et al., 2020), a well-known approach to semi-supervised learning.
Revised: In this way, these methods enforce consistency regularisation (Sohn et al., 2020), a well-known approach to semi-supervised learning.

Original: These contrastive methods often outperform the pre-text task methods and are the current state-of-the-art in self-supervised learning.
Revised: These contrastive methods often outperform the pre-text task methods and are the current state-of-the-art in self-supervised learning.

Original: However, most of these contrastive methods have several drawbacks, such as requiring prohibitively large batch sizes or memory banks, in order to retrieve the negative pairs of samples (Chen et al., 2020a;
Revised: However, most of these contrastive methods have several drawbacks, such as requiring prohibitively large batch sizes or memory banks, in order to retrieve the negative pairs of samples (Chen et al., 2020a;

Original: He et al., 2019).
Revised: He et al., 2019).

Original: The intuition behind our proposed module is that any system tasked with understanding images can beneﬁt from understanding the geometry of the image and the objects within it.
Revised: The intuition behind our proposed module is that any system tasked with understanding images can beneﬁt from understanding the geometry of the image and the objects within it.

Original: An afﬁne transformation is a geometric transformation that preserves parallelism of lines.
Revised: An afﬁne transformation is a geometric transformation that preserves parallelism of lines.

Original: It can be composed of any sequence of rotation, translation, shearing, and scaling.
Revised: It can be composed of any sequence of rotation, translation, shearing, and scaling.

Original: A homography is a generalisation of this notion to include perspective warping.
Revised: A homography is a generalisation of this notion to include perspective warping.

Original: A homography need not preserve parallelism of lines, however, it ensures lines remain straight.
Revised: A homography need not preserve parallelism of lines, however, it ensures lines remain straight.

Original: Mathematically, a homography is shown in Equation 1.
Revised: Mathematically, a homography is shown in Equation 1.

Original: It has 8 degrees of freedom and is applied to a vector in homogenous coordinates.
Revised: It has 8 degrees of freedom and is applied to a vector in homogenous coordinates.

Original: An afﬁne transformation has the same form, but with the added constraint that φ 3 , 1 = φ 3 , 2 = 0 .
Revised: An afﬁne transformation has the same form, but with the added constraint that φ 3 , 1 = φ 3 , 2 = 0 .

Original: [Equation]
Revised: [Equation]

Original: The ability to know how a source image was transformed to get to a target image implicitly means that you have learned something about the geometry of that image.
Revised: The ability to know how a source image was transformed to get to a target image implicitly means that you have learned something about the geometry of that image.

Original: An afﬁne transformation or, more generally, a homography is a natural way to encode this idea.
Revised: An afﬁne transformation or, more generally, a homography is a natural way to encode this idea.

Original: Forcing the network to estimate the parameters of a random homography applied to the source images thereby forces it to learn semantics about the geometry.
Revised: Forcing the network to estimate the parameters of a random homography applied to the source images thereby forces it to learn semantics about the geometry.

Original: This geometric information can supplement the signal provided by a contrastive loss, or loss in the latent space.
Revised: This geometric information can supplement the signal provided by a contrastive loss, or loss in the latent space.

Original: In this paper, we propose an additional module that can be used in tandem with contrastive selfsupervised learning techniques to augment the contrastive objective (the additional module is highlighted in Figure 1).
Revised: In this paper, we propose an additional module that can be used in tandem with contrastive selfsupervised learning techniques to augment the contrastive objective (the additional module is highlighted in Figure 1).

Original: The module is simple, model-agnostic, and can be used to supplement a contrastive algorithm to improve performance and supplement the information learned by the network to converge faster.
Revised: The module is simple, model-agnostic, and can be used to supplement a contrastive algorithm to improve performance and supplement the information learned by the network to converge faster.

Original: The module is essentially an additional stream of the network with the objective of regressing the parameters of an afﬁne transformation or homography.
Revised: The module is essentially an additional stream of the network with the objective of regressing the parameters of an afﬁne transformation or homography.

Original: In this way, there is a multi-task objective that the network must solve: 1.
Revised: In this way, there is a multi-task objective that the network must solve: 1.

Original: minimising the original contrastive objective, and 2.
Revised: minimising the original contrastive objective, and 2.

Original: learning the parameters of a homography applied to one of the input images from a vector difference of their latent representations.
Revised: learning the parameters of a homography applied to one of the input images from a vector difference of their latent representations.

Original: We force the latent space to encode the geometric transformation information by learning to regress the parameters of the transformation in an MLP that takes the vector difference of two latent representations of an input, x , and its transformed analogue, x (cid:48) .
Revised: We force the latent space to encode the geometric transformation information by learning to regress the parameters of the transformation in an MLP that takes the vector difference of two latent representations of an input, x , and its transformed analogue, x (cid:48) .

Original: By including the information in this way, the network is not invariant to the components of the transformation but is still able to use them as a self-supervised signal for learning.
Revised: By including the information in this way, the network is not invariant to the components of the transformation but is still able to use them as a self-supervised signal for learning.

Original: Moreover, this approach serves as a novel hybrid of the pre-text tasks and contrastive learning by enforcing consistency regularisation (Sohn et al., 2020).
Revised: Moreover, this approach serves as a novel hybrid of the pre-text tasks and contrastive learning by enforcing consistency regularisation (Sohn et al., 2020).

Original: Through extensive empirical studies, we show that the additional objective of regressing the transformation parameters serves as a useful supplementary task for self-supervised contrastive learning, and improves performance for all considered datasets in terms of linear evaluation accuracy and convergence speed.
Revised: Through extensive empirical studies, we show that the additional objective of regressing the transformation parameters serves as a useful supplementary task for self-supervised contrastive learning, and improves performance for all considered datasets in terms of linear evaluation accuracy and convergence speed.

Original: The remainder of the paper is structured as follows.
Revised: The remainder of the paper is structured as follows.

Original: In Section 2, we cover the related work in the area of self-supervised learning, going into detail where necessary.
Revised: In Section 2, we cover the related work in the area of self-supervised learning, going into detail where necessary.

Original: In Section 3 we detail our proposed method.
Revised: In Section 3 we detail our proposed method.

Original: We ﬁrst introduce a framework and set of notation to make the formalisation of the approach clear.
Revised: We ﬁrst introduce a framework and set of notation to make the formalisation of the approach clear.

Original: We then delve into the details behind the architecture and choices for the various part of the system.
Revised: We then delve into the details behind the architecture and choices for the various part of the system.

Original: This is followed by a comprehensive set of experiments in Section 4, including results of various datasets, as well as an ablative study.
Revised: This is followed by a comprehensive set of experiments in Section 4, including results of various datasets, as well as an ablative study.

Original: Finally, the paper is concluded with some closing remarks in Section 5.
Revised: Finally, the paper is concluded with some closing remarks in Section 5.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: SSL is a popular research area within computer vision.
Revised: SSL is a popular research area within computer vision.

Original: Previous approaches can be broadly classed into two main categories.
Revised: Previous approaches can be broadly classed into two main categories.

Original: The ﬁrst is where pre-text tasks are manually deﬁned, and the goal of the algorithms is to solve these hand-crafted tasks (Lee et al., 2020; Doersch et al., 2015; Gidaris et al., 2018; Zhang et al., 2016; Misra & Maaten, 2020).
Revised: The ﬁrst is where pre-text tasks are manually deﬁned, and the goal of the algorithms is to solve these hand-crafted tasks (Lee et al., 2020; Doersch et al., 2015; Gidaris et al., 2018; Zhang et al., 2016; Misra & Maaten, 2020).

Original: Examples of such methods include inpainting (Pathak et al., 2016), colourising (Zhang et al., 2016), jigsaw puzzles (Noroozi & Favaro, 2016), patch prediction (Doersch et al., 2015), and geometric image transformations (Dosovitskiy et al., 2014) such as using rotation as the pre-text task (Gidaris et al., 2018;
Revised: Examples of such methods include inpainting (Pathak et al., 2016), colourising (Zhang et al., 2016), jigsaw puzzles (Noroozi & Favaro, 2016), patch prediction (Doersch et al., 2015), and geometric image transformations (Dosovitskiy et al., 2014) such as using rotation as the pre-text task (Gidaris et al., 2018;

Original: Feng et al., 2019).
Revised: Feng et al., 2019).

Original: Some of these pre-text approaches that deal with geometric image transformations are similar in spirit to our method.
Revised: Some of these pre-text approaches that deal with geometric image transformations are similar in spirit to our method.

Original: Gidaris et al.
Revised: Gidaris et al.

Original: ; Feng et al.
Revised: ; Feng et al.

Original: (2019) are two variants of predicting image rotations as an auxiliary task for unsupervised learning.
Revised: (2019) are two variants of predicting image rotations as an auxiliary task for unsupervised learning.

Original: Perhaps closer to our method is Dosovitskiy et al.
Revised: Perhaps closer to our method is Dosovitskiy et al.

Original: (2014), in which a set of transformations is applied to image patches, and the network is trained in a fully-unsupervised manner to predict surrogate classes deﬁned by a set of transformed image patches by minimising the log loss.
Revised: (2014), in which a set of transformations is applied to image patches, and the network is trained in a fully-unsupervised manner to predict surrogate classes deﬁned by a set of transformed image patches by minimising the log loss.

Original: Our method, however, investigates a different, particular set of transformations (those that deﬁne an afﬁne transformation of general homography), and show this can be used to aid self-supervised performance, using the transformation parameters themselves as targets that need to be regressed (using mean-squared error) by the contrastive algorithm in a multitask manner.
Revised: Our method, however, investigates a different, particular set of transformations (those that deﬁne an afﬁne transformation of general homography), and show this can be used to aid self-supervised performance, using the transformation parameters themselves as targets that need to be regressed (using mean-squared error) by the contrastive algorithm in a multitask manner.

Original: The discrepancy in the network’s ability to predict the actual values of the parameters of the afﬁne transformation/homography serves as our additional supervision signal.
Revised: The discrepancy in the network’s ability to predict the actual values of the parameters of the afﬁne transformation/homography serves as our additional supervision signal.

Original: A somewhat related approach to our proposed method within the pre-text task domain is proposed by
Revised: A somewhat related approach to our proposed method within the pre-text task domain is proposed by

Original: Lee et al.
Revised: Lee et al.

Original: They propose to augment the learning process of a supervised learning algorithm with additional labels constructed using self-supervised labels.
Revised: They propose to augment the learning process of a supervised learning algorithm with additional labels constructed using self-supervised labels.

Original: These labels are rotation classes and colour permutations.
Revised: These labels are rotation classes and colour permutations.

Original: Importantly, they create a loss function which is based on a joint distribution of the original (supervised) labels and the self-supervised (augmented) labels.
Revised: Importantly, they create a loss function which is based on a joint distribution of the original (supervised) labels and the self-supervised (augmented) labels.

Original: In this way, the network is not forced to be invariant to the transformations under consideration, since this has been shown to hurt performance (Lee et al., 2020).
Revised: In this way, the network is not forced to be invariant to the transformations under consideration, since this has been shown to hurt performance (Lee et al., 2020).

Original: Our method is different to this in that we propose a module to be integrated speciﬁcally with self-supervised algorithms.
Revised: Our method is different to this in that we propose a module to be integrated speciﬁcally with self-supervised algorithms.

Original: Additionally, we regress the transformation parameters in real vector space and do not create classes for the parameters.
Revised: Additionally, we regress the transformation parameters in real vector space and do not create classes for the parameters.

Original: The other broad category of SSL is based on contrastive learning (Chen et al., 2020a; He et al., 2019; Caron et al., 2020), and this class of techniques represent the current state-of-the-art in selfsupervised learning, outperforming the hand-crafted pre-text task methods.
Revised: The other broad category of SSL is based on contrastive learning (Chen et al., 2020a; He et al., 2019; Caron et al., 2020), and this class of techniques represent the current state-of-the-art in selfsupervised learning, outperforming the hand-crafted pre-text task methods.

Original: These approaches learn representations by contrasting positive pairs of samples from negative pairs of samples in latent space.
Revised: These approaches learn representations by contrasting positive pairs of samples from negative pairs of samples in latent space.

Original: Such methods typically require that careful attention be paid to the negative samples.
Revised: Such methods typically require that careful attention be paid to the negative samples.

Original: Additionally, they have the disadvantage of requiring prohibitively large batch sizes (4096-16000), memory banks, or other mechanisms to retrieve the relevant negative samples.
Revised: Additionally, they have the disadvantage of requiring prohibitively large batch sizes (4096-16000), memory banks, or other mechanisms to retrieve the relevant negative samples.

Original: One popular such method is known as SimCLR (Chen et al., 2020a).
Revised: One popular such method is known as SimCLR (Chen et al., 2020a).

Original: SimCLR is a general framework for contrastive learning, and in its vanilla formulation consists of an encoder network parameterised by a CNN (usually a variant of ResNet (He et al., 2016)) and an MLP projection head.
Revised: SimCLR is a general framework for contrastive learning, and in its vanilla formulation consists of an encoder network parameterised by a CNN (usually a variant of ResNet (He et al., 2016)) and an MLP projection head.

Original: An input image is sampled, and two distinct views of that same input image are computed using a random augmentation.
Revised: An input image is sampled, and two distinct views of that same input image are computed using a random augmentation.

Original: The augmentation consists of colour jiterring, Gaussian blurring, and random cropping.
Revised: The augmentation consists of colour jiterring, Gaussian blurring, and random cropping.

Original: The two views are sent through the encoder network to produce two latent representations.
Revised: The two views are sent through the encoder network to produce two latent representations.

Original: These latent vectors are then sent through the projection head to produce ﬁnal latent vectors.
Revised: These latent vectors are then sent through the projection head to produce ﬁnal latent vectors.

Original: It is from these vectors that the loss is computed.
Revised: It is from these vectors that the loss is computed.

Original: In the case of SimCLR, the loss is normalised temperatured cross-entropy (NT-Xent).
Revised: In the case of SimCLR, the loss is normalised temperatured cross-entropy (NT-Xent).

Original: A recent approach proposed in Grill et al.
Revised: A recent approach proposed in Grill et al.

Original: (2020) (BYOL) somewhat overcomes the aforementioned disadvantages of requiring negative pairs of samples (which implicitly requires a large batch size).
Revised: (2020) (BYOL) somewhat overcomes the aforementioned disadvantages of requiring negative pairs of samples (which implicitly requires a large batch size).

Original: Two separate networks with their own weights are used in tandem to learn the representation.
Revised: Two separate networks with their own weights are used in tandem to learn the representation.

Original: An online network (consisting of an encoder, MLP projection head, and MLP prediction network) is trained to predict the representation outputted by a target network.
Revised: An online network (consisting of an encoder, MLP projection head, and MLP prediction network) is trained to predict the representation outputted by a target network.

Original: During training, the online network parameters are updated using backpropagation of error derivatives computed using a meansquared error loss.
Revised: During training, the online network parameters are updated using backpropagation of error derivatives computed using a meansquared error loss.

Original: However, the target network parameters are updated using an exponential moving average.
Revised: However, the target network parameters are updated using an exponential moving average.

Original: In this way, BYOL overcomes collapsed solutions in which every image produces the same representation.
Revised: In this way, BYOL overcomes collapsed solutions in which every image produces the same representation.

Original: We test our module with both SimCLR and BYOL, since these two methods serve as two popular, recent approaches to contrastive SSL.
Revised: We test our module with both SimCLR and BYOL, since these two methods serve as two popular, recent approaches to contrastive SSL.

Original: Some helpful ﬁndings for guiding self-supervised research were demonstrated in Kolesnikov et al.
Revised: Some helpful ﬁndings for guiding self-supervised research were demonstrated in Kolesnikov et al.

Original: Core among these are that 1) standard architecture designs that work well in the fullysupervised setting do not necessarily work well in the self-supervised setting, 2) in the selfsupervised setting larger CNNs often means higher quality learned representations, and 3) the linear evaluation paradigm for assessing performance may take a long time to converge.
Revised: Core among these are that 1) standard architecture designs that work well in the fullysupervised setting do not necessarily work well in the self-supervised setting, 2) in the selfsupervised setting larger CNNs often means higher quality learned representations, and 3) the linear evaluation paradigm for assessing performance may take a long time to converge.

Original: Moreover, Newell & Deng (2020) ﬁnd that the effectiveness of self-supervised pretraining decreases as the amount of labelled data increases, and that performance on one particular downstream task is not necessarily indicative of performance on other downstream tasks.
Revised: Moreover, Newell & Deng (2020) ﬁnd that the effectiveness of self-supervised pretraining decreases as the amount of labelled data increases, and that performance on one particular downstream task is not necessarily indicative of performance on other downstream tasks.

Original:  P ROPOSED M ETHOD
Revised:  P ROPOSED M ETHOD

Original: We ﬁrst introduce a mathematical framework for discussing our method.
Revised: We ﬁrst introduce a mathematical framework for discussing our method.

Original: Let B 1 be a set of base transformations .
Revised: Let B 1 be a set of base transformations .

Original: A base transformation is a transformation that cannot be decomposed into more basic transformations and is interpreted as per Grill et al.
Revised: A base transformation is a transformation that cannot be decomposed into more basic transformations and is interpreted as per Grill et al.

Original: ; Chen et al.
Revised: ; Chen et al.

Original: (2020a).
Revised: (2020a).

Original: Examples of base transformations include colour jittering, cropping, and horizontal ﬂipping.
Revised: Examples of base transformations include colour jittering, cropping, and horizontal ﬂipping.

Original: We deﬁne the possible base transformations a-priori, and |B 1 | < ∞ .
Revised: We deﬁne the possible base transformations a-priori, and |B 1 | < ∞ .

Original: Next, we deﬁne a new set of base spatial transformations B 2 that correspond to the general afﬁne transformations (i.e. rotation, translation, scaling and shearing) or the full homography (i.e. afﬁne transformations and perspective projection).
Revised: Next, we deﬁne a new set of base spatial transformations B 2 that correspond to the general afﬁne transformations (i.e. rotation, translation, scaling and shearing) or the full homography (i.e. afﬁne transformations and perspective projection).

Original: Further, we impose the following condition:
Revised: Further, we impose the following condition:

Original: [Equation]
Revised: [Equation]

Original: The reason for this restriction will be apparent later.
Revised: The reason for this restriction will be apparent later.

Original: A transformation t b,θ is parameterised by its associated base transformation b ∈ B 1 ∪ B 2 and transformation parameters θ ∈ Θ .
Revised: A transformation t b,θ is parameterised by its associated base transformation b ∈ B 1 ∪ B 2 and transformation parameters θ ∈ Θ .

Original: Then, the set of all possible transformations for a particular base transformation set B may be deﬁned as:
Revised: Then, the set of all possible transformations for a particular base transformation set B may be deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: Clearly, we may have that |T i | = ∞ , since some parameters may take on any value within compact subsets of R .
Revised: Clearly, we may have that |T i | = ∞ , since some parameters may take on any value within compact subsets of R .

Original: This is important because we want to be able to sample from an inﬁnite sample space during training to ensure the network sees a variety of samples.
Revised: This is important because we want to be able to sample from an inﬁnite sample space during training to ensure the network sees a variety of samples.

Original: We can now deﬁne an augmentation , which is an ordered sequence of n transformations.
Revised: We can now deﬁne an augmentation , which is an ordered sequence of n transformations.

Original: As such, each unique ordering will necessarily produce a unique augmentation (e.g. ﬂipping and then cropping is different from cropping and then ﬂipping).
Revised: As such, each unique ordering will necessarily produce a unique augmentation (e.g. ﬂipping and then cropping is different from cropping and then ﬂipping).

Original: Formally, an augmentation a is deﬁned as:
Revised: Formally, an augmentation a is deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: Denote the set of all possible augmentations for a transformation set T i as A T i .
Revised: Denote the set of all possible augmentations for a transformation set T i as A T i .

Original: Under this deﬁnition, A T 2 is the set of all possible afﬁne or homographic transformations.
Revised: Under this deﬁnition, A T 2 is the set of all possible afﬁne or homographic transformations.

Revised: Examples of the afﬁne transformations and homographies can be seen in Appendix A.

Original: Now, consider an input image x sampled at random from a dataset of images X ⊂ X , where X is the sample space of images.
Revised: Now, consider an input image x sampled at random from a dataset of images X ⊂ X , where X is the sample space of images.

Original: We sample augmentations a 1 , a 2 ∈ A T 1 , and apply them to x to produce augmented views x 1 and x 2 , respectively.
Revised: We sample augmentations a, b ∈ A T 1 , and apply them to x to produce augmented views x 1 and x 2 , respectively.

Original: We then sample an afﬁne/homographic transformation a φ ∈ A T 2 and apply it to x 1 to produce x (cid:48) 1 .
Revised: We then sample an afﬁne/homographic transformation c φ ∈ A T 2 and apply it to x 1 to produce x (cid:48) 1 .

Original: Note that x 1 and x (cid:48) 1 are related by a homography.
Revised: Note that x 1 and x (cid:48) 1 are related by a homography.

Original: This is a core assumption relied upon by further inductive biases we introduce into our model.
Revised: This is a core assumption relied upon by further inductive biases we introduce into our model.

Original: We now describe the proposed architecture as depicted in Figure 1.
Revised: We now describe the proposed architecture as depicted in Figure 1.

Original: Let the mapping f : X → R p be parameterised by a CNN, and the mappings g : R p
Revised: Let the mapping f : X → R p be parameterised by a CNN, and the mappings g : R p

Original: →
Revised: →

Original: R k and h :
Revised: R k and h :

Original: R p
Revised: R p

Original: → R m be parameterised by MLPs, where p , k , and m are the dimensionality of the encoder latent vector, projection head latent vector, and homography parameter vector, respectively.
Revised: → R m be parameterised by MLPs, where p , k , and m are the dimensionality of the encoder latent vector, projection head latent vector, and homography parameter vector, respectively.

Original: f and g are the encoder and projection head from the original SimCLR (Chen et al., 2020a) and BYOL (Grill et al., 2020) formulations, respectively, whereas h is a new MLP tasked with estimating the homography parameters.
Revised: f and g are the encoder and projection head from the original SimCLR (Chen et al., 2020a) and BYOL (Grill et al., 2020) formulations, respectively, whereas h is a new MLP tasked with estimating the homography parameters.

Original: Note that if we are regressing all parameters of a general afﬁne transformation, then m = 6 , whereas for a full homography we have m = 8 .
Revised: Note that if we are regressing all parameters of a general afﬁne transformation, then m = 6 , whereas for a full homography we have m = 8 .

Original: For brevity, we have denoted both streams in the architecture to be a network with the same shared weights, although it may be the case that the two streams consist of networks with different weights (as is the case with BYOL).
Revised: For brevity, we have denoted both streams in the architecture to be a network with the same shared weights, although it may be the case that the two streams consist of networks with different weights (as is the case with BYOL).

Original: The loss function for our method contains two terms.
Revised: The loss function for our method contains two terms.

Original: First is the original loss function as deﬁned by the original method: NT-Xent for SimCLR and squared L 2 for BYOL.
Revised: First is the original loss function as deﬁned by the original method: NT-Xent for SimCLR and squared L 2 for BYOL.

Original: We deﬁne this ﬁrst term as L 1 ( z 1 , z 2 ) , where z 1 = g ( f ( x 1 )) and z 2 = g
Revised: We deﬁne this ﬁrst term as L 1 ( z 1 , z 2 ) , where z 1 = g ( f ( x 1 )) and z 2 = g

Original: ( f ( x 2 )) .
Revised: ( f ( x 2 )) .

Original: The second term can be seen as forcing the network to explicitly learn the afﬁne transformation or homography between x 1 and x (cid:48) 1 .
Revised: The second term can be seen as forcing the network to explicitly learn the afﬁne transformation or homography between x 1 and x (cid:48) 1 .

Original: Let the latent representations of x 1 and x (cid:48) 1 be l 1 = f ( x 1 ) and l (cid:48) 

Revised: Let the latent representations of x 1 and x (cid:48) 1 be l 1 = f ( x 1 ) and l (cid:48)

Original: = f
Revised: = f

Original: ( x (cid:48) 1 ) .
Revised: ( x (cid:48) 1 ) .

Original: We send the vector difference l 1 − l (cid:48) 1 through h to produce an estimate of the homography’s parameters.
Revised: We send the vector difference l 1 − l (cid:48) 1 through h to produce an estimate of the homography’s parameters.

Original: We regress to these parameters using mean-squared error:
Revised: We regress to these parameters using mean-squared error:

Original: L 2 ( h ( l 1 − l (cid:48) 1 ) , φ ) , where φ are the ground truth afﬁne transformation parameters.
Revised: L 2 ( h ( l 1 − l (cid:48) 1 ) , φ ) , where φ are the ground truth afﬁne transformation parameters.

Original: Thus, the complete loss function is given by:
Revised: Thus, the complete loss function is given by:

Original: [Equation]
Revised: [Equation]

Original: The vector difference naturally describes the transformation needed to move from l 1 to l (cid:48) 1 .
Revised: The vector difference naturally describes the transformation needed to move from l 1 to l (cid:48) 1 .

Original: With our architecture and learning objective, we force this vector difference transformation vector to encode the homography between x 1 and x (cid:48) 1 .
Revised: With our architecture and learning objective, we force this vector difference transformation vector to encode the homography between x 1 and x (cid:48) 1 .

Original: This interpretation may be seen as natural and intuitive.
Revised: This interpretation may be seen as natural and intuitive.

Original: Hence, the L 1 term enforces invariance to the transformations in B 1 and L 2 enforces non-invariance to the transformations in B 2 .
Revised: Hence, the L 1 term enforces invariance to the transformations in B 1 and L 2 enforces non-invariance to the transformations in B 2 .

Original: Note that this is still completely self-supervised.
Revised: Note that this is still completely self-supervised.

Original: Moreover, the restriction imposed in Equation 2 is necessary because we cannot have any transformations in a φ ’s sequence that would destroy the fact that x 1 and x (cid:48) 1 are related through a homography.
Revised: Moreover, the restriction imposed in Equation 2 is necessary because we cannot have any transformations in c φ ’s sequence that would destroy the fact that x 1 and x (cid:48) 1 are related through a homography.

Original: For example, adding a cropping transformation would break the homography assumption.
Revised: For example, adding a cropping transformation would break the homography assumption.

Original: One may add transformations that do not break this restriction (e.g. colour jitter), however, we do not explore this here.
Revised: One may add transformations that do not break this restriction (e.g. colour jitter), however, we do not explore this here.

Original: We may interpret this extended architecture as solving a multi-task learning objective in which 1) the contrastive loss between differing augmented views of the image must be minimised, and 2) another network must be able to estimate the homography between images, which explicitly forces the latent space to encode this spatial information during training.
Revised: We may interpret this extended architecture as solving a multi-task learning objective in which 1) the contrastive loss between differing augmented views of the image must be minimised, and 2) another network must be able to estimate the homography between images, which explicitly forces the latent space to encode this spatial information during training.

Original:  E XPERIMENTS 4. E XPERIMENTAL S ETUP
Revised:  E XPERIMENTS 4. E XPERIMENTAL S ETUP

Original: This section presents an empirical study comparing the original SimCLR and BYOL techniques on the CIFAR10, CIFAR100 and SVHN benchmark datasets, with and without our proposed module.
Revised: This section presents an empirical study comparing the original SimCLR and BYOL techniques on the CIFAR10, CIFAR100 and SVHN benchmark datasets, with and without our proposed module.

Original: Our goal is not to achieve near state-of-the-art performance on the datasets, but is rather to demonstrate the effectiveness of the proposed additional homography estimation objective under consistent experimental settings.
Revised: Our goal is not to achieve near state-of-the-art performance on the datasets, but is rather to demonstrate the effectiveness of the proposed additional homography estimation objective under consistent experimental settings.

Original: In all cases, the proposed module improves the performance of a linear classiﬁer on the learned representation and improves the learning speed.
Revised: In all cases, the proposed module improves the performance of a linear classiﬁer on the learned representation and improves the learning speed.

Original: The experimental setup for the self-supervised training of SimCLR and BYOL can be found in
Revised: The experimental setup for the self-supervised training of SimCLR and BYOL can be found in

Original: Table 1.
Revised: Table 1.

Original: The batch size is somewhat lower than the original methods since the original methods focused on performance on ImageNet, which requires a considerably larger batch size to perform well.
Revised: The batch size is somewhat lower than the original methods since the original methods focused on performance on ImageNet, which requires a considerably larger batch size to perform well.

Original: In some additional experiments, we ﬁnd performance decreased for our datasets with batch sizes larger than 256 for all methods (original SimCLR and BYOL, as well as our method).
Revised: In some additional experiments, we ﬁnd performance decreased for our datasets with batch sizes larger than 256 for all methods (original SimCLR and BYOL, as well as our method).

Original: Further, we found alternative optimised hyperparameter values (learning rate, optimiser, and weight decay) that worked better than those proposed in the original formulations of SimCLR and BYOL, which can be attributed to similar reasons as the batch size arguments.
Revised: Further, we found alternative optimised hyperparameter values (learning rate, optimiser, and weight decay) that worked better than those proposed in the original formulations of SimCLR and BYOL, which can be attributed to similar reasons as the batch size arguments.

Original: We use the same type of learning rate decay as the previous methods, and train for the same number of epochs (and warmup epochs) as SimCLR.
Revised: We use the same type of learning rate decay as the previous methods, and train for the same number of epochs (and warmup epochs) as SimCLR.

Original: We use a temperature of 0.5 for the NT-Xent loss and keep all images at their default resolution of 32 × 32 .
Revised: We use a temperature of 0.5 for the NT-Xent loss and keep all images at their default resolution of 32 × 32 .

Revised: Lastly, all reported conﬁdence intervals are the average across 10 trials of the full pipeline trained from scratch (SSL pretraining + linear evaluation).

Original: Performance is measured as per the literature, using linear evaluation on the relevant dataset.
Revised: Performance is measured as per the literature, using linear evaluation on the relevant dataset.

Original: The experimental setup for linear evaluation can be seen in Table 1.
Revised: The experimental setup for linear evaluation can be seen in Table 1.

Original: We freeze the encoder and only optimise the weights of a ﬁnal linear layer using cross-entropy.
Revised: We freeze the encoder and only optimise the weights of a ﬁnal linear layer using cross-entropy.

Original:  We parameterise f as ResNet50, while g and h are parameterised as two-layer ReLU
Revised: [Table] We parameterise f as ResNet50, while g and h are parameterised as two-layer ReLU

Original: MLPs (Figure 1).
Revised: MLPs (Figure 1).

Original: Further, to ensure consistency with SimCLR, we have that B= { random crop , random horizontal ﬂip , colour jitter , Gaussian blur , random grayscale } .
Revised: Further, to ensure consistency with SimCLR, we have that B= { random crop , random horizontal ﬂip , colour jitter , Gaussian blur , random grayscale } .

Original: The output[Table] of h is a six-dimensional real vector, where the six components are deﬁned according to the parameters of a general afﬁne transform: 1) rotation angle, 2) vertical translation, 3) horizontal translation, 4) scaling factor, 5) vertical shear angle, and 6) horizontal shear angle.
Revised: The output of h is a six-dimensional real vector, where the six components are deﬁned according to the parameters of a general afﬁne transform: 1) rotation angle, 2) vertical translation, 3) horizontal translation, 4) scaling factor, 5) vertical shear angle, and 6) horizontal shear angle.

Original: For a homography, the output of h is instead an eight-dimensional vector.
Revised: For a homography, the output of h is instead an eight-dimensional vector.

Original: For details about the transformations, see Appendix A.
Revised: For details about the transformations, see Appendix A.

Original: 4. A FFINE AND H OMOGRAPHY O BJECTIVE
Revised: 4. A FFINE AND H OMOGRAPHY O BJECTIVE

Original: From Tables 2 and 3 (‘+ H’ and ‘+ A’ for homography and afﬁne, respectively) we can see that the estimation of the afﬁne transformation and the homography both assist performance and allow for faster learning.
Revised: From Tables 2 and 3 (‘+ H’ and ‘+ A’ for homography and afﬁne, respectively) we can see that the estimation of the afﬁne transformation and the homography both assist performance and allow for faster learning.

Original: In particular, we note statistically signiﬁcant improvements across all datasets for both SimCLR and BYOL with the afﬁne objective.
Revised: In particular, we note statistically signiﬁcant improvements across all datasets for both SimCLR and BYOL with the afﬁne objective.

Original: 1 We posit that the ability to explicitly estimate the afﬁne transformation or homography between input images in this way allows the encoder to learn complementary information early on in training that is not available from the contrastive supervision signal.
Revised: 1 We posit that the ability to explicitly estimate the afﬁne transformation or homography between input images in this way allows the encoder to learn complementary information early on in training that is not available from the contrastive supervision signal.

Original: The ability to estimate the afﬁne transform or homography means that the network is encoding the geometry of the input images.
Revised: The ability to estimate the afﬁne transform or homography means that the network is encoding the geometry of the input images.

Original: This explicit geometric information is not directly available from the contrastive signal.
Revised: This explicit geometric information is not directly available from the contrastive signal.

Original: Interestingly, the afﬁne objective outperforms the full homography in all cases, even though an afﬁne transformation is a subset of a homography.
Revised: Interestingly, the afﬁne objective outperforms the full homography in all cases, even though an afﬁne transformation is a subset of a homography.

Original: We perform a sweep of the distortion amount for the homography and ﬁnd it consistently performs similar to or a little worse than the afﬁne transform (see Appendix B).
Revised: We perform a sweep of the distortion amount for the homography and ﬁnd it consistently performs similar to or a little worse than the afﬁne transform (see Appendix B).

Original: When the distortion factor becomes too large, accuracy drops noticeably as the images are too distorted to learn effectively.
Revised: When the distortion factor becomes too large, accuracy drops noticeably as the images are too distorted to learn effectively.

Revised: We note that incorporating our module into a network results in an average 30% additional training time versus the respective original methods.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Figure 2 shows the linear evaluation accuracy trained on the embeddings extracted from the model at each epoch during the SSL training.
Revised: Figure 2 shows the linear evaluation accuracy trained on the embeddings extracted from the model at each epoch during the SSL training.

Original: We can see that performance and convergence improves with the inclusion of the proposed module.
Revised: We can see that performance and convergence improves with the inclusion of the proposed module.

Original: The module and its accompanying additional objective of  regressing the afﬁne transform/homography may be seen as a regulariser for the original contrastive objective.
Revised: The module and its accompanying additional objective of [Figure] regressing the afﬁne transform/homography may be seen as a regulariser for the original contrastive objective.

Original: This is further evidenced by the shaded regions in the ﬁgures, in which the proposed method results in more stable performance.
Revised: This is further evidenced by the shaded regions in the ﬁgures, in which the proposed method results in more stable performance.

Original: We note that the relative beneﬁt of our proposed module diminishes with longer training time for
Revised: We note that the relative beneﬁt of our proposed module diminishes with longer training time for

Original: SimCLR and BYOL.
Revised: SimCLR and BYOL.

Original: This makes sense as the relative beneﬁt of the module decreases with time[Figure] as the model learns to estimate the afﬁne transformation or homography more accurately as the epochs progress.
Revised: This makes sense as the relative beneﬁt of the module decreases with time as the model learns to estimate the afﬁne transformation or homography more accurately as the epochs progress.

Original: We performed additional experiments on SimCLR and BYOL training the model for longer, and note that the proposed module still outperforms or performs similarly to the original methods on all datasets.
Revised: We performed additional experiments on SimCLR and BYOL training the model for longer, and note that the proposed module still outperforms or performs similarly to the original methods on all datasets.

Original: This is shown in Table 4.
Revised: This is shown in Table 4.

Original: These results also verify the ﬁndings of previous works that ﬁnd that larger models trained for longer beneﬁts self-supervised architectures (Chen et al., 2020a;
Revised: These results also verify the ﬁndings of previous works that ﬁnd that larger models trained for longer beneﬁts self-supervised architectures (Chen et al., 2020a

Original: Grill et al.,2020; Chen et al., 2020b; Kolesnikov et al., 2019).
Revised: Grill et al., Chen et al , 2020b Kolesnikov et al

Original: [Table]
Revised: [Table]

Original: We note that the performance gap between SimCLR and BYOL evident in Tables 2 and 3 in general can be attributed to the fact that in the original works, BYOL was trained for 10x as long as SimCLR, whereas we trained both for the same number of epochs as the original SimCLR work.
Revised: We note that the performance gap between SimCLR and BYOL evident in Tables 2 and 3 in general can be attributed to the fact that in the original works, BYOL was trained for 10x as long as SimCLR, whereas we trained both for the same number of epochs as the original SimCLR work.

Original: We posit that BYOL has simply not converged sufﬁciently, since BYOL eventually outperforms SimCLR (as evidenced by Table 4).
Revised: We posit that BYOL has simply not converged sufﬁciently, since BYOL eventually outperforms SimCLR (as evidenced by Table 4).

Original: This is consistent with the ﬁndings from the original works.
Revised: This is consistent with the ﬁndings from the original works.

Original: 4. I 
Revised: 4. I NVARIANCE I S N OT A LWAYS D ESIRABLE

Original: [Table]

Original: In order for a function f to be invariant to a transformation T , we must have that, for all x , f ( x ) =
Revised: In order for a function f to be invariant to a transformation T , we must have that, for all x , f ( x ) =

Original: f
Revised: f

Original: ( Tx ) .
Revised: ( Tx ) .

Original: Thus, one way to encourage invariance to T in a neural network f is to add a term to the loss function which minimises:
Revised: Thus, one way to encourage invariance to T in a neural network f is to add a term to the loss function which minimises:

Original: [Equation]
Revised: [Equation]

Original: for some measure of similarity L .
Revised: for some measure of similarity L .

Original: If we rewrite our loss function from Equation 5 in terms of our input image x and augmentations a 1 , a 2 , a φ , we get:
Revised: If we rewrite our loss function from Equation 5 in terms of our input image x and augmentations a, b, c φ , we get:

Original: [Equation]
Revised: [Equation]

Original: The ﬁrst term of the above loss, corresponding to the SimCLR/BYOL loss, is clearly of the form of
Revised: The ﬁrst term of the above loss, corresponding to the SimCLR/BYOL loss, is clearly of the form of

Original: Expression 6.
Revised: Expression 6.

Original: This means that we are encouraging our representation to be invariant to the transformations within B 1 .
Revised: This means that we are encouraging our representation to be invariant to the transformations within B 1 .

Original: However, the second term in the loss (i.e. the term corresponding to the afﬁne transformation/homography parameter estimation) is not of the form of Expression 6, since we have recast the objective into a parameter prediction task.
Revised: However, the second term in the loss (i.e. the term corresponding to the afﬁne transformation/homography parameter estimation) is not of the form of Expression 6, since we have recast the objective into a parameter prediction task.

Original: Thus, we are not encouraging invariance to the transformations within B 2 .
Revised: Thus, we are not encouraging invariance to the transformations within B 2 .

Original: We provide some empirical evidence for this in Table 5.
Revised: We provide some empirical evidence for this in Table 5.

Original: When we recast the module to minimise L 2 ( f ( x 1 ) , f ( x (cid:48) 1 ))
Revised: When we recast the module to minimise L 2 ( f ( x 1 ) , f ( x (cid:48) 1 ))

Original: ( L 2 being the mean squared error loss), performance decreases notably on all datasets, with an average relative decrease of over 8%.
Revised: ( L 2 being the mean squared error loss), performance decreases notably on all datasets, with an average relative decrease of over 8%.

Original: This is because, with this loss, we have enforced invariance to the transformations in B 2 .
Revised: This is because, with this loss, we have enforced invariance to the transformations in B 2 .

Original: In particular, we have encouraged invariance to all the elements of an afﬁne transformation, which proves problematic.
Revised: In particular, we have encouraged invariance to all the elements of an afﬁne transformation, which proves problematic.

Original: [Table]
Revised: [Table]

Original: To delve deeper into the effect of transformation invariance on performance, we extract only the ‘6’ and ‘9’ classes of the SVHN dataset as a new dataset and repeat the SSL pre-training and linear evaluation tasks.
Revised: To delve deeper into the effect of transformation invariance on performance, we extract only the ‘6’ and ‘9’ classes of the SVHN dataset as a new dataset and repeat the SSL pre-training and linear evaluation tasks.

Original: The goal of this experiment is to observe how performance degrades when the neural network is encouraged to be invariant to certain transformations - including rotation - in a setting where certain invariance (i.e. rotation) is not desirable.
Revised: The goal of this experiment is to observe how performance degrades when the neural network is encouraged to be invariant to certain transformations - including rotation - in a setting where certain invariance (i.e. rotation) is not desirable.

Original: Results can also be seen in Table 5.
Revised: Results can also be seen in Table 5.

Original: This further suggests that invariance to certain transformations is not always desirable.
Revised: This further suggests that invariance to certain transformations is not always desirable.

Original: Evidence from Table 5 suggests that transformation invariance (for this particular class of transformations) in
Revised: Evidence from Table 5 suggests that transformation invariance (for this particular class of transformations) in

Original: SSL may not always be desirable, and may, in fact, hurt performance, even when this may not be expected (as in the case with CIFAR10 and CIFAR100, since no classes of these seem as if they should be affected by transformation invariance like the ‘6’ vs ‘9’ case).
Revised: SSL may not always be desirable, and may, in fact, hurt performance, even when this may not be expected (as in the case with CIFAR10 and CIFAR100, since no classes of these seem as if they should be affected by transformation invariance like the ‘6’ vs ‘9’ case).

Original: For more details on the invariance analyses, see Appendix C.
Revised: For more details on the invariance analyses, see Appendix C.

Original: 4. T RANSFORM C OMPONENT A NALYSIS
Revised: 4. T RANSFORM C OMPONENT A NALYSIS

Original: Table 6 shows the performance of the various components of an afﬁne transformation in terms of linear evaluation accuracy on the dataset.
Revised: Table 6 shows the performance of the various components of an afﬁne transformation in terms of linear evaluation accuracy on the dataset.

Original: To compute these results, the output dimensionality of mapping h needs to be changed accordingly.
Revised: To compute these results, the output dimensionality of mapping h needs to be changed accordingly.

Original: Namely, rotation, translation, scale, and shear have corresponding output dimensionalities m of 1, 2, 1, and 2, respectively.
Revised: Namely, rotation, translation, scale, and shear have corresponding output dimensionalities m of 1, 2, 1, and 2, respectively.

Original: Interestingly, shear alone outperforms the three other transforms on all datasets for both SimCLR and BYOL.
Revised: Interestingly, shear alone outperforms the three other transforms on all datasets for both SimCLR and BYOL.

Original: We hypothesise that this is because shear corrupts the image the most out of the four transforms, but still in a recognisable way.
Revised: We hypothesise that this is because shear corrupts the image the most out of the four transforms, but still in a recognisable way.

Original: This forces the networks to learn more complex geometry and information about the object that the other transforms.
Revised: This forces the networks to learn more complex geometry and information about the object that the other transforms.

Original: We leave further investigation of this to future work
Revised: We leave further investigation of this to future work

Original: [Table] [Table]
Revised: [Table] [Table]

Revised: 4. A DDITIONAL A BLATIONS

Revised: We perform various additional experiments to motivate the choice of architecture.

Revised: We experiment with other means of encoding the latent transformation, speciﬁcally, concatenation instead of vector difference.

Revised: However, this results in marginal performance gains of an average of 0.28 percentage points across the 3 datasets.

Original: C ONCLUSION
Revised: These results do not seem to justify the noticeable additional computational cost from the transformation representation being twice the size. For primarily this reason, we opt to stick with vector difference. Further, we experiment with having the module operate on the output of g instead of f . Performance degrades for all datasets (SimCLR): 64.38 for CIFAR10, 29.99 for CIFAR100, and 82.49 for SVHN.

Revised: Lastly, we perform some preliminary experiments into having two modules: one operating on x 1 and the other operating on x 2 (instead of just one module as per our original experimental setup).

Revised: The resulting performance difference is negligible with this setup: CIFAR10 65.28 ± 0.61, CIFAR100 31.68 ± 1.04, and SVHN 84.10 ± 0.23.

Revised: We posit that this is because if the one module can solve the homography estimation for x 1 , then a module operating on x 2 will have to be able to solve the homography estimation for it, since the same types of random homographies/afﬁne transformations are being applied to both streams.

Revised:  C ONCLUSION

Original: Network size and time of training is a bottleneck in modern SS architectures that can compete on a performance level like supervised alternatives.
Revised: Network size and time of training is a bottleneck in modern SS architectures that can compete on a performance level like supervised alternatives.

Original: We have shown that the proposed module that regresses the parameters of an afﬁne transformation or homography as an additional objective assists this training bottleneck with faster convergence and better performance.
Revised: We have shown that the proposed module that regresses the parameters of an afﬁne transformation or homography as an additional objective assists this training bottleneck with faster convergence and better performance.

Original: The architecture of the module does not encourage invariance to the afﬁne or homographic transformation, as invariance has been previously shown to be potentially harmful (Lee et al., 2020).
Revised: The architecture of the module does not encourage invariance to the afﬁne or homographic transformation, as invariance has been previously shown to be potentially harmful (Lee et al., 2020).

Original: Rather, the proposed module encourages these transformations to be encoded within the latent space itself by directly estimating the parameters of the transformation.
Revised: Rather, the proposed module encourages these transformations to be encoded within the latent space itself by directly estimating the parameters of the transformation.

Original: Lastly, we note that the afﬁne transformation performs better in all cases than the full homography, even though the homography is a superset of afﬁne transformations.
Revised: Lastly, we note that the afﬁne transformation performs better in all cases than the full homography, even though the homography is a superset of afﬁne transformations.

Original: The experiments suggest that the additional ability of perspective transform in a homography does not yield any tangible beneﬁt over a regular afﬁne transformation in such low-resolution settings.
Revised: The experiments suggest that the additional ability of perspective transform in a homography does not yield any tangible beneﬁt over a regular afﬁne transformation in such low-resolution settings.

Original: A D ATA A UGMENTATION D ETAILS
Revised: A D ATA A UGMENTATION D ETAILS

Original: Tables 7 and 8 detail the parameter values and value ranges used in the experiments for the base transformation sets B 1 and B 2 , respectively.
Revised: Tables 7 and 8 detail the parameter values and value ranges used in the experiments for the base transformation sets B 1 and B 2 , respectively.

Original: The transformations from B 1 are applied with a speciﬁed probability.
Revised: The transformations from B 1 are applied with a speciﬁed probability.

Original: We also normalise the parameters of the afﬁne transformations in the following way.
Revised: We also normalise the parameters of the afﬁne transformations in the following way.

Original: Consider a rotation angle α , translation values t x , t y  and shear angles s x , s y .
Revised: Consider a rotation angle α , translation values t x , t y , and shear angles s x , s y .

Original: We perform the following normalisation on these parameters: α := α/ 360 ; t x :=
Revised: We perform the following normalisation on these parameters: α := α/ 360 ; t x :=

Original: t x /W ; t y :=
Revised: t x /W ; t y :=

Original: t y /H ; s x := s x /s max ; s y :=
Revised: t y /H ; s x := s x /s max ; s y :=

Original: s y /s max , where H, W are the image height and width, respectively, and s max is the maximum allowed shear.[Table] Jitter [Table]
Revised: s y /s max , where H, W are the image height and width, respectively, and s max is the maximum allowed shear.

Original: [Table]

Original: B A FFINE VS H OMOGRAPHY
Revised: B A FFINE VS H OMOGRAPHY

Original: In addition to the perspective distortion factor of 0. [Table] we perform a sweep across this parameter for the values { 0 . 1 , 0 . 2 , 0 . 8 } .
Revised: In addition to the perspective distortion factor of 0.5, we perform a sweep across this parameter for the values { 0 . 1 , 0 . 2 , 0 . 8 } .

Original: The results can be seen in Table 9.
Revised: The results can be seen in Table 9.

Original: Interestingly, most distortion factors perform similarly on these datasets [Table] with a distortion factor of 0 .
Revised: Interestingly, most distortion factors perform similarly on these datasets, with a distortion factor of 0 .

Original: 5 performing best on average.
Revised: 5 performing best on average.

Original: However, when the factor gets too large [Table] as is the case for 0 . 
Revised: However, when the factor gets too large, as is the case for 0 . 8 , then the images become too corrupted for the neural network to seemingly learn anything useful.

Original: [Table]
Revised: [Table]

Original: then the images become too corrupted for the neural network to seemingly learn anything useful

Original: [Table] [Table]
Revised: [Table] [Table]

Original: C I NVARIANCE A NALYSIS
Revised: C I NVARIANCE A NALYSIS

Original: Figures3 and 4  shows the confusion matrices for a particular run of the SVHN dataset for SimCLR when enforcing (and not enforcing) transformation invariance.
Revised: Figures 4 and 5 shows the confusion matrices for a particular run of the SVHN dataset for SimCLR when enforcing (and not enforcing) transformation invariance.

Original: Interestingly, transformation invariance negatively affects most classes of the dataset.
Revised: Interestingly, transformation invariance negatively affects most classes of the dataset.

Original: Unsurprisingly, the classes ‘6’ and ‘9’ are most negatively affected when transformation invariance is enforced.
Revised: Unsurprisingly, the classes ‘6’ and ‘9’ are most negatively affected when transformation invariance is enforced.

Original: Rotation invariance in this context is prohibitive, and performance subsequently drops.
Revised: Rotation invariance in this context is prohibitive [Table] and performance subsequently drops.

Original: By recasting the module as proposed - encode the transformation and predict its parameters - we do not enforce invariance, and instead allow the network to learn from a richer supervision signal by learning to estimate a homography. 
Revised: By recasting the module as proposed - encode the transformation and predict its parameters - we do not enforce invariance, and instead allow the network to learn from a richer supervision signal by learning to estimate a homography. [Figure]