Self-Supervised Learning Through Efference Copies
Authors: Franz Scherr
Qinghai Guo
Timoleon Moraitis
Venue: NeurIPS 2022 Accept (2022)
Keywords: Self-supervised Learning
Theoretical Neuroscience
Sensory-Motor Learning
Representation Learning
Visual Features
Visual Perception
Deep Learning
Neural Networks
Embodied Intelligence
Inverse Models

Abstract: Self-supervised learning (SSL) methods aim to exploit the abundance of unlabelled data for machine learning (ML), however the underlying principles are often method-specific. An SSL framework derived from biological first principles of embodied learning could unify the various SSL methods, help elucidate learning in the brain, and possibly improve ML. SSL commonly transforms each training datapoint into a pair of views, uses the knowledge of this pairing as a positive (i.e. non-contrastive) self-supervisory sign, and potentially opposes it to unrelated, (i.e. contrastive) negative examples. Here, we show that this type of self-supervision is an incomplete implementation of a concept from neuroscience, the Efference Copy (EC). Specifically, the brain also transforms the environment through efference, i.e. motor commands, however it sends to itself an EC of the full commands, i.e. more than a mere SSL sign. In addition, its action representations are likely egocentric. From such a principled foundation we formally recover and extend SSL methods such as SimCLR, BYOL, and ReLIC under a common theoretical framework, i.e. Self-supervision Through Efference Copies (S-TEC). Empirically, S-TEC restructures meaningfully the within- and between-class representations. This manifests as improvement in recent strong SSL baselines in image classification, segmentation, object detection, and in audio. These results hypothesize a testable positive influence from the brain's motor outputs onto its sensory representations.

Self-Supervised Learning Through Efference Copies
Franz Scherr's Huawei Technologies franz. scherrghuanei com
Qinghai Guo Huawei Technologies gughan @huawei. com
Timoleon Moraitis
Huawei Technologies timole morai wwsdhhaweii.cccmmm com
Abstract
Self-supervised learning (SSL) methods aim to exploit the abundance of unlabelled data for machine learning (ML), however the underlying principles are often method-specific. An SSL framework derived from biological first principles of embodied learning could unify the various SSL methods, help elucidate learning in the brain, and possibly improve ML. SSL commonly transforms each training datapoint into pair of views, uses the knowledge of this pairing as a positive (i.e. non-contrastive) self-supervisory sign, and potentially opposes it to unrelated, (i.e. contrastive) negative examples. Here, we show that this type of self-supervision is incomplete implementation of concept from neuroscience, the Efference Copy (EC). Specifically, the brain also transforms the environment through efference, i.e. motor commands, however it sends to itself an EC of the full commands, i.e. more than mere SSL sign. In addition, its action representations are likely egocentric. From such a principled foundation we formally recover and extend SSL methods such SimCLR, BYO.. and ReLIC under common theoretical framework, ie. Self-supervision Through Efference Copies (S-TEC). Empirically, S-TEC restructures meaningfully the within- and between-class representations. This manifests as improvement in recent strong SSL baselines image classification, segmentation, object detection, and in audio. These results hypothesize testable positive influence from the brain's 's motor outputs onto its sensory representations.
Introduction
Deep Learning (DL. has drawn inspiration from Neuroscience and also offers models for under- standing aspects the brain (Richards et al., 2019). DL. has been extremely successful, largely owing labelled big datasets. However, such labelling is costly procedure carried out by human supervisors. Fully unsupervised ML techniques do exist, however they rarely reach the performance of supervised learning (Moraitis al., 2022: Journé al., 2022). On the other hand, recently, category of algorithms that are self-supervised has emerged. In self-supervised learning (SSL), the model itself generates the supervisory signal, so that human supervision is not needed, and then uses that signal for supervised learning. Recent SSL algorithms generate the supervisory signal by using pairs of inputs where it known whether they are of the same or of different instance, therefore celf-generating positive or negative labels. Examples can be associated as being positive, e.g. based on their temporal proximity, if the input is in a sequential domain (Oord al., 2018). Advanced SSL algorithms generate themselves the positive pairs of inputs, by augmenting the training dataset (He al., 2020; Chen al., 2020a; Grill et al., 2020; Caron al., 2020; Mitrovic et al., 2021). Further improving these algorithms even has the potential to outperform supervised learning (Tomasev et al., 2022), because more information exists in the comparison of complete input pairs than individual human-labelled examples. Therefore, improving SSL consists in devising representation-leammi methods that better capture that information. However, conceptual frameworks that unify these
Huawei Zurich Research Center, Switzerland, Huaawii ACS Lab, Shenzhen, China, *Corresponding author
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

A
Action
Environment
Efference
Copy (EC)
Object
Changed sensory input
Sensory processing
Pezt
Inverse go model
2t'
Selt-supervised learning
B
Aego
Efference Copy (EC)
Restitt
Augmentations
Ø
Lid
Shared parameters
MLP
Amnanip
y MLP
Cmanip
Image
Downstream tasks: classificaioon,
Discrate dasses
Figure Efference copy (EC), A) Censory-motor system: Efferent actions a (change focus, manipulations, etc.) yield changed sensory input x'. The internal copy of the motor command, i.e. the EC, we propose, may used as a self-supervisory signal for learning an inverse model qo- B) Abstract MI. setting: The action space comprises switching and augmenting images. Sensory processing and inverse models are implemented as neural networks, trained through EC's feedback.
principles existing SSL methods, and guide towards new improved ones, are scarce (Balestriero and LeCun, 2022).
2 Efference copies in the central nervous system
The operation of the biological central nervous system (CNS), which appears to learn mostly without external supervision, may provide such framework. Conversely, ML simulations within such framework may also generate testable hypotheses for biological SSL. In the present study we take this abstract hope and formulate it as concrete link from SSL to specific mechanism in the CNS. We begin by observing first, that the CNS of vertebrate animals is believed to have evolved with the main purpose of performing sensory-motor control and learning, and second, that the data manipulations that augment the training examples in ML implementations of SSL can be viewed as motor actions. The search for analogies then can focus on looking for possible self-supervisory signals within biological motor control and learning.
A particularly well-suited and well-established signal in the sensory-mold system is that of the Effer- ence Copy (EC) (von Helmholtz, 1867; McNamee and Wolpert, 2019) or Corollary Discharge (Sperry, 1950). Namely, it has been shown that when component of the CNS addresses the body's muscles with an efferent, i.e. outgoing. motor command or action, often it also sends a copy to the CNS itself, see Fig. AA. ECs have multiple functions and abundant supporting evidence (Kennedy et al., 2014; McNamee and Wolpert, 2019; Kilteni et al., 2020; Latash, 2021). I). For example, certain motor commands responsible for the locomotion of frogs generated in the spinal cord, but copied to the brainstem, which is responsible for motor control of the eye (von Uckermann et al., 2013). disturbances the visual field are then predicted and appropriately
The body- sollly-ovmmeee-ellleeeee counteracted by eye movements that stabilize the frog's gaze. Therefore, one function of EC is coordinate different motor controllers of the body. Another function of EC is focus sensory processing on externally-generraee generated and unpredicted stimuli by cancelling predictable sensations of self- generated actions. E.g., humans cannot tickle themselves effectively, because by using its ECs the CNS predicts the sensory consequence of its own action, and cancels it before it is per- ceived (Blakemore al., 1998). The role of ECs in humans actually broader and very central to motor control. Specifically, the control of bodily movements involves forward internal models that the brain maintains, i.e. models that predict the sensory inputs that result from each motor command (Kawato, 1999; McNamee and Wolpert, 2019). These forward models rely access to

---

A
B
C
PEC; 90
Object-identity-
related actions
Oid
Add
Odd
0dd
Same-object
manipulations
Oua
Smanip
O
unknown Gmanti
Single
Ggaanip
move up
Several Smmnip
Figure 2: Actions. A) Categories: New objects can be brought into focus Ond 1 (e.g. saccades).
Else (Oid = 0), the same object can be manipulated by Amanip (e.g. moving). B) Dependency structure graphical model. C) Corresponding decision tree if @manip further assumed to be discrete.
motor commands to generate their predictions, and that access is provided by ECs. Importantly, motor control also involves inverse models, which map representations of targeted movement sensations to their possible actions (Rizzolatti et al., 1998; Kawato, 1999) (Fig. IA). addition to motor control, ECs also underlie motor learning (Witney et al., 1999; Troyer and Doupe, 2000; Diedrichsen et al., 2003; Engert, 2013; Brownstone et al., 2015). For example, when learning an inverse model, the structures that calculate errors must access the efference.
Given the pervasive role of ECs in sensory processing, motor control, and motor learning, we hypothesize that ECs could play key role in the learning of sensory representations too, and that it does so through the learning of inverse models. More specifically, we hypothesize that, if the EC acts as sulf--uupry learning signal, then it improves the sensory learning process, e.g. improving the later classification of input examples. Rather than physiological experiments, or biologically detailed simulations, we will test the hypothesis in abstract ML setting. Nevertheless, we will addition, our model could
use mechanisms that do have plausible biophysical implementations. improve ML methods by providing more of the information content of paired input datapoints to SSL. That is because ECs can be rich and diverse signals, i.c. they can provide the full description of the actions that generate input pairs, and can do so for varied types of actions.
3 S-TEC: Self-supervision Through Efference Copies
3.1 Definitions and key principles
Our basic assumption is that (a) an EC is available, i.e. copy of the efferent motor commands, or actions. The essence of our strategy is to use this EC as a target label to (b) learn an inverse model, mapping sensory inputs to the motor outputs that caused the changed inputs in the first place (Kawato, 1999). We assume (c) a hierarchical model, c.g. a multilayer neural network. We conjecture that a model that improves on this motor-oriented task, will also improve its intermediate sensory representations as a direct consequence, which are then useful to a wider variety of sensory tasks. In our experiments, we use the representation for classification. In order to concretize the model, let x E DD generally denote sensory inputs. Furthermore, we define the motor commands as actions E Aego that result transformed inputs x' T(x, a), denoting with T the transformation function, see also Fig. 1A. In the following, we will simply write the EC as a probability distribution PEc(a) (a]ar, lr, x²) to indicate the distribution of values it will assume given the sensory inputs are before an action was taken, and are x' thereafter. We utilize this as ground truth that the inverse model needs to predict. More formally, we denote the to-be-learned inverse sensory-motor mapping by qo (alæ, _x') with free parameters 0 that surmise synaptic weights. Learning then is the minimization of the discrepancy between the ground truth pec and our model qo:
min
E A T(z.a)
(PE qo)
(1)
(Loss)
In the above formulation, we denote with DKL the Kullback-Leible divergence, and introduce the loss function L that will be helpful later. The broad concept given in Eq. (1) is so far agnostic to the specific types of actions and sensory inputs. To render the matter more concrete, and to align with the examples for sensory modalities in Section 2, we will focus on the visual sensory domain. This also facilitates the validation of our approach by MI. experiments on contemporary datasets and architectures, see Section 4. We assume that only one type of sensory object is observed with
3

---

each sensory input. We denote the set of possible actions as Aggo To account for the fact that the model concerns sensory- motor control in the physical world, (d) actions must account for two types of sensory transformations (see Fig. 2A), namely:
(dI) Object-identit--rlltted actions did- This category of action switches between sensed objects, e.g. by saccade of the eyes, bringing entirely new objects into focus, or not. In the context of standard vision datasets that are comprised of static images, we simply exchange the currently viewed image with a randomly sampled new one. The two types of actions in this category, i.e. switching or not, are and 1 or O:d respectively.
(d2) Same-object manipulations dmanip- This category is ddentity-preservingg i.e. maintain- ing the sensed object but the observer actively manipulates it or its view, e.g.: turning the object, or moving to closer vantage point. With static images, this kind of transformation is naturally formed by commonly used image augmentation operations. E.g. spatial trans- formations that crop an image with random size and random aspect ratio can simulate the movement to different point at closer distance whereas mirroring the image horizontally corresponds well to rotating a symmetric 3D object, see Fig. IB. We denote an action that transforms one augmented view x into the other augmented view x' by Omanip-
The
actions are of two types and E {0. 1). Therefore, this part of the action representation is categorical. Based on this, (e) we model the entire action representation as categorical, i.e. including Ømanip- This is to follow the biological evidence that the brain maintains uniform principles throughout its organization, e.g. throughout the cortex (Douglas et al., 1989), including motor areas (Bastos et al., 2012). Moreover, there is significant evidence that this uniform organization does specifically have categorical structure, were different actions are represented by different clusters of neurons (Graziano, 2016). Importantly, this allows learning the associated inverse model by means of classification task, as will be introduced later.
As the overall action a is composed by two parts, i.e. = (aid, Amanip), we can summarize this categorical structure as a graphical model and decision tree, shown in Fig. 2B and C. Hence, given the same object continues to be in focus, i.e. Oid = O, then there exist several options for the object- manipulating action Omanip. In the other case, where focus is switched to a different object, i.e. asd = 1, there is no value of the ohject-manipulation action Omanip that relates æ and x'. To formally represent this in the decision tree, we assign all probability to some unknown Amanip in that case.
So far we have not described how each class of action Amanip. and therefore its copy EC, is parametrized by the motor controller. Based on the fact that EC conveys to the observer the action that himself is taking, it is appropriate to (f) use an egocentric representation of actions instead of aligning the actions with an allocentric reference point, i.e. with the environment. To do so, notice that the spatial transformations introduced in (d2) are affine, thus can be represented with their associated transformation matrices. This logical parametrization allows us to conveniently compute the egocentric action that is needed to turn x into a': By multiplication of the transformation matrix that gave rise to one view from the original with the inverted transformation matrix that gave rise to the other. This highlights difference to the allocentric representation of actions that was chosen in other work (Lee al., 2021), where transformations were aligned to the original, allocentric reference frame (i.e. differences of scales rather than their quotient as it would emerge here).
3.2 Formalism
Through the preceding dependency structure (Fig. 2B, C), the inverse model naturally decomposes into two more specific inverse models, where one is attributed to object-identit--rlate actions, and the other to the same-object manipulations, to which we simply refer to as "identity-related inverse model' qo (0,alaz,, and "manipulation-related inverse model" qo (0 (i,,, x') respectively. Therefore we have that (o x') = gotaalzz x')o (Cmaanip (iid 2, 2 Apply- ing the same also for the ground truth Pec enables us to split the loss function into separate parts L Lid + Lmanip that reflect learning of the ceentit--relttd inverse model and learn- ing of the manipulation-relant
inverse model correspondingly. More precisely, the loss dedi- cated to the identity-related inverse model is given by Lia DKI. (kr.(pee(aaa jr, x²): (aaalar while similarly, the loss dedicated to the manipulation-related inverse model is given by Cmanip = DKL (PEC (Amanip (aia, x, x') ); qo (Amanipl (ia, x,, x') x²)), see also Appendix D for details.

---

In practice, we also include regularization losses Lreg. see Appendix C, and weight the relative importance of the loss terms by hyperparameters X. Therefore, the loss that we consider is given by:
L = Lid + Amanip-mannpp + AregCreg
(2)
Instantiation of the inverse models. The specific formulation of (o(a|22, x²) determines the loss function that will be optimized. We consider several options for the formulation of qo 22, x²) that allow us recover various contemporary approaches for SSL as we discuss in Results Section 4.1.
On the other hand, for the manipulation-relatt representation that clusters similar actions as advocated in key principle (e). We implemented this by subdividing the support of each single component Ot; manip of amanip (consisting of 6 components for the affine transformation) into number of K discrete bins. We indicate these discretized versions of the real actions with hat x and define the probability of mp,mmnnp being in bin as:
inverse model, we opted for a categorical action
qe (1k = b) jaa = 0, x. x²) = exp
E, exp (V (f(x), f(x'))
where we have introduced feature extractor f (ResNets in our case, see Fig. 1) and the functions t (for which used MLPs) to express the model's belief that ag mamip assumes value in discrete bin j. Note that the functions / and db both are learnable, but the dependence on @ is omitted for brevity. We refer to Fig. for an ablation study on alternative instantiations of the manipulation-related inverse model.
4 Results
4.1 Recovering contrastive & non-contrastive SSL from the identity-related inverse model
Depending on the specific instantiation of the identity-related inverse model, we recover several common approaches for SSL using the concept of ECs. In particular, we show that based the choice of the learned ge (0,a x'). we can recover from the identity-related loss either contrastive losses (i.e. instance discrimination) such as employed in SimCLR (Chen al., 2020a), ReLIC (Mitrovic et al., 2021) or ReLICv2 (Tomasev et al., 2022), or non-contrastive approaches such as BYOL (Grill al., 2020) (see below, and Appendices D.7.1, D.7.2, and D.7.3).
First, we consider here as an example the identity-related inverse model that gives rise to the contrastive loss of SimCLR (Chen et al., 2020a). We define this inverse model's probability of no identity-switch, i.e. and 0, in the common way used for the positive view in contrastive learning (Chen al., 2020a), for which we adopt the notation provided by Mitrovic et al. (2021):
(o(()))), f(2)))/T Ex, exp (b( (
qa(aia 0/2, x')
(4)
Here, á is function that computes a similarity between the features produced by f. i.e. it compares the intermediate sensory representations. We defined as a scalar product between projected features: o(h, h/) >. g: (h)g: (h'), whereas y is a multi-layer perceptron (MLP), following typical choices the literature (see also Appendix B). The scalar T a temperature hyperparameter, and the set C is composed of additional candidate inputs to which is compared to (through the denominator). Note that g is learnable also, thus depending on
We assume that the EC is a perfect copy of a, hence Prc(a|a, x') assigns all probability to the true action a that was applied. doing so, we obtain an upper bound of the objective (2) that we use for our S-TEC experiments (see Appendix D for the derivation). Its associated component dedicated to the identity-relatee inverse model the typical contrastive learning objective:
(())
L < log Σ nn E exp (d(f(x), (x.))/r)
exp (v
Amanip
log Σ, exp (Wh,, (f f(a"))) + >xeg Freg
(5)
where we introduced x to always represent an input that is related to x through did = We use x" also for the second term that concerns the manipulation-relate inverse model, to reflect that the loss
5

---

is only applied in that condition. For additional convenience we write j" to refer to the bin in which the value of mamippp relating and x", falls.
By integrating a confidence-estimate into the identity-related inverse model, we obtain and extend SSL methods such ReLIC and ReLICv2. Furthermore, by defining the identity-related inverse model as a normal distribution, we obtain non-contrastive losses such as that of BOLL. For these derivations, see Appendices D.7.2 and D.7.3. Interestingly, the loss for VICReg (Bardes al., 2022) is of the same type as derived from S-TEC's principles for a normally-distributed identity-related inverse model.
4.2 Experimental evaluation
Table 1: Accuracies obtained with linear classification (mean and std over 5 independent runs).
Architecture
Method
CIFAR-10
CIFAR-100
STL-10
SimCLR repr. (Chen al., 2020a)
91.5 10.1
65.3 10.3
91.5 10.4
ReLIC rept. (Mitrovic et al., 2021)
91.5 #0.2
65.6 #0.3
91.4 #0.2
MoCo v2 repr. (Chen al., 2020c)
90.9 #0.2
65.7 #0.4
88.6 #0.6
ResNet-18
BYOL repr. (Grill et al., 2020)
92.0 #0.2
66.4 #0.4
91.1 #0.2
S-TEC (ours)
92.0 10.2
66.6 10.3
91.6 10.2
S-TEC* (ours)
92.6 10.1
67.6 +0.44
91.4 10.2
SimCLR repr. (Chen al., 2020a)
93,2 #0.2
70.8 #0.2
94.0 #0.1
ReLIC rept. (Mitrovic et al., 2021)
93.2 +0.2
70.8 20.2
93.8 #0.1
ResNet-50
S-TEC (ours)
93.9 #0.2
71.9 #0.4
94.3 #0.2
S-TEC* (ours)
94.0 #0.1
72.4 #0.1
93.9 +0.2
In the preceding Sections we have derived our framework that connects the concept of ECs to current methods for SSL. Here, we present the results of our experimental evaluations that aim to assess the quality of the representations that can be learned with our approach. For this purpose, we considered various image datasets, including CIFAR-10/100 (Krizhevsky al., 2009), STL-10 (Coates et al., 2011) as well as the ImageNet ILSVRC-2012 dataset (Russakovsky et al., 2015), and compare S-TEC against several other SSL algorithms, such as SimCLR (Chen al., 2020a), MoCo v2 (Chen et al., 2020c), BYL (Grill et al., 2020) and ReLIC (Mitrovic et al., 2021). We follow the same procedure for experiments, where we first performed SSL and subsequently determine the class prediction accuracy of linear classifier that is trained on the emergent representations. During SSL the same data augmentation methods as in Chen et al. (2020a) are applied throughout, i.c. also colour augmentations, but these were not considered for training inverse model with S-TEC, see also Appendix A for details to the augmentations applied. We adopted the ResNet v1 framework (He et al., 2016), and used specifically ResNet- 18 and ResNet-50 architectures as the feature extractor f. while o and z were generally implemented as multi-layer perceptrons (MLPs), see Fig. 1B, and Appendix B for architectural details. These networks were optimized during SSL by gradient-descent using the adaptive rate scaling of the LARS algorithm (You et al., 2017) with learning rate warmup and decay. If not otherwise stated, SSL was performed for 1,000 epochs. See Appendix C for details to the optimization with SSL and for details of training the linear classifier. For all our comparisons and ablation studies, we stress that the overlap in the implementation maximal. Especially for the comparison between SimCLR and S-TEC (and between ReLIC and S-TEC*), we emphasize that
A
B
Allocentric
Egocentric
67-
Classification
65.8 10.3
66.6 20.9
Accuracy of lineardassitg
66-
L. Regression
66.1 10.6 65.5 10.1
65-
No Camaip
65.3 40.3
10"
10'
10
15
20
Amaaip
(K
6)
Number of bins K (Auass =
Figure 3: Ablation study on CIFAR-100 with ResNet-18. A) Instantiation of the manipulation- related inverse model and action representation. B) Hyperparameter sweep. For any setting we report mean and 95% confidence interval based on 25 independent runs.

---

only loss functions are changed and manipulation-relann relltte inverse models are added. As mentioned in Section 4.1, the identity-related inverse model can also be instantiated based on other methods for SSL, such ReLIC (Mitrovic et al., 2021) through suitable choice of qo (asal (z, x'), see Appendix D. We denoted this specific variation with S-TEC" (i.e. target networks etc.).
CIFAR-10/100 and STL-10. We report the accuracies that linear classifiers could attain after SSL on the respective datasets in Table 1. Using 5 independent runs for each setting that we considered revealed that the manipulation-relatee related inverse model in the case of S-TEC or S-TEC" consistently increased the accuracy of linear classifier over the respective baseline.
Table 2: Comparing with the results of (Lee et al., 2021) on STL-10, i see Table 7 thereof.
200 Epochs
1,000 epochs
SimSiam impl. Lee et (2021)
86.32t 86.03t
90.2 #0.3
SimSiam + AugSelf (Lee et al., 2021)
90.8 #0.2
SimCLR repr. (Chen et al., 2020a)
86.1 #0.2
91.5 #0.4
S-TEC (ours)
86.2 #0.2
91.6 #0.2
We also compared our approach with the results of Lee et al. (2021), who considered a similar augmentation-aware training setting, that was mainly focused on the transferability of represen- tations between domains. We considered the case in which their method exhibited the strongest im- provement on STL-10, see Table of (Lee al., 2021) ("crop"), and retrained their model using the same number of epochs (1,000). We used their implementation and employed the same im- age augmentations as we did. Results are shown in Table 2. Conversely, we also tested our meth- ods in a 200 epoch training budget, as originally done by Lee et al. (2021) and included the best reported performance that they obtained, see Ta- ble 7 of (Lee et al., 2021), noting that our method did not outperform in this case.
Table 3: ImageNet results (ResNet-50).
Method (100 epoch)
Top-1 (val.)
SimCLR rept. (Chen al., 2020a)
64.6
ReLIC repr. (Mitrovic et al., 2021)
66.2
S-TEC (ours)
64.8
S-TEC* (aurs)
66.3
Method (300 epoch)
Top-1 1 (val.)
ReLIC repr. (Mitrovic et al. 2021) S-TEC* (ours)
70.0
70.2
Method (2 800 epoch)
Top-1 (test)
MoCo v2 (Chen et al., 2020c)
71.1
SWAV (Caron 2020)
75.3
SimCLR (Chen et al., 2020u)
69.3
To investigate the differences between our ap- proach and that of (Lee et al., 2021), con- ducted an ablation study exchanging the ego- centric action representation that we used with an allocentric one. In addition, we probed the impact of replacing action classification with L2 regression. Experiments were performed on CIFAR- 100 using ResNet-18s,, with Egocen- tric+Classificattin yielding 66.6% accuracy over the next best setting Allocentric+L.2 Regression with 66.1 1 %, which was employed by Lee et al. (2021) (Fig. 3A and Appendix E). We hypoth- esize that classification affords the model more flexibility in its output distribution, thus it can handle uncertainty of its action prediction better.
BYOL (Grill 2020)
74.3
ReLIC (Mitrovic et al., 2021)
74.8
ReLICv2 2 (Tomasev et al., 2022)
77.1
VICReg (Bardes et al., 2022)
73.2
Table 4: Transfer learning on PASCAL VOC.
Method (300 epoch) Obj. Detection (AP50)
ReLIC repr. (Mitovicc al., 2021) S-TEC* (ours)
82.3 (taat2007)
82.5 (t 2007)
Method (300 epoch) Segmentation (mloU)
ReLIC repe. (Mitrovic al.. 2021)
69.9 (va12012)
S-TEC* (ours)
70.5 (va12012)
Method (1000 epoch) Segmentation (mloU)
ImageNet. We experimented with ImageNet ILSVRC-2012 (Russakovsky et al., 2015) to demonstrate that S-TEC and S-TEC* also scale.
BYOL (Grill al., 2020)
76.3 (va12012)
ReLICv2 (Tomusev et al., 2022)
77.9 (va12012)
We performed training for either 100 or 300 epochs and report the results in Table 3, confirming that S-TEC is not restricted to small datasets.
7

---

A
Standard SSL
Unsugmented
distance
Remove augmented images
example
Trucks
Planes
Magritudo augmentation
Color and cates dess
Strangly augmented
B
S-TEC
D
Increased di distance
Remove augmented imagees
Trucks
Planes
Figure 4: Analysis. A-B) Schematic of conjectured feature space organization: En-unaware SSL co-locates views of an object (A), while S-TEC separates according to the level of augmentation (B). C-D) UMAP embeddings of images and their augmentations. Collapsed representations become separated. E-F) UMAP embeddings of unaugmented images, using the projection of C-D. Increased class-separation is visible.
Object detection and semantic segmentation. SSL aims to install generally useful representa- tions. Thus, considered downstream tasks beyond classification: object detection and semantic segmentation on PASCAL VOC (Everingham et al., 2010) using Faster R-CNN (Ren et al., 2015) and fully convolutional networks (Long et al., 2015), respectively, along with a ResNet-50 backbone. We initialized this backbone with the parameters that resulted from SSL on ImageNet for 300 epochs, and then fine-tuned the network on the new task runs with different initializations of remaining parameters). Results are reported in Table 4, see also Appendix B and C for details.
Hy pepraarraeeeer dependence and learning dynamics. To assess the dependence of our results on hyperparameters, we carried out several studies on CIFAR-100 with ResNet-18s: We performed a sweep over Amanip that scales the impact of the manipulation number of bins K used in the classification for action components. Performance depends significantly on Amanip, but is less affected by K, as long as there are not too many bins (i.c. < 10), see Fig. 3B for results. Lastly, we also exhibit the loss dynamics and learning progress in Appendix E.3.
loss, and a sweep over the
Audio (LibriSpeech). Finally, we also attempted to improve representation learning in the audio domain. Specifically, we considered the same data and model as introduced by Oord al. (2018). In addition to the time-sensitive identity-related inverse model, as it emerges with CPC, we also added time-insensitive identity-related inverse model. This allowed to achieve 65.4% accuracy phoneme classification (with frozen features) as opposed to 65.1% 1% that we obtained with CPC.
5 Analysis and intuitions
Increased information content. A consequence learning a manipulation--ellted inverse model is that additional information must be expressed by the feature extractor f. Curiously, it had been shown in several other works that strong methods for SSL only perform well on downstream tasks, such classification, if intermediate representations are used. E.g. Chen al. (2020a) showed that inserting a nonlinear MLP between the feature extractor f and the loss for contrastive SSL resulted subsequent linear classification, opposed the control case
in significantly better performance where this MIP was missing. This effect explained by loss of information that is not necessarily important for the contrastive SSL objective, but for downstream tasks. In fact, recent work (Chen et al., 2020b; Mitrovic et al., 2021) observed better performance if the depth of MLP was further increased. This supports the viewpoint that additional information, albeit being potentially redundant to the contrastive SSL objective, is desired for downstream tasks of interest, see also Lee et al. (2021).
Better organization of class borders. Furthermore, we conjecture that learning the additional cinipulation-relatee inverse model using Emanip for S-TEC (see Eq. (2)) encourages the feature
8

---

space to be better organized. Firstly, note that conventional (contrastive or non-contrastive) SSL promotes representations of the same object in different views be co-located (Wang and Liu, 2021) (see Fig. 4A, e.g. truck and its tire). On the other hand, S-TEC, due to its EC-aware learning, encourages representations of different views of one object (e.g. full truck vs tire) to take different positions (truck and tire in Fig. 4B). As a result, we hypothesize, the representations of canonical, untransformed views of the same type of object (e.g. trucks) must become more concentrated, to allow the transformed ones to spread. This must then increase the separation between clusters of untransformed objects (Fig. 4: purple arrow, A vs B). Moreover, S-TEC's separation of augmented views from unaugmented ones within object-class allows the model to instead locate similar augmented views of different object classes. This then forms arguably semantically meaningful class-borders and transitions (Fig. 4B, truck tire and plane tire).
Experimentally, this hypothesis is supported by the features computed by ResNet-50 on images of the testing set of CIFAR-10, including also their augmentations. In Fig. 4C-F, we computed lower-dimensional projections by the means of UMAP (MeInnes et al., 2018), and colour-coded the minus the relative area of the cropped image). Comparing
magnitude of augmentation (defined Fig. 4C and D, the model trained with S-TEC (Panel D) is aware of the zoom level of the augmentation and places more augmented images in similar regions, i.e. the borders, while the model trained without the full EC (SimCLR in this case) is oblivious to it (Panel C). If we embed the original unaugmented images in the same projection (Fig. 4E-F), the apparent class centre distance increases for S- S-TEC (Panel F), due to the now missing augmented images on the border. Quantitatively, we computed for each class separately the distance between the centroid of augmented image representations and the centroid of unaugmented image representations. Averaged over all classes, we find that this distance is 10.5 for S-TEC and 0.4 for SimCLR (both latent spaces cover similar scales). This further confirms that SimCLR clusters these subsets (augmented and unaugmented images) of one class around a single centroid, whereas S-TEC separates them.
Importantly, S-TEC may offer a new method for avoiding representational collapse (Grill et al., 2020; Bardes et al., 2022: Balestriero and LeCun, 2022) in non-contrastive SSL, because it explicitly displaces representations of the same object if they correspond to different manipulations. We have shown how S-TEC's theoretical framework recovers non-contrastive learning and extends it with manipulation-relate inverse model, however experiments are left for future work.
6
Related work
SSL through auxiliary tasks. The idea of SSL by solving high-level queries about input manipula- tions was considered previously. E.g. Doersch et al. (2015); Noroozi and Favaro (2016) proposed to transform input images into patches and attempted to solve context prediction and jigsaw-puzzles respectively, while others found useful to predict a prior rotation transformation (Gidaris et al., 2018). In contrast to such spatial prediction tasks, different line of work by Zhang et al. (2016) discovered that colourization of boock--andwhitt images also creates useful features for downstream tasks. Since aforementioned auxiliary tasks are orthogonal at large, prior works studied combinations and/or extensions of those (Doersch and Zisserman, 2017; Zhang et al., 2019).
Contrastive SSL. Opposing to the preceding strategies of training on (handcrafted) auxiliary tasks algorithms that originated from the idea of mutual information (MI) maximization between the input and representations thereof. In particular, the prevalent strategy of mini-batch training rendered it practical to compare and contrast representations of different and related inputs, which enables the maximization of MI (Gutmann and Haränin,n 2010). Following this perspective, algorithms for deep networks were introduced (Oord et al., 2018), and further progress ensued, where the common blueprint for the algorithm is to transform input into pairs and to contrast those against other, unrelated ones (Henaff, 2020; He et al., 2020; Chen et al., 2020a,b, 2021). Tschannen et al. (2020) opened a discussion whether it works well due to maximization of MI, which resulted in the search for different explanations, e.g. through causal interventions (Mitrovic et al., 2021: Tomasev et al., 2022). On the other hand, the profound utility of contrastive SSL has also inspired other research that attempts to connect to hypothesized learning mechanisms in the brain. For instance, Illing et al. (2021) show that contrastive SSL can give rise deep representations with local learning rules.

---

Non-contrastive SSL. Several other studies explored non-contrastive avenues for SSL, which have gained more traction due their attractive properties, such as not needing negative examples. In general, these approaches require the network to produce consistent representations under content- preserving input transformations while addressing the problem of representational collapse (Grill et al., 2020; Chen and He, 2021; Zbontar et al., 2021; Bardes et al., 2022). Furthermore, Caron et al. (2020) demonstrate that enforcing consistency between cluster assignments can install potent feature extraction capabiliiies into a model, while also not requiring pair-wise contrasting.
Augmentation-awal self-suuervision. Algorithms based contrastive SSL typically aim for representations that are invariant to input transformations. While this seems appropriate in principle, subsequent studies have shown that this is not always favourable, this strategy can exclude certain information from representations that could otherwise make them more useful for varying downstream tasks (Xiao et al., 2021). Based on similar arguments, Lee et al. (2021) proposed to predict differences of certain transformation parameters in addition to training on a standard SSL objective (Chen et al., 2020c.a; Chen and He, 2021) to improve the transferability of learned representations to other domains. These are promising results; however, on the main performance tests of SSL., i.e. testing the representations in the same domain the training domain, these prior augmentation-aware approaches have not achieved the same performance advantage as compared to S-TEC, see Table 2.
The relation of our approach to prior work. Our approach generalizes methods that pair repre- sentations of paired inputs into framework that also introduces semantic structure between paired inputs, based on the known transformations between them. This generalized and unified framework emerges from the concept of ECs and its relation inverse models. Therefore, our approach is angmentation-wwrre but its foundation on sensory-motor principles and neuroscience instructs im- portant elements (Section 3) that are missing from earlier augmentation-awwan approaches (Xiao et al., 2021; Lee et al., 2021), but have been discussed analogously by Mineault et al. (2021), and studied in part by research on local learning (Illing et al., 2021). Currently, contrastive SSL is one of the dominant approaches in the literature, and our approach improves it (and can be combined with further improvements, e.g. ReLIC) in our tests (Tables and 3). Our theoretical framework also recovers and extends non-contrastive approaches, such as BYOL (Section 4.1).
Conclusion
S-TEC is a theoretical framework derived formally from first principles of biological sensory- control. It unifies and extends theoretically several SSL approaches, and improves them practically. Interestingly, designing S-TEC's details in a biologically-priccipeee way is crucial for performance. S-TEC is consistently better over several strong baselines in image classification, segmentation, and object detection. S-TEC as framework provides a new angle for future further improvements to SSL. By following established biological principles, S-TEC feeds back to neuroscience. Our results suggest that the availability of ECs to the nervous system for inverse-model learning may positively impact sensory skill. This hypothesis is testable. It predicts that subjects, exposed to motor learning task in a novel sensory environment through active movements, would perceive the new environment better than participants that only experience passive exploration of the environment. Supporting evidence from kittens and humans already exists (Held and Hein, 1963; Bach-y Rita, 1972). S-TEC's biological implications could be strengthened even within the computational setting, by using optimization algorithms that are more biologically plausible than backpropagation. Such options have recently been described, including within SSL (Illing et al., 2021). Adding further detail to S-TEC's ' neural networks, such as spiking neurons, could further enhance its biological relevance. Limitations. Even though our theoretical framework includes and extends various SSL methods, such as the very recent ReLICv2 (Tomasev et al., 2022), as well as non-contrastive SSL, e.g. BYOL (Grill et al., 2020), experimentally we have only extended the methods of Chen et al. (2020a); Mitrovic et al. (2021); Oord et al. (2018). In addition, we have employed only the basic commonly used augmentations for SSL without exploring other actions/augmmentation.. Moreover, we have experimented only with ResNets (He et al., 2016). Future research could test the advantage of S-TEC in other architectures, e.g. with self-attention (Vaswani et al., 2017).
Potential negative societal impacts. SSL can exploit unlabelled data, and S-TEC's rich feedback from ECs improves it, thus significantly expanding also the malicious applicability of MI. One concern is of privacy, i.c. S-TEC might assist the profiling of individuals from anonymized data.
10

---

Acknowledgments and Disclosure of Funding
This work is partially supported by the Science and Technology Innovation 2030-Major Project (Brain Science and Brain-Like Intelligence Technology) under Grant 2022ZD0208700, The authors would like to thank Lukas Cavigelli, Renzo Andri, Édouard Carré, and the rest of Huawei's Von Neumann Lab, for offering compute resources.
References
Bach-y Rita, P. (1972). Brain mechanisms in sensory substitution. Academic Press.
Balestriero, R. and LeCun, Y. (2022). Contrastive and non-contrastive self-supervised learning recover global and local spectral embedding methods. arXiv preprint arXiv:2205.11508.
Bardes, A., Ponce, J., and LeCun, (2022). VICReg: Varrinne-imariinneeovvvainee rew.:oooorrrinneeegggrrrrriaaaia regularization for self-supervised learning. In International Conference Learning Representations.
Bastos, A. M., Usrey, W. M., Adams, R. A., Mangun, G. R., Fries, P., and Friston, K. J. (2012).
Canonical microcircuits for predictive coding. Neuron, 76(4):695-711
Blakemore, S.-J., Wolpert, D. M., and Frith, tickle sensation. Nature neuroscience, 1(7):635-640.
(1998). Central cancellation of self-produced
Brownstone, R. M., Bui, T. V., and Stifani, N. (2015). Spinal circuits for motor learning. Current opinion neurobiology, 13:766677
Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., and Joulin, (2020). Unsupervised learning of visual features by contrasting cluster assignments. Advances in Neural Information Processing Systems, 33:9912-999 1299224.
Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. (2020a). A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597- 1607. PMLR.
Chen, T., Kornblith, S., Swersky, K., Norouzi, M., and Hinton, (2020b). Big self-supe models are strong semi-supervised learners. Advances in neural information processing systems, 33:22243-22255.
Chen, T., Luo, C., and Li, (2021). Intriguing properties of contrastive losses. Advances Neural Information Processing Systems, 34.
Chen, X., Fan, H., Girshick, R., and He, K. (2020c). Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297.
Chen, X. and He, K. (2021). Exploring simple siamese representation learning- In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 15750-15758.
Coates, A., Ng, A., and Lee, H. (2011). An Analysis of Single Layer Networks in Unsupervised Fea- ture Learning. In AISTATS. https://css sttps://ca.staaford.eee stw.ford. edu/ aistats 2011.pdf.
Diedrichsen, J., Verstynen, T., Hon, A., Lehman, S. L.,, and Ivry, R. B. (2003). Anticipatory adjustments the unloading task: an efference copy necessary for learning? Experimental Brain Research, 148(2):272-276.
Doersch, C., Gupta, A., and Efros, A. A. (2015). Unsupervised visual representation learning by context prediction. Proceedings of the IEEE international conference on computer vision, pages 1422-1430.
Doersch, C. and Zisserman. A. (2017). Multi-task self-supervised visual learning, In Proceedings of the IEEE International Conference on Computer Vision, pages 2051-2060.
Douglas, J., Martin, A., and Whitteridge, D. (1989). canonical microcircuit for neocortex. Neural computation, 1(4):480-488

---

Engert, F. (2013). Fish in the matrix: motor learning in a virtual world. Frontiers in neural circuits, 6:125.
Everingham, M., Van Gool, L., Williams, C. K., Winn, J., and Zisserman, (2010). The pascal visual object classes (voc) challenge. International journal of computer vision, 88(2):303-33
Gidaris, S., Singh, P., and Komodakis, N. (2018). Unsupervised representation learning by predicting image rotations. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings. OpenReview.net.
Graziano, M. S. (2016). Ethological action maps: paradigm shift for the motor cortex. Trends in cognitive sciences, 20(2):221-132.
Grill, J.-B., Strub, F., Altché, F., Tallec, C., Richemond, P., Buchatskaya, E., Doersch, C., Avila Pires, B., Guo, Z., Gheshlaghi Azar, M., et al. (2020). Bootstrap your own latent-a new approach to self-supervised learning. Advances in Neural Information Processing Systems, 33:21271-211
Gutmann, M. and Hyvärinen, A. (2010). contrastve estimation: A new estimation principle for unnormalized statistical models. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pages 297-304. JMLR Workshop and Conference Proceedings.
He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R. (2020). Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9729-9738.
He, K., Zhang, X., Ren, S., and Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778.
Held, and Hein, A. (1963). Movement- dlovement-producedd stimulation in the development of visually guided behavior. Journal of comparative and physiological psychology, 56(5):872.
Henaff, O. (2020). Data-efficient image recognition with contrastive predictive coding. In Interna- lional Conference on Machine Learning, pages 4182-4192. PMLR.
Illing, B., Ventura, J., Bellec, G., and Gerstner, W. (2021). Local plasticity rules can learn deep representations using self-supervised contrastive predictions. Advances in Neural Information Processing Systems, 34.
Journé, A., Rodriguez, H. G., Guo, Q., and Moraitis, (2022). Hebbian deep learning without feedback. arXiv preprint
Kawato, M. (1999). Internal models for motor control and trajectory planning. Current opinion in neurobiology, 06):718-727
Kennedy, A., Wayne, G., Kaifosh, P., Alviña, K., Abbott, L., and Sawtell, N. B. (2014). A temporal basis for predicting the sensory consequences of motor commands in electric fish. Nature neuroscience, 17634:416-422.
Kilteni, K., Engeler, P., and Ehrsson, H. H. (2020). Efference copy is necessary for the attenuation of self-generated touch. IScience, 2302):100843.
Krizhevsky, A., Hinton, G., et al. (2009). Learning multiple layers of features from tiny images.
Latash, M. L. (2021). Efference copy kinesthetic perception: copy of what it? Journal of neurophysiology, 12594):1079-1094.
Lee, H., Lee, K., Lee, K., Lee, H., and Shin, J. (2021). Improving transferability of representations via augmentation aware aware self-supervision. Advances in Neural Information Processing Systems, 34.
Long, J., Shelhamer, E., and Darrell, T. (2015). Fully convolutional networks for semantic segmenta- tion. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3441-3440.
McInnes, I.., Healy, J., and Melville, J. (2018). Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv: 1802.0346.
12

---

McNamee, D. and Wolpert, D. M. (2019). Internal models in biological control. Annual review of control, roboties, and autonomous systems, 2:339-364.
Mineault, P., Bakhtiari, S., Richards, B., and Pack, (2021). Your head is there to move you around: Goal-driven models of the primate dorsal pathway. Advances in Neural Information Processing Systems, 34:28757-28771.
Mitrovic, J., McWilliams, B., Walker, J. C.,, Buesing, L. H., and Blundell, C. (2021). Represen- tation learning via invariant causal mechanisms. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net.
Moraitis, T., Toichkin, D., Journé, A., Chua, Y., and Guo, Q. (2022). Softhebb: Bayesian inference in unsupervised hebbian soft winner-take-al networks. Neuromorphic Computing and Engineering. 2(4):044017.
Noroozi, M. and Favaro, (2016). Unsupervised learning of visual representations by solving jigsaw puzzles. In European conference computer vision, pages 69-84. Springer.
Oord, A. v. d., Li, Y., and Vinyals, O. (2018). Representation learning with contrastive predictive coding. arXiv preprint urXivv1807.0377
Ren, S., He, K., Girshick, R., and Sun, (2015). Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems, 28.
Richards, B. A., Lillicrap, T. P., Beaudoin, P.. Bengio, Y., Bogaez, R., Christensen, A., Clopath, C., Costa, R. P., de Berker, A., Ganguli, S., al. (2019). deep learning framework for neuroscience. Nature neuroscience, 22(11) 22(11):17611 17111:1761-1770
Rizzolatti, G., Luppino, G., and Matelli, M. (1998). The organization of the cortical motor system: new concepts. Eleetroencephaloggupyy and clinical neurophysiology. 106(4):283-296.
Russakovsky, O., Deng. J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla,
A., Bernstein, M., Berg, A. C., and Fei-Fei, L. (2015). ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 115(3):211-252.
Sperry, R. W. (1950). Neural basis of the spontaneous optokinetic response produced by visual inversion. Journal of comparative and physiological psychology, 43(6)4822
Tomasev, N., Bica, I. McWilliams, B., Buesing, L., Pascanu, R., Blundell, C., and Mitrovic, J. (2022). Pushing the limits of self-supervised resnets: Can we outperform supervised learning without labels on imagenet? arXiv preprint arXiv:2201.05119.
Troyer, T. W. and Doupe, A. J. (2000). An associational model of birdsong sensorimotor learning i. efference copy and the learning of song syllables. Journal of neurophysiology. 84(3):200-11223
Tschannen, M., Djolonga, J., Rubenstein, K., Gelly, S., and Lucie, M. (2020). On mutual information maximization for representation learning. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L.., Gomez, A. N., Kaiser, E.., and Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.
von Helmholtz, H. (1867). Handbuch der physiologischen Optik: mit 213 in den Text eingedruckter Holzschmitten und 11 Tafeln, volume 9, Voss.
von Uckermann, G., Le Ray, D., Combes, D., Straka, H., and Simmers, J. (2013), Spinal efference copy signaling and gaze stabilization during locomotion in juvenile xenopus frogs. Journal of Neuroscience, 33 33(10):4253-4264
Wang, F. and Liu, H. (2021). Understanding the behaviour of contrastive loss. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2495-2504.
Witney, A. G., Goodbody, S. J., and Wolpert, M. (1999). Predictive motor learning of temporal delays. Journal of neurophysiology, 82(5):2099-
13

---

Xiao, TT, Wang. X., Efros, A. A., and Darrell, T. (2021). What should not be contrastive in contrastive learning. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net.
You, Y., Gitman, I., and Ginsburg, B. (2017). Large batch training of convolutional networks, arXiv preprint arvi: 1w0.vv::000.000 770.00388...
Zbontar, J., Jing. L.. Misra, I., LeCun, Y., and Deny, S. (2021). Barlow twins: Self-supervised learning via redundancy reduction. In International Conference on Machine Learning, pages 12310-12320 PMLR.
Zhang. L., Qi, G.-J., Wang, L., and Luo. J. (2019). Act vs. aed: Unsupervised representation learning by auto-encoding transformations rather than data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2547-2555.
Zhang, R., Isola, P., and Efros, (2016). Colorful image colorization. European conference on computer vision, pages 649-666. Springer.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [Yes]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes] Will be in the supplemental material
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)? [Yes] Will be the supplemental material
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [No]
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [N/A]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you'l 're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsoureing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
14

---

(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
15


Reviews:
[{"review_id": "vv7TbWcu9i0", "reviewer": "Reviewer_wiNh", "summary": "Inspired by the concept of Efference Copy (EC), the authors propose a new self-supervised learning (SSL) framework, called Self-supervision Through Efference Copies (S-TEC). Specifically, the authors focus on inverse models of EC, which map sensory inputs to motor outputs, and make the analogy between EC and SSL by considering data augmentation as actions. The inverse models are implemented by the identity-related inverse model to determine whether two inputs are the same object or not, and the manipulation inverse model to figure out which augmentation is applied. Through the experiments, the authors demonstrate that S-TEC outperforms the reference algorithms including SimCLR and AugSelf on the image classification tasks, and analyze why S-TEC works well in a qualitative manner.", "questions": "- Please give more details about the egocentric representation of actions in lines 140-151.\n- Please clarify the difference with [1] in the experiments. For now, the comparison is done on only STL-10 and the number of training epochs for S-TEC is larger than [1], which makes me suspicious about whether the performance improvement comes from S-TEC or not. If the main difference between S-TEC and [1] is the egocentric representation of actions, please conduct an ablation study of S-TEC by replacing the egocentric representation of actions with the allocentric one as [1].\n- Please conduct ablation studies about the manipulated-related inversed model, such as varying the number of clusters.\n- Please conduct additional experiments on various tasks other than image classification.", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Originality: The motivation of this paper that brings the concept of EC into SSL sounds interesting. However, the novelty is a bit limited because the authors do not consider forward models that are critical parts of EC in S-TEC and it is difficult to see the significant differences from previous studies. The loss function of the identity-related inverse model is almost identical to a conventional contrastive loss used in SSL literature and the loss function of the manipulated-related inversed model is similar to [1].\n\nQuality: The authors' claims are not supported well. First, the comparison with [1] is not enough. Since both the proposed method and [1] are motivated by augmentation-aware SSL, the two methods should be compared intensively. However, the authors compare the two methods on only STL-10 and the experiment setups are not identical. Second, the ablation study is lacking completely. For example, it would be great to see the effect of the number of clusters for the manipulation-related inverse model. Finally, the experiments are done on only image classification tasks. Considering the purpose of SSL is to learn general features for various downstream tasks, the authors should conduct more experiments on various tasks, such as object detection. \n\nClarity: The paper is generally well-written and easy to follow. However, some important details are in the supplemental material not in the main text.\n\nSignificance: At this stage, the significance of this paper is limited because it is not clear what are the contributions compared with previous work and the experimental results are not enough to support the authors' claims as I wrote above.\n\nReference\\\n[1] Lee, H., Lee, K., Lee, K., Lee, H., and Shin, J. (2021). Improving transferability of representations via augmentation-aware self-supervision. Advances in Neural Information Processing Systems, 34.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "h96atdLhq4J", "reviewer": "Reviewer_fHUb", "summary": "Author propose a new neuroscience-inspired self-supervised learning process through adopting processes of the brain for developing a new framework. Specifically the use the concept of efference copy to derive a new loss function in a contrastive setting that can be used to perform self-supervised learning. They demonstrated the applicability of this method on a number of different datasets on image classification. There was a modest increase in performance when compared against some of the earlier work on contrastive SSL, i.e. SimCLR, which is actually the biggest limitation of this work. Nevertheless that is an interesting piece of work that is among the 1-2 other papers that have approached this problem from a more complex cognitive perspective.", "questions": "A) Did you consider comparing with other methods beyond SimCLR and Relic? It would have been positive irrespective of absolute performance\nB) When comparing with methods that provided results in 200 epochs (arguably very few labs can train for longer than 200/300 epochs, as usually methods that present 1000 epochs on image net for instance are large lab and companies **usually**), one should compare their own method on 200 epochs rather than comparing the 1000 vs 200.\nC) More of theoretical question, if this method can barely outperform SimCLR, what further improvements adopted from neuroscience could perhaps lead to further improvements that can compare with say VicReg?", "limitations": "I think the results section is a clear limitation and someone reading this paper will be confused as to what the current SOTA is. Contrastive methods have seen better days and we can safely say that most work now is non-contrastive - this has not been adequately articulated in the paper and further information should have been provided on other methods such as regularisation and VicReg, SWAV, etc.\n\nThe paper would have been extremely interesting 2 years ago, but as it is a different way of thinking I do not necessarily see this as a limitation, given that advances are not linear across sub-fields. What I mean by that is that it could be that in a year from now a neuroscience-inspired methods built upon this one could outperform SOTA in SSL. I just think that all these should have been covered in the main paper, and also not use the supplementary material to add so much information that 10-15% of which would fit best in the main paper. I literally looked at the supplementary material to get some essential information whereby essential information should be covered in the main paper, and leave additional context for the supplementary material.\n\n", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths.\n\nA) the paper is well founded and written. Although I am not a neuroscientist, I appreciate that the formulations used are inspired by how our central nervous system sends stimuli to our motor neuron system to perform an action, whilst also sending a signal to itself. That is grounded to the fact that our central nervous systems learns primarily without external supervision.\nB) There is a parallel presented around how to recover SSL methods such as contrastive learning by inverse models for efference copy using first principles. \n\nWeaknesses.\n\nA) Although it seems to be foundationally grounded, it mainly focuses on comparisons with “very early” (oxymoron I know) work on contrastive learning (SimCLR), which I dare say that is not that relevant anymore, and is mainly used as a baseline when comapred with other approaches. It is common to compare with SimCLR but when including BYOL, SWAV, MoCo, etc. I think the authors should have added comparisons with all these approaches, even if this methods did not outperform them. I am not taking the side at all on “SOTA” or “nothing”   -  I mean that from the point of of completeness of the experimental design and rigourous representation around how this method compares with others.\nB) Figure 1 is incomplete and notations are inconsistent but that has been acknowledged in the supplementary material and the authors have committed to fix these issues in the camera-ready version.\nC) Presentation could have been stronger, i.e. additional figures could have been provided on how the model trains, loss function, etc. over those 1000 epochs. There is limited information in general on the results part.\nD) Conclusion is rather weak, not only because it does not reflect on the neuroscience side more than it did, but it rather repeats and points to sections to draw conclusions. It is more like a summary/overview than a conclusion", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "fp4w2WmuwdQ", "reviewer": "Reviewer_3AEo", "summary": "The authors proposed to generalize the recent self-supervised learning (SSL) methods with the Efference Copy (EC) concept in neuroscience. The proposed principles of EC lead to another implementation/variant of the SSL method which is named S-TEC, which has a more generalizable loss function and richer feedback. The new model S-TEC is benchmarked with other SSL methods, where the authors showed that it leads to improved performance overall. The authors also conducted additional latent space analysis that shows S-TEC tends to provide more separated feature space.", "questions": "1. As currently there are more qualitative visualizations of the latent space, it would be really interesting if more quantitative analysis could be conducted to investigate how the latent space structure is affected (e.g. how strongly augmented images distribute compared to the others).\n\n2. The mathematical notations in the Formalism section could be more self-explainable without referring to the appendix. For example, line 181 'φ compares the outputs of f', it is unclear which type of function is used and how the outputs are compared. \n\n3. The related work section could be put after the Analysis section for better readability.\n\n4. The authors could include the performance trajectory of S-TEC along the training epochs to show if the model learns faster when compared to other models. \n\n5. The authors could also include ablation experiments based on different augmentations to show if the model learns better when fewer/weaker augmentations are used.", "limitations": "The authors addressed some limitations of the work. Based on the suggestions and questions above, more limitations could be included after the rebuttal process.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strength**:\n\t\n1. The proposed methodology/assumption is interesting and novel.\n- The authors made the theoretical connection between the EC and the SSL frameworks, which is particularly a suitable insight for the Neurips audience.\n- The resulting formulation of the loss function is novel and general, which indeed provides new insights into the SSL domain.\n\n2. The experiments seem to be nicely carried out.\n- Although there are weaknesses in the experiment sections, I would like to point out that the experiments overall are nicely conducted. The method is tested on the major datasets that are used to evaluate SSL methods, ResNet-18 and ResNet-50 are both used, and the results are compared with a few major benchmark models. More could be done, but the overall experiments seem relatively convincing (under the assumption that the authors do not have an extensive amount of compute to use).\n- The latent space analysis is interesting and insightful.\n\n3. The paper is clear and nicely written.\n\n**Weakness**:\n\n1. Methodology: while the neuroscience connections give additional insights into the SSL framework, the actual implementation of the loss function ends up having many similarities/connections with previous works. Although the ‘The relation of our approach to prior work’ partly addressed the issue, it would be great if the authors could specifically remark how their method could or could not be reduced to other SSL frameworks.\n\n2. Experiments could be more extensive:\n- More models could be benchmarked, especially widely used ones such as MoCo and BYOL. \n- ImageNet results seem to be lower than the published performance of the SimCLR (69.3% as in their paper, with ResNet-50). The details of this should be more carefully addressed.\n- The 200 epoch results of SimSiam are used to compare with the 1,000 epoch results of SimCLR and S-TEC, which is biased and should be replaced with a fair comparison.\n\n3. Others:\n- All figure captions (especially Figure 1 and 2) could be significantly improved. They are not self-explainable enough without carefully reading the main text.\n- The definition of the regularization loss was put into the Appendix, while it might be important to put some definition in the main text to help the readers better understand how it would affect the performance. Also, there should be some ablation experiments to evaluate how the weights of the regularization would affect the model performance.\n- Line 341-342 appeared twice (also in line 293-294): ‘Moreover, we also confirmed that the benefits of our approach transfer to other, orthogonal improvements, such as provided by ReLIC, see Table 3.’\n\nIf the above weaknesses could be addressed, especially the weaknesses regarding the experiments, I am willing to raise the score.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "0bnMu-aTTn", "reviewer": "Reviewer_bN43", "summary": "The paper states a broad framework for learning in a setting where actions lead to changes to the environment/input, via the (neuroscience-inspired) notion of an efference copy, in which the agent gets a copy of the action (\"motor command\") in addition to sending it off to the motor system. Using a vision setting, and by setting up two simple actions, \"look at the same object or not\" and \"change the view (affine transformation) of the object\", the paper derives a loss function which is identical to a typical contrastive learning objective, plus a term that handles the case of change of view, for which they use data augmentation. The paper shows that this framework yields representations that are better for linearly classifying in several visual tasks.", "questions": "Can you explain concisely why the notion of efference copies is particularly valuable for this example or for other possible examples?", "limitations": "The framework presented is so general that it is a bit tricky to think of ethical implications, but given that the paper rotates around one specific example, I would have suggested the authors dig a little more into the object or visual classification example to see what negative impacts they can think of of either better features, or perhaps the specific model described here (e.g. would it be possible to use this model to get better predictions of held-out views of objects or faces?)", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\nThis framework is very interesting, and raises lots of obvious questions about what would happen if one were to change the set of actions. In particular, I am curious how this could be applied to speech. Typical approaches are not merely contrastive, but predictive, i.e. they have a temporal dimension (wav2vec2, CPC), so there would perhaps be some way of modifying the current approach, or perhaps a more radical approach in the same framework could be developed that takes an analysis-by-synthesis angle.\n\nThe results obtained are somewhat better than the reference systems.\n\nWeaknesses\n\nI think the notion of efference copy is not really doing anything here, aside from re-framing the problem, given that we are assuming that the efference copy is always exactly accurate. As such, given that there is only one example given (the specific cashing-out of the framework for vision), and given that there are no ablation studies or error bars, it is not so clear to me what we are supposed to get out of the result. Is the notion of efference copy concretely useful to improve performance? Or is it merely the combination of contrastive learning with data augmentation? Or is it not either which is in fact improving the result over previous scores (perhaps implementation differences, random initialization, etc)? In other words, to clearly establish the value of this new framework, I would expect one of two things: more, diverse examples; or a detailed and careful analysis of one example. The discussion around Figure 3 would imply that the authors are suggesting that it is the combination of contrastive learning with data augmentation which is helping, but that is not really a property of the general framework from what I can see, but rather just of this specific example and the actions that the authors have chosen. Maybe this could be okay, if the paper were reframed as being less about the efference copy framework (which doesn't seem to have any specific value here) and more about combining contrastive SSL with augmentation, but by itself that seems like a small extension with not such big empirical improvements.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept