High Fidelity Visualization of What Your Self-Supervised Representation Knows About
Authors: Florian Bordes
Randall Balestriero
Pascal Vincent
Venue: Accepted by TMLR ()

Abstract: Discovering what is learned by neural networks remains a challenge. In self-supervised learning, classification is the most common task used to evaluate how good a representation is. However, relying only on such downstream task can limit our understanding of what information is retained in the representation of a given input. In this work, we showcase the use of a Representation Conditional Diffusion Model (RCDM) to visualize in data space the representations learned by self-supervised models. The use of RCDM is motivated by its ability to generate high-quality samples ---on par with state-of-the-art generative models--- while ensuring that the representations of those samples are faithful i.e. close to the one used for conditioning.
By using RCDM to analyze self-supervised models, we are able to clearly show visually that i) SSL (backbone) representation are not invariant to the data augmentations they were trained with -- thus debunking an often restated but mistaken belief; ii) SSL post-projector embeddings appear indeed invariant to these data augmentation, along with many other data symmetries; iii) SSL representations appear more robust to small adversarial perturbation of their inputs than representations trained in a supervised manner; and iv) that SSL-trained representations exhibit an inherent structure that can be explored thanks to RCDM visualization and enables image manipulation.

Published in Transactions on Machine Learning Research (08/2022)
High Fidelity Visualization of What Your Self-Supervised Representation Knows About
Florian Bordes
Meta AT Research
Mila, Université Montréal
Randall Balcstriero
Mela AI Research
Pascal Vincent
Meta Al Research
Mila, Université de Montréal
Canada CIFAR AI Chair
Reviewed on OpenReview: https: /ttps://openreeee opw.reveee.........mm net/ forum? id- wrd
Abstract
Discovering what is learned by neural networks remains challenge. Tn self-supervised learning, classification is the most common task used to evaluate how good a representation is. However, relying only on such downstream task can limit our understanding of what information retained the representation of given input. this work, we showcase the of Representation Conditional Diffusion Model (RCDM) visualize in data space the representations learned by self-supervised models. The use of RCDM is motivated by its ability to generate high-quality samples while ensuring that the representations of those samples are faithful i.c. close to the one used for conditioning. By using RCDM to analyze self-supervised models, we are able to clearly show visually that i) SSL (backbone) representation are not invariant to the data augmentations they were trained with thus debunking an often restated but mistaken belief; ii) SSL post-projector emboddings appear indced invariant to these data augmentation, along with many other data symmetries; iii) SSL representations appear more robust to small adversarial perturbation of their inputs than representations trained in supervised manner; and iv) that SSL-trained representations exhibit an inherent structure that can be explored thanks to RCDM visualization and enables image manipulation. Code and trained models are available at http://hhhb..caccccaoeeeeeeaaaa RRCDM
par with state-of-the-art generative models
Earth from space
untrained representation
supervised representation
SSL representation
We representations of the real picture of Earth the left (source: NASA) conditioning for RCDM. We show samples (resolution 256 256) cases where the representations (2048-dimensions) were obtained respectively with random initialized 50,, supervised-traiiee one, and SStntrained one. More samples in Fig. 36

---

Published in Transactions on Machine Learning Research (08/2022)
1 Introduction
Approaches aimed at learning useful representations, from unlabeled data, have long tradition in machine
learning. These include probabilistic latent variable models and variants of auto-encoders (Acklyy et al.)
1985
Hinton et al.] 2006 Salakhutdinov et al., 2007 Vincent et al., 2008 Kingma & Welling 2014 Rezende
ct al.
2014), that are traditionally put under the broad umbrella term of unsupervised learning (Bengio et al.
2013
More recent approaches, under the term of self-superuised learning (SSL) have used various kinds of
pretext-lasks" to guide the learning of a useful representations. Filling-in-the-llanke Lasks, proposed earlier in
Vincent et al.) 2008 2010), later proved remarkably successful in learning potent representations for natural
language processing Vaswani et al. 2017 Devlin et al. 2019p. Pretext tasks for the image domain include
solving Jigsaw-puzzles (Noroozi & Favaro 2016), predicting rotations or affine transformations (Gidaris et al.
2018
Zhang et al. 2019b) or discriminating instances (Wu et al.) 2018 van den Oord et al.
2018). The
latest, most successful, modern family of SSL approaches for images (Misra & Maaten 2020
Chen et al.
2020
Chen & He 2020 He et al., 20200 Grill et al., 2020. Caron et al. 2020 2021; Zbontar et al. 2021
Bardes et al. 2021 have two noteworthy characteristics that markedly distinguish them from traditional
models such autoencoder variants GANs (Goodfellow et al. 2014): a) their training criteria are not based on any inpul-space reconstruction or generation, but instead depend only on the obtained distribution in the representation or embedding space b) they encourage invariance to explicitly provided input transformations a.k.a. data-angmentations, thus injecting important additional domain knowledge.
unsupervised-lcariie
Despite their remarkable success in learning representations that perform well on downstream classification tasks, rivaling with supervised-trained models (Chen et al., 2020), much remains to be understood about SSL algorithms and the representations they learn. How do the particularities of different algorithms affect the representation learned and its usefulness? What information does the learned representation contain? Answering this question is the main focus of our work. Since SSL methods mostly rely on learning invariances to a specific set of handerafted data augmentations, being able to evaluate these invariances will provide insight. into how successful the training of the SST. criteria. has been. It is also worth to be noted that recent. SSL methods use a projector (usually small MLP) on top of a backbone network (resnet50 or vit) during training, where the projector is usually discarded when using the model on downstream tasks. This projector trick is essential to get competitive performance on ImageNet i. we often observe a performance boost of 10 to 30 percentage accuracy point when using the representation at the backbone level instead of the ones at the projector level on which SSL criteria are applied during training. Since SSL criteria are not applied at the backbone level, there remains a mystery regarding what the backbone does learn that make it better for classification than the projector. This is another question that we will be able to answer in this study.
Empirical analyses have so far attempted to analyse SSL algorithms almost exclusively through the limited lens of the numerical performance they achieve on downstream tasks such as classification. Contrary to their older unsupercised learning cousins, modern SSL methods do not provide any direct way of mapping back the representation in image space, to allow visualizing it. The main gool of our work is thas to enable the visualization of representations learned by SSL methods, as a tool to improve our understanding.
In our approach (Section 3), we propose to generate samples conditioned on a representation such that (i) the representation of those samples match the one used for conditioning, and (ii) the visual quality of the sample is as high as possible to maximize the preciseness of our visual understanding. For this, we employ a conditional generative model that (implicitly) models. For reasons that we will explain later, we opted for conditional diffusson model, inspired by Dhariwal & Nichol (0211.
This paper's main contributions are:
To demonstrate how recent diffusion models are suitable for conditioning on large vector representations such as SSL representations. The conditionally generated images, in addition to being high-quality are also highly representation-fatthee
i.c. they get encoded into a represontation that closely matches the representation of the images used for the conditioning (Tab. [b] Fig. 23).
To showcase its potential usefulness for qualitatively analyzing SSL representations and embeddings (also in contrast with supervised representations) by shedding light on what information about the input image is or isn't retained in them.
2

---

Published in Transactions on Machine Learning Research (08/2022)
Specifically, by repeatedly sampling from a same conditioning representation, one can observe which aspects are common to all samples, thus identifying what is encoded in the representation, while the aspects that vary greatly show what was not retained in the representation. We make the following observations: (i) SSL projector embeddings appear most invariant, followed by supervised-tree -traiied representation and last SSL representat J (Fig. 2 (ii) SST.-trained representations retain more detailed information on the content of the background and object style while supervised-traiie -raiind representations appear oblivious to these (Fig. 3
(iii) despite the invariant training criteria, SSL representations appear to retain information on object scale, grayscale color, and color palette of the background, much like supervised representation (Fig. 3
(iv)
Supervised representations appear more susceptible to adversarial attacks than SSL ones (Fig. We can explore and exploit structure inside SSL representations leading to meaningful manipulation of image content (such as splitting representation in foreground/ /bacckrroud components to allow background substitution) (Fig, 30 31.
(v)
2 Related Work
Many works (Erhan et al. 2009 Zeiler & Pergus 2014 Simonyan et al. 2013 Mahendran & Vedaldi 2015 Selvaraju et al. 2016 Smilkov et al. 2017 Olah et al. 2017) used gradient-bant techniques to visualize what is learned by neural networks. Some of them maximize the activation of specific neuron to visualize what is learned by this neuron, others offer visualization of what is learned at different layers by trying to 'invert' neural networks. All of these use some form of regularization, coist or prior to guide the optimization process towards realistic images. Dosovitskiy & Brox (2016) learn to map back a representation to the input space by using a Generative Adversarial Networks (Goodfellow ct al., 2014) which is trained to reconstruct an input given a representation. Since the mapping is deterministic, they obtain only single image with respect to specific conditioning. In contrast, we use stochastic mapping that allows to visualize the diversity of the images associated to a specific representation. Nguyen et al. (2016) also use GANs but instead of trying to invert the entire vector of representation, they try to find which images (by using an optimization process in the latent space of the generator) maximize a specific neuron. The following work (Nguyen et al. 2017) demonstrates how using this conditional itcrative optimization in the latent space of the generator lead to high quality conditional image generalion. Finally Lučié et al. (2019) leverage SSL methods to create discrete cluster that are used as conditioning for a GAN. Our work focuses only on continuous representation vectors. More recently, Caron et al. (2021) used the attention mask of transformers to perform unsupervised object segmentation. By contrast, our method is not model dependent, can plug any type of representation as conditioning for the diffusion model. Another possibility, explored in ZZho et al. (2021); Appalaraju et al.] (2020); Ericsson et al. (2021), is to learn to invert the DN features through a Deep Image Prior (DIP) model. In short, given a mapping f that produces a representation of interest, the DIP model go learns ming (( ([())), æ). However, as we will demonstrate in Figure 7 this solution not only requires to solve an optimization problem for each generated sample but also leads to low-quality generation.
Visualization methods:
Generative models: Several families of techniques have been developed as generative models, that can be trained on unlabeled data and then employed to generate images. These include reto-segressive models (Van Den Oord et al., 2016)) variational auto-encoders (Kingma & Welling 2014 Rezende et al.) 2014), GANs (Goodfellow et al.; 2014), autoregressive flow models (Kingma et al., 2016), and diffusion models (Sobl-Dickstein et al. 2015). Conditional versions are typically developed shortly after their unconditional versions MMizaa & Osindero 2014 van den Oord et al.) 2016). In principle one could envision training a conditional model with any of these techniques, to condition on an SSL or other representation for visualization purpose, as we are doing in this paper with a diffusson model. One fundamental challenge when conditioning on a rich representation such as the one produced by a SSL model, is that for a given conditioning h we will usually have available only a single corresponding input instance x, By contrast a particularly successful model such as the conditional version of BigGAN (Brock et al. 2019) conditions on categorical variable, the class label, that for each value of the conditioning has a large number of associated data. One closcly related work to ours is the recent work on Instance- Conditioned GANs (IC-GAN) of Casanova et al. (2021] Similar to us it also uses SSL or supervised representations as conditioning when training a conditional
2Th representation that is produced by Resnet50 backbone, before the projector.
3

---

Published in Transactions on Machine Learning Research (08/2022)
generative model, here GAN (Goodfellow et al. 2014 specifically variant of BigGAN (Brock et al. 2019
or StyleGAN2 (Karras et al. 20200. However, the model is trained such that, from a specific representation, it learns to generate not only images that should map to this representation, but a much broader neighborhood of the training data. Specifically up to 50 training points that are its nearest neighbors in representation space. It remains to be seen whether such a GAN architecture could be trained successfully without resorting to nearest neighbor set. IC-GAN is to be understood as conditional generative model of an image's broad neighborhood, and the primary focus of the work was on developing a superior quality controllable generative model. By contrast we want to sample images that map closely as possible to the original image in the representation space, as our focus is to build a tool to analyse SSL representations, to enable visualising what images correspond precisely to a representation. (See Fig. 23 for comparison.) Lastly, a few approaches have locused on conditional generation to unravel the information encoded in representations of supervised models. In Shocher et al. (2020), a hierarchical LSGAN generator is trained with a class- conditiona discriminator Zhang ct al. 2019a). While the main applications focused on inpainting and style-transfer, this allowod to visually quantily the increasing invariance of representations associated to deeper and deeper layers. This method however requires labels to train the generator. On the other hand, Nash et al. (2019) proposed to use an autoregressive model, in particular PixeICNN++ (Salimans et al., 2017), to specifically study the invariances that each layer of a DN inherits. In that case, the conditioning was incorporated by regressing a context vector to the generator biases. As far as we are aware, PixeICNN++ generator falls short on high-resolution images e.g. most papers focus on 32 x 32 Imagenet. Lastly, Rombach et al. (2020) proposes to learn a Variational Auto Encoder (VAE) that is combined with an invertible neural network (INN) whose role is to model the relation between the VAE latent space and the given representations. To allow for interpretable manipulation, second invertible network (Esser et al. 2020)) is trained using labels to disentangle the factors of variations present in the representation. By contrast we train end-to-end single decoder to model the entire diversity of imputs that correspond to the conditioning representation, without imposing constraints of a structured prior or requiring labels for image manipulation.
3 High-Fidelity Conditioning with Diffusion Models
We base our work on the Ablated Diffusion Model (ADM) developed by Dhariwal & Nichol 2021 which uses a UNet architecture (Ronneberger et al. 2015) to learn the reverse diffusion process. Our conditional variant - called Representation- CnarrsennttiionCCCnddtttnnne Diffusion Model (RCDM) iss illustrated in Fig. 6b To suitably condition on representation h (xz), we followed the technique used by Casanova al. 2021 for IC-GAN which rely conditional batch normalization (Dumoulin et al. 2017). More preciscly, we replaced the Group Normalization layers of ADM by conditional batch normalization layers that take h as conditioning. We also apply a fully connected layer to h that reduces dimension to a vector of size 512. This vector is then given as input to multiple conditional batch normalization layers that are placed in each residual block of the diffusson model. An alternative conditioning method is to use the original conditioning method built inside ADM, and replace the embedding vector used for class labels by a linear layer that reduces dimension to a vector of the size of the time steps embedding. We didn't 't observe any differences in term of experimental results between those two methods presented in Fig. 10 In this paper, we used the first conditioning method for most of the experiments in order to make a proper comparison with IC-GAN.
In contrast with Dhariwal & Nichol (2021) we don't use the input gradient of a classifier to bias the reversed diffusion process towards more probable images -aunsafeer-guddanee)) nor do we use any label information for training our model - recall that our goal is building a visualization tool for SST models that train on unlabelod data. Another drawback of classifier-guddace
the need to retrain a classifier on inputs generated by the diffusion process. Since training SSL models can be very costly, retraining them was not an option, thus we had to find a method that could use directly the representation of pretrained model.
Our first experiments aim at evaluating the abilitics of our model to generate realistic-looking images whose representations are close to the conditioning. To do so, we trained our Cepresentation-Connitttoo Model (RCDM), conditioned on the 2048 dimensional representation given by a Resnet50 (He et al. 2016 trained with Dino (Caron et al.] (021) on ImageNet (Russakovsky et al.] 2015).. Then we compute the representations of a set of images from ImageNet validation data to condition the sampling from the trained RCDM. Fig. Ta shows it is able to sample images that are very close visually from the one that is used to get.
Diffusion
4

---

Published in Transactions on Machine Learning Research (08/2022)
(b)
Figure 1: a. In-distribution conditional image generation. An image from ImageNet validation set (first column) is used to compute the representation output by a trained SSL model (Dino backbone). The representation used as conditioning for the diffusion model. Resulting samples that our conditional diffusion model produces samples that are very close to the original image. b) Out of distribution (OOD) conditioning. How well does RCDM generalize when conditioned different distribution? (here a WikiMedia Commons image, see Fig. 9 for more). Even with an OOD conditioning, the images produced by RCDM match some characteristics of the original image (which highlights that RCDM not merely overfitling linear interpolation between the SSL representation of images the first column and the representation of the images in the last column. We use the interpolated vector conditioning for our model, that produces the samples that are showed columns 6. Fig. 15 appendix shows more sampled interpolation paths.
shown in the subsequent columns (see Fig. R We observe
representations given by images from
ImageNet).
Interpolation between two images from ImageNet validation data. We apply
the conditioning. We also evaluated the generation abilities our model on out of distribution data. Fig. |b shows that our model able to sample new views of an OOD image. We also quant itatively validate that the generated images' representations are close to the original image representation in Tab. [b] Fig. [7 Fig. [8]
and Fig. 19 We do so by verifying that the representation used conditioning is the nearest neighbor of the representation of its generated sample.
This implies that there is much information kept inside the SSL representation so that the conditional generative model is able to reconstruct many characteristics of the original image. We also perform interpolations between two SST representations in Fig. Te This shows that our model able produce interpretable images even for SSL representations that correspond an unlikely mix of factors. Both the interpolation and OOD generation clearly show that the RCDM model is not merely outputting training set images that it could have memorized, This is also confirmed by Fig. 16 in the appendix that shows nearest neighbors of generated points.
The conditional diffusion model might also serve as a building block to hierarchically build an unconditional generative model. Any technique suitable for modeling and sampling the distribution of (lower dimensional) representations could be used. As this not our primary goal in the present study, we experimented only with simple kernel density estimation (see appendix B). This allow quantify the quality of our generative process in unconditional manner to fairly compare against state-of-the-art generative models such ADM;
5

---

Published in Transactions on Machine Learning Research (08/2022)
Model
SimCLR
SimCLR
Dino
Dino
Barlow
Barlow
VicReg
VicReg
Trunk
Head
Trunk
Head
Trunk
Head
Trunk
Head
Val acc.
69.1 %
61.2 %
74.8 %
64.9 %
72.6 %
62.9 %
72.3 %
62.2 %
Table a): ImageNet linear probe validation accuracy on representation given by various SSL models. We observe an accuracy gap between the linear probes at the trunk level and the linear probes trained at the head level of around 10 percentage point of accuracy.
Cond.
RCDM Samples
SimCLR
Dino
Trunk
Backbone
VicReg
Barlow T.
SimCLR
Dino
Head Projector
VicReg
Barlow
Supervised
Figure 2: What encoded inside various representations? First to fourth rows show RCDM samples conditioned the usual resnet50 backbone representation (size 2048) while fifth to eigth rows show samples conditionned the projector/heee represenlation of various sel models. (Note that a separate RCDM generative model was trained specifically for each representation). Commonn/stathe aspects among set of generated images reveal what is encoded in the conditioning representation. Aspects that vory show what is not encoded in the representation. We clearly that the projector representation only keeps global information and not its context, contrary to the backbone representation. This indicates that invariances in SSL models are mostly achieved in the projector representation, not the backbone. Furthermore, it also confirms the linear classification results of Table a) which show that backbone representation are better for classifications since they contain more information about an input than the ones at the projector level. Additional comparisons provided Fig.

---

Published in Transactions on Machine Learning Research (08/2022)
(b) For each encoder, we compute the rank and mean reciprocal rank (MRR) of the image used as conditioning within the closest set of neighbor in the representation space of the samples generated from the valid set (50K samples). A rank of one means that all of the generated samples for given model have their representations matching the representation used as conditioning.
(a) We report results for ImageNet to show that our ap- proach is reliable for generating images which look realistic. Since the focus of our work is not generative modelling but showcase and encourage the use of such model for repre- sentation analysis, we only show results for one conditional generative models. For each method, we computed FID and IS with the same evaluation selup in Pytorch.
Model
[Mean rank
1MRR
Method
Res.
LFID
118
ADM (Dhariwal & Nichol
2021
256
26.8
34.5
+ 1.8
Dino (Caron et al.
2021
1.00
0.99
IC-GAN Casanova et al
RCDM W/ KDE (ours)
IC-GAN
Casanova et
2021
202T
w/ KDE
256
256
20.8
21.6
19.0
38.6
51.3
51.9
+ 2.6
+ 2.2
1.1
Supervised
SimCER CCenn cl
Barlow T.
Swav (Caron &
Zbontar ct al.
2020
2020
2021
5.65
1.00
1.16
1.01
0.69
0.99
0.99
0.97
Table 1: a) Table of results on ImageNet. We compute the FID (Heusel et al. 2017) and IS (Salimans et al. 2016) on 10 000 samples generated by each models with 10 000 images from the validation set of ImageNet as reference. KDE means that we used the uncouditional representation sampling scheme based on KDE (Kernel Density Estimation) for conditioning IC-GAN instead of the method based on K-means introduces by Casanova et al.] (2021). b) Table of ranks and mean reciprocal ranks for different encoders. This table show that RCDM is faithful to the conditioning by generating images which have their representations close to the original one.
we provide some generative model metrics in Tab. 1a along some samples in Fig. 8 to show that our method is competitive with the current literature, even in unconditional generation setting.
4 Visual Analysis of Representations Learned by Self-Supervised Model
The ability to view generated samples whose representations are very close in the representation space to that of conditioning image can provide insights into what's hidden in such a representation, learned by self-supervised models. As demonstrated in the previous section, the samples that are generated with RCDM are really close visually to the image used as conditioning. This gives an important indication of how much is kept inside a SSL representation in general. However, it is also interesting to see how much this amount of 'hidden' information varies depending on what specific SSL representation is being considered. To this end we train several RCDM on SSL representations given by VicReg (Bardes al. 2021), Dino (Caron et al., 2021. Barlow Twins (Zbontar et al.) 2021) and SimCLR (Chen ct al., 2020). In many applications that use sel sell-supervieed models, the representation that is subsequently used is the one obtained at the level of the backbone of the ResNet50. Usually, the representation computed by the projector of the SST.-model (on which the SSL criterion is applied) is discarded because the results on many downstream tasks like classification is not as good as when using the backbone representation. However, since our goal is to visualize and better understand the differences between various SSL representations, we also trained RCDM on the representation ation given by the projector.
In Fig. 2 we condition all the RCDM with the image labelled conditioning and sample 7 images for each model. We observe that the representation at the backbone level does not allow much variance in the generated samples. Even information about the posc and size of the animal is kept in the representation. In contrast, when looking at the samples generated by using representations at the projector level (also coined as head in the figure), we observe much more variance in the generated samples, which indicates that significant information about the input has been lost. These qualitative differences are correlated 3 with the quantitative experiment we made in Fig. Table a) highlighting that when training a linear probe over the corresponding representations, the performances at the backbone level are better than the ones at the projector level.
4.1 Are Self- Supeviupe Representations Really Invariant to Aata-Auumeetttions??
In Fig. B
we apply specific transformations (augmentations) to a test image and we check whether the samples generated by the diffusion model change accordingly. We also compare with the bohavior of superviscd model.
Thhis point is further demonstrated Figure 12 in the Appendix.
7

---

Published in Transactions on Machine Learning Research (08/2022)
Conditioning
Dino
SimOLR
Supervised
Vertical shift
Zoom out
Zoom in
Grayscale
Color Jitter
Figure 3: Using our conditional generative model to gain insight about the invariance (or covariance) of representations with respect to several data augmentations. On an original image (top left) we apply specific transformations (visible in the first column). For each transformed image, we compute the 2048-dimensional representation of ResNet50 backbone trained with either Dino, SinCLR, or a fully supervised training. We then condition their corresponding RCDM on that representation to sample 3 images. We see that despite their invariant training criteria, the 2048 dimensional SSL representations appear to retain information on object scale, grayscale vs color, and color palette of the background, much like the supervised-rrained representation. They do appear insensitive to vertical shifts. We also see that supervised representation constrain the appearance much less, Refer to Fig. 25 in Appendix for comparison with using the lower dimensional projector- head embedding as the conditioning representation.
We note that despite their invariant training criteria, the 2048 dimensional SSL representations do retain information on object scale, grayscale status, and color palette of the background, much like the supervised representation. They do appear invariant to vertical shifts. In the Appendix, Fig. 25 applies the same transformations, but additionally compares using the 2048 representation with using the lower dinensional projector head embedding as the representation. There, we observe that the projector representation seems to encode object scale, but contrary to the 2048 representation, it appears to have gotten rid of grayscale-status and background color information. Currently, researchers need to use custom datasets (in which the factors of variation of specific image are annotated) verify how well the representations learned are invariant those factors. We hope that RCDM will help researchers in self-supervised learning to alleviate this concern since our method is "plug and play" and can be casily used any datasct with any type of representation.
4.2 Self- Supervieed and Supervised Models See Adversarial Examples Differently
Since our model is able to "project back" representations to the manifold of realistic-looking images, we follow the same experimental protocol as Rombach et al. (220)) to visualize how advensarial examples affect the content the representations, as seen through RCDM. We apply Fast Gradient Sign attacks (FGSM) (Goodfellow al. 2015) over a given image and compute the representation associated to the attacked image. When using RCDM conditioned on the representation of the adversarial examples, we can visualize the
generated images still belong to the class of the attacked image or not. In Fig. 4 and 29 the adversarial
8

---

Published in Transactions on Machine Learning Research (08/2022)
Eps Adversanial ex
0.0
0.01
0.025
0.06
0.1
0.25
0.1
1.0
SimCLR Supervised
RCDM
RCDM
Swav
RCDM
Figure 4: Using RCDM visualize the robustness of differently-traine representations adversarial attacks. We use Fast Gradient Sign to attack a given image (top-left corner) on different models with varions values for the attack coefficient epsilon. the first row, we only show the adversarial images obtained from supervised encoder: refer to Fig. 29 in the Appendix to see the (similar looking) adversarial examples obtained for each model. In the following rows we show, for differently trained models, the RCDM "stochastic reconstructions" of the adversarially attacked images, from their ResNet-50 backbone representation. For adversarial atlack a purely supervised model (second row), RCDM reconstructs an animal that belongs to another class, a lion in this case. Third and forth rows show what obtain with ResNet50 that pretrained with SimCLR Swav in SSL fasion, with only their linear softmax output layer trained supervised manner. contrast to the supervised model, with the trannt models, RCDM stably reconstruets dogs from the representation of adversarially attacked inputs, larger values for epsilon. Images classified incorrectly by a trained linear probe are highlight with a red square.
with quite
attacks change the dog in the samples to lion in the supervised setting whereas SSL methods doesn't 't to be impacted by the adversarial perturbations ie the samples are still dogs until the adversarial attack became visible to the human eye.
4.3 Self-Supervised Representations Locally Encode background and Object on Different Dimensions
Experimental manipulation of representations can be useful to analyze to what degree specific dimensions of the represeniation can be associated with specific aspects or factors of variations of the data. In a self-supervised setting in which we don't have access to labelled data, it can be difficult to gain insight as to how the information about the data is encoded in the representation. We showcase a very simple and heuristic setup to remove the most common information in the representations within a set of the nearest neighbors of a specific example. We experimentally saw that the nearest neighbors of a given representation share often similar factors of variation. Having this information in mind, we investigate how many dimensions are shared in between this set of neighbors. Then, we mask the most common non-zero dimensions by setting them to and use RCDM to decode this masked representation. In Fig, 5 this simple manipulation visibly yields the removal of all information about the background and the dog, to only keep information about clothing (only one dog had clothes in the set of neighbors used to find the most common dimensions). Since the information about the dog and the background are removed, RCDM produces images of different clothes only. In the third and fourth row, instead of sctting the most common dimensions zeros, we set them to the value they have other unclothed dog images. By using these new representations, RCDM is able generate the corresponding dog with clothes. This setup works better with SSL methods, supervised models learn to
9

---

Published in Transactions on Machine Learning Research (08/2022)
i
1
zero mask of most common indices where dim of representation is non zero
Least common dim where dim of representation
Figure 5: Visualization of direct manipulations in the representation space of a ResNet-50 backbone trained with SimCLR. In this experiment, find the most common space) of the image used as conditioning (top-left clothed dog). In the second row, we set these dimensions to zero and use RCDM to decode the thus masked representation. We that RCDM produces a variety of clothes (but no dog): all information about the background and the dog has been removed. In the third and forth row, instead of setting these dimensions to zero, we set them to the value they have the representation of the unclothed-dos image on the left. As we can see, the generated dog gets various clothes which were not present in the original image. Additional examples provided Figure 30
dimensions among the neighborhooo (in representation
remove from their representation most of the information that is not needed to predict class labels. We show similar experiment for background removal and manipulation in Figure 30 in the Appendix.
5 Conclusion
Most of the Self- Self-Supervised Learning literature uses downstream tasks that require labeled data to measure how good the learned representation is and to quantify its invariance to specific data-augmeentttions. However one cannot in this way see the entirety of what retained in a representation, beyond testing for specific invariances known beforchand, or predicting specific labeled factors, for a limited (and costly acquire) set, of labels. Yet, through conditional generation, all the stable information can be revealed and discerned from visual inspection of the samples. We showeased how to simple conditional generative model (RCDM) to visualize representations, enabling the visual analysis of what information is contained in self-supervised representation, without the need of any labelled data. After verifying that our conditional generative model produces high-quality samples (atlested qualitatively and by FID scores) and representatiin--aailiee samples, we turned to exploring representations obtained under different frameworks. Our findings clearly separate supervised from SSL models along a variety of aspects: their respective invariances or lack thereof specific image transformations, the discovery of exploitable structure in the representation's dimensions, and their differing sensitivity to adversarial noise.
10

---

Published in Transactions on Machine Learning Research (08/2022)
6 Reproducibility statement
The data and images in this paper were only used for the sole purpose of exchanging reproducible rescarch results with the academic communily.
Our results should be easily reproducible as:
RCDM, is based on the same code as Dhariwal & Nichol guided-diffusion and uses the same hyper-parameters (See Appendix of Dhariwal Nichol [2021) for details about the ssper-parameters).
2021 (https://githun) com.openaii.iiii.o
To obtain our conditional RCDM, one just needs to replace the GroupNormalization layers in that architecture by conditional batch normalization layer of Brock et al. (2019) (using the code from https://github. com/a http:///..a...../aaaaa/////a//// /itocc//iicAA--yyyrrrrrr.
The self-supervised pretrained models we used extract the conditioning representations were oblained from the model-zoo of VISSL (Goyal al., 2021 (code from https:://github.com//
facebookreesarcchviiss]]]
The unconditonal sampling process is straightforward, explained in Appendix B
We are working on cleaning and preparing release any remaining code glue to casily reproduce the results in this paper.
7 Broader impact statement
Our work aims to promote the use of conditional generative models to project back in image space the internal representation learned by latest and future techniques to train deep artificial neural networks - in order to better understand their inner workings. Such improved understanding through qualitative visualizations, in complement with quantitative metrics, is expected to foster the development of more robust and reliable neural network algorithms. Controlled generation of realistic images is not the goal and focus of this work, we merely use as tool for scientific understanding. Yet conditional generative models have already been and will likely continue to be used and improved to generate fake images, including of synthesized imaginary situations and people, that we expect will be increasingly realistic and hard to impossible to distinguish from real photographs. Such technology will be usable for positive creative pursuits, as well as for voluntarily misleading portrayals of fakes passed as truths and facts.
References
David Ackley, Geolfrey E. Hinton, and Terrence J. Sejnowski. A learning algorithum for boltzmann machines. Cognitive Science, 9:147-169, 169, 1985.
Srikar Appalaraju, Yi Zhu, Yusheng Xie, and István Fehérvári. Towards good practices in self-supervised representation learning. arXiv preprint orXiv:2012.000 000668 2020.
Adrien Bardes, Jean Ponce, and Yann LeCun. Vicreg: carianeei regularization for self-supervised learning. arXiv preprint arXim:2105.00 04906, 2021.
Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE transactions on pallern analysis and machine intelligence, 35(8):1798 1828, 2013. URL http:
/ /arxiv.org/abs//200000 5538 cite arxiv:1206.5338.
Andrew Brock, Jeff Donahuc, and Karen Simonyan. Large scale GAN training for high fidelity natural image synthesis. In International Conference on Learning Representations, 2019. URL https://openreviee het//foruumm:dd-B1aaajjj0011
11

---

Published in Transactions on Machine Learning Research (08/2022)
Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp- 9912-9924. Curran Associates, Inc., 2020. URL https://prroceedings htwp:occeddnng..nnrrsss...oom .pd
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervieed vision transformers. In Proceedings of the International Conference on Computer Vision (ICCV), 2021.
Arantxa Casanova, Marlène Careil, Jakob Verbeek, Michal Drozdzal, and Adriana Romero. Instance- conditioned gan. In Mhirty-Fifth Conference on Neural Information Processing Systems, 2021.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference machine learning, pp. 1597 1607. PMLR, 2020.
Xinlei Chen and Kaiming He. Exploring simple siamese representation learning. In CVPR, 2020.
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Euzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dalaset for semantic urban scene understanding. In CVPR, 2016.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers), pp. 1011853/v1/n199111 URL https: http::////ooooo///....o 1t//:/10..86////// 118653/v//19-11/0 1423
Association for Computational Linguistics, 2019. doi:
Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat GANs on image synthesis. In Thirty-Fifth Conference on Neural Information Processing Systems, 2021. URL https:///peenrevee..ne net/
forum Forum/ad=AAuuCzzaV
Alexey Dosovitskiy and Thomas Brox. Generating images with perceptual similarity metrics based on deep networks. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett. (eds.), Advances in Neural Information Processing Systems, volume 29. Curran Associates, Inc., 2016. URL https: pdf
Vincent Dumoulin, Jonathon Shiens, and Manjunath Kudlur. A learned representation for artistic style. In 5th International Conference Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net net, 2017. URL https://opeareviee.nnn wtw.arrrviie..e//ooouuudddd BuTig
Dumitru Erhan, Yoshua Bengio, Aaron Courville, and Pascal Vincent. Visualizing higher-layer features of deep network. University of Montreal, 1341(3):1, 2009.
Linus Ericsson, Henry Gouk, and Timothy M Hospedales. How well do self-supervised models transfer? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5414 5423, 2021.
Patrick Esser, Robin Rombach, and Bjorn Ommer. disentangling invertible interpretation network for explaining latent representations. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pottern Recognition, pp. 9223 9232, 2020.
Spyros Gidaris, Praveer Singh, and Nikos Komodakis. Unsupervised representation learning by predicting image rotations. In 6th International Conference on Learning Representations, ICLR 2018, Vancoweer, BC, Canada, April 30- May 3, 2018 Conference Track Proceedings. OpenReview.nes, 2018. URL http:///ppnneeeeeeee.eeeeeoreaes
12

---

Published in Transactions on Machine Learning Research (08/2022)
Ian Goodfellow, Jean Pouget-Abadie, Mchdi Mirza, Bing Xu, David Warde- Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
Warde-Farley, Farley, Sherjil Ozair, Aaron
lan Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations, 2015. URL
Priya Goyal, Quentin Duval, Jeremy Reizenstein, Matthew Leavitt, Min Xu, Benjamin Lefandeux, Mannat Singh, Vinicius Reis, Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Ishan Misra. Vissl. https
//github.com/////// htt.t///.coo..ccceooeeeeerraa/aa
2021.
Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec Pierre H. Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, Rémi Munos, and Michal Valko. Bootstrap your own latent: A new approach to self-supervised learning. In NeurIPS, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770 778, 2016.
Kaiming Hc, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pottern Recognition, pp. 1729-97338 2020.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by two time-scale update rule converge to local Nash equilibrium. In NewrIPS, 2017.
Geoffrey E. Hinton, Simon Osindero, and Yee Whye Teh. fast learning algorithm for deep belief nets. Neural Computation, 18:1527-1554 2006.
Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of StyleGAN. In CVPR, 2020.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. In Proceedings of the 2nd International Conference on Learning Representations (ICLR 2014), 2014.
Durk P Kingma, Tim Salimans, Rafal Jozefowicz, Xi Chen, llya Sutskever, and Max Welling, Im- proved variational inference with inverse autoregressive flow. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett (eds.), Advances in Neural Information Processing Systems, vol- ume 29. Curran Associates, Inc., 2016. URL https://prooceeding. ienpooeeddinnnnnuiiisssss. /c/pppper/216/ffile/ Paper...p .dd..
Alexander Kolesnikov, Alexey Dosovitskiy, Dirk Weissenborn, Georg Heigold, Jakob Uszkoreit, Lucas Beyer, Matthias Minderer, Mostafa Dehghani, Neil Houlsby, Sylvain Gelly, Thomas Unterthiner, and Xiaohua Zhai. An image is worth 16x16 words: Transformers for image recognition at scale. 2021.
Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales. Deeper, broader and articr domain generalization. In Proceedings of the IEEE international conference on compuler vision, pp. 5542 5550, 2017.
Mario Lučié, Michael Tschannen, Marvin Ritter, Xiaohua Zhai, Olivier Bachem, and Sylvain Gelly. High- fidelity image generation with fewer labels. In Kamalika. Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 4183 4192. PMLR, 09-15 Jun 2019. URL http:://proceciings.nt press//97/ lucic19a.htmm
Aravindh Mahendran and Andrea Vedaldi. Understanding deep image representations by inverting them. In CVPR, pp. 5188-5190 5196. IEEE Computer Society, 2015. ISBN 978-1-4673-6664-0 URL http://ablpp uni- mmi-trier.om de/ wwtp:dddddbbco/edeecceeeececce/ http/////coooe/cpp///////1111111 htm.:/........m.mm.mmmmmmmm Wathendaaan
13

---

Published in Transactions on Machine Learning Research (08/2022)
Mehdi Mirza and Simon Osindero. Conditional generative adversarial nets. arXiv preprint, (arXiv:1411.178.)) 2014.
Ishan Misra and Laurens van der Maaten. Self-supervised learning of pretext-invariant representations. Tn CVPR, 2020.
Charlic Nash, Nate Kushman, and Christopher KI Williams. Inverting supervised representations with auloregressive neural density models. In The 22nd International Conference on Artificial Intelligence and Statistics, pp. 1620-1629. PMLR, 2019.
Anh Nguyen, Alexey Dosovitskiy, Jason Yosinski, Thomas Brox, and Jeff Clune. Synthesizing the preferred inputs for neurons in neural networks via deep generator networks. In Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS'16, pp. 3393-33- 3403, Red Hook, NY, USA, 2016. Curran Associates Inc. ISBN 9781510838819.
Anh Nguyen, Jeff Clune, Yoshua Bengio, Alexey Dosovitskiy, and Jason Yosinski. Plug & play generative networks: Conditional iterative gencration of images in latent space. In Proceedings of the IEEE Conference on Compuler Vision and Pallern Recognilion. IEEE, 2017.
Mehdi Noroozi and Paolo Favaro. Unsupervised learning of visual representations by solving jigsaw puzzles. In Proceedings of the European Conference on Computer Vision (ECCV), 2016.
Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. Feature visualization. Distill, 2017. doi: 10.23915
/ distill.00007.
D. J. Rezende, S. Mohamed, and D. Wierstra. Stochastic backpropagation and approximate inference in deep generalive models. In International Conference Machine Learning (ICML 2014), 2014.
Robin Rombach, Patrick Esser, and Björn Ommer. Making sense of cmns: Interpreting deep representations and their invariances with inns. In Computer Vision ECCV 2020: 16th European Conference, Glasgow, UK, August 23 28, 2020, Proceedings, Part XVII 16, pp. 647 664. Springer, 2020.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. volume 9351, pp. 234-241, 10 2015. ISBN 978-3-319-24573-7. doi: 10.1007/ 078-3-319-24574- 28.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Foi. ImageNet Large Scale Visual Recognition Challenge. International Journal of Compuler Vision, 115(3):211 211-222,,, 2015. doi:
10.1007/811266-015.006667
Ruslan Salakhutdinov, Andriy Mnih, and Geoffrey Hinton. Restricted boltzmann machines for collaborative filtering. In Proceedings of the 24th International Conference on Machine Learning, ICML '07, pp. 791-798, New York, NY, USA, 2007. Association for Computing Machinery. ISBN 9781595937933, doi: 0..146/11733996.2775500 URL https://doi. https::/ddo..org/1... 1145/1273496. 1273596
T. Salimans, 1. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen. Improved techniques for training GANss. In NewrIPS, 2016.
Tim Salimans, Andrej Karpathy, Xi Chen, and Diederik P. Kingma. Pixelenn++: A pixelenn implementation with discretized logistic mixture likelihood and other modifications. In ICLR, 2017.
Ramprasaath R Selvaraju, Abhishek Das, Ramakrishna Vedantam, Michael Cogswell, Devi Parikh, and 2016.
Dhruv Batra. Grad-cam: Why did you say that? arXiv preprint
Assaf Shocher, Yossi Gandelsman, Inbar Mosseri, Michal Yarom, Michal Irani, William Freeman, and Tali
Dekel, Semantic pyramid for image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pottern Recognition, pp. 7457-74666 2020.
14

---

Published in Transactions on Machine Learning Research (08/2022)
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprinl orXio:1312.000 2013.
Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg, Smoothgrad: removing
noise by adding noise. Xiv preprint arx: 706. 006.09825, 2017.
Jascha Sohl-Dickstedd Eric A. Weiss, Niru Maheswaransthan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In Francis R. Bach and David M. Blei (ods.). Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015, volume 37 of JMLR Workshop and Conference Proceedings, pp. 2566-2265. JMLR.org, 2015. URL html
attp://proceedings.
mml.ing.mml......mmmmmm.....m..m mlr.
Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Deep image prior. In Proceedings of the IEEE Conference on Compuler Vision and Pollern Recognition (CVPR), June 2018.
Aaron van den Oord, Nal Kalchbrenner, Lasse Espeholt, koray kavukcuoglu, Oriol Vinyals, and Alex Graves. Conditional image generation with pixelenn decoders. In D. Lee, M. Sugiyama, U. Luxburg, Guyon, and R. Garnett (ods.), Advances in Neural Information Processing Systems, volume 29. Curran Associates, Inc., 2016. URL https://pprocedding wwwpeeddiinnnnuuiis.roeo neurips. cc/paperr2016/file// pdd
Aäron Van Den Oord, Nal Kalchbrenner, and Koray Kavukcuoglu. Pixel recurrent neural networks. In Proceedings of the 33rd International Conference on International Conference on Machine Learning Volume 48, (C, pp. 1747 1756. JMLR.org, 2016.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Reprosentation learning with contrastive predictive coding. arxiv preprinl ar Xrv: 1t0:1807000 007.03748, 2018.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, L ukasz Kaiser, and Illia Polosukhin. Attention all you need. In 1. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems, http://ddiiineeerrrraapcc/aapppe
volume 30. Curran Associates, Inc., 2017. URL pdf
Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. Extracting and composing robust features with dcnoising autoencoders. In Proceedings of the 25th International Conference on Machine Learning, ICML '08, pp. 1096 1103, New York, NY, USA, 2008. Association for Computing Machinery. URL
ISBN 9781605582054. doi: 10. 10.1145 10.1145/139015.... 1t51/1333166.902294
Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, and Pierre-A ntoine Manzagol. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. Journal of Machine Learning Research, 11(110):3371 3408, 2010. URL http:// org/ org/papers/vl1/
sincent10a.htmm
Zhirong Wu, Yuanjun Xiong, Stella X Yu, and Dahua Lin. Unsupervised feature learning via non-paramet instance discrimination. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3733-3742, 2018.
Jure Zbontar, Li Jing, Ishan Misra, Yann Lecun, and Stephane Deny. Barlow twins: Self-supervised lcarning via redundancy reduction. Marina Meila and Tong Zhang (eds.), Proceediigs of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 19310-12320 html
PMLR, 18-24 Jul 2021. URL httpps:/pproceeddggg/ mlr. weteesssvv39//oottttm
Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In European conference on compuler vision, pp. 818 833. Springer, 2014.
Han Zhang, Ian Goodfellow, Dimitris Metaxas, and Augustus Odena. Self-attention generative adversarial networks. Tn International conference on machine learning, pp. PMT.R, 2019a.
15

---

Published in Transactions on Machine Learning Research (08/2022)
Liheng Zhang, Guo-Jun Qi, Liqiang Wang, and Jiebo Luo. Act vs. aed: Unsupervised representation learning by auto-encoding transformations rather than data. arXiv preprint, (arXiv:1901.044)) (..4666.)) 2019b.
Nanxuan Zhao, Zhirong Wu, Rynson W. H. Lau, and Stephen Lin. What makes instance discrimination good for transfer learning? In 9th International Conference Learning Representations, ICLR 2021, Virtual Event, Austria, May 8-7, 2021. OpenReview.ne .nc, 2021. URL ottes://ppaaueveeeeeee/o/ooaaaad
16

---

Published in Transactions on Machine Learning Research (08/2022)
Input space X
Improsntation matching input (hh)
Sampling RCDM
U|xx h) deep netacre
fxx
rancom start
2014
gradiend-waeed reprosentaaionn inttthinng
Input used for conditioning
h
representation
conditiooning
Uff N) = Net(x.
In nih caDnC
X
de verte diRusion consRRondd Rt (REC34
x
(Reversed diffusion process)
Ui
207h
Training of RCDM
X
f(x)
data
manifold M
raal input
Training sample
h
I
Reconstruction loss
ylakds k-f(s)
Denolsing)
U kk)
(b)
Figure 6: (a) Massooaiion of considered image generation methods. A real input æ yields representation h. All methods start from random noise image (9)) maddient-based representation matching (light blue arrows) will move it towards S(h) i.e. until its representation matches h, but won't land the data-manifold M. Unconditional reverse diffusion (ADM model, green arrows) will move it towards the data manifold. Our representation-condiiione diffusion model (RCDM, red arrows) will move it towards M n S(h), yielding a different natural-looking image with the same given representation. (b) Representation- Represantation-CCndiiineeee Diffusion Model (RCDM). From diffusion process that progressively corrupts an image, the model learns the reverse process by predicting the noise that it should remove at each step. We also add as conditioning a vector h, which is the reprosentation given by SSL or supervised model for given image æ. Thus, the network is trained explicitly to denoise towards a specific example given the corresponding conditioning. The diffusion model used is the same as the one presented by Dhariwal & Nichol 2021 with the exception of the conditioning on the representations.
A Conditional and super-resolution sampling with RCDM
As presented in the main text, we introduce RCDM to generate samples that preserved well the semantics of the images used for the conditioning. The training of the model is simple and presented in Figure 6b We show in Figure X additional samples of RCDM when conditioning on the SSL representation of ImageNet validation set images (which were never used for training). We observe that the information hidden in the SSL representation is so rich that RCDM is almost able to reconstruct entirely the image used for conditioning. To further evaluate the abilities of this model, we present in Figure Da a similar experiment except. that we use out of distribution images as conditioning. We used cell images from microscope and photo of status (Both from Wikimedia Commons), sketch and cartoons from PACS (Li et al.) 2017), image segmentation from Cityscape (Cordts et al. 2016) and an image of the Earth by NASA. Even in the OOD scenario, RCDM is able to generate images that share common features to the one used as conditioning because of the richness of ssl representations. However, if the images used as OOD are too far from the training distribution, which is the case when using image segmentation mask from Cityscapes, the model will have more difficulty to reconstruct the images used as conditioning. To investigate this failure is due to the SST network used to producod the conditioning, we run the experiment in Figure 11 which we kept the same SSL model with an RCDM trained on ImageNet and another one on Cityscape. We observe that when using Cilyscapes segmentation mask conditioning with an RCDM trained on Cityscapes segmentation mask, despite using SSL model trained only on ImageNet, RCDM is able to reconstruct the conditioning very faithfully. This mean that the failure mode observed in OOD are mostly due to the visualization model (RCDM) and not due to the representation used for conditioning.
In those examples we used conditional batch-normalization (which the same tochnique as used by Casanoval et al. (2021)). ) However one can also use the sampling technique built-in in the ADM model of Dhariwal &
Nichol 2021 Instead of using embedding layer that take discrete representation, we can use a linear
17

---

Published in Transactions on Machine Learning Research (08/2022)
Random
Supervised
SSL (Dino backbone)
SSL (Dino proj)
Conditioning
DIP
L
Random
Supervised
SSL (Dino backbone)
SSL (Dino proj)
Conditioning
RCDM
We compare RCDM with the approach of Zhao et al. 2021) that use Deep Image Prior (DIP (Ulanov et al. )018)) to visualize the fcatures learn by Sclf-supervised models. We run this experiment by using as conditioning an In-Distribution image (with the dog from ImageNet validation set) and an Out-Of- Distribution image with an image from the Earth (Source: NASA). For both methods, we compare with representations extracted at the backbone level of Resnet (after average pooling) from a random initialized network, supervised network and network trained with SSL (Dino). We also use the representation extracted at the projector level for Dino. We observe that the samples we obtained with RCDM have a better quality than the ones generated with DIP and are also more insightful about the properties of the reprosentations. In this instance, we clearly see with RCDM that the supervised representation invariant to background which is something more difficult to assess with DIP.
Figure 7:
layer to map representation to the dimension of the time steps embedding and add it along the time step conditioning. A comparison with these two conditioning methods is shown in Figure 10
We also use the super-resolution model presented by Dhariwal & Nichol 2021 to generate images of higher resolutions. In Figure 13 we use the small images on the top of the bigger images as conditioning for a RCDM trained on images of size 128x128. Then, we feed the 128x128 samples into the super-resolution model of Dhariwal & Nichol 2001h) to get images of size 512x512. Since the model of DDariwal & Nichol (2021) is conditional and need labels, we used a random label when upsampling from RCDM. Despite using the "wrong" label, the high resolution samples are still very close to the conditioning. This show that RCDM can be used jointly with super-resolution model to sample high fidelity images in the close neighborhood of the conditioning.
To verily how well our model can produce realistic samples from different combinations of representations we take two images from which we compute their representations and perform a linear interpolation between those. This give us new vectors of representation that can be used as conditioning for RCDM. We can see on Figure 14aan and Figure 15 that RCDM is able to generate samples that contains the semantic characteristies of both images.
18

---

Published in Transactions on Machine Learning Research (08/2022)
Finally, in Figure 16 we search the nearest neighbors of series of samples in the ImageNet training set. As demonstrated by Figure T6 RCDM samples images that are new and far enough from images belonging to the training set of ImageNet.
19

---

Published in Transactions on Machine Learning Research (08/2022)
Figure 8: Generated samples from RCDM 256x256 images trained with representations produced by Dino. We put on the first column the images that are used to compute the representation conditioning. On the following column, are qualitatively close to the original image.
see the samples generated by RCDM. It is worth to denote our generated samples
20

---

Published in Transactions on Machine Learning Research (08/2022)
3
Figure 9: Generated samples from RCDM model 256x256 images trained with representations produced by Dino Out of Distribution data. We put the first column the images that used to compute the representation. On the following column, can see the samples generated by RCDM. It is worth to denote our generated sample are close to the original image. The images used for the conditioning are from Wikimedia Commons, Cityscapes (Cordts et al. 2016 PACS (Li et al. 2017 and the image of earth from NASA.
21

---

Published in Transactions on Machine Learning Research (08/2022)
Conditioinng
Using Cond. Batch Norm conditioning
Using ADM built-in conditioning
Conditioning
Figure 10: Comparison between conditioning RCDM with batch normalization and the built-in conditioning mechanism offered by ADM. For this example, took the representation backbone of dino trained on ImageNet with resolution 128x128 128. There doesn't 't seem to be any siguificant differences between both methods.
22

---

Published in Transactions on Machine Learning Research (08/2022)
Conditioinng
(Clyacapes
Dino trained on ImageNet, RCDM trained on ImageNet
segmentation mask)
Conditioning
(Cityccapes
Dino trained on ImageNet, RCDM trained on Cityscapes
Figure 11: We perform this experiment to see if the failures mode on OOD, especially when conditioning on segmentation mask of Cityscapes, are due to the sclf-supervised representations not containing enough information to reconstruct the image, or are due to RCDM not being able to reconstruct OOD images. On the first line, show the samples generated by an RCDM trained on ImageNet with the self-supervised representation of Dino that also trained on ImageNet. On the second line, we show the samples generated by an RCDM trained on the segmentation masks of cityscapes that the same self-supervised model of Dino that was trained on ImageNet. We can clearly see that despite using a SSL model trained on ImageNet, when RCDM is trained on CityScapes, the reconstruction almost match the original conditioning. Hence, one should train or fine-lune RCDM on any target dataset to then use it to sample representation conditioned images from a (frozen) pre-trained model.
23

---

Published in Transactions on Machine Learning Research (08/2022)
Most confident linear probe classification
Less confident linear probe classification
SimCLR (Trunk)
SimCLR (Head)
Dino (Trunk)
Dino (Head)
EE
a) RCDM samples using a condittonnd-eeprsentttiin having the highest probabilities under a linear classifier.
24

---

Published in Transactions on Machine Learning Research (08/2022)
Most confident linear probe classification
Less confident linear probe classification
Barlow T. (Trunk)
Barlow T. (Head)
Vicreg (Trunk)
Vicreg (Head)
b) RCDM samples conditioning that have the lowest probabilities under linear classifier.
25

---

Published in Transactions on Machine Learning Research (08/2022)
c) More RCDM samples on conditioning that have the highest probabilities under linear classifier.
d) More RCDM samples conditioning that have the lowest probabilitics under a linear classifier.
We trained a lincar probe for classification by using the representation (at the backbono and projector level) given by various SSL models. Then we find, among the ImageNet validation set, the images that yield the highest softmax probability under this linear probe and use RCDM generate samples with respect to their representations. We also find the images that yield the lowest softmax probabilities. On the first column and seventh column in a) and b), present the images used conditioning (thus, the ones with highest and lowest softmax probabilities). In the following columns we show the corresponding RCDM samples. We observe that all generated images belong to the same class as the one that was used as conditioning when looking at the most confident representation. However when looking at the least confident representation, the generated samples does not seem belong to a unique class. This experiment shows that the uncertainty the predictions of downstream classification task can also be predicted by simple visual inspection of samples produces by RCDM.
Figure 12:
26

---

Published in Transactions on Machine Learning Research (08/2022)
B A hierarchical diffusion model for unconditional generation
We provided novel and conditional generative model based on given latent representation e.g. from a SSL embedding, and a diffusion model. This allows visualizing and thus provides insight regarding what is or isn't encoded in a particular representations. We can go one step further and augment this conditional model with an unconditional one that can generate those representations. This will provide us with the ability to generate new samples without the need to condition on a given input. As a by-product, it will allow us to quantify the quality of our generative process in an unconditional manner to fairly compare against state-of-the-art generative models.
We shall recall that our goal is to employ the conditional generative model to provide understanding into learned (SSL) representations. The unconditional model is only developed to compare our generative model and ensure that its quality is reliable for any further down analysis. As such, we propose to learn the representation distribution in a very simple manner via the usual Keruel Density Estimation (KDE). That is, the distribution is modeled as
N Σ N(h; I(on);
p(h)
with a set to 0.01. By using the above distribution, we are able to sample representations h to then sample images æ conditionally to that h using our diffusion model. We provide some samples in Figure 20 to show that even with our very simple conditioning, our method is still able to generate realistic images.
C On the closeness of the samples in the representation space
Even if we show that RCDM is able to generate images that seems visually close to the image used for the conditioning, it's still unclear how close those images are in the representation space. We can compute euclidean distances but to know how close the generated samples are to the conditioning, we need to have references that can be used to compare this distance with. As references, between a conditioning image and random images in the validation set of ImageNet, random images belonging to the same class as the conditioning, the closest images in the training set, the conditioning image on which we applied single data augmentations and the conditoning image on which we applied the data augmentation performed by Swav and Dino (Caron et al. 2020 2021). D. The results can be seen Figure 21 for a RCDM trained with Dino representations and in Figure 22 for a RCDM trained with SimCLR representations. On both Figure, we observe that the generated images with RCDM are closer to the conditioning than the closest neighbors in the entire training set of ImageNet. We also computed the mean and reciprocal mean rank in the main paper (Table bb) which show that for most SSI. models the closest examples in the representation space of the generated images is the image used as conditioning. We also added Figure 17 to show which rank is associated to samples generated by RCDM. For SimCLR, the rank is mostly always 1 whereas we got more diversity for the supervised case. This difficulty of RCDM to generated samples which have their representation that map back to the one used for the conditioning can be explain by the nature of a supervisod training. In such scenario, the encoder is trained to map big set of images (often a specifie class) to specific type of representation whereas SSL models are explicitly train to push each examples farther away from each others. Thus, it seems more likely that a little perturbation on the supervised representation induces a change of nearest neighbor. This hypothesis is supported by Figure 29 which show that small adversarial attack are enough to induces a change of class in the representation which is not the case for SSL encoders.
compute the euclidean distance
D Analysis of representations learned with Self- model
Having generated samples that are close in the representation space to a conditioning image can give us an insight on what's hidden in the representations learned with self-supervised models. As demonstrated in the previous section, the samples that are generated with RCDM are really close visually to the image used as conditioning. This give an important proof of how much is kept inside a SSL representation. However, it's also important to consider how much this amount of 'hidden" information varied depending on the SSL representation that is used. Therefore, we train several RCDM SSL representations given by VicReg
27

---

Published in Transactions on Machine Learning Research (08/2022)
Bardes et al. 2021p, Dino (Caron et al., 2021), Barlow Twins (Zoontar et al. 2021 and SimCLR (Chen et al. 2020). In many applications that used self-supervised models, the representation that is used is the one corresponding to the backbone of the ResNet50. Usually, the representation given by the projector of the SSI-model (on which the SSL criterion is applicd) is discarded because the results on many downstream tasks like classification not as good as the backbone. However, since our work is to visualize and better understand the differences between SSL representations, we also trained RCDM on the representation given by the projector of Dino, Barlow Twins and SimCLR. In Figure 24 we condition all the RCDM with the image labelled conditioning and sample 9 images for each model. If we look at the projector of the SSL models, the generated samples have a higher variance.
To further compare and analyse the different SSL models, we visualize how much SSL representations can be invariant with respect to a transformation that applied on the conditioning image. In Figure 255 we apply several Data Augnentation: Vertical shift, Zoom out, Zoom In, Grayscale and a Collor Jitter on given conditioning image. Then we compute the SSL representations of the transformed image with different SSL models and use our corresponding RCDM see how much the samples have changed with respect to the samples generated on the vanilla conditioning image. We observe that the representation (the 2048 backbone onc) of all SSL methods are not invariant to scale and change of colors. Whereas the representation of the projector doesn't seem to take into account any small transformation in the original conditioning outside the scale for Dino. For SimCI., there is still some information about the background that is kept in the representation however the samples are not as close visually with respect to the 2048 representation. Barlow Twins is interesting because there isn't much differences between the backbone representation (2048) one and the representation of the projector (Size 8192). With the exception that this last representation seems to be more invariant to color shift than the backbone onc.
We also perform an experiment in Figure 28 using OOD images to ensure that the conclusions drawn with our methods about SSL representation are not specific to ImageNet.
D.1 Visualization of adversrial examples
We use RCDM to visualize adversarial examples for different models. For each model, we trained a linear classifier on top of their representations to predict class labels for the ImageNet dataset. Then, we use FGSM attacks over the trained model using a NLL loss to generate adversarial examples. In Figure 2 we show the adversarial examples that are created for each model, the samples generated by RCDM with respect to the representation of the adversarial perturbed example and the class label predicted by the linear classifier over the adversarial examples. The supervised model is very sensitive to the attack whereas SSL models seems more robust.
D.2 Manipulation of SSL representations
It is also possible to manipulate SSL representations to generate new images. We try to apply addition and subtractions over SSL represcntations (similarly to what has been done in NLP). From two different images, we compute the difference between the two corresponding representations and add the difference vector to a third image. Figure 32 shows that it is possible to apply such transformations meaningfully in the SSL space. We also used another setup where we choose specific dimensions in the representation based on how many times these dimensions are non zero in the representation space of set of neighbors. Then we set this dimension to xero which surprisingly induces the removing of the background in the generated images. We also replace them by the same corresponding dimension of another images which induces change of background toward the one of the new image. Results are shown in Figure 30
D.3
Experiments with vision transformers
All the experiments in this paper were conducted with Resnet50 since most the SSL baselines are available with this model. However, RCDM can work with any type of architecture, including vision transforms. In Figure 33 we show RCDM samples using representations of Dino trained with a VIT-B 16 (Kolesnikov et al. 2021
28

---

Published in Transactions on Machine Learning Research (08/2022)
D.4 Why is my model over-fitting the training set
By enabling the visualization of what is learned in a representation, RCDM can help researchers to gel a better understanding of the failures modes of their models. In one of our experiments, we trained a SSL models with VieReg by using only cropping as data augmentation (thus discarding the traditional colorjiterring/ grayscliing and other transforms that change the colors). Training a linear probe on such network resulted in training accuracy of 95% on the training set while the validation accuracy was only about 20%. To better understand how the model was able overfit on the training set, we trained RCDM the representations of this model. The samples obtained are shown in Figure 34 This experiment validate the hypothesis that removing color related augmenlations during the training of SSL models leads to learn representations that are only colors and textures based.
D.5 Visualizing how representations changing during training
RCDM, is to consider how representations are changing during training. In this experiment, we trained 3 RCDM models on the representation given by SSL models (VicReg) trained after epoch, 5 epochs and 50 epochs. In this experiment, we want to visualize what is changing in the representation during training. The hypothosis was that at the beginning of the training, the network is learning some easy feature, like some color information, and later in the training more complex features, probably containing more shape based information. In Figure (35 we observe that after 1 epoch of SSL training, the information retain in the representation mostly color /tetture based while after only 5 epochs, are better defined.
Another way
that the shape
29

---

Published in Transactions on Machine Learning Research (08/2022)
High resolution samples from our conditional diffusion generative model using the super resolution model Dhariwal Nichol 2021). We use the small images on the top of each bigger image as conditioning (source NASA for the earth picture) for a diffusion model trained with Dino representation on 128x 128 images. Then, feed the samples generated to the super resolution model of Dhariwal & Nichol (2021) which produces images of size 512x512. Since the super resolution model is conditional, sample random label. We note that the high issolution samples are still very close to the conditioning.
Figure 13:
30

---

Published in Transactions on Machine Learning Research (08/2022)
Conditioning
Conditioning Conditionnn2 2
Conditioning 2
2onditioning 2
(a) Linear interpolation between the image of the golden retriever conditioning with varions other images belonging
to the same class conditioning 2.
Conditioning
Conditioning 2
Conditioning 2
Conditioning 2
Conditioning 2
(b) Linear interpolation between the image of the pug conditioning 1 with various other images belonging the espresso class as conditioning 2.
Figure 14: Each vectors that result from the linear interpolation is feed to a RCDM trained with Barlow Twins representation.
31

---

Published in Transactions on Machine Learning Research (08/2022)
Figure 15: Diversity of the samples generated by RCDM on interpolated representations. Each row corresponds to different random noise for the same conditioning, On the first and last column are the real images used for the interpolation. All of the images in-between those rows are samples from RCDM.
32

---

Published in Transactions on Machine Learning Research (08/2022)
Samples
Croaset training Image
Stamppes
Closest training Fmsgge
Semples
Cfoeset training Imagee
(a) Closest real images (ImageNet training set) from images sampled with RCDM trained on Dino (backbone) representation (2018).
Samples
Closest training image
Samplee
Coseet trahing mage
Siamo
Cloxess treining image
(b) Closest real images (ImageNet training set) from images sampled with RCDM trained on Dino projector represen- tation (256).
Figure 16: We find the nearest neighbors in the represontation space of samples generated by RCDM. The images in the red squared are the ones used for conditioning.
33

---

Published in Transactions on Machine Learning Research (08/2022)
SimCLR
Supervised
RCDM
NN
Rank
1
1
1
1
1
1
1
1
4
1
2
RCDM
Rank
1
1
5
2
1
1
2
RCOM
NN
1
8
1
1
RCDM
Rank
1
1
1
6
2
5
NN
RCOM
Rank
1 1
1
1
1
2
8
RCDM
Rank
1
1
1
2
1
8
Figure 17: After generating samples with respect specific conditioning, we compute back the representation of the generated samples and find which are the closest neighbors in the validation set. Then, we compute the rank of the original image that was used as conditioning within the set of neighbors. When the rank is it implies that the nearest neighbors of the generated samples is the conditioning itself, meaning that the generated samples have their representation that is very close in the representation space to the one used as conditioning. We can see that for SimCLR, the generated samples are much closer in the representation space their conditioning than the supervised representation. This easily explain by the fact that supervised model learn map images from same class toward similar representation whereas SSL models try push further away different examples.
34

---

Published in Transactions on Machine Learning Research (08/2022)
Figure 18: Visual analysis of the variance of the generated samples for a specific image when using the trunk buukkbbackoone of a Barlow Twins encoder. The first image (in red) in the one used as conditioning.
35

---

Published in Transactions on Machine Learning Research (08/2022)
Figure 19: Visual analysis of the variance of the generated samples for a specific image when using the projector/heet hhaad of a Barlow Twins encoder. The first image (in red) in the one used as conditioning.
36

---

Published in Transactions on Machine Learning Research (08/2022)
Our simple generative model of representations consists applying a small Gaussian noise over representation computed from random training images ImageNet. We use these noisy vector as conditioning for our 256x256 RCDM trained with Dino representations. We note that the generated images looks realistic despite some generative artefact like two-headed dog and elephant-horse.
Figure 20: Unconditional generation following the protocol of section B
37

---

Published in Transactions on Machine Learning Research (08/2022)
Random
0.0
636.5
868.7
1042.9
712.5
1443.8
Same class
1833.4
1040.7
673.9
650.4
745.7
Closest Images
340.1
344.8
347.9
356.5
364.2
DAA
23.8
478.3
64.5
173.9
147.3
DA.2
417.7
010.0
444.5
445.6
379.5
Generated
132.5
93.8
91.4
122.0
124.5
Figure 21: Squared Euclidean distances in the Dino representation space. We show the squared euclidean distance between the conditioning image the leftmost column first row and different images to get an insight about how close the samples generated by the diffusion model stay close to the representation used conditioning. The distances with the conditioning is printed below each images. On the first row, we show random images from the ImageNet validation data. On the second row, we take random validation examples belonging to the same class as the conditioning. On the third row, we find the closest training neighbors of the conditioning in the representation space. On forth row, D.A. means Data Augmentation which consist in horizontal flip, CenterCrop, ColorJitter, GrayScale and solarization. On fifth row (D.A. 2), use the random data Augmentation used in the paper of (Caron et al. 2020: 2021 On the last row, show the generated samples from our conditional diffusion model that use Dino representation. The samples produces by our model are much closer to the conditioning than other images.
38

---

Published in Transactions on Machine Learning Research (08/2022)
UNLEADED
Random
0.0
8.8
7.1
4.8
26.4
Same class
35.4
13.7
6.0
Closest images
FAMILY PORTRAIT
GARBAGE TRUCK FAIL
2.6
2.6
2.6
2.7
D.A
DA.2
5.2
3.9
Generated
1.3
2.4
1.5
2.4
2.2
Figure 22: Squared Euclidean distances in the SimCLR projector head representation space. We show the squared euclidean distance between the conditioning image on the leftmost column on first row and different images to get an insight about how close the samples generated by the diffusion model stay close to the representation used as conditioning. The distances with the conditioning is printed below each images. On the first row, we show random images from the ImageNet validation data. On the second row, take random validation example belonging to the same class the conditioning. On third row, we find the closest training neigbords of the conditioning in the representation spacc. On forth row, D.A. means Data Augmentation which consist. in horizontal flip, CenterCrop, ColorJitter, GrayScale and solarization. On fifth row (D.A. 2), we use the random data Augmentation used in the paper of (Caron et al. 2020 2021 On the last row, we show the generated samples from our conditional diffusion model that use SimCLR projector head representation. The samples produces by our model are much closer to the conditioning than other images.
39

---

Published in Transactions on Machine Learning Research (08/2022)
RGDD
M
Distance: 0
17.9
21.
19.7
24.2
23.6
21.2
G
N
Distance:
35.5
33.6
42.4
32.9
29.6
37.2
R
C
M
Distance:
31.5
34.9
35.2
34.6
33.7
31.9
C
Distance:
43.2
38.5
45,4
42.4
43.2
44.4
R
M
Distance:
31
20.5
28.8
26.2
26.8
25.6
C
A
Distance:
48.2
50.6
43.8
43.2
40.7
43.1
Figure 23: Comparison of the euclidean distance between IC-GAN and RCDM. We use the same self- supervised representation as conditioning (Swav encoder) for RCDM and IC-GAN. We compute the euclidean distance between the representation of the generated images versus the representation used as conditioning. We observe that samples of RCDM are much closer the representation space (and also visually) to the conditioning. Samples of IC-GAN show higher variability, thus farther away in the representation space.
40

---

Published in Transactions on Machine Learning Research (08/2022)
Conditioning
Dino
Barlow
SiMCLR
VicRag
Dimo Head
Barlow Head
SmCLL MmCLRHHad Head
Supervised
Figure 24: Generated samples from RCDM trained with representation from various self-supervised models. We generate samples for each model with different random seeds. We observe that the representation given by dino isn't very invariant while the given by SimCLR or VieReg show much better invariance. We also show the samples of RCDM trained the representation given by the projector (The embedding which is usually applied the SSL criterion). There is a much higher variability in the generated samples. Maybe too much to be used for a classification task since we can observe class crossing.
41

---

Published in Transactions on Machine Learning Research (08/2022)
Conditioning
Dino
SimCLA
Barlow
Vertical shift
Zoom out
Zoom in
Grayscale
Color litter
Conditioning
head
SimCLR
Barlow
Vertical shirt
Zoom out
Zoom
Grayscale
Color Jitter
Figure 25: We compare how much the samples generated by RCDM change depending on different transfor- mations of given image and the model and layer used to produces the representation. Top half uses 2048 representation. Bottom half uses the lower dimensional projector head embedding. We observe that using the projector head representation lcads to much larger variance in the generated samples whereas using the trnal backbone (2048) representation leads to samples that are very close to the original image. We also observe that the projector representation seems to encode object scale, but contrary to the 2048 representation, it seems have gotten rid of grayscale-status and background color information.
42

---

Published in Transactions on Machine Learning Research (08/2022)
Conditioning
Barlow
SimCLR
Supervised
Vertical shift
Zoom out
Zcom in
Grayscale
Color Jitter
Conditioning
Barlow
SimCLR
Superviaed
Vertical shirt
Zoom out
Zoom
Gayscale
Color Jitter
Figure 26: Same setup as Figure 25 except with other images as conditioning.
43

---

Published in Transactions on Machine Learning Research (08/2022)
Dino
Barlow T.
SLMCLR
Dino head
Barlow T. Head
SmCLR head head
Supervised
Figure 27: Generated samples from RCDM using the mean representation for a specific class (golden retriever) in ImageNet for various SSL models.
44

---

Published in Transactions on Machine Learning Research (08/2022)
OOD
Dino (Resnet50) backbone
conditioning
OOD
Dino (Resnet50) projector
conditioning
Figure 28: We compare the visualization obtained with representations from Dino (resnet50) at the backbone level and also at the projector level on OOD images to ensure that conclusions drawn with our model about SSL representations are not specific to ImageNet. We confirm that we observe the sames phenomenons in an OOD settings as the ones we could get on an In- In-Distribution scenario.
45

---

Published in Transactions on Machine Learning Research (08/2022)
0.1
Adversariia rial eK.
Supervised
RCDM
Class
Retriever
Lion
Chain
Bubble
Egs Adversarie exc
0.0
0.01
0.05
0.1
0.5
1.0
2.0
SiMCLA
RCDM
Cless
Golden Retriever
Leonberg
Golden Retriever Golden Retriever
Coyotte
Chain mail
Chain mail
0.0
0.01
0.05
0.5
2.0
Eps Adversar ex
Swar
RGDM
Clasd
Golden Retriever
Golden Retriever
Lion
Chow chow
Brain coral
Brain coral
Nerntodee
0.5
Eps Adversarial
Battow I.
RCDM
predicted
Golden Retriever Golden Retriever Golden Retriever
Chow chow
Chainlink fence
Brain coral
Brain coral
Class
Figure 29: Visualization of adversarial examples We use RCDM to visualize adversarial examples for different models. For each model, we trained a linear classifier on top of their representations to predict class labels for the ImageNet dataset. Then, we use FGSM attack over the trained model using a NLL loss to generate adversarial examples towards the class lion. For each model, we visualize adversarial examples for different values of which the coefficient used in front of the gradient sign. In the supervised scenario, even for small values of epsilon which doesn't 't seem to change the original image, the decoded image well the predicted label by the linear classifier becomes a lion. However it's not the case in the self-supervised setting where the dog still get the same class or get another breed of dog as label until the adversarial attack becomes more visible to the human eye (For e value superior to 0.5).
46

---

Published in Transactions on Machine Learning Research (08/2022)
zero mask of most common indices where dim of representation is non zero Least common dim of where dim of representation is non zero
Figure 30: Background suppression and addition Visualization of direct manipulations over the representation space. On the first row, we used the full representation of the dog's 's image on the top-left as conditioning for RCDM. Then, we find the most common non zero dimension across the neighborhood of the image used as conditioning. On the second row, we set these dimensions to zero and use RCDM to decode the truncated representation. We observe that RCDM produces examples of the dog with a high variety of unnatural background meaning that all information about the background is removed. In the third and forth row, instcad of setting the most common zero dimension zero, set them to the value of corresponding dimension of the representation associated to the image the left. As we can see, the original dog get a new background and a new pose,
47

---

Published in Transactions on Machine Learning Research (08/2022)
zero mask of least common indices where dim of representation non zero Least common where dim representation is zero
Figure 31: Same setup as Figure
30 except that instead of using the most common non zero-dimensions as
mask, we used the least common non-zero dimensions as mask. On the second row, we observe that some information about the original dog is removed such that in each column, get slightly different breed of dog while the background stay fixed. On the third and forth row, background (grass) is propagated through the samples (which was not the case in Figure 30
that the information about the
+
Figure 32: Algebraic manipulation of representations from real images (left-hand side of =) allows RCDM generate new images with novel combination of factors. Here we use this technique with ImageNet images, attempt background substitutions.
48

---

Published in Transactions on Machine Learning Research (08/2022)
Figure 33: Conditional generation with RCDM using representation extracted from VIT-B 16 trained with Dino. This experiment shows that RCDM is able to successfully use the representation extracted form different kinds of architectures.
49

---

Published in Transactions on Machine Learning Research (08/2022)
5
Figure 34: Conditional generation with RCDM using representation extracted from a Resnet50 trained with VicReg using only cropping as data augmentation (thus discarding all transforms related color change). This experiment shows that training an SSL model without learning any invariances to colors lead learn only statistics about the colors in the representation. We can clearly that the samples generated from the guitar are clearly following the same colors statistics as the conditioning but totally fail to reconstruct anything related to the shape information. The source for the picture of the carth is NASA.
50

---

Published in Transactions on Machine Learning Research (08/2022)
trained for 1 epoch
trained for 5 epochs
trained for 50 epochs
Figure 35: Conditional generation with RCDM using representation extracted from a Resnet50 trained with VicReg for 1, 5 and 50 training epochs (a new RCDM generator is trained fully for each case). This experiment shows that the SSL model first (after epoch) lcarns to retain mostly information about color and texture in its representation (see e.g. how conditioning on the parrot representation yields vehicles with similar color- themee. It encodes accurate information on the more precise shape only later in training.
51

---

Published in Transactions on Machine Learning Research (08/2022)
Gondniciig
(a) Earth from an untrained representation (Random initialized Resnet 50).
Condbonig
(b) Earth from a supervised representation (Pretrained resnet50 on ImageNet)
Cordtoang
(c) Earth from SSL representation (Dino Resnet50 backbone).
Figure 36: Different samples of RCDM conditioned on satellite image of the earth (source: NASA). We show the samples we obtained in a) when using random initialized network to get representations, b) when using a pretrained resnet50, c) when using self supervised model (Dino).
52


Reviews:
[{"review_id": "uAh8Tkg2Wy", "reviewer": "Reviewer_W3ik", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n* Due to the excellent sample quality of diffusion models, this work presents the best visualizations of learned representations I have seen. \n* The presented method is a straightforward extension of existing work on diffusion models, and enables new applications beyond visualization such as image manipulation.\n* The high quality of visualizations as well as ability to successfully invert OOD images allows for more interactive probing and understanding of learned representations\n* The paper is well written and the motivation and methods are clear. \n\nWeaknesses:\n* Despite the quality of inversions, I found the experiments were unconvincing of the utility of this approach for understanding neural networks. If such visualizations are to help understand neural networks, what have we learned in the qualitative experiments that we did not know previously? I would love to see identification of a failure of some representations that could be exploited by altering inputs to confuse the representation (based off unexpected invariances), or some human study indicating that these visualizations are useful for any task.\n* Baselines and discussion of related work: there is tons of work on visualizing representations of DNNs, but there are no alternative methods presented or compared to in this paper. Could all the qualitative insights have come about from a visualization method that produced less realistic inversions? How do these other approaches compare in your setting? Appendix A is a start at evaluating this quesiton, but ther eis a huge gap between the quality of the baselines in Appendix A and published results in this area. \n* An additional challenge is that your conditional diffusion models will have to be retrained for every representation. As another idea, I believe you could do classifier guidance in the diffusion context by combining an unconditional p(x) model with a conditional p(h | x) model using something like log p(h|x) = ||f(x) - h_target||^2, i.e. you can use classifier guidance to guide your unconditional sample to have a similar representation as your target without retraining a diffusion model. \n* There’s no discussion of how the images that are used to train the conditional diffusion model will bias the inversions. Are failures on OOD inputs due to the representation failing on OOD inputs or the visualization failing?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents a new approach for visualizing learned representations with a conditional diffusion model. Given a fixed representation h = f(x) (from any method), a conditional diffusion model is trained on a given dataset (ImageNet) to model p(x | h). By sampling several images from p(x|h), we can then better understand what the representation is sensitive to (similar across samples) vs. invariant to (different across images). The visualization approach is applied to SOTA SSL methods Dino, Swav, SimCLR, Barlow Twins and supervised learned representations. Qualitatively, this work shows how different representations are sensitive to different things and how visualization can show the robustness of representations to adversarial perturbations, and quantitatively the resulting unconditional generative model performs well.", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "Critical:\n* While the reconstructions are high quality, I think this paper is still missing an experiment which can show us something we didn't know before about a learned neural network representation. Leveraging these visualizations to perform some downstream task (e.g. if a human is able to detect whether or not the network will get the correct answer on an adversarially perturbed input) would help to strengthen the argument that thsese visualizations are useful.\n* Baselines and discussion of related work: the related work needs to be expanded to cover the work of Dosovitsky & Brox, Mahendran & Vedaldi, Nguyen et al., and the DeepDream/feature viz work of Olah et al at a minimum. It would be great if there were additional comparisons with these approaches to highlight that they cannot yield the same qualitative conclusions as your proposed method (e.g. around invariance to vertical shift but sensitivity to scale). I also feel an explanation or further experiments to highlight why the baselines are so bad in Appendix A is critical.\n* Appendix A:  I don't think the argument that gradient-based methods only occupy a K-dimensional subspace is valid (see more comments below). An extended proof or discussion is needed.\n\nThe below are a mix of suggested change and notes as I went through the paper (organized by section). \n\nIntro:\n* Intro: it may be worth mentioning that the modern family of SSL methods are similar to prior work like metric learning with triplet losses and exemplar CNNs/instance discrimination that have both of the criteria you listed (objective depends on properties of the embedding not reconstruction, and leverages data augmentation to encourage invariance).\n* I don’t think “s.a” is a standard, please write out as “such as”\n* For the observations mentioned in the intro, is (ii) about post or pre-projector SSL representations?\n\nRelated work:\n* “Successful interpretation…. first few layers of a DN”: what does successful mean here? I think an issue throughout the paper is that there are vastly different definitions/success criteria for “understanding” so it may be worth fleshing out exactly what was understood in these earlier works and how they failed when applied to higher-level layers of DNNs\n* I believe the characterization of prior work that inverts DNNs with DIP is inaccurate, at least looking at Zhao et al. (2021), they minimize d(f(x), f(g(w))) where w are the weights of the generator and the loss is in feature space. Qualitatively your results in Appendix A are far worse than Figure 2 of Zhao et al. (2021). Any ideas what is leading to this discrepancy? \n* Deterministic visualization methods: almost all the methods listed have some stochastic component based on the initialization of the random image or random weights that are being optimized. There’s no baseline/comparison/discussion of how one could also use these methods to form a diverse set of images given a representation\n* There is a tremendous amount of work on studying learned representations through post-hoc inversion and visualization methods, and I would not expect this paper to cover all of them. However, I do think there are a number of fundamental papers that were missed that should be briefly discussed:\n* Mahendran & Vedaldi (2015, 2016): motivates the notion of natural pre-images, constructs a careful set of regularizers to yield good inversions that are more natural (e.g. jittering, smoothness, texture/style regularizer). There’s no discussion of DeepDream (2015): https://en.wikipedia.org/wiki/DeepDream and   Olah et al.’s work on feature visualization and activation atlases (e.g. https://distill.pub/2017/feature-visualization/), this Distill paper has a great table under “The Spectrum of Regularization” that mentions several other papers that are not discussed here, e.g. Nguyen et al. 2016a that uses a GAN generator to synthesize preferred inputs (but does not draw samples from the conditional, instead they optimize directly for the latent), Nguyen et al. 2016b that runs MCMC with conditional guidance based on features.\nAll of these approaches that optimize in image space with extra regularizers can be used to generate a diverse set of natural pre-images that could then be evaluated and compared against.\n* Dosovitskiy & Brox (2016) have two papers, one training generators with L2 loss which results in blurry reconstructions, the other training with GAN discriminator losses that produces better sample quality. Here they mention that their method does not work to sample diverse sets of pre-images, citing this would be great motivation for the current paper!\n* Lucic et al. (https://arxiv.org/abs/1903.02271): uses SSL to improve the quality of a generative model by clustering data using SSL + small amounts of labeled data and training a GAN conditioned on these learned clusterings. It’s similar in spirit to your work but it’s a GAN and leverages conditioning on the discrete cluster instead of continuous representation embedding. \n\n3: Conditioning a diffusion model on representation h\n   * This is the first time S(h) appears and it is not yet defined (you could define it when you introduce notation in the intro)\n   * The density p(x) is not a good statistic to measure whether a sample is realistic looking (see: https://arxiv.org/abs/1810.09136), so letting S(h) be the set of sufficiently high likelihood images will not yield good lookign samples. Similarly, I don’t think the description of the goal as sampling from M \\int S(h) makes sense when reasoning over high-dimensional continuous spaces.\n   * Do you actually use batchnorm and introduce dependences between elements in the batch or use conditional group norm where you modulate the scale and bias of each feature map?\n   * You could use an architecture closer to the super-res based approach by having a neural network that outptus a representation-dependent bias in only the first layer (corresponding to broadcasting the vector spatially after a transformation). I believe your approach is a generalization of this, but I’d be interested to see baselines of how different forms of conditioning impact the results.\n   * Need to mention what dataset you are training the diffusion model on. ImageNet?\n\n4: Experiments\n   * In terms of not memorizing, Figure 2 & Figure 17 are great. The argument in Figure (b) that the samples are “close visually” I don’t find so convincing. In particular the contrast of the diffusion models is often very different from the original, and for the semantic segmentation input, changing the color of a pixel corresponds to changing the class label, and so the resulting samples I’d think about as *very* different even if to our eye they have somewhat similar characteristics. For example, semantic segmentaiton masks on roads will never have splotches of other texture Fig2b, right, and if you trained a diffusion model on segmentation masks it likely would not generate images like this. IMO, Fig 2b highlighted that we should not expect these visualization methods to work well for really OOD inputs (like segmentation masks)\n   * Figure 3 should be listed as a Table not a Figure. Fig 3a results are neat and compelling! \n\n5: Analysis\n   * Fig 4 is a neat demonstration of how different heads and methods constrain the pre-iamge differently! It would be interesting to pair this with a supervised model to see whether the small differences in the representations of the sampled inputs confuse a classifier that is trained on the learned representation. \n   * Fig 6: it’d be useful to note where the classifier fails on the adversarial example in each row (e.g. by having a box around each image with a color red if it’s incorrect). I think there’s also a great opportunity for a behavioral study to show the utility of your method: see whether humans are better able to predict the classifier output by visualizing your reconstructions instead of the original adversarially perturbed input.\n   * Fig 7: this was an interesting idea and set of manipulations but I dont’ find the results very convincing. It does seem like this method is good at plugging in different backgrounds, but the clothes (to my eye) in the bottom row don’t match well with the initial set of clothes and the dogs in that image are of poorer quality than most your samples. I think there may be more convincing sets of manipulations in latent space that could highlight this method, e.g. taking an average of a bunch of dog images with clothes, and dogs without, and learning a single vector to add in latent space.\n   * Table 3b could use more explanation / discussion in the text as it’s the only quantitative result that involves your representational-conditional model (that’s not the unconditional FID)\n\n7: Reproducibility\n   * Would be great to have accessible code to understand some of the model details (like how you handle normalization exactly).\n\nAppendix A:\n   * This is a great appendix and highlighting the failure mode of prior methods in the main paper would be useful to motivate your approach. \n   * A.2 / Eqn 2: I don’t follow your logic here. The Jacobian of a deep network will change as the input is changed, and thus the space that the iterates span can be D-dimensional if you take many steps\n   * A.3: prior work has found that randomc rops + jittering can be very important for the quality of results. Did you try that here as well? Having seen the quality of earlier results (e.g. DeepDream, Mahendran &Vedaldi), I’m surprised how bad the results are in Fig 8\n\n", "broader_impact_concerns": "There was no broader impact statement. With any work on generative models, I believe it's important to carefully detail what datasets you are using to train it (e.g. ImageNet for your conditional diffusion models), and highlight how you do or don't expect it to fail on OOD inputs. In particular in your work you argue that your inversion works well on OOD inputs, but if images are far from ImageNet examples I do believe the method performs worse. There may also be many other naturalistic images that have the same hidden representaiton as an ImageNet image but are not in the dataset you trained on, and thus inversions will be heavily biased to be closer to ImageNet samples. ", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "AYDiwkvMYq", "reviewer": "Reviewer_BkYX", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\n- This paper touches on an interesting topic: the interpretability of learned representations is and remains an important problem, especially for large-scale SSL models such as CLIP, DINO, etc. Gaining insights into these representations via visualizations has potential (and is also an established approach).\n\n- Good qualitative samples and extensive experiments.\n- Analyzing the image manipulation capabilities of the approach is an interesting extra and seems promising (albeit more for a dedicated generative model).\n- The work is very related to the concurrent work unCLIP (a.k.a. DALLE-2, https://arxiv.org/abs/2204.06125), but with a different focus: it does not focus on the generative capability/synthesis quality, but rather on the interpretability that comes with this application.  \n  -  A side question, since this work also does some synthesis (especially in the appendix): How does the choice of SSL mesh affect the **generative** capability of the model? That is, suppose I want to build a generative model using RCDM. Which network should I choose, and how does this affect the generative application? \n\n**Weaknesses**\n-  The approach is not new, other conditional-generative approaches (as mentioned in the section on related work) have addressed the same problem. The paper would benefit from a direct comparison of these approaches, and it would be worth discussing whether such a comparison could also make clear which of the approaches has which advantages and disadvantages. (1)\n-  A somewhat mixed comment: I think this paper suffers a bit from trying to do the balancing act between being an \"interpretability paper\" and showing (very decent) generative results (mainly in the appendix). In my opinion, it would benefit greatly from focusing on only one of these two aspects. The latter also allows for very interesting insights, e.g. what kind of \"representation network\" is needed to achieve the best generative performance. (2)\n- The conditioning mechanism via the conditional batchnorm seems to be a bit ad-hoc. Is this more powerful than the mechanism already built into the ADM Unet (one would just have to leave out the \"class embedding\" layer)? (1)\n- I am not quite sure what exactly we learn in this paper: What **new** insights does the method offer, especially compared to [1] and [2] (when trained with a similar computational budget) (1)?  How can we be sure that the DM does not \"ignore\" the conditioning (and invent new details during synthesis) (1)? \n  - $\\rightarrow$  Is there any way to \"prove\" that the initial sample contains the same information as the representation from which it was predicted? (1)\n- Using a deterministic mapping from the input to the latent diffusion space (such as an inverted DDIM sampler, e.g., in https://arxiv.org/abs/2105.05233), this method could find an **explicit representation** of the invariances (rather than visualizing them implicitly via multiple decodes). I think it would be interesting to have a visualization of this. (2)\n- I would be very interested to see results obtained with classifier-free guidance (https://openreview.net/pdf?id=qw8AKxfYbI). How do such \"post-hoc\" truncation procedures (which also exist for other types of generative models) change/influence the interpretation of the results? (2)\n\n*References*\n- [1]  Charlie Nash, Nate Kushman, and Christopher KI Williams. Inverting supervised representations with\nautoregressive neural density models. In The 22nd International Conference on Artificial Intelligence and\nStatistics, pp. 1620–1629. PMLR, 2019.\n- [2] Robin Rombach, Patrick Esser, and Björn Ommer. Making sense of cnns: Interpreting deep representations\nand their invariances with inns. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow,\nUK, August 23–28, 2020, Proceedings, Part XVII 16, pp. 647–664. Springer, 2020\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents a technique for visualizing representations of fixed, pre-trained, and mostly self-supervised (SSL) image networks. Specifically, the learned representation of a neural network can be mapped back on the image space by using a conditional diffusion model, and by representing multiple samples corresponding to the same SSL representation, the invariances learned by the network which shall be interpreted can be visualized.\n", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "- See weaknesses, annotated as follows: \n\t- (1) needs to be answered in order to recommended the paper for acceptance\n\t- (2) not absolutely required, but would strengthen the work\n", "broader_impact_concerns": "Since the paper provides a fairly powerful generative approach, it also includes all the considerations one needs to make when publishing such models. Although this has been discussed in specific publications, I think the paper should at least briefly summarize the potential societal problems associated with such models (spread of misinformation, deliberate manipulation, misrepresentation of data, ...).", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "9VG3R0ll3C", "reviewer": "Reviewer_GRCY", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The paper gave a nice overview of related works, although at times they seemed to cover too much ground. For example, I didn’t think it was necessary to cover all families of generative models (e.g. PixelCNN++), as they weren’t relevant to the method and were distracting at times.\n- The paper also conveyed a lot of empirical results (in terms of samples, interpolations, etc.) throughout the main text and the appendix. The authors clearly demonstrated that RCDM worked well as an instance-conditional generative model for their purposes.\n\nWeaknesses:\n- The exact contribution of this work was unclear to me. The RCDM model is a pre-existing conditional variant of OpenAI’s diffusion model from (Dhariwal & Nichol 2021), just with a very minor architectural change (where the Group Normalization activations have been replaced with conditional batch normalization layers). Thus I’m not sure that this is a contribution of the work, and the fact that its quality is on-part with “state-of-the-start models” as mentioned on page 2 is not surprising (as well as the analysis in Section 4).\n- This implies that the main contribution of the work is centered around analyzing what kinds of information is encoded in SSL representations/embeddings, starting in Section 5. However, I felt that this was actually the weakest part of the paper. Although the authors examine 4 SSL algorithms (VicReg, Dino, Barlow Twins, SimCLR)\n- Parts of the text/paper were unclear. For example, what is the distinction between n SSL “projector embedding” and “SSL representation?” This wasn’t clarified until Section 5, which seems to be too late in the main text.\n- Parts of the exposition could definitely be improved. See “Requested changes” section.\n- There are some examples of overclaiming here: e.g., the OOD views generated by RCDM are noticeably worse quality than those from in-domain (Figure 2).\n- I think the authors should demonstrate that their approach works on datasets beyond ImageNet, since the method is pitched as a “general-purpose, plug-and-play visualization tool.” This goes beyond simple OOD synthesis experiments in Figure 2. I would expect the method to work well since diffusion models tend to work well on a variety of different (and complex) image datasets.\n- One thing that wasn’t clear to me was what the takeaway should be, even after visualizing such SSL representations. I have elaborated on this in the “Requested changes” section.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper aims to understand the information captured by neural network representations (specifically, self-supervised learning (SSL) methods) by going beyond the standard downstream classification task, and instead visualizing the representations with a class-conditional diffusion model (RCDM). They find that this RCDM helps to uncover the fact that SSL representations are robust to adversarial perturbations and demonstrate invariance to certain kinds of transformations, though with varying degrees across SSL methods.\n", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "- As mentioned in the “Weaknesses” section, I think the paper would benefit from a more clear explanation \n- In Section 3, the authors discuss how to develop an effectively “instance-conditional” generative model which takes in a representation h and learns a conditional probability distribution p(x|h). This section was quite confusing. I think the authors should either commit to a full-length discussion on (conditional) diffusion models, how they are trained, how they generate samples, etc. and refer to their schematic in Figure 1, or choose to leave out such details entirely. The current hybrid approach in Section 3 where the details are extremely high-level actually made it harder to understand what was going on, even though at the end of the day the point of the section was to elaborate upon the authors’ decision to replace a particular activation layer.\n- On that note, I think Figure 1 could be explained in more detail. The authors contextualize RCDM within 2 alternative frameworks (unconditional generation, gradient-based controllable generation), and it would have been helpful to highlight this more in the main text.\n- How much are the results specific to ImageNet? While it is important to evaluate SSL methods on this dataset, I think it’s important to demonstrate that the method works well on other datasets used in SSL such as CIFAR-10/CIFAR-100, CityScapes. \n- Although the authors performed a wide variety of experiments with the visualization of learned representations, the conclusions they drew seemed somewhat inconclusive. For example, if SSL representations do preserve information such as the object scale and color palette of the background, what are the implications of this? What should we do with the knowledge that SSL representations seem to demonstrate some signs of adversarial robustness to perturbations in the input space? I would have appreciated if the authors could have contextualized their findings better for either recommendations in practice, developments of some quantitative metrics to compare between representations, etc. At the moment, the experiments are largely qualitative visual comparisons, which are hard to assess.\n- Along this line, I think the paper would benefit from additional experiments that use the insights drawn from RCDM to verify empirical phenomena in downstream tasks. For example, if it is indeed the case that the SSL representations are “invariant to vertical shifts,” does this observation still hold in a downstream task where the test images are shifted downwards? Experiments along these lines would be extremely helpful for convincing the reader that the qualitative observations made from using a tool like RCDM is actually useful/predictive of certain properties of the representations in practice. \n- Addition of broader impacts section (see below).\n", "broader_impact_concerns": "The authors did not include a Broader Impacts statement in their submission. They should discuss the potential harmful impacts of generative models, especially with their potential for misuse in generating deepfakes and synthetic examples of people that may violate their privacy.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Many thanks to the authors for working with the reviewers on getting extra experiments and clarifications into the subsequent versions of this work. 

The main point of contention in the reviews is regarding the claim from the submission, that the insights uncovered by the visualization tool are good for a downstream task/experiment. I appreciate that the authors added some figures & experiments showing that in some circumstances the visualization tool can show whether the representation will be useful (for classification) or not. I am not necessarily convinced that using RCDM is in fact "easier" than training kNN (as the authors claim in their comments).  

Taking a look at Figures 12 and 13, it is hard to know whether the conclusions spelled out by the authors are cherry-picked based on those samples, even if the discussion at the end of Section 4 suggests there are some metrics to bolster those claims.

So, in rendering my decision, I am guiding myself on the two TMLR policies:

1. "Are the claims made in the submission supported by accurate, convincing and clear evidence?": It is clear that the claim about the usefulness of the visualization tool towards a downstream task is not believed by the reviewers to be very supported by accurate, convincing and clear evidence. But the authors have shown some good-faith attempts at clearing this up.

2. "Would at least some individuals in TMLR's audience be interested in knowing the findings of this paper?": I believe a non-trivial number of TMLR readers would be interested in knowing the findings of this paper.

For this to be accepted as is, I would want the authors to refine the experiments in figures 12 & 13 and the discussion at the end of Section 4:

  * At the very least, I would like to see a few more numbers and analysis. More specifically, I'm concerned that the Dino linear probing result on page 6 may not generalize at all to other models, so it'd be good to see more data on this (ideally with as many encoders as the authors used for the other experiments). I think this could go a long way towards understanding the extent to which the proposed method is "useful".
  * If those results don't quite generalize beyond Dino, I think the paper can still be accepted, provided the authors downplay the claims about "usefulness" (which were the most contentious with the reviewers).

Beyond that, I think the authors have done a good job at answering the reviewers' concerns.
Decision: Accept (poster)