Perceptual Regularization: Visualizing and Learning Generalizable Representations
Authors: Hongzhou Lin
Joshua Robinson
Stefanie Jegelka
Venue:  (2020)
Keywords: regularization
representation learning
visualization

Abstract: A deployable machine learning model relies on a good representation. Two desirable criteria of a good representation are to be understandable, and to generalize to new tasks. We propose a technique termed perceptual regularization that enables both visualization of the latent representation and control over the generality of the learned representation. In particular our method provides a direct visualization of the effect that adversarial attacks have on the internal representation of a deep network. By visualizing the learned representation, we are also able to understand the attention of a model, obtaining visual evidence that supervised networks learn task-specific representations. We show models trained with perceptual regularization learn transferrable features, achieving significantly higher accuracy in unseen tasks compared to standard supervised learning and multi-task methods.

Under review as a conference paper at ICLR 2020
VISUALIZING REPRESENTATIONS BY PERCEPTUAL REGULARIZATION
Anonymous authors Paper under double-blind review
ABSTRACT
A deployable machine learning model relies on a good representation. desirable criteria for good representation is to be interpretable by humans. We propose a technique termed perceptual regularization that enables visualization of the latent representation. By visualizing the learned representation, we also able to understand the attention of a model, obtaining visual evidence that supervised networks learn task-specific representations. Moreover our method provides direct visualization of the effect that adversarial attacks have on the internal representation of deep network. We show how this can be used systematically perform latent space interpolation to modify semantic features.
INTRODUCTION
Deep neural networks are known to be vulnerable, small changes known as adversarial attacks can completely change the model's prediction (Szegedy et al. 2014 Nguyen et al. [015 Kurakin et al. 2017). There is a rapidly growing body of work on how to perform effective ad vdvessaiial attacks, and how to defend against them (Papernot et al., 2017 Madry et al.) 2017 Athalyy et al. 2018). On the one hand, adversarial examples are named as such because they as able to trick a deep network using changes imperceptible to humans. On the other hand, the model makes predictions based on its internal representation (i.e. what the model perceives) which may be very different from human perception. Indeed, the existence of adversarial examples is evidence that human perception and machine perception are misaligned, and the internal representation of a model and a human differ significantly. The motivating question we would like address are adversarial examples really adversarial examples from the model's perspective?
In order get insight of this question, we extend an existing visualization technique from Dosovitskiy & Brox (2016b); Johnson et al. (2016); Dosovitskiy & Brox (2016a), stacking a decoder head on top of the model's 's latent representation with the aim of providing human interpretable reconstruction, see Figure I In particular, we train the decoder jointly with the network, terming the resultant objective function perceptual regularization.
similar strategy has also been explored in [Sabour ee al. (2017); Qin et (2019) for Capsule networks, and exploiting semi-supervised (unlabeled) data in Lasserre et al. (2006); Grabner et al. (2007); Larochelle & Bengio (2008): ) Le et al. (0118). One of the key insights we bring in work is that simple decoder head term is all that is needed to yield significant visual insight into a range of questions about the representations learned by deep networks.
Visualization is an active research area, with various standard techniques such as saliency maps Simonyan al. (2014) among others Zeiler & Fergus (2014); Mahendran & Vedaldi (2015) Sprin- genberg et al. (2015): Zhou et al. (2016) being used to visualize representations by inverting them. However, reconstructing the original input from the representation is computationally expensive. An alternative way is to introduce a forward decoder from latent space to image space, aiming to globally invert the representation. This technique has been used in Dosovitskiy & Brox (2016b);Johnse et al.) (2016); Dosovitskiy & Brox (2016a) and our method follows this line of work.
The main contributions of this paper are:
Extending the existing visualization method as a regularization method for jointly learning a visualization for deep representations.

---

Under review as a conference paper at ICLR 2020
Using this method visualize how the attention of model focuses only aspects that relevant for prediction. More generally we give promising evidence to suggest that our visualization method is of interest for model diagnostics and interpretabilityy.
Using this visualization method to shed light what effect adversarial attacks have on model.
Using adversarial attacks to identify direction vectors in the latent space that correspond to certain semantic features. We show that the decoder network can then be used synthesize images with modified semantic features.
2 PERCEPTUAL REGULARIZATION
We begin by introducing perceptual regularization. Consider deep image classification model such as AlexNet (Krizhevsky et al. 2012). VGG-Net (Simonyan & Zisserman, 2015), or ResNet (He et al. 2016). It usually consists of two stages: several convolutional layers, followed by dense layers. The convolution layers can be viewed as a feature extraction process which we denote by F. transforming images from the input space A' into latent space Z. The dense layers C: 2 + ) act classifier on the latent space. In order to regularize and visualize the latent representation, we introduce a decoder D 2 X on top of the latent space Z which we jointly train with the classification model. The architecture is shown in Figure]
F
reconstruction X'=XX)
Input X
class labels Y-H(()
Figure I: Perceptual regularization: we stack a decoder on top of the feature map and jointly train it with the classifier.
More precisely, we denote the classification model by H = o F and term the reconstruction P = D F a perception of the model. The regularization strategy we introduce is to jointly train H and P together by minimizing
min E X [((H(X). -XX))- P.H
(Perceptual regularization)
classificatin loss
reconstruction loss
The first term the standard classification loss and we take to be the cross-entropy loss throughout the paper. The regularization term is the & reconstruction loss scaled by hyper-parameter À >> O. Intuitively, the decoder tries to reconstruct the original image based on the same information the classifier uses for prediction. In particular, if the feature map F preserves all the information from the input, the perception should be able to perfectly recover X. Conversely, if lot of information has been thrown away, then we will get poor reconstruction and the regularization loss will be high. Therefore, our regularization biases the learned representation to maintain more information about the input compared vanilla supervised learning. This will be useful if one would like to reuse the learned features later for other tasks.
Notice that the regularization term vanishes when P is the identity mapping, in which case the feature map F is bijective. This can happen when the model is very powerful and therefore able to memorize any dataset. However, this situation is less interesting in the sense that a good representation should efficiently compress the information. This idea has been clearly understood with the information bottleneck objective, for which it is known that F(X) approximates the minimal sufficient statistic for Y given X (Shamir et al.) 2010). Therefore, to constrain the expressiveness the feature map, impose bottleneck structure in the network architecture. More precisely, the dimension of the
2

---

Under review as a conference paper at ICLR 2020
latent space 2 is made significantly smaller than the dimension of the input space X. This enforces the feature map to compress information and makes it very unlikely to be bijective.
As well as regularizing the model, perceptual regularization also provides a natural way to visualize the latent space. In particular, the perception P is designed decode the feature map F. which is very useful for model understanding, In the next three sections we begin to explore the insights that can be drawn using pereeptual regularization.
VISUALIZING FEATURES LEARNED BY A DEEP NETWORK
Extensive research have been conducted on attention- based models in object recognition and image captioning Bahdanau et al. (2014): Karpathy & Fei-Fei (2015): Ba et al. (2015); Gregor et al. (2013): Xu et al. (013 y Jaderberg et al. (2015):HH et al. 2017). where the attention mechanism is explicitly introduced as spatial map highlighting image regions. second application of perceptual regularization is to visualize what a classifier's 's attention focuses on. In particular, if a model is trained to predict the object the middle of an image, is commonly helieved that the learned representation "overfits" to the training task by focusing only on the central part of the image. This can be cause of negative transfer- when reusing learned representation for a new task actually hurts performance (Pan & Yang 2009, Kuzborskij & Orabona 2013). We show our visualization method can help understand when a representation will cause negative transfer. All experimental details, including architecture and hyper-parameters are given in full in the appendix.
1936
25
10
223
881
6
2
0
2
8
12858 6757
67
6
15
2
5
6
7
7
6
WIT 2066
7
17
2 6
7
7
$2388238888
65540
8
2
8 8
6
5
0
19 9
84
222559
299
9 8
2
5
9
2
Figure 2: Left: Original images from SVHN. Right: The model's perception of the images.
3.1 THE ATTENTION PROBLEM
It has long been intuitively understood that the representations in the final layers of a supervised deep network will only focus on aspects of the data relevant to its prediction task (Long et al. 20155] Yosinski et al.
2014, Alain & Bengio, 2017). In this section we demonstrate on the SVHN dataset (Netzer et al.
2011) that this phenomenon of focused attention can be visualized using perceptual
regularization.
The SVHN dataset consists of images of house numbers and the task is to predict the number in the center of the image. In particular, an image could contain multiple numbers but the label is given according to the middle one. A natural question that remembers numbers that are not in the center? To answer this question, we train CNN with perceptual regularization and then use perception to visualize the attention of the model, i.c. given any input X, we visualize P(X). As shown in Figure 2 our visualization technique shows that the model learns a representation that focuses on the middle of the image, and forgets what is at the edges of the image. This provides additional evidence to support the observation that a supervised model learns a representation specific to its prediction task. This is fine if there is only one task of interest, but makes transferring features between tasks problematic. For example, in our case, the representation would be completely ineffective for identifying the digit on the left or right hand sides.
will a supervised model learn a representation
3.2 PERCEPTUAL REGULARIZATIU TO CONTROL ATTENTION
The cause of this restricted attention problem is the guidance given by the labels. In this section, demonstrate that perceptual regularization can be useful to reduce the problem of overfitting learned features to the training task. In particular, our method allows for a smooth interpolation between supervised learning and unsupervised learning. In the extreme case, when À 0, recover
3

---

Under review as a conference paper at ICLR 2020
040
818
881 6
4 221
6
7
1
12 677
6
2
Y 7 1717
7
12
18
18866150
5
21561
245159
( = 10"'.Ace 88.5%.
(b) À = 10"1,000cA Acc: 87.9%.
(c) A= 10" A.c: 87.8%.
(d) A= 10" Acct 81.3%.
Figure 3: Visualizing the model's attention. Acc is the accuracy of the classifier on the test set.
the standard supervised learning setting, and when À oo we recover the classical auto-encoder objective. Hence, the parameter À can control how much information we would like our model's representation to keep.
We run experiments on the SVHN dataset by varying the regularization parameter A. We keep the architecture fixed (see Appendix B.3 for more details). Figure shows different regimes of features for different values of X. There are four main regimes:
Tiny (A 10-1, Figure aa): Since À is extremely small the contribution to the loss from the decoder branch is so negligible that there not much incentive to learn a good decoder and the visualization breaks down.
Delicate (10-1 < À < 10-2, Figure 3 b))) : A good decoder is learned and the bias injected into the learned feature map by the regularization term is small. The number in the center well reconstructed, but the edges of the image are not and the colors are missing. It is in this regime that we are able to see the restricted attention of a classifier.
Intermediate (10-2 < X < I, Figure 3)))): We obtain more faithful reconstructions compared to the "delicate" regime. In particular, the numbers at the edges are reasonably reconstructed but colors are still not well captured.
Large (A 1. Figure d)): The decoder is very good, both numbers and colors are recovered. The regularization has had a significant effect on the nature of the learned features. Since these features retain much more information than for smaller À they are more likely to transfer well to new unseen tasks.
Clearly the choice of X heavily affects the features learned by the model. The larger À is, the more information is preserved the features: small values of À only preserve the number in the middle of the image, intermediate values can also reconstruct the boundary of the image, and only for large À is the input's precise color correctly reconstructed. is also important to note that the accuracy on supervised task decreases as À increases, but at a relatively slow rate. This is to be expected. In the limit À + oo there is no longer any incentive to perform good classification, only reconstruction, An interesting future direction will be to understand why certain features are harder to learn than others, for instance, in our example why the color is recovered later than numbers along the edges.
important to note that perceptual regularization introduces an inductive bias towards learning representations that retain lot of information about the original input, which are different from what learned without it. one wanted to avoid this bias then one idea is to learn the feature map first, then learn the visualization decoder separately, with the feature map frozen. This strategy works reasonably well when the network's s size is small. But, as the network gets more complicated becomes computationally difficult to decode the feature map. Hence, as an alternative solution, train the decoder jointly with the feature map. What we have shown is that although the bias induced by the regularization is an apparent drawback, it can in fact be turned into strength by using À to control the behavior of the feature map.
Finally, the specific values of quoted are meant only serve very rough guide about trends. Making the magnitude of the reconstruction loss and classification loss comparable is often effective locate the "delicate" X.
HOW DO ADVERSARIAL ATTACKS AFFECT A CLASSIFIER'S 'S PERCEPTION?
Having illustrated how the decoder network can be used to study the information content of a learned representation, now turn attention to adversarial examples. Adversarial examples seem quite mysterious, and point to a way in which human and machine perceptions are misaligned (Han et al. 2019 Engstrom al. 2019). While intuitively reasonable, understanding of this phenomenon
4

---

Under review as a conference paper at ICLR 2020
has been limited by the difficulty of observing the effect of adversarial attacks the internal representation model (Zhagg & Zhu 2018 Olah et al. 2018). In this section we explore the first application of our perceptual regularization: to provide human understandable visualizations of model's internal representation adversarial examples.
First, we briefly recap how adversarial examples are obtained. Given a classification model H and sample (X, we aim fool the model's prediction by slightly perturbing the input X. We constrain the perturbation to be in small neighborhood A (X) of X, which is usually defined as an lp ball. Specifically, we look for
X adv E arg max (H((X',)) X'EA(X)
While many different strategies have been proposed to adversarially attack the model (Goodfellow al. 2015 Moosavi-Dezfooli al. (2016) Kurakin
2017 Athaaye al. 2018), focus on the projected gradient descent (PGD) attack (Madry et al. 2017) throughout this paper. Specifically, we set (XX) to be an fo ball and the adversarial example X obtained by performing 100 iterations of PGD attack. An example of an adversarial attacks can be found in the first row of Figures 4 All experimental details, including architecture and ayper-parameters are given full in the appendix.
+.02
X "female" 99.92% confidence
X "male" 99.08 confidence
PGD attack
| P
P
Figure 4: First row: by adding an imperceptibly small vector whose elements are obtained from performing PGD attack, we can change the classification of the image. Second row: by applying perception, we can understand the model's "incorrect" prediction.
By design, adversarial examples are hard to distinguish from the original image with human per- ception. The perceptual regularization architecture gives a classifier, and a separate branch that reconstructs the original image. We adversarially attack the classification branch 7/ Co F using (I) to obtain Xadv- If, instead of viewing X and X we look at the model's perception of each image P(X) and P P((aav), clear disparity emerges between how the model understands the two examples. What is extremely striking is that the model's perception and incorrect classification are semantically consistent to the human eye. Put simply: when the model misclassifies an image of female as male, the perception of the adversarial image really looks like male to a human, as shown Figure when the model misclassifies an image of a smiling face as not smiling, the perception of the adversarial image really looks like it is not smiling, shown in Figure 5
We provide further experiments of our method on the MNIST (LeCun et al. 1998 and CelebA (Liu et al. 2015) datasets. Our perception decoder allows one to visualize the effect of an adversarial attack deep network's representation. For CelebA train three different classifiers for three different binary classification tasks Drrglaases/SmmlinnGGenndd
All experiments support the same observations made in this section. See the appendix for many examples.

---

Under review as a conference paper at ICLR 2020
PGD attack
P(X) "smile"
P(Xah)) "not smile"
Figure 5: Images after applying perception the task of classifying smiling face, understand the model's "incorrect" prediction. The corresponding original images can be found in Appendix A.
This shows that given the model's feature representation, it is making prediction that appears reasonable on adversarial images. In other words, the representation learned by the model is very different from human perception. It is also interesting that the images obtained from perception (Figure 5 in the second row of Figure 4 ) are simultaneously close to each other and yet highly distinguishable in the characteristics most important for prediction. This gives interesting visual insight into the way in which adversarial attacks find a short path to a decision boundary. For example, Figure has several subtle modifications: lighter eyes, squarer forehead and jaw, thicker nose. The changes are very targeted, capturing the essence of the global feature.
This begs the question: can adversarial attacks be used to identify and modify high-level features? Global features are high level human-inte -nttaainiterrettbbll features such as, "wearing glasses". "is smiling", or "is female").. There has been much previous work on learning disentangled representations for which the latent variables correspond to disentangled human-interpretabl features (Tenenbaum & Freeman, 22000) [Chn al. 20166] Higgins et al. 2017). To then modify given global feature one needs to figure out which latent feature (or which combination) to adjust. However, the task of finding the "correct" latent direction is non-trivial.
ADVERSARIAL ATTACKS FOR LATENT SPACE INTERPOLATION
Our visualization method yields a novel way for finding directions in latent space corresponding to given global feature. Traditional approach applying generative adversarial network such as Radford et al. (015); ; Karras et al. 2019) involve significant manual effort to identify which latent direction corresponds to a given semantic feature. Our method provide a more straightforward way to obtain this direction by performing adversarial attacks. The recipe is as follows:
Train a model with perceptual regularization,
Given X, obtain Xnav by performing an adversarial attack toward the target label (c.g. smiling/wear eyeglasses),
The latent space direction corresponding to this global feature is = F(Xasv) - F(X),
Synthesize images with adjusted global feature C(F(X) + a%) for some a.

---

Under review as a conference paper at ICLR 2020
Larger values of will make larger adjustments the global feature. Figures and give simple examples of this for a = 1. Figure soows images synthesized by following a linear path through latent space for a range of values of E [0, 2]. The results show that technique allows precise modification of the target global feature, leaving other aspects of the image unaltered.
Figure 6: First row: smiling to not smiling, Second row: glasses to glasses. Third row: female to male. Each column corresponds sequentially to a E {0, 0.25, 0.5, 0.75, 1, 1.25, 1.5, 1..
We believe it is possible to obtain more realistic images using recent progress GANs, but we leave this for future work. Finally, we stress that our method requires labeled data while GANs not. However, in situations where one has labeled data, our method provides promising alternative since it does not require human intervention.
DISCUSSION AND CONCLUDING REMARKS
In this paper introduced perceptual regularization, method for visualizing representations and show that it can be used for a variety of purposes including: assessing the informational content of representation, visualizing the effect of adversarial attacks on the latent representation, and identifying semantically meaningful latent space direction vectors. To achieve this, our method combines generative and discriminative model. There has been previous work on the idea of combining generative and discriminative models (Lasserre et al.) 2006 Grabner et al. 2007 Larochelle & [Bengio] 20088] [e al. 2018), however these works have focused on idea using such a model to exploit semi-supervised (unlabeled) data. To the best of our knowledge our work is the first to identify the generative plus discriminative formulation as an effective method for practical, visual appraisal of the representation learned by model. We use this visualization to understand adversarial examples and study the attention of model. We also identify this formulation as suitable for learning representations that effective previously unseen tasks.
In this work we made the choice to consider the & loss as metric between images. However, the specific nature of the injected bias particular to &g: it permits lots of small errors, but penalizes large differences severely. It would be an interesting future direction to consider different metrics and consider what kind of representations are learned by regularizing with different generative model objectives such as GANs, VAEs, or InfoMAX (Goodiellow et al. 2014 Kingma & Welling 2014 Hjelm al. 2019).
Another interesting question is to understand what conditions are necessary to obtain good visu- alizations. We identify two possible factors. First: model complexity. As the task becomes more challenging we may need more powerful decoder and refined training strategy (e.g. applying methods from the GAN literature). Second: human meaningful features. There has been work suggesting that more complicated datasets have small but highly predictive features that humans do not notice (Tsipras et al. 2018). The model may learn to use these features for prediction. this case our visualizations may also not be human meaningful. Nevertheless, could be a useful certificate for whether the model learns human meaningful features.
visualization method
This raises the question of how to learn robust features that generalize across multiple tasks. There is evidence suggesting adversarial training may help learn more "human- aalinned" features. But so far work adversarial training has mainly focused supervised learning. We suspect that this will not
7

---

Under review as a conference paper at ICLR 2020
be sufficient to learn transferrable features. It would be an interesting next step combine perception regularization with adversarial training.
Our visualizations raise broader point about the supervised learning paradigm: models insist pigcon-holing every imput into one of fixed number of classes, no matter how out of sample the input is. Consider the following toy example: taking X to be random noise and computing the adversarial attack adv for a model H. trained on MNIST, then P(X looks like a digit to the human eye (see Figure 7.. This is undesirable and is sufficient simply add 11th "other" class. This is long way from the dynamic way that humans create and learn new classes of objects.
To conclude, have shown that perceptual regularization is promising approach answering many important questions in machine learning. However we believe this is just the beginning, and view perceptual regularization as broadly useful tool for model diagnostics with many more uses to be uncovered in the future.
3 3
7
Figure 7: First row: Xadv for random noise X. Second row: P(Xadv). Even for what is essentially random noise, the model insists on perceiving something that semantically looks like a number.
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes. In International Conference on Learning Representations (ICLR) Workshop. 2017.
A. Athalye, N. Carlini, and D. Wagner. Obfuscated gradients give a false sense of security: Cir- cumventing defenses to adversarial examples. In International Conferences Machine Learning (ICML), 2018.
J. Ba, V. Mnih, and K. Kavukcuoglu. Multiple object recognition with visual attention. In Interna- tional Conference on Learning Representations (ICLR), 2015.
D. Bahdanau, K. Cho, and Bengio. Neural machine translation by jointly learning to align and translate. In International Conference on Learning Representations (ICLR), 2014.
X. Chen, Y. Duan, R. Houthooft, Schulman, I. Sutskever, and P. Abbeel. Infogan: Interpretable representation learning by information maximizing generative adversarial nets. In Advances in Neural Information Processing Systems (NIPS), 2016.
A. Dosovitskiy and T. Brox, Generating images with perceptual similarity metrics based on deep networks. In Advances in Neural Information Processing Systems (NIPS). 2016a.
A. Dosovitskiy and Brox. Inverting visual representations with convolutional networks. In IEEE conference on computer vision and pattern recognition (CVPR), 2016b.
L. Engstrom, A. Ilyas, S. Santurkar, D. Tsipras, B. Tran, and A. Madry. Learning penceptually-aligned representations via adversarial robustness. arXiv preprint 1906:v/906.00900 2019.
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and
Y. Bengio. Generative adversarial nets. Advances Neural Information Processing Systems (NIPS), 2014.
1. Goodfellow, J. Shlens, and C. Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations (ICLR), 2015.
H. Grabner, P. Roth, and H. Bischof. Eigenboosting: Combining discriminative and generative information. In IEEE conference on computer vision and pattern recognition (CVPR), 2007.
K. Gregor, I. Danihelka, A. Graves, D. Rezende, and D. Wierstra. Draw: A recurrent neural network for image generation. In International Conferences on Machine Learning (ICML), 2015.
C. Han, W. Yoon, G. Kwon, S. Nam, and D. Kim. Representation of white-and black-box adversarial examples in deep neural networks and humans: A functional magnetic resonance imaging study, arXiv preprint 1905v 000422, 2019.
K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In IEEE conference on computer vision and pattern recognition (CVPR), 2016.
K. He, G. Gkioxari, P. Dollár, and R. Girshick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, 2017.
I. Higgins, L. Matthey, A. Pal, C. Burgess, X. Glorot, M. Botvinick. S. Mohamed, and A. Lerchner. beta-vae: Learning basic visual concepts with a constrained variational framework. In International Conference on Learning Representations (ICLR), 2017.
R. D. Hjelm, A. Fedorov, S. MoriiiMarchh,don, K. Grewal, P. Bachman, A. Trischler, and Y. Bengio. Learning deep representations mutual information estimation and maximization. In International Conference on Learning Representations (ICLR), 2019.
M. Jaderberg, K. Simonyan, A. Zisserman, and K. Kavukcuoglu. Spatial transformer networks. In Advances in Neural Information Processing Systems (NIPS), 2015.
J. Johnson, A. Alahi, and L. Fei-Fei. Perceptual losses for real-time style transfer and super-resolution.
In European Conference Computer Vision (ECCV), 2016.

---

Under review as a conference paper at ICLR 2020
A. Karpathy and L. Fei-Fei. Deep visual-semani alignments for generating image descriptions. In IEEE conference on computer vision and pattern recognition (CVPR), 2015.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pp. 4401-4410, 2019.
D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR), 2015.
D. P. Kingma and M. Welling. Auto-encoding variational bayes. In International Conference on Learning Representations (ICLR), 2014.
A. Krizhevsky, I. Sutskever, and G.E. Hinton. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems (NISS), 2012.
A. Kurakin, I. Goodfellow, and S. Bengio. Adversarial machine learning at scale. In International Conference on Learning Representations (ICLR), 2017.
I. Kuzborskij and F. Orabona. Stability and hypothesis transfer learning. In International Conference on Machine Learning, pp. 942-950, 2013.
H. Larochelle and Y. Bengio. Classification using discriminative restricted boltzmann machines. In International Conferences on Machine Learning (ICML), 2008.
Lasserre, C. Bishop, and T. Minka. Principled hybrids of generative and discriminative models. IEEE conference on computer vision and pattern recognition (CVPR), 2006.
L. Le, A. Patterson, and M. White. Supervised autoencoders: Improving generalization performance with unsupervised regularizers. In Advances in Neural Information Processing Systems (NIPS), 2018.
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, (6(11)22278-2324, 1998.
Z. Liu, P. Luo, X. Wang, and X. Tang. Deep learning face attributes in the wild. December 2015.
M. Long, Y. Cao, J. Wang, and M. I. Jordan. Learning transferable features with deep adaptation networks. International Conferences on Machine Learning (ICML). 2015.
A. Madry, L. Makelov, A.and Schmidt, D. Tsipras, and A. Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv: 06083, 2017.
A. Mahendran and A. Vedaldi. Understanding deep image representations by inverting them. In IEEE conference on computer vision and pattern recognition (CVPR), 2015.
S. Moosavi-Dezfooli, A. Fawzi, and P. Frossard. Deepfool: simple and accurate method to fool deep neural networks. In IEEE conference on computer vision and pattern recognition (CVPR), 2016.
Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng. Reading digits in natural images with unsupervised feature learning. 2011.
A. Nguyen, J. Yosinski, and J. Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. IEEE conference on computer vision and puttern recognition (CVPR), 2015.
C. Olah, A. Satyanarayan, I. Johnson, S. Carter, L. Schubert, K. Ye, and A. Mordvintsev. The building blocks of interpretabilityy Distill, 3(3)ee0, 2018.
J. Pan and Q. Yang. survey on transfer learning. JEEE Transactions on knowledge and data engineering, 22(10): 335-- 2009.
10

---

Under review as a conference paper at ICLR 2020
N. Papernot. P. McDaniel, I. Goodfellow, S. Jha, B. Celik, and A. Swami. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia conference on computer and communications security, 2017.
Y. Qin, N. Frosst, S. Sabour, Raffel C., Cottrell G., and G. Hinton. Detecting and diagnosing adversarial images with class-conditional capsule reconstructions. CoRR, abs/1907 00000000977,.. 2019. URL http: /ttp://aaxvvvv.. wttp:/iww..gg/bb////aaaa/
02957
A. Radford, Metz, and Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. In International Conference on Learning Representations (ICLR), 2016.
Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.0644 2015.
S. Sabour, N. Frosst, and G. Hinton. Dynamic routing between capsules. In Advances in Neural Information Processing Systems (NIPS), 2017.
O. Shamir, S. Sabato, and N. Tishby. Learning and generalization with the information bottleneck. Theoretical Computer Science, 1192933))2666--7111 2010.
K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations (ICLR), 2015.
K. Simonyan, A. Vedaldi, and A. Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. In ICLR Workshop, 2014.
J. Springenberg, A. Dosovitskiy, T. Brox, and M. Riedmiller. Striving for simplicity: The all convolutional net. In ICLR Workshop, 2015.
C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus. Intriguing properties of neural networks. In International Conference Learning Representations (ICLR), 2014.
J. B. Tenenbaum and W. T. Freeman. Separating style and content with bilinear models. Neural computation, 1246):124711283, 2000.
D. Tsipras, S. Santurkar, L. Engstrom, A. Turner, and A. Madry. Robustness may be odds with accuracy. arXiv preprint arXiv: 1805./:8805..2111 2018.
K. Xu, J. Ba, R. Kiros, K. Cho, A. Courville, R. Salakhudinov, R. Zemel, and Y. Bengio. Show, attend and tell: Neural image caption generation with visual attention. In International Conferences on Machine Learning (ICML), 2015.
Yosinski, J. Clune, Y. Bengio, and H. Lipson. How transferable are features in deep neural networks? In Advances in Neural Information Processing Systems (NIPS), 2014.
M. D. Zeiler and R. Fergus. Visualizing and understanding convolutional networks. In European conference on computer vision, 2014.
Q. Zhang and Zhu. Visual interpretability for deep learning: a survey. Frontiers of Information Technology & Electronic Engineering, 19)2:27-3,, 2018.
B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba. Learning deep features for discriminative localization. In JEEE conference on computer vision and pattern recognition (CVPR), 2016.
11

---

Under review as a conference paper at ICLR 2020
A ADDITIONAL IMAGES ON ADVERSARIAL EXAMPLES
PGD attack
X "not smile"
"smile"
||
|| P
P
Figure 8: Visualization of adversarial examples CelebA dataset with Smile labels. The adversarial examples are obtained by applying 100 iterations of PGD attack with to perturbation and 0.03. The regularization parameter set to À 10-3
12

---

Under review as a conference paper at ICLR 2020
Fore
igh
PGD attack
X "not smile"
X "smile"
||
||
P
P
Figure 9: Visualization of adversarial examples CelebA dataset with Smile labels. The adversarial examples are obtained by applying 100 iterations of PGD attack with foo perturbation and 0.03. The regularization parameter set to À 10 10-3
13

---

Under review as a conference paper at ICLR 2020
PGD attack
X "male"
X "female"
|| P
P
Figure 10: Visualization of adversarial examples CelebA dataset with eyeglasses labels. The adversarial examples are obtained by applying 100 iterations of PGD attack with foo perturbation and 0.03, The regularization parameter is set À 10-3
14

---

Under review as a conference paper at ICLR 2020
PGD attack
X "female"
X "male"
|| P
|| P
20
Figure 11:
Visualization of adversarial examples CelebA dataset with Gender labels. The adversarial examples are obtained by applying 100 iterations of PGD attack with for perturbation and 0.03, The regularization parameter is set À 10-3
15

---

Under review as a conference paper at ICLR 2020
PGD attack
X "without eyeglasses"
X "with eyeglasses"
||
|| P
P
Figure 12: Visualization of adversarial examples CelebA dataset with Eyeglasses labels. The adversarial examples are obtained by applying 100 iterations of PGD attack with fo perturbation and 0.03, The regularization parameter is set À 10-3
16

---

Under review as a conference paper at ICLR 2020
PGD attack
X
X "without eyeglasses"
"with eyeglasses"
||
|| P
P
Figure 13: Visualization of adversarial examples CelebA dataset with Eyeglasses labels. The adversarial examples are obtained by applying 100 iterations of PGD attack with fo perturbation and 0.03, The regularization parameter is set À 10-3
17

---

Under review as a conference paper at ICLR 2020
5
9 9.9.5 5
9
486
4808
8
3
PGD
/ 1
attack
87.99
8799
9
X 1
Xadr | P
P
9
4
1
5
4
9
7 3
4
8
6
8
3 8 5
/
3
5
8
1303
3
8
7
9
2 4
Figure 14: Visualization of adversarial examples on MNIST dataset with 0.2 (f.x perturbation).
The regularization parameter set to À= = 10-2
18

---

Under review as a conference paper at ICLR 2020
A.1
SOME FAILURE CASES
ED
ED
PGD attack
Pana
Parsh
X
X
||
P
P
Figure 15: Visualization of adversarial examples on CelebA dataset with Eyeglasses labels. The adversarial examples are obtained by applying 100 iterations of PGD attack with lo perturbation and 0.03. The regularization parameter is set to X = 10-3
19

---

Under review as a conference paper at ICLR 2020
B ARCHITECTURE DETAILS
B.1 EXPERIMENTAL SETTING ON CELEBA
For the CelebA experiments, we resize the images to 64 x 64 and we subsample the dataset to obtain relatively balanced dataset on the labels we are considering Eycglasee The statistics of the training set and testing set are as follows:
Size
Eyeglasses
Smile
Gender
Training Set
20000
50%
44.72%
59.60%
Test Set
6386
50%
43.83%
99.51%
Figure 16: The percentage indicates the proportion of positive labels among the dataset.
We use Adam optimizer (Kingma Ba 2015) with 3 = (0.9, 0.999), learning rate 10-3 3 the first 20 iterations and 10 for later iterations. No data augmentation is used. The architecture used in the perception regularization is similar to the one from DCGAN (Radford et al. 2016). The detailed architecture follows:
Input: RGB image a E R3x61161
x 4 conv. 128ch, ReLU, stride 2, padding
4 X 4 conv. 256ch, ReLU, stride 2, padding
4 x 4 conv. 512ch, ReI.U, stride 2, padding
x 4 conv. 1024ch, ReLU, stride 2, padding
x 4 conv. 128ch, ReLU, stride padding
Output: latent vector 12
(a) Feature Map FF
Input: latent vector E R 128
4 x 4 deconv. 128ch + 1024ch, ReLU, stride 1, padding 0
4 x 4 deconv. 1024ch 512ch, ReLU, stride 2, padding
x 4 decony. 512ch 256ch, ReLU, stride padding
x deconv. 256ch 128ch, ReLU, stride padding
deconv. 128ch + 3ch, ReLU, stride 2, padding
Sigmoid
Output: RGB image reconstruction x' 3x66644 (b) Decode D
Input: latent vector 2 E RR 128
Fully connected 128 - 256, ReLU,
Linear 256 + 2
Output: prediction IR2 (c) Classifier C
20

---

Under review as a conference paper at ICLR 2020
B.2 EXPERIMENTAL SETTING ON MNIST
For training the perceptual regularization, we perform 20 iterations of Adam optimizer Kingma
Ba 2015) with 3 = (0.9, 0.999) and learning rate 10-3, No data augmentation is used.
Input: Greyscale image E R 11x28x28
4 x conv. 16ch, ReLU, stride 2, padding
4 X conv. 16ch, ReLU, stride padding
4 X conv. 32ch, ReLU, stride padding
4 x conv. 32ch, ReLU, stride 1, padding
Output: latent vector E R33
(d) Feature Map F
Input: latent vector 2 RR
deconv. 32ch 32ch, ReI.U, stride 1, padding 0
deconv. 32ch - 32ch, ReLU, stride I, padding
4 X 4 deconv. 32ch + 16ch, ReLU, stride 2, padding 1
4 x 4 deconv. 16ch - 1ch, ReLU, stride 2, padding 1
Sigmoid
Output: Greyscale image reconstruction x' E R 1x2xx28 (e) Decode D
Input: latent vector & E R:32 Fully connected 32 + 64, ReLU,
Linear 64 -> 10
Output: prediction y R10, (f) Classifier C
B.3 EXPERIMENTAL SETTING ON SVHN
For the SVHN experiments, the inputs are RGB images of size 32 x 32. No data augmentation is used. Our architecture for the perception regularization is follows:
21

---

Under review as a conference paper at ICLR 2020
Input: RGB image x R3x32x32
4 x 4 conv. 16ch, ReLU, stride 2, padding
4 X 4 conv. 32ch, ReLU, stride 2. padding
4 X 4 conv. 64ch, ReLU, stride 2, padding
4 x 4 conv. 64ch, ReLU, stride 1, padding 0
Output: latent vector 2 R24
(g) Feature Map F
Input: latent vector 2 E TR.4
4 x 4 deconv. 128ch + 64, ReLU, stride I, padding 0
4 X 4 deconv. 64ch - 32, ReLU, stride 2, padding 1
4 X 4 deconv. 32ch 16, ReLU, stride 2, padding 1
4 deconv, 16ch 3ch, ReLU, stride 2, padding I
Sigmoid
Output: RGB image reconstruction x' E 3x332x32 (h) Decode D
Input: latent vector R64
Fully connected 64 -> 128, ReLU,
Linear 128 > 10
Output: prediction y E R100
(i) Classifier C
22


Reviews:
[{"review_id": "SylCVZigtH", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Claims: The authors present perceptual regularization as a method for learning a visualization of deep representations for promoting interpretability and understanding of vulnerability to adversarial attacks. Second, they show their method can explain negative transfer to new tasks. Finally, they show that the representations learned with this regularization method transfer to unseen tasks better than without the reconstruction regularization.\n\nDecision: Weak accept. The authors tackle an important problem with a very simple regularization technique and show how it can aid interpretability in adversarial attacks and improve transferability through strengthening attention of features to new tasks and in the multi-task setting.  While the two problems are important, results on multiple datasets would be necessary to strengthen the paper. Section 3 on perception claims to give results on MNIST and CelebA datasets, but there only seem to be results on CelebA. For transfer in section 5, MNIST or SVHN should also be included by removing classes of digits and checking performance. The conclusion addresses the use of different types of generative models, but I would be extremely interested to see how VAE compares with a deterministic auto encoder, as they are known to have much better generative properties. \n\nFor Fig 6, it would also be nice to know how robust these numbers are to lambda, and include some variation for different values of lambda, or at least some discussion of how much tuning is required.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "HJeJcEfMtS", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nIn this study, the authors propose a new architecture for visualizing the latent space of a network. This architecture is achieved by appending a secondary decoder 'head' to visualize the latent space by reconstructing the inputs.\n\nIn summary, I found the paper to provide a simple architectural change for attempting to address this issue of network interpretability and improve transfer learning. The results all appear sensible and expected, but the experiments are somewhat weak. My primary concerns are:\n\n1. The network architecture described is not terribly novel as many papers have explored pairing an auto-encoder with a classification task. Thus, I don't find the methods to be much of a contribution. The main contribution of the paper is thus in the interpretation and analysis of the method.\n\n2. The results on network interpretability are all qualitative.\nThat is, I must make a judgement on the resulting reconstructed image. I find this unsatisfactory for providing any quantitative assessment of the performance of the method. In particular, I would want to see a quantitative assessment of this method and compare it with other network introspection techniques. Currently, there is no benchmark nor any other methods to compare against.\n\n3. The results on transfer learning are quantitative but weak.\nIt is not clear to me how other simple transfer learning methods may perform on the task presented. That is, I do not know how hard the actual task is. (Would retraining a logistic classifier  with SIFT features work just as well?) I would expect to see far more experimental studies to justify these claims. Additionally, I would expect to see comparisons with other transfer learning techniques to see how well this method fares.\n\nMajor Comments:\n\n- Why do the authors allow for the decoder to regularize the latent space? Why not just stop the gradients at the latent space to provide a microscope to visualize the latent space without effecting it?\n\n- This reference appears to be in the same spirit as this paper and probably should be cited.\nAlain, Guillaume, and Yoshua Bengio.\n\"Understanding intermediate layers using linear classifier probes.\"\narXiv preprint arXiv:1610.01644 (2016).\n\n\n- Figure 2, 3 and 4 provide very nice qualitative demonstrations of the reconstruction providing insight into how an image was misclassified, however some issues remain:\n  1) A human must interpret the reconstruction. This is a subjective process and is not systematic.\n  2) Can you see examples in cross-validated test images where the visualization would predict a misclassification? Can you see counter-examples where the resulting image does not make sense?\n\n- Figure 6 and 7 provide interesting results indicating that the unsupervised objective improves transfer learning across several classification tasks related to faces. The results are encouraging but I am concerned that (a) the task is too easy, (b) tuning the hyperparameter for \\lambda can be difficult. For (b), I would like to ensure that \\lambda was tuned independent on a third validation test (and not the test set whose numbers are reported). For (a), I suspect that an important benchmark is to just take set \\lambda = \\infty and then perform linear classification on the embedding. It would be interesting to see how well these numbers fare as a baseline compared to the reported numbers.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "HJeEts4PqH", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary: This paper discusses the use of a reconstruction network which is fed an internal representation of a deep network and is trained to reconstruct the input. The classification and reconstruction networks are trained simultaneously with a combined loss function. Various experiments are carried out to test whether this approach can help investigate adversarial examples and the quality of learned features for transfer learning.\n\nReview: The idea presented in this paper is simple and easy to understand and the paper includes experiments testing its effectiveness in various settings. However, there are various issues with the paper in its current form: First, the idea of reconstructing hidden activations is not new and the paper fails to cite future work. \"Dynamic Routing Between Capsules\" by Sabour et al. consider adding a similar reconstruction network to Capsule networks. \"Detecting and Diagnosing Adversarial Images with Class-Conditional Capsule Reconstructions\" considers the use of this reconstruction network for detecting adverarsarial examples (like this paper) and also includes an unconditional reconstruction network which is added to a convnet (like this paper). I imagine there are other examples, but those are ones I know of. Second, the experiments are limited - they only include simple datasets (SVHN, MNIST, CelebA) which are normalized so that the object appears in the center of the image. This makes reconstructions difficult. The experiments also lack various important details like the model architectures and other hyperparameters, and overall are mostly \"proof of concept\" experiments rather than exhaustively testing out the proposed method. Finally, some of the claims made in the paper (particularly in the introduction) are dubious, for example that adversarial examples are a security concern and that transfer learning has been unsuccessful because it can harm performance. Based on these issues, I won't recommend acceptance. I think if the authors better situated the novelty of their work, included more exhaustive and difficult experiments, and/or focused on one specific application area it would be a stronger submission.\n\nSpecific comments:\n- \"to be understandable\" - can you be more specific? Internal representations of deep neural networks are not understandable whatsoever on their own, we usually use some post-hoc method (like your reconstruction network) to try to interpret them somehow.\n- \"[adversarial examples] raise many security concerns, such as the reliability of driverless cars, or the trustworthiness of facial recognition systems\" The field is moving towards a consensus that adversarial examples do not pose a realistic security threat in any of the settings they have been tested in. The most significant description of this viewpoint is in \"Motivating the Rules of the Game for Adversarial Example Research\" by Gilmer et al.\n- \"but has made the transferability of features to new tasks difficult, sometimes even harming performance\" This statement completely omits a huge body of work showing how effective transfer learning is, not only in computer vision (typically going from an ImageNet classifier to some downstream task with a small training set) but also in NLP (where transfer learning-based methods have state-of-the-art in virtually every benchmark). A good discussion of when transfer learning works and doesn't in CV is provided in \"Do Better ImageNet Models Transfer Better?\" by Kornblith et al.\n- The variables X and Y (without mathcal) in the \"(Perceptual regularization)\" definition equation are not defined.\n- The classification and reconstruction models are not described in Section 3. This is incredibly important. The quality of the reconstructions is likely extremely dependent on these architectural choices as they provide an implicit prior over the image space. This detail needs to be filled in. E.g. the pooling/downsampling in your classification network  will have a strong impact on the reconstruction quality, as will the size, depth, and configuration of the reconstruction network. Furthermore, I imagine the regularization parameter lambda is extremely important (as shown in section 4.2) but its value is not mentioned.\n- The experiments are on SVHN, MNIST, and CelebA which are all \"normalized\" datasets, i.e. all of their constituent images are normalized so that the object of interest appears in the same position in the center of the image. This makes reconstruction much, much easier. Results should be presented on non-normalized datasets (e.g. CIFAR-10, ImageNet, etc) in order to be convincing.\n- \"But, as the network gets more complicated it becomes computationally difficult to decode the feature map.\" Can you quantify this claim or back it up with experiments?\n- While the results on transferring to different CelebA classification tasks are interesting, it is much more common to pre-train on a diverse task like ImageNet than an extremely fine-grained task like classifying smiling or not smiling. I think these results would be more impressive if you followed modern experimental practice more closely.\n- \"The cost is that our method performs slightly poorer than the standard training method when S = T\" I would not call a difference of upwards of 8% absolute small.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a new mechanism to visualize the latent space of a neural network. The idea is simple and the paper includes several experiments to test the effectiveness of the method. However, the method bears similarity to previous work and the evaluation does not sufficiently show quantitative improvements over other introspection techniques. The reviewers found this was a substantial problem and for this reason the paper is not ready for publication. The paper should improve its discussion of prior work and better establish its place in this regard.
Decision: Reject