Gradients as Features for Deep Representation Learning
Authors: Fangzhou Mu
Yingyu Liang
Yin Li
Venue:  (2020)
Keywords: representation learning
gradient features
deep learning

Abstract: We address the challenging problem of deep representation learning -- the efficient adaption of a pre-trained deep network to different tasks. Specifically, we propose to explore gradient-based features. These features are gradients of the model parameters with respect to a task-specific loss given an input sample. Our key innovation is the design of a linear model that incorporates both gradient and activation of the pre-trained network. We demonstrate that our model provides a local linear approximation to an underlying deep model, and discuss important theoretical insights. Moreover, we present an efficient algorithm for the training and inference of our model without computing the actual gradients. Our method is evaluated across a number of representation-learning tasks on several datasets and using different network architectures. Strong results are obtained in all settings, and are well-aligned with our theoretical insights.

Published as a conference paper at ICLR 2020
GRADIENTS AS FEATURES FOR DEEP REPRESENTATION LEARNING
Fangzhou Mu, Yingyu Liang Department of Computer Sciences University of Wisconsin-Madises (fmu, yliang, Jécs hww.e/iss........o............ee wisc.
Yin Li
Departments of Biostatistics & Computer Sciences University of Misconsin-Madisse yin. 11 wwwiww..e................o
ABSTRACT
We address the challenging problem of deep representation learning the effi- cient adaption of a pre-trained deep network to different tasks. Specifically, we propose to explore gradient-based features. These features are gradients of the model parameters with respect to a task-specific loss given an input sample. Our key innovation is the design of a linear model that incorporates both gradient and activation of the pre-trained network. We demonstrate that our model provides local linear approximation to underlying deep model, and discuss important theoretical insights. Moreover, we present an efficient algorithm for the training and inference of our model without computing the actual gradients. Our method is evaluated across number of representation-llariig tasks on several datasets and using different network architectures. Strong results are obtained in all settings, and are well-alige aligned with our theoretical insights"]
INTRODUCTION
Despite tremendous success of deep learning, training deep neural networks requires a massive amount of labeled data and computing resources. The recent development of representation learn- ing holds great promises for improving data efficiency of training, and enables an easy adaption to different tasks using the same feature representation. These features can be learned via either unsu-
pervised learning using deep generative models (Kingma & Welling 20144] Dumoulin et al. 2016), or self-supervised learning with "pretext" tasks and pseudo-labels (Noroozi & Favaro [016 Zhang
et al., 2016 Gidaris et al. 2018), or transfer learning from another large-scale dataset (Yosinski al. 2014 Oquab al. [0144] Girshick et al.) 2014). After learning, the per-sample activation of the network is considered as generic features. By leveraging these features, simple classifiers such as linear models can be learned for different tasks. However, given sufficient amount of training data, the performance of pppresentation-larriig methods lags behind fully supervised deep models.
As step towards bridging this gap, we propose make use of gradient- based features from pre- trained network, i.e., gradients of the model parameters with respect to task-specific loss given an input sample. Our key intuition is that these per-sample gradients contain task-relevant discrimina- tive information. More importantly, we design novel linear model that accounts for both gradient- and iidivatton-basee features. The design of our linear model stems from the recent advances in the theoretical analysis of deep models. Specifically, our gradient- based features inspired by the neural tangent kernel (NTK) (Jacot et al. 2018 Lee et al. 2019; Arora et al. 2019b). and adapt NTK in the setting of finite-width networks. Therefore, our model provides local approximation of fine-tuning an underlying deep model, and the accuracy of the approximation is controlled by the semantic gap between the appresentation-leamiin and the target tasks. Finally, the structure of the gradient features and the linear model allows us to derive an efficient and scalable algorithm for training and inference.
To evaluate our method, we focus on visual representation learning in this paper, although our model can be easily modified for natural language or speech data. To this end, we consider a number of learning tasks in vision, including unsupervised, self self-super and transfer learning. Our method
Project wehpage at http:: /aattt::/ppggsse /ttp///aaaaaace/eaa wtt.:aaew............e.e/ wtw.:/.edddaammmaaaaaaaaaaaaaaee

---

Published as a conference paper at ICLR 2020
is evaluated across tasks, datasets and network architectures and compared against set of baseline methods. We observe empirically that our model with the gradient features outperforms the tradi- tional activation- logistic regressor by significant margin in all settings. Moreover, model compares favorably against fine-tuning of network parameters.
Our main contributions are thus summarized as follows.
We propose novel representation-leamee model that builds on gradients model parameters as the feature representation.
method. At the core of our method lies in linear
From a theoretical perspective, we claim that our linear model provides a local approximation of tieettniigg underlying deep model. From a practical perspective, we devise an efficient and scalable algorithm for the training and inference of our method.
We demonstrate strong results of our method across various representation-leaniie tasks, dif- ferent network architectures and several datasets. Furthermore, these empirical results are well- aligned with our theoretical insight.
RELATED WORK
Representation Learning. Learning good representation of data without expensive supervision remains challenging problem. Representation learning using deep models has been recently ex- plored. For example, different types of deep latent variable models (Kingma & Welling 2014]
Higgins et al. [017 Berthelot et al.] 2019; Dumoulin et al., 2016, Donahue et al. 2016 Dinh
et al.) 2017] Kingma & Dhariwal 2018; Grathwohl et al.) 2019) were considered for representa- tion learning. These models
designed to fit the distribution of data, yet their intermediate responses were found useful for discriminative tasks. Another example is self-supervised learning. This paradigm seeks to learn from discriminative pretext task whose supervision comes almost for free. These pretext tasks for images include predicting rotation angles (Gidaris et al. (2018). solv- ing jigsaw puzzles (Noroozi & Favaro 2016) and colorizing grayscale images (Zhang et al. 2016). Finally, the idea of transfer learning hinges on the assumption that features learned from a large and generic dataset can be shared across closely related tasks and datasets (Girshick et al. 2014
Sharif Razavian et 20144 (qquab et al.) (2014). The most successful models for transfer learning so far are those pre-trained on the ImageNet classification task (Yosinski et al. 2014).
As opposed to proposing new representation-leameee out of the existing tasks. Hence, our method is broadly applicable offers a generic framework that can be readily combined with any eepresentation-larmiie paradigm.
tasks, our work studies how to get the most
Gradient of Deep Networks. Our method makes use of the Jacobian matrix of a deep network as feature representation for a downstream task. Gradient information is traditionally employed for visualizing and interpreting convolutional networks (Simonyan et al. 2013), and more recently for generating adversarial samples (Szegedy al. (013)) crafting defense strategies (Goodfellow al.)
2015), facilitating knowledge distillation (Sinha et al. 2018 [Srinivas & Fleuret 2018). and boosting
multi-task and meta learning (Sinha al., 2018 Achille et al. 2019).
Our work draws inspiration from Fisher vectors (FVs) (Jaakkola & Haussler 1999) gradient-based features from a probabilistic model (e.g.. mixture of Gaussians). The FV pipeline has demonstrated its success for visual recognition based on hand-crafted local descriptors (Perronnin & Dance (007). More recently, FV's have shown promising results with deep models, first as an ingredient of hybrid system (Perronnin & Larlus 2015). and then as task embeddings for meta- meta-learning (Achille et al., ).19. Our method differs from the FV approaches in two folds. First, it is not built around a prob- abilistic model, hence has distinct theoretical motivation as we describe later. Second, our method enjoys exact gradient computation with respect to network parameters and allows scalable training. whereas Achille et al. (2019) employs heuristics in their method to aggressively approximate the computation of FVs.
Another line of relevant research is from Zinkevich et al. (2017). They also explored similar gra- dient features of deep networks, which they call holographic features, by drawing inspiration from generalized linear models (GLM). They showed that linear model trained on the gradient features can faithfully recover the output of the original network for the same task, and also showed some other desired properties of gradient features. In contrast, our work is motivated by the NTK theory,
2

---

Published as a conference paper at ICLR 2020
1
2
fx(x)
W2
Output
w (Linear weight)
xn (x)
Activation
Actiratio
Duripst
Graddeentbbased teatures)
Linear weight
(x(x)
(a)
(b)
Figure 1: (a) An illustration of our parameterization. We consider a deep network F(x; w) - w" fo( that consists of ConvNet fo with its parameters 0 - (0). 02) and linear weights w. (b) An overview of our proposed model. Our model takes the activation fx( and the gradient Ja, (œ) as input (see illustration in (a)), and learns linear weights wi (matrix) and w2 (vector) for prediction.
and we study the more challenging problem of adapting the features for a different task. would be interesting to investigate the connection between GLM and NTK.
Neural Tangent Kernel (NTK) for Wide Networks. Jacot et al. (2018) established the connection between deep networks and kernel methods by introducing the neural tangent kernel (NTK). Lee iet al. (2019) further showed that a network evolves as a linear model in the infinite width limit when trained on certain losses under gradient descent. Similar ideas have been used analyze wide networks (see, e.g., Arora et al. (2019b a): Li & Liang] (2018) 8): Allen-Zhu et al. (2019a); Du et al. 2019 Allen-Zhu et al. (209bb); Cao & Gu (020); Mei al. (2019)). Our method is, to our best knowledge, the first attempt to materialize the theory in the regime of practical networks. Instead of assuming random initialization of network parameters as all the prior works do, we for the first time
empirically evaluate the implication of pre-training the linear approximation theory.
GRADIENT-BASED FEATURES FOR REPRESENTATION LEARNING
We start by introducing the setting of representation learning using deep models. Consider a feed- forward deep neural network Fu, (x) & w fo (z)) that consists of backbone fo(æ) with its vector- ized parameters and linear model defined by w (italic for vectors and bold for matrix). Specif- ically, Se encodes the input æ into vector representation fa()) IRd w Raxe are thus linear weights that map feature vector into output dimensions. For this work, we focus on convolutional networks (ConvNets) for classification tasks. With trivial modifications, our method can easily ex- tend beyond ConvNets and classification, e.g., for recurrent network as the backbone and/or for regression task.
Following the setting of representation learning, we assume that a pre-trained fø is given with 0 as the learned parameters. The term representation learning refers to a set of learning methods that do not make use of discriminative signals from the task of interest. For example, f can be the encoder of a deep generative model (Kingma & Welling 2014; Dumoulin et al. 2016 Donahue let al. 2016), or a ConvNet learned by using proxy tasks (self-supervieed learning) (Goyal et al.) 2019 Kolesnikov et al., 2019) or from another large-scale labeled dataset such as ImageNet (Deng al. 2009). Given target task, it is common practice to regard fa fixed feature extractor (activation-basee features) and train a set linear weights, given by
(1) We omit the bias term for clarity. Note that @ and 0 are instantiations of w and 6, where w is the solution of the linear model and is given by representation learning. Based on this setup, we now describe our method, discuss the theoretical implications and present an efficient training scheme.
(xx = T fa(z),
3.1 METHOD OUTLINE
Our method assumes a partition of 0 A (01, 0z), where 02 and 0₂ parameterize the bottom and top layers of the ConvNet ff (see Figure
a for an illustration). Importantly, we propose to use w Jj, (x) in addition to activation-based features fa(æ). Specifically, Já, (a:) Rdx is the Jacobian matrix of fa with respect to the pre-trained parameter
gradient-based features Va, F (x)
3

---

Published as a conference paper at ICLR 2020
82 from the top layers of f. Given the features (fo(x) (z) @ (x)) for æ, our linear model hereby considered as a classifier for concreteness, takes the form
Own, wn (a) = w/ fa(x)) 5" (x) = Iwn (x: + WJ Jáz (z))
where wi E Rd are linear classifiers initialized from @, W2 E RR |01 are shared linear weights for gradient features, and [02] is the size of the parameter 22 Both wi and W2 are our model parameters that need to be learned from target task. An overview of the model is shown in Figure [b).
Our model subsumes the linear model in Eq. I as the first term, and includes a second term that is linear in the gradient- meadentt-based features. We note that this extra linear term is different from standard linear classifiers as in Eq. In this case, the gradient-hasee features form a matrix and the weight vector W2 is multiplied each row of the feature matrix. Therefore, W2 is shared for all output di- mensions. Similar to standard linear classifiers, the output of ÿ is further normalized by the softmax function and trained with the cross-entropy loss using labeled data from the target dataset.
Conceptually, our method can be summarized into three steps.
Pre-train the ConvNet fa. This is accomplished by substituting in any existing representation- learning algorithm. Examples include unsupervised, supervse and transfer learning.
Train linear classifiers w using (a)).. This is a standard step in representation learning by fitting a linear model using "generic features".
Learn the linear model Dwr (a)). linear model the form of Eq. ? learned using gradient- and activation-bart baaedd features. Note that the features are obtained when 0 = 0 is kept fixed, hence our method requires no extra tuning of the parameter 0.
3.2 THEORETICAL INSIGHT
The key insight is that our model provides a local linear approximation to (a)). This approximation comes from Eq. ! the crux of our approach. Importantly, our linear model is mathematically well motivated - it can be interpreted as the first-order Taylor expansion of Fs.w with respect to its parameters (02, w) around the point of (02, (). More formally, we note that
(x) R TJ fa(x) -
02) -w- 1oo-a1) - fa(Ø)
w fo(x)) -5 (z)) 02)
وw 02 - (z).
With = wi and 02 02 W2 Eq. provides a linear approximation of the deep model Foz w (x) around the initialization (0g, ). Our key intuition is that given a sufficiently strong base network, training model approximates fine-tuning Fez.w-
The quality of the linear approximation can be theoretically analyzed via the recent neural tangent kernel approach (Jacot et al. 2018 Lee et al. 2019 Arora et al. 2019b) some related ideas (Arora et al., 2019a] Li & Liang, 2018 Allen-Zhu et al. 2019a Du et al.) 2019 Allen-Zhu et al. 2019b] Cao & Gu 20200) Mei et al. 2019) when the base network Fe. is sufficiently wide and random initialization. Unlike prior works, we apply the linear approximation on pre-trained networks of practical sizes. We argue that such an approximation is useful in practice for the following reasons:
The pre-trained network provides a strong starting point. Thus, the pre-trained network parameter 0 is close to a good solution for the downstream task. The key to good linear ap- proximation is that the network output is stable with respect to small changes in the network parameters. The pre-trained base network also has such stability properties, which are sup- ported by empirical observations. For example, the pre-trained network has similar predictions for significant fraction of data in the downstream task fine-tuned network.
The network width required for the linearization to hold decreases data becomes more structured. An assumption made in existing analysis is that the network is sufficiently or even infinitely wide compared to the size of the dataset, that the approximation can hold for any dataset. We argue that this is not necessary in practice, since the practical datasets are well- structured, and theoretically it has been shown that long the trained network is sufficiently
4

---

Published as a conference paper at ICLR 2020
wide compared to the effective complexity determined by the structure of the data, then the approximation can hold (Li & Liang 2018 Allen-Zhu et al. 2019a). Our approach thus takes advantage of the bottom layers to reduce data complexity in the hope that lincarization of the top (and often the widest) layers can be sufficiently accurate.
3.3 SCALABLE TRAINING
Moving beyond the theoretical aspects, practical challenge of our method is the high cost of com- puting I during training and inference. A naïve approach requires evaluating and storing wf Jø, (æ) for all This is computationally expensive and can become infeasible the number of output di- mensions e and the parameter size [02] grow. Inspired by Pearlmutter (1994), we design an efficient training and inference scheme for 9. Thanks to this scheme, the complexity of training our model using gradient features is of the same magnitude training a linear classifier on network activation.
Central to our scalable approach is the inexpensive evaluation of the Jacobian -eetorr product (JVP) @ Jú, W2) whose size is the same as c. First, we note that
Fo+rog (a)) Fo rw= Jū (ar) o(r²)
(4)
by first-order Taylor expansion around scalar r = 0. Rearrange and take the limit of r to O, we get lim P-0
F Fetroy (2x) Fa. (x)
wn Jog (x))
Fo
(5)
dr
which can be conveniently evaluated via forward- mode automatic differentiation.
More precisely, let consider the basic building block of convollutio layers. These layers are defined as a linear function h(ze; We: be) = w ze + be. where we and be. which are part of 0, are the weight and bias respectively, and z, is the layer input. We denote the counterparts of w. and b in w2 w. and be, i.e., W. and b. are the linear weights applied w. and b.. It can be shown that
0h(zz; + rw., be + rbc) Mizes Wes be) + h
Oze Ör
we; 0),
Ör
Uze is the JVP coming from the upstream layer. where dr
When nonlinearity is encountered, we have, using the ReLU function as example,
Re Oze
@ 12,20:
(7)
Ör where ① is the element-wise product, is the element-wise indicator function and ze is the layer input. Other nonlinearities as well as pooling layers can be handled in the same spirit. For batch normalization, we fold them into their preceding convolutions.
Ör
Importantly, Eqs. 66) and ( provide efficient approach to evaluating the desired JVP in Eq. (5) by successively evaluating a set JVPs on the fly. This chain of evaluations starts with the seed dzn O, where 20 is the output of the section of f parameterized by 0, and can be pre-computed. oF TT Jj, (x)wz can be computed along with the standard forward propagation through f. Moreover, during the training of ÿ. its parameters uy and W2 can be learned via standard back-propagation. In summary, our approach only requires a single forward pass through the fixed f for evaluating Ó. and a single backward pass for updating the parameters wi and w2.
Complexity Analysis. We further discuss the complexity of our method in training and inference, and contrast our method to the fine-tuning of network parameters 02. Our forward pass, demon- strated by Eqs. (6) and 7 is a chain of linear operations intertwined by element-wise multipli- cations. The second term in Eq. 6 forms the "main stream" of computation, while the first term merges into the main stream at every layer of the ConvNet f. The same reasoning holds for the backward pass. Overall, our method requires twice as many linear operations as fine-tuning A of the ConvNet. Note, however, that half of the linear operations by our method are slightly cheaper due to the absence of the bias term. Moreover, in the special case where 02 only includes the topmost layer of f. our method carries out the same number of operations as fine-tuning since the second term in Eq. (6) can be dropped. For memory consumption, our method requires to store an addi- tional "copy" (linear weights) of the model parameters compared to fine-tuning As the size of 6g typically small, this minor increase of computing and memory cost puts our method on the same page fine-tuning.
5

---

Published as a conference paper at ICLR 2020
4 EXPERIMENTS
Our experimental results are organized into two parts. We first perform ablation studies to under- stand the representation power of the gradient features. Next, evaluate method on three representation-leaming tasks: learning deep generative models, self-supervised learning using a pretext task, and transfer learning from ImageNet.
We report results on several datasets and using different base networks to demonstrate the strength of our method. In all cases, our method achieves significant better performance than the standard logistic regressor on network activation. In the most interesting scenario where the pre-training task is similar to the target task, our method achieves comparable or even better performance than fine-tuning, Before we present our results, we summarize our implementation details.
Implementation Details. We adopt the NTK parametrization (Jacot et al. 2018) for 0g and fold batch normalization into their preceding convolutional layers prior to training our model. For the SVHN and CIFAR-I0/1I experiments, We train the models for 80K iterations with initial learning rate 1e-3, halved every 20K iterations. For the VOCO7 and COC02014 experiments, we train the models for 50 epochs with initial learning rate le-3, halved every 20 epochs. All models are trained with the Adam optimizer (Kingma & Ba 2015) with batch size 64, B1 0.5, 83 0.999 and weight decay 1e-6. In addition to Adam, we also use the SGD optimizer with weight decay 5e-5, momentum 0.9 and the same learning rate schedule for fine-tuning, and report the better result between the two runs.
4.1 ABLATION STUDY
We conduct ablation studies address the following two questions: Does pre-training encourage more powerful gradient features? What is the optimal size of the gradient features? We answer the first question by comparing gradient features derived from various combinations of random and pre-trained network parameters, and the second by varying the number of layers that contribute to the gradient features.
Baselines. We compare our linear model in Eq. 2), referred to as the full model in the sequel, against two baselines. The first baseline Iwn (æ), or the activation model, is the first term in the full model. It is a logistic regressor on network activation and is widely used for representation learning in practice. The second baseline w' Ja, (x) wu2 or the gradient model, is the second term in the full model. It is linear in the gradient features and can serve as linear classifier on its own. Moreover, we compare our method against fine-tuning in order to assess the validity of our theoretical insight.
Settings. For ablation study, we consider both unsupervised- and report classification accuracy. In the unsupervised setting (Table D. we transfer features learned from deep generative model for image classification. Specifically, we train BiGAN CIFAR- 10 (Krizhevsky et al. 2009) and use its encoder as the base network for classification of the same dataset. In the supervised setting (Table 2 consider fine-tuning ImageNet pre-trained models on a different classification tasks. Concretely, we use the PyTorch (Paszke et al., 2017) distribution of ImageNet pre-trained ResNet18 (He al. 2016) the base network for VOCO7 (Everingham et al., 2010) object classification. For both settings, activation features are always output from the pre-trained base network. In contrast, gradient features are evaluated with respect to all possible combinations of random and pre-trained network parameters. In this way, we ensure that variation in performance of the full model can be solely attributed to the gradient features.
settings and
Results and Conclusions. Our results are shown in Table I (unsupervised) and Table 2 (supervised). We summarize our main conclusions from the ablation study.
The discriminative power of the gradient features is a consequence of pre-training. In particular, the full model's performance gain is not a consequence of an increase in parameter size. The full model supplied with random gradients no better than the activation baseline.
Gradient from the topmost layer of pre-trained network suffices to ensure a reasonable performance gain. Further inflating the gradient features from bottom layers has diminishing returns, introducing little accuracy gain and sometimes hurting the performance. The results in Zinkevich et al. (2017) also suggest that gradient features from top layers are the most effective in terms of approximating the underlying network.

---

Published as a conference paper at ICLR 2020
Table 1: Ablation study in the unsupervised setting. We train BiGAN on CIFAR-10 and use its encoder as the base network for classification of the same dataset. All results are reported using classification accuracy. We obtain the pre-trained 0 and 22 from the pre-trained base network, and @ by first training the activation baseline for the target task. The specifications of 01, 8g and only affects gradient evaluation. Activation features are always output from the pre-trained base network.
02: conv5
02: conv4-5
02: conv3-5
grad
full
grad
full
grad
full
random 0g pre-trained
random w
23.09
62.83
26.30
62.83
27.52
62.85
Random
pre-trained w
40.23
63.11
43.36
63.42
44.36
63.38
0
random w
32.50
62.98
33.20
63.10
35.13
63.10
0g
pre-trained w
44.07
63.99
45.87
64.21
48.88
64.51
random 0
random w
53.39
63.74
56.24
64.30
57.20
64.60
Pre-trained
pre-trained w
59.98
65.46
61.47
65.38
59.89
64.73
01
pre-trained 0g
random w
70.28
69.84
71.08
70.10
71.60
70.64
pre-trained u
70.14
70.51
70.89
70.78
71.55
71.37
activation
62.87
fine-tuning
71.78
73.18
74.30
Table Ablation study in the supervised setting. use the PyTorch distribution ImageNet pre-trained 18 the base network for VOCO7 object classification. All results reported using mean average precision (mAP). Predictions are based on a single center crop at test time.
02: layer4 block2 full
02: layer4 block1-2
grad
grad
full
random
random
15.63
82.84
17.96
82.85
Random
02
pre-trained
17.43
82.79
19.90
82.82
01
pre-trained
random
18.15
82.85
20.15
82.21
0g
pre-trained w
20.78
82.76
23.57
82.73
random
random 62.35
82.84
69.34
82.82
Pre-trained
pre-trained ~
65.75
82.84
71,73
82.86
0
pre-trained 0g
random w
80.74
83.15
80.30
82.97
pre-trained
83.05
83.50
83.12
83.40
activation
82.65
fine-tuning
82.97
82.50
Our results improve as the dataset and base network grow in complexity. Our method always outperforms the activation baseline by significant margin when the gradient features are derived from pre-trained parameters, Moreover, our method consistently beats the gradi- ent baseline and even fine-tuning on the more challenging VOCO7 dataset when using a more complicated residual network.
Remarks. We obtain the pre-trained 0= 02) from representation learning, and the pre-trained by first training the standard logistic regressor for the target task. We follow the same training procedure in later experiments. According to the ablation study, a good heuristic for our method is to use the gradients from the topmost convolutional layer (or residual block) of pre-trained network. A more principled strategy for selecting gradient-contributing layers is left for future work.
4.2 RESULTS ON REPRESENTATION LEARNING
We now present results on three different representation-lean leamning tasks: unsupervised learning using generative modeling, sele-supervised learning using pretext tasks and transfer learning from pre- trained models. For all experiments in this section, we contrast our proposed linear classifier (i.c., the full model in Eq. (2) with the activation and gradient baselines as well as fine-tuning 02 for the

---

Published as a conference paper at ICLR 2020
Table Unsupervised Learning Results. We consider two base networks, namely the encoder of a BiGAN and that of a VAE, trained on three datasets. Our target task is image classification on the same datasets. All results are reported using classification accuracy.
SVHN
CIFAR- 10
CIFAR-100
activation gradient full
82.04
62.87
34.30
BiGAN (02: conv5)
89.91
70.14
38.99
89.96
70.51
39.37
fine-tuning
91.15
71.78
41.05
activation
81.95
52.05
29.20
VAE (0): conv
gradient
91.44
61.47
35.10
full
90.90
61.16
34.83
fine-tuning
93.61
65.16
37.86
Table 4: Self-supervised and Transfer Learning Results. We consider ResNet50 pre-trained on the jigsaw pretext task, and ResNet18 pre-trained on the ImageNet classification task. Our target task is VOCO7 and COCO2014 object classification. All results are reported using mean average precision (mAP). Predictions are averaged over ten random crops at test time.
VOCO7
COC02014
activation gradient
57.83
42.10
Self-supervised (Jigsaw)
57.73
41.02
full
61.70
45.62
fine-tuning
67.88
52.66
Self-supervised (Colorization)
activation
52.30
n/a
activation
83.59
62.98
Transfer
gradient
84.63
66.33
(ImageNet)
full
84.95
66.45
fine-tuning
84.14
63.43
target task. Following our conclusion from the ablation study, we make use gradient features only from the topmost layer (or residual block) of pre-trained base network.
Unsupervised Learning. Similar to our ablation study, we consider unsupervised representation learning using deep generative models. We train both BiGAN and VAE, and use their encoders as the pre-trained ConvNet f- Our BiGAN and VAE models follow the architecture and training setup from Dumoulin et al. (2016) and Berthelot al. (0119). We average-pool the ConvNet output for activation features, and obtain gradient features from the topmost convolutional layer. We train on the t train split of CIFAR-10/100 and the extra split of SVHN, and report classification accuracy on their test splits.
Unsupervised Learning Results. Our results are summarized in Table 3 Our model consistently improves the activation baseline by over 10% on all three datasets. Moreover, we observe good agreement of performance between our model and fine-tumin ane-ttning. Finally, we notice that our model does not have significant advantage over the gradient baseline. We think this is because the datasets and base networks used here are so simple that the gradient features alone can be equally effective.
Self-supervised Learning. Moving beyond unsupervised learning, we also consider self-supervised setting, where the representation is learned via a pretext task and pseudo-labels. In this case, we consider ResNet50 pre-trained on the jigsaw self-supervising task on ImageNet provided by Goyal et al. (2019). See Noroozi & Favaro (2016) for details on the jigsaw task. We average-pont the ConvNet output for activation features, and use gradient features from the last residual block. We train on the t trainval split of VOCO7 and the train split of COC02014 for object classification, and report the mean average precision (mAP) scores their respective test and va 1 splits.
8

---

Published as a conference paper at ICLR 2020
Self-supervised Learning Results. Our results are summarized in the top part of Table We again observe a significant improvement of up to 4.5% over the two baselines on both datasets. Our method also outperforms competitive self- suff-supeeiieed learning method (Colorization) (Zhang al. 2016) using the activation from the same backbone network (ResNer50) and pre-trained on the same dataset (ImageNet). We note that our method still lags behind fine-tuning in this setting with large gap (-6% on VOCO7 and -7% on COCO2014).
Transfer Learning. Finally, we consider the most widely used transfer learning setting, where the representation is learned by pre-training on a large scale labeled dataset, e.g., ImageNet. We experiment with an ImageNet pre-trained ResNet18 from PyTorch. We follow the same setting as self-supervised learning and report results on VOCO7 and COC02014 datasets.
Transfer Learning Results. Our results are presented in the bottom part of Table Not surpris- ingly, our results consistently outperforms the two baselines across both datasets. However, the gap between our method and gradient baseline is much smaller. To our surprise, our method outperforms fine-tuning on both datasets by large margin. In comparison to fine-tuning, our method has gain of +0.8% and +3.0% on VOCO7 and the larger COCO2014 dataset.
4.3 FURTHER DISCUSSIONS
We provide further discussions of our results. Specifically, analyze the performance gap between out method and fine-tuning under three different settings. We then compare our method with learning linear classifier only using gradient features. Moreover, we contrast our method with the recent effort of learning infinitely wide networks using exact NTK (Arora et al. 2019b).
The performance gap between our method and fine-tuning. We argue that this gap controlled by the degree of semantic overlap between the representation-leariin and target tasks. The jigsaw pretext task, though proven effective for learning features, still semantically distant an image classification task, and hence results in the large performance gap we observe in the self-supervised setting. On the other hand, is reasonable expect classifiers for datasets with overlapping cate- gories to share common semantics. This may explain why our method outperforms fine-tuning in the transfer-learning experiments, where the source and target datasets are conceptually close.
Comparing our method with training linear classifier on gradient features alone. In general, both methods are considerably more powerful than the standard activation based logistic regressor. In the unsupervised setting, the gradient model performs on par with, or even better than the full model, which consumes both activation and gradient features (see Table 3). This raises the question of whether network activations is not necessarily the case. In fact, as we demonstrate in the self-supervised and transfer-learming experiments (see Table 4 combining activation and gradient features is formula for delivering the best performance on challenging representation-leariig tasks (jigsaw pretext task) and datasets would be interesting to further characterize the "representation bias"
redundant in the presence of gradient features. We argue that this
(VOC07 and COCO2014), induced by the two types of features.
Contrasting our method with learning infinitely wide networks from seratch. Arora et al. (2019)) recently proposed exact NTK computation algorithms for learning infinitely wide Con- vNets, and proved that such networks are equivalent kernel regression using NTK. They obtained 66% accuracy on CIFAR-10 classification using a vanilla network and achieved an extra 11 % boost after including customized network modules. Their approach, at its current stage, only supports full-batch learning and is limited to single output dimension. Our method, though not directly com- parable with theirs, does not suffer from these limitations and delivers competitive results (e.g., we obtain 70.5% accuracy on CIFAR-10 using four-layer BiGAN encoder as the base network, see Table 3). Hence, out work highlights another promising direction to capitalize on the insights generated by the theoretical analysis of NTK.
5 CONCLUSION
We presented novel method for deep representation learning. Specifically, given a pre-trained deep network, we explored as features the per-sample gradients of the network parameters relative to a task-specific loss, and constructed linear model that combines network activation with the gradient

---

Published as a conference paper at ICLR 2020
features. We showed that model can be very efficient in training and inference, and may be understood as a local linear approximation to an underlying deep model by an appeal to the neural- tangent-kernel kernel (NTK) theory. We empirically demonstrated that the gradient features are highly discriminative for downstream tasks, and our method can significantly improve over the baseline method of representation learning across pre-training tasks, network architectures and datasets. We believe that our work offers a novel perspective to improving deep representation learning. Future research directions include using NTK distance metric for measuring sample similarity in low- shot classification and assessing model similarity in knowledge distillation.
Acknowledgement: This work was supported in part by FA9550-18 1000-88-1-0166 The authors would also like to acknowledge the support provided by the University of Wisconsin-Madison Office of the Vice Chancellor for Research and Graduate Education with funding from the Wisconsin Alumni Research Foundation.
REFERENCES
Alessandro Achille, Michael Lam, Rahul Tewari, Avinash Ravichandran, Subhransu Maji, Charless Fowlkes, Stefano Soatto, and Pietro Perona. Task2vec: Task embedding for meta-learning. In ICCV, 2019.
Zeyuan Allen-Zhu, Yuanzhi Li, and Yingyu Liang. Learning and generalization in overparameter- ized neural networks, going beyond two layers. In NeurIPS, 2019a.
Zeyuan Allen-Zhu, Yuanzhi Li, and Zhao Song. convergence theory for deep learning via over- parameterization. In ICMI, 2019b.
Sanjeev Arora, Simon Du, Wei Hu, Zhiyuan Li, and Ruosong Wang. Fine-grained analysis of opti- mization and generalization for overparameterized two-layer neural networks. ICMI., 2019a.
Sanjeev Arora, Simon S Du, Wei Hu, Zhiyuan Li, Ruslan Salakhutdinov, and Ruosong Wang. On exact computation with an infinitely wide neural net. In NeurIPS, 2019b.
David Berthelot, Colin Raffel, Aurko Roy, and Ian Goodfellow. Understanding and improving interpolation in autoencoders via an adversarial regularizer. In /CLR, 2019.
Yuan Cao and Quanquan Gu. Generalization error bounds of gradient descent for learning over- parameterized deep relu networks. In AAAI, 2020.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, 2009.
Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using real nvp. In ICLR, 2017.
Jeff Donahue, Philipp Krähenbühh,, and Trevor Darrell. Adversarial feature learning. ICLR, 2016.
Simon Du, Jason Lee, Haochuan Li, Liwei Wang, and Xiyu Zhai. Gradient descent finds global minima of deep neural networks. ICML, 2019.
Vincent Dumoulin, Ishmael Belghazi, Ben Poole, Olivier Mastropietro, Alex Lamb, Martin Ar- jovsky, and Aaron Courville. Adversarially learned inference. In ICLR, 2016.
Mark Everingham, Lue Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (VOC) challenge. IJCV, (8(2):303-338, 2010.
Spyros Gidaris, Praveer Singh, and Nikos Komodakis. Unsupervised representation learning by predicting image rotations. In ICLR, 2018.
Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich feature hierarchies for accu- rate object detection and semantic segmentation. CVPR, 2014.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR, 2015.
10

---

Published as a conference paper at ICLR 2020
Priya Goyal, Dhruv Mahajan, Abhinav Gupta, and Ishan Misra. Scaling and benchmarking self- supervised visual representation learning. ICCV,V 2019.
2019.
Will Grathwohl, Ricky TQ Chen, Jesse Betterncurrt, llya Sutskever, and David Duvenaud. Ffjord: Free-form continuous dynamics for scalable reversible generative models. In ICLR, 2019.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. CVPR, 2016.
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-VAE: Learning basic visual concepts with constrained variational framework. In ICLR, 2017.
Tommi Jaakkola and David Haussler. Exploiting generative models in discriminative classifiers, In NeurIPS, 1999.
Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kernel: Convergence and gen- eralization in neural networks. In NeurlPS, 2018.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.
Diederik P Kingma and Max Welling, Auto-encoding variational bayes. In ICLR, 2014.
Durk P Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible 1x1 convolutions, In NewrIPS, 2018.
Alexander Kolesnikov, Xiaohua Zhai, and Lucas Beyer. Revisiting self-supervised visual represen- tation learning. In CVPR, 2019.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Technical report, 2009.
Jaehoon Lee, Lechao Xiao, Samuel S Schoenholz, Yasaman Bahri, Jascha Sohl-Dickstein, and Jef- frey Pennington. Wide neural networks of any depth evolve as linear models under gradient descent. In NeurIPS, 2019.
Yuanzhi Li and Yingyu Liang. Learning overparameterized neural networks via stochastic gradient descent structured data. In NewrlPS, 2018.
Song Mei, Theodor Misiakiewiez, and Andrea Montanari. Mean-field theory of two-layers neural networks: dimension-free bounds and kernel limit. In COLT, 2019.
Mehdi Noroozi and Paolo Favaro. Unsupervised learning of visual representations by solving jigsaw puzzles. In ECCV, 2016.
Maxime Oquab, Leon Bottou, Ivan Laptev, and Josef Sivie. Learning and transferring mid-level image representations using convolutional neural networks. In CVPR, 2014.
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary De Vito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. In NeurlPS, 2017.
Barak A Pearlmutter. Fast exact multiplication by the hessian. Neural computation, 6(1):147-160 1994.
Florent Perronnin and Christopher Dance. Fisher kernels on visual vocabularies for image catego- rization. In CVPR. 2007.
Florent Perronnin and Diane Larlus. Fisher vectors meet neural networks: A hybrid classification architecture. In CVPR, 2015.
Ali Sharif Razavian, Hossein Azizpour, Josephine Sullivan, and Stefan Carlsson. Cnn features off- the-shelf: astounding baseline for recognition. In CVPR Workshops, 2014.

---

Published as a conference paper at ICLR 2020
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Vi- sualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034, 2013.
Ayan Sinha, Zhao Chen, Vijay Badrinarayanan, and Andrew Rabinovich. Gradient adversarial train- ing of neural networks. arXiv preprint arXiv: 0r8.vv1800.08028, 08028, 2018.
Suraj Srinivas and Francois Fleuret. Knowledge transfer with Jacobian matching. JCML, 2018.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 2013.
Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. How transferable are features in deep neural networks? In NeurIPS, 2014.
Richard Zhang, Phillip Isola, and Alexei A Efros. Colorful image colorization. In ECCV, 2016.
Martin A Zinkevich, Alex Davies, and Dale Schuurmans. Holographic feature representations of deep networks. UAI, 2017.
12


Reviews:
[{"review_id": "r1l25t7-9S", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary: This paper considers the use of a neural network's Jacobian as additional features for semi-supervised, unsupervised, and transfer learning of representations. The idea is simple and the authors motivate this choice by connecting it to the literature on the neural tangent kernel (although nothing is proven in this paper). Some experiments are performed in which the authors demonstrate some improvements over the simple baseline of using the neural network's final layer alone.\n\nStrengths:\n- The idea is simple and motivated by the Fisher vector work (Jaakkola & Hausler, 1999), which the authors cite.\n\nWeaknesses:\n- Comparing to a baseline with half as many features (just using the final layer of the neural network) is not a compelling baseline. The authors could have considered other alternatives: just using the gradients when comparing against the current baseline, augmenting the baseline with random features of the data.\n- The discussion of the connection to theory added very little to the paper. This is not a theoretical paper, and the hand-wavy connections to the theoretical literature did not bolster the case. Instead, I found it distracting. I agree with the authors that this connection is important to point out, but a paragraph or two suffices.\n- The ablation study could be more compelling. I'm not particularly surprised that a model improves as one increases the number of parameters that are pre-trained on the same kind of data (even if the pre-trianing objective is somewhat distinct).", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I did not assess the derivations or theory."}, "extra_text": {}}, {"review_id": "S1xP7SAaFS", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary: This paper proposes to use the gradients of specific layers of convolutional networks as features in a linearized model for transfer learning and fast adaptation. The method is theoretically backed by an appeal to the recently proposed neural tangent kernel and seems like it could be practically useful.\n\nEdit: Post rebuttal, I am somewhat satisfied by the authors' response and find their ablation study somewhat compelling hence am increasing my score to a weak accept. However, I'm not completely convinced that their choice of layers to use as gradient features is reasonable. If this paper does end up getting accepted, I'd very strongly advocate for being extremely clear as to \n1) why the gradient features used are used, \n2) a good choice of gradient features in practice\n3) inclusion of all of the relevant ablation studies in the final version.\n\nI tend to weakly reject this paper currently despite liking the simplicitly and timeliness of the approach. Specifically, I would like further discussion of the choice of the layers to use as gradient features and an ablation study on supervised trained networks.\n\nOriginality: I believe that this is one of the first papers to explicitly use the Taylor approximation (neural tangent kernel) in a transfer learning setting, making the approach timely and potentially practically useful. \n\nSignificance: The approach is a quite nice merging of theoretical insights with a neat practical implementation. Although the main methodological advance is a straightforward application of the thinking behind Jacobian vector products, the method is well described and ought to be practically useful. However, I have a bit of a concern as to its practical necessity in comparison to simple fine-tuning of the final softmax layer (referred to in Table 1 as \\theta_2) on a new dataset. \n\nClarity: While the approach described in Section 3 is quite generic – theoretically, the method should simply consist of training the final layer of the neural network (w_1) and weights from the Jacobian features around the Taylor expansion. However, the experimental approaches suggest that different layers were used in each experiment – see e.g. “[we] … compute our gradient based features from one, two, or all of the top-3 conv layers” (Section 4.2 for the BiGAN architectures) versus “our gradient based features are from the last 2 conv layers” (Table 2 for the AlexNet architectures). Why was the entire network (modulo the last feed forward layer) not simply used for the Jacobian features? \n\nSimilarly, why was the entirety of the model (again modulo the last feedforward layer) not used for forwards model in the AlexNet experiments?\n\nBased on the ablation study in Section 4.1, the authors find that “it suffices to set the very top layer as \\theta_2 to enjoy a reasonably large performance gain.” When this is the case, it is a bit tough to distinguish the approach from standard final layer transferring approaches (Sharif Razavian et al, 2014) and indeed Bayesian final layer approaches that simply retrain an output layer (e.g. Perrone et al, 2018). Indeed this seems to be the case in Table 1, if that is so, then why not just re-train \\theta_2 as well throughout the experiments as it will typically just be another stochastic convex optimization problem?\n\nCould the authors quickly describe an implementation of the scalable Jacobian inner products in Section 3.3? It seems like outputs will have to be cached during the forwards passes, thereby requiring a somewhat significant amount of software engineering (and memory overhead) to be have to do this process for each new layer. Is this the correct understanding of how one would implement the procedure? A quick paste of PyTorch pseudo-code would be sufficient here.\n\nQuality: The experiments seem to be relatively convincing – it seems exciting that a linear model + the network’s features itself can typically perform as well as fine-tuning and occasionally even better than fine-tuning itself. \n\nHowever, I am a bit concerned by the fact that the ablation studies themselves only utilize models trained in an un-supervised fashion. \nI’d instead like the authors to run an ablation study on supervised trained models (perhaps 8 layer conv nets or VGG16) in the same manner as in Table 1 and Section 4.1. Specifically, I’d like to see this done to see whether the gradients in fact are as interesting as features when they have been trained in a supervised fashion. \n\nSimilarly, I’d like to see the features themselves used as a linear classifier (no network forwards passed) in the same two ablation studies. That is, could the authors use w_1^T J w_2 as the features for their linear classifier. If they have already done so and I’ve missed that in the tables somehow, I apologize. This experiment should help to test out how _useful_ the features defined by the Jacobian matrix are in comparison to the network’s forward pass itself.\n\nMinor Comments: \n\nIntroduction: “After learning, the the (sic) activation of the deep network are considered as generic features.” Not only is there a small typo, but you should either include a citation here or be more specific as to what “the activation” of the deep network is here.\n\nIntroduction: “And the accuracy of the rthe (sic) …” typo + please do not start sentences with and if at all possible.\n\nSection 3.1 (beneath eq. 2): “liner” should be linear.\n\nTable 3: “Self-supervise” should be “Self-supervised.”\n\nReferences: \n\nPerrone et al, Scalable Hyperparameter Transfer Learning, NeurIPS, 2018. http://papers.nips.cc/paper/7917-scalable-hyperparameter-transfer-learning.pdf\n\nSharif Razavian et al, CNN Features off-the-Shelf: an Astounding Baseline for Image Recognition, CVPRW, 2014. https://arxiv.org/abs/1403.6382\n\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "B1gcXmQr5H", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The key idea in this paper is to generate a feature vector based on a Fisher information idea for intermediary levels of a deep neural network. I liked the idea of using some form of linearization of the Fisher information matrix to learn a feature vector. The connection to tangent plane ideas in terms of robustness also made sense. The issue I have is what layer to apply this idea to and formally relating this idea to the variational VAE type framework. Using the top layer seems arbitrary. Also using one layer seems arbitrary. Is there a way to argue via a perturbation analysis of the variational problem of what makes the most sense. I feel the paper hints at this but does not make this idea explicit. ", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper makes a reasonable contribution to extracting useful features from a pre-trained neural network.  The approach is conceptually simple and sufficient evidence is provided of its effectiveness.  In addition to the connection to tangent kernels there also appears to be a relationship to holographic feature representations of deep networks.  The authors did do a reasonable job of providing additional ablation studies, but the paper would be improved if a clearer study were added to investigate applying the technique to different layers.  All of the reviewer comments appear worthwhile, but AnonReviewer2 in particular provides important guidance for improving the paper.
Decision: Accept (Poster)