It's DONE: Direct ONE-shot learning with Hebbian weight imprinting
Authors: Kazufumi Hosoda
Keigo Nishida
Shigeto Seno
Tomohiro Mashita
Hideki KASHIOKA
Izumi Ohzawa
Venue: NeurIPS 2022 Submitted (2022)
Keywords: One-shot learning
Few-shot learning
Weight imprinting
Hebbian theory
ImageNet models
New class addition

Abstract: Learning a new concept from one example is a superior function of the human brain and it is drawing attention in the field of machine learning as a one-shot learning task. In this paper, we propose one of the simplest methods for this task with a nonparametric weight imprinting, named Direct ONE-shot learning (DONE). DONE adds new classes to a pretrained deep neural network (DNN) classifier with neither training optimization nor pretrained-DNN modification. DONE is inspired by Hebbian theory and directly uses the neural activity input of the final dense layer obtained from data that belongs to the new additional class as the synaptic weight with a newly-provided-output neuron for the new class, by transforming all statistical properties of the neural activity into those of synaptic weight. DONE requires just one inference for learning a new concept and its procedure is simple, deterministic, not requiring parameter tuning and hyperparameters. DONE overcomes a problem of existing weight imprinting methods that interfere with the classification of original-class images. The performance of DONE depends entirely on the pretrained DNN model used as a backbone model, and we confirmed that DONE with current well-trained backbone models perform at a decent accuracy. 

It's DONE: Direct ONE-shot learning with Hebbian weight imprinting
Anonymous Author(s)
Affiliation Address email
Abstract
Learning a new concept from one example is a superior function of the human brain and drawing attention in the field of machine learning as a one-shot learning task. In this paper, we propose one of the simplest methods for this task with a nonparametric weight imprinting, named Direct ONE-shot learning (DONE). DONE adds new classes to pretrained deep neural network (DNN) classifier with neither training optimization by Hebbian theory and directly uses the neural activity input of the final dense layer obtained from data that belongs to the new additional class as the synaptic weight with statistical properties of the neural activity into those of synaptic weight. DONE requires just one inference for learning new concept and its procedure sim- ple, deterministic, not requiring parameter tuning and hyperparameters. DONE overcomes problem of existing weight imprinting methods that interfere with the classification of original-class images. The performance of DONE depends entirely on the pretrained DNN model used as a backbone model, and we confirmed that DONE with current well-trained backbone models perform at a decent accuracy.
2
3
pretrained-DNY -NN modification. DONE is inspired
neuron for the new class, by transforming all
10
11
12
15
16
1 Introduction
18 As is well known, artíficial neural networks are initially inspired by the biological neural network in
19 the animal brain. Subsequently, Deep Neural Networks (DNNs) achieved great success in computer
20 vision 9 14. 20 and other machine learning fields. However, there are lots of tasks that are easy
21 for humans but difficult for current DNNs. One-shot learning is considered as one of those kinds 22 27 Humans can add a new class to their large knowledge from only one 23 input image but is difficult for DNNs unless another specific optimization is added. Usually,
22 of tasks
24 additional optimizations require extra user skills and calculation costs for tuning parameters and hyperparameters. Thus, for example, if an ImageNet model [6] [66] that learned 1000 classes can learn new class "baby" from one image of a baby with neither additional training optimization nor Pretrained-DNN modification, it will be useful in actual machine learning uses.
26
27
28 For a DNN model trained with a sufficiently rich set of images, a reasonable representation of
29 unknown images must exist somewhere in the hidden multi-dimensional space. Indeed, weight imprinting, proposed by Qi et al. [26], can add novel classes to Convolutional Neural Networks (CNNs) by using final-dense-layer input of new-class image without extra training. Qi's s weight imprinting method needs just a few CNN-architecture modifications and can provide decent accuracy Submitted to 36th Conference Neural Information Processing Systems (NeurIPS 2022). Do distribute.
31
32
33 in one-shot image classification task (e.g., accuracy for novel-class images was 21 % when novel

---

49 equal to its change, i.e., Weat = 0 0+ Aw A Weat- Hebbian theory is about this Aweat and (R) secions
aa)
Original model
New-class-adee model
(b)
oblained
vector
X
X
W
Ranking information
ari
No change
Frequency
Ware
Dog
Neciial activits
Synsptc weight
Dog
F( 00, Wave
data
Transformation by quantile normalization
Wor tatore AWaa
dess vector)
Transformation by quantile normalization
= of + Fxomd
M x, Mave
OO OR Synaptic ght
Figure I: Scheme of DONE, (a) The neural activity input of final dense layer (orange a vector in original model) obtained from a new-class data (an image of a cat) is directly used for the transformation to the new-class vector (orange Weat) in the new weight matrix (W) without any modification to the backbone model. (b) An example case of transformation from æ to Weat, with actual distribution data when the backbone DNN is EfficientNet-BO.
34
100 classes were added to the original 100 classes in CUB- (UB-2000-211 2011 dataset). Moreover, some studies show that the capabilities of DNN itself have the potential to enable Out-of-Disrribttion Detection (OOD). For the relationship between the brain and artificial computation, not only some of the features of the lower 4 and higher visual cortex [ are explained by filtering or DNN, but also cooncpttlearning functions into the hidden space has been analyzed [39].
35
21
36
36
37
38 the embedding of such
39 In this paper, we introduce very simple method, Direct ONE-shot learning (DONE) with nonpara- metric weight imprinting. As shown in Figure aa), DONE directly transform the input of the final dense layer (æ vector in the figure) obtained by one input image belonging to a new class (e.g.. a cat in the figure) into the weight vector for the new additional class (Weat. a row vector the weight matrix W). Then, it is done. DONE uses weight imprinting but never modifies backbone DNN including original weight matrix W ori unlike Qi's method. Qi's method was inspired by the context inspired by Hebbian theory [2]. This difference in inspiration
40
41
42
43
44
of metrie learning, but DONE source makes small but important difference in method procedures, and this study proposes new formulation of Hebbian theory for weight imprinting.
46
47
48 In weight imprinting, we can assume that the new weight vector What born out of nothing and thus
50 states that synaptic weight is strengthened when both its presynaptic and postsynaptic neurons
51 active simultancously. When single image of a new class (cat) is presented visual input, some of the presynaptic neurons x become active. Simultaneously. postsynaptic neuron corresponding
52
53 to cat is active (e.g., Veat = 1), while postsynaptic neurons for all the i-th original classes are not
54 (y = 0), because the training image is known to be a cat. In simple and conventional formulation
55
of Hebbian theory, the change in the weight vector can be described as AWeat X æ Yeat. thus because y/: O. Therefore, the mechanism of weight imprinting without modification of W.ri ori can be explained by Hebbian theory.
56
Weat AWeat & x, while Aw;
57
Here, a problem arises with this simple formulation alone, because neural activity and synaptic weight 59 are different in scale and those relationships would not be linear. For example, Figure (i)- and (i) show frequency distributions of neural activity in x and weight in w,, which are different in 61 shape, an actual DNN. If only the new Weat had far different statistical properties compared the
58
60
62 other ws the comparison between classes would be unequal, and the additional Weat could inhibit the classification of the original classes (shown later). Therefore, the implementation of Hebbian theory here must include function for the nonlinear scale transformation, i.c., AWeat Fa-ww( Meat)..
83
65
DONE takes into account this transformation by quantile normalization [3, so that the frequency distribution of Weat becomes equal to that of Wave (the average vector of original w, vectors), i.c., Aurat F(x. Wave) (Figure I (ii().... Quantile normalization is an easy and standard technique in Bioinformatics [ 3) and it suitable for implementing Hebbian theory. The statistical properties of Weat that result from the transformation from neural activity to synaptic weight should be similar to the statistical properties of original synaptic weights. For example, could apply linear transformation
86
67
88

---

71 so that the mean and variance (i.e., 1st and 2nd central moments) of the elements of Weat are the same
72 as those of aave However, it is not clear such adjustment for only 1st and 2nd central moments is
73 enough in this situation where the 3rd or higher central moments could be different (such as shown in
74 Figure b))) One of the simplest solutions for every situation is to make the statistical properties identical. As above, DONE can simply add a new class by West F(z Wave) nonparametrically. and we call this transformation Hebbian weight imprinting (see Methodology section for details).
75
77
Our method's 's basis and procedure are very simple, but it achieves similar accuracy to Qi's method and does not interfere with the original classification compared to Qi's method. DONE achieved over 50% accuracy (approximately 80% of classification well-trained original classes) in a one-shot image classification task that adds eight new classes to a model pretrained for the ImageNet 1000 classes (ViT (Vision Transformer) [4 EfficientNet
78
79
80
81
[29] as backhone model (note that the chance level is less than 0. 0.). In typical five-way one-shot classification task, DONE with ViT achieved over 80% accuracy.
84 The advantages of DONE over other weight imprinting methods are (i) Hebbian- -inpiired simpler basis 85 and procedure, (ii) no modification to backbone models, and (iii) nonparametric procedure for a wide range of backbone models including future models. The advantages of DONE as a weight imprinting (iv) optimization thus little calculation cost and (v) no parameters or hyperparameters thus reproducible for anyone. In addition proposing the new methods, this paper contains the following useful information: generic task to add new classes to 1000-class ImageNet models, and the difference in backbone DNNs, specifically, between Transfomer (ViT) and a CNN (EfficientNet). Moreover, DONE may provide a useful insight when exploring the learning principles of the brain
86
87
89
90
81
92
because DONE is inspired by the Hebbian theory.
2
Related work
2.1
One-shot and few-shot image classification
95
Typical learning approaches for one- or few-shot image classification are metric learning, data augmentation, and meta learning. Weight imprinting has come out from metric learning. Each of these approaches has its own advantages and purposes, and they are not contradictory but can be used in mixed manner.
96
97
98
Metric learning uses classification at feature space metric space 8
22 30
Roughly speaking,
100
metric learning aims to decrease the distances between training data belonging to the same class and increase the distance between the data belonging to different classes. Metric learning such as using Siamese network [13] useful for tasks that require one-shot learning, e.g., face recognition. A Data-augmentttion approach generatively increases the number of training inputs 119 27 45). This
101
102
103
104
approach includes various types such cumi-supervised approaches and example generation using Generative adversarial networks (10). Meta learning approaches train the abilities of learning systems
to learn [18 23 43 The purpose of meta learning is to aim to increase the learning efficiency itself, and this is a powerful approach for learning from small amount of training data, typically one-shot
107
108
learning task 41
2.2
Weight imprinting
Weight imprinting is a learning method that initially arose from an innovative idea "learning without optimization" (26), and DONE is a type of weight imprinting. Weight imprinting does not contain any optimization algorithm and is basically inferior to other optimization methods by themselves in 112
110
111
113 accuracy. However, comparisons of DONE with other optimization methods are useful in evaluating
114 those optimization methods, because the performance of weight imprinting methods is uniquely determined by the backbone DNN without any randomness. Thus, weight imprinting does not aim for the highest accuracy but for practical convenience and reference role bascline method.
3

---

117
We here explain the basis of weight imprinting and then specific procedure of Qi's method. Let us consider the classification at the final dense layer of DNN models in general. In most cases, the output vector y = (8h, ... yN' of the final dense layer denotes the degree to which an image belongs to each class and is calculated from the input vector the final dense layer x = (#1,, ,Tr),, weight matrix
118
119
120
121
W (N MM, and bias vector b (b1..... bx). ). Here, for i-th class in N classes (i 1, . .N.
122
scalar Yi is calculated from the corresponding weight vector w, (W51,'' wiss) (i-th row vector
of W matrix) and bias scalar b, the following equation:
y x w. + b; ||~/|| ||w, cos + bj.
124
where the cosine similarity is a metric that represents how similar the two vectors æ and w, are irrespective of their size. Thus, this type of model contains cosine similarity as a part of its objective function. It is also possible to use the cosine similarity alone instead of the dot product [25].
125
126
Weight imprinting uses this basis of the cosine similarity. The cosine similarity will have the maximum value 1 if and w, are directly proportional. Thus, if certain is directly used for the weight of new j-th class wy (j = N 1,..,, the cosine similarity for j-th class becomes large when another æ with a similar value comes.
129
130
131
In Qi's s method, to focus only on the cosine similarity as a metric for the objective function, the backbone DNN models are modified in the following three parts:
132
133
Modification 1 Adding Le normalization layer before the final dense layer so that æ becomes unit vector, i.e., | =1
135
Modification 2: Normalizing all w; to become unit vectors, i.e., /1sll2 = 1 for all i.
Modification 3 Ignoring all bias values b, i..., b vector.
Then, the final-dense-layer input obtained from a new-class image Fnew (Lz-normalized, in Qi's method) is used as the weight vector for the new class Wj, i.e.
W
Knew-
139
Qi's method is already simple and elegant, but it still requires some modifications to the backbone DNN, which involves changes in the objective function. Whether a modification is good or bad depends on the situation, but if not necessary, it would be better without modification in order to avoid unnecessary complications and unexpected interference with the original classification because the backbone DNN would be already well optimized for certain function. Also, Qi's method uses linear transformation for conversion of x into W as a result of focusing on the cosine similarity, without considering the difference in statistical properties between æ and Wy, which limits the range of backbone DNNs used. There have been various researches that make Qi's method more complex and applicable 32 38 40 44 46), but to the best of our knowledge, none that make it simpler or
140
144
145
146
147
148
solve the transformation problem.
3
Methodology
149
150
3.1
Procedure and basis of DONE
151
DONE does not modify backbone DNN and just directly applies Fnew to w. (j = N + 1,,..) as shown in Figure j
152
W = F (Dnew, Wave);
(3)
by bori-
(4)
153 where F(#new; Wave) is a quantile normalization of Dnew, using the information of the average weight vector for original classes (Wave) as the reference distribution, and b is the median of the original bias vector boci. Then, it is done.

---

156
In the quantile normalization, the elements value of resultant wy become the same as that of the reference ave- Specifically, for example, first we change the value of the most (1st) active neuron in nuew vector into the highest (Ist) weight value Wave- We then apply the same procedure for the 2nd, 3rd,- ... M-th highest neurons. Then, the ranking of each neuron in &new remains the same, and the value of each ranking is all identical to Wave. This resultant vector is w, Namely, all statistical properties of the elements of wy and Wave identical (the frequency distributions the same).
157
158
159
180
162
For Wave, to implement the concept of Hebbian theory, we used all the N x M elements of flattened ori divided these elements into M parts in ranking order, and obtained the M median values in each element as A -diimnnsional Wave- For example, in the case of ViT-B/32 (N = 1000, M 768), the highest value of Wave is the median of 1st to 1000th highest elements in 768,000 elements of W or and the lowest value is the median of 767,001th to 768,000th elements.
163
164
165
166
167
3.2
Limitations, applications, and potential negative societal impacts
168
As limitations, DONE requires a neural network model that has dense layer for classification as above. DONE can be used for a wide range of applications with DNN classifiers, including OOD applications 4 There can be various potential negative societal impacts associated with these broad applications. One example is immoral classification discrimination when classifying human-related data, such as facial images, voices, and personal feature data.
189
170
172
173
3.3
Implementation and Dataset
As backbone models, employed ViT-B/32 34 and EfficientNet-BO [29] as two representative
DNNs with different characteristics, using "vit-keras" 48 and "Effficienteet Keras (and TensorFlow Keras)" 47). respectively. Also, for transfer learning, we employed InceptionVI [15] (employed in
178
177
the paper by Qi et al. 266 and ResNet-12 (20), using "Trained image classification models for Keras" and Tensorflow
178
49
respectively. All models used this study were pretrained with ImageNet.
179
We used CIFAR-100 and CIFAR-10 7 for additional classes, using Tensorflow 12 Also, for transfer learning, we used CIFAR-FS 28 by Torchmeta (31) We used ImageNet (ILSVRC2012)
images 6 16] for testing the performance of the models. We used information of 67 categorization
182
33 of ImageNet 1000 classes, for a coarse 10 categorization in Figure [). All images were resized
183
to (224 x 224) by OpenCV (50] or the preprocessing resizing layer of Tensorflow [1]
4
Results and Discussion
184
185
4.1 One-class addition by one-shot learning
First, according our motivation, we investigated the performance of DONE when new class from one image added to DNN model pretrained with ImageNet (1000 classes). As new additional classes, we chose eight classes, "baby", "woman", "man", ''aterpiilarr", "cloud", "forest", "maple_tree", and "sunflower" from CIFAR-100, which were not in ImageNet (shown in Figureßaa). The weight parameters for the additional one class wy is generated from one image, thus the model had 1001 classes. To conduct stochastic evaluations, 100 different models were built by using 100 different training images for each additional class.
188
189
190
191
192
193
2 b) shows letter-value plots of the accuracy for each additional class (chance level 1 1/1001). The mean of the median top-1 accuracy of 8 classes by DONE were 56.5% and 92. % for ViT and EfficinetNet, respectively (black line). When the mean accuracy was compared with the accuracy of ImageNet validation test by the original 1000-class model (orange line; 65% and 69%), the mean with ViT had no significant difference and the mean of EfficinetNet was significantly greater (one sample i-test; two-sided with a=0.05, in all statistical tests in this study). The higher accuracy than the original classes in EfficientNet is strange, and it is considered that EfficientNet tends to recognize the new-class images as just OOD (see later, Figure 4
Figure
194
195
196
197
198
5

---

(a)
#1 Women
#d Calerpiler
#4 Cloud
ei Madle Tree Sumlooer
(1 Vision Transformer
(1) EfficienNNet
Batry
Forest
Highest
Synaptc w
weight
Median
Lovest
-u0 ua use 6no
6.00
Neural activity in x obtained by "Ruddy tumstoon" image
DONE
(b)
Wortowe in
Median nes--lles Onginaliclass class accuracy Intorference by new class
(i) Vision Transformer
(ii) EfficientNet
100.
Top-1 accuracy (%)
80.
Interference fraction (%)
f
40-
20-
A
A
4.A
A
A'A
A7A
4
A
A
A
40
21
#2 Men
43 Catrrlarr
444 Cloud
455 Forest
a6
Baby
Woman
Maple Tree Surllower
Baby
Woreen
Man
Caterilaar
Cloud
Forest
Meçle Surfower
Newly-added class
Figure 2: One-class addition by one-shot learning. (a) Representative images of the newly-added CIFAR-100 classes. Each image was chosen as representative because the model that learned the image showed the highest, median, and lowest accuracy in each class in (b)-(i). (b) Letter-value plots of top-1 accuracy of the one-class-adde models obtained by one-shot learning with DONE in classification of new-elass images. The median top-1 accuracy of the new-class classification (black circles), top-1 accuracy in original-class classification (orange circles), and the fraction of the interference with the original-class classification by the newly-added class (blue) are also plotted for DONE (closed) and Qi's method (open). Black and orange lines show the mean of the closed circles. (c) The relationship between and vectors when image of "Ruddy turnstone" is input and it miss-classífied as "Sunflower" only in the case Qi's method with EfficientNet. The frequency distributions of elements of each vector are also shown outside the plot frames.
201
An obvious fact in one-shot learning is that a bad training image produces a bad performance, e.g.. the accuracy was 6% VIT when the training image was baby image shown at the bottom left in 202
Figuregaa). But in practical usage, user is supposed to use representative image for the training. We therefore think that the low performance due to bad training image is not significant issue.
205
We investigated the interference of the class addition with the classification performance of the original 1000 classes. We evaluated the original 1000-class model and eight 1001-class models that showed the median accuracy, by using all 50,000 ImageNet validation images (Figure (b))). The difference between the accuracy of the original 1000-class model (orange line) and the mean accuracy of the eight 1001-class models (orange closed circles) was less than 1% (0.004% and 0.664% for ViT and EfficinetNet, respectively).
206
207
211 Figure b) also shows the fraction of ImageNet validation images in which the output top-1 answer
212 of the added model was the new class (thus incorrect) in the 50,000 images (blue closed triangles; right axis). This interference fraction was low in VIT, and for example, only images out of 50,000 classified "baby". When checked the images, both images indeed contained baby though its class in ImageNet was "Bathtub". Therefore, observed interference in ViT was not mistake but just the result of another classification. EfficientNet shows a significantly greater fraction of interference than ViT (Wilcoxon signed-rank test), but we also confirmed that a similar thing happened, e.g.. 198 of the 204 ImageNet-validation images classified as "baby" in EfficientNet
213
216
217
218
219
contained human or doll.
We also compared DONE with Qi's method. Open circles and triangles show the results using Qi's method instead of DONE in the same tests described above. When the backbone model

---

(a)
1-shot
(b)
10-shot
(c)
100-shot
(d)
[mm]
Vision Transformer
Vision Transformer
Top-1 Iiterference fracilon (%)
50-
98%
85%
75%
Mean top-t spcuracy
DONE
Accuracy Interference
Output
[mm]
[m))
Qi's
[m]]
100.
Accuracy Intarferance
X:
15%
75-
5%
50-
(ii EfficientNet
(i) EfficientNet
7(m)
7(m)
Class of input images
K-shot
Figure 3: Multi-class addition and K-shot learning. (a). (b), and (c) show the results of the 1008-class model constructed by 1, 10, and 100-shot learning, respectively. The horizontal and vertical axes show the class of the input images, and the output class, respectively. The class numbers are those shown in Figure aa. The class [im] contains 1000 classes ImageNet. (d) Summary the mean accuracy and the interference fraction with original-class classification by DONE and Qi's method.
222
was EfficientNet, the strangely-high accuracy (paired sample t-test) and the interference fraction (Wilcoxon signed-rank test) were significantly greater by Qi's method than by DONE, Also, a significant outlier of decreased accuracy in the ImageNet validation test was observed (orange open circle for "Sunflower"; Smirnv-Grrubbs test). On the other hand, those differences were not significant in the case of VIT.
223
224
225
228
To investigate the cause of the difference between DONE and Qi's method, especially about the greater interference by Qi's method in EfficientNet, we plotted WSunflower and WRuddy turnstone against æ obtained from an image of "Ruddy turnstone" (Figure (c) Note that all the vectors here are Le nermalizzdd, and thus DONE and Qi's method have common WRuddy_turnste turnstone and æ. In the case of ViT, the shape of the frequency distributions of all these vectors are similar, and WSnffoower of DONE and Qi's method are similar. On the other hand, in EfficientNet, the shape of frequency distributions are more different between WRuddy turnstone and than ViT, and thus the shape of frequency distributions are more different between Winddy turnsstne and WSunflower by Qi's method
228
"Ruddy_turnstone
229
230
231
232
235
than by DONE. Then, by Qi's method, æ is more similar to WSunflower than ttansttee because not neuronal match but statistical properties are similar. This is the basis of the problem by linear transformation of neural activity to synaptic weight. Therefore, the difference between DONE and
236
237
Qi's s method appears in the interference when the statistical properties of and wn vectors in the 238
239
backbone DNN are different (thus the results in ViT are similar between DONE and Qi's method).
240
4.2
Multi-class addition and K-shot learning
241
DONE was able to add new class as above, but it might just be because the models recognized the new-class images as OOD, i.c., something else. Therefore, it is necessary to add multiple new similar classes and check the classification among them. In addition, it is necessary to confirm whether the accuracy increase by increasing the number of training images, because in practical uses, users will prepare not just one training data but multiple training data for each class.
242
243
244
245
246
Specifically, we used one image from each of the eight classes and added new eight classes to the original 1000 classes, using DONE as one-shot learning. We evaluated this 1008-class model by 100 CIFAR test images for each of 8 classes and 10,000 ImageNet validation images. Figure 3 a) shows the results of the output of the representative model constructed by one-shot learning in which one
247
248
249
250
image that showed median accuracy in Figure 2 b)) was used as standard training image of each
class. In both backbone DNNs, the fraction of output of the correct class was the highest among the 1008 classes, and mean top-1 accuracy of the 8 classes was 51.8% and 61. 1% in ViT and EfficientNet.
7

---

Vision Transformer& & DONE
(b) EfficientNet & DONE
(c) EfficientNet & Qi's
Hi veclors for original 1000 classes
03-
inverlecrete
reptle
1.5
5
plant-lkk
02-
Tohlko
02-
M| vectors for added ddesses by images
1
6
0.1-
PC2
0.5
7
6
0.C
H] vectors by imageNet images
-0.1
5
-0.1
1
-02
-02-
peraton
sceneeyy
hoavies
Intal point: 1-shot by 100 100-shot
PC1
Figure 4: Principal component analysis of weight vectors. PCA of each w, and wy vector the one- shot 1008-class models shown in Figure a.). Different colors for w, show coarse 10 categorization of the classes. Also, 100 w vectors obtained by inputting 100 ImageNet images are shown.
253
respectively. That is, DONE was also able to classify newly added similar classes together with the original classes, in both DNNs.
254
Next, we increased the number of training images as K-shot learning. In the case of 10-shot learning (Figure Bb)), each of the ten images was input to obtain each æ, and the mean vector of the ten æ vectors was converted into Wy, according to the Qi's method. For this representative 10-shot model,
256
257
258
we used 10 images whose index CIFAR-100 was from the front to the 10th each class. We also tested 100-shot learning in the same way (Figure ()). As a result, we found that such a simple averaging operation steadily improved the accuracy (Figure 3 3d) summaries the mean accuracy).
259
260
261
When we used Qi's method, compared to the case of DONE, ImageNet images were significantly more often categorized to the new classes as interference only when the backbone model was EfficientNet (paired sample f-test), while there was no significant difference in the mean accuracy of
262
263
254
1, 10, 100-shot 1008-class models for the added classes between DONE and Qi's method with both
285
backbone DNNs (Figure ())) Thus, again the interference the case of EfficientNet is significantly
greater in Qi's method than DONE.
287
4.3
Principal component analysis of weight vectors
268
Qi's method showed greater interference in classification of the original-class images than DONE only when the backbone DNN is EfficientNet. Moreover, even by DONE, EfficientNet showed greater interference than VIT and strangely-high accuracy at 1001-class model, even though DONE
289
270
271
did not change the weights for the original classes and transformed the new-class weights so that the statistical properties were the same as those of the original-class weights. Therefore, there should
272
273
be at least two reasons for these results only shown in EfficientNet, and DONE cannot correct least of them. To investigate those reasons, analyzed W matrix (w; and w, vectors) of the one-shot 1008-class models shown in Figure 3 (a) (and corresponding models by Qi's s method) by
278
Principal component analysis (PCA; Figure 4)
277 In VIT by DONE (Figure a): 4a): Qi's methods showed similar results, see Figure S1), newly added un vectors (black circles, with the ID number of newly-added classes) were comparable those of the original classes w, (colored circles), e.g., wy vector of class "caterpillar (3 in a)))" was near w, of original "invertebrate" classes. Also, even when we got wy by inputting ImageNet images (red crosses; validation ID from the front to the 100th), those ImageNet w, vectors distributed in similar range.
278
279
281
282
283
On the other hand, in EfficientNet by DONE (Figure most of newly-added 8-class wn were of the distribution (meaning out of minimal bounding ellipsoid) w; of original 1000 classes, while most of the ImageNet w, (red crosses) were inside the distribution of w,, Therefore, in the case of DONE, the main reason for the observed greater interference and strangely accuracy
284

---

(a) 5-way 1-shot classification
(b) w-way 5-shot classification
100
60-
DONE
Accuracy
60-
Qr's method
60-
Bendou et al Zhang
20
InceptionV1
BO InceptionV1
ResNel-12
Backbone DNN
Figure 5: 5-way 1-shot (a) and 5-shot (b) classification accuracy on CIFAR- FS with various backbone DNNs, Error bars show standard errors. Asterisks mean significant differences between DONE and Qi's s method (Dwass-Steer Steel-Critchlor Fritchlow-Fligner test).
287
in EfficientNet than ViT would be the difference between ImageNet and CIFAR. These results are consistent with known facts that ViT is considered be better predictive uncertainty estimation
288
24
377, more robust input perturbations [35]. and more suitable classifying OODs (36] than
CNNs like EfficientNet.
281
In EfficientNet by Qi's method (Figure 4 cc),, most of not only 8-class wy but also the ImageNet
292
w were out of the distribution w, of original 1000 classes. The difference in the distributions
293
between the original w, and the ImageNet wj is considered to indicate the difference in the statistical and w, vectors in EfficientNet.
294
properties
In the case of 100-shot learning (the terminal points of the gray arrows in Figure 4p. wy went away from the cluster of original w; in all three cases, although their performance was better than one-shot learning. Therefore, 100-shot wj were considered to work somehow in a different way from the original wt
296
297
298
299
4.4
Transfer few-shot learning
300
DONE is recommended for the easy addition of new classes, not for transfer learning. However, DONE can work for it (Figure S2) and is convenient for the evaluation of DNNs and other few-shot learning methods. We examined the 5-way (5 classes) I I-shot task CIFAR-FS, which is kind of standard task ohe-shot classification. Specifically, we used each single image in 5 classes out of
301
302
303
100 classes of CIFAR- 100 for constructing model, and evaluate the model by 15 images in each class. The combination of the 5 classes (and corresponding training images) randomly changed in 100 times (Figure a)). Also 5-way 5-shot task was tested in similar way (Figure 5()))
a)).
307
We found VIT significantly outperformed the other DNNs in all conditions (Dwass- Trrass-Stecl-oee Cretc-Crthhhow- Fligner test) by both DONE and Qi's method. Compared to Qi's method, DONE shows significantly greater accuracy with some CNN models, and never significantly worse, although it is not an expected advantage of DONE and there would be particular reason for it.
308
309
311
Figure also clearly shows that how much other state-of-the-art one-shot learning methods with optimization (methods in [53] and (45) outperform DONE as the baseline without optimization, at the same test with a common backbone DNN (ResNet-12))
312
313
5
Conclusion and Future work
This paper has proposed DONE, one of the simplest one-shot learning methods that allows to add new classes to pretrained DNN at a decent accuracy without optimization or modification of the DNN. DONE applies Hebbian weight imprinting, which is a new implementation of Hebbian theory by quantile normalization, to the final dense layer of a DNN model. Given the simplicity and wide applicabilityy, not only DONE but also Hebbian weight imprinting alone are expected to be applied in wide range of the field of neural networks. This study has just proposed the method, and its
317
318
319
320
scalability (Figure S3) and expected applications are yet to elucidated. Since the performance of
DONE is completely dependent backbone DNNs and further development of DNN is certain, the
situation to obtain sufficient accuracy with DONE may soon come.

---

References
324
325
http://www.....p//////////tttt/s [1] D. Amaratunga and J. Cabrera, "Analysis of Data From Viral DNA Microchips." Journal of the American Statistical Association, vol. 96, no, 456, pp- 1161-1170, Dec. 2001, doi: 10.1 111119996614551153334
326
[2] D. O. Hebb, The Organization of Behavior; A Neuropsychological Theory. New York: Psychology Press, 2002. doi: 10.4524/9781410612405
328
[3] B. M. Bolstad, R. A. Irizarry, M. Astrand, and T. P. Speed, "A comparison of normalization methods for high density oligonucleotide array data based variance and bias," Bioinformatics, vol. 19, no. 2, pp. 185-193, Jan. 2003, doi: 10. 109//iiinforrmaiics/// 19......cc///........ 185.
330
331
332
[4] M. Carandini al., "Do We Know What the Early Visual System Does?," Neurosci. vol. 25, no. 46, pp. 10577-10597, Nov. 2005, doi: 10. 00.522/NEUURSSC133726600..000
333
334
[5] L. Fei-Fei, R. Fergus, and P. Perona, "One-shot learning of object categories," IEEE Transactions on Pattem Analysis and Machine Intelligence, vol. 28, no. 4, pp. 594-6944496 611, Apr. 2006, doi: 10.1 109.109MM....000
335
336
[6] Deng, W. Dong, R. Socher, L.-J. Li, Li, and L. Fei-Fei, "Imagenet: A large-scale hierarchical image datahase," in 2009 IEFE conference on computer vision and pattern recognition, 2009, pp. 248-255.
337
338
[7] A. Krizhevsky and G. Hinton, "Learning multiple layers of features from tiny images." 2009.
[8] K. Q. Weinberger and L. K. Saul, "Distance Metric Learning for Large Margin Nearest Neighbor Classifi- cation," J. Mach. Learn. Res., vol. 10. pp. 207-244, Jun. 2009.
340
[9 A. Krizhevsky, 1. Sutskever, and G. E. Hinton, "ImageNet Classification with Deep Convolutional Neural Networks," " in Advances Neural Information Processing Systems, 2012, vol. 25. Available: http://ppppppp..aaaaaaaaaoooaoba
342
343
344
[10]
Goodfellow
al., "Generative Adversarial Nets," " Neural Information Processing Systems, 2014, vol. 27. wwwpreccddingnnnuuiss.cc..e
Advances
345
Available:
346
wff.cfff3l-a-cc--ooo httpr//aaaccaa-tttaattt.t. Abstracttttttm
347
[1]] D. L. K. Yamins. H. Hong, C. F. Cadieu, E. A. Solomon, D. Seibert, and J. DiCarlo, "Performance- optimized hierarchical models predict neural responses in higher visual cortex." Proceedings of the National Academy of Sciences, vol. 11 111, no. 23, pp. 8619-8624, Jun. 2014, doi: 10. 10.1073/pnas. 14031 12111.
348
350
[12]
M. Abadi et al.. "TensorFlow: Large-scale machine learning on heterogeneous systems." 2015.
351
[13] G. Koch, R. Zemel, and R. Salakhutdinov. "Siamese neural networks for one-shot image recognition." in ICML deep learning workshop, 2015, vol. p. O.
352
353
[14] K. Simonyan and A. Zisserman. "Very Deep Convolutional Networks for Large-Scale Image Recognition." "
354
in 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings. 2015. Available: http://arxiv.orr//bb///499 155511555/
356
[15] C. Szegedy et al., "Going Deeper With Convolutions." in Proceedings of the ICEE Confer- ence on Computer Vision and Pattern Recognition, 2015, pp. 1-9. Available: https:/www.ev- foundation.org/ope wwtp://pwaccs//ooeeee...pp./blla Deepee Doeper__itt__2111 Sitthh2005_CVP__ppeeree
358
Going
[16] O. Russakovsky al., "ImageNett Large Scale Visual Recognition Challenge." 2015, Available: http://arriv.orr//as///00....000
Jan.
360
[17] B. M. Lake, R. Salakhutdinov, and B. Tenenbaum, "Human-level concept learning through probabilistic program induction," Science, vol. 350, no. 6266, pp. 1332- 1338, Dec. 2015, doi: 10. 1122:icience.aaa30555
382
[18] M. Andrychowicz et al.. "Learning learn by gradient descent by gradient descent." Advances in Neural Information Processing Systems, 2016, vol. 29. Available: Abstraacttt
384
365
386
[19] O. Vinyals, C. Blundell, Lillicrap. Koray Kavukcuoglu, and D. Wierstra, "Matching Networks for One Shot Learning." " in Advances in Neural Information Processing Systems, 2016, vol. 29. Available: https:/papers.ips.ccrppp 1ttec//aaprr000///h/0091111111
367
358
369
[20]
K. He, x. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition." " in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2016, pp. 778. doi: 10.1109/CVPP.2011..90
370
10

---

372
hwtpp//www./ppp///////////////o. [21] B. Lakshminarayanan, A. Pritzel, and C. Blundell, "Simple and Scalable Predictive Uncertainty Estimation
using Deep Ensembles." in Advances in Neural Information Processing Systems, 2017, vol. 30, Available:
374
375
[22]
Snell, K. Swersky, and R. Zemel, "Prototypical Networks for Few-shot Learn ing." in Advances in Neural Information Processing Systems, 2017, vol. 30. Available: hups://pooeeedings.nerrrp
376
377
378
[23] C. Finn, P. Abbeel, and S. Levine, "Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks." in Proceedings of the 34th International Conference on Machine Learning, Jul. 2017, pp. 1126-1135. Available:
http://prooeedinss.lrppssss/ mlr plrppess//7000fmm 70t.m///////m/aaaaamlll
380
381
[24] C. Guo, G. Pleiss. Y. Sun, K. Q. Weinberger, "On Calibration of Modern Neural Networks." in Proceedings of the 34th International Conference on Machine Learning, Jul. 2017, pp. 1321-1330. Available:
382
383
http://wwwoodnngnsmmrrer///////e 1 7a.html
384
[25] C. Luo, J. Zhan, X. Xue, L. Wang, R. Ren, and Q. Yang. "Cosine Normalization: Using Cosine Similarity Instead of Dot Product in Neural Networks," in Artificial Neural Networks and Machine Learning ICANN 2018, Cham, 2018, pp. 382-391. doi: 10. 0000/978-3-030-1444
386
387
[26] H. Qi. M. Brown, and D. G. Lowe. "Low-Shot Learning With Imprinted Weights," in Proceedings of the IEFF Conference on Computer Vision and Pattern Recognition, 2018, pp. 5822-5830. Ac- cessed: May 02, 2022. Available: Shot_Learning_ 1ith__VPR_2000 20ppp_ppr_ppppmmmlll
388
389
[27] E. Schwartz et al., "Delta-encoder: effective sample synthesis method for few-shot object recognition," " in Advances in Neural Information Processing Systems, 2018, vol. 31. Available: hups://proceedings hetp:iiggneerrrpssc/aaprr/000000 http://aaprr2001//aa/a///1222220
392
393
394
[28] L.. Bertinetto, J. F. Henriques, P. Torr, and A. Vedaldi, "Meta-lcarning with differentiable closed-form solvers," presented at the International Conference on Learning Representations, Scp. 2018. Available: https//ppenneviee..eeeffouur wwtp://ewweeee.....oomooo////aaa
385
396
397
[29] M. Tan and Q. Le. "EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks." in
Proceedings of the 36th International Conference Machine Learning, May 2019, pp. $105-6114. Available: https:/prooceedings.mm www.:mmm....ps.... press/97//ann prrp 19..h//
389
400
[30] M. Kaya and H. S. Bilge, "Deep Metric Learning: A Survey," Symmetry, vol. 11, no. 9, Art. no. 9, Sep. 2019, doi: (0.33300/ym1199900009
401
402
[3]] T. Deleu, T. Wurfl, M. Samiei, J. P. Cohen, and Y. Bengio, "Torchmeta: A Meta-Learning library for PyTorch,"
403
Sep. 2019, Available: http://axii..or/aas///99..0
[32] G. S. Dhillon, P. Chaudhari, A. Ravichandran, and S. Soatto, "A Baseline for Few-Shot Image Classifica- tion." presented the International Conference on Learning Representations, Sep. 2019. Accessed: May
405
408
18. 2022. Available:
407
[33]
N. Eshed, "Novelty Detection and Analysis Convolutional 10. 00.729877233-6118.
Neural Networks," " May 2020, doi:
408
409
[34] A. Dosovitskiy et al., "An Image is Worth 16x 16 Words: Transformers for Image Recognition at Scale," presented at the International Conference Learning Representations, Sep. 2020. Available: wtwp://eeeeeeeeeeeeee//oomo/aaaa
412
[35] S. Bhojanapalli, A. Chakrabarti, D. Glasner, D. Li, T. Unterthiner, and A. Veit, "Un- derstanding Robustness of Transformers for Image Classification." in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 10231- https//openaccess.theee www.eeawcccssstece....ooooooeeee wwtp://.oooooeee//cc/////////eee palli_ Understanding Robustness of Transformers for Classificatiio cicpiiaaaaaaiiiccccceee 2021 paper.hhl
413
414
415
10241. Available:
Bhojana-
[36] S. Fort, J. Ren, and B. Lakshminarayanan. "Exploring the Limits of Out-of-Distributin Detection." Advances in Neural Information Processing Systems. 2021, vol. 34. pp. 7068-7081, Available: Attr//ta000-AAstttac.hh.tmm
418
419
420
[37] M. Minderer et al., "Revisiting the Calibration of Modern Neural Networks." in Advances in Neural Information Processing Systems, 2021, vol. 34. pp. 15682-15699 Available:
421

---

423
[38]
N. Passalis, A. Iosifidis, M. Gabbouj, and A. Tefas, "Hypersphere--asse Based Weight Imprinting for Few-Shot Learning on Embedded Devices." IEEE Transactions Neural Networks and Learning Systems, vol. 32, no. 2, pp. 925-930, Feb, 2021, doi: 10119//TNNSS.00022997977
425
[39]
B. Sorscher, S. Ganguli, and H. Sompolinsky, "The Geometry of Concept Learning" hioRxiv, p- 2021. 2211.03.21.0 4436244, May 2021, doi: 111100021.03..1..25000
427
428
[40]
Li al., "Classification Weight Imprinting for Data Efficient Object Detection," in 2021 IEEE 30th International Symposium on Industrial Electronics (ISIE), Jun. 2021, pp. 1-5. doi: 10. 2009ISSEE45552222199575666
429
430
431
[41] M. Huisman, J. N. van Rijn, and A. Plaat, "A survey of deep meta-learning"" Artif Intell Rev, vol. 54, no.
432
6, pp. 4483-4544 Aug. 2021, doi: 10. (007//10422-021-100000
433
[42] J. Yang. K. Zhou, Y. Li, and Z. Liu, "Generalized Out-of-Distibattoo Detection: A Survey,"
arXiv:2110.11 1334, Oct. 2021, Available: http://arxiv.oor/aas//11 10.1 13334
435
[43] B. Zhang et al., "Sample-Centric Feature Generation for Semi-Supervised Few-Shot Learning," IEEE Transactions Image Processing, vol. 31, pp. 2309-2322 2022, doi: 10.1 1..1099922222231545955
[44] P. Cristovao, H. Nakada, Y. Tanimura, and H. Asoh, "Few Shot Model based on Weight Imprinting with Multiple Projection Head," in 2022 16th International Conference Ubiquitous Information Management Communication (LMCOM), Jan. 2022. pp. 1-7. doi: I0.119/MMCMM5555332222.27777
439
[45] Y. Bendou et al., "EASY: Ensemble Augmented-Shot Y-shaped Learning: State Slate-Ot-hheAArt
Few-Shot Classification with Simple Ingredients." " arXiv:2201 .96699, Feb. 2022, Available: http://anxiv.or/abb/2201..1
442
443
[46] H. 7hu al., "Weight Imprinting Classification Bassdd Force Grasping With Variahle-Stiffness Robotic
444
Gripper," IEEE Transactions on Automation Science and Engineering, vol. 19, no. 2, pp. 969-981, Apr. 2022, doi: 10.1 11T.AASE..02..3556555
446
[47] P.
lakubovskii, EfficientNet https://gttub.co//ubbeeeefffffot
Keras
(and
TensorFlow
Keras),
Available:
448
[48] F. Morales, vit-keras. Available: http://giiialoommm/aattoooaailit
449
[49] M. Andrews, Trained image classification models for Keras. Available: https//githob.ccmmmdddddeepp loadning-models
451
[50] openev-python:
Wrapper package
for
OpenCV
python
bindings.
Available:
https//gilhub.com/kaaakoppppp
Checklist
454
1. For all authors...
455
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
456
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Methodology.
458
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See
459
Methodology.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results...
463
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
464
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you experiments...
486
(a) Did you include the code, data, and instructions needed reproduce the main ex- perimental results (either in the supplemental material URL)? [Yes] in the supplemental material
12

---

489
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]
470
(c) Did you report error bars (e.g.. with respect the random seed after running experi- ments multiple times)? [Yes]
473
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [No] Because the amount of compute small and no special hardware is required.
474
475
476
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
477
(a) your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [No] Because they are all OSS.
479
(c) Did you include any new assets either in the supplemental material or as URL2 [Yes] in the supplemental material
480
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [No] Because they are all OSS.
482
483
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [No] Because they are all OSS.
484
485
5. If you used crowdsourcing or conducted research with human subjects...
486
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
487
488
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
489
480
(c) Did you include the estimated hourly wage paid participants and the total amount spent participant compensation? [N/A]
492
A
Appendix
Code and data are submitted supplemental material (the same as 1st submission). Supplemental figures are added in this revised submission (NeurIPS SouurlPS_FigurS..ppl
13


Reviews:
[{"review_id": "vQdbXCjuDLd", "reviewer": "Reviewer_j4P6", "summary": "The paper proposes a one-shot learning mechanism that adds a new class to the network's output using quantile normalization of the new input (to the last layer) w.r.t. weights in the last layer that correspond to the old classes.", "questions": "## Large concerns\n### Hebbian learning\nThis work does not describe a Hebbian mechanism of learning. Hebbian means $\\Delta W_{ij} = f(x_i, y_j)$ for input neuron $x_i$ and output neuron $y_j$ (in the classical Hebbian sense, $f(x, y)=xy$). Eq. 3 is fundamentally not Hebbian because it uses information about other weights. Lines 154-155 say quantile normalization \"is suitable for implementing Hebbian theory\", but this claim is not backed up. The authors should either thoroughly explain what makes their method Hebbian, or remove mentions of Hebbian learning.\n\n### Comparison with previous methods\nIn lines 104-109 the authors try to explain why they’re not comparing their method to previous work directly:\n> It is meaningless to compare the above three approaches with weight imprinting, because weight imprinting does not contain any optimization algorithm. Therefore, in principle, there is no reason for weight imprinting methods to outperform other methods by themselves in accuracy. The performance of weight imprinting methods is uniquely determined by the backbone DNN without any randomness, hence its performance is suitable as a reference baseline for other methods. Thus, weight imprinting does not aim for the highest accuracy but for practical convenience and reference role as a baseline method.\n\n– I can agree with not chasing the best accuracy, but then there must be a discussion of concrete practical benefits of the method. (No optimization != faster, as quantization takes some time too.)\n\n### Overall reason behind the method\nSection 3.1: I did not understand why quantile normalization helps. Shouldn’t it make the response in each neuron $y$ similar to that of the new class? Either way, the authors should spend more time explaining their method and why it works, as it is the core contribution.\n\n## Unclear parts in the text\n\nLines 135-136: \"the backbone DNN model is a very good model as a heritage of mankind and should not be changed as much as possible (especially for many non-expert users)\"\n– this is not an explanation.\n\n153: what probability distributions? Everything has been deterministic so far, so it's worth explaining what distributions are discussed here. Is it over individual weights? Individual neurons?\n\n166: \"The range of applicable models is yet unclear, but in principle it is wider than Qi’s method.\"\n– what does it mean? The line before said that you need a network with a dense final layer, which clearly defines the range of models.\n\n223-224: what’s good accuracy for practical uses? It’s never explained how the authors came up with that value\n\n## Small corrections\n**Note**: the paper contains a lot of grammatical errors. One simple way to fix it is to copy-paste the source code into a google doc and go through all suggested corrections.\n\nLine 15 (abstract): performs \"at a\", not \"a\", practical-level accuracy?\n\nAlso next sentence: Can write as \"DONE overcomes ...mentioned issues of DNNs...\"; Currently the sentence doesn’t read well.\n\n92: uses \"a\" classification\n\n93: aim\"s\"\n\n145: applies to, not apply into\n\n212-214: black and orange said twice\n\nWhy is the supplementary material just a corrected version of the main paper?\n", "limitations": "The authors can mostly addressed the limitations of their work (although see below), but they did not account for the potential negative impacts. I'm not flagging it for an ethics review as it's just a small correction though.\n\nChecklist 1C: the described model can be easily adapted for face recognition. This does imply potential negative societal impacts and should be discussed by the authors, even though previous work can be used in a similar way.\n\nRelated, the authors refer to practical uses throughout the whole paper, in particular when talking about performance of DONE. But it’s never discussed what those are, and why the achieved performance of the method is good enough for them.", "rating": 5, "confidence": 4, "soundness": 1, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths:\n1. The proposed one-shot classification method is simple and, unlike previous work, doesn't require layer normalization in the second last layer.\n2. The method works reasonably well.\n3. The method is novel, as far as I know.\n\n## Weaknesses:\n1. This work has \"Hebbian\" in the title and some part of the text, but as far as I can tell has nothing to do with Hebbian learning. (Elaborated in the Questions section)\n2. Performance of the method is not adequately compared to previous work. (Again, elaborated below) \n\n## Summary\n\nMy current score is 4 (borderline reject). The paper proposed an interesting method, but it doesn't properly explain why it works, and it doesn't justify the lack of comparison (in terms of performance and computational complexity) with previous methods. I'm willing to increase the rating if my points are addressed, however.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "VVq0XJG-PtS", "reviewer": "Reviewer_8B2d", "summary": "The paper presents a method for the one-shot classification of novel inputs. For the classification of novel inputs, the method uses representations obtained by the final dense layer of a pretrained backbone model to create novel weight vectors of new classes. Importantly, DONE does not optimize or change the backbone model in any way, making it applicable to various backbone models and computationally efficient.", "questions": "My main suggestion is that you would make a further effort to convince this method's benefits over Qi's. In the current paper, the benefits are argued in text. Try to argue using a test where Qi's modification to the backbone model impairs performance, while DONE does not.\n\nAlso, I suggest adding more details about the practical implementation of quantile normalization.\n\nFinally, the paper needs to be thoroughly edited.", "limitations": "The paper adequately discusses the method limitations.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1) Introducing quantile normalization in machine learning, a method that was unknown to me.\n\nWeaknesses:\n1) The link to Hebbian learning is weak and not meaningful for this paper goals.\n2) The idea that the last dense layer of a DNN includes representations that allow novel classifications is not novel, without any model modification, is not novel (e.g., [1] analyzed it theoretically).\n3) There are many writing problems. Too many to mention them all. Examples: a. talking about the heritage of mankind in line 135 (unprofessional); b. figure 2 has an irrelevant screen cursor at the right-most data point. More importantly, essential legend details of this figure appear in the text instead of the caption, which is confusing; c. “the simplest” in line 324 is also not professional (I think that Qi’s method is simpler). \n4) The method does not improve upon previous works Figure 2 shows that Qi’s method works similarly or better. I am not convinced that the backbone modifications it does are detrimental.\n5) Accepting the updated paper that was submitted via the supplementary materials, at a later deadline, is unfair to other authors.\n\n[1] Sorscher, B., Ganguli, S., & Sompolinsky, H. (2021). The geometry of concept learning. bioRxiv.‏", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "8F-C0hW1ue", "reviewer": "Reviewer_RJBn", "summary": "The authors present Direct ONE-shot learning with Hebbian imprinting (DONE) a method for one-shot learning inspired by Hebbian learning in the brain. The method uses neural activations from the final layer of an “encoder” network (such as a vision transformer or EfficientNet) on a single example from an unseen class to create a weight vector for the class of the new class. The presented method is closely related to and improves upon a that of a 2018 paper by Qi et al, which the authors cite. \n\nThe authors present experiments in which DONE is compared to Qi’s method. Specifically, the authors one-shot-learn 1 or 8 classes, evaluate the performance on those classes, and measure the degree to which the new classes interfere with initially trained classes. The authors also present results on k-shot learning. \n", "questions": "1. The authors write on line 31 that “The human brain does not necessarily have more complex processes than DNNs…”. I recommend to remove this statement. Many researchers would disagree with the statement, and the statement is not essential to the paper. \n\n2. The authors write on line 33 that “a series of simple processes such as linear filtering followed by a nonlinearity can describe the function of lower visual cortex”. I recommend to rephrase this slightly, as learning in the lower visual cortex is part of the lower visual cortex’s function, but that aspect is not described by a series of simple processes such as linear filtering followed by a nonlinearity. \n\n3. “modify” on line 48 should be “modifies”\n\n4. On line 61, the authors write “neural activity and synaptic strength are different in dimension.” Do the authors mean  “neural activity and synaptic strength are different in scale.”? Line 157 also mentions “different dimension” when the authors may have meant “different scale”. \n\n5. On line 135, the authors write “the backbone DNN model is a very good model as a heritage of mankind”. I did not understand this sentence. Could the authors rephrase it?\n\n6. Figure 2 is impossible to understand without referring to explanations of data point markers in section 4.1, which means that a reader needs to jump back and forth between the Figure 2 and the text on the page after. I recommend explaining the figure markers in Figure 2’s caption instead of in section 4.1. \n\n7. On line 185, the authors write “(not 1008 classes here)”. At this point, the reader has not yet read about the 1008-class experiments. How about removing the comment in parentheses?\n\n8. The EfficientNet architecture is misspelled in a number of places in the paper, e.g. as “EfficinetNet” and “EfficientNnet”\n\n9. Koray Kavukcuoglu’s name should be upper-cased on line 377. ", "limitations": "The authors have adequately addressed the limitations and potential negative societal impact of their work.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The paper presents a novel and simple method for few-shot learning. \n2. While the paper does not claim to model the brain, it is exciting to see that one-shot learning with brain-line Hebbian imprinting can work so well. \n3. This paper is a pleasure to read. It is well-structured, and the writing is mostly clear. \n\nWeaknesses\n1. The authors compare their method with only one method, when it would have been helpful for readers to compare to a broader range of existing one-shot learning approaches for image classification. The authors state “It is meaningless to compare the above three approaches with weight imprinting, because weight imprinting does not contain any optimization algorithm. “ . I disagree with this, for two reasons. First, papers such as the one about one-shot learning with siamese networks (https://www.cs.cmu.edu/~rsalakhu/papers/oneshot1.pdf), which the authors cited) are quite similar to DONE in that a big model gets trained on large amounts of data in a time-consuming process, and later one-shot classification becomes cheap. In other words, both were optimized at some point. Second, even if two models fall into different categories (e.g. because one is much more computationally expensive than the other), it’s useful for readers to know how much accuracy they lose (if any) by using a more flexible methodology. \n2. While the algorithm from the paper is novel and undoubtedly very elegant, it is very similar to Qi’s method and other papers that the authors cite. In this sense, the paper is not a must-read for most researchers in the community. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Reject