A Closer Look at Prototype Classifier for Few-shot Image Classification
Authors: Mingcheng Hou
Issei Sato
Venue: NeurIPS 2022 Accept (2022)
Keywords: few-shot
meta-learning
prototypical network
fine-tuning
prototypical classifier

Abstract: The prototypical network is a prototype classifier based on meta-learning and is widely used for few-shot learning because it classifies unseen examples by constructing class-specific prototypes without adjusting hyper-parameters during meta-testing.
Interestingly, recent research has attracted a lot of attention, showing that training a new linear classifier, which does not use a meta-learning algorithm, performs comparably with the prototypical network.
However, the training of a new linear classifier requires the retraining of the classifier every time a new class appears.
In this paper, we analyze how a prototype classifier works equally well without training a new linear classifier or meta-learning.
We experimentally find that directly using the feature vectors, which is extracted by using standard pre-trained models to construct a prototype classifier in meta-testing, does not perform as well as the prototypical network and training new linear classifiers on the feature vectors of pre-trained models.
Thus, we derive a novel generalization bound for a prototypical classifier and show that the transformation of a feature vector can improve the performance of prototype classifiers.
We experimentally investigate several normalization methods for minimizing the derived bound and find that the same performance can be obtained by using the L2 normalization and minimizing the ratio of the within-class variance to the between-class variance without training a new classifier or meta-learning.

A Closer Look at Prototype Classifier for Few-shot Image Classification
Mingcheng Hou The University of Tokyo wee.:e4teechooo@@aaaae com
Issei Sato The University of Tokyo satoOg ecc. u-tokyo... ac jp
Abstract
The prototypical network is prototype classifier based meta-learning and is widely used for few-shot learning because it classifies unseen examples by constructing class-specific prototypes without adjusting hyper-parameters during meta-testing, Interestingly, recent research has attracted a lot of attention, showing that training a new linear classifier, which does not use meta-learning algorithm, performs comparably with the prototypical network. However, the training of new linear classifier requires the retraining of the classifier every time a new class appears. In this paper, we analyze how prototype classifier works equally well without training new linear classifier or meta-learning. We experimentally find that directly using the feature vectors, which is extracted by using standard pre-trained models to construct prototype classifier in meta-testing, does not perform well as the prototypical network and training new linear classifiers on the feature vectors of pre-trained models. Thus, we derive a novel generalization bound for prototypical classifier and show that the transformation of a feature vector can improve the performance of prototype classifiers. We experimentally investigate several normalization methods for minimizing the derived bound and find that the same performance can be obtained by using the L2 normalization and minimizing the ratio of the within-class variance to the between-class variance without training a new classifier or meta-learning.
Introduction
Few-shot learning is used to adapt quickly to new classes with low anmotation cost. Meta-learning is standard training procedure tackle the few-shot learning problem and the prototypical network [. a.k.a ProtoNet is a widely used meta-learning algorithm for few-shot learning. In ProtoNet, we prototype classifier based meta-learning predict the classes of unobserved objects by constructing class-specific prototypes without adjusting the hyper-parameters during meta-testing.
ProtoNet has two advantages. (1) Since the nearest neighbor method is applied query data and class prototypes during the meta-test phase, no hyper-parameters are required the meta-test phase. (2) The classifiers for the support set in the meta-test phase when new classes appear. Moreover, the generalization bound of ProtoNet in relation to the number of shots in support set has been studied [2]. The bound suggests that the performance of ProtoNet depends on the ratio the within-clas variance to the between-class variance of the features extracted using the meta-trained model.
quickly adapt to environments because they do not have to be re-trained
There have been studies on training new linear classifier on the features extracted using a pre-trained model without meta-learning, which can perform comparably with the meta-learned models 3 We call this approach the linear-evalluaionn-en based approach. these studies, the models are trained with the standard classification problem, i.e., models are trained with cross-entropy loss after linear projection from the embedding space to the class-probability space. The linear-evaluation-
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

approach has three advantages over meta-learning (1) Training converges faster than meta-learnin (2) Implementation simpler. (3) Meta-learning decreases in performance if the number of shots does not match between meta-training and meta-testing 22: however, the tinear-evaluation-bee basee approach does not need to take this into account. However, requires retraining a linear classifier every time new class appears.
To avoid meta-learning during the training phase and the linear evaluation during the testing phase, we focus on using a prototype classifier in the lesting phase and training models in a standard classification manner. As we discuss in section 4 we found that when we directly constructed prototypes from the feature vectors extracted using pre-trained models and applied the nearest neighbor method as in the testing phase in ProtoNet, this does not perform as well as the innear-ealluation-eee -bsse approach. We hypothesize that the reason is the difference between how the loss is calculated in ProtoNet and pre-trained models. As described in section 3 if we consider a prototype as a pseudo sample average of the features in each class, the loss function of ProtoNet can be considered as having regularizing effect that makes it closer to the sample average of a class. Since standard classification training computes cross-entropy loss with dot-products to make the features linearly separable, the loss function does not have such an effect and can cause large within-class variance. Figure sshows scatter plot of the features extracted using a neural network with two dimension output trained on CIFAR-10 with ProtoNet [[a) and cross-entropy loss with a linear projection layer [(b). This figure indicates that the features extracted using a model trained in a standard classification manner get distributed away from the origin and cause large within class variance along the direction of the class mean vectors, while those of ProtoNet more centered to its class means. This phenomenon is also observed in face recognition literature 518)
We now focus on a theoretical analysis for prototype classifier. A recent study 2 analyzed an upper bound of risk by using a prototype classifier. The bound depends on the number of shots of support set, between-class variance, and within-class variance. However, it has two drawbacks. The analysis requires the class-conditioned distribution of features to be Gaussian and to have the same covariance matrix among classes. Moreover, since the bound does not depend on the norm of the feature vectors, it is not clear from the bound what tranuurrrtannnoo method can lead to performance improvement. Thus, need to derive novel bound for a prototype classifier.
Our contributions are threefold.
1. We relax the existing assumption; specifically, the bound does not require that the features be distributed in Gaussian distribution, and each covariance matrix does not have to be the same among classes.
2. We show that our generalization bound consists of three terms: (I) the variance of the norm of feature vectors, (2) the difference in the distribution shape constructed from each class embedding, and (3) the ratio of the within-class variance to the between-class variance.
3. From our theoretical analysis and empirical investigation, we found that reducing two terms is a critical factor to improve the performance of prototype classifiers: the variance of the norm of feature vectors and the ratio of the within-class variance to the between-class variance.
While it is empirically known that the 1.2normalization of feature vector and minimize the ratio of the within-class variance the between-cland variance can improve the performance of prototype classifier, our work is the first to theoretically show that these transformation can improve the performance. The bound of Cao et al. 2 cannot explain the performance improvement of L2- normalization because 72-normalization does not reduce the ratio of the within-class variance to the between-class variance as shown in Section !
A prototype classifier can be applied to any trained feature extractor. It is simpler than linear- evaluation methods such as logistic regression and support vector machine (SVM) for three reasons. (1) A prototype classifier does not require training in the test phase. The training requires time and memory consumption. (2) Since the nearest neighbor method is applied on query data and class prototypes, the computation in the test phase does not depend on the number of classes while linear-evaluation methods do. (3) A prototype classifier requires fewer hyperparameters than linear- evaluation methods. Note that our goal is not to replace linear-evaluation methods with prototype classifier, but to show that a prototype classifier can be used as a counterpart of the linear-evaluation methods and can be practically useful first step in tackling few-shot learning problems.

---

Activation of 2nd neuron
Activation
-20 -35 63 50 200
Activation of 1st neuron
Activation of 1st neuron
(a) ProtoNet
(b) dot-product with cross-entropy loss
Figure I: Distribution of features extracted using a neural network with dimensional final layer trained on CIFAR-10 with (a): ProtoNet loss and (b): dot-product with cross-entropy loss. The ProtoNet features get distributed closer its class center than the features extracted using the model trained on cross-entropy loss with linear projection layer.
2 Related Work
We summarize related work by describing a prototype classifier with meta-learning, finear-vvaluation based approach without meta-learning, prototype classifier without meta-learning and theoretical analysis related to the few-shot learning problem.
Prototype classifier with meta-learning On the basis of the hypothesis that features well distin- guished in the training phase are also useful for classifying new classes, constructing one multiple prototypes for classifying unseen examples is widely used approach [1 1-51. Certain algorithms compute similarities between multiple prototypes and unseen examples by using their own modules, such as an attention mechanism 9 [4, a relation network [12. reweighting mechanisms by taking into account between-class and within-class interaction [Л and latent clusters [13]. Another line of research is transforming the space of extracted features to a better distinguishable space [16 or taking the variance of features into account 20
Update-basse based meta-learning In contrast to the prototype classifier with meta-learning, update- based meta-learming adjusts the model parameters to adapt to new classes. Model-agnose meta- learning (MAML) and its variants [211223 search for good initialization parameters that adapt to new classes with few labeled data and few update steps. These approaches require additional learning of hyper-parameters and training time; thus, they prevent quick adaptation to new classes.
linear-evaluation--asss approach without meta-learning Interestingly, recent studies have shown that training a new linear classifier with features extracted using a model trained with cross- entropy loss on base-dataset performs comparably with meta-learning based methods [3] 24 A more
effective method for training a new classifier in few-shot settings has been proposed 25 26 such as calibrating distribution generated by support knowledge to obtain better embeddings [4]. However, similar to update-bass studies require additional ceper-parameters and training for application to new classes; thus if tackle these points, we would have an alternative method that is easier and more convenient to use.
self-supervised learning 26 and distilling
meta-learning these
prototype pe classifier without meta-learing Recent studies have shown that using prototype classifier without meta- meta-learning or a linear classifier can lead to comparable performance 22] 27 Wang [4 has empirically shown that Centering and normalizing the feature vectors can improve the performance of a prototype classifier. Another study empirically shows that applying NCA loss in the training phase can improve the performance requires model to be trained with NCA loss while our studies do not require a model to be trained with any specific loss. These studies do not show any theoretical insights on the performance of prototype classifier and they do not compare with linear-evaluation-basse methods.
prototype classifier (27). However, their study
Theoretical analysis of few-shot learning Even though much improvement has been empirically made on few-shot learning, theoretical analysis is scarce. In the context of meta- meta-learning, Du et al.
3

---

provided risk bound on the meta-testing phase that is related to the number of meta- veta--raining data and meta-testing data. Lucas et al. [29] derived information-Hie -heeortttt lower-bounds on min- max rates of convergence for algorithms that are trained on data from multiple sources and tested on novel data. Cao al. [2] derived a new bound on a prototype classifier and theoretically demonstrated that the mismatch regarding the number of shots in support set between meta-training and meta-testing degrades the performance of prototypical networks, which has been only experimentally observed. However, their bound depends two assumptions: the class-conditional distributions of features are Gaussian and have the same covariance matrix among classes. In contrast, we derive a novel bound that does not depend any specific distribution.
28
3 Theoretical Analysis of Prototype Classifier in Terms of Variance of Norm of Feature Vectors
In this section, we first formulate our problem settings and explain the recent theoretical analysis on a prototype classifier. Next we provide our novel bound for a prototype classifier with the bound related to the variance of the norm of the feature vectors. Finally, we list several methods that can improve the performance of a prototype classifier based on our bound.
3.1 Problem Setting
Let y be a space of a class, T a probability distribution over V, X' a space of input data, a probability distribution over X' D, probability distribution over X' given class y. We define Doan and D by Din II Dy and D = II D respectively. We sample N classes from T to form the N-way classification problem. Denote by K the number of annotated data in each class and E X. E y as imput data and its class respectively. We define set of support data of class sampled from as S. {æz (æs, vi) X' x ) n data in the N-way K-shot classification problem as S U Se. Suppose a feature extractor computes a function Φ X + RD, where D the number of the embedding dimensions. ((S.) is defined by d(S.) K Emes. c(x)). Let & be a space of the extractor function d. Denote by M: & x X x (X x y) NK + RR a prototype classifier function that computes the probability of input x belonging class as follows.
c) and a set of support
exp (-1((())-
M(0 n. S)c = PM S,
(1)
Σ exp (-llo(x) -
where ||0||2= = (ul4)), and y(d is the d-th dimension of vector The prediction of input x, denoted by y y. is computed by taking argmax for M(ó, S), i.e., ÿ = argmaxM x, S).
We denote by E [g==(())g(())) an operation to take the expectation of g(z) over distributed as q(2).
and we simply denote Ez-g(=) (2)) as Ex [=)]] when obviously distributed on g(2). We define Var Varz~at=)le [9 [=)]
operation to take the variance of g(2) over z distributed g(2). With I
denoting the indicator function, we define the expected risk RMM of a prototype classifier as
RM(Ø)= =
[[laagmaxMMM x, S) + c].
(2)
We show case of multi-class classification in Appendix A.5 due to lack of space. We obtain the bound on multi-class case by Frechet's 's inequality.
Let Cj and C2 denote any pair of classes sampled from т. We consider that a query data point belongs to class ej and support sets consist of class ei 's support set and 02's support set. Then, equation 2 is written as follows.
(x(()) = Es.c, M(rgmax x. Sc, U US,.) # cill.
(3)
where ES Es- Ee
3.2
What Feature- Method Is Expected to Be Effective?
The current theoretical analysis for prototype classifier 2 has the following two drawbacks (see Appendix A.2 for the details). The first is that the modeling assumption requires class-conditioned

---

distribution of the features to follow a Gaussian distribution with the same covariance matrix among classes. For example, when we use the ReLU activation function in the last layer, it is not normally distributed and the coadd-cooddiioned distribution does not have the same covariance matrix as shown in Figure The second drawback is that it not clear what catuuretrranformmatio the upper bound. A trauurretransoomaiioo method that maximizes the between-class variance and minimizes the within-class variance such as linear discriminant amalysis (LDA) t0 and Embedding Space Transformation (EST) ? can be expected to improve performance; however, it is not clear how the second term of the denominator changes.
method can reduce
From Figure I the distribution of each class feature stretches in the direction of its class class-mean mean feature vector. This property is also observed in metric learning literatures [5] 6 A model trained in the cross-entropy loss after linear projection from the embedding space to the class-pre daass-probbbilityy space computes a probability of input belonging to class c with a weight matrix W Dx given by
exp (Ø(x) W.)
ply = c/ W. σ) Σラ exp
Comparing equation | with equation T we found that equation D does not have regularization term similar to the one appearing in equation T that forces the features to be close to its class-mean feature vector. This implies that the features extracted using a model trained with the cross-entropy loss using equation 4 are less close to its class-mean feature vector than the ProtoNet loss on equation [ Through this observation and the property mentioned above, we hypothesize that normalizing the norm of the feature vectors can push the features each class-mean feature vector and can boost the performance of a prototype classifier trained with cross- entropy loss on equation
3.3 Relating Variance of Norm to Upper Bound of Expected Risk
To understand what effect the variance of the norm of the feature vectors has on the performance of prototype classifier, we analyze how variance contributes to the expected risk when an embedding function á is fixed. The following theorem provides a generalization bound for the expected risk of a prototype classifier in terms of the variance of the norm of the feature vectors computed using a feature extractor.
Let M be an operation of a prototype classifier for binary classification defined by equation For pe ((e Σg L. (e)(<(()) pe) ]. μ Ee-r(pe)
Theorem 1.
[ee]. Ee- (ike (1 - [1) and E,, = [ if <(x) has the variance of its norm, then the misclassification risk of the prototype classifier for binary classification R. satisfies
((T(E))
RM(d) $1 EV[hra(((x)) + (I + Vait (E, E, (1) + dist (Hor Moz)
(5)
where EV[h12(0(2))]
K Eowr VarzaD [|16(2)11]]
(6)
2
Vir(Eee)) =
+
Varewr [Tr (T(E.)))
K
k2
8
Vwit (2i, 25,,2,,4 11)
(Ir(2ir) (IT(2) ATTH 14) + 4 (Tr(2) + n 1))
K
+ 4lE
Tr (Eer) (Hez - Ho.)) (Hes - Mo.)
(8)
E d Moo) =Eoooo (wo Moz)" (Mor - -moz)) 27
Remark. The term EV(haa(((x)) is the variance of the norm of the feature vectors. The term (I,(E,,) is the variance of the summation with the diagonal element of the covariance matrix from each class embedding; it can be interpreted as the difference in the distribution shape constructed from each class embedding. The term Vwit (E,, E, 1)) is related to the within-class variance. The last
term of equation is approximately linear with Tr Ee,
because (Mo, -H,,) ((M,,, -Hop) is supposed
TT(E)
Tr())
to be constant. Thus (Se, is secondary expression for the ratio of the within-class variance TY(E)
5

---

to the between-class variance. The term E dist Mo,) is the expectation of the Euclidean distance between the class-mean vectors.
This bound has the following properties.
1. Its derivation does not require the features to be distributed in Gaussian distribution and the class-conditioned covariance matrix does not have to be the same among classes
2. The bound can decrease when any of four statistics decreases with fixed between-class variance (2): (i) the variance of the norm of the feature vectors, as discussed in Section
(ii) the difference in the distribution shape constructed from each class embedding, (iii) the ratio of the within-class variance (2,) to the between-class variance (2), and (iv) the Euclidean distance between the class-mean vectors.
3.2
As result, our bound loosens the modeling assumption of Theorem 2 in Appendix A.2 and has its property. We show the proof in Appendix A.4
3.4 Feature-Transformatio Methods
We hypothesize from Theorem T that in addition to a Cature-transformattoo method related to equation lowering the ratio of between-class variance within-class variance can improve the performance of a prototype classifier. As shown in Figure left in experimantal analysis, the value
2
of equation is relatively small compared equation 6 and equation g] Regarding equation g
the ratio of the Euclidean distance between the class mean vectors and the between-class variance supposed to be constant. Thus, we focus on transformation methods related equation 0 and equation We analyze four traturrtransforrmatoonn methods: Eg-normalization (Ly-norm). EST,LDA, EST+L2-norn -norm and LDA+Lg norm. We give the details on each method in Section A.I due lack of space.
4 Experimental Evaluation
In this section, we experimentally analyzed the effectiveness of the trature-trassoomaation methods mentioned in adaptation. The center loss [ and affinity loss [3 have been proposed to efficiently pull the features of the same class to their centers in the training phase; however.our goal is not achieve SOTA but, through comparison with existing studies, show that we can achieve the same level of performance without fine-tuning and thus we focus on a widely used pre-trained model in the experiments following the line of studies Chen et al. B and Tian et al.
under scenarios: (1) standard object recognition and (2) cross-domain
4.1 Datasets and Evaluation Protocol
For standard object recognition, we use the minilmageNet dataset, tieredImageNet dataset, the CIFARFS dataset, and the FC100 dataset. minilmageNet. The minilmageNet dataset + is standard bench-mark for few-shot learning algorithms in recent studies. It contains 100 classes randomly sampled from ImageNet. Each class contains 600 images. We follow widely used splitting protocol
22
to split the dataset into 64/16/ for training/validattonnnessnnngg respectively.
mage The tieredImageNer dataset 3 is another subset of ImageNet but has 608 classes. These classes grouped into 34 higher level categories in accordance with the ImageNet hierarchy and these categories are split into 20 training (351 classes), 6 validation (97 classes), 8 testing categories (160 classes). This splitting protocol ensures that the training set is distinctive enough from the testing set and makes the problem more realistic than minilmageNet since we generally cannot assume that test classes will be similar those seen in training.
The CIFAR-FS dataset 34] is a recently proposed fewshot image classification bench- mark, consisting of all 100 classes from CIFAR-100 [5 The classes are randomly split into 64, 16, CIFAR-FS
and 20 for training meta- vetaavvlid,tion, and meta-testing respectively.
containing 100 classes which are grouped into categories. These categories are split into categories for training (60 classes), from 4 categories for validation (20 classes), 4 categories for testing (20 classes). This
FC100 The FC100 dataset [36] is another dataset derived from CIFAR-100 35

---

Table 1: Classification accuracies with ResNet12 minilmageNet, fieredlmageNet. CIFARES, and
FC100 of ProtoNet, linear-evaluation-ben based methods [3, centering with L--norm [24] and ours. The Baseline without finear-vvaluation methods with accuracy greater than the lower 95% confidence margin of the accuracy of ProtoNet and Baseline are in bold. Regarding to Baseline++, Baseline++ without linear-evaluation methods with accuracy greater than the lower 95% confidence margin of the accuracy of ProtoNet and Baseline++ are in bold. All the methods are our reimplementation.
Nis Set
Nerdiinggeee
CIFAR-FS
EC100
S-hhot
I-but
lisear enailaation
FroteNed
09:45 & 0.02
13.06 a 0.65
56,40 a 0.98
17.07 & 0.70
68.26 0.99
70.91 a: 0.72
1 0.77
Baseline
G4.44 di 0.80
76,00
0.62
81.67 a 0.92
81,02 & 0.64
60.62 d: 0.80
79.79 a: 0,70
39.72 à DUS
16,04 0,76
A pooorge clncgjer with pontre--onggooaaaeii suerisod
Baseline wie linear conteringt norn []]
55.96
75,98
90.42
0.66
79.35
41.58 0.74
Baselis linea
46.36
73.97 0.60
50,00
75.10
46.91 00.80
75.81
31:00
52.00
0.77
Reellss-ebb-iieee Lymeen
59,60 + 1.42
17.12 1:12:::00 0.44
64.34 +: 0,39
K1.98 0.66
63.78
0,87
S0.14 =: 0.75
40.34
0.71
56,61
0.71
Reclaa--oh--isee EST
51 0-0.3
73.47
0.60
50.94
8 0.95
78.47 0.06
56.08
0.94
76.01
45.90
0:30--20.85 0.85
64.32
0.78
Baselie wio linerr LST-L docm
$7.34 0.84
76,90
0.62
64.14 0:91
X1.39 2: 9.64
65.19
0.93
$1.13 2: 0.09
49.54 1: 0.91
64.12
1: 0.09
Baseline win liner LIA
51 00.00-2 2.103
T1.47
2: 0.00
30.96
8 01533
T% 07 0.000
26.0%
12 01:1
70.
0-000 70
45.99
1: 0.55
64:12
1: 0.75
Basclizz-ile liner L.
60.29 = 0.50
75.99
2: 0.66
64.16
+: 0.90
81.96 + 0.65
65.57
:: 0.59
90.50
+: 0.68
30.97
2: 0.51
64.91
0.76
mvtllde with austo-boorinng or Inveremmisaionn
Resclinze+
56.33
0.60
63.02
90:8-+0075
0.72
postoge elargfor with Semtre-/tongiorarrinn sertiody
swellass+--wh--iisee comcringseeeeemm [3]
57.50 =: 1.81
70:00 0.00
63.31 0.991
79:19
0.06
65.76 St 0.90
79.35 5: 0.73
37.51 0: 0.71
50.55 :: 0.72
0.40
10.59
73.85
18:08 0.80
12.79
0.57
06.83 0.67
57.96 =
8.61
65.36
81,88
0.09
+:
90.23 +
0.67
51.10 0.71
Baselierr linesr
47:11
0.63
32.01
59.72 0.05
72.10 0:74
37.06
0.70
Buellazss-shh-iccs LDA
08.43::0.44
47.26
74.41
70.01
0.75
Baselinze L--mm
58.36 1: 0,87
78.33 0.66
65.10 0.99
80.19
0.09
0.96
79,09 1: 0.72
44.8 0.83
58.7 0.74
Table 2: Classification accuracies with ResNet12 on minilmageNet, tiered CIFARFS, and FC100 of methods in current studies and ours. The best performing methods and any other runs within 95% confidence margin are in bold. Methods with & show the results of our reimplementation following Section 4.2
Nereollmape
CHARLE
EC100
1 Shot
5-bht
1 1.stor
S-bbot
1-shon
5-sbo
Lishot
5-sbot
mviildee aurtrslorning or finaorernboniion
ManoOgiNar
0600
Taphin 17
pestinge clasnisrr with festtre-ttunnommmiinn murffaly
Raelias-wh-lince Lyytten
50.60 + 1.12
77.12 0.44
66.21 0.8.0
$1.98 +: 0.66
65.78 01-78-t-030 0.87
00.14
10:31 0.7.1
56.61 6.7.1
Breliaz-n--lice, ES7-1l--uum
57.81 + 0.84
76.90 2: 8.62
64.14
=
$1.30
2: 0.64
65.19 0.05
$1.13 = 0.69
10.51 0.91
64.12 :: 0.69
Baselme-wie wio linear L.De-ly-oom
60.29 :: 1180
75.14 0.06
04.16 0.10
$1.96 1: 0.68
65.37 0.89
50,00 11 0.08
50.97
0.51
64,91 0.76
Baselmer lites. nopm
90.23
37.16
Baellane++-sh--liee FST-f.-ttem
58.22
0.79
Baellaze--shh-iice
68.26
65.10
80.50
0.96
70.00
0.74
splitting protocol is similar to the protocol used in tieredImageNet so that the training set is distinctive enough from the testing set and makes the problem more realistic than CIFAR-FS.
For cross-domain scenario, we use the minilmageNet dataset during pre-training stage and we use the CUB-200-201 dataset [Р a.k.a CUB during testing stage.
CUB The CUB dataset contains 200 classes and 1 11,788 images in total. Following the evalua- tion protocol of Chen et al. respectively.
3 we split the dataset into 64/16/2 /2 for training/validttionn/essi
4.2 Implementation Details
We compared the feature-transformation methods against ProtoNet []] Baseline, and Baseline++ G For Baseline and Baselinc++, we trained the linear projection layer on support set. The difference between Baseline and Baseline++ is that the norm of the linear projection layer and features in Baseline++ are normalized to be constant. We call Baseline and Baseline++ as "linear evaluation" methods. We also compared with the
method proposed in a previous study 24 In that study they transformed the features so that the mean of all features was the origin before L2 Ig-normalizatio and then used a prototype classifier without training new linear classifier. We call this operation centering+ Lg-norm. We re-implemented these methods following the training procedure in previous study 3 In the pre-training stage, where the cross-entropy loss was used and meta-learning was not used, we trained 400 epochs with batch size of 16. In the meta-training stage for ProtoNet, we trained 60, 000 episodes for 1-shot and 40, 000 episodes for 5-shot tasks. We
7

---

used the validation set to select the training episodes with the best accuracy. We evaluated the model every 5 epochs, not every epoch as experimented in Chen et al. G and we performed early stopping when the validation score does not improve for 50 epochs. In each episode, we sampled N classes to form N- W-way classification (in meta-training N=20 and meta-testing N=5 following the original study for ProtoNet [ For each class, we selected K' labeled instances as our support set and 16 instances for the query set for a K K-shot task.
In the linear evaluation or meta-testing stage for all methods, we averaged the results over 600 trials. In each trial, we randomly sampled classes from novel classes, and in each class, we also selected K' instances for the support set and 16 for the query set. For Baseline and Baseline++, we used the entire support set to train a new classifier for 100 iterations with a batch size of With ProtoNet, we used the models trained in the same shots as meta-testing stage since a mismatch in the number of shots between veta-nraining and meta-testing degrades performance 22. All methods were trained from scratch, and the Adam optimizer with an initial learning rate of 10-3 was used. We applied standard data augmentation including random crop, horizontal- and color jitter in both the training stages. We used ResNet12 network and ResNet18 network following the previous study 3
For LDA we set À 0.0001 for equation I1 Appendix A.I and for EST we set the dimensions of the projected space to 60 following the settings of the original study 2 Following the procedure of EST, we calculate the transformation matrix of LDA based on the features of the training dataset.
4.3 Results
We present the experimental results of standard object recognition in Table on the basis of backbones with ResNet12 for comprehensive comparison. We show the result of backbones with 8 in Table
We present the discussion on cross-dataset scenario and the result of the scenario in 4.5
Comparison of feature-transformation methods with ProtoNet, linear evaluation methods, and centering+T2-nnrmm From Table [ we can observe that the prototype classifier with Iz-norm, EST+Ig-norm, I22 performs comparably with ProtoNet and the linear-eaaluation-bar approach in all settings. Comparing the centering+L2--now can slightly improve the performance of the prototype classifier in several 1-shot settings However, in 5-shot settings, the boost decreases and even performs worse than linear-evaluation methods, e.g. minilmageNet and rieredImageNet.
methods described in Section 3.4
with centering+L2-nome
Comparison among feature-transformation methods In the 1-shot setting, although EST falls short of ProtoNet and the enear-evaluation based approach, also improves the performance of a prototype classifier. The performance gain of both L2-norm, EST, LDA, EST+L2-norm and LDA + L2-norm decrease when the number of shots increases. This phenomenon can be explained through Theorem The term relating to the variance of the norm and the ratio of the within-class variance to the between-class variance depends on K. Since the term diminishes as K increases, the performance gain of the
methods decreases.
Comparison of feature-transformatie methods with current studies Table 2 shows the perfor- mance of MetaOptNet [36]. TapNet [17 and cettur-transfoommtton methods that performs compara- methods in Table [ We can observe that prototype classifier can achieve comparable performance with current studies in most of the settings with the feature transformations. Moreover, can further boost the performance of prototype classifier by combining L2-norm and the methods to reduce the ratio of the within-class variance to the between-class variance, e.g. CIFAR-FS and FC100.
ble with ProtoNet or Imear-evaluattionbaae
Discussion on Ceature-transformattie methods LDA and EST in the 5-shot setting do not improve the performance so much compared with Lg-norm variants while in the 1-shot settings, LDA and EST improve the performance. This is because, the 5-shot settings, the values computed from equation & get smaller with larger K, and the effect of equation S on the risk of a prototype classifier in the 1-shot settings is larger. Especially, EST and LDA outperform L2-norm in FC100. We found from Figure that the features of FC100 show the largest ratio of the within-class to between-class variance among all datasets. Thus the method of reducing the ratio works better with FC100 features than with any other dataset's features. As discussed in Section the combination of fcature-transfomee transformation
8

---

L2nomalization
EST L2-nommllzzaion
LDA
E--ommaltzaion
02
08
Ee Te(I
Reshlet12 misilnagenet
ReeHett2 Ueredimagenet
ResNett2
TriE
Trc
CIFARPS
FC100
67 divided by Tr(E)?. The values are computed from test-split of each dataset with ResNet12 and ResNet18. We scaled the values so that each dataset's Ywa (XE, 22,44
Figure 2: Left: We plotted the values shown in equations
Tv(2))
be for simplicity. Right: We plotted the ratio the within-class variance to between-class variance Tr(--) before and after applying Lg-norm, EST, LDA, EST L2-norm and L2A+-T2-nomm of each dataset. TT(3)
methods that reduce the variance of the norm and the ratio of the within-class variance to the between- class variance can further boost the performance prototype classifier. This indicates that reducing the term equation 6 and equation is important factor for the performance of prototype classifier.
4.4 The comparison of the ratio of the within-class variance to the between-class variance before and after applying L2rrm, EST and LDA
We show the ratio of the within-class variance to the between-class variance before and after applying L2- L2-norm, EST, LDA, EST + Lnrmm, LDA L2-norm in Figure right. We calculated the ratio of the between-class variance each class's variance of each dataset. This figure shows that L2- normalization slightly changes the ratio while EST and LDA can reduce the ratio. The combination of EST and L2-norm LDA and L2-norm the norm feature vectors to O. Therefore, EST + L2-norm and LDA + Lz-norm performs better than L2-norm. Moreover, L2-norm does not affect the bound of Cao et al. [2] because their bound depends on the ratio of the between-clate variance to each class's 's variance as shown in 2 Thus the bound of Cao al. 2 cannot explain the performance improvement of prototype classifier with L2-norm while our bound can.
reduce the ratio while decreasing the variance of
4.5 Performance results cross dataset
We show in Table the performance results of cross-dataset scenario in this section with 95% confidence margin. From the table, we can observe that the prototype classifier with Iz-norm, EST+Lg-norm and LDA+Igg performs comparably with linear-valluation-aasee methods. Therefore, our analysis results still hold on the cross-domain scenario.
We also found that ProtoNet performs worse than linear-evaluation-aaees methods and prototype classifier with feature- trauurrrtansooo methods. In this scenario, we should focus on improving the performance of prototype classifier with features extracted from model trained without meta- learning, c.g, Baseline and Baselinet+. Since there is no guarantee that the features of model trained with cross-entropy loss will be distributed in Gaussian distribution, Cao et al. [2's bound cannot explain the performance improvement of the prototype classifier while our analysis can.
4.6
The gap between the performance of linear classifier and prototype classifier
Since we hypothesize that the performance gap between protototype classifiers and ProtoNet is related to the difference of how the loss function calculated, we experimentally analyzed that how the gap changes when cosince distance innerproduct is used in ProtoNet instead of Euclidean distance. We show in Table the performance of ProtoNet with cosine similarities and innerproduct.
The performance gap between a prototype classifier based on the features of Baseline and ProtoNet with cosine distance or inner product is decreased compared to the gap between prototype classifier based on the features of Baseline and a ProtoNet with Euclidean distance.

---

Table Classification accuracies with ResNet12 and ResNet18 on cross-domain scenario (minilmageNet - CUB) of ProtoNet, eneearevaluatton-bee based methods 3 centering with L2 norm
24 and ours. The notation same as Table
münlIaageeet CUB ResNer12 minllaggeeet
CUB ResNer18
S.hhot
Sahoot
methods with metar-larning or lineor-evalaution
ProtoNett
38.47 1: 0.69
61.47 0.68
44.01 1 0.79
61.79 4 0.75
Baseline
46.91 0.81
68.55
46.36
0.78
66.97 0.73
A prototype classifier with tolfure-ranaformmziinn merheuss
Baseline seelineewwo-llnnen centering n.rm 24
46.70 d: 0.79
65.36 0.70
46.56 1 0.77
65.96 0.72
Bascline-w/o waooiiiewwooiiaarr
43.05 1: 0.72
63.17 1: 0.67
12.11
0.71
61.31 2:51::00 0.73
Baseline w/o lineur, Lg -tom
47.64 d: 0.79
66.31 1: 0.73
46.55 J: 0.83
67.21
1: 0.73
Baseline-wii weaiineewwo-lineerr FST
43 0.77
0.77
Bascline-woo-linaar EST-L2-uorm
46.33 + 0.78
66.28 + 0.71
47.10
0.83
66.33
0.74
Baseling w/o lineur LDA
43.22 1: 0.74
62.47
1: 0.71
44.02
1: 0.74
65.34
0.75
Baseline-wlo- www.-inooor.......... 1.D Ty-nomm
47.65 0.80
66.94 0.72
48,47 1 0.82
67.21
0.69
methodl with meta-leaming or fineur-vviaatiio
Baseline ++
1
45,56 + 0.82
66.00 0.74
46.26 0.87
63.53 0.71
prototype classifier with frnture-rumnformmttion methous
Baseline --woo-linear- centering Ia-norm [E]
47.36 d: 0.82
64.44 0.75
46.55 1: 0.79
64.52 +: 0.71
Bascling+--wir woo-lineer
40.68 +: 0.71
61.02
0.71
39.72 0.70
60.00 0.71
Baseline wiw---iiear......... Ly-norm
0.82
Baselinc+-wwo-iineg EST
40.28 1: 0.73
56.79
0.72
40.11 0.74
31.95 0.73
Baseline en-wwoolinerrr ng-norm
46.64 J: 0.84
65.39
1: 0.74
45.96 J: 0.78
64.23 :: 0.77
Bsclince+-woo-lince
0:75
Bssclinc++ LDAm
46.32 +: 0.81
65.35 0.76
45.60 +: 0.81
64.17 + 0.73
Table 4: Classification accuracies with ResNet12 on miniImageNet and tirredImageNet of ProtoNet with feature- transformation methods.
muieclmageNee
NiewallmageNet
I-shot
S-hbot
I-shot
5-shol
ProtoNes
54.42 0.86
73.56
56.56
0.58
78.39 0.71
ProtoNet- cosine similarity
0.83
50.17
ProeoNet-w-innee Se-eeNeee---mmerroduduu
63.87 0.83
0.69
48.10
70.11 0.84
Brseline wio lincur
46.36 11 0.58
73.97 1 0.02
50.00 a 0.87
78.10 11 0.67
Conclusion
We theoretically and experimentally analyzed how the variance of the norm of feature vectors affects the performance of a prototype classifier. We derived generalization bound that does not require the features to be distributed in Gaussian distribution and the class-conditioned covariance matrix does not have to be the same among classes. We found that using EST+L2 noom makes the classifier comparable with ProtoNet and the eeeerrrrvvlluatoon-baaeeee the number of shots in a support set increases, the performance gain from natture-tranffommatie method decreases, which is consistent with the results of theoretical analysis.
approach. Our experiments show that when
One limitation of this paper is that, in the first place, the prototype classifier works well under the assumption that the data distribution for each class is unimodal and isotropic modeling. However, consider that this assumption may be sound in practice because the multimodality of the data distribution in a class is typically caused by the way the data is annotated. Since the number of a support set is small in few-shot settings (for example, in 5-way 5-shot, the number of labeled data is 25), we can easily check the labels and re-label them so that they don't 't become multi-modal.
Broader impact. We believe that our work shows that prototypical classifier is expected to be a practically useful first step in tackling few-shot learning problems because of its simplicity. The progress in few-shot learning can impact importnant problems such medical images and re-identification problems. We also recognize our work might constitute a threat that authoritarian entities deploy few-shot learning algorithms for surveillance.
6 Acknowledgments
This work was supported by JSPS KAKENHI Grant Number 20H04239 Japan.
10

---

References
[1] Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In Advances in Neural Information Processing Systems, 2017.
[2) Tiansi Cao, Marc T.L.w, and Sanja Fidler. theoretical analysis of the number of shots in few-shot learning. In International Conference on Learning Representations (ICL.), 2020.
[3] Wei- Yu Chen, Yen-Cheng Liu, Zsolt Kira, Yu-Chiang Wang, and Jia-Bin Huang. A closer look few-shot classification. International Conference on Learning Representations (ICIR)) 2019.
[4] Yonglong Tian, Yue Wang, Dilip Krishman, Joshua B. Tenenbaum, and Phillip Isola. Rethinking few-shot image classification: good embedding is all you need? In European Conference Computer Vision (ECCV), 2020.
[5] Yandong Wen, Kaipeng Zhang, Zhifeng Li., and Yu Qiao. A discriminative feature learning approach for deep face recognition. In European Conference on Computer Vision (ECCV), 2016.
[6] Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, and Le Song. Sphereface: Deep hypersphere embedding for face recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), 2017.
[7] Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji. Dihong Gong, Jingchao Zhou, Zhifeng Li, and Wei Liu. Cosface: Large margin cosine loss for deep face recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), 2018.
[8] Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Areface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE conference computer vision and pattern recognition (CVPR), 2019.
[9] Oriol Vinyals, Charles Blundell, Timothy Lillicrap, koray kavukcuoglu, and Daan Wierstra. Matching networks for one shot learning. In Advances in Neural Information Processing Systems, 2016.
[10) Frederik Pahde, Mihai Puscas, Tassilo Klein, and Moin Nabi. Multimodal prototypical networks for few-shot learning. In Winter Conference on Applications of Computer Vision( WACV), pages 2021.
[11] Zhong Ji, Xingliang Chai, Yunlong Yu, and Zhongfei Zhang. Reweighting and information- guidance networks for few-shot learning. In Neurocomputing, volume 423, pages 13-23, 2021.
[12] Flood Sung, Yongxin Yang. Zhang, Tao Xiang, Philip H.S. Tom, and Timothy M. Hospedales. Learning to compare: Relation network for few-shot learning. In Computer Vision and Pattern Recognition (CVPR). 2018
[13] Kelsey R. Allen, Evan Shelhamer, Hanul Shin, and Joshua B. Tenenbaum. Infinite mixture prototypes for few-shot learning. PMLR, pages 232-241, 2019.
[14] Carl Doersch, Ankush Gupta, and Andrew Zisserman. Crosstransformers: spatially-aware few-shot transfer. In Advances in Neural Information Processing Systems, pages 4003-4014 2020.
[15] Hang Qi. Matthew Brown, and David G Lowe. Low-shot learning with imprinted weights. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pages 5822-5830, 2018.
[16] Christian Simon, Piotr Koniusz, Richard Nock, and Mchrtash Harandi. Adaptive subspaces for few-shot learning. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), 2020.
[17] Sung Whan Yoon, Jun Seo, and Jackyun Moon. Tapnet: Neural network augmented with task- adaptive projection for few-shot learning. In Proceedings of the 36th International Conference on Machine Learning (ICML), 2019.
[18] Debasmit Das, J. H. Moon, and C. S. George Lee. Few-shot image recognition with manifolds. In International Symposium on Visual Computing (ISVC), pages 3-14, 2020.
[19] Debasmit Das and C.S. George Lee. A two-stage approach to few-shot learning for image recognition. In IEEE Transactions on Image Processing, volume 29, 2020.
11

---

[20] Peyman Bateni, Raghav Goyal, Frank Wood Vaden Masrani, and Leonid Sigal. Improved few-shot visual classification. In Proceedings of the IEEE conference on computer vision and paltern recognition (CVPR), pages 3-14, 2020.
[21] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adap- tation of deep networks. In Proceedings of the 34th International Conference Machine Learning (ICML), pages 1126-1135. PMLR, 2017.
[22] Chelsea Finn, Kelvin Xu, and Sergey Levine. Probabilistic model-agnostic meta- meta-learning. In Advances in Neural Information Processing Systems, pages 9516-9527. 2018.
[23] Aravind Rajeswaran, Chelsea Finn, Sham M. Kakade, and Sergey Levine. Meta-learning with implicit gradient. PMIR, pages 1 1126-1135, 1135, 2017.
[24] Yan Wang, Wei-L.un Chao, Kilian Weinberger, and Laurens der Maaten. Simpleshot: Revisiting nearest-neighhor classification for few-shot learning. volume abs/:1911.04623, 2019.
[25] Shuo Yang, Lu Liu, and Min Xu. Free lunch for few-shot learning: Distribution calibration. In International Conference Learning Representaaioons/[LRRR, 2021.
[26] Cheng Perng Phoo and Bharath Hariharan. Self-training for few-shot transfer across extreme task difference. International Conference on Learning Representatioss((C1RR 2021.
[27] Steinar Laenen and Luca Bertinetto. On episodes, prototypical networks, and few-shot learning. In Advances in Neural Information Processing Systems, 2021.
[28] Simon Shaolei Du, Wei Hu, Sham M. Kakade, Jason D. Lee, and Qi Lei. Few-shot learning via learning the representation, provably. In International Conference Learning Representa- tions(ICLR), 2021.
[29] James Lucas, Mengye Ren, Irene Raissa KAMENI KAMENI, Toniann Pitassi, and Richard Zemel. Theoretical bounds on estimation error for meta-learning. In International Conference on Learning Reprexentations((CLR) 2021.
[30] Keinosuke Fukunaga. Introduction to Statistical Pattern Recognition (2Nd Ed.). Academic
Press Professional, Inc., 1990.
[31] Munawar Hayat, Salman Khan, Syed Waqas Zamir, Jianbing Shen, and Ling Shao. Max-margin class imbalanced learning with gaussian affinity. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 6469-6479, 2019.
[32] Sachin Ravi and Hugo Larochelle. Optimization as a model for few-shot learning. In Interna- tional Conference on Learning Representations (ICLR), 2017.
[33] Mengye Ren, Sachin Ravi, Eleni Triantafillou, Jake Snell, Kevin Swersky, Josh B. Tenen- baum, Hugo Larochelle, and Richard Zemel. Meta-learning for semi-supervised fewshot classification. In International Conference on Learning Representations (ICLR), 2018.
[34] Luca Bertinetto, João Henriques, Philip H.S. Torr, and Andrea Vedaldi. Meta-learning with differentiable closed-form solvers. In International Conference on Learning Representations (ICLR), 2019.
[35] Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifar-
[36] Kwonjoon Lee, Subhransu Maji, Avinash Ravichandran, and Stefano Soatto. Meta-learning with differentiable convex optimization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 10657-10665, 2019.
[37] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The
caltech-ucsd birds-200-2011 dataset. 2011.
[38] Douglas P. Wiens. On moments of quadratic forms in non-spherically distributed variables. In
A Journal of Theoretical and Applied Statisties, volume 23, pages 265-270, 1992.
[39] Michael D. Perlman. Jensen's 's inequality for convex vector-valued function on an infininite- dimensional space. Multivar Anal, pages 52 65, 1974.
12


Reviews:
[{"review_id": "oPCaJvKd31P", "reviewer": "Reviewer_bREF", "summary": "The authors derive a more relaxed generalization bound for prototypical networks. Using the generalization bound, the authors propose the use of feature transformation to improve generalization of classifiers. The authors experiment with different normalization techniques on multiple few shot learning datasets and find that L2 based normalization can produce similar recognition performance as the one linear discriminant regularization.\n", "questions": "In Figure 1, don’t the shape of the clusters vary because Euclidean distance is used for ProtoNets while dot-product is used for Linear layers ? Is it because of the loss function ? The authors can elaborate on this further.\n\nCan t-SNE feature visualization be shown to see how feature transformation affects discriminability ?\n\nCan this feature transformation approach improve non-prototypical few shot learning methods ?\n\n--------- POST REBUTTAL ----------\n\nAfter rebuttal I have increased form borderline accept to weak accept.\n", "limitations": "No societal impacts \n", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "— Strengths —\n\nThe authors derive a generalization bound that requires fewer assumptions about the data distribution family.\n\nExperimental results on multiple datasets are consistently supported by their developed theorem.\n\nThe authors can produce reasonable amount of recognition performance without any fine-tuning and therefore can be useful for devices where backpropagation is not supported.\n\n— Weaknesses —\n\nThe novelty of the proposed bound is incremental compared to the generalisation bound Cao et al. It might be good for the authors to elaborate point-wise differences between the bounds\n\nComparison studies have only been carried out against a few few shot learning methods.\n\nMore complex feature transformation methods should also have been explored such as exponential, non-linear kernels, power transforms etc to see whether they can also produce better generalization.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "eCnzsDeAIXo", "reviewer": "Reviewer_up5k", "summary": "This paper analyzes which factors lead to better performance of a prototype classifier directly applied on top of a learned representation. This stands in contrast to recent work that has shown that training a linear classifier on top is effective but computationally expensive and requires more hyperparameter tuning. Contributions include a new generalization bound on the performance of a prototype classifier that uses fewer assumptions than previous work and better shines light on the critical factors, including variance of the norm, within-class variance, and between-class variance. Experiments are conducted on several benchmark datasets and shows that with appropriate feature transformation and L2 normalization, a prototype classifier can achieve similar performance to a learned linear classifier.\n", "questions": "* As mentioned above, the analysis here is done for a prototype classifier based on Euclidean distance, but cosine distance has also shown to be effective if distances can be scaled appropriately (Oreshkin et al. 2018). To what extent can the current analysis be extended to cosine distance? See also ll. 47-49, where the gap in performance is hypothesized to be due to the different in loss function. Does the gap decrease when cosine distance-based prototype classifiers are used instead?\n* The generalization bound presented in Section 3.3 suggests that performance would be improved by controlling the variance of the feature norm. In previous work, hyperspherical prototype networks have been proposed (Mettes et al. 2019), where the norm of feature vectors is enforced to be 1. How does such an approach relate to the  analysis in the paper?\n* How do the multiclass results compare to the binary results? Are there any qualitatively different takeaways?", "limitations": "More discussion about the limitations of the analysis would be useful: e.g. choice of distance and how this could possibly be addressed.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n+ The aim is well-motivated. Eliminating the need for retraining a linear classifier and instead simply applying a prototype classifier would make it easier to adapt pre-trained representations to the few-shot setting.\n+ The generalization bound removes simplifying assumptions of previous work and in doing so paints a more nuanced picture of the important underlying factors.\n+ Experiments are extensive and consist of four datasets and multiple variations of the prototype classifier, including two feature transformation methods both with and without L2 normalization.\n\nWeaknesses\n- The clarity could be improved in some places. One is that the discussion of LDA and EST is rather light relative to the important role they play in the experiments. Another is a discussion of how the multiclass bound relate to the binary bound presented in the main paper.\n- The discussion of the results could be improved. The important terms are explained at the end of Section 3.3 but it is not very intuitive how future work should go about trying to better control them. I also found Figure 2 somewhat confusing and it was not immediately clear to me how the figure relates to the experimental results.\n- The analysis is limited to Euclidean distance, which is a valuable first step, but coupling this with a discussion of e.g. cosine distance would be useful. This is relevant since standard classifiers use inner products to compute logits, which is more closely related to cosine distance than Euclidean.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "KM7CD0chfA0", "reviewer": "Reviewer_7Qcy", "summary": "Recently, the linear classifier on the top of pre-trained model without meta-learning performs comparably to the prototypical networks for few-shot learning. However, these approaches require re-training every time a new class arrives. This paper focuses on prototype classifier which doesn't require re-training or any meta-learning approach. It founds that the variance of the norm of feature vectors affect the performance of prototype classifiers. It then derives a generalization bound that does not depend on assumptions on the class-conditional distributions. Moreover, the paper theoretically shows that this bound decreases as the variance in norm of the feature vectors decreases, and it empirically investigaes different transformation approaches to lower this variance. Finally, the authors analyze the effectiveness of the feature transformations on multiple datasets that include mini-ImageNet, tiered-ImageNet, CIFAR-FS, CUB, and FC-100.", "questions": "Please see my questions in the weakness section.", "limitations": "Yes, authors adequately addressed the limitations and potential negative societal impact of their work.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths:\n\n* The paper investigates an interesting problem for few-shot learning by proposing to use prototype classifiers which generally don't require re-training or meta-learning algorithm.\n\n* It theoretically analyze the prototype classifier in terms of variance of the norm of the feature vectors\n\n* It investigates several transformation approaches to minimize this variance that includes L2 normalization, EST,LDA,\nEST+L2-norm and LDA+L2-norm\n\n* Experiments on several datasets corresponding to the two different task i.e. i) standard object recognition and ii) cross-domain adaptation.\n\n### Weakness:\n\n* Novelty of generalization bound is limited as it derives a the bound by modifying (Cao et al., 2020)\n\n* 1-shot performance looks good, but for 5-shot experiment, there is negligible performance boost compared to the baseline++ for miniImageNet, tieredImageNet, and CIFAR-FS in Table 1. Could authors provide any explanation or give remarks about it? It is written in the manuscript that all the methods are reimplemented. It could be that the baseline++ was not properly tuned or trained. Even in Table 2, the performance gap is negligible compared to the other methods. Moreover, the same phenomenon can be observed for cross-domain experiments in Table 4 in supplementary materials\n\n* The authors have proposed several feature transformation methods, but there is not a single one which stands out? Can the authors please explain this behavior?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept