Invariance-Guided Feature Evolution  for  Few-Shot Learning
Authors: Wenming Cao
Zhineng Zhao
Qifan Liu
Zhihai He
Venue: ICLR 2022 Submitted (2022)
Keywords: Few-shot learning
Invariance loss

Abstract: Few-shot learning (FSL) aims to characterize the inherent visual relationship between support and query samples which can be well generalized to unseen classes so that we can accurately infer the labels of query samples from very few support samples. We observe that, in a successfully learned FSL model, this visual relationship and the learned features of the query samples should remain largely invariant across different configurations of the support set. Driven by this observation, we propose to construct a feature evolution network with an ensemble of few-shot learners evolving along different configuration dimensions. We choose to study two major parameters that control the support set configuration: the number of labeled samples per class (called shots) and the percentage of training samples (called partition) in the support set. Based on this network, we characterize and track the evolution behavior of learned query features across different shots-partition configurations, which will be minimized by a set of invariance loss functions during the training stage. Our extensive experimental results demonstrate that the proposed invariance-guided feature evolution (IGFE) method significantly improves the performance and generalization capability of few-shot learning and outperforms the state-of-the-art methods by large margins, especially in cross-domain classification tasks for generalization capability test. For example, in the cross-domain test on the fine-grained CUB image classification task, our method has improved the classification accuracy by more than 5%.

Under review as a conference paper at ICLR 2022
INVARIANCE-GUDDER FEATURE FEW-SHOT LEARNING
EVOLUTION FOR
Anonymous authors Paper under double-blind review
ABSTRACT
Few-shot learning (FSL) aims to characterize the inherent visual relationship be- tween support and query samples which can be well generalized to unseen classes so that we can accurately infer the labels of query samples from very few sup- port samples. We observe that, in a successfully learned FSL model, this visual relationship and the learned features of the query samples should remain largely invariant across different configurations of the support set. Driven by this obser- vation, propose to construct feature evolution network with ensemble of few-shot learners evolving along different configuration dimensions. We choose study two major parameters that control the support set configuration: the num- ber of labeled samples per class (called shots) and the percentage of training sam- ples (called partition) in the support set. Based on this network, we characterize and track the evolution behavior of learned query features across different shots- partition configurations, which will be minimized by a set of invariance loss fune- tions during the training stage. Our extensive experimental results demonstrate that the proposed invariance-guides feature evolution (IGFE) method significantly improves the performance and generalization capability of few-shot learning and outperforms the state-of-the-ard methods by large margins, especially cross- domain classification tasks for generalization capability test. For example, in the cross-domann test on the fine-grat are-graiddd CUB image classification task, our method has improved the classification accuracy by more than 5%.
INTRODUCTION
Few-shot image classification aims to learn to predict the labels of query images from very small sets of support samples (Finn et al. 2017 Yu et al. 2020). More importantly, this learned representation needs to generalize well onto unseen classes during the test stage. Since the number of support images per class, also called shots, is very small, often in the range of I to 5, the characteristics of support samples will vary significantly. Therefore, a successful few-shot learning (FSL) should be able to survive this large variation of specifcc characteristics in the support set (Yu et al. 2020 Bronskill et al.] (020). This requires that the FSL be able to capture the underlying or inherent visual relationship between the query and support images, instead of being distracted or over-fitted by the image- specific characteristics, especially those non-salient features from the background. In other words, this inherent visual relationship, once successfully learned, should remain relatively invariant across different sample configurations of the support sets. Otherwise, its classification performance with very few support samples and generalization capability onto novel classes will degrade significantly.
Based on this observation, we put forward the following observation: the feature representation generated by the FSL, if successfully learned, should remain relatively invariant to sample configu- rations of the support sets, for example, the number of shots (i.e, the number of samples per class) and the selection of support samples for training. Driven by this observation, we propose to con- struct a feature evolution network with an ensemble of few-shot learners which evolve the feature embedding along dimensions of the support set configuration, as illustrated in Figure In this work, we choose to study two major parameters that control the support set configuration: the number of labeled samples per class (called shots) and the percentage of training samples (called partition) in the support set. This network aims to characterize and track the evolution and variation of learned features of the query samples across different shot-partition configuration of the support samples.

---

Under review as a conference paper at ICLR 2022
Invariance Constraint
Shot-Partition Variations
Query Set
Support Set
Figure 1: Illustration of the proposed idea for invariance-guide guiiedd feature evolution for few-shot learn- ing.
We then introduce an invariance metric to measure the variation of the learned query features. This invariance metric serves a loss function to be optimized during the FSL process. Our exten- sive experimental results demonstrate that the proposed IGFE method significantly improves the performance of FSL and outperforms existing state-of-the-art methods by large margins various benchmark datasets.
RELATED WORK
There three major categories of methods that have been developed for few-shot learning: meta learning, transfer learning, and metric learning.
(1) In meta-learning based methods, episode training, which mimics the test tasks during the train- ing stage, is explored to improve the knowledge transfer and generalization capability from meta- training to meta-testing (Finn al., [017 Rusu al.) 2018). Zhang al. (2020a); Ye et al. 2020); Chen et al. (2020b), meta-learning is combined with pre-training to further improve the performance. Tian et al. (2020) Rizve et al. (202T) decoupled the learning procedure into the base- class embedding pre-training and novel-class classifier learning and develop a multivariate logistic regression and knowledge distillation method to improve the leatanng performance.
(2) Transfer-learning methods (Qiao et al. 2018 Gidaris & Komodakis) 2018 Chen et al. 2019)
pre-train the model with large set of base classes and then adapt model to unseen classes. Transmatch(Yu et al. (2200) pre-trains feature extractor on the base-class data, uses it as the feature extractor to initialize the classifier weights for unseen classes, and updates the model using semi- supervised learning methods. Gidaris & Komodakis (2018) propose an attention module to predict the classifier weights for novel classes dynamically. Instead of separating the base classes into a set of few-shot tasks, methods in Qiao al. (2018): 8); Chen et al. (2019) all base classes to pre- the few-shot model, which is then adapted to novel-class recognition.
(3) Metric-based methods (Sung et al. 2018: Zhang al. 2020b) aim to learn effective measure- ment space and identify query samples based on the feature distance between samples. MatchingNet (Vinyals al. 2016) shows that neural networks can be successfully trained to measure the simi- larity between query and support samples. The prototype network method (Snell et al. 2017)) intro- duces prototype, which represents class. The distance between prototypes and query vector is used for the classification. Recent methods utilize task-spe prototypes. For example, FEAT (Ye transformation on the prototypes. BSNet (Li et al. 2021)
al.) 2020) applies self-attention-bee uses dual-metric distance and achieves better performance than single-metric distance.
Recently, graph-based methods have been explored to model and predict the correlation between samples, aiming to propagate meta-knowledge and the labels from base classes to novel classes (Rodríguez et al. 2020 Satorras & Estrach, 2018). Chen et al.] (2020a) develop a graph neural network-based few-shot learning method which iteratively aggregates features from neighbors and leam the correlation between samples. DPGN (Yang et al., (2200) has proposed a two-fold complete graph network to model instance-level and distribution-level relations with label propagation and transduction. The MCGN method (Tang et al. 2021) combines GNN and conditional random field (CRF) to achieve better affinity prediction.
2

---

Under review as a conference paper at ICLR 2022
Knowledge distillation. This work is related knowledge distillation, which learns small student network using the supervision signals from both ground truth labels and a larger teacher network (Gou et al. 2021). Three different types of knowledge supervision signals have been explored in
the literature. (1) Response-based knowledge distillation uses the neural response of the last output
layer of the teacher network the supervision information (Hinton al. 2015 Zhang al. 2019
Meng et al. 2019). (2) Feature-bass knowledge distillation uses the feature map of the middle
layer as the supervision signal train the student network (Romero et al. 2014 Wang
2020
Chen et al. 2021). (3) Relationship- hassss knowledge distillation further explores the relationship
between different layers or data samples (Yim et al. 2017 Passalis et al.) 2020; Dong et al. 2021)
For example, the second order statistics between different network layers are used in Yim et al. (2017) as the supervision signals.
METHOD
In this section, we present our proposed method invariance-guided feature evolution (IGFE) for few-shot learning.
3.1 CLASS FEATURE VARIATIONS ACROSS DIFFERENT SUPPORT SET CONFIGURATIONS
In this section, experimentally study the variation of query set features across different support set configurations, which motivates us to make the invariance observation discussed in Section 1. In typical setting, K K-way N-shot FSL has K classes with N support samples per class in its training batch. The training samples from one class will be divided into M batches. In total, we have K x M samples per class.
Closs Variance
1.8
1.8
1.4
1.2
1-shot
2 2-shor
shot
0.8
0.
0.4
0.2
4-shot
5-shot (a)
10-shot
100
200
300
Batches (b)
400
500
600
Figure 2: (a) visualization of class features across partitions for different numbers of shots; (b) average variance of class features across shots for different numbers of batches of support samples used for training.
The performance of the FSL directly depends on the configuration of the support set. In this work, study two important parameters that plays important roles in FSL: (1) N, the number of shots the number of support samples per class, and (2) M, the number of sample batches or the selected partition of the support set used for training. For simplicity, we refer these types of configuration as shot configuration and partition configuration. In the following experiments, we perform few- shot learning on the mini-ImageNet dataset. Figure a) shows the t-SNE plots for query sample features of 5 classes for different shots N. In each shot configuration, we choose different subset partitions of the support samples for training. We can see that, for FSL with larger N, the variation of query class features is becoming smaller. This implies that, in better learned FSL. model, the feature variation across different support set partition is becoming smaller. Figure 2 shows the experimental results for different numbers of batches subset partitions of the support set used for training. For each batch or support set partition, we perform the FSL with different N and compute the average variation of query class features. We can see that, more batches of support samples used, the FSL better learned, the query class feature variation across different shots is becoming smaller.
These experimental results motivate us to make the following observation: in a better learned FSL model, the variation of query sample feature across different shot and partition configurations of
3

---

Under review as a conference paper at ICLR 2022
the support will be smaller. Driven by this observation, develop the invariance invariance-guided guided feature evolution (IGFE) method for few-shot learning, which will be explained int the following section.
Query Labels
3-Shot
L 33
T3-11
2-Shot
22-1
h 1-2
Classification
Query Set
Query Labels
Shot Configuration Variation
Support Set +Labels
D
Training
Partition Configuration
Support Set
Variation
(b) Figure 3: (a) Overview of the proposed IGFE method; (b)The IGFE testing scheme.
(a)
3.2 INVARIANCE- GUIDED FEATURE EVOLUTION
Figure 3(a) provides an overview of the proposed invariance-guided feature evolution (IGFE) method. use the one-shot learning the partition variation, we use three different subsets of the support samples for training: (1) Sw which is the whole support set; (2) AA and Sg which two randomly chosen subsets of the sup- port set. For example, in our experiments, each subset contains 60% of the whole support samples. For the shot variation, besides the baseline one-shot learning module, we introduce two additional FSL modules with N 2 (two-shot) and N 3 (three-shot). In this way, we have the follow- ing four FSL modules: & (two-shot) and $ (thee-shot) trained on the whole support set, & A (one-shot) and 2 (one-shot) trained on support subsets S and SB, respectively. All of these four FSL modules are using the same graph neural network (GNN) design which will be explained the following section.
example explain proposed IGFE method. For
In this work, we propose to explore the invariance property of the learned FSL feature across dif- ferent support set configurations to optimize the generalization capability of the FSL. As illustrated Figure a]..
X be the input image. { and $ 1 (X) be the features generated by the one-shot learning networks trained support subsets S and Sg. respectively. Let f and i (X) be the features generated by the two-shot and three-shot networks trained on the whole support set W, respectively. During our experiments, measure and optimize the distance between the one-shot and two-shot features. Instead, we intro- duce two auto-encoder networks, T2-1 and T3 to transform the feature vectors & (X) and $ (X), respectively, so as to match the feature vectors generated by the one-shot networks. Then, introduce the following two feature invariance loss functions for the one-shot network $1
observe that it is not efficient directly
[x (X) T2-.1 ? (x))
CT Ex T3 @ (x))
(2)
Here, Ex represents the statistical average over the training batch. Similarly, can define the feature variance loss 2 and 6E for the one-shot network $
Note that one-shot networks $ (X) and $ B (X) are trained over two different subsets of the sup- port samples. This unique design motivated by the following two considerations. (1) It can be used to measure and minimize the variation across the partition configuration variation, i.e., different partitions of the support set for training. (2) The fusion of the features generated by these network result in more efficient shot classification with improved generalization capability. To mea- sure and minimize the invariance between these two one-shot networks, we propose to analyze the their softmax outputs. Specifically, let (X) (0), 02 03 ..., 0%1 be the output at the softmax
4

---

Under review as a conference paper at ICLR 2022
layer of network 1 Then, the probability for sample X to be in class i is given by
exp((*))
PA( E exp
(3)
Similarly, we can define the class probability P's (X) for network $ We use the Kullback- (KI.) Divergence to measure the invariance between the outputs of these two one-shot networks
[x)(X)
Cp ΣΣ
PA (X) log
(4)
[Pb(
In our experiments, we use the loss function Cp + L1-12 + L1-13 to train both one-shot networks.
should noted that the loss function in is defined for the whole batch of samples. other words, the training process aim to the minimize the overall variation of a set of query samples, instead of an individual one. From our experiments, we observe that, for each individual query sample, the features generated by these two one-shot networks will differ slightly. To produce the feature F' (X) for each individual query sample, we propose to fuse these two features using the following average operation
$ 2 + (X)
F F
(5)
It should be noted that the above shot-partition configuration variations are only performed on the support set during the training stage. During the testing stage, once successfully trained, only the baseline 1-shot network is used for testing. Figure shows the network structure in the test stage. The support samples (with labels) and the query samples (without labels) are passed to both trained networks 2 and & to produce the features for the each query sample. These two features averaged according to (5) and then passed to the classification head to predict the query labels.
example to explain the proposed IGFE method. It can be easily extended to other settings of few-shot learning. For example, for 5-way 5-shot, those two networks D2 (2-shot) and T (3-shot) introduced for shot variations will be replaced with 10- shot and 15-shot configurations. Note that both the shot and the partition configurations are simply different ways to use the support samples during the training stage.
We have used the 5-way 1-shot learning
3.3 BIPARTITE GRAPH NEURAL NETWORK FOR SUPPORT- QUERY SAMPLE ANALYSIS
bipartite graph neural network (GNN) perform joint analysis of the support and query samples. In K K-way N-shot learning, given K classes, each with N support samples {. need learn the FSL network predict the labels for K query samples )Q&). This implies, in each training batch, we have K x (N + 1) support samples and query samples. As illustrated in Figure 4 we use backbone network, for example, ResNet-10 or ResNet- extract feature for each of these support and query samples. Denote their features by S = (s } and Q These support-query
In this work, propose
} where I represents the update iteration index the GNN. Initially, sample features form the nodes for the GNN, denoted by (x ر S J), J = K x (N + 1), for the simplicity of notations. The edge between two graph node represents the correlation rx(x;, x/) between nodes x and x Note that our GNN has two types of nodes: support sample nodes and query sample nodes. The support samples nodes have labels while the labels of the query samples need be predicted. If x and x are both support nodes, we have
1 if [(xx) = ((x)) * x({xxx) = 0 lf + (xx)).
(6)
Here 1(-) represents the label of the corresponding support sample. Since the labels for the query nodes are unknown, the correlation for edges linked to these query need to be learned by the GNN. Initially, they can be set as random values between and 1. Note that the edge connection between the query node and support node represents the correlation between them. Therefore, we can use this correlation provide very important information for inferring the label for the query node from the label of the other support node.
Each node of the GNN combines features from these neighboring nodes with the corresponding cor- relation weights and updates its own feature by learning multi-layer perceptron (MIP) network
5

---

Under review as a conference paper at ICLR 2022
Evolution
Labels
L(S)
Encoder Network
Features
Figure 4: Overview of the graph neural network for few-shot feature embedding.
Fl:] as follows
x =F vx
At each edge, another MLP network I| is learned predict correlation between its two nodes,
*x xx) 9 g[x xf],
(8)
whose ground-truth values determined using the scheme discussed in the above. This bipartite GNN is jointly trained with the feature transform (or evolution) networks, and the final classification network which predicts the query labels.
Table 1: Classification accuracy of general few-shot classification mini-Image
mini ImageNer
Methods
Backbone
5-way I-shot
S-way 5-shot
LBO Rusu et al
2018
WRN 28
61.76 1 0.08%
77.59 1: 0.12%
PPA
Ciao et
2018
WRN-28
59.60 : 0. 0.41%
73.74 +: 0.19%
wDAE Gidaris
Komadakks
2019
WRN-28
61.07 1 0.15%
76.75 0.11%
62.93
79.87
Yan
Res-18
6.63%
78.85
ProtoNet (C Chen ei al. 2019
Res-10
51.98 : 0.84%
72.64 0.64%
Matching (Chen el mill
(Chee
2019
Res 10
54.49 1 0.81%
68.82 1: 0.65%
RelationNer (Cben ot al
2019
Res-10
52.19 = 0 83%
70.20 + 0.66%
Cross Domann [sengg el
2020
Res-10
66.32 U.80%
81.98 0.55%
FEAT [Fii
20200
Res-12
66.78 0.20%
82.05 0.14%
FRN Wertheimer
2021
Res-12
66.45 0.19%
infoPatch Liu et al.
20:1
Res-12
67.67 = U. 0.45%
82.44 0.31%
IER-Distil Rizve al
2021
Res-12
67. 28 0. 8.8%
84.78 0.52%
Our IGFE Method
Res 10
71.48 1 0. 0.26%
86.22 0.16%
Res-12
72.06 + 0.25%
86.59 0.16%
Table 2: Classification accuracy of fine-grained few-shot classification on CUB.
CUB
Methods
Backhone
I-shot
5-way 5-shot
D&CC) Li
2021
Conv4
65.20 + 0.92%
84.18 + 0.641%
DN4 Li al
2019
Conv8
64.26 1: 1.01%
84.41 1: 0.63%
Baselinc==
Chen ct al.
2019
Res-18
67.02 + 0.90%
83.58 + 0.5-1%
MatchNet
Vinyals et
2016
Res-18
73.49 + 0.89%
84.43 - 0.3%6
Neg
0.85%
0.43%
DSN+
Simon
2020
Res-12
0.21%
0.34%
CTX
Doerscl
2022
Res-12
79.34 + 0.21%
91.42 - 0.11%
FRN
Werthemmer et
2021
Res-12
83.55 + 0.19%
92.92 1 0.10%
Our IGFE Method
Res 10
85.47
0.22%
94.33 1 0.10%
Res
86.54 0.22%
94.53 1 0.10%
4 EXPERIMENTS
In this section, conduct experiments to evaluate the performance of the proposed IGFE method and compare it with the state-of-the-art methods. We also provide ablation studies to further under- stand the proposed method.

---

Under review as a conference paper at ICLR 2022
4.1 DATASET
We evaluate method on five widely used few-shot learning benchmarks: mini-Image VVinyals et al. 2016). CUB-200-2011 (Wah et al. 2011), Cars (Krause et al. 2013). Places and Plantae (Van Hom et al. 2018). More details of dataset presented in Appendix A.I.
4.2 IMPLEMENTATION DETAILS
We conduct experiments with backbone networks, ResNetl0 and ResNet12 (Tseng al. 2020 which are extensively used in few-shot learning tasks. The feature transformation ML. has three hidden layers with dimensions of 480, 480 and 458, respectively. Meanwhile, normalization and activation operations are performed after each hidden layer. Here we use LeakyReLu (Xu et al., 2013) as the activation function. We use the Adam optimizer in all experiments. We first pre- train a feature extractor with 400 epochs on the base class (e.g., the mini- Imni-ImmgeNNet and set the classification category 200. The GNN-based feature embedding networks are trained with 400 epochs. In all training and testing experiments, we resize the picture to 224x224.
Evaluations are conducted in 5- 5-way 1-shot and 5-shot settings on benchmark datasets, including mini-ImageNet. CUB-200-2011, Cars, Places, and Plantae. In the N-way K-shot setting, meta- test task has N classes, each with K labeled support samples. We randomly sample 10000 meta-test tasks from the test dataset and then report the mean accuracy and the 95% confidence interval. We also randomly sample 15 queries for each of the five classes in 5-way 1-shot and 5-shot settings for each meta-test task.
In order to evaluate the effectivenesss of our proposed IGFE method, following existing work in the literature, we conduct FSL experiment in the following three scenarios: (1) general few-shot classification in the same object domain, (2) fine-grained few-Shot classification in the same domain, and (3) cross-Domain few-shot classification. For the cross-domain FSL, the image classes for the training and those for the testing are from different domains, for example, mini-ImageNet images for training and CUB images for testing.
(1) General Few-Shot Image Classification. In the first part of our few-shot learning experiments, we evaluate the FSL performance on the mini- Imni-ImaeeNet dataset. Table D shows the results of our method and existing methods in the literature. We can see that our IGFE method outperforms the current best EI-Distill method (Rizve et al., (02T)) by a large margin. For example, for the --shot classification, our method improves the accuracy by 4.78% with the ResNet-12 12 backbone. For the 5-shot classification, the performance gain is 1.81%.
(2) Fine-Grained Few-Shot Classification. The CUB-200-2011 dataset is often used to evaluate the performance FSL methods on fine-grained image classification. Table ? summarizes the performance comparison results. It can be seen that our method outperforms existing sfate-of-the- art methods. For the 1-shot fine-grained classification task, our method improves the classification accuracy by 2.99% upon the current best FRN method (Wertheimer et al. 2021). For the 5-shot classification task, our method outperforms the FRN method by 1.61%.
Table 3: Cross-domain few-shot classification results. We train the model on the mini-ImageNet domain and evaluate the trained model in other domains.
5-way 1-shot
Backbonc
CUB
Cars
Places
Plantac
MatchingNet+FT (Tsene
2020
Res-10
36.61
0.53%
29.82 + 0.41%
51.07 # 0. 0.68%
34.48 # 0.50%
RelationNet+FT [Teng et al.
2020
Re
44.07 = 0.77%
28.63 1 0.59%
50.68 1: 0.87%
33.14 1: 0.62%
GNN+FT 20
Res 10
47.47 1 0.73%
31.61 1 0.53%
55.77 1 0.79%
35.95 1 0.58%
GNN+ATA WWang Dene
2021
Res-10
45.00
0.50%
33.61 + 0.40%
53.57 = 0.50%
34.42 0.40%
GRRG (Snn 2021
Res-10
0.51%
32.78 1 0.39%
54.83 0.56%
37.49
0.43%
Our IGFE Method
Res
49.94
0.24%
34.13 1 0.20%
60. 1 0.28%
39 0.21%
Res. 12
53.16 1 0.25%
33.94 1: 0.19%
59.56 1. 0.27%
0.21% 40.51
3-way 5-shor
Backbone
CUB
Cars
Places
Plantac
MatchingNet+FF (Txeng et
2020)
Res: 10
55.23 = 0.83%
41.24 1 0.65%
64.55 1 0.75%
41.69 1: 0.63%
RelationNet+#FT Tiene et
2020
Res-10
59.46
U.71%
39.91 + 0.69%
66.28 : 0.72%
45.08 0.59%
GNN+FT TTeng el
Res
0.68%
1: 0.64%
73.94
0.67%
53.85
GNN+ATA Wang
2021
Res 10
66.22
0.50%
49.14 1 0.40%
75.48 1 0.40%
52.69
0.40%
LPR-GNN Sun al
2021
Re
64.44
48%
46.20 + 0.46%
75. # 0.47%
54.46 + 0.46%
Our IGFE Method
Ro
69.12 + 0. 21%
49.43 = 0.21%
78.01 + 0.20%
59 99 0.21%
Res: 12
71.11 1 0.21%
49.72 1: 0.21%
77.58 1 0.20%
59 88 1 0.20%
7

---

Under review as a conference paper at ICLR 2022
experiments aim to evaluate the general- ization capability of few-shot learning (Chen et al. 2019). Following existing work in the literature, train the model on base classes of the mini- ImageNet domain and evaluate the cross-domain performance of the model on the CUB, Cars, Places, and Plantae domains respectively. Table shows the performance comparison for 5-way 1-shot image classification with existing methods. In the second column we also include the intra-domain few-shot classification accuracy on the mini- ImageNet. We can see that our IGFE method is able significantly improve the cross-domain classification accuracy and demonstrates good generalization capability. For example, on the Places dataset, the accuracy is improved by 4.99% upon the GNN+FT (Tseng et al. 2020) method, which is quite significant. For other datasets, the performance improvement is between 1.4% and 2.2%, which is quite consistent. We also include results of our IGFE method with the ResNet-12 backbone. Table [3 (bottoo) shows the performance comparison for the 5-way 5-shot image classification task. Our method outperforms the current state-of-the-an methods by large margins.
(3) Cross-domain few-shot classification.
4.3 ABLATION STUDIES
In this section, we conduct ablation studies to further understand the proposed IGFE method and evaluate the performance contributions of major algorithm components. In our IGFE method, we have introduced two major support set configurations: shot and partition configurations. The shot configuration determines the number of samples assigned to each training classes in the support set. Partition configuration determines the subset of support samples assigned to each FSL module. It should be noted that this shot and partition configurations are only performed for the training sam- ples. In the test stage, once the base FSL module is successfully learned, these two configurations of the support set are removed. For example. in the 5-way 1-shot learning, although 2-shot and 3- shot configurations of the support set are used during the training, in the test stage, we only use the 5-way 1-shot network to predict the labels of query samples, exactly following the same protocol as in existing methods in the literature. Table summarizes the performance contributions of these two algorithm components for 5-way I-shot (top) and 5-way 5-shot (bottom) classification tasks on five benchmark datasets. the first row shows the classification accuracy of the baseline method Tseng et al. 2020) on which we have implemented our algorithm. The second row shows the per- formance with the shot variation with 1-shot, 2-shot, and 3-shot configurations. The third row shows the performance with both shot and partition variations where two different partition or subsets of the support set are used to train two 1-shot networks, both guided by the shot variations. We can that both algorithm components contribute significantly the overall performance. However, on different datasets, their portions of contribution are vary slightly. Experiments for the 5-way 5-shot classification task are shown in Table 4 bottom). We can see that, for 5-shot classification, the partition variation has more significant contribution than the shot configuration. This is the be- cause the number of shots in this 5-shot classification task is already large (5). So, the importance of introducing extra shot variation degrades.
In shot variation, we have included 1-shot, 2-shot, and 3-shot. It is observed that, if more shot variations are used, the performance will be further improved. For example, in Table we compare the 5-way 1-shot classification accuracy with two shot variations (1-shot + 2-shot) and three shot variations (I-shot 2-shot + 3-shot). We can see that the three shot variations scheme obtains better classification accuracy than the two shot variations scheme.
Table 4: The ablation study on mini-ImageNet, CUB, Cars, Places, and Plantae datasets. We train the model on the mini-ImageNet domain and evaluate the trained model on other domains.
5-way I-shot
mini-ImageNet
CUB
Cars
Places
Plantac
Baseline
66.32 1 0.80%
47.47
0.75%
31.61 1. 0.53%
55.77 1 0.79%
35.95 1 0.58%
Shot Variation
69.63 + 0.25%
47.86
0.21%
33.20 + 0.16%
58.14 + 0.25%
39.53 + 0.17%
Shot and Purtition Variation
71.48 1 0.26%
49.94
1 0.24%
34.13 1 0.20%
60.76 + 0.28%
39.77 1 0.21%
5-way 5-shor
mini-ImageNNt
CUB
Cars 44.90 0.64%
Places
Plantac
Baseline
81.98 0.55%
66.98 0.68%
73.94 0.67%
53.85 1 0.62%
Shot Variation
83.81 0.17%
61.03
0.21%
16.21 0.21%
71.12
0.21%
51.16 + 0.20%
Shot and Partition Variation
86.22 + 0.16%
69. 12 + 0 2.1%
49. 43 0.21%
78.01 + 2.20%
59.99 + 0.21%
Table 5: The IGFE method with different shot configurations.
Shot Configurations
mini-ImageNeet
CUB
Cars
Places
Plantac
(1 (1-shot+ + 2-shol) Variation
68.92 1 0.25%
16 1 0.22%
32.70 1 0.16%
57.83 1: 0.25% 35.91 + 18%
(I-shot+ + 2-shot + 3-shat) Variation
69.63 + 0.25%
47.86
0.21% 33.20 + 1.1%
58.44 # 0.25% 39.53 - 0.1%
8

---

Under review as a conference paper at ICLR 2022
5 UNIQUE CONTRIBUTIONS AND FURTHER DISCUSSIONS
Compared to the existing work in the literature, the unique contributions of this work can be sum- marized as follows: (1) This work is motivated by an important observation that, in a successfully learned FSL model, the generated features for query samples should remain largely invariant across different support sample configurations. We then construct a network structure to characterize their feature evolution and explore their variations. (2) We introduce two types of configuration vari- ations, namely, shot and partition variations, to construct different configurations of the support samples during the training stage. Using the GNN, we explore the relationship between the query samples and these different configurations of the support samples. We introduce an invariance loss function to train this network to minimize the feature variation across different support sample con- figurations. (3) Our experimental results have demonstrated that the proposed IGFE method is able to significantly improve the FSL performance, outperforming existing state-of-the-art methods by large margins. The ablation studies have also demonstrated that the proposed shot-partition con- figuration variation of the support set and the invariance loss on the query samples are able sig- nificantly improve the few-shot classification accuracy upon the baseline GNN+FT method (Tseng al. 2020), on which our algorithm implemented.
Successful generalization of the network model learned from the training set to the test set is an important problem in machine learning (Zhou 2014). It has been recognized that optimizing the marginal distribution by maximizing the marginal mean and minimizing the marginal variance si- multaneously can lead to better generalization performance for traditional machine learning method, such as support vector machine (SVM) classification (Zhou) 2014, Zhang & Zhou 2020). In this work, we explore the invariance constraint to minimize the feature variance of query classes so as to improve the generalization capability of the few-shot classification. From the experimental results, We can see that our proposed performs much better on cross-domate tasks than intra-domain tasks.
The proposed invariance loss across multiple networks is related to the consistency loss (Ouali et al, 2020 [Abuduweili et al.] 202T) and the student-teacher or dual student network (Ke et al. 20199) that have been extensively used in semi-supervised learning research. In this work, we explore this in- variance and mutual consistency in the few-shot learning framework where the feature invariance or consistency is enforced between the query samples with different configurations of support samples over a graph neural network.
6 CONCLUSION
In this work, we have developed a new few-shot learning method, called invariance-guided feature evolution (IGFE), aiming to characterize the inherent visual relationship between support and query samples which can be well generalized to unseen classes. From our experimental studies, we have observed that, a successfully learned FSL model, this visual relationship and the learned features of the query samples should remain largely invariant across different configurations of the support set. Driven by this observation, we have constructed a feature evolution network with an ensemble of few-shot learners evolving along different configuration dimensions. We have introduced the shot and partition configurations that control different assignments of the support samples during training. Using bipartite graph neural network, we analyze the feature relationship between query samples and different configurations of support samples. We characterize and track the evolution behavior of learned query features across different shots-partition configurations, which is minimized by a set of invariance loss functions during the training stage. Our extensive experimental results demonstrate that the proposed invariance-guided feature evolution (IGFE) method significantly improves the performance and generalization capability of few-shot learning and outperforms the state-of-the-an methods by a large margin.
REFERENCES
Abulikemu Abuduweili, Xingjian Li, Humphrey Shi, Cheng-Zhong Xu, and Dejing Dou. Adaptive the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pp. 6923-6932, 2021.
consistency regularization for supi-super transfer learning. In Proceedings the
John Bronskill, Jonathan Gordon, James Requeima, Sebastian Nowozin, and Richard Turner. Tas- knorm: Rethinking batch normalization for meta-lcarning. In International Conference on Ma- chine Learning, pp. 1153-1164. PMIR,, 2020.

---

Under review as a conference paper at ICLR 2022
Defang Chen, Jian-Ping Mei, Yuan Zhang, Can Wang. Zhe Wang, Yan Feng, and Chun Chen. Cross- layer distillation with semantic calibration. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 7028-70383 2021.
Riquan Chen, Tianshui Chen, Xiaolu Hui, Hefeng Wu, Guanbin Li, and Liang Lin. Knowledge graph transfer network for few-shot recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 10575-10582. 2020a.
Wei-Yo Yu Chen, Yen-Cheng Liu, Zsolt Kira, Yu-Chiang Frank Wang, and Jia-Bin Huang. A closer look at few-shot classification. arXiv preprint (.XXi::904.042233 2019.
Yinbo Chen, Xiaolong Wang. Zhuang Liu, Huijuan Xu, and Trevor Darrell. A new meta-baseline for few-shot learning. arXiv preprint arXiv:2003.04390 2020b.
Carl Doersch, Ankush Gupta, and Andrew Zisserman. Crosstransformers: spatially-aware few-shot transfer. arXiv preprint arXiv:2007. 14498, 2020.
Songlin Dong, Xiaopeng Hong, Xiaoyu Tao, Xinyuan Chang, Xing Wei, and Yihong Gong. Few- shot class-incremental learning via relation knowledge distillation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 1255-1263, 2021.
Nanyi Fei, Zhiwu Lu, Yizhao Gao, Jia Tian, Tao Xiang, and Ji-Rong Wen. Meta-learning across meta-tasks for few-shot learning. arXiv preprint arXiv:2002.04274 2020.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference Machine Learning, pp. 1126-1135. PMLR, 2017.
Spyros Gidaris and Nikos Komodakis. Dynamic few-shot visual learning without forgetting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4367- 2318.
Spyros Gidaris and Nikos Komodakis. Generating classification weights with gnn denoising autoen- coders for few-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 21-30, 2019.
Spyros Gidaris, Andrei Bursuc, Nikos Komodakis, Patrick Pérez, and Matthieu Cord. Boosting few-shot visual learning with self-supervision. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8059-8068, 2019.
Jianping Gou, Baosheng Yu, Stephen Maybank, and Dacheng Tao. Knowledge distillation: A survey, International Journal of Computer Vision 129(6):789-18888) 2021.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in neural network. arXiv preprint arXiv: 500.00233.....,, 2015.
Zhanghan Ke, Daoye Wang, Qiong Yan, Jimmy Ren, and Rynson WH Lau. Dual student: Breaking the limits of the teacher in semi-supervised learning. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pp. 0728-6736,77 2019.
Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained categorization. In Proceedings of the IEEE international conference on computer vision work- 2013.
shops. pp.
Wenbin Li, Lei Wang, Jinglin Xu, Jing Huo, Yang Gao, and Jiebo Luo. Revisiting local descriptor based image-to-class measure for few-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7260-7268, 2019.
Xiaoxu Li, Jijie Wu, Zhuo Sun, Zhanyu Ma, Jie Cao, and Jing-Hao Xue. Bsnet: Bi-similarity network for few-shot fine-grained image classification. IEEE transactions on image processing: publication of the IEEE Signal Processing Society, 10:1811113 2021.
Bin Liu, Yue Cao, Yutong Lin, Qi Li. Zheng Zhang, Mingsheng Long, and Han Hu. Negative margin matters: Understanding margin in few-shot classification. In European Conference on Computer Vision, pp. 438-455. Springer, 2020.
10

---

Under review as a conference paper at ICLR 2022
Chen Liu, Yanwei Fu, Chengming Xu, Siqian Yang, Jilin Li, Chengjie Wang, and Li Zhang. Learn- ing a few-shot embedding model with contrastive learning. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, volume 35, pp.
2021.
Zhong Meng, Jinyu Li, Yong Zhao, and Yifan Gong. Conditional teacher-student learning. In ICASSP 2019-2019 IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pp. 6445-6449. IEEE, 2019.
Yassine Ouali, Céline Hudelot, and Myriam Tami. Semi-supervised semantic segmentation with cross-con training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12674-12660 12684, 2020.
Nikolaos Passalis, Maria Tzelepi, and Anastasios Tefas. Heterogeneous knowledge distillation using information flow moddling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2338, 2020.
Siyuan Qiao, Chenxi Liu, Wei Shen, and Alan L Yuille. Few-shot image recognition by predicting parameters from activations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7229-7236 2018
Mamshad Nayeem Rizve, Salman Khan, Fahad Shahbaz Khan, and Mubarak Shah. Exploring com-
plementary strengths of invariant and equivariant representations for few-shot learning. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10836- 10846, 2021.
Pau Rodríguez, Issam Laradji, Alexandre Drouin, and Alexandre Lacoste. Embedding propagation: Smoother manifold for few-shot classification. In European Conference Computer Vision, pp- 121-138. Springer, 2020.
Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. Fitnets: Hints for thin deep nets. arXiv preprint arXiv:1412.65500 2014.
Andrei A Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascanu, Simon Osin- dero, and Raia Hadsell. Meta-learning with latent embedding optimization. arXiv preprint arXiv:1807.055960 2018.
Victor Garcia Satorras and Joan Bruna Estrach. Few-shot learning with graph neural networks. In International Conference Learning Representations, 2018.
Christian Simon, Piotr Koniusz, Richard Nock, and Mchrtash Harandi. Adaptive subspaces for few-shot learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 4145- 2020.
Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. Ad- vances Neural Information Processing Systems, 30:4077-4087 2017.
Jiamei Sun, Sebastian Lapuschkin, Wojeiech Samek, Yunqing Zhao, Ngai-Man Cheung, and Alexander Binder. Explanation-guit guideed training for cross-domain few-shot classification. In 2020 25th International Conference on Pattern Recognition (ICPR), pp. 7609-7616. IEEE, 2021.
Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip HS Torr, and Timothy M Hospedales. Learning compare: Relation network for few-shot learning. In Proceedings of the IEEE con- ference on computer vision and pattern recognition, pp. 1199-1208, 2018.
Shixiang Tang, Dapeng Chen, Lei Bai, Kaijian Liu, Yixiao Ge, and Wanli Ouyang. Mutual crf- gun for few-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2329 22299,339,22. 2021.
Yonglong Tian, Yue Wang. Dilip Krishnan, Joshua B Tenenbaum, and Phillip Isola. Rethinking few-shot image classification: good embedding is all you need? In Computer ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XIV 16, pp. 266-282, Springer, 2020.
11

---

Under review as a conference paper at ICLR 2022
Hung- Yu Tseng, Hsin- Hsin-Ying Lee, Jia-Bin Huang, and Ming Husnn Yang. classification via learned feature-wise transformation. International Conference on Learning Representations, 2020.
few-shot
Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pictro Perona, and Serge Belongie. The inaturalist species classification and detection dataset. In Proceedings of the IEFE conference on computer vision and pattern recognition, pp. 8769-8778, 2018.
the
Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. Advances in neural information processing systems, 29:3630-3638, 2016.
Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-ucsd birds-200-2011 dataset. 2011.
Haoqing Wang and Zhi-Hong Deng. Cross-domain few-shot classification via adversarial task aug- mentation. arXiv preprint arXiv:2 104. /4385, 2021.
Xiaobo Wang, Tianyu Fu, Shengeai Liao, Shuo Wang, Zhen Lei, and Tao Mei. Exclusivity- consistency regularized knowledge distillation for face recognition. In Computer Vision Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXIV 16, pp. 325-342. Springer, 2020.
Davis Wertheimer, Luming Tang, and Bharath Hariharan. Few-shot classification with feature map reconstruction networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Paitern Recognition, pp. 2012-8021, 2021.
Bing Xu, Naiyan Wang, Tianqi Chen, and Mu Li. Empirical evaluation of rectified activations in convolutional network. arXiv preprint 1rXiv:1505000855 2015.
Baoming Yan, Chen Zhou, Bo Zhao, Kan Guo, Jiang Yang. Xiaobo Li, Ming Zhang, and Yizhou Wang. Augmented bi bi-path network for few-shot learning. In 2020 25th International Conference on Pattern Recognition (ICPR). pp. 4661-448. IEEE, 2021.
Ling Yang, Liangliang Li, Zilun Zhang. Xinyu Zhou, Erjin Zhou, and Yu Liu. Dpgn: Distribution propagation graph network for few-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13390-13399, 2020.
Han-Jia Ye, Hexiang Hu, De- he-Chhan Zhan, and Fei Sha. Few-shot learning via embedding adaptation with set-to-set functions. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 8808-8817, 2020.
Junho Yim, Donggyu Joo, Jihoon Bae, and Junmo Kim. A gift from knowledge distillation: Fast optimization, network minimization and transfer learning. Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 4133-4144 2017.
Zhongjie Yu, Lin Chen, Zhongwei Cheng, and Jiebo Luo. Transmatch: transfer-learning scheme for supi-supe few-shot learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp.
2020.
Chi Zhang, Yujun Cai, Guosheng Lin, and Chunhua Shen. Deepemd: Few-shot image classifica- tion with differentiable earth mover's 's distance and structured classifiers. Proceedings of the IEFF/CVF conference on computer vision and pattern recognition, pp. 12203-12213, 2020a.
Feng Zhang, Xiatian Zhu, and Mao Ye. Fast human pose estimation. Proceedings of the IEFF/CVF Conference Computer Vision and Pattern Recognition, pp. 3517-3526, 2019.
Teng Zhang and Zhi-Hua Zhou. Optimal margin distribution machine. IEEE Trans. Knowl. Data
Eng., 32(6): 1156, 2020.
Xiaolin Zhang, Yunchao Wei, Yi Yang, and Thomas S Huang. Sg-one: Similarity guidance network for one-shot semantic segmentation. IEEE transactions on cybernetics, 9009):3855-3865 2020b.
12

---

Under review as a conference paper at ICLR 2022
Zhi- Hua Zhou. Large margin distribution learning. In Artificial Neural Networks in Pattern Recog- nition 6th IAPR TC 3 International Workshop. ANNPR 2014, Montreal, QC, Canada, October 6-8, 2014. Proceedings, volume 8774 of Lecture Notes in Computer Science, pp. 1-11. Springer, 2014.
A APPENDIX
In this appendix, we provide more details of experimental settings and additional results for further understanding of the proposed method.
A.1 DATASETS
We evaluate method on five widely used few-shot learning benchmarks. (1) mini- mini-ImageNNet (Vinyals et al., 2016) is a subset of ImageNet, which includes 100 classes and each class consists of 600 images. We follow the standard protocol that utilizes 64 classes for training, 16 classes for validation, and 20 classes for testing.
(2) CUB-200-2011 (Wah et al., 2011) is a fine-grained classification dataset, which includes 200 classes and contains 11,788 images. Following |inyals) iet al. (2016), we randomly split the dataset into 100 classes for training, 50 classes for validation, and 50 classes for test, respectively.
(3) Cars (Krause et al.) 2013) created by Stanford University contains 16,185 images of 196 types of cars. We randomly selected 196 categories include 98 training. 49 validation and 49 testing for the experiment. (4) Places (Krause al.) 2013) a dataset of scene images, containing 73,000 training images from 365 scene categories, which are divided into 183 categories for training, 91 for validation and 91 for testing. (5) Plantae (Van Horn et al.)
2018) is sub-set of the 7Nat2017 dataset, which contains 200 types of plants and total of 47,242 images. We split them into 100 classes for training, 50 for validation, and 50 for testing. The number of training, validation, testing categories for each dataset are summarized in Table 6
Table 6: Summarization of the datasets (domains).
Datasets
mini- mini-ImageNet
CUR
Cans
Places
Plantae
Source
Vinyals et al
2016
Wah et al 2011
Krause et al.
2013
Krause et al.
2013
Van Horn et
2018
Training categories
58
183
100
Validation categories
16
Testing categories
20
49
91
50
Split setting
Tseng al.
2020
Vinyals al. 2216
randomly split
randonly split
randomly split
A.2 EXTENSION TO N-SHOT IMAGE CLASSIFICATION
In the main paper, we used the 5-way 1-shot as an example to present the method of invariance- guided feature evolution (IGFE). This is method can be naturally extended to generic K-way N- shot image classification scenarios. Note that the shot is just a way to partition the support samples during the training stage. For example, for a 5-way 5-shot image classification, we construct 10- shot network and 15-shot network to guide the training of the 5-shot network. Once successfully trained, in the test stage, on the 5-shot network is used for predicting the query labels.
A.3 VARIANCE OF QUERY SAMPLE FEATURES
As discussed in the paper, the proposed IGFE method aims to minimize the feature variance across different shot and partition configurations of the support samples. 5 4 examples from the mini-lmageNet few-shot image classification. The second row shows the t-SNE visualization of the query sample features from 5 query classes. Each class has 100 samples. The first row shows the query features obtained by the baseline GNN+FT (Tseng et al. 2020) method upon which our IGFE method was implemented. We compact feature clusters for the query classes than the baseline method. This leads to significant improvement of accuracy and generalization capability in few-shot image classification.
see that the IGFE is able generate much more aggregated and
Figure 6 plots the feature variance of each query class. Each sub-figure shows one experiment of 5-way T-shot image classification the mini-ImageeNet dataset with 100 query samples. In each
13

---

Under review as a conference paper at ICLR 2022
Baseline
IGFE
Figure 5: Visual comparison 5-shot model on minilmageNet dataset, select 100 query samples for each category.
sub-figure, the dashed line represents the variance for each of these query classes generated by the baseline method. The solid line shows the variance of our IGFE method. We can see that our method is able significantly reduce the feature variance of the query classes.
Oue
Car
Due
Due
Verente
Vortande
Varfemne bo
0.4
02
0.
Sample
Sampo
cendro
Vanancce
Variamee
0.
02
02
Sample
Sample
Sampe
Samp
Figure 6: Variance comparison of 5-shot model on minilmageNet dataset.
14


Reviews:
[{"review_id": "x1bui7hkK4d", "reviewer": "Reviewer_yhon", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n1.\tThe experiments on features with different configurations are insightful.\n\nCons:\n1.\tThe relationship of different shot number has been studied in the former work [1]. The authors can have some discussion about the difference.\n2.\tThe main problem comes with the implementation of GNN. Briefly speaking, the authors implement GNN in such a way: For a N-way K-shot episode with M query samples for each class, the GNN is used M iterations, with each iteration one query sample from each class and all support samples are used as input. This leads to two problems:\na)\tThis procedure seems to be a transductive process since GNN can gather information from multiple query samples instead of just one sample which is the definition of inductive setting. This means it is unfair to some extent to compare the model with former inductive methods. Besides, the proposed method is not comparable with the state-of-the-art transductive methods.\nb)\tI am not sure if such a design can help model learn the prior knowledge that query samples in each iteration have different labels, which can be a kind of information leakage. \nThe authors can explain about the above questions. In my opinion, it would be better if the authors can provide results without GNN as an ablation study to see if the proposed method is truly effective.\n\n[1]  Regularising Knowledge Transfer by Meta Functional Learning. IJCAI 2021\n\n\n", "summary_of_the_paper": "This paper mainly targets few-shot learning. The authors propose to enhance the invariance of features among different configurations. They merge the regularization with GNN architecture to achieve good results on several datasets.", "summary_of_the_review": "This paper provides insightful method and strong results. However I have questions about the implementation for now, which makes the proposed method not reliable enough.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "n-5P49WdPSs", "reviewer": "Reviewer_wbRK", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n+ I think this paper puts forward an interesting question: how does the FSL configuration affect the feature distribution? Studying this question could potentially guide the how to design an effective few-shot learner. \n\n+ This paper provides code, which makes the paper more reproducible.  \n\nCons:\n- I am extremely confused by the motivation/idea proposed in this paper. This paper is not quite easy to follow. There are several modules proposed in this paper but the motivations behind the proposed modules are not clear. Why does this paper split the training set into 4 pieces? Why are 1 & 2 & 3 shots selected?     \n\n- It seems that the main novelty of this paper is to use GNN to refine features and make them invariant to hyper-parameters such as number of shots. However, this has been studied extensively in prior arts. What's the difference between the propose module and the previous GNN based few-shot learners?\n\n- What does the evolution mean in the proposed GNN module? Is it the forward of the GNN module? \n\n- The evaluation to existing methods is not fair. Most of the methods of comparison use images of 84x84 (or 80x80)  as inputs while this paper resizes all images to 224x224. So I am not able to interpret the Table 1&2&3 and to assess whether the proposed method is effective. Can authors provide the results under the same training protocols with other methods? ", "summary_of_the_paper": "This paper studies feature variance in a FSL model. It observes that support set configuration can greatly affect the feature distribution. To that end, this paper proposes modules based on GNN to learn features invariant to hyper-parameters. Specifically, this method subsamples training set into 4 subsets and each subset is encoded by the same backbone network + GNN. Then, feature invariance is enforced across subsets. By doing so, this method learns features that are invariant to both number of shots and the percentage of training samples, which are beneficial to few-shot learning evaluation. ", "summary_of_the_review": "My assessment is based on the three aspects: motivation, novelty, and results. The motivations behind proposed modules in this paper do not make sense to me and the novelty of this paper is limited. Most importantly, the comparison to previous methods is not fair. So I recommend rejection of this paper. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "d3DNZNgdq72", "reviewer": "Reviewer_dXjh", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strenghts:\nThis method simultaneously utilizes four FSL modules with different shot-partition configurations to train a GNN model that perform joint analysis of the support and query samples.\nBesides, they propose a feature invariance loss function which reduce the variance among different configurations efficiently. The experiments show the efficiency of the proposed IGFE.\n\nWeaknesses:\n1. Some definitions are not clearly. Firstly, the two configurations considered in this paper is the shots and the partition, where the partition is the percentage of training samples in the support set, but I do not quite understand the M, the number of sample batches or the selected partition of the support set used for training, which is mentioned in Section 3. Is the M the number of samples in each class that can be used as support set in all episodes (e.g., if M=300 and there are 600 samples in each class, the support set of this class in each episode is selected in these 300 samples)? And whether the query set is selected in these M samples or in all samples of each class? If so, the expression “In total, we have K*M samples per class.” is not right. And the interpretation of M should be clearer.\n2. The efficiency of GNN module should be introduced in experiments. This paper does not show the experimental results with other modules. Therefore, it’s not convictive to show the efficiency of IGFE. What’s the performance if only utilizing the linear function?\n3. The efficiency of the feature invariance loss function is not introduced in experiments. Why it is not efficient to directly measure and optimize the distance between the one-shot and two-shot features?\n4. The GNN module is a transductive method. What’s the performance when employing the inductive setting (Constructing each graph with only one query sample)?\n5. The number of base classes is 64 in mini-ImageNet. Why to set the classification category to 200?\n6. There are some typos. For example, the last sentence in Section 3.1, “which will be explained int the following section”, the “int” should be “in”. And “Φ_A^1(one-shot) and Φ_B^2(one-shot) trained on support subsets S_A and S_B”, the “Φ_B^2” should be “Φ_B^1”.\n", "summary_of_the_paper": "This paper experimentally studied the variation of query set features across different support set configurations and observed that in a successfully learned FSL model, the visual relationship between support and query samples and the learned features of the query samples should remain largely invariant across different configurations of the support set. Motivated by this, this paper proposed an invariance-guided feature evolution (IGFE) method to maintain the invariance across (a) different number of labeled samples per class (called shots) and (b) different percentage of training samples (called partition) in the support set to promote the performance of FSL.", "summary_of_the_review": "The contributions are increamental, and the writing could be improved.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "6aC73Q9Klx", "reviewer": "Reviewer_ZKYs", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength:\n\n- The problem statement of the paper to build a training strategy with an adapted architecture to increase the generalization looks interesting. \n- The paper is well written. \n\n\nWeaknesses:\n\n- Unfortunately, I had some difficulties with the illustration of the problem in the beginning section of section 3 (Method). While understanding the reason for having an invariant model for meta-learning, I am not sure if the paper accurately describes the problem. The obtained tsne results applied on a pertained depends on several *missing factors* such as backbone, loss function, and a number of epochs on the tested dataset miniImagNet. For example, tsne visualizing on an overfitted deep backbone such as the ResNet (trained with many iterations on the dataset) using tSNE would result in discriminative feature embedding for 1-shot if we are visualizing the training samples.  \n\n- Applying graph neural networks at top of ResNet and testing on the small datasets might increase the generalization due to the over-parameterization. Therefore, I recommend the author to discuss the number of parameters in their proposed method and ablate the effect of the applied extra-module. For example, what happens if we train the model with a cross-entropy loss with a hybrid ResNet12-GNN model.\n\n \n- I totally respect the effort of the author by evaluations with miniImageNet and CUB and other small datasets.  However, large few-shot benchmarks are required to measure the effectiveness of the method \n\n- MiniImageNet dataset contains images of size 84x84 which is used in few-shot literature. Could you please justify the reason for using 224x224?  \n", "summary_of_the_paper": "This work first illustrates a problem in episodic pretraining. Particularly, the paper claims that as we increase the number of shots in the episodic pertaining, the feature space becomes more discriminative because low-shot-based training is not invariant to the extracted features of the support set. Then, the paper proposes to use an extra trainable graph neural network module to extract “invariant features”. Finally, the paper evaluates the proposed method using miniImageNet, CUB datasets using ResNet10 and ResNet12.   ", "summary_of_the_review": "I think the illustration of the problem discussed in the paper is not clear, and part of the evaluation is not clear enough (please see above for the full review). ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: Three out of four of the reviewers are leaning (weakly or strongly) towards rejecting this paper. Unfortunately, the authors only responded to the concerns of the most positive reviewer, making it difficult to disregard the concerns from the three more negative reviewers.

I also took a look at the paper myself, and share a number of the reviewers' concerns. First, the proposed method appears to be performing transductive inference for its predictions, while many baselines it compares with rely on inductive inference. Transductive inference is generally known to outperform inductive inference, therefore some of the improvements in accuracy can potentially be accounted to that. The authors did mention in their one author response that they generated results in the inductive setting and still saw an improvement, however the submission was not updated with details around that new experiment, making it hard to rely on it. Second, the paper is using a 224x224 resolution for images, while the original mini-ImageNet benchmark (and the majority of baselines evaluated on it) assume a 84x84 resolution. Here too, using the former resolution is known to outperform the latter. Third, I too found the paper to lack clarity at a number of places in the writing.

I also notice that the final predictions is made following the averaging of features from two models (A and B, as in Eq. 5). This is a form of model ensembling, which generally is a principle know to help improve generalization. It seems appropriate to wonder whether the baselines are worse partly due to not relying on any ensembling at all. 

Finally, I've found a recent method from ICJAI 2021 (Cross-Domain Few-Shot Classification via Adversarial Task Augmentation) which appears to beat the proposed method in the cross-domain setting for the majority of domains. 

Given the above, and the lack of rebuttals to the reviewers with the most concerns, I'm afraid I must recommend this paper be rejected at this time.
Decision: Reject