Alleviating the Sample Selection Bias in Few-shot Learning by Removing Projection to the Centroid
Authors: Jing Xu
Xu Luo
Xinglin Pan
Yanan Li
Wenjie Pei
Zenglin Xu
Venue: NeurIPS 2022 Accept (2022)
Keywords: Few-shot learning
metric learning
image classification

Abstract: Few-shot learning (FSL) targets at generalization of vision models towards unseen tasks without sufficient annotations. Despite the emergence of a number of few-shot learning methods, the sample selection bias problem, i.e., the sensitivity to the limited amount of support data, has not been well understood. In this paper, we find that this problem usually occurs when the positions of support samples are in the vicinity of task centroid—the mean of all class centroids in the task. This motivates us to propose an extremely simple feature transformation to alleviate this problem, dubbed Task Centroid Projection Removing (TCPR). TCPR is applied directly to all image features in a given task, aiming at removing the dimension of features along the direction of the task centroid. While the exact task centoid cannot be accurately obtained from limited data, we estimate it using base features that are each similar to one of the support features. Our method effectively prevents features from being too close to the task centroid. Extensive experiments over ten datasets from different domains show that TCPR can reliably improve classification accuracy across various feature extractors, training algorithms and datasets. The code has been made available at https://github.com/KikimorMay/FSL-TCBR.

Alleviating the Sample Selection Bias in Few-shot Learning by Removing Projection to the Centroid
Jing Xu Xu Luo', Xinglin Pan', Wenjie Pei', Yanan Li* Zenglin Xu+
Harbin Institute of Technology, Shenzhen
Universityy of Electronic Science and Technology of China Penggchnng Laboratory *Zhejiang Lab j mag.xuuummmmmmmlammm..mm com, frank. luox@outlook. com, kp6001680gmai1. com com, ynli 22j..jjun mjuugmail..coo com, zenglin0gmail. com
Abstract
Few-shot learning (FSL) targets at generalization of vision models towards unseen tasks without sufficient annotations. Despite the emergence shot learning methods, the sample selection bias problem, i.c., the sensitivity to the limited amount of support data, has not been well understood. In this paper, we find that this problem usually occurs when the positions of support samples are in the vicinity of task centroid the mean of all class centroids in the task. This motivates to propose an extremely simple feature transformation to alleviate this problem, dubbed Task Centroid Projection Removing (TCPR). TCPR applied directly to all image features in given task, aiming at removing the dimension of features along the direction of the task centroid. While the exact task centoid cannot be accurately obtained from limited data, estimate it using base features that each similar to one of the support features. Our method effectively prevents features from being too close the task centroid. Extensive experiments over ten datasets from different domains show that TCPR can reliably improve classification accuracy across various feature extractors, training algorithms and datasets. The code has been made available at hitps: /etpps//gitthb. Koom//KiiiiooMaa/SLLLLTTT
number of few-
Introduction
Deep networks have made great progress image classification 14 47 31 However, large-scale
annotated datasets are costly and even infeasible to obtain in many real-world applications. To explore the possibility vision models to learn visual concept quickly, few-shot learning (FSL) [10] 41] has raised attention recently. In this problem, vision model, trained needs to recognize novel classes of data (query set) given only a few labeled images (support set).
base dataset with labeled data,
Typical few-shot learning methods usually first learn a good feature extractor from the base dataset. Then in each test-time task, the extracted features of all images the support set are used to construct linear classifier to recognize category of images from the query set 40 g. While good feature extractor may help cluster unseen data, the task distribution shift between training and testing (25 still makes it hard to estimate novel class distribution using a small number of samples from the support set. Thus, the performance strongly correlated with the sample quality of the support data. If the support data are not typical enough represent the whole class, or if there exists outlier data that is not representative, the constructed classifier will be biased, which called the sample selection bias problem [6]. In this paper, we delve into this problem in the context of few-shot learning, and find that the task centroid, defined as the mean of all class centroids in a task, plays
"Corresponiing author
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

before
Cara Texi coanel
Tark
Tak2
Taki
Atturnc:: T6.4%
Accurncy: 48.5%
Adco wil Virres rance
Support Data
Dechion Boundary
Anevoc Drnniceess Comted
(a) Positions of sampled support data
(b) Accuracy with various distances to task centroids
The decision boundary is extremely sensitive to the position of support features in FSL, resulting in high performance fluctuations, especially when they are in the vicinity of the task centroid. [(b] The correlation between the classification accuracy and the average distance of all support data to the task centroid. The accuracy gets lower and the variance gets larger when the support samples are closer to the task centroid. After applying the proposed transformation, the bias is alleviated.
Figure 1: ((a)
an important role. Intuitively speaking, the direction of the task centroid can be regarded as the commonality (or the shared pattern) among all classes in that task. Thus a support feature close to the task centroid contains this phenomenon in Figure I(a) where we consider the 1-shot case-each class contains one support sample. As seen, when these samples are close the task centroid, the constructed classifier is very likely to be biased-a small perturbation to the support sample will lead to a large rotation of the classification boundary.
ambiguous information that may confuse the classifier. We demonstrate
To verify our intuition, we randomly choose two classes "Lion" and "Dalmatian" from the test set of minilmagenet, and randomly sample 10000 binary 1-shot classification tasks using these two classes. In Figure [(b)] we show how accuracy varies with the distance of support data to the task centroid. As expected, small distance of support data the task centroid generally results in low average accuracy (plotted in blue) while producing high variance (plotted in red).
To alleviate this problem, we propose simple transformation features that removes the projection on the direction of the task centroid, named Task Centroid Projection Removing(TCPR) Since the ground-truth task centroid is not available and a small amount of support data is inadequate for obtaining a good approximation, we seek the assistance of the base set based on the observation: features from novel dataset tend to cluster around direction in the feature space [39), and small number of base features close to that direction may be strongly correlated with these features. Inspired by this, we estimate the direction using the most similar top & base samples to the support data. Since our proposed transformation removes the projection to the centroid, the impact of the commonality of classes is mitigated and the feature space becomes more discriminative for these classes. After transformation, the problem is relieved as shown in Figure [(b) The transformation helps the classifier capture the object of interest from the misleading shared patterns (see Figure (
Our method is generic, flexible, and can be used agnostic to the pre-training based FSL methods. Empirical experiments show that the transformation function can consistently and largely boost the classification performance on ten datasets with various gaps between base and novel classes, demonstrating the importance of reducing the widely existing sample selection bias. Our contributions are summarized as follows: 1) To the best of our knowledge, we are the first to unravel the connection of the sample selection bias probelm with the task centroid in few-shot learning which has been ignored so far; 2) To reduce the sample selection bias, we propose simple transformation to remove projection an approximated task centroid estimated with the assistance of related neighbors in the base set, and 3) Comprehensive experiments show the effectiveness of the proposed transformation.
2 Related Work
Most recent few-shot learning approaches can be categorized into three groups: optimization-aast generation-based and metric-based methods. Optimization-ban based methods aim to learn models with good initialization so that the model can quickly adapt novel set with a limited number of gradient descent on a few labeled examples, including MAMI [IO 11, Reptile LFO(36),Met Meta-B McTAL S/ S/T(22), etc. Generation-based methods deal with data deficiency problem by generating

---

pseudo support samples based on extra prior knowledge [45]. Some work utilize the transferable
44 Other methods use high level semantic embedding of classes as extra prior information to generate novel samples, i.e.,
intra-class deformations in base set like SGM [22. VFD 48 and DC
class-level attributes [50,, word2vec 28 52 or unlabeled query data (1]. Metric-based methods attempt to encode images into a well-shaped feature embedding space and perform classification with distance-based classifiers, e.g., MatchingNe([4]), ProtoNet [37] and Relation Network (38]. Some studies reveal that the standard transfer learning paradigm can achieve surprisingly competitive performance in few-shot learning 44 [ 73]. In this paradigm, the models are trained on base set to feature extractors and only the classifiers are learned with novel samples. Based on this observation, several work consider improving the generalization ability of learned image representations on the base set with some auxiliary pretext tasks. Inspired by recently proposed self-supervised learning methods 13 3. some FSL methods propose to use the contrastive learning with some regularization techniques to learn high-quality feature representation, e-g.. S2M2 (27]. Inv-Equ [4), VLCL [23]. PAL [26]. Arl. [52) and POODLE [7]. Those methods demonstrate the significance of powerful feature representations in FSL..
learn
However, the generalization ability will be damaged when there exists a large distribution shift between base and novel classes, leading to bias in the estimation of feature distributions of novel classes (39] Luo et al. [24] manifest that image background is one harmful knowledge that prevent models from accurately capturing feature distributions of novel classes. Luo et al. (25) reveal that channels of features may have different importance in different tasks, and the estimation bias of channel importance reduces the discrimination between novel classes. Some simple transformation functions reshape the skewed distribution of novel features to improve the classification performance. The 12 normalization can remove the negative effect caused by the magnitude of features 4 20 40)
Based on that, SEN (29 with l2 normalization transforms features onto the hypersphere while keeping features repel from all other prototypes. SimpleShot [44] subtracts the mean of whole training set features before l normalization. However, when there exists a large distribution shift between base and novel set, the performance of SimpleShot drops quickly since the mean of all training set features is not always a good information for the various class distributions of novel set. ZN [9] discovers the existence of the hubness problem in FSL and uses ZN normalization to address it. For every component of an individual novel feature, the ZN transformation subtracts the mcan of all components and then divides the standard deviation of all components. DCM [39] calibrates the distribution of novel samples to approach zero centered mean and unit standard deviation using support data to improve the evaluation performance. Compared with ZN and DCM, the motivation of our TCPR is different. We focus on reducing the classification bias when the support samples in the vicinity of local centroid. Besides, both ZN and DCM transform novel features without the help of base set, which may lose some useful information. On the other hand, using few samples in novel classes also results in a bias task sampling. Some studies II 49 39 46 show that modern few-shot learning algorithms are extremely sensitive to the data used for adaptation. However, none of them analyzes the connection of the sampling bias to the effect of task centroid.
3 Methods
3.1 Preliminaries
In few-shot classification, we given base set DD and a novel set D, with disjiont classes C. and C.. i.c., C, n Cr 0. Training takes place on D,. and few-shot evaluation takes place on Dn, Each FSL evaluation episode consists of a N-way K-shot task, which evaluates model's ability to learn new classifier discriminating N novel classes of Dn with only K labeled data available in each class.
The N x K labeled samples comprise the support set S = (zx, v:) N Based on the labeled support set S and the base dataset Dz, our goal is to predict the category of all the unlabeled samples N K
in the query set Q = (2,, yt) >>> XK+1 correctly, where denotes the number of images for each class in the query set. The query set sampled from the same N novel classes as S in each task.
Our work based on the popular pre-training pipeline. The sufficient labeled base samples are used for pre-training. For novel classes, we fix the parameters 0 in feature extractor So unchanged. We learn new classifier fw using the limited samples in the support set S to predict labels for query samples Q. Our approach is of feature-level and is independent of the pre-trained backbones.
3

---

Support Datn (Nove Clases)
Base Neighbors (Top
Task Centrold Approsination
Tisk C Centroid Projection
Clasa
Oue of Dan
Gue lou Cores
Step
Step 2
Step
Class
Base Neighbors
Support Data
Task Centrold
(a) Feature Distribution
(b) The Flowsheet of TCPR
Figure 2: 2(a) The feature distribution of base classes(mindlmagened and novel classes(CUB) show
the novel classes skew to some direction and are more related to limited base samples. ((b)) The overall process of TCPR. Given novel support samples, our TCPR consists of three steps: 1) seeking the most related top k samples in base set; 2) approximating the task centroid of the novel data; 3)
removing the harmful effect of the direction of the task centroid. The visualization is conducted by randomly choosing two classes and projecting all data into 3-dimensional subspace using PCA.
Pre-training Stage: Recently, some studies 440, g have demonstrated that the significance of powerful feature representations with simple feature embedding network fo learned on base classes through a image classification proxy task has good transferability to novel samples. Furthermore, to strengthen the generalization of for the supe-siue learning with other proxy tasks, e.g., colorization and rotation, is used as auxiliary loss in FSL. To show our method is agnostic to the feature extractor, we implement the proposed transformation on different algorithms, including Baseline++ [4]( which trains Sa with vanilla cross-entropy loss); S2M2 [27 (which uses mixup regularization and an auxiliary task predict the rotation angles); Inv-Equ (3 (which uses sixteen
geometric transformations for self-supervised learning and knowledge distillation techniques).
Evaluation stage: For each N N-way K-shot task, we train new linear classifier fw using the support set S with N x K limited labeled samples. Let W = [W], W2, ..., Wx| E RR denotes the learn-able parameters of the classifier, where d is the dimension of the extracted features. We normalize the weight vector for each class c, where ||w=|| = 1, E [1, 2, ..., N]. to eliminate the effect of the magnitude of the feature vector 42). The probability of a sample belonging to class can be formulated as:
P(y clx) e
Σ eT(ww)"²=
where y is scaling hyper-parameter. After training on S, the weights vectors [w], W0, ..., wo] can be seen as prototypes of novel classes and used to predict the samples in the query set Q. The classification result is based on the distance of the query image features to these learned prototypes.
Although the simple pre- pre-training strategy has achieved promising performance with well-designed feature extractors, the ambiguity in classification caused by the few samples in the novel classes is still unsolved. Besides, the ambiguity is further aggravated in the vicinity of the task centroid.
3.2 Task Centroid Projection Removing
In this work, we propose a simple transformation to remove the sample bias exacerbated by the task centroid, named Task Centroid Bias Removing (TCPR). The overview of the proposed TCPR is shown in Figure 2(b) which consists of three steps: 1) given support data S from novel classes, we find the most related top & samples in the base set; 2) using the top A neighbors, we approximate the task centroid vector with the help of the statistics of the base set, and 3) we alleviate the harmful effect of the approximated centroid by removing the component of novel features along the direction.
Seeking the base neighbors. Our work based on the phenomenon that the domain shift between base and novel sets causes the skewness of novel class distribution to some specific directions [39]. As shown in Figure [(a) we visualize the distributions of the base dataset, i.e., miniImagenet (which is coarse-grained dataset with wide range of object categories) and the
Step I:

---

novel dataset, i.c., CUB (which is fine-graing dataset that only contains various bird categories), showing that the novel classes are close to only some specific base classes and skew to the direction. Inspired by the observation, we try to approximate the task centroid by looking back upon similar base samples.
the N-way K-shot task, given novel support normalized features x E S, ||>²|| 1, i E [1, 2, ..... N x K]. extracted by the pre-trained feature extractor fø. we calculate the mean of S by averaging the extracted features all support samples, that is,
Nxk Σ 1=1
(2)
NXK
K
Using the mean of the support features, we seek the top is base samples with the closest angular similarity (evaluated by the cosine distance) between the 2n and x! E Dbase: E [1, ..., Nbase]:
Deasine = [cos("", 2)) E {1,2, Noasel)
Diopk {i]cos(z"; ,²4) lopk (D (D Pensine))
where the notation -os(:;-) denotes the cosine distance between two vectors and (opK(.) is an operator to select the largest & elements in the distance set D Doosine The set Dtopk contains the most related & samples in base classes.
Step 2: Approximating the task centroid. We approximate the task centroid by weighted aggregation of 2 i E Piopk, E {1,2, k} based on the cosine similarity in D Peosine: The weight wy of each z can be constructed as:
cos(I", .r//
(4)
ΣiEDrops cos(i", [""")"))"""
where p is the hyper-parameter controlling the relative weights of the base samples. The approximated task centroid Crask is formulated as:
Ctask = L2(
Σ
wrz,),
(5)
ZvEDtopK
where the notation (2(+) denotes the normalization. The approximation of task centroid by the base neighbors, Crask, is more accurate and stable than the means of support set, .². With only few examples, _ easy to overfit on the support data and far from mirroring the ground truth task centroid of the novel classes. Since the novel classes skew in certain direction toward relevant limited base samples, extrapolating the task centroid from the top k similar samples in base set can be seen as more precise alternative. We can change the hyyer-parameee & to cover the distribution of novel classes with different domain gaps.
Step 3: Removing the projection to the task centroid. To reduce the sampling bias caused by the support samples in the vicinity of the task centroid, we propose to eliminate the effect of the direction of the task centroid by pushing novel features away from it. For all the novel samples in both support and query set, x E U Q, we use a simple transformation which extracts the projection the approximated task centroid Ctask:
22
Clack
ft
cos(uf, Cseek )
22
fn Lz - cos(æ", Ctask Ctast:)
Figure Removing the projection along the Crask- The distribution gap between two classes (ccolored in red) and class 2(colored in blue) becomes larger.
After removing the component of features along the direc- tion of Ctask, the distribution novel classes is pushed away from the approximated centroid, shown in Figure 3
Since the direction of task centroid can be seen as the commonality of classes of each task, it contains the non-discrimintavee information which inhibits the classification results. TCPR projects out the harmful direction and constructs a better latent space.
TThe complexity calculating the similarity matrix O(Niase x d), and selecting the topK feature O (Naas x (k)).)). For empirical verification, the increase of latency is small (+11 ms) compared to the original time of training the classifier (298ms) for one 5-way 1-shot meta meta-testing task. The latency is averaged over tasks.
5

---

I-shot with g-1
I-shot with 2m0.5
Anarage AccraayT
5000
Aocurayjia (2%)
#Numbos of Thcks
Accesces with Ditarce
Numbar Tasks
1
I-shot with
Aouarayy 52%0
S. 3.4 1 2.8.
Acueeu wilh Vani of Bestanoo
Khunbes of TTTT
AccursoyW
Accunsoy
Average Accuracyc32
Accuracios with Varied Distance
Number Tasks
Number Tasks
10-shot with 1-1
I-shot with am3
Serraye Accerned22
5000
6% mommmorr of Ta/35
ENUm0005 10015
Accuracyli g
Accuracyy38.66M (866))
Number
Number Tasks
Average distance with centrodd
Average distance with task centroid
(a) Sampling bias with different number of (b) Sampling bias with different value of a shots
Figure Classification accuracy vs the average distance between two class prototypes to the centroid.
3.3 The Connection of Task Centroid to Ambiguity in FSL
In this subsection, we analyze the classification ambiguity aggravated by the task centroid, and further reveal that it naturally-occurrigg phenomenon in FSL since the limited samples result in poor estimation of the class prototype, and the small distribution margin among novel classes makes the ambiguity more severe. For better understanding, we design a toy simulation of binary classification task. Specifically, let D1 = Ni1, E,). D2 = ((2 Σn) be two Gaussian distributions, where μ = [-a, 0), μ4 [a, 0j and Σ =1=E; Σg The task simply to discriminate between samples from the two classes. The task centroid of the two classes is the origin point (0, 0). use Pearest-Centrond Classifier (37] without any regularization to learn the decision boundaries, which first averages points from each class in the support set to form class prototypes and assigns query points to the class of the nearest prototypes. We randomly sample 10, 000 tasks for each simulation experiment. The connection between classification accuracy and the average distance from two class prototypes to the task centroid is shown in Figure 4 With the decrease of the distance, the average of accuracy results gets lower, while the variance of accuracy results gets larger, as shown in the 1-shot task with a = 1.
Intuitively, the sampling bias in the vicinity of task centroid ties with the number of shots in the FSL task. With the increase of the shots, the variance of the class prototypes will diminish. In K-shot task, the covariance matrix of class prototype is Σ I/K². When the number of shots is large enough, the class prototypes can estimate the class centroid accurately with a low variance and the sampling bias is alleviated. This phenomenon is clearly shown in Figure 4(a) when K = 10. The poor estimation of the class prototypes caused by limited labeled data is one of the reasons why the sampling bias decreases the performance of FSL.
On the other hand, the bias correlates with the discriminability of the two classes' distribution. We vary the value of a from = 0.5 to a = 3 with wide range to simulate distribution with different diseriminability in 1-shot tasks. The two classes are more well- separated with larger a. As shown in Figure [bb). the bias problem begins to emerge with a = 0.5, when there exists large overlap between two classes' distribution. With the increase of a, the bias problem is more severe at first and is relieved later. It almost disappears when = 3, where the inter-class distance is much larger than the intra-class variance. The feature distribution which is suffering from the sampling bias highly corresponding to the sho- learning classification problem. Recent studies have shown that the

---

Table 1: The 5-way 1/5-shot classification accuracies on minImageNet. The denotes we randomly crops 9 patches for testing proposed in DeepEMD [5]].
Methods
Backbones
5-way 1-shot
5-way 5-shot
MAML [0]
ResNet--18
49.61 0.92
65.72 0.77
NatchingNet [41]
ResNet-18
52.91 + 0.88
68.88 = 0.69
ProtoNet [37
ResNet-18
54.16 + 0.82
73.68 = 0.65
DeepEMD [3]
ResNet-12
65.91 = 0.82
82.41 0.56
DeepEMD [5]
ResNet-11
68.77 0.29
84.13 = 0.53
CSEI 19
ResNet-12
0.83
81.93 = 0.36
Arl. [52]
ResNct-12
65.21 + 0.58
80.41 = 0.49
McTAL 2
ResNet-12
66.61 0.28
81.43 0.25
8/7 22
ResNet-12
68.03 + 0.52
82.53 = 0.47
POODLE 17
ResNet-12
67.80
83.50
PAL 26
ResNet-12
69.37 1 0.64
84.40 = 0.44
DC
49
WRN-28
68.57 +: 0.55
82.88 + 0.42
PAL 26
ResNet-12
69.37 : 0.64
84.40 0.44
COSOC
24
ResNet-12
69.28 0.49
85.16 0.42
Baseline++ 4
ResNet-18
54.23 4: 0.46
76.11 = 0.33
Baseline++ + +TPR
ResNet-18
58.67 0.41
77.31 + 0.32
Baseline++ 4
ResNet-12
61.67
79.32 0.34
Baseline++ + TCPR
ResNet-11
64.92 0.40
80.87 1 0.33
Inv Iny-Equ [34]
ResNet-12
65.57
84.43 = 0.28
Inv-Equ+TCPP
ResNet-12
68.92 1..2++0.4 0.44
85.32 + 0.25
S2M2 27
WRN-28
64.63
83.50 0.29
S2M2+TCPR
WRN-28
68.05
84.51 + 0.27
S2M2*
WRN-28
67.19
85.63 = 0.26
S2M2*+TCPR
WRN-28
70.67 1 0.43
86.61 1 0.26
high-quality image representations can be transferred from base to novel set, but there does not exist enough margin among novel classes due to the domain gaps. This leads to another factor causing the classification ambiguity aggravated by the task centroid. Similar phenomenon can be observed among the real data in higher dimensional space, as illustrated Figure [(b)
Experiments
Datasets. We evaluate our approach on four widely-used FSL datasets. The minilmageNet 32 is coarse-grained dataset with distinct classes of animals or objects. It is a subset of ILSVRC-12 B5
including 600 images of each class. Following the previous work (32), the categories are split into 64, 16, 20 classes for training, validation and evaluation respectively. miered is much larger and more challenging dataset. It is made up of 779, 165 images from 608 classes sampled from hierarchical category structure. We adopt 351 classes as base categories, 97 as validation categories and 160 as novel categories as suggested in [33]. The CUB fine- ane-ggrieed dataset consisting of 11, 788 images from 200 bird classes. We spilt the dataset into 100, 50 and 50 categories as base, validation and test categories, following 4.. Meta-dataset a significantly larger-scale dataset which comprised of multiple datasets of diverse data distributions, presenting more realistic tasks.
Implementation Details. We adopt the ResNet-18/12 4 34] and WRN (27 as the feature extractors fe for fair comparison with published results. All the extracted features are normalized using the l2 normalization. We use the cosine classifiers to learn the decision boundaries of the novel set. The p in Equation 4 is set to be 0.5. The number of related base neighbors are selected according to the degree of domain difference between base and novel set. We set a larger l with small task distribution shift, e.g., transferred from mini-train to mini-test, and lower l with larger distribution shift, e.g., transferred from mini-train to Quick Draw. The details of the selection of l: can found in Section 4.3 Hyper-parameter selection. All the ablation studies and analysis experiments are conducted with S2M2 27). We report the mean accuracy as well as the 95% confidence interval on 2000 randomly generated episodes.
7

---

14
Table 2: Results on rieredImageNet.
28
Meihods
5-way
Shhul
5-way
5-shoi
MAML 10
51.67 1811
70.93
0.08
LEO E6
66.33
+ 0.05
81.41 - 0.09
4.4
MeteOpt
0.53
CSFT [9
72.57
+ 0.95
86.72
0.63
DeepEMD
0.67
CAN [15
70.65
+ 0.99
84.08
0.68
before
after
Inv-Equ 24
72.21
0.90
87
0.58
S2M2
74.87
= 0.13
87.61
0.26
S2M2 + TCPR
77.67
1: 0.41
88.89 0.25
Figure 5: The cosine similarities among 20
novel class centers
Table 3: The 5-way
1-shot results on Meta-dataser
Test Set
mini-tost
CUB
Fungi
Omini
Sign
QDraw
Flower
DID
baseline
64.63
47.75
42,36
77.28
53.50
51.60
70.33
50.47
TCPR =
69.52 44.89
53.83
46.28 46.28
80.88
56.65+3.13
57.31 +5.m
15.37 +5.07
34.38
TCPR
66.88
50.48.2.75
43.1540.82
77.89-0.77
20.22
54..003..00
70.33
51.49-1.00
TCPR +
68.06
51.87
51.87
44.38
78.51
54.83
54.62
72.55_2.53
52.50.2.08
SimpleShot [44
67.18 1.88
49.68 11.93
43.79
78.19 9..91
54.04 0o.54
54.50 (2:50
71.68
51.19 0.772
ZN 9
07.05+2.42
48.1540.10
43.2440.5
78.80_1.52
53.9240.42
52.86
70.21-0:2
52.20+1.m
4.1 Experimental Results
Note that any FSL method which follows the pre-training pipeline can be employed as the baseline in our method. To show our method is agnostic to feature extractors, we use the algorithms proposed in Baseline++ 4 S2M2 [27). Inv-Equ 34 to obtain the embedding features as baseline models, without loss of generality. After applied TCPR transformation. we get 3% - 4% improvement on 1-shot task and nearly 1 % on 5-shot task. The improvements are larger in 1-shot task than 5-shot, since the sampling bias is more severe in 1-shot task, as shown in 3.3 Furthermore, we combine them to improve the generalization ability brand-new images. Given original images, the strategy randomly samples 9 patches with different sizes and shapes, followed by resizing these patches to 84 x 84. We achieve remarkable 70.67% and 86.61% accuracies for 1/5-shot tasks showing that the most Kate-of-the-art FSL methods still suffer from the biased problem caused by task centroid and the problem can alleviated by our method. The stable improvements validate the general applicability of TCPR for FSL. We also observe stable improvement on rieredImagenet shown in Table
4.2 Statistical Analysis
The approximation of task centroids. We approximate the task centroid of novel task using four different statistic approximations, the mean of the support and query set æuSq.. the mean the support set z and the weighted summation of base neighbors Crask. We can observe from Table 3 that: (1) The 36 CCCoog is the most accurate approximation of the task centroid, which is not available in inductive few-shot tasks and be seen as the transductive setting. The improvements datasets by a large margin show the existence of the detrimental impact task centroid, and can be (2) Compared with the limited information in zy. seeking the related samples in the base set as assistance, Ctask, can be a better estimation.
all
alleviated by our TCPR transformation;
The comparison with alternative normalization operations. We also compare different trans- formations on Meta-dataset shown in Table 3 based on pre-trained models. The SimpleShot [44] subtracts the mean of whole training set features before l2 normalization. The improvements limited when the train-test gaps are large. The Z-score normalization [9] is applied to each feature vector independently, causing loss of utilizing the task centroid information. Our proposed TCPR achieves better performance with the assistance the approximation of task centroid, which makes it insensitive to the base-novel gap.
The task centroid clusters different novel classes the same direction and hurts the discrim- inability. We visualize the cosine similarities among 20 novel class centers of minilmagenet before and after the proposed transformation, as shown in Figure 5 It is clear that the centers of novel classes are scattered in the same direction (task centroid of the novel set), which forms clusters with

---

Retriever
Lion
Dalmatian
Crab
Scoreboard
Albatross
Warbler
Bobolink
Flycat
Cormorant
after before
Transforred from mini-train mini-lest
Transferred from mini-train CUB
Figure 6: The Grad-Cam class activation maps of query samples.
mini test
CUB Fungi
Sign QDraw
A/Accuracy
Coco
b(Accuracy
A(Accuracy)
DTD
30
Flowers
2.
10
05
500010000
500010000
10
100
500
5c0010000
The Number of
The Number of
Figure 7: The effect of the number of base neighbors different novel sets.
large cosine similarity before transformation. After the transformation removes the projection along the direction, samples of different classes in the feature space are more separable.
The projection on the task centroid vector distracts the neural network from the main objects. Figure/6Jshww the Grad-Cam activation maps of some query samples with and without the TCPR transformation. We observe that our transformation by removing the projection of the task centroid helps the model adjust attention to the objects for classification with only few support images. Before transformation, skewness of features makes the classifier confused by the shared structure and distracts the neural network from the main objects. After removing the projection on task centroid vector of task direction, easier for the classifier to capture the objects of interest.
4.3 Ablation Study
Hyper-parameete selection. Selection of the number is of similar base neighbors plays an essential role in approximating the task centroid of novel classes. We show how hyperparameter k influences the few-shot classification performance on different novel sets with varied domain differences in Figure The optimal value of l is different according to the difference of the base-novel domain. Evidently, datasets that are similar to the train split of minilmagenet, like the test split of minilmagenet and Coco, often result in larger number optimal Datasets with less domain gap require small k. like the QDraw and Omniglot. It is necessary to mention that the DTD is a texture dataset. So. it is hard to find the most related base samples, using the whole training set can be better estimation. The results also manifest importance of good approximation of task centroid since different approximations will cause significant change in accuracy.
Performance with different number of ways and shots. We show the results with different numbers of ways and shots before and after the simple transformation in Table The transformation improvement is more significant when the number of shots is lower since the bias problem is more severe the low-shot situation.
Table 4: The accuracy with wide range of number of ways and shots.
N
N
N
10
N-way K-shot
K=1
K K=5 5
K= 10
K
1
=
K
10
K=1
K
K
before
84.52
94.47
95.95
64.63
83.50
87.89
50.68
73.97
79.15
after
88.39
94.91
96.21
68.05
84.52
88.14
52.80
74.52
79.63

---

Conclusion and Future Work
In this work, we disclose a phenomenon that few-shot classification performance would be very sensitive to the position of support samples if they are in the vicinity of the task centroid, which leads a decrease of accuracy with a large variance in different tasks. A simple feature-level transformation, named TCPR, is proposed to address bias by removing the component of novel features to an this
approximated task centroid direction achieving consistent improvements on Meta-datasets, paper, consider the sampling bias effect of task centroid. In the future, will explore the influence of variance of data. Besides, a self-adaptive selection algorithm of the number of l in base set is worth to explore to improve the generalization of TCPR. Furthermore, plan to apply our method to dense prediction FSL tasks like object detectioon/segmenationn in the near future.
Acknowledgments and Disclosure of Funding
This paper was partially supported National Key Research and Development Program of China (No. 2018AAA01002044, and a key program of fundamental research from Shenzhen Science and Technology Innovation Commission (No. JCYJ202001091 13403826), and the Major Key Project of PCL (No. PCL2021A06), Foundation of Zhejiang Province (No. LQ20F030007), and the Open Research Projects of Zhejiang Lab (NO.2022RCOAB04) and the National Natural Science Foundation of China (No. 62206256).
References
[1] Mayank Agarwal, Mikhail Yurochkin, and Yuckai Sun. On sensitivity of meta-learning to support data. In Advances in Neural Information Processing Systems, 34, pages 20447-20460, 2021.
[2] Sungyong Baik, Janghoon Choi, Heewon Kim, Dohee Cho, Jaesik Min, and Kyoung Mu Lee. Meta-learning with adsktideptive loss function for few-shot learning. ICCV, 2021.
[3] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey E. Hinton. A simple framework for contrastive learning of visual representations. In ICML, volume 119 of Proceedings of Machine Learning Research, pages 1597-1607. PMLR, 2020.
[4] Yu Chen, Yen-Cheng Liu, 7solt Kira, Yu-Chiang Frank Wang, and Jia-Bin Huang. A closer look at few-shot classification. In ICLR (Poster). OpenReview.net, 2019.
[5] Yinbo Chen, Xiaolong Wang, Zhuang Liu, Huijuan Xu, and Trevor Darrell. Meta-baseline: exploring simple meta-learning for few-shot learning. In ICCV, 2021.
[6] Corinna Cortes, Mehryar Mohri, Michael Riley, and Afshin Rostamizadeh. Sample selection hias correction theory. In International conference on algorithmic learning theory, pages 38-53. Springer, 2008
[7] Debasmit Das and C. S. George Lee. A two-stage approach to few-shot learning for image recognition. IEEE Transactions on Image Processing, 2019.
[8] Guneet Singh Dhillon, Pratik Chaudhari, Avinash Ravichandran, and Stefano Soatto. A baseline for few-shot image classification. In ICLR. OpenReview.net, 2020.
[9] Nanyi Fei, Yizhao Gao, Zhiwu Lu, and Tao Xiang. Z-score normalization, hubness, and few- shot learning. In 2021 IEEE/CVF International Conference Computer Vision, ICCV 2021, Montreal, QC. Canada, October 10-17, 2021, pages 142-151. IEEE, 2021.
[10] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adap- tation of deep networks. In ICML, volume 70 of Proceedings of Machine Learning Research, pages 1126-11355 PMLR, 2017.
[11] Chelsea Finn, Kelvin Xu, and Sergey Levine. Probabilistic model-agiostic meta-learning. In NeurIPS, pages 9537-9548, 2018
10

---

[12] Bharath Hariharan and Ross B. Girshick. Low-shot visual recognition by shrinking and hallucinating features. ICCV, pages 3037-3046. IEEE Computer Society, 2017.
[13] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In CVPR, pages Foundation IEEE, 2020.
Computer Vision
[14] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, pages 770-778. IEEE Computer Society, 2016.
[15] Ruibing Hou, Hong Chang, Bingpeng Ma, Shiguang Shan, and Xilin Chen. Cross attention network for few-shot classification. In NeurIPS, pages 4005-4016, 2019.
[16] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, pages 1106-1114 1114,,11,22200 2012.
[17] Duong Le, Khoi D Nguyen, Khoi Nguyen, Quoc-Huy Tran, Rang Nguyen, and Binh-Son Hua. Poodle: Improving few-shot learning via penalizing out-of-distribution samples. NeurIPS, 34, 2021.
[18] Kwonjoon Lee, Subhransu Maji, Avinash Ravichandran, and Stefano Soatto. Meta-learning with differentiable convex optimization. In CVPR, pages 10657-10665. Computer Vision Foundation JEEE, 2019.
[19] Junjie Li, Zilei Wang, and Xiaoming Hu. Learning intact features by erasing-inpainting for few-shot classification. In AAAI, pages 8401-8409. AAAI Press, 2021.
[20] Bin Liu, Yue Cao, Yutong Lin, Qi Li, Zheng Zhang, Mingsheng Long, and Han Hu. Negative margin matters: Understanding margin in few-shot classification. ECCV (4), volume 12349 of Lecture Notes in Computer Science, pages 455. Springer, 2020.
[21] Jinlu Liu, Liang Song, and Yongqiang Qin. Prototype rectification for few-shot learning. In ECCV (I), volume 12346 of Lecture Notes in Computer Science, pages 741-756. Springer, 2020.
[22] Su Lu, Han-Jia Ye, and De-Chuan Zhan. Support-target protocol for meta-learning, NeurIPS, abs/2104.03736, 2021.
[23] Xu Luo, Yuxuan Chen, Liangjian Wen, Lili Pan, and Zenglin Xu. Boosting few-shot classifi- cation with view-lcarnable contrastive learning. In 2021 IEEE International Conference on Multimedia and Expo (ICME), pages 1-6. IEEE, 2021.
[24] Xu Luo, Longhui Wei, Liangjian Wen, Jinrong Yang. Lingxi Xie, Zenglin Xu, and Qi Tian. Rectifying the shortcut learning of background for few-shot learning. Advances in Neural Information Processing Systems, 34:13073-13085 2021.
[25] Xu Luo, Jing Xu, and Zenglin Xu. Channel importance matters in few-shot image classification. In International Conference on Machine Learning, pages 14542-14559, PMLR, 2022.
[26] Jiawei Ma, Hanchen Xie, Guangxing Han, Shih-Fu Chang, Aram Galstyan, and Wael Abd- Almageed. Partner-assisted learning for few-shot image classification. ICCV, abs/2109.07607, 2021.
[27] Puncet Mangla, Mayank Singh, Abhishek Sinha, Nupur Kumari, Vineeth N. Balasubramanian, and Balaji Krishnamurthy, Charting the right manifold: Manifold mixup for few-shot learning. In WACV, pages 2207-2216. IEEE, 2020.
[28) Tomás Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. Distributed representations of words and phrases and their compositionality. In NIPS, pages 3111-3119, 2013.
[29] Van Nhan Nguyen, Sigurd Løkse, Kristoffer Wickstrøm, Michael Kampffmeyer, Davide Roverso, and Robert Jenssen. Sen: A novel feature normalization dissimilarity measure for prototypical few-shot learning networks. In European Conference on Computer Vision, pages 118-134. Springer, 2020.

---

[30] Alex Nichol and John Schulman. Reptile: a scalable metalearning algorithm. arXiv preprint urXiv:1803.02999, 2(3):4, 2018.
[31] Xinglin Pan, Jing Xu, Yu Pan, Liangjian Wen, Wenxiang Lin, Kun Bai, Hongguang Fu, and Zenglin Xu. Afinet: Attentive feature integration networks for image classification. Neural Networks, 155:360-368, 2022.
[32] Sachin Ravi and Hugo Larochelle. Optimization as a model for few-shot learning. In ICLR. OpenReview.net, 2017.
[33] Mengye Ren, Eleni Triantafillou, Sachin Ravi, Jake Snell, Kevin Swersky, Joshua Tenen- baum, Hugo Larochelle, and Richard S. Zemel. Meta-learning for semi-supervised few-shot classification. ICLR (Poster). OpenReview.net, 2018.
[34] Mamshad Nayeem Rizve, Salman Khan, Fahad Shahbaz Khan, and Mubarak Shah. Exploring complementary strengths of invariant and equivariant representations for few-shot learning. In CVPR, 2021.
[35] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael S. Bernstein, Alexander C. Berg, and Fei-Fei Li. Imagenet large scale visual recognition challenge. Int. J. Comput. Vis., 11(33)211-222, 2015.
[36] Andrei A. Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascanu, Simon Osindero, and Raia Hadsell. Meta-learning with latent embedding optimization. In ICLR (Poster). OpenReview.net, 2019.
[37] Jake Snell, Kevin Swersky, and Richard S. Zemel. Prototypical networks for few-shot learning. In NIPS, pages 4077-4087, 2017.
[38] Flood Sung, Yongxin Yang. Li Zhang, Tao Xiang, Philip H. S. Torr, and Timothy M. Hospedales. Learning to compare: Computer Vision Foundation / IEEE Computer Society, 2018.
Relation network for few-shot learning. In CVPR, pages 1199-1208.
[39] Ran Tao, Han Zhang, Yutong Zheng, and Marios Savvides. Powering finetuning for few-shot learning: Domain-agnostic bias reduction with selected sampling. AAAI, 2022.
[40] Yonglong Tian, Yue Wang, Dilip Krishnan, Joshua B. Tenenbaum, and Phillip Isola, Rethinking few-shot image classification: a good embedding is all you need? ECCV, abs/2003. 2020.
[41] Oriol Vinyals, Charles Blundell, Tim Lillicrap, Koray Kavukcuoglu, and Daan Wierstra. Match- ing networks for one shot learning. In NIPS, pages 3630-3638, 2016.
[42] Feng Wang, Xiang Xiang, Jian Cheng, and Alan Loddon Yuille. Normface: L2 hypersphere embedding for face verification. In Proceedings of the 25th ACM international conference on Multimedia, pages 1041-1049, 2017.
[43] Jingquan Wang, Jing Xu, Yu Pan, and Zenglin Xu. Semantically proportional patchmix for few-shot learning. In ICASSP 2022-2022 IEEE International Conference on Acousties, Speech and Signal Processing (ICASSP), pages 1895-1889 IEEE, 2022.
[44] Yan Wang. Wei-Lun Chao, Kilian Q Weinberger, and Laurens van der Maaten. Simpleshot: Re- visiting nearest-neighbor classification for few-shot learning. arXiv preprint arXiv: 19Xv:1111.004633 2019.
[45] Yu-Xiong Wang, Ross B. Girshick, Martial Hebert, and Bharath Hariharan. Low-shot learning from imaginary data. In CVPR, pages 7278-7286 Computer Vision Foundation IEFE Computer Society, 2018.
[46] Jing Xu, Xinglin Pan, Wenjie Pei, and Zenglin Xu. Exploring category-correlated feature for few-shot image classification. CoRR, abs/2112.07224, 2021.
12

---

[47] Jing Xu, Yu Pan, Xinglin Pan, Steven Hoi, Zhang Yi, and Zenglin Xu. Regnet: self-regulated network for image classification. IEEE Transactions on Neural Networks and Learning Systems, 2022.
[48] Jingyi Xu, Mingzhen Huang, ShahRukh Athar, and Dimitris Samaras. Variational transfer learning for finegrrainet few-shot visual recognition. ICCV, abs/2010.03255, 2021.
[49] Shuo Yang, Lu Liu, and Min Xu. Free lunch for few-shot learning: Distribution calibration. In ICLR. ICLR, 2021.
[50] Baoquan Zhang, Xutao Li, Yunming Ye, Zhichao Huang, and Lisai Zhang. Prototype completion with primitive knowledge for few-shot learning. In CVPR, pages 3754-3762. Computer Vision Foundation / IEEE, 2021.
[51] Chi Zhang, Yujun Cai, Guosheng Lin, and Chunhua Shen. Deepemd: Few-shot image classi- fication with differentiable earth mover's distance and structured classifiers. In CVPR, pages 12200-12210. Computer Vision Foundation / IEEE, 2020.
[52] Hongguang Zhang, Piotr Koniusz, Songlei Jian, Hongdong Li, and Philip H. S. Torr. Rethinking class relations: Absolute-relative supervised and unsupervised few-shot learning. In CVPR, pages 9432-9441. Computer Vision Foundation IEEE, 2021.
Checklist
The checklist follows the references. Please read the checklist guidelines carefully for information on how to answer these questions. For each question, change the default [TODO] to [Yes], [No], or [N/A] . You are strongly encouraged to include justification to your answer, either by referencing the appropriate section of your paper or providing brief inline description. For example:
Did you include the license to the code and datasets? [Yes] See Section ??.
Did you include the license the code and datasets? [No] The code and the data are proprictary.
Did you include the license to the code and datasets? [N/A]
Please do not modify the questions and only use the provided macros for your answers. Note that the
Checklist section does not count towards the page limit. In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answees below.
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type
of GPUs, internal cluster, cloud provider)? [Yes]
13

---

4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [N/A]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "uXdbV9KD6id", "reviewer": "Reviewer_i4sq", "summary": "This paper proposes a novel few-shot learning method of task centroid projection removing (TCPR). The proposed TCPR aims to resolve one of the limitations of few-shot learning, called the sampling bias problem. By adjusting the classification boundary via moving the data centroids by removing some components of features, TCPR can improve the few-shot classification accuracies for various datasets, feature extractors, and base few-shot classification algorithms.    \n\n-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\nAfter Rebuttal: I appreciate the authors for kindly addressing my concerns and questions. Most of my concerns are resolved. As a result, I keep my original score.", "questions": "1. Related to weakness 2, can we utilize the base class centroids instead of the base class data? Since there exist many data for the base classes, we can easily compute the true class centroids for the base classes. Isn’t it more realistic to store the base class centroids, not the entire base class data? How does this affect performance?\n2. The process of removing the projection to the task centroid seems to make the features orthogonal to the task centroid and remove the effect of the dimension of task-centroid. Is there any other way to remove the dimension of task-centroid? For example, [1] utilizes the linear nulling of the vector to remove the effect of specific dimensions. \n\n[1] Yoon et al. TapNet: Neural Network Augmented with Task-Adaptive Projection for Few-Shot Learning, ICML 2019.\n", "limitations": "1. Sampling process for computing task centroid may be computationally inefficient.\n2. To obtain the gain of TCPR, we need to store some extra base data, which can be unreal condition. ", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\n1. Idea of approximating task centroid with the help of base class data is interesting and powerful.  \n2. The concept of utilizing the task centroid to resolve sampling bias problem is intuitive and powerful, and it can be easiliy applied to various FSL techniques. Experimental results show that the proposed TCPR is able to improve the performances of various few-shot learning methods.\n3. Analysis results on sampling bias problem would be helpful for further research on sampling bias problem. \n\nWeakness\n\n1. It seems the process of sampling the most related samples from the base class samples seems computationally inefficient. Since there are many base class data and N_base is large, it seems that computing cosine similarity and choosing top-K samples from base data require many computations. \n2. The task centroid has to be computed anew for the given task, and we need to keep the entire base data, not just the trained model. For me, it seems that keeping all the base class samples for the approximation of task centroid seems somewhat unreal. \n3. I am not sure whether the comparing TCPR-extended few-shot learning with the base few-shot learning model. In Figure 7, it seems that TCPR requires the large k to obtain meaningful gain, which means TCPR utilizes many extra data, although the extra data come from the base classes. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "o-lKcYAsJbQ", "reviewer": "Reviewer_5bg4", "summary": "This paper proposes a post-processing technique to improve the quality of prototypes used in few-shot learning (FSL). The main idea is to remove a bias, by first identifying a large set of neighbors from the base samples (training data used in designing the model) and performing a weighted averaging to identify the origin of decision space (my interpretation). Once the center is identified, the support and query samples are normalized according to the center (by first a weighted bias removal and then normalization). Experiments on various datasets show that the method helps in having a less biased decision space, hence improving the results. ", "questions": "As for questions, I would like to ask\n\n1- Can the method be used to address dense prediction FSL? if yes, how?\n\n2- I am not convinced that the change of origin is the source of bias. Somehow I feel that the algorithm tries to use the base data to construct a better latent space. A natural question to ask is, can we consider the PCA space from the base samples as a way of denoising the decision space? That is, aside from the task-center, what will happen if the PCA space is also taken into account. Can you kindly comment on this?", "limitations": "It is not discussed but the method extensively uses training data towards its decisions so could potentially generate unfair and biased outputs", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n-------------\n\n- the paper is written very clearly.\n- focusing on classification, the authors did a comprehensive study and contrast their algorithm on a broad set of experiments, including meta-dataset.\n\nWeaknesses\n-------------\n\n- despite the fact that the authors explained the motivation behind the work (closeness to a center), I have a lot of difficulties convincing myself that the so-called task centroid is the source of the problem. Overall, I think one can scale samples, and distances to the task-center cannot be the only reason behind improvements here. I do understand that the solution works on the hyper-sphere so do not crusade me but my point is, the relative closeness cannot be the sole reason behind the improvement obtained in this work.\n\n- the method is quite hefty in compute as it requires identifying k-neighbors of support samples, where k in practice can be in the range 10,000. As a matter of fact, Fig.7 suggests the algorithm underperforms on mini-imagenet if k <100.\n\n- the problem studied is solely classification. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "eKqlzOLjrBp", "reviewer": "Reviewer_ZuAk", "summary": "This paper studies the sampling bias issue in few-shot learning and identifies its connection to task centroids. To mitigate the sampling bias, this paper proposes to perform Task Centroid Projection Removing (TCPR) before training the classifier for the test task. When performing TCPR, it approximates the task centroid by using top K data samples in base classes. Experimental results show the proposed method improves the classification accuracy for different pre-trained feature extractors.", "questions": "1. The concern about the large value of k. Could the authors provide experiment results about the running time cost of obtaining top k base samples and training the classifier?\n2. The toy case in Sec 3.3. TCPR cannot be applied in this case. I think it would be better to project the data into a 3-d hypersphere and perform TCPR to see how TCPR helps to improve the performance. I think TCPR would work for some cases and fail for other cases. \n3. TCPR is somehow similar to DC[43] as both methods try to improve the performance by using the information of base samples. I think it would be better to provide more discussion about DC in related works and provide more experiment results of DC on the same pre-trained feature extractors as TCPR, e.g. Baseline++, Inv-Equ, S2M2, S2M2*. Besides, is it possible to retrieve top k base classes prototypes (like DC) instead of top k base samples, so that not all base class samples are required?\n4. About Experiment in Table 3: It would be desirable to provide the distance between the approximation of the different task centroids and the real task centroid. In addition, the results of the 5-shot experiments should be provided in Table 3 for a better understanding of the improvement of c_task on the 5-shot experiments.", "limitations": "The paper discusses the limitation about selection of k. ", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Originality: \nThe studies about the connection between sampling bias and task centroid and the proposed TCPR method are novel.\n\t\nQuality:\nThe proposed TCPR is technically sound. \nThe paper tries to support the claim about the connection between sampling bias and task centroid through toy cases and experiments. But the toy case in Sec 3.3 is not a good toy case, because it does not match with the cases where TCPR is applicable.\nAnother issue is computation cost. The value of k is very large. The time cost to get top k base samples may be high. Moreover, this method requires access to all base samples, which may be not practical.\n\t\n\nSignificance: \nThe sampling bias is an important issue in few-shot learning. This paper studies the connection between sampling bias and task centroid and proposes TCPR to mitigate it. Applying TCPR on various pre-trained feature extractors, significant performance improvements are achieved.\n\nClarity:\nThe paper is not difficult to understand. It clearly presents the motivation and the method and provides sufficient technique and experiment details. But the writing can be further improved. \n\n----\nAfter rebuttal:\nThe response from the authors addresses my questions. I still hold some minor concerns about the failure of TCPR on the toy case, as the authors did not provide a revised version of the paper, and also about the large value of k.\nI keep my score weak accept. \n\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Zc6Aj2Ke-VM", "reviewer": "Reviewer_HDBL", "summary": "The paper conducts experiments which show that classification boundary in a few-shot setting is very sensitive to the position of support samples (few examples seen during evaluation) if they are in the vicinity of the data centroid (referred to as task centroid). Hence, a feature transformation named Task Centroid Projection Removing (TCPR) is proposed to remove the component of features along the direction of approximated task centroid. Experimental evaluation is conducted on several datasets.", "questions": "Is the \"Removing the projection to the task centroid\" similar to VLAD/Fisher Vector encodings which have previously been used in image classification? Is it possible to distinguish with these methods?\n\n[1] https://openaccess.thecvf.com/content_cvpr_2015/papers/Xu_A_Discriminative_CNN_2015_CVPR_paper.pdf This was used for few shot event recognition.\n\nsentence correction:\n\n99: DCM [36] using support data to improve the evaluation performance calibrates the distribution of novel samples to approach zero centered mean and unit standard deviation . \n\nusing support data to improve the evaluation performance calibrates the distribution  -> uses support data to improve the performance by calibrating the distribution\n\n101: our method has different motivation to reduce the classification bias when the support samples in the vicinity of local centroid.\n\nNot sure what this means", "limitations": "yes, they are addressed", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nThe paper motivates the problem well with the “Lion” and “Dalmatian” example which highlights the problem with existing techniques. Later on, the paper addresses this problem with a simple solution by picking nearest samples from the base class and then picking a vector which points in the direction along the k nearest points in the base class, along a direction away from the class mean. Extensive experiments show the effectiveness of the method.\n\nWeakness:\nSome of the terminology being used in the paper is a bit non-intuitive to understand and this being coupled with incorrect sentences makes the paper hard to read.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "GA4PgjiM0m", "reviewer": "Reviewer_XMzd", "summary": "This paper presents a new technique, Task Centroid Projection Removing (TCPR) for few-shot classification. The paper highlights the phenomenon of the distance of samples from a task centroid affecting task performance. The paper then builds on this insight to create the TCPR technique. ", "questions": "Why is cosine distance used for all similarity measurements? Could any distance metric be used?\n\nOn line 233, what are ‘cosine classifiers’? Could you give a more thorough explanation of how the final classification was done for the query set.", "limitations": "The authors have addressed the limitations of their work.", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "___Strengths___: The paper is attacking a significant problem in machine learning and is presenting a novel technique to address that significant problem. Learning with less labels, which includes few-shot learning, is one of the greatest problems hindering machine learning. So, this paper is directly addressing a very significant problem. The technique proposed as well as the insight about support samples’ positions in the embedded space is rather novel and straightforward. The idea of pre-training bias affecting subsequent few-shot classification is novel and also reflects some of the known wisdom regarding domain adaptation in machine learning. The use of base set in constructing the task centroid is a clever and straightforward means of controlling the variability that can arise in sampling for few-shot classification. \n\nThe paper also does a thorough empirical evaluation, including an ablation study with different models and hyperparameters. The thoroughness of this study combined with the released code greatly inspires the reader to try the proposed technique on their own work.\n\n___Weaknesses___: The paper has a few weaknesses. First, there are some minor clarity issues. The placement of figures is wonky in that Table 3 comes before Table 2. And, while this is an acknowledged limitation, the setting of $k$ for the number of nearest neighbors really needs more guidelines for practical use. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "4zmMtNSeeH", "reviewer": "Reviewer_mLQK", "summary": "This paper claims to be the first to investigate bias of the supports that causes low accuracy in few-shot learning. Specifically, when the supports are too close to the centroid of a task, the quality of the decision boundaries would be unstable and hence a lower accuracy. They propose a framework to remove such bias by firstly estimating the centroid with base and support features from a pre-trained feature extractor and removing the components that lie in the same direction as the centroid, pushing away the novel features, and thus mitigating the bias. With the experiments on several benchmarks, they showed the effectiveness of the method.", "questions": "1.\tThe assumption is novel features tend to mingle with base features, and one could find K nearest of them for task centroid estimation. Any guarantees on this? What if novel features lie in a sparse region where the closest base features are far? Would it cause instability for centroid estimation? More exploration and evidence on this would be greatly appreciated and would bolster the work.\n\n2.\tThe simulation experiments are in 2-d, difference in high dimensional space?\n\n3.\tI would imagine in some cases, removing the projection to the centroid could potentially cause performance degradation? For instance, what if supports from two classes lie in the upper sphere while supports for the other two lie in the other sphere? Then, in this case, initially we could use the direction along the centroid for differentiation, and yet after projection, we might not be able to?\n", "limitations": "No discussion on limitations. ", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\n1.\tThe idea is intuitive and easy to understand and follow, which they also show the effectiveness by applying the method on different pre-trained few-shot models.\n2.\tThey showed SoTA performance on different benchmarks.\n3.\tA set of ablation studies are performed on the design choices.\n\nWeaknesses\n\n1.\tNo analysis on failure cases, if any. Discussion on limitations is limited. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept