DiffKendall: A Novel Approach for Few-Shot Learning with Differentiable Kendall's Rank Correlation
Authors: Kaipeng Zheng
Huishuai Zhang
Weiran Huang
Venue: NeurIPS 2023 poster (2023)
Keywords: Few-shot learning

Abstract: Few-shot learning aims to adapt models trained on the base dataset to novel tasks where the categories were not seen by the model before. This often leads to a relatively concentrated distribution of feature values across channels on novel classes, posing challenges in determining channel importance for novel tasks. Standard few-shot learning methods employ geometric similarity metrics such as cosine similarity and negative Euclidean distance to gauge the semantic relatedness between two features. However, features with high geometric similarities may carry distinct semantics, especially in the context of few-shot learning. In this paper, we demonstrate that the importance ranking of feature channels is a more reliable indicator for few-shot learning than geometric similarity metrics. We observe that replacing the geometric similarity metric with Kendall’s rank correlation only during inference is able to improve the performance of few-shot learning across a wide range of methods and datasets with different domains. Furthermore, we propose a carefully designed differentiable loss for meta-training to address the non-differentiability issue of Kendall’s rank correlation. By replacing geometric similarity with differentiable Kendall’s rank correlation, our method can integrate with numerous existing few-shot approaches and is ready for integrating with future state-of-the-art methods that rely on geometric similarity metrics. Extensive experiments validate the efficacy of the rank-correlation-based approach, showcasing a significant improvement in few-shot learning.

DiffKendall: A Novel Approach for Few-Shot Learning with Differentiable Kendall's Rank Correlation
Kaipeng Zheng'
Huishuai Zhang
Weiran Huang lff
1 Qing Yuan Research Institute, SEIEE, Shanghai Jiao Tong University
Microsoft Research Asia
kaipengm20gmail. com,
huzhang@miccoosoft. com, weiran. huang@outlook.com
Abstract
Few-shot learning aims to adapt models trained on the base dataset to novel tasks where the categories were not seen by the model before. This often leads to a rela- tively concentrated distribution of feature values across channels on novel classes, posing challenges in determining channel importance for novel tasks. Standard few-shot learning methods employ geometric similarity metrics such as cosine similarity and negative Euclidean distance gauge the semantic relatedness be- tween two features. However, features with high geometric similarities may carry distinct semanties, especially in the context of few-shot learning. In this paper, we demonstrate that the importance ranking of feature channels a more reliable indicator for few-shot learning than geometric similarity metries. We observe that replacing the geometric similarity metric with Kendall's rank correlation only during inference able to improve the performance of few-shot learning across a wide range of methods and datasets with different domains. Furthermore, we propose a carefully designed differentiable loss for meta vetta-trining to address the non non-differeniiabiiiy issue of Kendall's 's rank correlation. By replacing geometric similarity with differentiable Kendall's rank correlation, our method can integrate with numerous existing few-shot approaches and is ready for integrating with future state-of-the-ar art methods that rely on geometric similarity metrics. Extensive exper- iments validate the efficacy of the mone-corelation-baae based approach, showcasing a significant improvement in few-shot learning.
1 Introduction
Deep learning has achieved remarkable success in various domains. However, obtaining an adequate amount of labeled data is essential for attaining good performance. In many real-world scenarios, obtaining sufficient labeled data can be exceedingly challenging and laborious. This makes it major bottleneck in applying deep learning models in real- world applications. In contrast, humans are able to quickly adapt to novel tasks by leveraging prior knowledge and experience, with only a very small number of samples. As result, few-shot learning has received increasing attention recently.
The goal of few-shot learning to adapt models trained on the base dataset to novel tasks where only few labeled data are available. Previous works mainly focus classification tasks and have been extensively devoted to metric-learning-aasee methods [1, 2, 3, 4]. In particular, they learn enhanced embeddings of novel samples by sampling tasks with similar structure to the novel task from sufficiently labeled base dataset for training. Geometric similarity metrics, such as negative Euclidean distance [1] and cosine similarity [4, 5. 6, 7], commonly utilized to determine the semantic
I "Correspondence Weiran Huang.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

base class
novel class
0.34
Density
0.32
0.30
69
1.0 1 2.0
2.5
19020001 test DVK DKOM
kungi
Hove
CUR
Channel value
(a)
(b)
Figure (a) Distribution of feature channel values for a base class and a novel class on mini- ImageNet. The novel class is previously unknown for the model, where clear difference can be observed from the base class. (b) Comparison of the Mcan Variance Feature Channel Values between the base dataset (mini-train) and various distinct new datasets. It is observed that the variance of feature channel values on the new datasets is consistently much lower than that on the base dataset.
similarities between feature embeddings. Recent studies [8, 9, 10] also show that simply employing cosine similarity instead of inner product in linear classifier also yields competitive performance of novel tasks, when the model is pre-trained on the base dataset.
Since the categories of the base class and novel class are disjoint in few-shot learning, there exists a significant gap between training and inference for the model. classes, when the feature extractor faces a novel class that is unseen before, the feature channel values become more concentrated, i.c., for a novel class, most non-core features' channels have small and closely clustered values in the range [0.1, 0.3] (see Figure I(a)). This phenomenon occurs because the model is trained on the base data, and consequently exhibits reduced variation feature values when dealing with novel data. Empirically, channel values between the base dataset and various novel datasets (see Figure 1(b).. The results reveal significantly smaller variance in feature channel values in the novel datasets compared the base dataset. A smaller variance means values are closer to each other, demonstrating that this is universally valid conclusion. This situation creates challenge in employing geometric similarity to accurately distinguish the importance among non-core feature channels. To provide a concrete example, consider distinguishing between dogs and wolves. While they share nearly identical core visual features, minor features play vital role in differentiating them. Suppose the core feature, two minor features are represented by channels 2, and 3, respectively, in the feature vector. A dog prototype may have feature (1, 0.28, 0.2), and wolf prototype may have feature (1, 0.25, 0.28). Now, for a test image with feature (0.8, 0.27, 0.22), it appears more dog-like, as the 2nd feature more prominent than the 3rd. However, cosine distance struggles to distinguish them clearly, misleadingly placing this test image closer to the wolf prototype (distance=0.003) rather than the dog prototype (distance=0. 0.044) Contrastingly, the importance ranking of feature channels is able to distinguish dogs and wolves. The test image shares same channel ranking (1, 2, 3) as the dog prototype, whereas the wolf prototype's is channel ranking is (1, 3, 2).
found that compared to base
additionally compare the variance of feature
Motivated by the above observations, aim to boost the few-shot learning based on the importance ranking of feature channels in this paper. Specifically, we propose a simple and effective method, which replaces the commonly-used geometric similarity metric (c.g.. cosine similarity) with Kendall's rank correlation to determine how closely two feature embeddings are semantically related. We demonstrate that using Kendall's rank correlation at test time can lead to performance improvements on wide range of datasets with different domains. Furthermore, we investigate the potential benefits of employing Kendall's rank correlation episodic training. One main challenge that the calculation of channel importance ranking is non-differentiable, which prevents from directly optimizing Kendall's rank correlation for training. address this issue, we propose a smooth approximation of Kendall's 's rank correlation and hence make it differentiable. We verify that using

---

the proposed differentiable Kendall's 's rank correlation at the meta- meta-training stage instead of geometric similarity achieves further performance improvements.
In summary, our contributions to this paper are threefold as follows: 1) We reveal an intrinsic property of novel sample features in few-shot learning, whereby the vast majority of feature values are closely distributed across channels, leading to difficulty in distinguishing their importance; 2) We demonstrate that the importance ranking of feature channels can be used as a better indicator of semantic correlation in few-shot learning. By replacing the geometric similarity metric with Kendall's 's rank correlation at test time, significant improvements can be observed in multiple few-shot learning methods on wide range of popular benchmarks with different domains; 3) We propose a differentiable loss function by approximating Kendall's rank correlation with a smooth version. This enables Kendall's 's rank correlation to be directly optimized. We verify that further improvements can be achieved by using the proposed differentiable Kendall's s rank correlation the meta-training stage instead of geometric similarity.
2 Related Works
Meta-Leearning-Basee Basee Few-Shot Learning. Previous research on few-shot learning has been extensively devoted to meta-learming- based methods. They can be further divided into optimization- based methods [11, 12, 13] and metric learning-basee -baee methods [1, 2, 3, 4, 14, 15, 16], with metric learning-baas based methods accounting for the majority of them. ProtoNets [1] proposes to compute the Euclidean distance between the embedding of query samples and the prototypes on the support set for nearest-neighhor classification. Meta-baseline [4] uses cosine similarity instead and proposes two-stage training paradigm consisting of pre-training and episodic training, achieving competitive performance compared to ofate-of-the-art the-art methods. Recent research [3, 17] has begun to focus aligning support and query samples by exploiting the local information on feature maps. Li et al. [17] represent support samples with set of local features and to classify query samples. CAN [3] uses cross-attention to compute the similarity score between the feature maps of support samples and query samples. ConstellationNet [6] proposes learn enhanced local features by constellation models and then aggregate the features through self-attention blocks. Several studies [7. 18] investigate methods for calibrating novel sample features in few-shot learning. Xue and Wang [18] learn transformation that moves the embedding of novel samples towards the class prototype by episodic training. Zhang et al. [7] propose to use additional attribute annotations that are generalizable across classes to complete the class prototypes. However, this requires additional labeling costs and the common properties will not exist when the categories of base class data and novel class data are significantly different. Unlike previous studies, we explore determining semantic similarities between novel sample features with the correlation of channel importance ranking in few-shot learning, which has never been studied before.
E-nearest-neighbor classifier
Hansfer-Learning Based Few-Shot Learning. Transfer-learning- based few-shot learning methods have recently received increasingly widespread attention. Prior research [8, 9, 10] has verified that competitive performance in few-shot learning can be achieved by pre-training models on the base dataset with cross-entropy loss and using cosine similarity for classification on novel tasks, without relying on elaborate meta-learning frameworks, Recent studies [19, 20, 21] also focus on introducing data from the pre-training phase to assist in test-time fine-tuning. For instance, Afrasiyabi et al. [19] propose to introduce samples from the base dataset in fine-tuning that are similar to the novel samples in the feature space. POODLE [21], on the other hand, uses the samples in the base dataset as negative samples and proposes to pull the embedding of novel samples away from them during fine-tuning.
3 Problem Definition
Typically, a few-shot task T consists of a support set S and a query set Q. The objective of the few-shot task is to accurately predict the category for each query sample I; E Q based on the support set The support set S is commonly organized as N-way K-shot, which means that there are a total of N categories of samples included in this task, with each class containing K annotated samples. In few-shot scenarios, K is usually a very small number, indicating that the number of samples available for each category is extremely small.
3

---

It is infeasible to train a feature extractor fo on few-shot task directly from scratch. Thus, the feature extractor fo usually trained on a base dataset Dasse with sufficient annotation to learn a prior. The training process typically involves pre-training and episodic training. After that, few-shot tasks are sampled on the novel dataset Doovel for performance evaluation. The categories of the samples in Dasse are entirely distinct from those in novel- The trained feature extractor fe is then used to obtain the embedding of the samples for novel tasks, followed by calculating the similarity between the embedding of support samples and query samples for nearest-neighhor classification, namely,
(xp(sim((f((z)).))))
P(y = le x) = Σラ- exp(sim(fola).c)))
where x E Q denotes the query sample, C₁ represents the class prototype, which is usually represented by the mean feature of the support set samples in each category, sim(-) denotes a similarity measure, N denotes the total number of classes included in the few-shot task, and is used to perform a scaling transformation on the similarities.
In supervised learning, since there is distribution gap between the training and test data, we can obtain high-quality embedding of test data by training the model with a large number of in-domain samples. However, in few-shot scenarios, the categories in the training data and the test data have no overlap. As there is no direct optimization towards the novel category, the features of novel samples in few-shot learning can be distinct from those learned through conventional supervised learning [7, 18, 22].
Geometric similarities are commonly used to determine semantic similarities between features in few-shot learning, among which cosine similarity is widely employed in recent studies [4, 6, 8). Given two vectors with the same dimension x (fi,..... Fin.) and y (V1, ..., Un), denoting the features of novel samples, cosine similarity is calculated as | 2 I.Ui-- It can be seen that the importance of each channel is directly correlated with the numerical feature value xy and y. However, as shown in Figure ((a), the feature distribution of the novel class samples are largely different from that of the base class samples. The values of the feature channels are highly clustered around very small magnitudes, making difficult for cosine similarity to differentiate their importance in classification. Consequently, the classification performance will be dominated by those very few channels with large magnitude values, while the small-valued channels, which occupy the majority of the features, will be underutilized. Although the embedding has already been projected onto the unit sphere in cosine similarity to reduce this effect, we verify that the role of the small-valued channels in classification is still largely underestimated.
4 Warm-Up: Using Kendall's Rank Correlation During Inference
In this paper, for the first time, we explore the utilization of channel importance ranking in few-shot learning. Converting numerical differences into ranking differences enables effective discrimination between small-valued channels that exhibit similar values, and reduces the large numerical differences between large-valued and small-valued channels. To achieve this, we start with investigating the use of Kendall's 's rank correlation in few-shot learning. Kendall's rank correlation gauges the semantic similarity between features by assessing how consistently channels are ranked, which aligns precisely with our motivation. In this section, we demonstrate that leveraging Kendall's rank correlation simply during the inference stage can lead to a significant performance improvement.
4.1 Kendall's Rank Correlation
Given two n-dimensional feature vectors x (#11..., y (V1...,, Vn), Kendall's rank correla- tion is determined by measuring the consistency of pairwise rankings for every channel pair (I, xz) and (y,, y.). This coefficient can be defined as the disparity between the number of channel pairs (z,, 2z) and (ye, yj) that exhibit concordant ordering versus discordant ordering, namely,
T(x y) Neo Nai
Nival
where Nam represents the count of channel pairs with consistent importance ranking, i.e., either (xr > æz)) A (3: Uj} or (x; < x.) A (ye < 1j), Na, reflects the count of channel pairs with inconsistent ordering represented by either (ar, > xz)) A (y < yj) or (a, < x;)) A (2r > y5)- Nuvill represents the total number of channel pairs.

---

Table I: Performance improvements by using Kendall's rank correlation at test time. The training set of mini-Iman ImageNet used the base dataset and the average accuracy (%) of randomly sampled 5-way 1 1-shot tasks on test sets with different domains is reported.
Method
Backbone
mini-test
CUB
Traffic Signs
VGG Flowers
Quick Draw
Fungi
CF + cosine
Conv-4
48.57
36.97
38.89
59.92
45.75
35.99
CE CIM
Conv-4
48.94
37.41
39.35
59.79
45.56
35.89
Kendall
Conv-4
51.50
39.01
39.04
61.55
46.00
36.73
CE + cosine
RosNet-12
62.2
45.12
55.54
69.41
53.73
40.68
CE + CIM
ResNet-12
60.4
45.20
56.64
69.61
55.14
40.34
Kendall
ResNet-12
47.07
60.84
71.38
55.99
41.68
Meta-B cosine
ResNet-12
62.84
45.38
54.88
69.14
53.27
40.57
CIM CTM
ResNer-12
61.60
45.24
55.31
68.87
54.08
40.41
Meta-B Kendall
ResNet-12
63.36
47.15
59.70
70.57
55.78
41.70
CE + cosine
ResNet-18
62.92
43.7
47.17
62.35
52.33
38.89
CE + CIM
ResNet-18
61.91
43.75
47.23
61.89
52.21
39.07
CE Kendall
ResNet-18
62.83
45.54
54.32
67.08
54.15
39.64
CE + cosine
WRN-28-10
60.08
43.64
47.01
66.03
47.99
39.27
CE + CIM
WRN-28-10
59.34
43.43
46.30
64.42
48.37
39.42
CB + Kendall
WRN-28-10
61.68
45.80
51.39
69.72
53.52
41.52
S2M2 + cosine
WRN-28-10
64.52
47.44 47.44
$2.30
68.93
51.41
41,76
S2M2 + CIM
WRN-28-10
63.60
47.59
53.84
70.91
53.89
42.54
S2M2 Kendall
WRN-28-10
63.97
47.74
57.88
71.48
54.63
43.49
Avg Improvements (Kendall vs. cosine)
0.92 f
1.69
4.56 f
2.67
2.60
1.27 1
Avg Improvements (Kendall vs. CIM)
1.81 f
1.63
4.13
2.88
1.80
1.18
4.2 Performace Improvements by Using Kendall's Rank Correlation at Test Time
We conduct comprehensive experiments and verify that directly using Kendall's rank correlation at test time can significantly improve performance in few-shot learning.
Specifically, recent studies [8, 9, 10] have confirmed that pre-training the model the base dataset with cross-entropy loss, and utilizing cosine similarity for classification on novel tasks, can yield competitive performance. This approach has proven to outperform number of meta-learnin- based methods. Therefore, our initial comparison involves assessing the performance of cosine similarity and Kendall's s rank correlation when the model is pre-trained with cross-entropy loss (CE) on the base dataset. The evaluation is conducted on different backbone networks that are commonly used in previous studies, including Conv-4, ResNet- 12, ResNet-18, and WRN-28-10 10. addition, the comparison is also made based on Deta-Baseline (Meta-B) [4], representative meta-learning based approach in few-shot learning, and an advanced method S2M2 [5]. We keep all other settings unchanged and replace the originally utilized cosine similarity with Kendall's rank correlation solely during the testing phase. Furthermore, we expand our comparison to include recently proposed method (CIM) [22], which suggests a test-time transformation for feature scaling in few-shot learning. The train set of mini-ImageNer [23] is used as the base dataset train the model. Since the novel task confronted by the model can be arbitrary in real-world applications, we employ wide range of datasets for testing that exhibit significant differences in both category and domain, including the test set of mini-ImageNer (mini-test), CUB [24]. Traffic Signs [25]. VGG Flowers [26]. Quick Draw [27] and Fungi [28]. In the testing phase, we randomly sample 2000 few-shot tasks from the test dataset in the form of 5-way 1-shot and report the average accuracy all tasks for performance evaluation. The results are shown in Table 1. It can be seen that simply using Kendall's rank correlation at test time in few-shot learning achieves significant improvements compared with cosine similarity. It also vastly outperforms the latest test-time feature transformation method CIM.
Moreover, we also demonstrate the effectiveness of Kendall's rank correlation in few-shot learning through another closer observation of performance. Specifically, we pre-train the model on base dataset from scratch using standard cross-entropy loss and make comparison between the performance of Kendall's rank correlation and cosine similarity on few-shot tasks after each training epoch. The results shown in Figure 2. It can be seen that almost throughout the entire training
5

---

525
50.0
Kordal
Korde has
47.5
45.0
Accuracy Ok
42.5
15
40.0
37.5
35.0
32.5
Kendall
100
125
150
175
200
Epoch
Figure 2: Comparisons between the performance of Kendall's 's rank correlation and cosine similarity on the test of mini-ImgeNee under 5-way 1- shot after each epoch, when pre-training Conv-4 network from scratch.
Figure Comparisons between the performance of leveraging Kendall's rank correlation for both training and testing, and for testing only.
process, Kendall's rank correlation achieves better performance compared cosine similarity, demonstrating the effectiveness of leveraging channel importance ranking in few-shot learning.
5 DiffKenall: Learning with Differentiable Kendall's Rank Correlation
We have shown that the mere utilization of Kendall's 's rank correlation test time yields substantial enhancement in few-shot learning performance. In this section, we investigate the potential for further improvements through the integration of Kendall's rank correlation into the meta learning process. The main challenge is that the calculation of channel importance ranking is non-differentiable, which hinders the direct optimization of Kendall's rank correlation for training. To tackle this problem, in our study, we propose a differentiable Kendall's rank correlation by approximating Kendall's 's rank correlation with smooth functions, hence enabling optimizing ranking consistency directly in episodic training.
5.1 A Differentiable Approximation of Kendall's Rank Correlation
Given two n-dimensional vectors x (......, Fn). = (91 .., yn), we define (x. y) as:
1 ea(z; e -((== ca(w= a No ca(z:-25) + e c-((===))) ea(w:-us) e-a(
Fa (x,
where a > is hyperparameter, and No n(n-1) represents the total number of channel pairs. Lemma 1. Fa (æ. y) is differentiable approximation of Kendall's 's rank correlation T(x, 8).
T(x, y) lim a-+ (x.
Please refer to the appendix for the proof. The main idea of the proof involves using a sigmoid to approximate the sgn function in the sgn expression of Kendall's ' rank correlation where T(x, y)
m(n-1) Eie sg, I.)) sgn(y: V;).
5.2 Integrating Differentiable Kendall's Rank Correlation with Meta-Baseline
By approximating Kendall's rank correlation using Eq. (3), we are able to directly optimize Kendall's rank correlation in training, addressing the issue non-differentiabiiiiyy in rank computation. This implies that our method can integrate with numerous existing approaches that rely on geometric similarity, by replacing geometric similarity with differentiable Kendall's s rank correlation in episodic training. thereby achieving further improvements.

---

Algorithm 1: Episodic training with differentiable Kendall's rank correlation (2,
Input:
Base class dataset Druse = { (Ti yilli = =1, 1....N
Output:
The episodic-traing loss C.
Randomly sample categories from base class dataset Dasse
Randomly sample m, samples in each category to build the support S;
3 Randomly sample my samples in each category to build the query set Q;
Compute class prototypes: C*
Σ
fo (z:). Sk denotes the subset of S where k;
[
(x.y)
Compute the episodic-training loss:
Σ (x.2)EE
logp(ulæ:). p(ulæ) is obtained by Eq. (1),
where Fa (æ. is used as the similarity measure sim(-).
Table 2: Comparison studies mini-ImageNNet and tiered- Lrreed-ImaeNee... The average accuracy (%)
with 95% confidence interval of the 5-way I-shot setting and the 5-way 5-shot setting is reported.
Dataset
Method
Backbone
5-way 1-shot
5-way 5-shot
ProtoNet [I]
Conv-4
49.42 +: 0.78
68.20 +: 0.66
MatchingNet [23]
Conv-4
43.56 0.84
55.31 1: 0.73
MAML [11]
Conv-4
48.70
63.11 0.92
GCR [30]
Conv-4
53.21 0.40
72.34 + 0.32
SNAIL [31]
ResNet-12
55.71 = 0.99
68.88 + 0.92
mini- Nmageeet
AdaResNet [32]
ResNet-12
56.88 0.62
71.94 1 0.57
TADAM [14]
ResNet-12
58.50 1 0.30
76.70 0.30
MTL. [33]
ResNet-12
61.20
75.50
MetaOptNet [12]
ResNet-12
62.64 0.61
78.63
TapNet [34]
ResNet-12
61.65
7.10
CAN [3]
ResNet-12
63.85 = 0.48
79.44
ProtoNet TRAML [35]
ResNet-12
60.31 0.48
77.94 0.57
SLA--G [36]
ResNet-12
62.93
79.63 0.47
ConstellationNet [6]
ResNet-11
64.89 0.23
79.95 0.17
Meta-Baseline [4]
ResNet-12
63.17 0.23
79.26 1. 0.17
Meta-Baseline DiffKendall (Ours)
ResNet-12
65.56 0.43
80.79 + 0.31
ProtoNet [1]
Conv-4
53.31 0.89
72.69 + 0.74
tiered- ImageNet
Relation Networks [2]
Conv-4
54.48 0.93
71.32 - 0.78
MAML [11]
Conv-4
51.67 1.81
70.30 - 1.75
MetaOptNer [12]
ResNet-12
65.99 0.72
81.56 0.53
CAN [3]
ResNet-12 12
69.89 + 0.51
84.23 + 0.37
Meta-Baseline
Meta- Baseline [4]
ResNet-12
68 08.62
83.74 0.18
Meta-Bascline- Difflendall (Ours)
ResNet-12
70.76 0.43
85.31 + 0.34
straightforward application is the integration of our method with simple and widely-adopter baseline in few-shot learning. In Meta-Baseline cosine similarity cos(æ, y) employed as the similarity measure sim(.) in Eq. (1), to determine the semantic similarity between query samples' embedding and prototypes in episodic training and testing. Hence, we replace the cosine similarity originally utilized Meta-Baseline with Kendall's 's rank correlation by employing differentiable Kendall's isss rank correlation during the meta-training phase and adopting Kendall's 's rank correlation during the testing phase. The outline calculating the episodic-training loss with differentiable Kendall's rank correlation 7, (x, y) is demonstrated in Algorithm 1.
[4], which is
5.3 Results
Settings. We conduct extensive experiments on mini- ImageNet [23] and tiered-Imagede ImageNet [29] for performance evaluation, both of which are widely used in previous studies. We use ResNet-12 12 as the backbone network and first pre-train the feature extractor on the base dataset. a Eq. (3) set to 0.5 for the differentiable approximation of Kendall rank correlation. The performance evaluation conducted on randomly sampled tasks from the test set, where the average accuracy and the 95% confidence interval reported.
7

---

66
55
Kendal llest only)
+
Kendal Iran test)
Accurats 1961
60
-00
cosine
23
Sendell (rest only)
20
0.0
0.4 0.8
11
(b)
Figure 4: Average accuracy of 5-way I-shot tasks on the test-set of mini mnni--mageeNer using the masked features. (a) Channels with values less than Lg are masked out (b) Channels with values larger than Hu are masked out.
Comparison Studies. Table 2 shows the results of the comparison studies. It can be that both the datasets, compared with the original Meta-Basellne that uses cosine similarity in episodic training. we achieve a clear improvement by replacing cosine similarity with the proposed differen- tiable Kendall's rank correlation, with 2.39%, 2.16% in the 1-shot and 1.53%, 1.57% in the 5-shot, respectively. Moreover, our method also outperforms methods like CAN [3] and ConstellationNet [6], where cross-attention and self-attention blocks are used. It is worth noting that there are no additional architectures or learnable parameters introduced in method, just like the original Meta Baseline.
Furthermore, we also conduct a comprehensive comparison to demonstrate the role of incorporating differentiable Kendall's rank correlation during episodic training. The results are presented in Figure 3. Compared with solely adopting Kendall's rank correlation at test time, it can observed that leveraging the differentiable Kendall's 's rank correlation in episodic training leads to 1%-2% improvement under 5-way 1-shot on test sets with varying domain discrepancies. This clearly demonstrates that the proposed differentiable Kendall's rank correlation can effectively serve soft approximation to directly optimize ranking consistency for further performance improvements in few-shot learning.
5.4 Analysis
Channel-VV Wise Ablation Studies: A closer look at the performance improvements. We aim to carry out an in-depth analysis to uncover the underlying reasons behind the performance gains observed in few-shot learning upon utilizing Kendall's rank correlation. By determining semantic similarities between features with the correlation channel importance ranking, we can effectively distinguish the role and contribution of small-valued channels that overwhelmingly occupy the feature space of novel samples for classification. As a result, these previously neglected small-valued channels be fully harnessed to enhance classification performance. To validate this, propose a channel-wist ablation study in which we test the performance models on few-shot tasks using the small-valued and large-valued channels separately, allowing for more detailed and nuanced understanding of their respective roles in classification. Concretely, given an n-dimenssinal feature (I,, ), we define two types of masks (4,,..,,)) (hi, .., hn), follows,
4 = S
if 24 < Lo else
h4 = {0
if x; > Ho else
The masked feature is then calculated as H O I or E = & h, where C denotes Hadamard Product. We selectively preserve channels with large values in mask while masking out small- valued channels. Conversely, we exclude channels with large values in mask h to exclusively evaluate performance of small-valued channels on classification. Subsequently. we utilize embedding to compare the performance of Kendall's 's rank correlation and cosine similarity in few-shot learning under various settings threshold Top and Hg with the corresponding results illustrated in Figure 4.
masked

---

80.0
70
72/14
80.31
75,30
05025
80223
77.5
Acquracy 180
72.5
1-shot S-shet
1-chat
S-shct
67.5
65.0
65.0
1.25n
nir-332
0.25
0.75
Number of charnel pairs
Hyperparametera
Figure 5: Performance of the linear time com- plexity calculation method for Kendall's rank correlation on mini-ImageNet (n = 640).
Figure 6; Ablation studies of the hyperparameter a Eq. (3) on minn--mmaeNNe.
Figure 4(a) shows that when small-valued channels masked during testing, both cosine similarity and Kendall's rank correlation achieve similar performance, but significant improvements are observed by utilizing differentiable Kendall's 's rank correlation for episodic training. small-valued valued channels are gradually unmasked, Kendall's ' rank correlation significantly outperforms cosine similarity. This demonstrates that the improvement in performance achieved by utilizing Kendall's rank correlation is due to a more effective utilization of small -valud channels in novel sample features. This effect further reflected in Figure 4(b), where masking only large-valued channels and utilizing only small-valued channels for classification results in substantial improvement of approximately 9% in performance using Kendall's rank correlation compared cosine similarity.
Calculating Kendall's Rank Correlation within Linear Time Complexity. Kendall's rank cor- relation requires us compute the importance ranking concordance of any pair of channels. This results in a higher time complexity compared to cosine similarity, increasing quadratically with the total number of channels. We investigate whether this time complexity could be further reduced to improve the computational efficiency of Kendall's rank correlation at test time. Specifically, we propose a simple approach calculate the importance ranking concordance by randomly sampling a subset of channel pairs instead of using all channel pairs. The experimental results are presented in Figure 5, where represents the total number of channels in the features of novel samples. can be observed that by randomly sampling 5n channel pairs, we achieve performance that is very close to using all channel pairs. It should be noted that this performance has already surpassed that the original Meta- deta-Baseline method while the time complexity is maintained linear, equivalent to cosine similarity.
Hyperparameter Sensitivity. We also investigate the impact of the hyperparameter a in Eq. (3), and the experimental results are presented in Figure 6. The best results are obtained around value of 0.5, and the performance is found to be relatively insensitive to variations of a within a certain range. Setting value for that is too large or too small may lead to a decrease performance. When value for is too large, the model may overfit to the base classes during episodic training, which can result in decreased generalization performance on novel classes. Conversely, a value that is too small is used, this may lead to poor approximation of Kendall's rank correlation.
5.5 Visualization
Further visual analysis is demonstrated in Figure 7. Specifically, employ Kendall's rank correlation and cosine similarity to visualize the feature maps the query samples. By computing the semantic similarity between the class prototype and each local feature of the query samples, the regions wherein the salient targets are located on the feature maps are illustrated. Hence, we can observe whether the significant features in the query samples accurately detected. It is evident that the utilization of Kendall's 's rank correlation results in a more precise localization of the distinctive regions within the query sample.
Furthermore, we conduct in-depth visual experiments involving channel ablation. We mask the channels with values greater than Ho in the features of both the class prototype and query sample,

---

golden retriever
crab
da mation
triffe
lion
Query
Cosine
Kendall
Figure 7: The results of the visual analysis on the test set of mini-ImageNet with cosine similarity and Kendall's Rank Correlation respectively. can be that Kendall's rank correlation demonstrates more accurate localization of salient targets and superior differentiation of key features.
Cosine
Kendall
Query Image
Hg=0.2
Hg=0.3
Hy=0.4
Hg=1.0
Full
Figure 8: The results of the channel ablation visualization experiments, where channels with values greater than Ho are masked out, and 'Full' indicates when all channels are used. It can seen that the key features of the category are held on the small-valued channels and successfully uncovered by Kendall's 's rank correlation, while cosine similarity misses these critical features.
just like the channel-wise ablation experiments Section 5.4. The results are shown Figure from which we observe that Kendall's rank correlation captures the discriminative features in the query sample when only utilizing the small-valued channels. In contrast, cosine similarity ignores these critical features, resulting in inability to correctly locate salient regions when all channels used. Therefore, we can infer that the small-valued channels that occupy the majority the features indeed play vital role in few-shot learning. This also explicitly demonstrates that the improvement achieved by Kendall's rank correlation in fho-shot learning is essentially due to its ability to fully exploit the imall-valued channels in features.
Conclusion
This paper exposes a key property of the features of novel samples in few-shot learning, resulting from the fact that values on most channels are small and closely distributed, making it arduous to distinguish their importance in classification. To overcome this, we propose to replace the commonly used geometric similarity metric with the correlation the channel importance ranking to determine semantic similarities. Our method can integrate with numerous existing few-shot approaches without increasing training costs and has the potential to integrate with future state-of-the-arr methods that rely geometric similarity metrics to achieve additional improvement.
10

---

Acknowledgment
This work supported by 2023 CCF-Baidu Open Fund and Microsoft Research Asia.
We would like to express our sincere gratitude to the reviewers NeurIPS 2023 for their insightful and constructive feedback. Their valuable comments have greatly contributed to improving the quality of our work.
References
[1] Jake Snell, Kevin Swersky, and Richard S. Zemel. Prototypical networks for few-shot learning. In NeurIPS, 2017.
[2] Flood Sung. Yongxin Yang, Li Zhang. Tao Xiang, Philip H. S. Torr, and Timothy M. Hospedales. Learning to compare: Relation network for fow-shor learning. In CVPR, 2018.
[3] Ruibing Hou, Hong Chang, Bingpeng Ma, Shiguang Shan, and Xilin Chen. Cross attention network for few-shot classification. In NewrlPS, 2019.
[4] Yinbo Chen, Zhuang Liu, Huijuan Xu, Trevor Darrell, and Xiaolong Wang. Meta- Meta-baseline: Exploring simple meta-learning for few-shot learning. In ICCV, 2021.
[5] Puneet Mangla, Mayank Singh, Abhishek Sinha. Nupur Kumari. Vineeth N. Balasubramanian, and Balaji Krishnamurthy. Charting the right manifold: Manifold mixup for few-shot learning. WACV, 2020.
Weijian Xu, Yifan Xu, Huaijin Wang, and Zhuowen Tu, Attentional constellation for few-shot learning. In ICLR, 2021.
Baoquan Zhang, Xutao Li, Yunming Ye, Zhichao Huang, and Lisai Zhang. Prototype completion with primitive knowledge for few-shot learning. In CVPR, 2021.
[8] We Chen, Yen-Cheng Liu, 7salt Kira, Yu-Chiang Frank Wang, and Jia-Bin Huang. A closer look few-shot classification. ICLRR 2019.
Guneet Singh Dhillon, Pratik Chaudhari, Avinash Ravichandran, and Stefano Soatto. baseline for few-shot image classification. /CLR, 2020.
[10] Yan Wang. Wei-Lun Chao, Kilian Q. Weinberger, and Laurens van der Maaten. Simpleshot: Revisiting nearest-neighbor classification for few-shot learning. arxiv:1911.04623,, 2019.
[11] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learnin for fast adaptation of deep networks. In JCML., 2017.
[12] Kwonjoon Lee, Subhransu Maji, Avinash Ravichandran, and Stefano Soatto. Meta-learning with differen- tiable convex optimization. In CVPR, 2019.
[13] Andrei A. Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals. Razvan Pascanu, Simon Osindero, and Raia Hadsell. Meta-learning with latent embedding optimization. In ICLR, 2019.
[14] Boris N. Oreshkin, Pau Rodríguez López, and Alexandre Lacoste. TADAM: task dependent adaptive metric for improved few-shot learning. In NewrIPS, 2018.
[15] Chi Zhang, Yujun Cai, Guosheng Lin, and Chunhua Shen. Deepemd: Few-shot image classification with differentiable earth mover's 's distance and structured classifiers. In CVPR, 2020.
[16] Han-Jia Ye, Hexiang Hu, De-Chuan Zhan, and Fei Sha. Few-shot learning via embedding adaptation with set-to-set functions. In CVPR, 2020.
[17] Wenbin Li, Lei Wang, Jinglin Xu, Jing Huo, Yang Gao, and Jiebo Luo. Revisiting local descriptor based image-to-elass measure for few-shot learning. In CVPR, 2019.
[18] Wanqi Xue and Wei Wang. One-shot image classification by learning to restore prototypes. In AAAI, 2020.
[19] Arman Afrasiyabi, dran-FFançois Lalonde, and Christian Gagné. Associative alignment for few-shot image classification. In ECCV, 2020.
[20] Rajshekhar Das. Yu-Xiong Wang, and José M. F. Moura. On the importance of distractors for few-shot classification. In ICCV, 2021.

---

[21] Duong Hoang Le, Khoi Duc Nguyen, Khoi Nguyen, Quoc-Huy Tran, Rang Nguyen, and Binh-Son Hua. POODLE: Improving few-shot learning via penalizing out-of-distribution samples. In NeurlPS, 2021.
[22] Xu Luo, Jing Xu, and Zenglin Xu. Channel importance matters in few-shot image classification. In ICMI.,
2022.
[23] Oriol Vinyals, Charles Blundell, Tim Lillicrap. Koray Kavukcuoglu, and Daan Wierstra. Matching networks for shot learning. In NeurIPS, 2016.
[24] C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. The caltech-uesd 20rds-200-2011 dataset. Technical Report CNS-TR-2011-00 California Institute of Technology. 2011.
[25] Sebastian Houben, Johannes Stallkamp, Jan Salmen, Mare Schlipsing, and Christian Igel. Detection of traffic signs in real-world images: The german traffic sign detection benchmark. In IJCNN 2013.
[26] Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In ICVGIP, 2008
[27] Jonas Jongejan, Henry Rowley, Takashi Kawashima, Jongmin Kim, and Nick Fox-Gieg. The quick, draw! experiment. mniickdra..ithheeee withgoogle com, 2016.
[28] Brigit Schroeder and Yin Cui. Fgvex fungi classification challenge 2018. github.com/vise. com.visipedia fgvcx_fungi_comg 2018.
[29] Mengye Ren, Eleni Triantafillou, Sachin Ravi, Jake Snell, Kevin Swersky, Joshua B. Tenenbaum, Hugo Larochelle, and Richard S. Zemel. Meta-learning for semi-supervised few-shot classification. In ICLR, 2018
[30] Aoxue Li, Tiange Luo, Tao Xiang, Weiran Huang, and Liwei Wang. Few-shot learning with global class representations. ICCV, 2019.
[31] Nikhil Mishra, Mostafa Rohaminejad, Xi Chen, and Pieter Abbecl. A simple neural attentive meta-Icarner. In ICLR, 2018
[32] Tsendsuren Munkhdalai, Xingdi Yuan, Soroush Mchri, and Adam Trischler. Rapid adaptation with conditionally shifted neurons. JCMI,, 2018.
[33] Qianru Sun. Yaoyao Liu, Tat-Seng Chua, and Bernt Schiele. Meta-transfer learning few-shot learning. In CVPR, 2019.
[34] Sung Whan Yoon, Jun Sco, and Jackyun Moon. Tapnet: Neural network augmented with task-adaptive projection for fow-shot learning. In /CML, 2019.
[35] Aoxue Li, Weiran Huang, Xu Lan, Jiashi Feng, Zhenguo Li, and Liwei Wang. Boosting few-shot learning with adaptive margin loss. CVPR, 2020.
[36] Hankook Lee, Sung Hwang, and Jinwoo Shin. supe-super label augmentation via input transforma- tions. /CML, 2020.
12

---

Appendix
A Proof for Lemma
Lemma 1. To (x, y) is differentiable upproximation of Kendall's rank correlation T (ax y) T(x, y)
lim Fa(x, y).
a-+oc
Proof.
First, consider the scenario where channel pairs exhibit consistent importance ranking, specif-
ically, either 2; > Iú A > yj or < 2z A yo Vý- In the case where I₁ > 23 A ys Vj. we obtain:
e ca( xc(== - e a(2:-zz) ga( (1))))))))))))) 1z) + e a=(
lim ea(u vz) e aavii
lim
calui (al()-v)) 4z) -a(1i- 1.
On the other hand, if 23 < zj A 3t < Yj we have:
lim ((w)= e -(( ( [(;=-;;
lim a e a((c-
t co(n - #; e-a( -
Hence, when xj Ij Vi yj or मं zj yi < Yj, the following conclusion holds:
( e a(z ca (y) (35
lim a-+00 e + e e-((=_2,, ca(w vリ) -a 1.
(4)
Second, consider the scenario where channel pairs exhibit inconsistent importance ranking, that is, either 2, > 2j A yi < yj or I; < A yi > Vj- In the case where 2; > 2j A y/ < Yj. we obtain:
ا - e 22, ca(a; - + E -a(=;=,))
(u(--v E ax(ー))
lim
lim a+++00 cn mil + e -a(a 95)
On the other hand, I; < xj > y/ > Yj, we have:
aly Us) -a lim ea(a;- ( mz) + a(===15)) -1, lim
a()==))) e x(=
a(v Ms)
1.
a eaty U5) + e-as 93))
Hence, when x, Dj 8i < 9 or 4; 2j A > yj the following conclusion holds:
ا - e - a u)) aa( 1.
lim a-+ ea(=:=a,) e -((,,_.,) c vo) -a( 25
(5)
When considering all channel pairs, combining Eq. (4) and Eq. (5), is evident that: (0()=)))) - e -o( ( mz) c (0(z)-2)) + e- ( + -ee eva(w))s))
x i-1
lim ia (z 4) lim a a++00 t No i=
Neon Nas
No
T y),
where Noon represents the total count of channel pairs with consistent importance ranking, NNis represents the count of channel pairs with inconsistent importance ranking.
13


Reviews:
[{"review_id": "rv3mis9f5J", "reviewer": "Reviewer_7vc7", "summary": "This paper aims at addressing the uniform distribution of values across features on novel classes, and proposes to use Kendall's rank corerlation instead of geometric similarity metrics to improve the performance of few-shot learning pipelines. Moreover, the authors propose a designed loss for meta-training to make Kendall's rank correlation differentiable. The experimental results demonstrate the usefulness of the propose method.", "questions": "- It seems the work is based on prototype. Is it possible to perform it between support-query pairs rather than support proto-query pairs? As it highlights the importance of feature channels, it should be more meaningful to perform support-query correlation. ", "limitations": "Please refer to the weaknesses.", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The idea is technically sound. Theoretically, the channel importance is definitely beneficial to improve the accuracy of FSL. \n- The expeirment results show the usefulness of the propose method.\n- The presentation is well and easy to follow.", "weaknesses": "- The novelty and significance of the proposed method is roughly limited. The core idea of Kendall's rank correlation is to measure the consistency of pairwise rankings for each channel pair. However, such a issue has been considered in previous cross-matching related works such as DeepEMD. If this design can be incorporated with DeepEMD or other cross-matching pipelines, it would be interesting to see the overall performance and the conclusions will be more convincing.\n- The experimental results is not impressive enough. Though this method achieves improvements compared to metric-based methods, its performance is still below the SOTA. \n- In figure 1, it is proper to show more cases to demonstrate the advanceness of Kendall's rank correlation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Q1: The proposed method's novelty is limited, since the consistency of pairwise rankings for each channel pair has been considered in previous works such as DeepEMD. Can this design be incorporated with DeepEMD?\n\nA1: Thanks for your insightful feedback. While it may seem that both DeepEMD and our method consider the consistency of pairwise rankings, they do so from **two different perspectives**. Specifically, DeepEMD employs the Earth Mover's Distance to match various **local regions** in the spatial domain of the image. In contrast, our method leverages Kendall’s Rank Correlation to achieve consistency across **feature channels**.\n\nMoreover, we would like to highlight that our method can easily be integrated into existing methods **without increasing training costs**. As an example, **our method can indeed be incorporated with DeepEMD** by substituting the cosine similarities used therein with Kendall’s Rank Correlations. Through this modification, we have observed **a large improvement for DeepEMD** (1%-2%). All the latest experimental results, demonstrating this enhancement, are detailed in Table A of the attached PDF in our \"global\" response.\n\nQ2: The experimental results are not impressive enough. Though this method achieves improvements compared to metric-based methods, its performance is still below the SOTA.\n\nA2: Thanks for bringing this to our attention. It's important to clarify that our current experiments are conducted **based on a simple and widely-adopted baseline** (meta-baseline) to **demonstrate the effectiveness** of our method. While this may lead to performance that is below the SOTA, we believe it adequately demonstrates the capabilities and potential of our method.\n\nMoreover, as mentioned in the response to the previous question, our method could be easily integrated with existing methods. **Combining our method with a stronger baseline**, DeepEMD, we can **achieve the current SOTA performance**, as shown in Table A of the attached PDF in our \"global\" response. \n\nFurthermore, even for most recently proposed methods such as CIM, we have shown that simply replacing cosine similarity with Kendall’s Rank correlation (our method) at the inference stage can result in significant improvements across various datasets with domain differences (see Table 1 of the submitted paper). This indicates that **our method is ready for integrating with future SOTA methods** to achieve additional improvement. \n\nWe hope this explanation can address your concern, and provide a clearer demonstration of the value and adaptability of our method.\n\nQ3: In figure 1, it is proper to show more cases to demonstrate the advanceness of Kendall's rank correlation.\n\nA3: Thank you for this feedback. Indeed, there are numerous such examples. We would like to kindly remind you that, in the **supplementary material's visual analysis section**, we provide more intuitive illustrations of the superior performance of Kendall’s ranking correlation. \n\nConcretely, we employ Kendall’s Rank Correlation and cosine similarity to visualize the feature maps of the query samples with the aim of confirming the accurate localization of salient objects in the images. It is evident that the utilization of Kendall’s Rank Correlation results in a more precise localization of the distinctive regions within the query sample. \n\nMoreover,  we also conduct an in-depth visual analysis involving channel ablation. It is noticeable that the discriminative key features of the dog predominantly exist in channels with lower values. Utilizing Kendall's rank correlation effectively captures these essential features, whereas cosine similarity disregards them, providing evidence of the effectiveness of our method.\n\nQ4: It seems the work is based on prototype. Is it possible to perform it between support-query pairs rather than support proto-query pairs? As it highlights the importance of feature channels, it should be more meaningful to perform support-query correlation.\n\nA4: Thanks for your insightful feedback. Indeed, our method can be applied to not only prototype-query matching **but also support-query matching**. For example, we replace cosine similarity with Kendall’s Rank Correlation in InfoPatch which is a **contrastive-learning-based few-shot learning method**, and observe an improvement in both 1-shot and 5-shot tasks on mini-ImageNet, as shown in Table A of the attached PDF in our \"global\" response."}, {"review_id": "JZ94KEQZF3", "reviewer": "Reviewer_5zcq", "summary": "The authors suggest a new similarity metric that utilizes differentiable Kendall’s rank correlation instead of the commonly used geometric similarity metrics like cosine similarity in few-shot learning. By levitating the importance of small-valued feature channels, the proposed approach significantly improves the few-shot performance across multiple datasets from various domains.", "questions": "- The paper does a great job in pointing out a source of problem in few-shot classification and a good job in proposing a way to address it. However, it feels somewhat weak in terms of looking in-depth into the problem. Following the ‘weaknesses’ discussion above, it would have been great to improve the paper on this end and explore the advantages/disadvantages of some simple potential alternative fixes such as (i) instance/group normalization, (ii) temperature scaling combined with l2 normalization, or (iii) a simple attention mechanism such squeeze&excitation attention.\n- Suggestion: The name of the dataset on which the presented results are obtained can be added in the discussions (or captions) of Figures 5 and 6.\n- Suggestion: The overall figure quality can be improved.", "limitations": "No additional comments.", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- The presented idea is simple and appears to be effective, and the overall approach is clearly expressed. The work is built on a simple yet powerful observation about the feature activation statistics (and their differences on base vs novel classes).\n- Utilizing a differentiable version of Kendall’s rank coefficient measure as an alternative similarity metric is an original idea to the best of my knowledge.\n- The proposed method enables efficient end-to-end few-shot learning without introducting problematic hyper-parameters.\n- It is impressive that using the proposed metric directly for testing, even without any pre-training, improves the result.\n- The proposed approach has proven to be effective in many common few-shot datasets across various domains, outperforming competitive methods in terms of performance improvements.\n- Ablation studies provide valuable insights that demonstrate the effectiveness of the proposed approach.", "weaknesses": "- While a solution to the Figure 1 observation is proposed based on Kendall’s rank correlation, I am not sure if this is the most simple way to handle the problem. In particular, could a simple instance-statistics-driven normalization scheme, such as group norm or instance norm or similar, could address the feature-scale issues?\n- Similarly, can’t the cosine similarity, combined with temperature scaling (a well known practice in , see Baseline++ paper for a detailed discussion) , where temperature may be different at test time, also address the problem of scale?\n- The explanations on Kendall’s rank correlation can be extended to make it more explanatory and to shed even more light on its complexities to make the paper more self-contained.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Q1: Could a simple instance-statistics-driven normalization scheme, such as group norm or instance norm or similar, could address the feature-scale issues?\n\nA1: **No.** Actually, solely adopting Kendall's rank correlation during the inference stage far exceeds what can be achieved by employing simple feature scaling methods, and we conduct experiments to demonstrate this. Specifically, on top of the raw features from the model's output, we apply GroupNorm, InstanceNorm, and squeeze&excitation attention for feature scaling transformations, followed by classification using cosine similarity (Please find detailed results in Table B of the attached PDF in our \"global\" response).\n\nRegarding **GroupNorm**, we test two values for the \"num_group\" parameter, namely 16 and 32. Although its performance occasionally surpasses the original cosine similarity, we observe that it is generally inferior to Kendall's rank correlation in terms of performance. Concerning **InstanceNorm**, we notice that this operation results in a substantial performance decline, with a decrease of over 10% compared to directly using cosine similarity. Furthermore, we also explore the use of **Squeeze&Excitation Attention**. We integrate this module into the backbone network for training, and the results also show that it does not demonstrate a clear advantage. \nActually, in Table 1 of Section 4, we have compared our method with the **CIM** method, which is a most recently proposed simple test-time feature scaling method in few-shot learning. Across multiple datasets with diverse domain differences, our method also consistently outperforms CIM. These findings provide strong evidence that the improvements obtained through Kendall ranking correlation are not attainable merely by simple feature scaling. We will include this discussion in the paper to emphasize the superiority of our proposed method. Thank you for your valuable review.\n\nQ2: Can’t the cosine similarity, combined with temperature scaling also address the problem of scale?\n\nA2: **No, it can't.** In fact, the temperature coefficient is also used in the meta-baseline method to adjust the output probabilities. When reproducing their experimental results, we follow the original settings of the meta-baseline and set this parameter as learnable during training. We conduct experiments accordingly, and if we solely adjust this parameter, the performance would be lower compared to the results reported in the meta-baseline paper, as shown below. \n\n| Method   | Backbone |   T: Learnable | T= 1 |T= 0.1 |T= 0.01 |\n| :---: | :---: | :---: |:---: |:---: |:---: |\n|Meta-Baseline (cosine)|ResNet-12|63.17|62.35|62.84|62.76|\n\nClearly, adjusting the temperature parameter alone cannot achieve the level of performance improvement obtained by using Kendall’s ranking correlation.\n\nQ3: The explanations on Kendall’s rank correlation can be extended to make it more explanatory and to shed even more light on its complexities to make the paper more self-contained.\n\nA3: Thank you for bringing this to our attention. Let us provide a detailed explanation below. \n\nOur approach emerged from observing an apparent difference in feature channel values between base data and novel data. We found that compared to base classes, when the feature extractor faces a novel class that is unseen before, the feature channel values become more uniform, i.e., **for a novel class, most non-core features' channels have small and closely clustered values** in the range [0.25, 0.5] (see Figure 1 of the submitted paper). \n\nThis phenomenon occurs because the model is trained on the base data, and consequently exhibits reduced variation of feature values when dealing with novel data. This situation creates a challenge in employing geometric similarity to accurately distinguish the importance among non-core feature channels.\n\nTo provide a concrete example, consider distinguishing between dogs and wolves. **While they share nearly identical core visual features, minor features play a vital role in differentiating them**. Suppose the core feature, two minor features are represented by channels 1, 2, and 3, respectively, in the feature vector. A dog prototype may have feature (1, 0.3, 0.2), and a wolf prototype may have feature (1, 0.2, 0.3). Now, for a test image with feature (0.9, 0.28, 0.22), it appears more dog-like, as the 2nd feature is more prominent than the 3rd. However, cosine distance misleadingly places this test image closer to the wolf prototype (distance=0.031) rather than the dog prototype (distance=0.048).\n\nContrastingly, the test image shares the same channel ranking (1, 2, 3) as the dog prototype, whereas the wolf prototype's channel ranking is (1, 3, 2). Inspired by this, we employ Kendall’s rank correlation to more accurately discern between dogs and wolves, highlighting the utility of our approach.\n\nWe hope this clarification better conveys the underlying rationale for our method, and we will carefully review this section in the revised paper.\n\nQ4: It would have been great to explore the advantages/disadvantages of some simple potential alternative fixes such as (i) instance/group normalization, (ii) temperature scaling combined with l2 normalization, or (iii) a simple attention mechanism such squeeze&excitation attention.\n\nA4: Thank you for the comment. In the response to Q1 and Q2, we have verified and demonstrated that the performance improvement achieved by using Kendall's rank correlation far exceeds what can be achieved by employing simple feature scaling methods, which is related to this question.\n\nSuggestion: i) The name of the dataset on which the presented results are obtained can be added in the discussions (or captions) of Figures 5 and 6. ii) The overall figure quality can be improved.\n\nA5: Thank you very much for your valuable feedback. We will make the necessary revisions to the manuscript according to your suggestions."}, {"review_id": "G7FkjYFIyG", "reviewer": "Reviewer_oKqH", "summary": "This paper introduces a novel approach for few-shot learning using Kendall's Rank Correlation. The authors demonstrate that feature channel importance ranking is a more reliable indicator for few-shot learning than geometric similarity metrics. They propose replacing the geometric similarity metric with Kendall's rank correlation for inference, which improves the performance of few-shot learning across different datasets and domains. Additionally, the paper presents a carefully designed differentiable loss for meta-training to address the non-differentiability issue of Kendall's rank correlation. The contributions of this paper can be summarized as follows: \n1. Introducing the use of feature channel importance ranking for few-shot learning.\n2. Demonstrating the effectiveness of Kendall's rank correlation in improving few-shot learning performance.\n3. Proposing a differentiable approximation of Kendall's rank correlation for meta-training, leading to further performance improvements.\n", "questions": "1. “This often leads to a relatively uniform distribution of values across feature channels on novel classes, making it difficult to determine channel importance for novel tasks.” I can't understand the meaning of this sentence? Please provide a detailed explanation.\n\n2. “When we compare the values of different feature channels on the base dataset and novel dataset, we observe that the novel dataset has a much more uniform value distribution than the base dataset. ” Can this conclusion hold true? This is only an observation on one dataset, and there is no evidence to suggest that the new dataset has a multiple more uniform value distribution than the base dataset.\n\n3. The third paragraph in the introduction uses an example of dogs and wolves to illustrate that cosine similarity cannot effectively distinguish between dogs and wolves, while Figure 1 (b) shows the identification results of dogs and crabs.\n\n4. In Table 1, for deeper backbones, why does the Kendall's rank correlation have a decrease in performance compared to cosine distance?\n", "limitations": "The authors do not provide limitations and potential negative societal impact of their work.", "rating": 3, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. Originality: The paper presents a novel approach for few-shot learning that uses Kendall's rank correlation. This is a unique and innovative idea that has not been explored in previous research.\n\n2. Quality: The paper is well-researched and presents a thorough analysis of the proposed method. The authors provide detailed experimental results and ablation studies to validate their approach. The proposed differentiable loss function is carefully designed and addresses the non-differentiability issue of Kendall's rank correlation.\n\n3. Clarity: The paper is well-written and easy to understand. The paper is also well-organized, making it easy to follow the flow of ideas.\n\n4. Significance: The use of Kendall's rank correlation has shown to be effective in improving the performance of few-shot learning across different datasets and domains.  \n", "weaknesses": "1. Lack of comparison with state-of-the-art methods: The paper does not compare the proposed method with state-of-the-art few-shot learning methods.  \n\n2. Lack of theoretical justification: The paper does not provide a theoretical justification for why feature channel importance ranking and Kendall's rank correlation are better suited for few-shot learning than geometric similarity metrics. Providing such a justification would strengthen the paper's argument and make it more convincing.\n\n3. The motivation described in the paper is not clearly explained：Why replace geometric similarity metrics with Kendall's rank correlation?\n\n4. From the results in Table 2, it can be seen that the performance improvement is marginal, and the final result is not SOTA.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Q1: Lack of comparisons with SOTA methods.\n\nA1: Thanks for the suggestion. In our latest experimental results, we have added comparisons with SOTA methods (See Table A of the PDF in our \"global\" response). We would like to highlight that by integrating Kendall's Rank Correlation into a stronger backbone DeepEMD, **our method achieves SOTA performance**.\n\nQ2: Lack of theoretical justification for why Kendall's rank correlation are better suited for FSL.\n\nA2: Thanks. Our method is inspired by empirical observations, making it an algorithmic paper rather than a theoretical one. Extensive experiments support our intuitive observations, which align with our expectations. \nWhile we will further provide intuitive justification in our next response, we leave the theoretical one as our future work.\n\nQ3: The motivation behind replacing geometric similarity metrics with Kendall's rank correlation is not clearly explained.\n\nA3: It seems that our original explanation of the motivation might not have been clear enough. Let us provide a detailed explanation below. \n\nOur approach emerged from observing an apparent difference in feature channel values between base data and novel data. We found that compared to base classes, when the feature extractor faces a novel class that is unseen before, the feature channel values become more uniform, i.e., **for a novel class, most non-core features' channels have small and closely clustered values** in the range [0.25, 0.5] (see Figure 1 of the paper). This phenomenon occurs because the model is trained on the base data, and consequently exhibits reduced variation of feature values when dealing with novel data. This situation creates a challenge in employing geometric similarity to accurately distinguish the importance among non-core feature channels. \n\nTo provide a concrete example, consider distinguishing between dogs and wolves. **While they share nearly identical core visual features, minor features play a vital role in differentiating them**. Suppose the core feature, two minor features are represented by channels 1, 2, and 3, respectively, in the feature vector. A dog prototype may have feature (1, 0.3, 0.2), and a wolf prototype may have feature (1, 0.2, 0.3). Now, for a test image with feature (0.9, 0.28, 0.22), it appears more dog-like, as the 2nd feature is more prominent than the 3rd. However, cosine distance misleadingly places this test image closer to the wolf prototype (distance=0.031) rather than the dog prototype (distance=0.048). Contrastingly, the test image shares the same channel ranking (1, 2, 3) as the dog prototype, whereas the wolf prototype's channel ranking is (1, 3, 2). Inpired by this, we employ Kendall’s rank correlation to more accurately discern between dogs and wolves, highlighting the utility of our approach. \n\nWe hope this clarification better conveys the underlying rationale for our method, and we will carefully review this section in the revised paper to ensure that the motivation is articulated more clearly.\n\nQ4: The results in Table 2 show marginal improvements, and the final result is not SOTA.\n\nA4: It's important to clarify that our current experiments are conducted based on a simple and widely-adopted baseline (meta-baseline). By simply substituting cosine similarity with Kendall's rank correlation, we achieve an improvement of 2%. Moreover, as mentioned in the response to Q1, our method could be easily integrated with existing methods. Combining our method with a stronger baseline, DeepEMD, we can achieve the current SOTA performance.\n\nQ5: Can't understand the meaning of the sentence mentioned.\n\nA5: In the response to Q3, we have detailed the motivation behind our approach, which is related to this question. To further clarify, let's consider an extreme scenario where all feature values are nearly equal. In this case, a minor perturbation in a feature vector could lead to significant changes in the channel ranking, while the geometric distance to other features might remain largely unchanged.\n\nQ6: Can the conclusion drawn from Figure 1 hold true?\n\nA6: We believe this is a universally valid observation. Intuitively, the model is trained on base data and hence the learned feature extractors should capture the largest variation features within that base data. For novel data that belongs to different classes from base data, the extracted features will exhibit less variation. Empirically, we additionally compare the variance of feature channel values between the base dataset and various novel datasets (See Table C). The results reveal a significantly smaller variance in feature channel values in the novel datasets compared to the base dataset. A smaller variance means values are closer to each other, which aligns with the observations depicted in Figure 1(a), validating the correctness of this conclusion.\n\nQ7: The third paragraph uses an example of dogs and wolves, while Figure 1 shows dogs and crabs.\n\nA7: The reason we use wolves and dogs as examples is that it is an intuitive idea. However, in the existing commonly used few-shot learning datasets, there are no classes that include both wolves and dogs. In our subsequent work, we will make examples and figures consistent.\n\nQ8: In Table 1, for deeper backbones, why does the Kendall's rank correlation have a decrease in performance?\n\nA8: Actually Table 1 serves as an exploratory experiment where we solely adopt Kendall’s rank correlation at test time, without any episodic training. Expecting any model to achieve performance improvement without training by simply replacing cosine similarity at test time, would be impractical. In fact, the results in Table 1 show that, in the vast majority of cases, this simple replacement leads to significant improvements. Despite the slight performance decrease on the mini-test, it is evident that across the other five datasets with more substantial domain variations, the use of Kendall’s rank correlation yields significant performance gains."}, {"review_id": "2rDWbkRVFd", "reviewer": "Reviewer_zjGF", "summary": "This paper proposes a new similarity metric for few-shot learning, named Kendall’s rank correlation, which originally come from the statistical concept of Kendall's rank. The motivation of this paper is from an experiment that the corresponding channel value for base and novel classes have a distribution difference. Additionally, the authors design a differentiable approximation for Kendall’s rank correlation and demonstrate favorable experimental results in comparative experiments.", "questions": "Please see the Weaknesses.", "limitations": "The primary concern lies in the reproducibility of the source code, as it is currently non-functional.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.\tThe paper is well-written and easy to follow, with a clear and well-organized structure.\n2.\tThe method is straightforward and simple.\n", "weaknesses": "1.\tThe ablation experiments only employ cosine distance, lacking a comprehensive consideration of Euclidean distance.\n2.\tThe performance of the differentiable approximation for various hyperparameter values is not adequately analyzed or visualized.\n3.\tThe optimal value for the hyperparameter is stated as 0.5 but lacks experimental or theoretical evidence to support this claim.\n4.\tTable 3 includes the \"meta-baseline\" method, which belongs to another approach and should not be classified as part of the proposed method.\n5.\tThe testing procedure is not described at all, including whether the differentiable approximation or Kendall's rank is used during testing.\n6.\tThe reproducibility of the source code is not well identified.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Q1: The ablation experiments only employ cosine distance, lacking a comprehensive consideration of Euclidean distance.\n\nA1: Thank you for your feedback. We would like to point out that the reason why we did not include the Euclidean distance in our experiments is that on the unit sphere, cosine similarity and Euclidean distance are equivalent. Furthermore, in few-shot learning, only ProtoNet adopts Euclidean distance, while many subsequent works are based on cosine similarity and have shown that using cosine similarity generally yields better results compared to using Euclidean distance. Nonetheless, we have also included comparative experiments with Euclidean distance (See Table B of the attached PDF in our \"global\" response). The experimental results demonstrate the superiority of utilizing Kendall's rank correlation over Euclidean distance.\n\nQ2:The performance of the differentiable approximation for various hyperparameter values is not adequately analyzed or visualized.\n\nA2: Thank you for this feedback. We would like to point out that the performance of differentiable Kendall ranking correlation shows relatively low sensitivity to variations in the hyperparameter $\\alpha$ within a specific range. Our experimental findings suggest that setting alpha to approximately 0.5 yields favorable performance. Setting this parameter too high may lead to overfitting on the base class data, while setting it too low may result in inadequate approximation to Kendall ranking correlation. A relevant analysis of this phenomenon has been carried out in the ablation experiments presented in Section 5.4. Here, we also conducted more detailed ablation experiments concerning this hyperparameter as follow.\n\n| Method| $\\alpha$=0.1 | $\\alpha$=0.2 |$\\alpha$=0.3 |$\\alpha$=0.4 |   $\\alpha$=0.5 | $\\alpha$=0.6 |$\\alpha$=0.7 |$\\alpha$=0.8 |\n|:---:|:---:|:---: |:---:|:---:|:---:|:---: |:---:|:---:|\n|Kendall| 63.99|64.66|64.92|65.21|65.56|65.02|64.56|63.79|\n\nQ3: The optimal value for the hyperparameter is stated as 0.5 but lacks experimental or theoretical evidence to support this claim.\n\nA3: Thank you for this feedback. In Section 5.4 and in response to Q2, we have extensively investigated this hyperparameter through ablation experiments. We did not meticulously tune this hyperparameter to deliberately seek a better result. What we discovered from the ablation experiments is that setting $\\alpha$ around 0.5 yields a relatively favorable outcome.\n\nQ4: Table 3 includes the \"meta-baseline\" method, which belongs to another approach and should not be classified as part of the proposed method.\n\nA4: Sorry for any confusion caused. In fact, Meta-Baseline is a a simple and effective method in few-shot learning, which proposes a two-stage training paradigm. Specifically, the model is first pre-trained on the base dataset using cross-entropy loss, following conventional supervised learning. During the meta-training stage, tasks are sampled from the base dataset, simulating the construction of test tasks in the form of N-way K-shot. The training objective is to accurately classify the query samples from the sampled tasks using cross-entropy loss as the loss function. In Meta-Baseline, cosine similarity is employed as the similarity measure to determine the semantic similarity between the query samples' embeddings and prototypes for nearest-neighbor classification. Our proposed method simply replaces the cosine similarity used in Meta-Baseline with the differentiable Kendall similarity for episodic training while keeping all other settings consistent. The relevant details can be found in Section 5.2 of the paper.\n\nQ5: The testing procedure is not described at all, including whether the differentiable approximation or Kendall's rank is used during testing.\n\nA5: Sorry for any confusion caused. In fact, in the **implementation details** provided in Section 5.2, we thoroughly describe our testing process: In the testing phase, we employ Kendall's rank correlation to compute the similarity between the embeddings of query samples and class prototypes for nearest-neighbor classification. Performance evaluation is conducted on 2000 randomly sampled tasks from the test set, and the average accuracy along with the 95% confidence interval are reported. The purpose of proposing the differentiable Kendall correlation is to address the non-differentiable issue of the original Kendall rank correlation in the ranking computation. During the testing phase, we use the original Kendall’s rank correlation for evaluation.\n\nQ6: The reproducibility of the source code is not well identified.\n\nA6: Sorry for any confusion caused. We would like to provide a detailed explanation of the content in our submitted code. The code for the inference phase is included in the 'eval.py' file, where you have the flexibility to set the testing mode. Specifically, setting the mode to 'kendall_test' implies the usage of Kendall ranking correlation in the inference phase. Regarding the training phase, it is divided into two parts, 'train_pretrain.py' and 'train_meta.py'. 'train_pretrain.py' corresponds to the pretraining process using the cross-entropy loss function. On the other hand, 'train_meta.py' is responsible for the meta-training stage, where the training mode is set to 'kendall_meta', indicating the utilization of our proposed differentiable Kendall ranking correlation for meta-training. The implementation of both the original Kendall ranking correlation and our proposed differentiable Kendall ranking correlation for meta-training can be found in the 'Models/models/kendall_fsl.py' file within the code.\n"}, {"review_id": "NEsyFPReOy", "reviewer": "Reviewer_DzME", "summary": "This paper is motivated by the observation that the value of channels in novel dataset has a much more uniform distribution than that in the base dataset.\nBased on the observation, the author proposes a new similarity metric, Kendall’s rank correlation, to utilize the ranking information of channels instead of values to calculate the similarity of two embeddings.\nTo address the non-differentiability issue of Kendall’s rank correlation, the author also proposes a differentiable approximation for meta-learning.\nExperiments show that the Kendall’s rank correlation and proposed approximation Kendall’s rank correlation are useful in some cases.\n", "questions": "Please refer to \"weaknesses\" part for all my concerns.", "limitations": "Overall, this paper has limited contribution and unconvincing experimental results.  ", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1.\tThe idea that uses the ranking information of channels instead of values to calculate the similarity of two embeddings is interesting.\n2.\tThe analysis in Section 5.4 is helpful to understand how the magnitude of channels’ value affects the performance at test-time.\n", "weaknesses": "1.\tThe proposed Kendall’s rank correlation is constrained with FSL methods that uses similarity metric to determine the similarity between two images, which limits the application scope of it. \n2.\tThe comparison in Section 4.2 is not convincing. The authors only consider a baseline that uses cosine similarity metric, which is not a very popular design for many FSL methods. Compared with more baselines using other similarity metrics (like Square Euclidean Distance in ProtoNet and a learnable metric in RelationNet) may help me to make sure the effectiveness and generality of proposed Kendall’s rank correlation.\n3.\tThe reported results in Table 1 are not clear. For example, why the results of baseline (cos+CE) and of CIM is much lower than that in Table 1 of [1] (lower than 10~20 points)?\n4.\tThe comparison methods (e.g. CAN and ConstellationNet) in Table 2 and 3 are not state-of-the-art methods as the paper claimed. Compared with more recent FSL methods (e.g. [2],[3],[4]) maybe more convincing to support the opinion that proposed method outperforms the state-of-the-art methods.\n\n[1] Channel importance matters in few-shot image classification. ICML 2022.\n[2] Alleviating the Sample Selection Bias in Few-shot Learning by Removing Projection to the Centroid. NeurIPS 2022.\n[3] Improving Task-Specific Generalization in Few-Shot Learning via Adaptive Vicinal Risk Minimization. NeurIPS 2022.\n[4] Matching Feature Sets for Few-Shot Image Classification. CVPR 2022.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Q1: The proposed Kendall’s rank correlation is constrained with FSL methods that uses similarity metric to determine the similarity between two images, which limits the application scope of it.\n\nA1: Thank you for your feedback. In fact, research in few-shot learning can be broadly categorized into metric-learning-based and optimization-based methods, with **metric-learning-based methods constituting a significant portion**. **Similarity measurement is an essential component of metric-learning-based approaches**. Our proposed method holds the potential to seamlessly integrate with this category by replacing cosine similarity with Kendall's rank correlation. On the other hand, as our method leverages Kendall's Rank Correlation to achieve consistency across feature channels, it can be easily integrated with those methods that address few-shot learning from other perspectives. \n\nMoreover, **the applicability of our method extends beyond the realm of few-shot learning**, as its underlying motivation is from observing an apparent difference in feature channel values between base data and novel data. We found that compared to base classes, when the feature extractor faces a novel class that is unseen before, most features' channels have small and closely clustered values, making it difficult for the model to distinguish the importance of individual channels. At this point, channel importance ranking can effectively accentuate the differences between channels. This property is applicable not only to few-shot learning but also to any task with cross-domain generalization attributes and holds inspiration. \n\nBased on the above aspects, we are confident that our method enjoys a broad application scope, far beyond the limitations initially perceived.\n\nQ2: The comparison in Section 4.2 is not convincing. The authors only consider a baseline that uses cosine similarity metric, which is not a very popular design for many FSL methods. Compared with more baselines using other similarity metrics (like Square Euclidean Distance in ProtoNet and a learnable metric in RelationNet) may help me to make sure the effectiveness and generality of proposed Kendall’s rank correlation.\n\nA2: Thanks for the comments. The **learnable metric-based methods** require meta-learning to train and obtain the optimal parameter values. However, in Section 4.2, we aimed to demonstrate that our approach does not require any training and solely relies on using Kendall rank correlation in the inference phase to achieve noticeable performance improvement. Additionally, the reason why we did not include the **Euclidean distance** in our experiments is that on the unit sphere, cosine similarity and Euclidean distance are equivalent. Furthermore, in few-shot learning, only ProtoNet adopts Euclidean distance, while many subsequent works are based on cosine similarity and have shown that using cosine similarity generally yields better results compared to using Euclidean distance. \n\nNonetheless, we also **conduct experiments using the learnable metric in RelationNet and Euclidean distance** (See Table B of the attached PDF in our \"global\" response). The experimental results demonstrate the superiority of utilizing Kendall's rank correlation over the learnable metric in RelationNet and Euclidean distance.\n\nQ3: The reported results in Table 1 are not clear. For example, why the results of baseline (cos+CE) and of CIM is much lower than that in Table 1 of [1] (lower than 10~20 points)? \n\nA3: Sorry for any confusion caused. In fact, CIM’s Table 1 reports results for the **5-way 5-shot** setting, while our Table 1 presents results for the **5-way 1-shot** setting. We also conduct experiments in the 5-way 5-shot setting (See Table D of the attached PDF in our \"global\" response). As you can see, in fact, our reimplementation outperforms the results reported in CIM's Table 1, and **the effectiveness of Kendall's rank correlation is demonstrated**.\n\nQ4: The comparison methods (e.g. CAN and ConstellationNet) in Table 2 and 3 are not state-of-the-art methods as the paper claimed. Compared with more recent FSL methods (e.g. [2],[3],[4]) maybe more convincing to support the opinion that proposed method outperforms the state-of-the-art methods.\n\nA4: Thank you for bringing this to our attention. We will ensure the inclusion of the methods you've highlighted in our list of citations. It is pertinent to highlight that our latest experimental results encompass comparisons with more recent FSL methods, including those you've referenced (Please refer to Table A in the PDF in our \"global\" response). \nWe would like to highlight that by integrating Kendall's Rank Correlation into a stronger backbone DeepEMD, **our method achieves SOTA performance**.\n"}]

Author rebuttal:
We sincerely appreciate the valuable comments from all reviewers. We have endeavored to provide explanations for the questions raised in the respective comments section. 

Additionally, the supplementary experiments have been incorporated into the PDF attached to the global response. Specifically, the appended PDF encompasses the following:
1. **Comparison with state-of-the-art methods** when integrating our approach with DeepEMD and InfoPatch. This provides evidence that when integrating our method with stronger baselines, it is capable of achieving state-of-the-art (SOTA) performance.
2. **Comparison among Kendall's rank correlation, distances other than cosine similarity, and simple feature scaling transformation modules.** This highlights the superiority of Kendall's rank correlation over alternative distances in few-shot learning, while also confirming that this improvement cannot be surpassed by mere simple feature scaling methods.
3. **Comparison of the variance of feature channel values between the base dataset and various novel datasets.** This validates the widely applicable conclusion that features' channel values exhibit a high degree of clustering and similarity for novel classes unseen by the model.
4. **Additional experiments of Kendall's rank correlation on 5-way 5-shot tasks** across multiple datasets with diverse domain variations.

Meta-review: This paper obtained diverging reviews: 1 weak accept, 1 borderline accept, 2 borderline reject, and 1 reject. While two reviewers appreciated the simple yet effective idea and the consistent improvement over standard methods, the other reviewers raised concerns about limited novelty, unclear motivation, lack of theoretical justification, and missing comparisons/analyses. The authors provided very detailed rebuttals, addressing most of them with additional experiments/analyses. Despite the authors’ constant rebuttals and requests for discussion, some reviewers remain negative without responding to the authors. AC thus carefully examined the reviews, the rebuttal, and the discussion. Checking the details, AC finds that the raised concerns are mostly resolved. For example, the authors clarified their motivation based on the observation about feature channel distributions, explained the difference of the proposed Kendall's rank correlation from DeepEMD, and provided SOTA results when combined with DeepEMD. The lack of theoretical motivation may be a drawback, but the authors’ empirical justification is intuitive and reasonable enough. In particular, AC agrees with reviewer 5zcq that the differentiable version of Kendall’s rank correlation as an alternative similarity metric is an original idea and can be very useful for metric-based few-shot classification and beyond. Therefore, AC recommends acceptance.
Decision: Accept (poster)