Improving RENet by Introducing Modified Cross Attention for Few-Shot Classification
Authors: Ching-Han Chang
Tian-Li Yu
Venue: NeurIPS 2022 Submitted (2022)
Keywords: few-shot classification
attention

Abstract: Few-shot classification is challenging since the goal is to classify unlabeled samples with very few labeled samples provided. It has been shown that cross attention helps generate more discriminative features for few-shot learning. This paper extends the idea and proposes two cross attention modules, namely the cross scaled attention (CSA) and the cross aligned attention (CAA). Specifically, CSA scales different feature maps to make them better matched, and CAA adopts the principal component analysis to further align features from different images. Experiments showed that both CSA and CAA achieve consistent improvements over state-of-the-art methods on four widely used few-shot classification benchmark datasets, miniImageNet, tieredImageNet, CIFAR-FS, and CUB-200-2011, while CSA is slightly faster and CAA achieves higher accuracies.

Improving RENet by Introducing Modified Cross Attention for Few-Shot Classification
Anonymous Author(s)
Affiliation
Address
email
Abstract
Few-shot classification challenging since the goal classify unlabeled samples with very few labeled samples provided. It has been shown that cross attention helps generate more discriminative features for few-shot learning. This paper extends the idea and proposes two cross attention modules, namely the cross scaled attention (CSA) and the cross aligned attention (CAA). Specifically, CSA scales different feature maps to make them better matched, and CAA adopts the principal component analysis to further align features from different images. Experiments showed that both CSA and CAA achieve consistent improvements over state-of- the-art methods on four widely used few-shot classification benchmark datasets, miniImageNet, tieredImageNet, CIFAR-FS, and CUB-200-2011, while CSA is slightly faster and CAA achieves higher accuracies.
5
7
11
Introduction
12
Few-shot classification has drawn lots of attentions in recent years [52 It originates from the observation that humans can learn new concepts with very few samples, and the goal is to classify unseen query samples given very few support samples. One may consider fine-tuning pre-trained 16 model using the labeled samples from the unseen classes; however, this usually causes severe overfitting, which can be alleviated by regularization and data augmentation but cannot be fully solved.
14
18
The meta-learning [17 has been widely used for few-shot learning recently. However, they usually 20 do not focus enough on relevant features as shown in Figure 1, taking the prototypical network [46 for example, and those irrelevant features causes the limitation of generalization to the unseen classes. The cross attention network (CAN) 18 and the relational embedding network (RENet) [00 remedy the above issue by proposing the cross attention. It has been shown that humans tend to locate
22
24 the most relevant regions in the pair of labeled and unlabeled samples first to recognize a sample from unseen class given few labeled samples [18]. Inspired by that, CAN and RENet generate the attention maps across the support class features and the query sample features to make the network attends more the target object regions.
26
27
28 this work, we make improvements for RENet by further enhancing feature discriminability for few-shot classification. We propose the cross scaled attention (CSA) and the cross aligned attention (CAA). CSA seales different feature maps to make them better matched. CAA further considers the alignment issue between different images by adopting the principal component analysis (PCA).
30
32
Our main contributions are as follows:
33
We propose two cross attention modules, CSA and CAA, to improve RENet.
34
Both proposed modules surpass the results state-of-the-art methods on minilmageNet, NieredlmageNet, CIFAR-FS, and 2UB-200-2011.
Submitted to 36th Conference on Neural Information Processing Systems (NeurIPS 2022). Do not distribute.

---

Figure 1. An example of the class activation maps 59 of an image (left) of an existing method 44 (middle) and our method (right). The warmer color indicates the higher value.
36
.
CSA is slightly faster than CAA, while CAA achieves higher accuracies than CSA. Users can choose the one suitable for their needs.
37
36
The remaining of this paper is organized as follows. Section 2 provides background knowledge highly related to this work. Section 3 presents our approaches. Section 4 shows the experiment results. Section concludes this work.
39
40
Related Work
41
Few-Shot Classification
Few-shot classification can be categorized into three groups, optimization-
43
parameler--ennerating basti methods [3] 32 33 and metric- 20 Optimization-base methods learn to update model parameters by designing the meta-learner as an optimizer. To adapt to new tasks efficiently for the learner, it methods predict parameters by designing the
based methods [1 29
10
44
47
44
based methods 46
48
50
18
58
45
46
learn good initialization. sarameter-generating--oe network. Mltric-based methods learn an embedding function that maps images to metric space such that the relevance between images is distinguished based on a distance metric.
47
meta-learner
48
49
Our method belongs to metric-based methods. The prototypical network [46], CAN 18), and RENet 20 are highly related our work. Following CAN and RENet, exploit the relation
50
51
between the support set and query set. However, the prototypical network extracts the support and the query features independently which makes the model distracted by irrelevant features. The cross attention network improves the performance by using an attention network to refine features, which makes the model focus on the relevant regions. RENet further improves the performance by 55 integrating a module that matchs the features in an image itself. Inspired by these works, we follow
52
59
54
56 some of their structures and integrate module that matchs the features between the support and the query images.
57
58
RENet We follow the structure of RENet 20 and integrate our module RENet. the self- correlational representation (SCR) and the correlatall attention (CCA) are proposed in RENet. SCR exploits the sliding window and the dilation to match the features an image itself. CCA computes the cosine similarity between the support and the query images and generate attention maps. We consider the cross attention between the support and the query images by exploiting the sliding window and the di dilation, which is similar to SCR. In addition to matching the features between the support and the query images, we also deal with the scaling and the alignment issues.
59
60
61
62
64
Approach
65
86
The network that addresses the challenge of generalization to unseen target classes presented in this section. The overall structure is composed of five modules: an embedding module, SCR, CCA, and CSA/CAA, and a classification module. The embedding module extracts features of the input image. It consists of several cascaded convolutional layers, mapping an input image into feature map. We use the ResNet-12 12 [16] network as our embedding module, which identical CAN [88] and RENet (20]. Following the prototypical network [46), CAN, and RENet, the support feature of
87
88
69
70
71
72 a class is defined as the mean of its support set in the embedding space. The embedding module
takes the support set and query sample as inputs and produces the support feature map 2, and
74 query feature map Zy. Each pair of feature maps (Z, and 7,,) are then fed through SCR, CCA, and CSA/CAA, which highlight the relevant regions and output more discriminative feature pairs (s and

---

RENet
exhXw
2
X w
F
A₂
Self-
Cross-
Cross Scaled Attension or Cross Aligned Attention
Correlational
Correlational Atlention
Representation
F
Vg
x ax
x x
Figure 2: The overall architecture.
78
9) for classification. We first present a brief definition of the problem and a concise overview of the proposed architecture in Section 3.1 and Section 3.2 respectively. We then present technical details of CSA and CAA in Section 3.3 and describe our training objective in Section 3.4.
78
3.1
Problem Definition
80
The datasets for few-shot classification are split into the training set and the testing set, and each of them are further split into the support and the query The support contains few labeled samples and the query set contains unlabeled samples. Given the support set, few-shot classification aims correctly classify the query set. The problem is called N-way K-shot if the support is composed of N classes and K labeled samples per class.
82
83
84
Because deep neural networks vulnerable to overfitting with few labeled samples [20), most few-shot classification methods adopt meta- meta-learning framework with episodic training. Following them, we adopt the episodic training mechanism, which has been shown effective for few-shot learning
86
88
46
50
45
15
3.2
Architecture Overview
90
The overall architecture illustrated in Figure 2. For each pair of support classes and query samples, we obtain proper feature representations. The network can model and exploit the semantic relevance between the support feature and query feature. Our approach is different from many previous methods which extract the support and the query features independently. We resort to metric learning this work. To be helpful to the subsequent matching, we integrate attention to the features.
91
82
93
94
The support feature map Z3 Rexxhxx is extracted from the support samples and the query feature map Zy E Rexhxw is extracted from the query sample, where c, h, and w denote the number of channels, height, and width of the feature maps respectively. The network generates attention maps for the input pair, which is then used to weight the feature map to achieve more discriminative feature and q. The architecture in Figure consists of three main learnable modules: SCR, CCA, and CSA or CAA. Since SCR and CCA have already proposed in we start our description from CSA and CAA. More detail can refer (0
96
88
representation, and the final outputs
100
101
RENet 20
102
3.3
Cross Scaled Attention (CSA) and Cross Aligned Attention (CAA)
103
Figure illustrates the structure of CSA and CAA. Inspired by SCR 20), We propose two similar modules CSA and CAA. SCR only considers about the correlation in the image itself, and we further think about the correlation between the support and the query images. CCA (20 also consider the correlation between the support and the query images. It computes the cosine similarity between the
104
105
106
107
support and the query images and generate attention maps. On the other hand, CSA and CAA match
the features between the support and the query images by computing the Hadamard product. They further help our model focus on more important features. SCR focuses on the target object in an image, and CSA and CAA focus the target objects in both the support and the query images. The
109

---

1.
Let scaling factor s the portion of pisels where attention values are higher than the average the whole attention map.
Filter all pixals with the everage of the whole ottenton map, If ther ettenson valuee are higher, keep the piali Othervice, diccard thom. Find the first and the second principal componeetts by PCA, Le! them be el and e2 respectively. Rotala the image with angle where @rarcsos(e Paarcossef--u//alle (ltl
comolation computation
lt x xisargmax A(x') as the center of the cropped image.
uf=t1,
1.. CCCC)
the scaling factors be sf=je1] 62#(e2) respectively. Lot xarggmax Air)
(H, (K.WUU.V.CC)
Crop image with shw sth. Compute the Hadamard procuct.
of alopped
Convit .13.3),, CIaC)
Grop with size for edh. Compute the Hadamard product.
Conven 1.111 CC
(..0))
squesce
(K.WC))
0)
CSA
GAA
Figure 3: The structure of CSA and CAA.
111
structure is almost identical to SCR, and the only difference is the input. Similar to CCA, CSA and CAA take an input pair of support and query, Y, and Yg. and produces the final embeddings, s and g.
112
113
Correlation computation Similar to SCR, we exploit the sliding window and the dilation to match the features. However, instead of matching the features of each position and its neighborhood which is presented in SCR, we match the features between the support and the query. The Hadamard product of
115
vector at each position [1, H) x [1, W] and vectors at the neighborhood x' E [1, H] x [1, W] is computed and collected into a cross-correlation tensor R. We represent the tensor R as a function
117
with vector output:
R(x,x',)) = Yx(x) Yg(x' +p [x]
+p
119
where p E (-dyy.d dul x d-dy, d It corresponds to a relative position in the neighborhood window such that 2dg - 1 = U and 2dy + 1 = V. which includes the center position. The edges of the feature map are zero- prrro-padedddeee for sampling off the edges.
120
121
122 To make the training process more efficient, we do not iterate through the whole image for x'. We take the position attention map produced previously that has maximum value the center and crop the image to find the scope of the target object, where the region we iterate through. For CSA, the size of the cropped region s'w w x s* h, where w and h are the width and the height of the image respectively, and s* is the scaling factor which is the portion of pixels where attention values are higher than the average of the whole attention map. If we change the scaling factor, which is the only difference between CSA and CAA, the module will become CAA. We elaborate on how we tune the scaling factor in the next two paragraphs.
123
124
125
126
127
128
129
130
Cross scaled attention (CSA) In this paragraph, we first elaborate on how we tune the scaling factor. In general, we have two scaling factors, 81 and 82, to crop the image with size s1 x s2h, where and h are the width and the height of the image respectively. We tune the scaling factors 81 and 82 in three different ways. Firstly, we let 8* = 81 = 82 and fix 8" to 0.5. Secondly, we let s* be the portion of pixels where attention values are higher than the average of the whole attention
131
135
135 map, which is the scaling factor adopted in CSA. Finally, we adopt PCA to determine the scaling factors, and this is what CAA does. We find out that the first method achieves the lowest accuracy, so
137
we regard it as a bascline. CAA achieves higher accuracies than CSA, but its training time is longer compared to CSA. The results of different methods are presented in Section 4.2.
138
139
The scaling factor s* for CSA is obtained by the following equation:
N
A'
140 where N is the number of pixels with attention values higher than the average of the attention map, 141 and A is the is the total number of pixels.
142 Cross aligned attention (CAA) We first filter all the pixels with the threshold of the average value of the whole attention map. If the attention values of the pixels are lower than the threshold, we

---

support
query1
queryz
without CSA/CAA
CSA
CAA
Figure 4. The class activation maps 59 of the support image (top) and the query images (middle and bottom). In CAA, the target objects in the support and the query images are aligned after rotation. The warmer color indicates the higher value.
144
will discard them. PCA is then conducted to find the first and the second principal components, and the image rotated with the angle where is the angle between the first principal component and the horizontal line. We crop the image with size $ W X s2h centered at the position where the attention value is maximum, where 81 and 82 are the magnitudes of the first and the second principal components respectively, and w and h are the width and the height of the image respectively.
145
148
147
148
149
Consider data matrix X with column-wise zero empirical means, which indicates that the sample mean of each column has been shifted to zero. The transformed is defined by set of coefficient vectors v. and each coefficient vector is constrained to be unit vector. To maximize variance, the first coefficient vector 11 has to satisfy the following equation:
150
152
Tx
argmaz
(3)
153 With ei found, the first principal component is X0yu7.
The second principal component e2 Xeaun" be found by the second coefficient vector 02- 02 can be found by the following equations:
155
x= X - X XUIUi
(4)
156
X
02 argmaa
(5)
TT
157
The rotation angle can be derived from the following equation:
= cos 1
61 U1
(6)
||e||||uz||
where un is the horizontal unit vector (1, 0).
159 As shown in Figure 4, CAA aligns the target objects in the support and the query images since we
rotate the image to help us match the features of the target objects in both images.
161 Cross attention learning A series of 2D convolutions applied to analyze the self-correlation patterns in R. For computational efficiency, the convolutional block follows a bottleneck structure as shown in Figure consists of point-wise convolution layer for channel size reduction, two 3 x 3 convolution layers for transformation, and another point-wise convolution layer for channel size
163
5

---

185
recovery. We insert batch normalization and ReLU between the convolutions. The spatial dimensions of local correlation patterns are reduced from U V x such that the output g(R) and Y, (Y₂) has the same size since they are gradually aggregated by the convolution block g()) without padding. The process of analyzing structure patterns could be complementary to appearance patterns in the representation Y, (Y2). Therefore, we combine the two representations to produce the final
186
168
169
170
embeddings and q:
g R) + Ys: q -= g (R) + Yg
171
(8)
172
which reinforces the base features with relational features and helps the few-shot learner focus on the target objects in the images.
173
3.4
Training and Testing (Inference)
175
Training Following [18] and [20]. we train the network via minimizing the classification loss on the query samples of the training set. The classification module is composed of the nearest neighbor classifier and a global classifier. The nearest neighbor classifier classifies the query samples into N support classes based on pre-defined similarity measures. Each position in the query feature maps constrained to be correctly classified to obtaín precise attention maps. We define the nearest neighbor classification loss L1 as the negative log- proobabbbyllty according to the true class label. A fully connected layer followed by softmax to classify each query sample among all available training classes is used in the global classifier. We compute the global classification loss L2. Finally, define the overall classification loss as I. = AL, + L2, where À the weight to balance the effects of different losses. We train the network end-to-end by optimizing L with the stochastic gradient descent algorithm.
177
179
181
182
189
185
Testing (Inference) Many existing methods including the prototypical network [46] and RENet (2) use the inductive inference. The global average pooling is performed to the features to get the mean support and query features. The label for query sample is predicted by finding the class which has the nearest mean support feature under a distance metric.
187
189
190
However, each class has very few labeled samples in few-shot classification task, so the support features of classes can hardly represent the true class distribution. To alleviate the problem, [18] proposed a simple and effective transductive inference algorithm that utilizes the unlabeled query samples to enrich the support features of classes.
191
192
193
194
Following [8], we use the transductive inference. In this way, the support features of classes can be more representative and robust. Experiment shows that the transductive inference achieves higher performance than the inductive inference especially in s-ohot where the problem described above is more serious.
195
196
197
4
Experiment Results
198
199
4.1 Experiment Setup
Datasets
We use four standard benchmarks for classffctttonn for evaluation: minilmageNet, fieredImageNet, CIFAR-FS, and CUB-200-2011 (Caltech-UCSD Birds-200-2011),
201
minilmageNet 50 is subset of ImageNet (ILSVRC-2012) [21] which consists of 60000 images. It contains 100 object classes with 600 images per class. These classes are randomly split into 64, 16, and 20 classes for training, validation, and testing respectively. All images are of size 84 x 84.
203
205
208
tieredImageNet [42] is much larger subset of ImageNet (ILSVRC-2012) [21]. It contains 608 classes grouped into 34 high-level categories. These are divided into 20, 6, and 8 categories for training, validation, and testing respectively, which corresponds to 351, 97, and 160 classes for training, validation, and testing respectively. All images are of size 84 x 84.
207
208
209
210
CIFAR-FS 2 is subset of CIFAR-100 which consists of 60000 images. It contains 100 object classes with 600 images per class. These classes are randomly split into 64, 16, and 20 classes for training, validation, and testing respectively. All images are of size 32 x 32.
212

---

214
LUB-200-2011 [53] is an image dataset with photos of 200 bird species (mostly North American). It consists of 100, 50, and 50 classes for training, validation, and testing respectively.
215
216
217
Experiment setting We conduct experiments for our approach on 5-way 1-shot and 5-way 5-shot settings. For an N-way K-shot setting, we form the episode with N classes and each class includes K support samples. We use 15 query samples per class in an episode for both training and testing. We randomly sample 2000 episodes from the testing set when testing. The average accuracy and the corresponding 95% confidence interval are reported over the 2000 episodes.
219
221
Implementation details We use Pytorch to implement all our experiments one NVIDIA RTX- 3080 GPU. The ResNet-12 [16] network is used as our embedding module. The input images size is 84 x 84 for minilmageNet and tieredImageNet. and 32 x 32 for CIFAR-FS. Horizontal flip, random crop, and random erasing are adopted as data augmentation during training. We use SGD as the optimizer. Each batch contains 8 episodes. For minilmageNet, CIFAR-FS, and CUB-200-200 model is trained for 90 epochs, with each epoch consisting of 1200 episodes, and the initial learning rate is 0.1 and decreased to 0.006, 0.0012, and 0.00024 60, 70, and 80 epochs, respectively. For magred the model is trained for 80 epochs, with each epoch consisting of 13980 episodes, and the initial learning rate is set to 0.1 with decay factor of 0.1 at every 20 epochs. We set the and 5 otherwise, and we set the weight hyperparameter (A) in the overall loss function 0.25, 0.5, and 1.5 for ImageNet derivatives, CIFAR- FS, and CUB-200-2011 I respectively. We set U=5 and V=5 in our experiment. We cross-validate all
223
225
the
227
229
230
231
lemperature hyperparameter (20) to 2 for
232
233
234
typrraarameters in the validation sets and fix them afterward in all experiments.
235
Comparison with state-of-the-an methods Table I shows the comparison between our method and existing few-shot method on minilmageNet, tieredImageNet, CIFAR-FS, and CUB-200-2011 1. All results in Table 1 except our work are directly adopted from their papers. "_" indicates the results are not available in their papers. Many existing methods extract features of support and query samples independently, making the features focus on the non-target objects. To avoid the issue, CAN [18).
236
237
238
239
240
RENet (2),, and our method highlights the target object regions and gets more discriminative features instead. Compared to CAN and RENet, our method achieves higher accuracies.
242
4.2
Ablation Study
243
We show the effectiveness of each component of the network by empirical results and compare the time cost in this subsection. In [18] and 120, a series of experiments in their ablation study has already been completed. Following them, we experiment on minilmageNet in this subsection. We show the effectiveness of CAA and compare the performances of CAA and CSA. We firstly introduce a baseline to be used for comparison. If we remove SCR, CCA, and CSA/CAA, the model almost become the prototypical network [46] with 12 [16 as the backbone, and the only difference is a global classifier. Therefore, we create a variant named R12 R12-proto by removing SCR, CCA, and CSA/CAA. In R R12-proto. the features from the embedding module are directly fed to the nearest neighbor and global classifier, and the model is trained with the joint of global and nearest neighbor classification loss. The comparison between all variants shown in Table 2. Time cost is shown in Table 3.
244
245
247
249
251
253
Influence of SCR, CCA, and CSA/CAA By comparing RENet+CSA/CAA and R12-proto, observe consistent improvements on both 1-shot and 5-shot scenarios as shown in Table 2. The reason is that when using SCR, CCA, and CSA/CAA, our model can highlight the relevant regions and extract more discriminative features. The performance gap shows that (1) conventionally independently extracted features tend focus on the non-target regions and produce inaccurate similarities. (2)
255
257
258
259
SCR, CCA, and CSA/CAA can help to highlight target regions and reduce such inaccuracy. As shown
260
in Table 2, RENe:+CSA/CAA outperforms R 12p-rooto consistently, which further demonstrates the effectiveness of the attention mechanism.
281
262
Influence of CSA and CAA To verify the effectiveness of CSA and CAA, we test another variant without the modules. We remove the component of CSA/CAA. That is, after we get the feature maps
283
We remimeeemnn the prototypical network with ResNet- 12 as the backbone in Table 1.
7

---

Table Performance comparison in terms of accuracy (%) with 95% confidence intervals on 5-way classification on (a) minilmageNet and tieredImageNet and (b) CIFAR-FS and SUB-200-2011
(a) Results on manilmageNet and tieredI datasets.
minilmageNet
IieredImageNet
Model
Backbone
1-shot
5-shot
1-shot
5-shot
MAML [10]
ConvNet
48.70 0.84
55.31
0.73
51.67 1.81
70.30 1.75
cosine classifier
ResNet-12
55.43 + 0.81
77.18 + 0.61
61.49 + 0.91
82.37 + 0.67
MTL 44
ResNet-12
61.20 + 1.80
75,50 - 0.80
TADAM [36
ResNet-12
58.50
76.70 0.30
PPA [39
WRN-28-10
59.60
7.194 0.41
65.65 0.92
83.40 + 0.65
wDAE-GNN
15
WRN-28-10
61.07 0.15
76.75 + 0.11
68.18 0.16
83.09 + 0.12
SimpleShot [51
ResNet-18
02.85
80.02
TPN 28
ResNet-11
59.46
75.65
59.91 0.94
73.30 + 0.75
RFS-simple
49
ResNet-12
0.66
79.64 0.44
19.74:0.77
84.41 +: 0.55
LEO 44
WRN-28-10
61.76
77.59 0.12
66.33 + 0.05
81.44 + 0.09
MetaOpt 22
ResNet-12
6.66
78.63 + 0,46
65.99 + 0.72
81.56 + 0.53
adaNet
33
ResNet-12
71.94 0.62
0.57
DC 226
ResNet-18
0.19
79.77 0.19
Shot-Free
ResNet-12
59.04
77.64
63.52
82.59
S2M2
30
ResNet-34
79.45 0.12 6.114
MN
50
ConvNet
60.60 0.71 4.77
MN
50
ResNet-12
0.80
75.99 + 0.60
68.50 0.92
80.60 + 0.71
RN
ConvNet
65.32 0.8
0.70
54.48 0.93
71.32 + 0.78
PN
46
ConvNet
68.20 0.66 4.78
53.31 + 0.89
72.69 1 0.74
PN
ResNet-12
6..46
73.65 0.37
64.56 + 0.56
76.78 + 0.43
MegMargin
27
ResNet-12
0.8
81.57 + 0.56
CTM
ResNet-18
80.51 1 0.13 0.2
68.41 0.39
84.28 + 1.73
FEAT [56
ResNet-12
66.78
82.05 = 0.14
70.80 + 0.23
84.79 0.16
DccpEMD 58
ResNet-12
82.41 1 0.56 0.82
71.16 0.87
86.03 - 0.58
CAN 18
ResNet-12
0.48
1
69.89 0.51
CAN+T 18
ResNet-11
0.35 80.64 0.55
73.21 + 0.58
84.93 + 0.38
RENet 20
ResNet-12
67.60
82.58 - 0.30
71.61 + 0.51
85.28 + 0.35
RENet+CSA (ours)
ResNet-12
73.18 0.51
84.20 0.31
75.58
0.57
85.74 + 0.39
RENet+CAA (ours)
ResNet-12
73.61 + 0.51
84.43 0.30
76.71 0.55
86.38 + 0.35
(b) Results CIFAR-FS and CUB-200-2011 datasets.
CIFAR-F'
CUB-200-2011
Model
Backbone
I-shot
5-shot
1-shot
5-shot
MAML.
10
ConvNet
58.9 1.9
71.5 1.0
MAMI. 10
ResNet-34
67.28 + 1.08
83.47 + 0.59
cosine classifier
ResNet-12
67.30 0.86
84.75 - 0.60
cosine classifier
8
ResNet-34
60.39 0.28
72.85 + 0.65
MetaOpt
22
ResNet-12
7.77
84.3 0.5
Shot-Free
41
ResNet-12
69.2
84.7
RFS-simple
49
ResNet-12 7.85
86.0 0.5
NegMargin
27
ResNet-18
72,66 + 0.85
89.40 + 0.43 86.55 - 0.61
S2M2
30
ResNet-34
62.77
75.75 0.13
72.92 0.83
Boosting
13
WRN-28-10 0.3
86.0 0.2
FEAT 56
ResNet-12
73.27 4 0.22
85.77 + 0.14
MN
150
ResNet-
71.87 + 0.85
85.08 + 0.57
RN
48
ConvNet
55,0 1.0
69.3 + 0.8
RN 48
ResNet-34
66.20
82.30 + 0.58
PN 46
ResNet-12
70.21 + 0.52
80.60 + 0.40
66.09 0.92
82.50 + 0.58
DeepEMD
58
ResNet-12
75.65 0.83
88.69 + 0.50
CAN 18
ResNet-12
71.65 +
83.72 + 0.38
CAN+T
ResNet-12
76.61 0.56
84.37 0.38
RENet 20
ResNet-12
74.51 + 0.46
86.60 0.32
79,49 + 0.44
91.11 + 0.24
CENet+CSA (ours)
ResNet-12
80.02 0.51
87.63 0.33
85.89 0.45
92.03 0.25
RENet+CAA (ours)
ResNet-12
80.40 + 0.50
87.76 + 0.33
86.63 + 0.44
92.88 + 0.22

---

Table 2: Ablation study on minilmageNet with performance comparison in terms of accuracy (%).
Variant
5-way I-shot
5-way 5-shot
R12-proto
66.36
75.65
without CSA/CAA
72.81
83.84
unscaled
72.91
84.06
with CSA
73.18
84.20
with CAA
73.61
84.43
Table 3: Time four datasets. All models implemented in PyTorch and tested on Nvidia RTX-3080.
Time
Model
minilmageNet
tieredImageNet
CIFAR-FS
CUB-200-2011
RENet
5 m
48 h 05 m
4 m
1 08 m
training
CEN
5 52 m
48 h 45 m
5 55 m
1 10 m
RENet+CAA
6h 16
h 17 m
6 60200m
1 15 m
RENet
2 2 06
2 10
2 m
2 m 03
inference
RENet+CSA
2 2 08
2 11
2 2 09
04
RENe++CAA
2 2m 12
2 2m 15
2 2m 14
2 m 08 s
Y. and Yg from CCA, the features fed the nearest neighbor and global classifier, and the model 285 is trained with the joint of global and nearest neighbor classification loss. As shown in Table 2, both the network with CSA and the network with CAA outperform the variant model. The improvement indicates that CSA and CAA can help to highlight target regions more effectively compared to the model without CSA/CAA.
288
267
268
289
Influence of PCA Using PCA achieves higher accuracies for our model compared to other methods
270
mentioned in Section 3.3. We compare the performances the unscaled version. CSA, and CAA in
Table 2. As described in Section 3.3, the unscaled version is the variant whose scaling factor is fixed 0.5. As shown in Table 2, CAA achieves the highest accuracy, and we conclude that the alignment of the target objects in the support and the query images benefits classification.
272
274
Speed comparison We compare the training time and inference time of RENet, RENe++CSA, and RENet+CAA in Table 3. As can be seen, the training time of RENet+CSA is slightly longer than RENet, and the training time of RENe++CAA slightly longer than RENe++CSA but not by much. The inference time of RENe++CSA is longer than RENet, and the training time of RENet+CAA is longer than RENet+CSA, but the differences are so slim that they are practically insignificant.
275
276
278
Conclusion
279
This work improves RENet for few-shot classification by introducing two cross attention modules, CSA and CAA, which model the semantic relevance between the support and query features. Specifically, CSA scales different feature maps to make them better matched, and CAA adopts the principal component analysis to further align features from different images. As result, the proposed modules focus more relevant regions by considering both the support and the query images rather than only the latter ones. Empirically, RENet with both CSA and CAA outperformed ofte--of-the-rr methods minilmageNet, tieredImageNet. CIFAR-FS, and CUB-200-201 four widely used datasets for few-shot learning, in terms of accuracy. The ablation study further verified that the improvements are achieved owing to the proposed modules.
281
283
285
287
289
Our work indicated that few-show learning information contained in those few support samples should be exploited as much possible, and the cross attention is one such way to do it. Although
291
such techniques may require slightly longer training time, we believe that it worthwhile especially
in the scenarios where labeled data are valuable and few.

---

References
293
294
[1] Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W. Hoffman, David Pfau, Tom Schaul, Brendan Shillingford, and Nando de Freitas. Learning to learn by gradient descent by gradient descent, 2016.
296
297
[2] Luca Bertinetto, Joao F. Henriques, Philip Torr, and Andrea Vedaldi. Meta-learning with differentiable closed-form solvers. International Conference on Learning Representations. 2019.
296
299
[3] Luca Bertinetto, João F. Henriques, Jack Valmadre, Philip H. S. Torr, and Andrea Vedaldi. Learning feed-forward one-shot learners, 2016.
301
[4] Wieland Brendel and Matthias Bethge, Approximating cnns with hag-of-local-featrre models works surprisingly well on imagenet, 2019.
303
304
[5] Qi Cai, Yingwei Pan, Ting Yao, Chenggang Yan, and Tao Mei. Memory matching networks for one-shot image recognition, 2018.
305
306
[6]
Kaidi Cao, Maria Brbic, and Jure Leskovee. Concept learners for few-shot learning, 2021.
307
[7] Long Chen, Hanwang Zhang, Jun Xiao, Liqiang Nie, Jian Shao, Wei Liu, and Tat-Seng Chua. Sca-cnn: Spatial and channel-wise attention in convolutional networks for image captioning, 2017.
308
309
[8]
We Chen, Yen-Cheng Liu, Zsolt Kira, Yu-Chiang Wang, and Bia-Bin Huang. A closer look at few-shot classification. In International Conference on Learning Representations, 2019.
311
312
[9] Carl Doersch, Ankush Gupta, and Andrew Zisserman. Crosstransformerr: spatially-aware few-shot transfer, 2021.
313
314
[10] Chelsea Finn, Picter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adapta- tion of deep networks, 2017.
316
[11] Peng Gao, Pan Lu, Hongsheng Li, Shuang Li, Yikang Li, Steven Hoi, and Xiaogang Wang. Question-guided hybrid convolution for visual question answering, 2018.
317
[12] Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. Imagenet trained enns are biased towards texture; increasing shape bias improves accuracy and robustness, 2019.
319
321
[13] Spyros Gidaris, Andrei Bursuc, Nikos Komodakis, Patrick Pérez, and Matthieu Cord. Boosting few-shot visual learning with supervuperi In Proceedings of the IEEE International Conference on Computer Vision, 2019.
322
323
324
[14] Spyros Gidaris and Nikos Komodakis. Dynamic few-shot visual learning without forgetting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4367-4375, 2018
326
327
[15] Spyros Gidaris and Nikos Komodakis. Generating classification weights with gnn denoising autoencoders for few-shot learning, 2019.
328
329
166] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for im- age recognition. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), June 2016.
331
332
[17] Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. Meta-learning in neural networks: A survey, 2020.
333
334
[18] Ruibing Hou, Hong Chang, Bingpeng Ma, Shiguang Shan, and Xilin Chen. Cross attention network for few-shot classification. In NeurIPS, 2019.
335
396
[19] Jie Hu, Li Shen, Samuel Albanie, Gang Sun, and Enhua Wu. Squeeze-and-excitatoo networks, 2019.
10

---

338
[20] Dahyun Kang, Heeseung Kwon, Juhong Min, and Minsu Cho. Relational embedding for few-shot classification. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021.
339
[21] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In F. Pereira, C.J. Burges, L. Bottou, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 25. Curran Associates, Inc., 2012.
342
344
345
[22]
Kwonjoon Lee, Subhransu Maji, Avinash Ravichandran, and Stefano Soatto. Meta-learning with differentiable convex optimization. In CVPR, 2019.
347
[23] Hongyang Li, David Eigen, Samuel Dodge, Matthew Zeiler, and Xiaogang Wang. Finding Task-Relevant Features for Few-Shot Learning by Category Traversal. In CVPR, 2019.
348
349
[24] Wenbin Li, Lei Wang, Jinglin Xu, Jing Huo, Yang Gao, and Jiebo Luo. Revisiting local descriptor based image-to-class measure for few-shot learning. 2019.
350
351
[25] Zhenguo Li, Fengwei Zhou, Fei Chen, and Hang Li. Meta-sgd: Learning to learn quickly for few-shot learning, 2017.
[26]
Yann Lifchitz, Yannis Avrithis, Sylvaine Picard, and Andrei Bursuc. Dense classification and implanting for few-shot learning, 2019.
354
355
[27] Bin Liu, Yue Cao, Yutong Lin, Qi Li, Zheng Zhang, Mingsheng Long, and Han Hu. Negative margin matters: Understanding margin in few-shot classification, 2020.
357
[28] Yanbin Liu, Juho Lee, Minscop Park, Sachoon Kim. Eunho Yang. Sung Ju Hwang, and Yi Yang. Learning to propagate labels: Transductive propagation network for few-shot learning, 2019.
358
359
[29] Yaoyao Liu, Bernt Schiele, and Qianru Sun. An ensemble of epoch-wise empirical bayes for few-shot learning, 2020.
380
361
[30] Puneet Mangla, Nupur Kumari, Abhishek Sinha, Mayank Singh, Balaji Krishnamurthy, and Vineeth N Balasubramanian. Charting the right manifold: Manifold mixup for few-shot learning. In The IEEE Winter Conference on Applications of Computer Vision, pages 2218-2221 2020.
363
364
[3]] Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, and Pieter Abbeel. A simple neural attentive meta-learner, 2018
366
[32]
Tsendsuren Munkhdalai and Hong Yu. Meta networks, 2017.
367
[33] Tsendsuren Munkhdalai, Xingdi Yuan, Soroush Mehri, and Adam Trischler. Rapid adaptation with conditionally shifted neurons, 2018
388
369
[34] Alex Nichol, Joshua Achiam, and John Schulman. On first-order meta-learning algorithms,
370
2018.
371
[35] Avital Oliver, Augustus Odena, Colin Raffel, Ekin D. Cubuk, and Ian J. Goodfellow. Realistic evaluation of deep semi-supervised learning algorithms, 2019.
[36] Boris N. Oreshkin, Pau Rodriguez, and Alexandre Lacoste. Tadam: Task dependent adaptive metric for improved few-shot learning, 2019.
374
375
[37] Jongchan Park, Sanghyun Woo, JoonJ YoungY Lee, and In So Kweon. Bam: Bottleneck attention module, 2018
376
377
[38] Marco Pedersoli, Thomas Lucas, Cordelia Schmid, and Jakob Verbeek. Areas of attention for image captioning, 2017.
378
379
[39] Siyuan Qiao, Chenxi Liu, Wei Shen, and Alan L. Yuille. Few-shot image recognition by
380
predicting parameters from activations. In CVPR, 2018.
[40] Sachin Ravi and Hugo Larochelle. Optimization as a model for few-shot learning. In In International Conference on Learning Representations (ICLR), 2017.
381
11

---

383
[41] Avinash Ravichandran, Rahul Bhotika, and Stefano Soatto. Few-shot learning with embedded class models and shot-free meta training, 2020.
384
[42] Mengye Ren, Eleni Triantafillou, Sachin Ravi, Jake Snell, Kevin Swersky, Joshua B. Tenen- baum, Hugo Larochelle, and Richard S. Zemel. Meta-learning for semi-supervised few-shot classification. In Proceedings of 6th International Conference on Learning Representations ICLR, 2018.
386
388
389
[43] Mengye Ren, Eleni Triantafillou, Sachin Ravi, Jake Snell, Kevin Swersky, Joshua B. Tenen- baum, Hugo Larochelle, and Richard S. Zemel. Meta-learning for semi-supervised few-shot classification, 2018
390
391
[44] Andrei A. Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascanu, Simon Osindero, and Raia Hadsell. Meta-learning with latent embedding optimization, 2019.
393
[45] Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. One-shot learning with memory-augmente neural networks, 2016.
395
396
[46]
Jake Snell, Kevin Swersky, and Richard S. Zemel. Prototypical networks for few-shot learning. 2017.
397
398
[47] Qianru Sun, Yaoyao Liu, Tat-Seng Chua, and Bernt Schiele. Meta-transfer learning for few-shot learning, 2019.
400
[48] Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip H. S. Torr, and Timothy M. Hospedales. Learning to compare: Relation network for few-shot learning, 2018.
402
[49] Yonglong Tian, Yue Wang, Dilip Krishman, Joshua B. Tenenbaum. and Phillip Isola. Rethinking few-shot image classification: good embedding all you need?, 2020.
403
[50] Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Koray Kavukcuoglu, and Daan Wierstra. Matching networks for one shot learning, 2017.
405
[51] Yan Wang, Wei-Lun Chao, Kilian Q. Weinberger, and Laurens van der Maaten. Simpleshot: Revisiting nearest-neighhor classification for few-shot learning, 2019.
407
408
[52] Yaqing Wang, Quanming Yao, James Kwok, and Lionel M. Ni. Generalizing from a few examples: A survey on few-shot learning, 2019.
409
410
[53] Welinder, S. Branson, T. Mita, C. Wah, F. Schroff, S. Belongic, and P. Perona. Caltech-UCSD Birds 200. Technical Report CNS-TR-2010-00,, California Institute of Technology, 2010.
412
[54] Sanghyun Woo, Jongchan Park, Joon- Joun-Young Lee, and In So Kweon. Cbam: Convolutional block attention module, 2018
413
414
[55] Huijuan Xu and Kate Saenko. Ask, attend and answer: Exploring question-guidee spatial attention for visual question answering. 2016.
415
[56] Han-Jia Ye, Hexiang Hu, De-Chuan Zhan, and Fei Sha. Few-shot learning via embedding adaptation with set-to-set functions. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 202.
417
419
[57] Sergey Zagoruyko and Nikos Komodakis. Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer. 2017.
420
[58] Chi Zhang, Yujun Cai, Guosheng Lin, and Chunhua Shen. Deepemd: Few-shot image clas- sification with differentiable earth mover's distance and structured classifiers. In IEEF/CVF Conference Computer Vision and Patlern Recognition (CVPR), June 2020.
422
424
[59] Bolei Zhou, Aditya Khosla, Agata I Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In Computer Vision and Pattern Recognition, 2016.
12

---

Checklist
428
427
The checklist follows the references. Please read the checklist guidelines carefully for information on how to answer these questions. For each question, change the default [TODO] to [Yes], [No], or [N/A]- - You are strongly encouraged to include a justification your answer, either by referencing the appropriate section of your paper or providing a brief inline description. For example:
428
429
430
431
Did you include the license to the code and datasets? [Yes] See Section ??.
432
Did you include the license the code and datasets? [No] The code and the data are proprictary.
433
434
Did you include the license to the code and datasets? [N/A]
Please do not modify the questions and only use the provided macros for your answers. Note that the Checklist section does not count towards the page limit. In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.
436
1. For all authors...
439
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes] See Section 1.
440
441
(b) Did you describe the limitations of your work? [No]
442
(c) Did you discuss any potential negative societal impacts of your work? [No]
443
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
444
2. If you including theoretical results...
448
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proof's of all theoretical results? [N/A]
448
3. If you ran experiments..
449
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or URL.)? [No] The code and
450
451
the data are proprietary.
452
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Section 4.1.
453
454
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes] See Table 1.
456
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See Section 4.1.
457
458
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
459
(a) If your work uses existing assets, did you cite the creators? [Yes]
460
(b) Did you mention the license of the assets? [N/A]
461
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
463
(d) Did you discuss whether and how consent was obtained from people whose data you rr using/curating? [N/A]
484
465
(e) Did you discuss whether the data you are using/curating contains personally identifiable
486
information or offensive content? [No] There is no personally identifiable information or offensive content.
467
488
5. If you used crowdsourcing or conducted research with human subjects...
469
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
471
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
472
473
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
474
13


Reviews:
[{"review_id": "kPylwBQLdI", "reviewer": "Reviewer_5kD8", "summary": "The paper extends RENet[20] by adding two variants of attention modules that rotate and crop the query image around the predicted (by attention) target object that best matches the support sample via dense matching of the feature maps. Good results are demonstrated on mini/tiered-imagenet, cub, and cifar-fs, albeit using transductive inference (that is utilizing the query set as a set of additional unlabeled samples from the support classes) thus making a somewhat unfair comparison when put together with the inductive methods in the same table.", "questions": "please see weaknesses", "limitations": "not discussed", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n- I like the use of dense matching, cross attention and position (crop) and rotation refinement, I can see how those are useful for the set task\n- good results were obtained, albeit in a transductive setup\n\nWeaknesses:\n- a proper comparison to transductive sota needs to be made, this sota can easily be found e,g, on papers-with-code:\nhttps://paperswithcode.com/sota/few-shot-image-classification-on-mini-2\nas we can learn from the numbers there the true transductive sota is much higher than what is compared in the current paper. The situation is similar also with the other benchmarks (tiered, cub, cifar-fs, etc)\n- novelty (w.r.t. the RENet, CAN and other dense matching methods) seems to be limited, the added modules are basically attention-driven affine transformation modules that afaik have been applied before to this task\n- writing could be improved, as it is written the method is not reproducible and the code is stated to be \"proprietary\" so no code is provided nor code release is planned - all this reduces the value of the current paper as no1 will be able to reproduce these results based on the description.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "eDPXo5q7Ypq", "reviewer": "Reviewer_8rzp", "summary": "This paper aims to address the problem of few-shot learning. Inspired by the recent advances in cross attention, this paper explores to extend this paper and proposes two cross attention modules, i.e., cross scaled attention and cross aligned attention. Particularly, with the help of principal component analysis, cross aligned attention tries to align features from different images. In the experiments, the proposed method is evaluated on multiple datasets.", "questions": "I am not clear about the motivation of using principal component analysis.", "limitations": "The authors have addressed the limitations and potential negative societal impact of their work.", "rating": 3, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Using cross attention to extract discriminative features is reasonable for few-shot learning task.\n\nWeakness:\n\n1. The motivation of this paper is not clear. The Introduction is very simple. I am not clear about the motivation of the proposed cross scaled attention and the cross aligned attention. Meanwhile, the authors should interpret the advantages of the proposed method. Finally, I am not clear why use principal component analysis. I recommend the authors revise this section carefully.\n\n2. The proposed method only consists of some commonly used operations. And this paper does not propose any inspired ideas. Therefore, this paper lacks novelty. Meanwhile, during introducing approach details, the authors should further interpret the motivation of the used operations, which is helpful for better understanding of the proposed method.\n\n3. The writing of the approach section is tedious. I advise the authors to modify the method section carefully. Meanwhile, I am not clear why the proposed method could address the problem of few samples. In the experiments, the authors do not sufficiently analyze the proposed method. The authors should give more visualization examples and training curves to show the effectiveness of the proposed method.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "GuPT4d0fzL", "reviewer": "Reviewer_qcLF", "summary": "The paper proposes two cross-attention architectures that augment RENet to improve few-shot classification accuracy. Motivated by the increased performance of methods where attention is deployed effectively, CSA (cross-scaled attention) and CAA (cross-aligned attention) introduce a further hierarchy of attention to RENet to improve performance. The former improves feature matching by better scaling the maps while the latter mitigates the alignment issue by PCA. Extensive experiments demonstrate state-of-the-art performance while ablation studies justify the efficacy of each component.", "questions": "Please answer the questions noted in the above section with respect to the results. Specifically, first how much of the improved performance is simply a factor of the proposed architectures having more parameters than RENet. Second, following on that, do CSA and CAA improve performance over other simpler attention architectures that add the same number of parameters to RENet?", "limitations": "There is no explicit discussion of limitations or potential negative social impact of the work. The submission could benefit from a dedicated section that addresses the technical limitations of the work, lays out areas for future improvement, and explores potential societal impacts of the work.", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- Paper is very well-written and easy to follow (but some small typos in the related work section)\n- Experiments and ablation studies demonstrate state-of-the-art performance of both CSA and CAA\n- The motivation behind the proposed architecture is sound\n\nWeaknesses:\n- Details on parameter counts of RENet+CSA and +CAA are required to determine if the speed of inference reported is simply correlated with the parameter count of the model.\n- In some tables, only the best values are in bold indicating state-of-the-art. However, when 95% confidence intervals are considered CSA and CAA are matching statistically speaking, and therefore, the claim for CSA's superior performance is much more marginal. The results classification and the resulting claims must be updated to reflect this.\n- There are also two major questions from the results that weaken some of the claimed contributions: First, how much of the improved performance is simply a factor of the proposed architectures having more parameters than RENet. Second, following on that, do CSA and CAA improve performance over other simpler attention architectures that add the same number of parameters to RENet? This could be simply addressed by providing parameter count information and if possible, evaluating a simple standard attention architecture with a similar parameter count added to RENet.\n- It is also noted that the architectures are evaluated in a transductive manner rather than inductively. Transductive evaluation gives a natural advantage in performance. Therefore, all results tables must be updated to indicate whether the baseline is transductive or inductive. Furthermore, inductive evaluation of CSA and CAA would be useful to see if there performance gain is directly from the architectures and not from the advantages in transductive inference.\n- Paper is written around RENet and assumes considerable existing knowledge of the RENet architecture. This results in the language reading like a simple extension paper and some details being confusing before reviewing RENet.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "A4sG2fx29F", "reviewer": "Reviewer_bTff", "summary": "This paper extends relational embedding network [18] by introducing two novel cross attention modules namely the cross scaled attention (CSA) and the cross aligned attention (CAA). While CSA normalizes different feature maps to be in the same scale, the CAA aligns features from different images via PCA. The experiments are conducted on four public datasets and show new state of the art results. ", "questions": "The presentation of the results in Table 1 can be improved by grouping different methods into inductive, semi-supervised, transductive learning. Which compared methods in Table 1 belong to transductive learning?  ", "limitations": "I cannot find the limitations and potential negative societal impact in the paper.", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\n1. The proposed cross-attention modules seem to be novel in few-shot learning field.\n2. The experiments show significant improvement over the baselines, which looks quite promising\n\nWeakness\n\n1. Although the results in Table 1 are impressive, it is unclear if the comparison is completely fair. Specifically, this paper is in the transductive learning setting, which directly uses the test samples as the unlabeled samples, while many methods in Table 1 do not use any unlabeled examples or they are in the standard semi-supervised setting where the unlabeled samples are not the test samples. Comparing with inductive methods or semi-supervised methods with the proposed transductive method is not fair.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Reject