Original: Rethinking Generalization in Few-Shot Classification
Revised: Rethinking Generalization in Few-Shot Classification

Original: Anonymous Author(s)

Original: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: One-hot image-level annotations only correctly describe an often small subset ofan image’s content, particularly when complex real-world scenes are depicted.
Revised: Single image-level annotations only correctly describe an often small subset of an image’s content, particularly when complex real-world scenes are depicted.

Original: While this is tolerable in many classification scenarios, it poses a significantchallenge for applications where the set of classes differs significantly betweentraining and test time.
Revised: While this might be acceptable in many classification scenarios, it poses a significant challenge for applications where the set of classes differs significantly between training and test time.

Original: In this paper, we take a closer look at the implicationsin the context of few-shot learning .
Revised: In this paper, we take a closer look at the implications in the context of few-shot learning .

Original: Splitting the input samples into patches andencoding these via the help of Vision Transformers allows us to establish semanticcorrespondences between local regions across images and independent of theirrespective class.
Revised: Splitting the input samples into patches and encoding these via the help of Vision Transformers allows us to establish semantic correspondences between local regions across images and independent of their respective class.

Original: The most informative patch embeddings for the task at hand arethen determined as a function of the support set via online optimization at inferencetime, additionally providing visual interpretability of ‘ what matters most ’ in theimage.
Revised: The most informative patch embeddings for the task at hand are then determined as a function of the support set via online optimization at inference time, additionally providing visual interpretability of ‘ what matters most ’ in the image.

Original: We build on recent advances in unsupervised training of networks viamasked image modelling to overcome the lack of fine-grained labels and learn themore general statistical structure of the data while avoiding negative image-levelannotation influence, aka supervision collapse.
Revised: We build on recent advances in unsupervised training of networks via masked image modelling to overcome the lack of fine-grained labels and learn the more general statistical structure of the data while avoiding negative image-level annotation influence, aka supervision collapse.

Original: Experimental results show thecompetitiveness of our approach, achieving new state-of-the-art results on fourpopular few-shot classification benchmarks for 5 -shot and 1 -shot scenarios.
Revised: Experimental results show the competitiveness of our approach, achieving new state-of-the-art results on four popular few-shot classification benchmarks for 5 -shot and 1 -shot scenarios.

Original:  Introduction
Revised:  Introduction

Original: Images depicting real-world scenes are usually comprised of several different entities, e g., a personwalking their dog in a park surrounded by trees or a  dogplaying with a ball (Figure 1).
Revised: Images depicting real-world scenes are usually comprised of several different entities, e g., a family walking their dog in a park surrounded by trees, or a person patting their dog (Figure 1).

Original: Nevertheless,popular computer vision datasets like ImageNet [35] assign a single image-level annotation to classifytheir entire content.
Revised: Nevertheless, popular computer vision datasets like ImageNet [38] assign a single image-level annotation to classify their entire content.

Original: Hence, such a label only correctly applies to an often small subset of the actualimage.
Revised: Hence, such a label only correctly applies to an often small subset of the actual image.

Original: As a result, models trained on such data via gradient-based methods learn to ignore allseemingly irrelevant information, particularly entities that occur across differently labelled images.
Revised: As a result, models trained on such data via gradient-based methods learn to ignore all seemingly irrelevant information, particularly entities that occur across differently labelled images.

Original: While this is acceptable for conventional classification methods that encounter a diverse number oftraining examples for all classes they are expected to classify it poses a major but often overlookedchallenge for applications where the set of classes differs between training and test time.
Revised: While this might be acceptable for conventional classification methods that encounter a diverse number of training examples for all classes they are expected to distinguish, it poses a major but often overlooked challenge for applications where the set of classes differs between training and test time.

Original: One suchaffected area is few-shot learning (FSL) where approaches are expected to correctly classify entirelynew classes at test time that have never been encountered during training, just by being providedwith a few (e.g., one or five) samples for each of these new categories.
Revised: One such affected area is few-shot learning (FSL) where approaches are expected to correctly classify entirely new classes at test time that have never been encountered during training, just by being provided with a few (e.g., one or five) samples for each of these new categories.

Original: During test time, entities thathave not been part of the set of training classes and have possibly been perceived as irrelevant mightvery well be part of the set of test classes – yet, the method was taught to ignore these.
Revised: During test time, entities that have not been part of the set of training classes and have possibly been perceived as irrelevant might very well be part of the set of test classes – yet, the method was taught to ignore these.

Original: Similarly, amethod might overemphasize the importance of certain image patterns learned during training thatare however of no relevance for the test classes, resulting in supervision collapse [8].
Revised: Similarly, a method might overemphasize the importance of certain image patterns learned during training that are however of no relevance for the test classes, resulting in supervision collapse [9].

Original: Few previous works [8; 17] partially tackle the above challenges.
Revised: Few previous works [9, 18] partially tackle the above challenges.

Original: Particularly, CTX [8] proposes tolearn the spatial and semantic alignment between CNN-extracted query and support features using atransformer-style attention mechanism.
Revised: CTX [9] proposes to learn the spatial and semantic alignment between CNN-extracted query and support features using a Transformer-style attention mechanism.

Original: The authors further show that self-supervised learning tasks(i.e. , SimCLR) can be integrated into episodic training along with normal supervised tasks to learn[Figure] more generalized features, which benefits solving unseen tasks and mitigates supervision collapse.
Revised: The authors further show that self-supervised learning tasks (i.e., SimCLR) can be integrated into episodic training along with normal supervised tasks to learn more generalized features, which benefits solving unseen tasks and mitigates supervision collapse.

Original: CAN [17] achieves this in a similar manner by performing cross-attention between class prototypesand query feature maps, highlighting the region of a feature map important for classification duringinference.
Revised: CAN [18] achieves this in a similar manner by performing cross-attention between class prototypes and query feature maps, highlighting the region of a feature map important for classification during inference.

Original: While both methods propose important contributions towards tackling supervision collapse,there exist important drawbacks.
Revised: While both methods propose important contributions towards tackling supervision collapse, there exist important drawbacks.

Original: Firstly, both methods build their ideas around aligning prototypesbased on each query.
Revised: Firstly, both methods build their ideas around aligning prototypes based on each query.

Original: Such prototypes are merely class-aware and ignore all inter-class informationpresent in the support set – a part that has  been shown to be crucial for few-shot learning [32; 46].
Revised: Such prototypes are merely class-aware and ignore all inter-class information present in the support set – a part that has however been shown to be crucial for few-shot learning [34, 48].

Original: Furthermore, learning query-aligned class representations requires performing the same operation foreach query, rendering such approaches rather inefficient at inference time.
Revised: Furthermore, learning query-aligned class representations requires performing the same operation for each query, rendering such approaches rather inefficient at inference time.

Original: Summarizing our and previous works’ observations, we aim to address what we see as the two maincriteria: 1) building an understanding about the image’s structure and content that generalizes towardsnew classes, and 2) providing the ability to interpret the provided samples in context , i.e., finding theintra-class similarities and inter-class differences while jointly considering all available information.
Revised: Summarizing our and previous works’ observations, we aim to address what we see as the two main criteria: 1) building an understanding about an image’s structure and content that generalizes towards new classes, and 2) providing the ability to interpret the provided samples in context , i.e., finding the intra-class similarities and inter-class differences while jointly considering all available information.

Original: Our work.
Revised: Our work.

Original:  To alleviate the negative influence of image-level annotations and to avoid supervisioncollapse, we decompose the images into patches representing local regions, each having a higherlikelihood of being dominated by only one entity.
Revised: 2 To alleviate the negative influence of image-level annotations and to avoid supervision collapse, we decompose the images into patches representing local regions, each having a higher likelihood of being dominated by only one entity.

Original: To overcome the lack of such fine-grainedannotations we employ self-supervised training with Masked Image Modelling as pretext task [57]and use a Vision Transformer architecture [9] as encoder due to its patch-based nature.
Revised: To overcome the lack of such fine-grained annotations we employ self-supervised training with Masked Image Modelling as pretext task [59] and use a Vision Transformer architecture [10] as encoder due to its patch-based nature.

Original: We buildour classification around the concept of learning task-specific similarity between local regions as afunction of the support set at inference time.
Revised: We build our classification around the concept of learning task-specific similarity between local regions as a function of the support set at inference time.

Original: To this extend, we first create a prior similarity map byestablishing semantic patch correspondences between all support set samples irrespective of theirclass, i.e. also between entities that might not be relevant or potentially even harmful for correctclassification.
Revised: To this extent, we first create a prior similarity map by establishing semantic patch correspondences between all support set samples irrespective of their class, i.e., also between entities that might not be relevant or potentially even harmful for correct classification (Figure 1, step (1)).

Original: Assume a reduced support set with only two classes: ‘person’ and ‘cat’.
Revised: Consider the depicted support set with only two classes: ‘person’ and ‘cat’.

Original: Thinkingback to Figure 1, the right image is part of our support set for ‘cat’ – and the dog just happens to bein the image.
Revised: The lower-right image is part of our support set for ‘cat’ – and the dog just happens to be in the image.

Original: Now in the query sample that shall be classified,assume the image depicts a personwalking their dog.
Revised: Now in the query sample that shall be classified, the image depicts a person patting their dog.

Original: We will thus correctly detect a correspondence of the two dogs across those twoimages, as well as between the person patches and the other samples of the person support set class.
Revised: We will thus correctly detect a correspondence of the two dogs across those two images, as well as between the person patches and the other samples of the person support set class.

Original: While the correspondences between the person regions are helpful, there is no ‘dog’ classis in theactual support set (i.e., ‘dog’ is out-of-task information), rendering this correspondence harmful forclassification since it would indicate that the query is connected to the image with the ‘cat’ label.
Revised: While the correspondences between the person regions are helpful, there is no ‘dog’ class in the actual support set (i.e., ‘dog’ is out-of-task information), rendering this correspondence harmful for classification since it would indicate that the query is connected to the image with the ‘cat’ label.

Original: Thisis where our token importance reweighting comes into play.
Revised: This is where our token importance weighting comes into play.

Original: We infer an importance weight for eachtoken based on its contribution towards correct classification of the other support set samples, activelystrengthening intra-class similarities and inter-class differences by jointly considering all availableinformation – in other words, we learn which tokens ‘help’ or ‘harm’ our classification objective.
Revised: We infer an importance weight for each token based on its contribution towards correct classification of the other support set samples, actively strengthening intra-class similarities and inter-class differences by jointly considering all available information – in other words, we learn which tokens ‘help’ or ‘harm’ our classification objective (Figure 1, step (2)).

Original: These importance-reweighted support set embeddings are then used as basis for our similarity-basedquery sample classification.
Revised: These importance-reweighted support set embeddings are then used as basis for our similarity-based query sample classification (step (3)).

Original: Our main contributions include the following
Revised: Our main contributions include the following:

Original: We demonstrate that Transformer-only architectures in conjunction with self-supervisedpretraining can be successfully used in few-shot settings without the need of convolutional

Original: backbones or any additional data.

Original: We show that meta fine-tuning of Vision Transformers combined with our inner loop tokenimportance reweighting can successfully use the supervision signal of provided support set

Original: labels while avoiding supervision collapse.

Original: We provide insights into how establishing general similarities across images independent ofclasses followed by our optimization-based selection at inference time can boost generalization while allowing visual interpretability at the same time, and show the efficacy of our

Original: C×H×W
Revised: C×H×W

Original: N·K·M×C ... ...
Revised: N·K×M×C ... ...

Original: N·K·L×D L×D M×C *
Revised: N·K×L×D 1×L×D M×C *

Original: N·K·L×L
Revised: N·K×L×L

Original: Token importance weights generator ...
Revised: Token importance weights generator ...

Original: N·K×L
Revised: N·K×L

Original: N·K·L×L
Revised: N·K×L×L

Original: N·K×C×H×W
Revised: N·K×C×H×W

Original: Logits aggregation low high
Revised: Logits aggregation low high

Original: Class probability x * Pair-wise similarity x
Revised: Class probabilities x

Original: Broadcast & reweighting
Revised: Broadcasting & Reweighting

Revised: Pair-wise similarity

Original: N
Revised: N

Revised: [Figure]

Revised: We demonstrate that Transformer-only architectures in conjunction with self-supervised pretraining can be successfully used in few-shot settings without the need of convolutional backbones or any additional data.

Revised: We show that meta fine-tuning of Vision Transformers combined with our inner loop token importance reweighting can successfully use the supervision signal of provided support set labels while avoiding supervision collapse.

Revised: We provide insights into how establishing general similarities across images independent of classes followed by our optimization-based selection at inference time can boost generalization while allowing visual interpretability at the same time, and show the efficacy of our method by achieving new state-of-the-art results on four popular public benchmarks.

Original:  Few-shot classification via reweighted embedding similarity
Revised:  Few-shot classification via reweighted embedding similarity

Original: We start this section by briefly introducing the problem setting we are tackling in this work: inductivefew-shot classification.
Revised: We start this section by briefly introducing the problem setting we are tackling in this work: inductive few-shot classification.

Original: We then provide an overview of our proposed method FewTURE – Fewshot classification with Transformers Using Reweighted Embedding similarity (Figure 2) beforeelaborating on the main elements in more detail.
Revised: We then provide an overview of our proposed method FewTURE – Fewshot classification with Transformers Using Reweighted Embedding similarity (Figure 2) before elaborating on the main elements in more detail.

Original: Problem definition .
Revised: Problem definition .

Original: Inductive N -way K -shot few-shot classification aims to generalize knowledgelearned during training on D train to unseen test data D test , with classes C train ∩C test = ∅ , using onlya few labelled samples.
Revised: Inductive N -way K -shot few-shot classification aims to generalize knowledge learned during training on D train to unseen test data D test , with classes C train ∩C test = ∅ , using only a few labelled samples.

Original: We follow the meta-learning protocol of previous works [46] to formulate thefew-shot classification problem with episodic training and testing.
Revised: We follow the meta-learning protocol of previous works [48] to formulate the few-shot classification problem with episodic training and testing.

Original: An episode E is composed of asupport set X s = { ( x nks , y nks ) | n = 1 , . .
Revised: An episode E is composed of a support set X s = { ( x nks , y nks ) | n = 1 , . .

Original: N ; k = 1 , . . .
Revised: N ; k = 1 , . . .

Original: , K ; y nks ∈ C train } , where x nks denotes thek -th sample of class n with label y nks , and a query set X q
Revised: , K ; y nks ∈ C train } , where x nks denotes the k -th sample of class n with label y nks , and a query set X q

Original: = { ( x nq , y nq ) | n = 1 , . .
Revised: = { ( x nq , y nq ) | n = 1 , . .

Original: . , N } , where x nqdenotes a query sample 1 of class n with label y nq .
Revised: . , N } , where x nq denotes a query sample 3 of class n with label y n q .

Original: 2.1 Overview on FewTURE
Revised: 2.1 Overview of FewTURE

Original: As depicted in Figure 2, we encode the image patches P s of the support set samples along with thequery sample patches p q via f θ and obtain corresponding sets of tokens Z s and z q , respectively.
Revised: As depicted in Figure 2, we encode the image patches P s of the support set samples along with the query sample patches p q via f θ and obtain corresponding sets of tokens Z s and z q , respectively 4 .

Original: It isto be noted that while we choose to illustrate our method via the use of one single query sample, theclassification of all query samples is computed at the same time in one single pass in practice.
Revised: It is to be noted that while we choose to illustrate our method via the use of one single query sample, the classification of all query samples is computed at the same time in one single pass in practice.

Original: Weretrieve our ‘prior’ correspondence map S expressing the token-wise similarity between the encodedsemantic content of the local regions in the query sample and all patches of all support samples,allowing us to consider all available information jointly without incurring information loss due toaveraging or similar operations.
Revised: We retrieve our ‘prior’ correspondence map S expressing the token-wise similarity between the encoded semantic content of the local regions in the query sample and all patches of all support samples, allowing us to consider all available information jointly without incurring information loss due to averaging or similar operations.

Original: This ‘prior’ similarity map represents correspondences betweenregions of samples irrespective of their individual class, i.e. also between entities that might not berelevant or potentially even harmful for correct classification.
Revised: This ‘prior’ similarity map represents correspondences between regions of samples irrespective of their individual class, i.e. also between entities that might not be relevant or potentially even harmful for correct classification.

Original: Using the annotated support set samples,we now infer a task-specific importance weight factor v j for each support token z js representing itscontribution to correctly classify other samples in the support set via online optimization at inferencetime (Section 2.4).
Revised: Using the annotated support set samples, we now infer a task-specific importance weight factor v j for each support token z js representing its contribution to correctly classify other samples in the support set via online optimization at inference time (Section 2.4).

Original: We then reweight the prior similarities to obtain our classification result for thequery sample, jointly considering all available information.
Revised: We then reweight the prior similarities to obtain our classification result for the query sample, jointly considering all available information.

Original: 2.2 Self-supervised pretraining against supervision collapse
Revised: 2.2 Self-supervised pretraining against supervision collapse

Original: To overcome the problem of supervision collapse induced by image-level annotations, we split theinput images into smaller parts where each region has a higher likelihood of only containing onemajor entity and hence a more distinct semantic meaning.
Revised: To overcome the problem of supervision collapse induced by image-level annotations, we split the input images into smaller parts where each region has a higher likelihood of only containing one major entity and hence a more distinct semantic meaning.

Original: Since no labels are available for this morefine-grained data, we encode the information of each local region via an unsupervised method.
Revised: Since no labels are available for this more fine-grained data, we encode the information of each local region via an unsupervised method.

Original: We build our approach around the recently introduced idea of using Masked Image Modeling(MIM) [3; 57] as a pretext task for self-supervised training of Vision Transformers.
Revised: We build our approach around the recently introduced idea of using Masked Image Modeling (MIM) [3, 59] as a pretext task for self-supervised training of Vision Transformers.

Original: In contrastto previous unsupervised approaches [5; 7] which focused mainly on global image-level representations, MIM randomly masks a number of patch embeddings ( tokens ) and aims to reconstruct themgiven the remaining information of the image.
Revised: In contrast to previous unsupervised approaches [5, 7] which focused mainly on global image-level representations, MIM randomly masks a number of patch embeddings ( tokens ) and aims to reconstruct them given the remaining information of the image.

Original: Thein this way introduced token constraints helpour Transformer backbone to learn an embedding space that yields semantically meaningful representations for each individual image patch.
Revised: The introduced token constraints help our Transformer backbone to learn an embedding space that yields semantically meaningful representations for each individual image patch.

Original: We then leverage the information of the provided labelsthrough fine-tuning the pretrained backbone in conjunction with our inner loop token importanceweighting described in the following sections while successfully avoiding supervision collapse (seeexperimental results in Section 3.2).
Revised: We then leverage the information of the provided labels through fine-tuning the pretrained backbone in conjunction with our inner loop token importance weighting described in the following sections while successfully avoiding supervision collapse (see experimental results in Section 3.2).

Original: 2.3 Classification through reweighted token similarity
Revised: 2.3 Classification through reweighted token similarity

Original: As illustrated in Figure 2, we split each input image x ∈ R H × W × C into a sequence of M = H · W / P 2patches p
Revised: As illustrated in Figure 2, we split each input image x ∈ R H × W × C into a sequence of M = H · W / Ppatches p

Original: = { p i } Mi =1 , with each patch p i ∈
Revised: = { p i } Mi =1 , with each patch p i ∈

Original: R P 2 × C .
Revised: R P 2 × C .

Original: We then flatten and pass all patches of thesupport and query images as input to the Transformer architecture, obtaining the set of supporttokens Z s = f θ ( P s ) with Z s =
Revised: We then flatten and pass all patches of the support and query images as input to the Transformer architecture, obtaining the set of support tokens Z s = f θ ( P s ) with Z s =

Original: { z nks | n =1 , . . .
Revised: { z nks | n =1 , . . .

Original: , N, k =1 , . .
Revised: , N, k =1 , . .

Original: , K } ,
Revised: , K } ,

Original: z nks = { z nkls | l =1 , . . .
Revised: z nks = { z nkls | l =1 , . . .

Original: , L ; z nkls ∈131 R D } and query tokens
Revised: , L ; z nkls ∈ R D } and query tokens

Original: z q = f θ ( p q ) with z q = { z lq | l = 1 , . . .
Revised: z q = f θ ( p q ) with z q = { z lq | l = 1 , . . .

Original: , L ; z lq ∈ R D } .
Revised: , L ; z lq ∈ R D } .

Original: Vision Transformerslike ViT [9; 43] satisfy L = M
Revised: Vision Transformers like ViT [10, 46] satisfy L = M

Original: whereas hierarchical Transformers like Swin [25] generally emit areduced number of tokens L < M due to internal merging strategies.
Revised: whereas hierarchical Transformers like Swin [27] generally emit a reduced number of tokens L < M due to internal merging strategies.

Original: Having obtained all patch embeddings, we establish semantic correspondences by computing thepair-wise patch similarity matrix between the set of support tokens Z s and query tokens
Revised: Having obtained all patch embeddings, we establish semantic correspondences by computing the pair-wise patch similarity matrix between the set of support tokens Z s and query tokens

Original: z q as S ∈
Revised: z q as S ∈

Original: R N · K · L × L , where each element in S is obtained by s l s ,l q nk = sim
Revised: R N · K · L × L , where each element in S is obtained by s l s ,l q nk = sim

Original: ( z nkl s s , z l q q ) , where l s =1 , . . .
Revised: ( z nkl s s , z l q q ) , where l s =1 , . . .

Original: , L andl q =1 , . . .
Revised: , L and l q =1 , . . .

Original: , L .
Revised: , L .

Original: Note that local image regions representing similar entities exhibit higher scores.
Revised: Note that local image regions representing similar entities exhibit higher scores.

Original: Whileany distance metric can be used to compute the similarity ( sim ), we found cosine similarity to workparticularly well for this task.
Revised: While any distance metric can be used to compute the similarity ( sim ), we found cosine similarity to work particularly well for this task.

Original: We then use the task-specific token importance weights v ∈ R N · K · L × 1inferred via online optimization based on the annotated support set samples (see Section 2.4) toreweight the similarities through column-wise addition and obtain our task-specific similarity matrixas ˜ S
Revised: We then use the task-specific token importance weights v ∈ R N · K · L ×inferred via online optimization based on the annotated support set samples (see Section 2.4) to reweight the similarities through column-wise addition and obtain our task-specific similarity matrix as ˜ S

Original: = S + (cid:2) v · 1 1 × L (cid:3) , with elements ˜ s l s ,l q nk .
Revised: = S + (cid:2) v · 1 1 × L (cid:3) , with elements ˜ s l s ,l q nk .

Original: Note that this addition of our reweighting logitscorresponds to multiplicative reweighting in probability space.
Revised: Note that this addition of our reweighting logits corresponds to multiplicative reweighting in probability space.

Original: We temperature-scale the adaptedsimilarity logits with 1 / τ S and aggregate the token similarity values across all elements belongingto the same support set class via a LogSumExp operation, i.e. aggregating K · L 2 logits per classfollowed by a softmax – resulting in the final class prediction ˆ y q for the query sample x q as
Revised: We temperature-scale the adapted similarity logits with 1 / τ S and aggregate the token similarity values across all elements belonging to the same support set class via a LogSumExp operation, i.e. aggregating K · L 2 logits per class followed by a softmax – resulting in the final class prediction ˆ y q for the query sample x q as

Original: [Equation]
Revised: [Equation]

Original: 
Revised: L (cid:88) [Table] (cid:88)

Original: [Equation]
Revised: [Table]

Revised: exp (cid:16)

Original: 
Revised: [Table] [Table] (cid:17)  N n =

Original: [Equation]
Revised: [Table]   .

Original: 2.4 Learning token importance at inference time
Revised: 2.4 Learning token importance at inference time

Original: We use all samples of a task’s support set together with their annotations to learn the importance foreach individual patch token via online optimization at inference time.
Revised: We use all samples of a task’s support set together with their annotations to learn the importance for each individual patch token via online optimization at inference time.

Original: As depicted in Figure 3, weformulate the same classification objective as in the previous section but aim to classify the supportset samples instead of a query sample.
Revised: As depicted in Figure 3, we formulate the same classification objective as in the previous section but aim to classify the support set samples instead of a query sample.

Original: In other words, we use two versions of the support set samples:one with labels as ‘support’ Z s and one without as ‘pseudo-query’ Z sq , and obtain the [-similaritymatrix S s ∈ R N · K · L × N · K · L . The token importance weights are initialized to v 0 = 
 ∈ R N · K · L × 1and column-wise added to form ˜ S s = S s + (cid:2) v 0 · 1 1 × N · K · L (cid:3) .-]
Revised: In other words, we use two versions of the support set samples: one with labels as ‘support’ Z s and one without as ‘pseudo-query’ Z sq , and obtain the [-similaritymatrix S s ∈ R N · K · L × N · K · L . The token importance weights are initialized to v 0 = 
 ∈ R N · K · L × 1and column-wise added to form ˜ S s = S s + (cid:2) v 0 · 1 1 × N · K · L (cid:3) .-]similarity

Original: The goal is now to determine which tokens of Z s are most helpful in contributing towards correctlyclassifying Z sq , and which ones negatively affect this objective. To prevent tokens from simplyclassifying themselves, we device the following strategy for our N -way K -shot tasks. For scenarioswith K > 1 samples per class, we apply block-diagonal masking with blocks of size L × L to thesimilarity matrix ˜ S s – meaning that we enforce classification of each token in Z sq exclusively basedon information from other images. Since there are no other in-class examples available in 1 -shotscenarios, we slightly weaken the constraint and apply local masking in a m × m window aroundeach patch, forcing the token to be classified based on the remaining information in the image.
Revised: N·K·L×N·K·L

Original: Wefound a local window of m = 5 to work well throughout our experiments for both architectures.

Original: N·K·L×D N·K·L×N·K·L N·K·L×N·K·L...
Revised: N·K·L×N·K·L

Original: Class probability
Revised: Class probability

Original: Class label
Revised: Class label

Original: Loss
Revised: Loss

Original: Logits aggregation
Revised: Logits aggregation

Original: N·K×L ...
Revised: N·K×L ...

Original: Update

Original: Backpropagate

Original: Forward pass

Original: Pair-wise similarity
Revised: Pair-wise similarity

Original: Broadcast&addition
Revised: Broadcast&addition

Original: N·K×L
Revised: N·K×L

Original: Masking
Revised: Masking

Revised: Update

Original: N·K·L×D
Revised: Backpropagate Forward pass N·K×L×D ... ... N·K×L×D ... ...

Revised: matrix S s ∈ R N · K · L × N · K · L .

Original: {+The token importance weights are initialized to v 0 = 
+}
Revised: {+The token importance weights are initialized to v 0 = 
+}

Revised: ∈ R N · K · L ×and column-wise added to form ˜ S s

Revised: = S s + (cid:2) v 0 · 1 1 × N · K · L (cid:3) .

Revised: The goal is now to determine which tokens of Z s are most helpful in contributing towards correctly classifying Z sq , and which ones negatively affect this objective.

Revised: To prevent tokens from simply classifying themselves, we devise the following strategy for our N -way K -shot tasks.

Revised: For scenarios with K > 1 samples per class, we apply block-diagonal masking with blocks of size L × L to the similarity matrix ˜ S s – meaning that we enforce classification of each token in Z sq exclusively based on information from other images.

Revised: Since there are no other in-class examples available in 1 -shot scenarios, we slightly weaken the constraint and apply local masking in an m × m window around each patch, forcing the token to be classified based on the remaining information in the image.

Revised: We found a local window of m = 5 to work well throughout our experiments for both architectures.

Original: We use temperature-scaling and aggregate all modified similarity logits across all elements belongingto the same support set class of the annotated Z s for each element z nksq , apply a softmax operationand obtain the predicted class probabilities ˆ y nks for each support set sample ( cf.
Revised: We use temperature-scaling and aggregate all modified similarity logits across all elements belonging to the same support set class of the annotated Z s for each element z nksq , apply a softmax operation and obtain the predicted class probabilities ˆ y nks for each support set sample ( cf.

Original: Equation (1)).
Revised: Equation (1)).

Original: Giventhat the prediction is now dependent on the initialized token similarity weights v , we can formulatean online optimization objective by using the support set labels y nks as
Revised: Given that the prediction is now dependent on the initialized token similarity weights v , we can formulate an online optimization objective by using the support set labels y nks as

Original: [Equation]
Revised: [Equation]

Original: It is to be noted that by using column-wise addition of v , we share the importance weights of eachsupport token across all ‘pseudo-query’ tokens and thus constrain the optimization to jointly learntoken importance with respect to all available information.
Revised: It is to be noted that by using column-wise addition of v , we share the importance weights of each support token across all ‘pseudo-query’ tokens and thus constrain the optimization to jointly learn token importance with respect to all available information.

Original: In other words, task-specific intra-classsimilarities will be strengthened while inter-class ones will be penalized accordingly.
Revised: In other words, task-specific intra-class similarities will be strengthened while inter-class ones will be penalized accordingly.

Original: We furtherdo not require any second order derivatives like other methods during meta fine-tuning, since theoptimization of v is decoupled from the network’s parameters.
Revised: We further do not require any second order derivatives like other methods during meta fine-tuning, since the optimization of v is decoupled from the network’s parameters.

Original:  Experiments and discussion 
Revised:  Experiments and discussion 3.1 Implementation details

Original: 3.1 Implementation details

Original: Our training strategy is divided into two parts: self-supervised pretraining followed by meta finetuning.
Revised: Our training strategy is divided into two parts: self-supervised pretraining followed by meta finetuning.

Original: It is to be noted that each architecture is exclusively trained on the training set data of therespective dataset that is to be evaluated, and no additional data is used.
Revised: It is to be noted that each architecture is exclusively trained on the training set data of the respective dataset that is to be evaluated, and no additional data is used.

Original: Datasets .
Revised: Datasets .

Original: We train and evaluate our methods using four popular few-shot classification benchmarks,namely mini ImageNet [46], tiered ImageNet [34], CIFAR-FS [4] and FC-100 [32].
Revised: We train and evaluate our methods using four popular few-shot classification benchmarks, namely mini ImageNet [48], tiered ImageNet [37], CIFAR-FS [4] and FC-100 [34].

Original: Architectures .
Revised: Architectures .

Original: We compare two different Transformer architectures in this work, the monolithic
Revised: We compare two different Transformer architectures in this work, the monolithic

Original: ViT architecture [9; 43] in its ‘small’ form ( ViT-small aka DeiT-small ) and the multi-scale Swinarchitecture [25] in its ‘tiny’ version ( Swin-tiny ).
Revised: ViT architecture [10, 46] in its ‘small’ form ( ViT-small aka DeiT-small ) and the multi-scale Swin architecture [27] in its ‘tiny’ version ( Swin-tiny ).

Original: Self-supervised pretraining .
Revised: Self-supervised pretraining .

Original: We employ the strategy proposed by [57] to pretrain our Transformerbackbones and mostly stick to the hyperparameter settings reported in their work.
Revised: We employ the strategy proposed by [59] to pretrain our Transformer backbones and mostly stick to the hyperparameter settings reported in their work.

Original: Our ViT and Swinarchitectures are trained with a batch size of 512 for 1600 and 800 epochs, respectively.
Revised: Our ViT and Swin

Original: [-We use 
-]
Revised: [-We use 
-][Figure]

Original: Nvidia A100 GPUs with 40GB each for our ViT and 8 such GPUs for our Swin models.
Revised: [Figure]

Original: Meta fine-tuning .
Revised: Meta fine-tuning .

Original: We use meta fine-tuning to further refine our embedding space by using theavailable image-level labels in conjunction with our token-reweighting method.
Revised: We use meta fine-tuning to further refine our embedding space by using the available image-level labels in conjunction with our token-reweighting method.

Original: We generally trainfor up to 200 epochs but find most architectures to converge earlier.
Revised: We generally train for up to 200 epochs but find most architectures to converge earlier.

Original: We evaluate at each epoch on600 randomly sampled episodes from the respective validation set to select the best set of parameters.
Revised: We evaluate at each epoch on 600 randomly sampled episodes from the respective validation set to select the best set of parameters.

Original: During test time, we randomly sample 600 episodes from the test set to evaluate our model.
Revised: During test time, we randomly sample 600 episodes from the test set to evaluate our model.

Original: Token importance reweighting and classifier .
Revised: Token importance reweighting and classifier .

Original: We use cosine similarity to compute S .
Revised: We use cosine similarity to compute S .

Original: While thetemperature τ S used to scale the logits can be learnt during meta fine-tuning, we found τ S = 1 / √ dto be a good default value (or starting point if learnt, see supplementary material).
Revised: While the temperature τ S used to scale the logits can be learnt during meta fine-tuning, we found τ S = 1 / √ d to be a good default value (or starting point if learnt, see supplementary material).

Original: We use SGD asoptimizer with a learning rate of 0. [Figure]
Revised: We use SGD as optimizer with a learning rate of 0.1 for the token importance weight generation.

Original: Please refer to supplementary material of this paper for a more detailed discussion regarding datasets,implementation and hyperparameters.
Revised: Please refer to supplementary material of this paper for a more detailed discussion regarding datasets, implementation and hyperparameters.

Original: 3.2 Self-supervised pretraining and token-reweighted fine-tuning improve generalization
Revised: 3.2 Self-supervised pretraining and token-reweighted fine-tuning improve generalization

Original: In this section, we investigate the influence of self-supervised pretraining compared to its supervisedcounterpart using the provided image-level labels (de-facto standard in most current state-of-the-artmethods).
Revised: In this section, we investigate the influence of self-supervised pretraining compared to its supervised counterpart using the provided image-level labels (de-facto standard in most current state-of-the-art methods).

Original: We only vary the training strategy of our backbone and use our introduced token similaritybased classifier with 15 inner-loop reweighting steps for all experiments.
Revised: We only vary the training strategy of our backbone and use our introduced token similaritybased classifier with 15 inner-loop reweighting steps for all experiments.

Original: Figure 4 illustrates that both
Revised: Figure 4 illustrates that both

Original: ViT-small and Swin-tiny with self-supervised pretraining alone ( w/o meta fine-tuning) learn moregeneralizable features than their respective supervised version for both 1 -shot (left-hand side) and5 -shot (right-hand side) scenarios.
Revised: ViT-small and Swin-tiny with self-supervised pretraining alone ( w/o meta fine-tuning) learn more generalizable features than their respective supervised versions for both 1 -shot (left-hand side) and 5 -shot (right-hand side) scenarios.

Original: In fact, across all cases w/o meta fine-tuning, the self-supervisedpretrained Transformers outperform their supervised counterparts by more than 10% and up to asignificant 18 .
Revised: In fact, across all cases w/o meta fine-tuning, the self-supervised pretrained Transformers outperform their supervised counterparts by more than 10% and up to a significant 18 .

Original: 19% in the case of Swin-tiny.
Revised: 19% in the case of Swin-tiny.

Original: This clearly demonstrates that our self-supervisedpretraining strategy explores information that is richer and beyond the labels.
Revised: This clearly demonstrates that our self-supervised pretraining strategy explores information that is richer and beyond the labels.

Original: Another interestinginsight is that the meta fine-tuning stage does not improve supervised backbones as much as theirunsupervised counterparts.
Revised: Another interesting insight is that the meta fine-tuning stage does not improve supervised backbones as much as their unsupervised counterparts.

Original: Specifically after unsupervised pretraining, meta fine-tuning of the
Revised: Specifically after unsupervised pretraining, meta fine-tuning of the

Original: Transformers in conjunction with our token-reweighting strategy is able to boost the performance by6 .
Revised: Transformers in conjunction with our token-reweighting strategy is able to boost the performance by

Original: 77% (ViT-small) and 13 .
Revised: 77% (ViT-small) and 13 .

Original: 01% (Swin-tiny) for 1 -shot, and 9 .
Revised: 01% (Swin-tiny) for 1 -shot, and 9 .

Original: 94% (ViT-small) and 10 .
Revised: 94% (ViT-small) and 10 .

Original: 10% (Swintiny) for 5 -shot.
Revised: 10% (Swintiny) for 5 -shot.

Original: While such significant improvements cannot be observed across the supervisednetworks, the Swin versions seem to generally start off lower after pretraining but benefit more fromthe fine-tuning than ViT.
Revised: While such significant improvements cannot be observed across the supervised networks, the Swin versions seem to generally start off lower after pretraining but benefit more from the fine-tuning than ViT.

Original: The observed results clearly indicate that our token-reweighted fine-tuningstrategy is able to further improve the generalization of self-supervised Transformers, thus performingbetter on the novel tasks of the unseen test set.
Revised: The observed results clearly indicate that our token-reweighted fine-tuning strategy is able to further improve the generalization of self-supervised Transformers, thus performing better on the novel tasks of the unseen test set.

Original: Figure 5 additionally depicts projected views of thetokens of 5 instances from a novel class  in embedding space.
Revised: Figure 5 additionally depicts projected views of the tokens of 5 instances from a novel class as well as the entire novel support set in embedding space.

Original: While the representations obtainedfrom the network meta fine-tuned by using common averaging over the embeddings (‘ average ’)does not exhibit any clear separation of the instances, the embeddings obtained with our classifierseem to retain the instance information (‘ w/o v ’) and separation is improved when using tokenimportance reweighting (‘ w/ v ’). 
Revised: Representations obtained with our classifier seem to retain the instance information (‘ w/o v ’) and separation is improved when using token importance reweighting (‘ w/ v ’). While the projected tokens of the entire support set show partial overlap between classes as is expected due to commonalities like e.g. similar background, our reweighting clearly determines the class-characteristic tokens (displayed in their original class-respective color).

Original: These results indicate that our similarity-based classifier coupledwith task-specific token reweighting is able to better disentangle the embeddings of different instancesfrom the same class, which prevents the network from supervision collapse and achieves the higher performance observed on the benchmarks.
Revised: These results indicate that our similarity-based classifier coupled with task-specific token reweighting is able to better disentangle the embeddings of different instances from the same class as well as other classes, which prevents the network from supervision collapse and achieves the higher performance observed on the benchmarks.

Original: They further show that self-supervised pretraining ishelpful but not sufficient to achieve well-separated representations without supervision collapse thatare suitable for few-shot classification.
Revised: They further show that self-supervised pretraining is helpful but not sufficient to achieve well-separated representations without supervision collapse that are suitable for few-shot classification.

Original: 3.3 Selecting helpful patches at inference time
Revised: 3.3 Selecting helpful patches at inference time

Original: Figure 6 shows a visualisation of the patch importance weights v that are learned at inference timeduring the inner loop adaptation for the support set images.
Revised: Figure 6 shows a visualization of the patch importance weights v that are learned at inference time during the inner loop adaptation for the support set images.

Original: Brighter regions represent a higherimportance weight, meaning that these patches will contribute most to the classification of querysamples if matches with high similarity can be found.
Revised: Brighter regions represent a higher importance weight, meaning that these patches will contribute most to the classification of query samples if matches with high similarity can be found.

Original: Judging from the visualized weights, FewTUREseems to consistently select characteristic regions of the depicted objects, e.g., the rim of the bowls,strings of the guitar or the dogs’ facial area, and to exclude unimportant or out-of-task information.
Revised: Judging from the visualized weights, FewTURE seems to consistently select characteristic regions of the depicted objects, e.g., the rim of the bowls, strings of the guitar or the dogs’ facial area, and to exclude unimportant or out-of-task information.

Original: We further investigate the influence of the number of optimization steps in our inner loop token importance weighting  using ViT-small on mini ImageNet.
Revised: We further investigate the influence of the number of optimization steps in our inner loop token importance weighting method using ViT-small on mini ImageNet.

Original: The results in Figure 7 indicate that increasingthe steps up to 20 aligns with increased performance, both during validation and testing.
Revised: The results in Figure 7 indicate that increasing the steps up to 20 aligns with increased performance, both during validation and testing.

Original: While theinitial increase in test accuracy when using our token reweighting (steps > 0 ) is rather significant with1 . 
Revised: While the initial increase in test accuracy when using our token reweighting (steps > 0 ) is rather significant with 1 . 15% , the contribution of higher step numbers comes at the cost of higher [Figure] [Table]

Original: 15% , the contribution of higher step numbers comes at the cost of higher computational complexity,and we generally found anything between 5 and 15 steps to be a good performance vs .
Revised: [Table] computational complexity [Table] and we generally found anything between 5 and 15 steps to be a good performance vs .

Original: inference-timetrade-off.
Revised: inference-time trade-off (see supplementary material for further details) [Table]

Original: 3.4 Evaluation on few-shot classification benchmarks
Revised: 3.4 Evaluation on few-shot classification benchmarks

Original: We conduct experiments using the few-shot settings established in the community, namely 5 -way1 -shot and 5 -way 5 -shot
Revised: We conduct experiments using the few-shot settings established in the community, namely 5 -way 1 -shot and 5 -way 5 -shot

Original: meaning the network has to distinguish samples from 5 novel classesbased on a provided number of 1 or 5 images per class.
Revised: meaning the network has to distinguish samples from 5 novel classes based on a provided number of 1 or 5 images per class.

Original: We evaluate our method FewTURE usingtwo different Transformer backbones and compare our results against the current state of the artin Table 1 for the mini ImageNet and tiered ImageNet, and in Table 2 for the CIFAR-100 and FC100datasets.
Revised: We evaluate our method FewTURE using two different Transformer backbones and compare our results against the current state of the art in Tablefor the mini ImageNet and tiered ImageNet [Table] and in Table 2 for the CIFAR-FS and FC100 datasets.

Original: It is to be noted that in contrast to previous works, we do not employ the help of anyconvolutional backbone but instead (and as far as we are aware for the first time) use a Transformerbackbone together with our previously introduced token importance reweighting method to achievethese results.
Revised: It is to be noted that in contrast to previous works, we do not employ the help of any convolutional backbone but instead (and as far as we are aware for the first time) use a Transformer backbone together with our previously introduced token importance reweighting method to achieve these results.

Original: We are able to set new state of the art results across all four datasets in both 5 -shot[Figure] and 1 -shot settings, improving particularly the 1 -shot results on mini ImageNet, tiered Imagenet and 
Revised: We are able to set new state of the art results across all four datasets in both 5 -shot and 1 -shot settings, improving particularly the 1 -shot results on mini ImageNet and tiered ImageNet by significant margins of 3 .

Original: CIFAR-FS by significant margins of 3 .

Original: 74% and 1 . 93% , respectively.
Revised: 03% and 1 . 74% , respectively.

Revised: 3.5 Increasing efficiency by pruning the token sequence

Original: [Table]
Revised: [Table]

Revised: To further improve the computational efficiency of our method, we investigate its behaviour when limiting the number of tokens that are considered to establish patch-wise correspondences to only a subset – allowing FewTURE to scale to potentially large many-way many-shot settings.

Revised: We use the attention maps inherent in our approach (averaged over all heads) to prune the number of patch tokens by only using the ones within the top-k attention values to compute the similarity matrix S .

Revised: The results obtained with our ViT-small backbone for pruning the number of tokens to 75% , 50% , 25% and 10% of the original token number indicate that such pruning might be an interesting avenue for future work, with our method still achieving 96 .

Revised: 4% of its original performance when only retaining 10% of the number of tokens.

Revised: 3.6 Ablation on the type of classifier

Original: [Table]
Revised: [Table]

Revised: We train a linear classifier as well as prototypical approach using our pre-trained ViT-small backbone for a 5-way 5-shot setting on mini Imagenet [48] to investigate the influence of the choice of classifier (Table 4).

Revised: We were able to obtain a test accuracy of 82 .

Revised: 80% for the prototypical network after optimising the pre-trained backbone with metafinetuning, which is competitive to the results we obtain for our method without reweighting (’0 step’ in Figure 7(b))

Revised: , but is clearly outperformed by our reweighting-based approach.

Revised: To provide a fair comparison, we optimize the linear classifier at inference time to adapt to the support set and obtained a maximum test accuracy of 82 .

Revised: Both results indicate the quality of embedding our backbone is able to produce but also demonstrate the importance of our task-specific reweighting-based approach.

Revised: For further ablation studies, please refer to the supplementary material.

Original: 3.5 Limitations
Revised: 3.7 Limitations

Original: Our introduced method is arguably more powerful for cases where multiple examples of the sameclass are provided, i.e. in N -way K -shot settings with K > 1 .
Revised: Our introduced method is arguably more powerful for cases where multiple examples of the same class are provided, i.e. in N -way K -shot settings with K > 1 .

Original: While FewTURE works well in1 -shot settings (Table 1 and 2), the inner loop adaptation procedure still aims to exclude cross-classsimilarities that hurt classification performance, but has less diverse information for selecting themost helpful regions due to the lack of other in-class comparison samples – which might thus yieldslightly less-refined token selections compared to multi-shot scenarios ( supplementary material ).
Revised: While FewTURE works well in 1 -shot settings (Table 1 and 2), the inner loop adaptation procedure still aims to exclude cross-class similarities that hurt classification performance, but has less diverse information for selecting the most helpful regions due to the lack of other in-class comparison samples – which might thus yield slightly less-refined token selections compared to multi-shot scenarios ( supplementary material ).

Original: While we did not face significant problems regarding the comparably very small size of our datasets(e.g. mini ImageNet with 38K compared to the usually used ImageNet with 1.28M training images),more specialized applications with highly limited training data might be negatively impacted andsuccessfully training our method due to the lack of inductive bias  in the Transformer architecturecould prove challenging.
Revised: While we did not face significant problems regarding the comparably very small size of our datasets (e.g. mini ImageNet with 38K compared to the usually used ImageNet with 1.28M training images), more specialized applications with highly limited training data might be negatively impacted and successfully training our method due to the reduced inductive bias present in the Transformer architecture could prove challenging.

Original:  Related work
Revised:  Related work

Original: Over the past few years, the family of few-shot learning (FSL) has grown diverse and broad.
Revised: Over the past few years, the family of few-shot learning (FSL) has grown diverse and broad.

Original: Thoseclosely related to this work can be categorized into two groups: metric-based methods [28; 29; 37;38; 46; 51; 52] and optimisation-based methods [12; 20; 31; 36; 59].
Revised: Those closely related to this work can be categorized into two groups: metric-based methods [30, 31, 40, 41, 48, 53, 54] and optimisation-based methods [13, 21, 33, 39, 61].

Original: Metric-based methods, such as
Revised: Metric-based methods, such as

Original: ProtoNet [38], DeepEMD [52], and RelationNet [40] aim to learn a class representation (prototype)by averaging the embeddings belonging to the same class and employ a predefined (ProtoNet and
Revised: ProtoNet [41], DeepEMD [54], and RelationNet [43] aim to learn a class representation (prototype) by averaging the embeddings belonging to the same class and employ a predefined (ProtoNet and

Original: DeepEMD) or learned (RelationNet) metric to perform prototype-query matching.
Revised: DeepEMD) or learned (RelationNet) metric to perform prototype-query matching.

Original: FEAT [51] takesthis a step further and uses a transformer head to adapt the  features given the novel tasks, whichresults in a more adaptive method.
Revised: FEAT [53] and TDM [22] take this a step further and use attention mechanisms to adapt the extracted features to the novel tasks.

Original: Our method instead fully utilizes the embeddings of local imageregions (patch tokens), preventing loss of information and supervision collapse occurring in theaforementioned prototype-based approaches (see supplementary material).
Revised: Our method instead fully utilizes the embeddings of local image regions (patch tokens), preventing loss of information and supervision collapse occurring in the aforementioned prototype-based approaches (see supplementary material).

Original: Optimisation-based methods such as MAML [12] and Reptile [31] propose to learn a set of initialmodel parameters that can quickly adapt to a novel task.
Revised: Optimisation-based methods such as MAML [13] and Reptile [33] propose to learn a set of initial model parameters that can quickly adapt to a novel task.

Original: However, updating all model parameters isoften not feasible given large backbones and only few labeled samples during inference.
Revised: However, updating all model parameters is often not feasible given large backbones and only few labeled samples during inference.

Original: To alleviatethis so-called meta-overfitting problem, CAVIA [59] and LEO [36] propose to learn and adapt alower dimensional representation that is mapped onto the network.
Revised: To alleviate this so-called meta-overfitting problem, CAVIA [61] and LEO [39] propose to learn and adapt a lower dimensional representation that is mapped onto the network.

Original: Our method is inspired by suchlower-dimensional adaptation strategies and learns a tiny set of context-aware re-weighting factorsonline for each novel task without requiring higher-order gradients, resulting in a flexible and efficientframework.
Revised: Our method is inspired by such lower-dimensional adaptation strategies and learns a tiny set of context-aware re-weighting factors online for each novel task without requiring higher-order gradients, resulting in a flexible and efficient framework.

Original: Self-supervised learning for FSL .
Revised: Self-supervised learning for FSL .

Original: Although self-supervised learning is underrepresented in thecontext of few-shot learning, some recent works [14; 30; 39] have shown that self-supervision viapretext tasks can be beneficial when integrated as auxiliary loss.
Revised: Although self-supervised learning is underrepresented in the context of few-shot learning, some recent works [15, 32, 42] have shown that self-supervision via pretext tasks can be beneficial when integrated as auxiliary loss.

Original: S2M2 [30] employs rotation [15]and exemplars [10] along with common supervised learning during the pre-training stage.
Revised: S2M2 [32] employs rotation [16] and exemplars [11] along with common supervised learning during the pre-training stage.

Original: CTX [8]demonstrates that SimCLR [6] can be combined with supervised-learning tasks in an episodictraining manner to learn a more generalized model.
Revised: CTX [9] demonstrates that SimCLR [6] can be combined with supervised-learning tasks in an episodictraining manner to learn a more generalized model.

Original: In contrast, FewTURE demonstrates that theself-supervised pretext task (i.e., Masked Image Modelling [2; 3; 22; 16; 33; 41; 57]), can be used ina standalone manner to learn more generalized features for few-shot learning on small-scale datasets.
Revised: In contrast, FewTURE demonstrates that the self-supervised pretext task (i.e., Masked Image Modelling [2, 3, 24, 17, 35, 44, 59]), can be used in a standalone manner to learn more generalized features for few-shot learning on small-scale datasets.

Original: Vision Transformers in FSL .
Revised: Vision Transformers in FSL .

Original: Transformers have been immensely successful in the field of Natural
Revised: Transformers have been immensely successful in the field of Natural

Original: Language Processing.
Revised: Language Processing.

Original: Recent studies [9; 21; 25; 44] suggest that encoding the long-range dependencyof data via self-attention also yields promising results for vision tasks (e.g., image classification, jointvision-language modeling, etc.).
Revised: Recent studies [10, 23, 27, 47] suggest that encoding the long-range dependency of data via self-attention also yields promising results for vision tasks (e.g., image classification, joint vision-language modeling, etc.).

Original: However, there is an important trade-off between leveraging therich representation capacity and the lack of inductive bias.
Revised: However, there is an important trade-off between leveraging the rich representation capacity and the lack of inductive bias.

Original: Transformers have gained a reputation togenerally require significantly more training data compared to convolutional neural networks (CNNs),since properties like translation invariance, locality and hierarchical structure of visual data have to beinferred from the data [24].
Revised: Transformers have gained a reputation to generally require significantly more training data compared to convolutional neural networks (CNNs), since properties like translation invariance, locality and hierarchical structure of visual data have to be inferred from the data [26].

Original: While this data-hungry nature largely prevented the use of Transformersin problems with scarce data like few-shot learning, some recent works demonstrate that a singletransformer head can be successfully adopted to perform feature adaptation [8; 51].
Revised: While this data-hungry nature largely prevented the use of Transformers in problems with scarce data like few-shot learning, some recent works demonstrate that a single Transformer head can be successfully adopted to perform feature adaptation [9, 53].

Original: To the best ofour knowledge, FewTURE is the first approach that uses a fully Transformer-based architecture toobtain representative embeddings while being trained exclusively on training data of the respectivefew-shot dataset.
Revised: To the best of our knowledge, FewTURE is the first approach that uses a fully Transformer-based architecture to obtain representative embeddings while being trained exclusively on training data of the respective few-shot dataset.

Original:  Conclusion
Revised:  Conclusion

Original: In this paper, we presented a novel approach to tackle the challenge of supervision collapse introducedby one-hot image-level labels in few-shot learning.
Revised: In this paper, we presented a novel approach to tackle the challenge of supervision collapse introduced by one-hot image-level labels in few-shot learning.

Original: We split the input images into smaller parts withhigher probability of being dominated by only one entity and encode these local regions using a Vision
Revised: We split the input images into smaller parts with higher probability of being dominated by only one entity and encode these local regions using a Vision

Original: Transformer architecture pretrained via Masked Image Modeling (MIM) in a self-supervised wayto learn a representative embedding space beyond the pure label information.
Revised: Transformer architecture pretrained via Masked Image Modeling (MIM) in a self-supervised way to learn a representative embedding space beyond the pure label information.

Original: We devise a classifierbased on patch embedding similarities and propose a token importance reweighting mechanismto refine the contribution of each local patch towards the overall classification result based aroundintra-class similarities and inter-class differences as a function of the support set information.
Revised: We devise a classifier based on patch embedding similarities and propose a token importance reweighting mechanism to refine the contribution of each local patch towards the overall classification result based around intra-class similarities and inter-class differences as a function of the support set information.

Original: Ourobtained results demonstrate that our proposed method alleviates the problem of supervision collapseby learning more generalized features, achieving new state-of-the-art results on four popular few-shotclassification datasets. 
Revised: Our obtained results demonstrate that our proposed method alleviates the problem of supervision collapse by learning more generalized features, achieving new state-of-the-art results on four popular few-shot classification datasets. Acknowledgements.

Revised: The authors would like to thank Zhou et al.

Revised: [59] for sharing their insights and code regarding self-supervised pretraining, as well as Dosovitskiy et al.

Revised: [10], Touvron et al.

Revised: [46] and

Revised: Liu et al.

Revised: [27] for sharing details of the ViT and Swin architectures.

Revised: Parts of this research were undertaken using the LIEF HPC-GPGPU Facility hosted at the University of Melbourne.

Revised: This Facility was established with the assistance of LIEF Grant LE170100200.