Original: Rethinking Generalization in Few-Shot Classification
Revised: Rethinking Generalization in Few-Shot Classification

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AffiliationAddress email
Revised: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Single image-level annotations only correctly describe an often small subset of animage’s content, particularly when complex real-world scenes are depicted.
Revised: One-hot image-level annotations only correctly describe an often small subset ofan image’s content, particularly when complex real-world scenes are depicted.

Original: Whilethis might be acceptable in many classification scenarios, it poses a significantchallenge for applications where the set of classes differs significantly betweentraining and test time.
Revised: While this is tolerable in many classification scenarios, it poses a significantchallenge for applications where the set of classes differs significantly betweentraining and test time.

Original: In this paper, we take a closer look at the implicationsin the context of few-shot learning .
Revised: In this paper, we take a closer look at the implicationsin the context of few-shot learning .

Original: Splitting the input samples into patches andencoding these via the help of Vision Transformers allows us to establish semanticcorrespondences between local regions across images and independent of theirrespective class.
Revised: Splitting the input samples into patches andencoding these via the help of Vision Transformers allows us to establish semanticcorrespondences between local regions across images and independent of theirrespective class.

Original: The most informative patch embeddings for the task at hand arethen determined as a function of the support set via online optimization at inferencetime, additionally providing visual interpretability of ‘ what matters most ’ in theimage.
Revised: The most informative patch embeddings for the task at hand arethen determined as a function of the support set via online optimization at inferencetime, additionally providing visual interpretability of ‘ what matters most ’ in theimage.

Original: We build on recent advances in unsupervised training of networks viamasked image modelling to overcome the lack of fine-grained labels and learn themore general statistical structure of the data while avoiding negative image-levelannotation influence, aka supervision collapse.
Revised: We build on recent advances in unsupervised training of networks viamasked image modelling to overcome the lack of fine-grained labels and learn themore general statistical structure of the data while avoiding negative image-levelannotation influence, aka supervision collapse.

Original: Experimental results show thecompetitiveness of our approach, achieving new state-of-the-art results on fourpopular few-shot classification benchmarks for 5 -shot and 1 -shot scenarios.
Revised: Experimental results show thecompetitiveness of our approach, achieving new state-of-the-art results on fourpopular few-shot classification benchmarks for 5 -shot and 1 -shot scenarios.

Original:  Introduction
Revised:  Introduction

Original: Images depicting real-world scenes are usually comprised of several different entities, e g., a personwalking their dog in a park surrounded by trees or a dog playing with a ball (Figure 1).
Revised: Images depicting real-world scenes are usually comprised of several different entities, e g., a personwalking their dog in a park surrounded by trees or a dog playing with a ball (Figure 1).

Original: Nevertheless,popular computer vision datasets like ImageNet [37] assign a single image-level annotation to classifytheir entire content.
Revised: Nevertheless,popular computer vision datasets like ImageNet [35] assign a single image-level annotation to classifytheir entire content.

Original: Hence, such a label only correctly applies to an often small subset of the actualimage.
Revised: Hence, such a label only correctly applies to an often small subset of the actualimage.

Original: As a result, models trained on such data via gradient-based methods learn to ignore allseemingly irrelevant information, particularly entities that occur across differently labelled images.
Revised: As a result, models trained on such data via gradient-based methods learn to ignore allseemingly irrelevant information, particularly entities that occur across differently labelled images.

Original: While this is acceptable for conventional classification methods that encounter a diverse number oftraining examples for all classes they are expected to classify, it poses a major but often overlookedchallenge for applications where the set of classes differs between training and test time.
Revised: While this is acceptable for conventional classification methods that encounter a diverse number oftraining examples for all classes they are expected to classify it poses a major but often overlookedchallenge for applications where the set of classes differs between training and test time.

Original: One suchaffected area is few-shot learning (FSL) where approaches are expected to correctly classify entirelynew classes at test time that have never been encountered during training, just by being providedwith a few (e.g., one or five) samples for each of these new categories.
Revised: One suchaffected area is few-shot learning (FSL) where approaches are expected to correctly classify entirelynew classes at test time that have never been encountered during training, just by being providedwith a few (e.g., one or five) samples for each of these new categories.

Original: During test time, entities thathave not been part of the set of training classes and have possibly been perceived as irrelevant mightvery well be part of the set of test classes – yet, the method was taught to ignore these.
Revised: During test time, entities thathave not been part of the set of training classes and have possibly been perceived as irrelevant mightvery well be part of the set of test classes – yet, the method was taught to ignore these.

Original: Similarly, amethod might overemphasize the importance of certain image patterns learned during training thatare however of no relevance for the test classes, resulting in supervision collapse [9].
Revised: Similarly, amethod might overemphasize the importance of certain image patterns learned during training thatare however of no relevance for the test classes, resulting in supervision collapse [8].

Original: Few previous works [9, 18] partially tackle the above challenges.
Revised: Few previous works [8; 17] partially tackle the above challenges.

Original: Particularly, CTX [9] proposes tolearn the spatial and semantic alignment between CNN-extracted query and support features using atransformer-style attention mechanism.
Revised: Particularly, CTX [8] proposes tolearn the spatial and semantic alignment between CNN-extracted query and support features using atransformer-style attention mechanism.

Original: The authors further show that self-supervised learning tasks(i.e.
Revised: The authors further show that self-supervised learning tasks(i.e.

Original: , SimCLR) can be integrated into episodic training along with normal supervised tasks to learn
Revised: , SimCLR) can be integrated into episodic training along with normal supervised tasks to learn

Original: [Figure]
Revised: [Figure]

Original: more generalized features, which benefits solving unseen tasks and mitigates supervision collapse.
Revised: more generalized features, which benefits solving unseen tasks and mitigates supervision collapse.

Original: CAN [18] achieves this in a similar manner by performing cross-attention between class prototypesand query feature maps, highlighting the region of a feature map important for classification duringinference.
Revised: CAN [17] achieves this in a similar manner by performing cross-attention between class prototypesand query feature maps, highlighting the region of a feature map important for classification duringinference.

Original: While both methods propose important contributions towards tackling supervision collapse,there exist important drawbacks.
Revised: While both methods propose important contributions towards tackling supervision collapse,there exist important drawbacks.

Original: Firstly, both methods build their ideas around aligning prototypesbased on each query.
Revised: Firstly, both methods build their ideas around aligning prototypesbased on each query.

Original: Such prototypes are merely class-aware and ignore all inter-class informationpresent in the support set – a part that has been shown to be crucial for few-shot learning [33, 47].
Revised: Such prototypes are merely class-aware and ignore all inter-class informationpresent in the support set – a part that has been shown to be crucial for few-shot learning [32; 46].

Original: Furthermore, learning query-aligned class representations requires performing the same operation foreach query, rendering such approaches rather inefficient at inference time.
Revised: Furthermore, learning query-aligned class representations requires performing the same operation foreach query, rendering such approaches rather inefficient at inference time.

Original: Summarizing our and previous works’ observations, we aim to address what we see as the two maincriteria: 1) building an understanding about the image’s structure and content that generalizes towardsnew classes, and 2) providing the ability to interpret the provided samples in context , i.e., finding theintra-class similarities and inter-class differences while jointly considering all available information.
Revised: Summarizing our and previous works’ observations, we aim to address what we see as the two maincriteria: 1) building an understanding about the image’s structure and content that generalizes towardsnew classes, and 2) providing the ability to interpret the provided samples in context , i.e., finding theintra-class similarities and inter-class differences while jointly considering all available information.

Original: Our work.
Revised: Our work.

Original: To alleviate the negative influence of image-level annotations and to avoid supervisioncollapse, we decompose the images into patches representing local regions, each having a higherlikelihood of being dominated by only one entity.
Revised: To alleviate the negative influence of image-level annotations and to avoid supervisioncollapse, we decompose the images into patches representing local regions, each having a higherlikelihood of being dominated by only one entity.

Original: To overcome the lack of such fine-grainedannotations we employ self-supervised training with Masked Image Modelling as pretext task [58]and use a Vision Transformer architecture [10] as encoder due to its patch-based nature.
Revised: To overcome the lack of such fine-grainedannotations we employ self-supervised training with Masked Image Modelling as pretext task [57]and use a Vision Transformer architecture [9] as encoder due to its patch-based nature.

Original: We buildour classification around the concept of learning task-specific similarity between local regions as afunction of the support set at inference time.
Revised: We buildour classification around the concept of learning task-specific similarity between local regions as afunction of the support set at inference time.

Original: To this extent, we first create a prior similarity map byestablishing semantic patch correspondences between all support set samples irrespective of theirclass, i.e. also between entities that might not be relevant or potentially even harmful for correctclassification.
Revised: To this extend, we first create a prior similarity map byestablishing semantic patch correspondences between all support set samples irrespective of theirclass, i.e. also between entities that might not be relevant or potentially even harmful for correctclassification.

Original: Assume a reduced support set with only two classes: ‘person’ and ‘cat’.
Revised: Assume a reduced support set with only two classes: ‘person’ and ‘cat’.

Original: Thinking backto Figure 1, the right image is part of our support set for ‘cat’ – and the dog just happens to be in theimage.
Revised: Thinkingback to Figure 1, the right image is part of our support set for ‘cat’ – and the dog just happens to bein the image.

Original: Now in the query sample that shall be classified, assume the image depicts a person walkingtheir dog.
Revised: Now in the query sample that shall be classified, assume the image depicts a personwalking their dog.

Original: We will thus correctly detect a correspondence of the two dogs across those two images, aswell as between the person patches and the other samples of the person support set class.
Revised: We will thus correctly detect a correspondence of the two dogs across those twoimages, as well as between the person patches and the other samples of the person support set class.

Original: While thecorrespondences between the person regions are helpful, there is no ‘dog’ class  in the actual supportset (i.e., ‘dog’ is out-of-task information), rendering this correspondence harmful for classificationsince it would indicate that the query is connected to the image with the ‘cat’ label.
Revised: While the correspondences between the person regions are helpful, there is no ‘dog’ class is in theactual support set (i.e., ‘dog’ is out-of-task information), rendering this correspondence harmful forclassification since it would indicate that the query is connected to the image with the ‘cat’ label.

Original: This is whereour token importance reweighting comes into play.
Revised: Thisis where our token importance reweighting comes into play.

Original: We infer an importance weight for each tokenbased on its contribution towards correct classification of the other support set samples, activelystrengthening intra-class similarities and inter-class differences by jointly considering all availableinformation – in other words, we learn which tokens ‘help’ or ‘harm’ our classification objective.
Revised: We infer an importance weight for eachtoken based on its contribution towards correct classification of the other support set samples, activelystrengthening intra-class similarities and inter-class differences by jointly considering all availableinformation – in other words, we learn which tokens ‘help’ or ‘harm’ our classification objective.

Original: These importance-reweighted support set embeddings are then used as basis for our similarity-basedquery sample classification.
Revised: These importance-reweighted support set embeddings are then used as basis for our similarity-basedquery sample classification.

Original: Our main contributions include the following
Revised: Our main contributions include the following

Original: We demonstrate that Transformer-only architectures in conjunction with self-supervisedpretraining can be successfully used in few-shot settings without the need of convolutional
Revised: We demonstrate that Transformer-only architectures in conjunction with self-supervisedpretraining can be successfully used in few-shot settings without the need of convolutional

Original: backbones or any additional data.
Revised: backbones or any additional data.

Original: We show that meta fine-tuning of Vision Transformers combined with our inner loop token importance reweighting can successfully use the supervision signal of provided support set
Revised: We show that meta fine-tuning of Vision Transformers combined with our inner loop tokenimportance reweighting can successfully use the supervision signal of provided support set

Original: labels while avoiding supervision collapse.
Revised: labels while avoiding supervision collapse.

Original: We provide insights into how establishing general similarities across images independent ofclasses followed by our optimization-based selection at inference time can boost generalization while allowing visual interpretability at the same time, and show the efficacy of our
Revised: We provide insights into how establishing general similarities across images independent ofclasses followed by our optimization-based selection at inference time can boost generalization while allowing visual interpretability at the same time, and show the efficacy of our

Original: C×H×W
Revised: C×H×W

Original: N·K·M×C ... ...
Revised: N·K·M×C ... ...

Original: N·K·L×D
Revised: N·K·L×D

Original: L×D M×C *
Revised: L×D M×C *

Original: N·K·L×L
Revised: N·K·L×L

Original: Token importance weights generator ...
Revised: Token importance weights generator ...

Original: N·K×L
Revised: N·K×L

Original: N·K·L×L
Revised: N·K·L×L

Original: N·K×C×H×W
Revised: N·K×C×H×W

Original: Logits aggregation low high
Revised: Logits aggregation low high

Original: Class probability x * Pair-wise similarity x
Revised: Class probability x * Pair-wise similarity x

Original: Broadcast & reweighting
Revised: Broadcast & reweighting

Original: N
Revised: N

Original:  Few-shot classification via reweighted embedding similarity
Revised:  Few-shot classification via reweighted embedding similarity

Original: We start this section by briefly introducing the problem setting we are tackling in this work: inductivefew-shot classification.
Revised: We start this section by briefly introducing the problem setting we are tackling in this work: inductivefew-shot classification.

Original: We then provide an overview of our proposed method FewTURE – Fewshot classification with Transformers Using Reweighted Embedding similarity (Figure 2) beforeelaborating on the main elements in more detail.
Revised: We then provide an overview of our proposed method FewTURE – Fewshot classification with Transformers Using Reweighted Embedding similarity (Figure 2) beforeelaborating on the main elements in more detail.

Original: Problem definition .
Revised: Problem definition .

Original: Inductive N -way K -shot few-shot classification aims to generalize knowledgelearned during training on D train to unseen test data D test , with classes C train ∩C test = ∅ , using onlya few labelled samples.
Revised: Inductive N -way K -shot few-shot classification aims to generalize knowledgelearned during training on D train to unseen test data D test , with classes C train ∩C test = ∅ , using onlya few labelled samples.

Original: We follow the meta-learning protocol of previous works [47] to formulate thefew-shot classification problem with episodic training and testing.
Revised: We follow the meta-learning protocol of previous works [46] to formulate thefew-shot classification problem with episodic training and testing.

Original: An episode E is composed of asupport set X s
Revised: An episode E is composed of asupport set X s

Original: = { ( x nks , y nks )
Revised: = { ( x nks , y nks )

Original: | n = 1 , . .
Revised: | n = 1 , . .

Original: N ; k = 1 , . . .
Revised: N ; k = 1 , . . .

Original: , K ; y nks ∈ C train } , where x nks denotes thek -th sample of class n with label y nks , and a query set X q
Revised: , K ; y nks ∈ C train } , where x nks denotes thek -th sample of class n with label y nks , and a query set X q

Original: = { ( x nq , y nq ) | n = 1 , . .
Revised: = { ( x nq , y nq ) | n = 1 , . .

Original: . , N } , where x nqdenotes a query sample 1 of class n with label y nq .
Revised: . , N } , where x nqdenotes a query sample 1 of class n with label y nq .

Original: 2.1 Overview on FewTURE
Revised: 2.1 Overview on FewTURE

Original: As depicted in Figure 2, we encode the image patches P s of the support set samples along with thequery sample patches p q via f θ and obtain corresponding sets of tokens Z s and z q , respectively.
Revised: As depicted in Figure 2, we encode the image patches P s of the support set samples along with thequery sample patches p q via f θ and obtain corresponding sets of tokens Z s and z q , respectively.

Original: It isto be noted that while we choose to illustrate our method via the use of one single query sample, theclassification of all query samples is computed at the same time in one single pass in practice.
Revised: It isto be noted that while we choose to illustrate our method via the use of one single query sample, theclassification of all query samples is computed at the same time in one single pass in practice.

Original: Weretrieve our ‘prior’ correspondence map S expressing the token-wise similarity between the encodedsemantic content of the local regions in the query sample and all patches of all support samples,allowing us to consider all available information jointly without incurring information loss due toaveraging or similar operations.
Revised: Weretrieve our ‘prior’ correspondence map S expressing the token-wise similarity between the encodedsemantic content of the local regions in the query sample and all patches of all support samples,allowing us to consider all available information jointly without incurring information loss due toaveraging or similar operations.

Original: This ‘prior’ similarity map represents correspondences betweenregions of samples irrespective of their individual class, i.e. also between entities that might not berelevant or potentially even harmful for correct classification.
Revised: This ‘prior’ similarity map represents correspondences betweenregions of samples irrespective of their individual class, i.e. also between entities that might not berelevant or potentially even harmful for correct classification.

Original: Using the annotated support set samples,we now infer a task-specific importance weight factor v j for each support token z js representing itscontribution to correctly classify other samples in the support set via online optimization at inferencetime (Section 2.4).
Revised: Using the annotated support set samples,we now infer a task-specific importance weight factor v j for each support token z js representing itscontribution to correctly classify other samples in the support set via online optimization at inferencetime (Section 2.4).

Original: We then reweight the prior similarities to obtain our classification result for thequery sample, jointly considering all available information.
Revised: We then reweight the prior similarities to obtain our classification result for thequery sample, jointly considering all available information.

Original: 2.2 Self-supervised pretraining against supervision collapse
Revised: 2.2 Self-supervised pretraining against supervision collapse

Original: To overcome the problem of supervision collapse induced by image-level annotations, we split theinput images into smaller parts where each region has a higher likelihood of only containing onemajor entity and hence a more distinct semantic meaning.
Revised: To overcome the problem of supervision collapse induced by image-level annotations, we split theinput images into smaller parts where each region has a higher likelihood of only containing onemajor entity and hence a more distinct semantic meaning.

Original: Since no labels are available for this morefine-grained data, we encode the information of each local region via an unsupervised method.
Revised: Since no labels are available for this morefine-grained data, we encode the information of each local region via an unsupervised method.

Original: We build our approach around the recently introduced idea of using Masked Image Modeling(MIM) [3, 58] as a pretext task for self-supervised training of Vision Transformers.
Revised: We build our approach around the recently introduced idea of using Masked Image Modeling(MIM) [3; 57] as a pretext task for self-supervised training of Vision Transformers.

Original: In contrast toprevious unsupervised approaches [5, 7] which focused mainly on global image-level representations, MIM randomly masks a number of patch embeddings ( tokens ) and aims to reconstruct them giventhe remaining information of the image.
Revised: In contrastto previous unsupervised approaches [5; 7] which focused mainly on global image-level representations, MIM randomly masks a number of patch embeddings ( tokens ) and aims to reconstruct themgiven the remaining information of the image.

Original: The  introduced token constraints help our Transformerbackbone to learn an embedding space that yields semantically meaningful representations for eachindividual image patch.
Revised: The in this way introduced token constraints helpour Transformer backbone to learn an embedding space that yields semantically meaningful representations for each individual image patch.

Original: We then leverage the information of the provided labels through fine-tuningthe pretrained backbone in conjunction with our inner loop token importance weighting describedin the following sections while successfully avoiding supervision collapse (see experimental resultsin Section 3.2).
Revised: We then leverage the information of the provided labelsthrough fine-tuning the pretrained backbone in conjunction with our inner loop token importanceweighting described in the following sections while successfully avoiding supervision collapse (seeexperimental results in Section 3.2).

Original: 2.3 Classification through reweighted token similarity
Revised: 2.3 Classification through reweighted token similarity

Original: As illustrated in Figure 2, we split each input image x ∈ R H × W × C into a sequence of M = H · W / P 2patches p
Revised: As illustrated in Figure 2, we split each input image x ∈ R H × W × C into a sequence of M = H · W / P 2patches p

Original: = { p i } Mi =1 , with each patch p i ∈
Revised: = { p i } Mi =1 , with each patch p i ∈

Original: R P 2 × C .
Revised: R P 2 × C .

Original: We then flatten and pass all patches of thesupport and query images as input to the Transformer architecture, obtaining the set of supporttokens Z s = f θ ( P s ) with Z s =
Revised: We then flatten and pass all patches of thesupport and query images as input to the Transformer architecture, obtaining the set of supporttokens Z s = f θ ( P s ) with Z s =

Original: { z nks | n =1 , . . .
Revised: { z nks | n =1 , . . .

Original: , N, k =1 , . .
Revised: , N, k =1 , . .

Original: , K } ,
Revised: , K } ,

Original: z nks = { z nkls | l =1 , . . .
Revised: z nks = { z nkls | l =1 , . . .

Original: , L ; z nkls ∈ 131 R D } and query tokens
Revised: , L ; z nkls ∈ 131 R D } and query tokens

Original: z q = f θ ( p q ) with z q = { z lq | l = 1 , . . .
Revised: z q = f θ ( p q ) with z q = { z lq | l = 1 , . . .

Original: , L ; z lq ∈ R D } .
Revised: , L ; z lq ∈ R D } .

Original: Vision Transformerslike ViT [10, 45] satisfy L = M
Revised: Vision Transformerslike ViT [9; 43] satisfy L = M

Original: whereas hierarchical Transformers like Swin [26] generally emit areduced number of tokens L < M due to internal merging strategies.
Revised: whereas hierarchical Transformers like Swin [25] generally emit areduced number of tokens L < M due to internal merging strategies.

Original: Having obtained all patch embeddings, we establish semantic correspondences by computing thepair-wise patch similarity matrix between the set of support tokens Z s and query tokens
Revised: Having obtained all patch embeddings, we establish semantic correspondences by computing thepair-wise patch similarity matrix between the set of support tokens Z s and query tokens

Original: z q as S ∈
Revised: z q as S ∈

Original: R N · K · L × L , where each element in S is obtained by s l s ,l q nk = sim
Revised: R N · K · L × L , where each element in S is obtained by s l s ,l q nk = sim

Original: ( z nkl s s , z l q q ) , where l s =1 , . . .
Revised: ( z nkl s s , z l q q ) , where l s =1 , . . .

Original: , L andl q =1 , . . .
Revised: , L andl q =1 , . . .

Original: , L .
Revised: , L .

Original: Note that local image regions representing similar entities exhibit higher scores.
Revised: Note that local image regions representing similar entities exhibit higher scores.

Original: Whileany distance metric can be used to compute the similarity ( sim ), we found cosine similarity to workparticularly well for this task.
Revised: Whileany distance metric can be used to compute the similarity ( sim ), we found cosine similarity to workparticularly well for this task.

Original: We then use the task-specific token importance weights v ∈ R N · K · L × 1inferred via online optimization based on the annotated support set samples (see Section 2.4) toreweight the similarities through column-wise addition and obtain our task-specific similarity matrixas ˜ S
Revised: We then use the task-specific token importance weights v ∈ R N · K · L × 1inferred via online optimization based on the annotated support set samples (see Section 2.4) toreweight the similarities through column-wise addition and obtain our task-specific similarity matrixas ˜ S

Original: = S + (cid:2) v · 1 1 × L (cid:3) , with elements ˜ s l s ,l q nk .
Revised: = S + (cid:2) v · 1 1 × L (cid:3) , with elements ˜ s l s ,l q nk .

Original: Note that this addition of our reweighting logitscorresponds to multiplicative reweighting in probability space.
Revised: Note that this addition of our reweighting logitscorresponds to multiplicative reweighting in probability space.

Original: We temperature-scale the adaptedsimilarity logits with 1 / τ S and aggregate the token similarity values across all elements belongingto the same support set class via a LogSumExp operation, i.e. aggregating K · L 2 logits per classfollowed by a softmax – resulting in the final class prediction ˆ y q for the query sample x q as 
Revised: We temperature-scale the adaptedsimilarity logits with 1 / τ S and aggregate the token similarity values across all elements belongingto the same support set class via a LogSumExp operation, i.e. aggregating K · L 2 logits per classfollowed by a softmax – resulting in the final class prediction ˆ y q for the query sample x q as [Equation]  [Equation]

Revised: 

Original: [Equation]
Revised: [Equation]

Original: 2.4 Learning token importance at inference time
Revised: 2.4 Learning token importance at inference time

Original: We use all samples of a task’s support set together with their annotations to learn the importance foreach individual patch token via online optimization at inference time.
Revised: We use all samples of a task’s support set together with their annotations to learn the importance foreach individual patch token via online optimization at inference time.

Original: As depicted in Figure 3, weformulate the same classification objective as in the previous section but aim to classify the supportset samples instead of a query sample.
Revised: As depicted in Figure 3, weformulate the same classification objective as in the previous section but aim to classify the supportset samples instead of a query sample.

Original: In other words, we use two versions of the support set samples:one with labels as ‘support’ Z s and one without as ‘pseudo-query’ Z sq , and obtain the similaritymatrix S s ∈ R N · K · L × N · K · L .
Revised: In other words, we use two versions of the support set samples:one with labels as ‘support’ Z s and one without as ‘pseudo-query’ Z sq , and obtain the similaritymatrix S s ∈ R N · K · L × N · K · L .

Original: The token importance weights are initialized to v 0 = 

Revised: The token importance weights are initialized to v 0 =

Original: ∈ R N · K · L × 1and column-wise added to form ˜ S s
Revised: ∈ R N · K · L × 1and column-wise added to form ˜ S s

Original: = S s + (cid:2) v 0 · 1 1 × N · K · L (cid:3) .
Revised: = S s + (cid:2) v 0 · 1 1 × N · K · L (cid:3) .

Original: The goal is now to determine which tokens of Z s are most helpful in contributing towards correctlyclassifying Z sq , and which ones negatively affect this objective.
Revised: The goal is now to determine which tokens of Z s are most helpful in contributing towards correctlyclassifying Z sq , and which ones negatively affect this objective.

Original: To prevent tokens from simplyclassifying themselves, we devise the following strategy for our N -way K -shot tasks.
Revised: To prevent tokens from simplyclassifying themselves, we device the following strategy for our N -way K -shot tasks.

Original: For scenarioswith K > 1 samples per class, we apply block-diagonal masking with blocks of size L × L to thesimilarity matrix ˜ S s – meaning that we enforce classification of each token in Z sq exclusively basedon information from other images.
Revised: For scenarioswith K > 1 samples per class, we apply block-diagonal masking with blocks of size L × L to thesimilarity matrix ˜ S s – meaning that we enforce classification of each token in Z sq exclusively basedon information from other images.

Original: Since there are no other in-class examples available in 1 -shotscenarios, we slightly weaken the constraint and apply local masking in a m × m window aroundeach patch, forcing the token to be classified based on the remaining information in the image.
Revised: Since there are no other in-class examples available in 1 -shotscenarios, we slightly weaken the constraint and apply local masking in a m × m window aroundeach patch, forcing the token to be classified based on the remaining information in the image.

Original: Wefound a local window of m = 5 to work well throughout our experiments for both architectures.
Revised: Wefound a local window of m = 5 to work well throughout our experiments for both architectures.

Original: N·K·L×D
Revised: N·K·L×D

Original: N·K·L×N·K·L N·K·L×N·K·L ...
Revised: N·K·L×N·K·L N·K·L×N·K·L ...

Original: Class probability
Revised: Class probability

Original: Class label
Revised: Class label

Original: Loss
Revised: Loss

Original: Logits aggregation
Revised: Logits aggregation

Original: N·K×L ...
Revised: N·K×L ...

Original: Update
Revised: Update

Original: Backpropagate
Revised: Backpropagate

Original: Forward pass
Revised: Forward pass

Original: Pair-wise similarity
Revised: Pair-wise similarity

Original: Broadcast&addition
Revised: Broadcast&addition

Original: N·K×L
Revised: N·K×L

Original: Masking
Revised: Masking

Original: N·K·L×D
Revised: N·K·L×D

Original: We use temperature-scaling and aggregate all modified similarity logits across all elements belongingto the same support set class of the annotated Z s for each element z nksq , apply a softmax operationand obtain the predicted class probabilities ˆ y nks for each support set sample ( cf.
Revised: We use temperature-scaling and aggregate all modified similarity logits across all elements belongingto the same support set class of the annotated Z s for each element z nksq , apply a softmax operationand obtain the predicted class probabilities ˆ y nks for each support set sample ( cf.

Original: Equation (1)).
Revised: Equation (1)).

Original: Giventhat the prediction is now dependent on the initialized token similarity weights v , we can formulatean online optimization objective by using the support set labels y nks as
Revised: Giventhat the prediction is now dependent on the initialized token similarity weights v , we can formulatean online optimization objective by using the support set labels y nks as

Original: [Equation]
Revised: [Equation]

Original: It is to be noted that by using column-wise addition of v , we share the importance weights of eachsupport token across all ‘pseudo-query’ tokens and thus constrain the optimization to jointly learntoken importance with respect to all available information.
Revised: It is to be noted that by using column-wise addition of v , we share the importance weights of eachsupport token across all ‘pseudo-query’ tokens and thus constrain the optimization to jointly learntoken importance with respect to all available information.

Original: In other words, task-specific intra-classsimilarities will be strengthened while inter-class ones will be penalized accordingly.
Revised: In other words, task-specific intra-classsimilarities will be strengthened while inter-class ones will be penalized accordingly.

Original: We furtherdo not require any second order derivatives like other methods during meta fine-tuning, since theoptimization of v is decoupled from the network’s parameters.
Revised: We furtherdo not require any second order derivatives like other methods during meta fine-tuning, since theoptimization of v is decoupled from the network’s parameters.

Original:  Experiments and discussion
Revised:  Experiments and discussion

Original: 3.1 Implementation details
Revised: 3.1 Implementation details

Original: Our training strategy is divided into two parts: self-supervised pretraining followed by meta finetuning.
Revised: Our training strategy is divided into two parts: self-supervised pretraining followed by meta finetuning.

Original: It is to be noted that each architecture is exclusively trained on the training set data of therespective dataset that is to be evaluated, and no additional data is used.
Revised: It is to be noted that each architecture is exclusively trained on the training set data of therespective dataset that is to be evaluated, and no additional data is used.

Original: Datasets .
Revised: Datasets .

Original: We train and evaluate our methods using four popular few-shot classification benchmarks,namely mini ImageNet [47], tiered ImageNet [36], CIFAR-FS [4] and FC-100 [33].
Revised: We train and evaluate our methods using four popular few-shot classification benchmarks,namely mini ImageNet [46], tiered ImageNet [34], CIFAR-FS [4] and FC-100 [32].

Original: Architectures .
Revised: Architectures .

Original: We compare two different Transformer architectures in this work, the monolithic
Revised: We compare two different Transformer architectures in this work, the monolithic

Original: ViT architecture [10, 45] in its ‘small’ form ( ViT-small aka DeiT-small ) and the multi-scale Swinarchitecture [26] in its ‘tiny’ version ( Swin-tiny ).
Revised: ViT architecture [9; 43] in its ‘small’ form ( ViT-small aka DeiT-small ) and the multi-scale Swinarchitecture [25] in its ‘tiny’ version ( Swin-tiny ).

Original: Self-supervised pretraining .
Revised: Self-supervised pretraining .

Original: We employ the strategy proposed by [58] to pretrain our Transformerbackbones and mostly stick to the hyperparameter settings reported in their work.
Revised: We employ the strategy proposed by [57] to pretrain our Transformerbackbones and mostly stick to the hyperparameter settings reported in their work.

Original: Our ViT and Swinarchitectures are trained with a batch size of 512 for 1600 and 800 epochs, respectively.
Revised: Our ViT and Swinarchitectures are trained with a batch size of 512 for 1600 and 800 epochs, respectively.

Original: We use 

Revised: We use

Original: Nvidia A100 GPUs with 40GB each for our ViT and 8 such GPUs for our Swin models.
Revised: Nvidia A100 GPUs with 40GB each for our ViT and 8 such GPUs for our Swin models.

Original: Meta fine-tuning .
Revised: Meta fine-tuning .

Original: We use meta fine-tuning to further refine our embedding space by using theavailable image-level labels in conjunction with our token-reweighting method.
Revised: We use meta fine-tuning to further refine our embedding space by using theavailable image-level labels in conjunction with our token-reweighting method.

Original: We generally trainfor up to 200 epochs but find most architectures to converge earlier.
Revised: We generally trainfor up to 200 epochs but find most architectures to converge earlier.

Original: We evaluate at each epoch
Revised: We evaluate at each epoch

Original: on600 randomly sampled episodes from the respective validation set to select the best set of parameters.
Revised: on600 randomly sampled episodes from the respective validation set to select the best set of parameters.

Original: During test time, we randomly sample 600 episodes from the test set to evaluate our model.
Revised: During test time, we randomly sample 600 episodes from the test set to evaluate our model.

Original: Token importance reweighting and classifier .
Revised: Token importance reweighting and classifier .

Original: We use cosine similarity to compute S .
Revised: We use cosine similarity to compute S .

Original: While thetemperature τ S used to scale the logits can be learnt during meta fine-tuning, we found τ S = 1 / √ dto be a good default value (or starting point if learnt, see supplementary material).
Revised: While thetemperature τ S used to scale the logits can be learnt during meta fine-tuning, we found τ S = 1 / √ dto be a good default value (or starting point if learnt, see supplementary material).

Original: We use SGD asoptimizer with a learning rate of 0.
Revised: We use SGD asoptimizer with a learning rate of 0.

Original: [Figure]
Revised: [Figure]

Original: Patch embeddings of the entire support set , stepw/ , step

Original: Please refer to supplementary material of this paper for a more detailed discussion regarding datasets,implementation and hyperparameters.
Revised: Please refer to supplementary material of this paper for a more detailed discussion regarding datasets,implementation and hyperparameters.

Original: 3.2 Self-supervised pretraining and token-reweighted fine-tuning improve generalization
Revised: 3.2 Self-supervised pretraining and token-reweighted fine-tuning improve generalization

Original: In this section, we investigate the influence of self-supervised pretraining compared to its supervisedcounterpart using the provided image-level labels (de-facto standard in most current state-of-the-artmethods).
Revised: In this section, we investigate the influence of self-supervised pretraining compared to its supervisedcounterpart using the provided image-level labels (de-facto standard in most current state-of-the-artmethods).

Original: We only vary the training strategy of our backbone and use our introduced token similaritybased classifier with 15 inner-loop reweighting steps for all experiments.
Revised: We only vary the training strategy of our backbone and use our introduced token similaritybased classifier with 15 inner-loop reweighting steps for all experiments.

Original: Figure 4 illustrates that both
Revised: Figure 4 illustrates that both

Original: ViT-small and Swin-tiny with self-supervised pretraining alone ( w/o meta fine-tuning) learn moregeneralizable features than their respective supervised version for both 1 -shot (left-hand side) and5 -shot (right-hand side) scenarios.
Revised: ViT-small and Swin-tiny with self-supervised pretraining alone ( w/o meta fine-tuning) learn moregeneralizable features than their respective supervised version for both 1 -shot (left-hand side) and5 -shot (right-hand side) scenarios.

Original: In fact, across all cases w/o meta fine-tuning, the self-supervisedpretrained Transformers outperform their supervised counterparts by more than 10% and up to asignificant 18 .
Revised: In fact, across all cases w/o meta fine-tuning, the self-supervisedpretrained Transformers outperform their supervised counterparts by more than 10% and up to asignificant 18 .

Original: 19% in the case of Swin-tiny.
Revised: 19% in the case of Swin-tiny.

Original: This clearly demonstrates that our self-supervisedpretraining strategy explores information that is richer and beyond the labels.
Revised: This clearly demonstrates that our self-supervisedpretraining strategy explores information that is richer and beyond the labels.

Original: Another interestinginsight is that the meta fine-tuning stage does not improve supervised backbones as much as theirunsupervised counterparts.
Revised: Another interestinginsight is that the meta fine-tuning stage does not improve supervised backbones as much as theirunsupervised counterparts.

Original: Specifically after unsupervised pretraining, meta fine-tuning of the
Revised: Specifically after unsupervised pretraining, meta fine-tuning of the

Original: Transformers in conjunction with our token-reweighting strategy is able to boost the performance by6 .
Revised: Transformers in conjunction with our token-reweighting strategy is able to boost the performance by6 .

Original: 77% (ViT-small) and 13 .
Revised: 77% (ViT-small) and 13 .

Original: 01% (Swin-tiny) for 1 -shot, and 9 .
Revised: 01% (Swin-tiny) for 1 -shot, and 9 .

Original: 94% (ViT-small) and 10 .
Revised: 94% (ViT-small) and 10 .

Original: 10% (Swintiny) for 5 -shot.
Revised: 10% (Swintiny) for 5 -shot.

Original: While such significant improvements cannot be observed across the supervisednetworks, the Swin versions seem to generally start off lower after pretraining but benefit more fromthe fine-tuning than ViT.
Revised: While such significant improvements cannot be observed across the supervisednetworks, the Swin versions seem to generally start off lower after pretraining but benefit more fromthe fine-tuning than ViT.

Original: The observed results clearly indicate that our token-reweighted fine-tuningstrategy is able to further improve the generalization of self-supervised Transformers, thus performingbetter on the novel tasks of the unseen test set.
Revised: The observed results clearly indicate that our token-reweighted fine-tuningstrategy is able to further improve the generalization of self-supervised Transformers, thus performingbetter on the novel tasks of the unseen test set.

Original: Figure 5 additionally depicts projected views ofthe tokens of 5 instances from a novel classas well as the entire support set in embedding space.
Revised: Figure 5 additionally depicts projected views of thetokens of 5 instances from a novel class in embedding space.

Original: Representations obtained with our classifier seem to retain the instance information (‘ w/o v ’) andseparation is improved when using token importance reweighting (‘ w/ v ’).While token embeddingsshow partial overlap between support set classes as is expected due to commonalities like e.g. similarbackground, our reweighting clearly determines the class-characteristic tokens.
Revised: While the representations obtainedfrom the network meta fine-tuned by using common averaging over the embeddings (‘ average ’)does not exhibit any clear separation of the instances, the embeddings obtained with our classifierseem to retain the instance information (‘ w/o v ’) and separation is improved when using tokenimportance reweighting (‘ w/ v ’).

Original: These results indicatethat our similarity-based classifier coupled with task-specific token reweighting is able to betterdisentangle the embeddings of different instances from the same class as well as other classes, which prevents the network from supervision collapse and achieves the higher performance observed onthe benchmarks.
Revised: These results indicate that our similarity-based classifier coupledwith task-specific token reweighting is able to better disentangle the embeddings of different instancesfrom the same class, which prevents the network from supervision collapse and achieves the higher performance observed on the benchmarks.

Original: They further show that self-supervised pretraining is helpful but not sufficient toachieve well-separated representations without supervision collapse that are suitable for few-shotclassification.
Revised: They further show that self-supervised pretraining ishelpful but not sufficient to achieve well-separated representations without supervision collapse thatare suitable for few-shot classification.

Original: 3.3 Selecting helpful patches at inference time
Revised: 3.3 Selecting helpful patches at inference time

Original: [Figure] Figure 6 shows a visualisation of the patch importance weights v that are learned at inference timeduring the inner loop adaptation for the support set images.
Revised: Figure 6 shows a visualisation of the patch importance weights v that are learned at inference timeduring the inner loop adaptation for the support set images.

Original: Brighter regions represent a higherimportance weight, meaning that these patches will contribute most to the classification of querysamples if matches with high similarity can be found.
Revised: Brighter regions represent a higherimportance weight, meaning that these patches will contribute most to the classification of querysamples if matches with high similarity can be found.

Original: Judging from the visualized weights, FewTUREseems to consistently select characteristic regions of the depicted objects, e.g., the rim of the bowls,strings of the guitar or the dogs’ facial area, and to exclude unimportant or out-of-task information.
Revised: Judging from the visualized weights, FewTUREseems to consistently select characteristic regions of the depicted objects, e.g., the rim of the bowls,strings of the guitar or the dogs’ facial area, and to exclude unimportant or out-of-task information.

Original: We further investigate the influence of the number of optimization steps in our inner loop token importance weighting using ViT-small on mini ImageNet.
Revised: We further investigate the influence of the number of optimization steps in our inner loop token importance weighting using ViT-small on mini ImageNet.

Original: The results in Figure 7 indicate that increasingthe steps up to 20 aligns with increased performance, both during validation and testing.
Revised: The results in Figure 7 indicate that increasingthe steps up to 20 aligns with increased performance, both during validation and testing.

Original: While theinitial increase in test accuracy when using our token reweighting (steps > 0 ) is rather significant with1 .
Revised: While theinitial increase in test accuracy when using our token reweighting (steps > 0 ) is rather significant with1 .

Original: 15% , the contribution of higher step numbers comes at the cost of higher computational complexity,and we generally found anything between 5 and 15 steps to be a good performance vs .
Revised: 15% , the contribution of higher step numbers comes at the cost of higher computational complexity,and we generally found anything between 5 and 15 steps to be a good performance vs .

Original: inference-timetrade-off.
Revised: inference-timetrade-off.

Original: 3.4 Evaluation on few-shot classification benchmarks
Revised: 3.4 Evaluation on few-shot classification benchmarks

Original: We conduct experiments using the few-shot settings established in the community, namely 5 -way1 -shot and 5 -way 5 -shot
Revised: We conduct experiments using the few-shot settings established in the community, namely 5 -way1 -shot and 5 -way 5 -shot

Original: meaning the network has to distinguish samples from 5 novel classesbased on a provided number of 1 or 5 images per class.
Revised: meaning the network has to distinguish samples from 5 novel classesbased on a provided number of 1 or 5 images per class.

Original: We evaluate our method FewTURE usingtwo different Transformer backbones and compare our results against the current state of the artin Table 1 for the mini ImageNet and tiered ImageNet, and in Table 2 for the CIFAR-100 and FC100datasets.
Revised: We evaluate our method FewTURE usingtwo different Transformer backbones and compare our results against the current state of the artin Table 1 for the mini ImageNet and tiered ImageNet, and in Table 2 for the CIFAR-100 and FC100datasets.

Original: It is to be noted that in contrast to previous works, we do not employ the help of anyconvolutional backbone but instead (and as far as we are aware for the first time) use a Transformerbackbone together with our previously introduced token importance reweighting method to achieve
Revised: It is to be noted that in contrast to previous works, we do not employ the help of anyconvolutional backbone but instead (and as far as we are aware for the first time) use a Transformerbackbone together with our previously introduced token importance reweighting method to achievethese results.

Revised: We are able to set new state of the art results across all four datasets in both 5 -shot

Original: [Figure]
Revised: [Figure]

Original: these results. We are able to set new state-of-the-art results across all four datasets in both 5 -shotand 1 -shot settings, improving particularly the 1 -shot results on mini ImageNet, tiered ImageNet and
Revised: and 1 -shot settings, improving particularly the 1 -shot results on mini ImageNet, tiered Imagenet and

Original: CIFAR-FS by significant margins of 3 .
Revised: CIFAR-FS by significant margins of 3 .

Original: 74% and 1 .
Revised: 74% and 1 .

Original: 93% , respectively.
Revised: 93% , respectively.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 3.5 Limitations
Revised: 3.5 Limitations

Original: Our introduced method is arguably more powerful for cases where multiple examples of the sameclass are provided, i.e. in N -way K -shot settings with K > 1 .
Revised: Our introduced method is arguably more powerful for cases where multiple examples of the sameclass are provided, i.e. in N -way K -shot settings with K > 1 .

Original: While FewTURE works well in1 -shot settings (Table 1 and 2), the inner loop adaptation procedure still aims to exclude cross-classsimilarities that hurt classification performance, but has less diverse information for selecting themost helpful regions due to the lack of other in-class comparison samples – which might thus yieldslightly less-refined token selections compared to multi-shot scenarios ( supplementary material ).
Revised: While FewTURE works well in1 -shot settings (Table 1 and 2), the inner loop adaptation procedure still aims to exclude cross-classsimilarities that hurt classification performance, but has less diverse information for selecting themost helpful regions due to the lack of other in-class comparison samples – which might thus yieldslightly less-refined token selections compared to multi-shot scenarios ( supplementary material ).

Original: While we did not face significant problems regarding the comparably very small size of our datasets(e.g.
Revised: While we did not face significant problems regarding the comparably very small size of our datasets(e.g.

Original: mini ImageNet with 38K compared to the usually used ImageNet with 1.28M training images),more specialized applications with highly limited training data might be negatively impacted andsuccessfully training our method due to the lack of inductive bias in the Transformer architecturecould prove challenging.
Revised: mini ImageNet with 38K compared to the usually used ImageNet with 1.28M training images),more specialized applications with highly limited training data might be negatively impacted andsuccessfully training our method due to the lack of inductive bias in the Transformer architecturecould prove challenging.

Original:  Related work
Revised:  Related work

Original: Over the past few years, the family of few-shot learning (FSL) has grown diverse and broad.
Revised: Over the past few years, the family of few-shot learning (FSL) has grown diverse and broad.

Original: Thoseclosely related to this work can be categorized into two groups: metric-based methods [29, 30, 39,40, 47, 52, 53] and optimisation-based methods [13, 21, 32, 38, 60].
Revised: Thoseclosely related to this work can be categorized into two groups: metric-based methods [28; 29; 37;38; 46; 51; 52] and optimisation-based methods [12; 20; 31; 36; 59].

Original: Metric-based methods, such as
Revised: Metric-based methods, such as

Original: ProtoNet [40], DeepEMD [53], and RelationNet [42] aim to learn a class representation (prototype)by averaging the embeddings belonging to the same class and employ a predefined (ProtoNet and
Revised: ProtoNet [38], DeepEMD [52], and RelationNet [40] aim to learn a class representation (prototype)by averaging the embeddings belonging to the same class and employ a predefined (ProtoNet and

Original: DeepEMD) or learned (RelationNet) metric to perform prototype-query matching.
Revised: DeepEMD) or learned (RelationNet) metric to perform prototype-query matching.

Original: FEAT [52] takesthis a step further and uses a transformer head to adapt the features given the novel tasks, whichresults in a more adaptive method.
Revised: FEAT [51] takesthis a step further and uses a transformer head to adapt the features given the novel tasks, whichresults in a more adaptive method.

Original: Our method instead fully utilizes the embeddings of local imageregions (patch tokens), preventing loss of information and supervision collapse occurring in theaforementioned prototype-based approaches (see supplementary material).
Revised: Our method instead fully utilizes the embeddings of local imageregions (patch tokens), preventing loss of information and supervision collapse occurring in theaforementioned prototype-based approaches (see supplementary material).

Original: Optimisation-based methods such as MAML [13] and Reptile [32] propose to learn a set of initialmodel parameters that can quickly adapt to a novel task.
Revised: Optimisation-based methods such as MAML [12] and Reptile [31] propose to learn a set of initialmodel parameters that can quickly adapt to a novel task.

Original: However, updating all model parameters isoften not feasible given large backbones and only few labeled samples during inference.
Revised: However, updating all model parameters isoften not feasible given large backbones and only few labeled samples during inference.

Original: To alleviatethis so-called meta-overfitting problem, CAVIA [60] and LEO [38] propose to learn and adapt alower dimensional representation that is mapped onto the network.
Revised: To alleviatethis so-called meta-overfitting problem, CAVIA [59] and LEO [36] propose to learn and adapt alower dimensional representation that is mapped onto the network.

Original: Our method is inspired by suchlower-dimensional adaptation strategies and learns a tiny set of context-aware re-weighting factorsonline for each novel task without requiring higher-order gradients, resulting in a flexible and efficientframework.
Revised: Our method is inspired by suchlower-dimensional adaptation strategies and learns a tiny set of context-aware re-weighting factorsonline for each novel task without requiring higher-order gradients, resulting in a flexible and efficientframework.

Original: Self-supervised learning for FSL .
Revised: Self-supervised learning for FSL .

Original: Although self-supervised learning is underrepresented in thecontext of few-shot learning, some recent works [15, 31, 41] have shown that self-supervision viapretext tasks can be beneficial when integrated as auxiliary loss.
Revised: Although self-supervised learning is underrepresented in thecontext of few-shot learning, some recent works [14; 30; 39] have shown that self-supervision viapretext tasks can be beneficial when integrated as auxiliary loss.

Original: S2M2 [31] employs rotation [16]and exemplars [11] along with common supervised learning during the pre-training stage.
Revised: S2M2 [30] employs rotation [15]and exemplars [10] along with common supervised learning during the pre-training stage.

Original: CTX [9]demonstrates that SimCLR [6] can be combined with supervised-learning tasks in an episodictraining manner to learn a more generalized model.
Revised: CTX [8]demonstrates that SimCLR [6] can be combined with supervised-learning tasks in an episodictraining manner to learn a more generalized model.

Original: In contrast, FewTURE demonstrates that theself-supervised pretext task (i.e., Masked Image Modelling [2, 3, 23, 17, 34, 43, 58]), can be used ina standalone manner to learn more generalized features for few-shot learning on small-scale datasets.
Revised: In contrast, FewTURE demonstrates that theself-supervised pretext task (i.e., Masked Image Modelling [2; 3; 22; 16; 33; 41; 57]), can be used ina standalone manner to learn more generalized features for few-shot learning on small-scale datasets.

Original: Vision Transformers in FSL .
Revised: Vision Transformers in FSL .

Original: Transformers have been immensely successful in the field of Natural
Revised: Transformers have been immensely successful in the field of Natural

Original: Language Processing.
Revised: Language Processing.

Original: Recent studies [10, 22, 26, 46] suggest that encoding the long-range dependency of data via self-attention also yields promising results for vision tasks (e.g., image classification,joint vision-language modeling, etc.).
Revised: Recent studies [9; 21; 25; 44] suggest that encoding the long-range dependencyof data via self-attention also yields promising results for vision tasks (e.g., image classification, jointvision-language modeling, etc.).

Original: However, there is an important trade-off between leveraging therich representation capacity and the lack of inductive bias.
Revised: However, there is an important trade-off between leveraging therich representation capacity and the lack of inductive bias.

Original: Transformers have gained a reputation togenerally require significantly more training data compared to convolutional neural networks (CNNs),since properties like translation invariance, locality and hierarchical structure of visual data have to beinferred from the data [25].
Revised: Transformers have gained a reputation togenerally require significantly more training data compared to convolutional neural networks (CNNs),since properties like translation invariance, locality and hierarchical structure of visual data have to beinferred from the data [24].

Original: While this data-hungry nature largely prevented the use of Transformersin problems with scarce data like few-shot learning, some recent works demonstrate that a singletransformer head can be successfully adopted to perform feature adaptation [9, 52].
Revised: While this data-hungry nature largely prevented the use of Transformersin problems with scarce data like few-shot learning, some recent works demonstrate that a singletransformer head can be successfully adopted to perform feature adaptation [8; 51].

Original: To the best ofour knowledge, FewTURE is the first approach that uses a fully Transformer-based architecture toobtain representative embeddings while being trained exclusively on training data of the respectivefew-shot dataset.
Revised: To the best ofour knowledge, FewTURE is the first approach that uses a fully Transformer-based architecture toobtain representative embeddings while being trained exclusively on training data of the respectivefew-shot dataset.

Original:  Conclusion
Revised:  Conclusion

Original: In this paper, we presented a novel approach to tackle the challenge of supervision collapse introducedby one-hot image-level labels in few-shot learning.
Revised: In this paper, we presented a novel approach to tackle the challenge of supervision collapse introducedby one-hot image-level labels in few-shot learning.

Original: We split the input images into smaller parts withhigher probability of being dominated by only one entity and encode these local regions using a Vision
Revised: We split the input images into smaller parts withhigher probability of being dominated by only one entity and encode these local regions using a Vision

Original: Transformer architecture pretrained via Masked Image Modeling (MIM) in a self-supervised wayto learn a representative embedding space beyond the pure label information.
Revised: Transformer architecture pretrained via Masked Image Modeling (MIM) in a self-supervised wayto learn a representative embedding space beyond the pure label information.

Original: We devise a classifierbased on patch embedding similarities and propose a token importance reweighting mechanismto refine the contribution of each local patch towards the overall classification result based aroundintra-class similarities and inter-class differences as a function of the support set information.
Revised: We devise a classifierbased on patch embedding similarities and propose a token importance reweighting mechanismto refine the contribution of each local patch towards the overall classification result based aroundintra-class similarities and inter-class differences as a function of the support set information.

Original: Ourobtained results demonstrate that our proposed method alleviates the problem of supervision collapseby learning more generalized features, achieving new state-of-the-art results on four popular few-shotclassification datasets.
Revised: Ourobtained results demonstrate that our proposed method alleviates the problem of supervision collapseby learning more generalized features, achieving new state-of-the-art results on four popular few-shotclassification datasets.