Original: Improving Task-Specific Generalization in Few-Shot Learning via Adaptive Vicinal Risk Minimization
Revised: Improving Task-Speciﬁc Generalization in Few-Shot Learning via Adaptive Vicinal Risk Minimization

Original: Anonymous Author(s)

Original: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Recent years have witnessed the rapid development of meta-learning in improvingthe meta generalization over tasks in few-shot learning.
Revised: Recent years have witnessed the rapid development of meta-learning in improving the meta generalization over tasks in few-shot learning.

Original: However, the task-specificlevel generalization is overlooked in most algorithms.
Revised: However, the task-speciﬁc level generalization is overlooked in most algorithms.

Original: For a novel few-shot learningtask where the empirical distribution likely deviates from the true distribution, themodel obtained via minimizing the empirical loss can hardly generalize to unseendata.
Revised: For a novel few-shot learning task where the empirical distribution likely deviates from the true distribution, the model obtained via minimizing the empirical loss can hardly generalize to unseen data.

Original: A viable solution to improving the generalization comes as a more accurateapproximation of the true distribution; that is, admitting a Gaussian-like vicinaldistribution for each of the limited training samples.
Revised: A viable solution to improving the generalization comes as a more accurate approximation of the true distribution; that is, admitting a Gaussian-like vicinal distribution for each of the limited training samples.

Original: Thereupon we derive theresulting vicinal loss function over vicinities of all training samples and minimizeit instead of the conventional empirical loss over training samples only, favorablyfree from the exhaustive sampling of all vicinal samples.
Revised: Thereupon we derive the resulting vicinal loss function over vicinities of all training samples and minimize it instead of the conventional empirical loss over training samples only, favorably free from the exhaustive sampling of all vicinal samples.

Original: It remains challengingto obtain the statistical parameters of the vicinal distribution for each sample.
Revised: It remains challenging to obtain the statistical parameters of the vicinal distribution for each sample.

Original: Totackle this challenge, we further propose to estimate the statistical parameters asthe weighted mean and variance of a set of unlabeled data it passed by a randomwalk starting from training samples.
Revised: To tackle this challenge, we further propose to estimate the statistical parameters as the weighted mean and variance of a set of unlabeled data it passed by a random walk starting from training samples.

Original: To verify the performance of the proposedmethod, we conduct experiments on four standard few-shot learning benchmarksand consolidate the superiority of the proposed method over state-of-the-art fewshot learning baselines.
Revised: To verify the performance of the proposed method, we conduct experiments on three standard few-shot learning benchmarks and consolidate the superiority of the proposed method over state-of-the-art few-shot learning baselines.

Original:  Introduction
Revised:  Introduction

Original: In recent years, deep learning algorithms have made tremendous progress and have been widelyapplied in vision [14], language [31] and speech [8] areas.
Revised: In recent years, deep learning algorithms have made tremendous progress and have been widely applied in vision [16], language [37] and speech [10] areas.

Original: Despite the prevalence, deep learningalgorithms rely heavily on a sufficient amount of labeled data and often fail when only a limitednumber of labeled samples are available.
Revised: Despite the prevalence, deep learning algorithms rely heavily on a sufﬁcient amount of labeled data and often fail when only a limited number of labeled samples are available.

Original: Considering the high cost of extensive data collection andannotation, it has attracted increasing attention to develop algorithms learning from a limited numberof training samples, which has been widely known as few-shot learning [33, 4].
Revised: Considering the high cost of extensive data collection and annotation, it has attracted increasing attention to developing algorithms learning from a limited number of training samples, which has been widely known as few-shot learning [39, 5].

Original: The recent focus of few-shot learning is built on the meta-learning framework [6, 33, 27].
Revised: The recent focus of few-shot learning is built on the meta-learning framework [9, 39, 33].

Original: The goalof meta-learning is to train a model that can be quickly adapted to a new task with a small number oflabeled samples by leveraging the knowledge from previous related tasks.
Revised: The goal of meta-learning is to train a model that can be quickly adapted to a new task with a small number of labeled samples by leveraging the knowledge from previous related tasks.

Original: Typically, a meta-learningmodel consists of the meta-training stage where a well-generalized feature extractor is learned frommeta-training tasks and the meta-testing stage during which a classifier is built on top of the featureextractor for novel tasks.
Revised: Typically, a meta-learning model consists of the meta-training stage where a well-generalized feature extractor is learned from meta-training tasks and the meta-testing stage during which a classiﬁer is built on top of the feature extractor for novel tasks.

Original: While the majority of meta-learning works revolved around developingbetter meta-training strategies to reduce the meta generalization error over all tasks, how to fullyleverage the meta-trained feature extractor to build a robust classifier and improve a task-specificgeneralization in the meta-testing stage remains less explored.The challenge to building such a classifier lies in the significant deviation of the empirical distribution(i.e. the distribution formed by the training data) from the population distribution (i.e. the true
Revised: While the majority of meta-learning works revolved around developing better meta-training strategies to reduce the meta generalization error over all tasks, how to fully leverage the meta-trained feature extractor to build a robust classiﬁer and improve a task-speciﬁc generalization in the meta-testing stage remains less explored.

Original: [Figure]
Revised: [Figure]

Original:  distribution).
Revised: The challenge to building such a classiﬁer lies in the signiﬁcant deviation of the empirical distribution (i.e. the distribution formed by the training data) from the population distribution (i.e. the true distribution).

Original: As illustrated in Figure 1, a model well fitting the training data (support set) maygeneralize poorly on the testing data (query set) from the true distribution due to the bias in theempirical distribution.
Revised: As illustrated in Figure 1, a model well ﬁtting the training data (support set) may generalize poorly on the testing data (query set) from the true distribution due to the bias in the empirical distribution.

Original: One viable solution is to approximate the true distribution more accurately –by viewing the empirical distribution as the assembling of delta functions located on each trainingsample, Chapelle et al.
Revised: One viable solution is to approximate the true distribution more accurately – by viewing the empirical distribution as the assembling of delta functions located on each training sample, Chapelle et al.

Original: [3] proposed the Vicinal Risk Minimization loss function where the deltafunctions are replaced by some density estimate in the vicinity of each training data.
Revised: [4] proposed the Vicinal Risk Minimization loss function where the delta functions are replaced by some density estimate in the vicinity of each training data.

Original: In the eraof deep learning, the VRM learning principle is implemented as Mixup [41, 32], which is simpleyet effective given a sufficiently large number of training data.
Revised: In the era of deep learning, the VRM learning principle is implemented as Mixup [47, 38], which is simple yet effective given a sufﬁciently large number of training data.

Original: However, as we have verified in
Revised: However, as we have veriﬁed in

Original: Table 3, the performance improvement by applying Mixup on the support set is limited in few-shotlearning where the vicinal distributions assembled by Mixup is still significantly biased.
Revised: Table 5, the performance improvement by applying Mixup on the support set is limited in few-shot learning where the vicinal distributions assembled by Mixup is still signiﬁcantly biased.

Original: To improvethe generalization of a few-shot classifier during meta-testing, we are motivated to revisit the VRMprinciple to 1) generate adaptive vicinal distribution for each sample in support set and 2) minimizethe expected vicinal loss over the vicinal distributions.
Revised: To improve the generalization of a few-shot classiﬁer during meta-testing, we are motivated to revisit the VRM principle to 1) generate adaptive vicinal distribution for each sample in support set and 2) minimize the expected vicinal loss over the vicinal distributions.

Original: However, both are challenging.
Revised: However, both are challenging.

Original: Without loss of generality, we assume that the vicinity of each training point admits a Gaussiandistribution.
Revised: Without loss of generality, we assume that the vicinity of each training point admits a Gaussian distribution.

Original: In [3], the means of the vicinal distributions are located on the original training points.
Revised: In [4], the means of the vicinal distributions are located on the original training points.

Original: As illustrated in Figure 1(b), however, setting the mean in this way is not sufficient to approximatethe true distribution, as an original training point likely deviates from the mean of the ground-truthdistribution of its corresponding class.
Revised: As illustrated in Figure 1(b), however, setting the mean in this way is not sufﬁcient to approximate the true distribution, as an original training point likely deviates from the mean of the ground-truth distribution of its corresponding class.

Original: In fact, it is difficult to estimate the mean and variance basedonly on the labeled training samples.
Revised: In fact, it is difﬁcult to estimate the mean and variance based only on the labeled training samples.

Original: In this paper, we assume that unlabeled samples are alsoavailable, and leverage these unlabeled samples to estimate the mean and variance of the vicinaldistribution for each labeled sample.
Revised: In this paper, we assume that unlabeled samples are also available, and leverage these unlabeled samples to estimate the mean and variance of the vicinal distribution for each labeled sample.

Original: Concretely, inspired by [42], we first construct a transitionprobability matrix based on pairwise distances between features of all samples, and secondly performthe lazy random walk among unlabeled samples starting from each labeled sample.
Revised: Concretely, inspired by [48], we ﬁrst construct a transition probability matrix based on pairwise distances between features of all samples, and secondly perform the lazy random walk among unlabeled samples starting from each labeled sample.

Original: After walkinga certain number of steps, we obtain the visit probabilities of all the samples and infer the vicinaldistribution whose mean and variance are the weighted mean and variance of all samples visited.
Revised: After walking a certain number of steps, we obtain the visit probabilities of all the samples and infer the vicinal distribution whose mean and variance are the weighted mean and variance of all samples visited.

Original: All that remains is the loss function over the vicinal distributions of all training samples.
Revised: All that remains is the loss function over the vicinal distributions of all training samples.

Original: An intuitiveway to do this is to exploit the Monte Carlo method, which repeats random sampling to explicitlygenerate data points from each vicinal distribution and minimizes the loss on all the sampled datapoints.
Revised: An intuitive way to do this is to exploit the Monte Carlo method, which repeats random sampling to explicitly generate data points from each vicinal distribution and minimizes the loss on all the sampled data points.

Original: Unfortunately, it requires to draw a large amount of samples to minimize the Monte Carloapproximation error, which significantly increases the computation cost.
Revised: Unfortunately, it requires drawing a large amount of samples to minimize the Monte Carlo approximation error, which signiﬁcantly increases the computation cost.

Original: Instead, we follow [3] toderive a closed form of the expected vicinal loss over the vicinal distributions.
Revised: Instead, we follow [4] to derive a closed form of the expected vicinal loss over the vicinal distributions.

Original: In this paper, we focuson building an optimization-based classifier in optimization-based meta-learning algorithms [6, 16],which has been proved with much more adaptability and generalization capacity in novel meta-testingtasks than a nearest-neighbour classifier in metric-based meta-learning algorithms [33, 27, 28, 21].
Revised: In this paper, we focus on building an optimization-based classiﬁer in optimization-based meta-learning algorithms [9, 19], which has been proved with much more adaptability and generalization capacity in novel meta-testing tasks than a nearest-neighbour classiﬁer in metric-based meta-learning algorithms [39, 33, 34, 24].

Original: Particularly, we consider both the cross-entropy loss and the SVM loss functions which are used asclassifier in the state-of-the-art meta-learning algorithms [37, 16].
Revised: Particularly, we consider both the cross-entropy loss and the SVM loss functions which are used as classiﬁer in the state-of-the-art meta-learning algorithms [43, 19].

Original: For the vicinal version of crossentropy loss, it cannot be computed in closed form.
Revised: For the vicinal version of crossentropy loss, it cannot be computed in closed form.

Original: We instead derive its quadratic approximation.
Revised: We instead derive its quadratic approximation.

Original: For SVM, we derive its closed-form vicinal loss.
Revised: For SVM, we derive its closed-form vicinal loss.

Original: In summary, our contributions are threefold:
Revised: In summary, our contributions are threefold:

Original: • We revisit vicinal risk minimization (VRM) in deep few-shot learning and derive the resultingexpected vicinal loss functions for both the cross-entropy loss and SVM.•
Revised: • We revisit vicinal risk minimization (VRM) in deep few-shot learning and derive the resulting expected vicinal loss functions for both the cross-entropy loss and SVM.

Original:  We propose to harness unlabeled data and a lazy random walk algorithm to generate adaptivestatistical parameters for the vicinal distribution of each training sample.
Revised: • We propose to harness unlabeled data and a lazy random walk algorithm to generate adaptive statistical parameters for the vicinal distribution of each training sample.

Original: • We have validated the superiority of the proposed algorithm with extensive experiments on threebenchmarks, i e., miniImageNet, CUB, CIFAR-FS.
Revised: • We have validated the superiority of the proposed algorithm with extensive experiments on three benchmarks, i.e., miniImageNet, CUB, CIFAR-FS.

Original:  Related Works2.1 Few-Shot Learning and Meta-Learning
Revised:  Related Works 2.1 Few-Shot Learning and Meta-Learning

Original: Few-shot learning aims to recognize novel classes with insufficient labeled instances.
Revised: Few-shot learning aims to recognize novel classes with insufﬁcient labeled instances.

Original: The recentfocus of few-shot learning algorithms has been on the meta-learning framework, which trainsthe model with multiple episodes to enable quick learning and applies the model to improve thelearning in novel tasks.
Revised: The recent focus of few-shot learning algorithms has been on the meta-learning framework, which trains the model with multiple episodes to enable quick learning and applies the model to improve learning in novel tasks.

Original: The meta-learning methods fall into two major categories: metric-basedmethods and optimization-based methods.
Revised: The meta-learning methods fall into two major categories: metric-based methods and optimizationbased methods.

Original: Metric-based methods [13, 33, 27, 28, 20, 39] learn ametric to compare two samples’ feature embeddings encoded by the feature extractor and classify asample in the query set based on its distances to the samples in the support set.
Revised: Metric-based methods [15, 39, 33, 34, 23, 45] learn a metric to compare two samples’ feature embeddings encoded by the feature extractor and classify a sample in the query set based on its distances to the samples in the support set.

Original: Optimization-basedapproaches [6, 23, 25, 29, 35] learn a good initialization such that it can be fast adapted to novel tasksbased on a few instances.
Revised: Optimization-based approaches [9, 27, 31, 35, 41] learn a good initialization such that it can be fast adapted to novel tasks based on a few instances.

Original: To tackle the data deficiency issue in few-shot learning, researchers have explored the idea ofexploiting unlabeled data and developed transductive learning or semi-supervised learning techniquesfor few-shot learning, either by leveraging nearest neighbor graph for propagating labels [36, 12, 18]or by leveraging predicted soft or hard labels on unlabeled samples to update the class prototypesor metric [24, 10, 15, 22, 39].
Revised: To tackle the data deﬁciency issue in few-shot learning, researchers have explored the idea of exploiting unlabeled data and developed transductive learning or semi-supervised learning techniques for few-shot learning, either by leveraging nearest neighbor graph for propagating labels [42, 14, 21] or by leveraging predicted soft or hard labels on unlabeled samples to update the class prototypes or metric [29, 12, 17, 26, 45].

Original: Our method is not built on a metric-based method.
Revised: Our method is not built on a metric-based method.

Original: We make useof the unlabeled data to derive the vicinal distribution for each support sample and optimize theconsequential vicinal loss to learn a classifier.
Revised: We make use of the unlabeled data to derive the vicinal distribution for each support sample and optimize the consequential vicinal loss to learn a classiﬁer.

Original: Our approach shares a similar spirit of [37].
Revised: Our approach shares a similar spirit of [43].

Original: Yang et al.[37] proposed to calibrate the distribution of the support samples by transferring statistics from thebase classes and trained the classifier on a set of samples sampled from the calibrated distributions.
Revised: Yang et al. [43] proposed to calibrate the distribution of the support samples by transferring statistics from the base classes and trained the classiﬁer on a set of samples sampled from the calibrated distributions.

Original: Unlike [37], our method obtains more accurate vicinal distributions by making use of the unlabeleddata from the same task.
Revised: Unlike [43], our method obtains more accurate vicinal distributions by making use of the unlabeled data from the same task.

Original: Besides, instead of sampling extensive data from the distributions, ourmodel directly optimizes the expected vicinal loss over the distributions, resulting in more efficienttraining and higher performance.
Revised: Besides, instead of sampling extensive data from the distributions, our model directly optimizes the expected vicinal loss over the distributions, resulting in more efﬁcient training and higher performance.

Original: 2.2 Vicinal Risk Minimization
Revised: 2.2 Vicinal Risk Minimization

Original: The Vicinal Risk Minimization (VRM) principle is introduced in [3, 30].
Revised: The Vicinal Risk Minimization (VRM) principle is introduced in [4, 36].

Original: It assumes each trainingsample admits a vicinal distribution and optimizes the expected loss over the vicinal distributions.
Revised: It assumes each training sample admits a vicinal distribution and optimizes the expected loss over the vicinal distributions.

Original: In[3], only binary VRM classifiers are considered.
Revised: In [4], only binary VRM classiﬁers are considered.

Original: Our method considers multi-class classifiers and thecross-entropy loss, which is more challenging to derive the expected vicinal loss.
Revised: Our method considers multi-class classiﬁers and the cross-entropy loss, which is more challenging to derive the expected vicinal loss.

Original: Besides, Chapelleet al.
Revised: Besides, Chapelle et al.

Original: [3] consider Gaussian vicinities but only estimate the variances, keeping the mean the same asthe original training sample.
Revised: [4] consider Gaussian vicinities but only estimate the variances, keeping the mean the same as the original training sample.

Original: Our approach further estimates the mean with the unlabeled data.
Revised: Our approach further estimates the mean with the unlabeled data.

Original: In the era of deep learning, the most popular technique under the VRM principle is Mixup [41, 32].
Revised: In the era of deep learning, the most popular technique under the VRM principle is Mixup [47, 38].

Original: Mixup constructs the vicinal distributions as the convex combination of the training data, whichcan be understood as a data augmentation technique that encourages the model to behave linearlyin-between training examples.
Revised: Mixup constructs the vicinal distributions as the convex combination of the training data, which can be understood as a data augmentation technique that encourages the model to behave linearly in-between training examples.

Original: It has been demonstrated that Mixup is simple yet efficient andimproves the generalization of deep models given sufficiently many training samples.
Revised: It has been demonstrated that Mixup is simple yet efﬁcient and improves the generalization of deep models given sufﬁciently many training samples.

Original: Mixup is alsointroduced on the meta-learning algorithm in [38, 7], but is applied on the query set for meta-modeloptimization.
Revised: Mixup has been introduced on the meta-learning algorithm in [44, 6], but is applied to the query set, focusing on improving the meta generalization.

Original: The experimental results  (Table 6)  in [38] show that applying Mixup on the support setcould even hurt the performance.
Revised: As the experiment results in [44] (Table 6) and in Table 5 shows, applying Mixup on the support set could even hurt the performance.

Original: Our method instead improves the performance by producing anaccurate estimate of the vicinal distribution for the small support set using unlabeled data.
Revised: Our method instead improves the performance by producing an accurate estimate of the vicinal distribution for the small support set using unlabeled data.

Original:  Adaptive Vicinal Few-Shot Learning
Revised:  Adaptive Vicinal Few-Shot Learning

Original: In this section, we will introduce the proposed algorithm, AD aptive V icinal Few-Shot Learning(ADV).
Revised: In this section, we will introduce the proposed algorithm, AD aptive V icinal Few-Shot Learning (ADV).

Original: We will first introduce the VRM, then derive the expected vicinal loss for a linear classifiertrain with cross-entropy loss and for SVM, respectively, and finally propose the algorithm based onlazy random walks to obtain the adaptive vicinal distribution for each labeled support sample.
Revised: We will ﬁrst introduce the VRM, then derive the expected vicinal loss for a linear classiﬁer train with cross-entropy loss and for SVM, respectively, and ﬁnally propose the algorithm based on lazy random walks to obtain the adaptive vicinal distribution for each labeled support sample.

Original: Wesummarize the proposed method in Algorithm 1.
Revised: We summarize the proposed method in Algorithm 1.

Original: Note that our method focuses on improving thetask-specific generalization in meta-testing set and is compatible with feature extractor trained by anymeta-learning algorithm.
Revised: Note that our method focuses on improving the task-speciﬁc generalization in meta-testing set and is compatible with feature extractor trained by any meta-learning algorithm.

Original: 3.1 Vicinal Risk Minimization
Revised: 3.1 Vicinal Risk Minimization

Original: Given data x, y following the distribution P ( x, y ) , the loss function L , we would like to optimize amodel f θ ( x ) by minimizing the expected risk
Revised: Given data x, y following the distribution P ( x, y ) , the loss function L , we would like to optimize a model f θ ( x ) by minimizing the expected risk

Original: [Equation]
Revised: [Equation]

Original: Here, x is the input data, y is its label and θ is the model parameters.
Revised: Here, x is the input data, y is its label and θ is the model parameters.

Original: In practice, P ( x, y ) is unknownand a set of training data { x i , y i } Ni =1 is available, where N is the size of training data.
Revised: In practice, P ( x, y ) is unknown and a set of training data { x i , y i } Ni =1 is available, where N is the size of training data.

Original: To learn amodel, one can minimize the empirical risk, which is equivalent to minimizing the expected loss withrespect to an empirical density estimate d P δ ( x, y )
Revised: To learn a model, one can minimize the empirical risk, which is equivalent to minimizing the expected loss with respect to an empirical density estimate d P δ ( x, y )

Original: = 1 N (cid:80) Ni
Revised: = 1 N (cid:80) Ni

Original: δ x i ( x ) δ y
Revised: δ x i ( x ) δ y

Original: i
Revised: i

Original: ( y ) , where δ a ( · ) is a Diracdelta function centered at a .
Revised: ( y ) , where δ a ( · ) is a Dirac delta function centered at a .

Original: Given a limited number of training sample, the empirical density estimate d P δ ( x, y ) could bias fromthe true density d P ( x, y ) .
Revised: Given a limited number of training sample, the empirical density estimate d P δ ( x, y ) could bias from the true density d P ( x, y ) .

Original: For a more accurate density estimate, Chapelle et al.
Revised: For a more accurate density estimate, Chapelle et al.

Original: [3] proposed toreplace the delta functions δ x i ( x ) by some estimate of density in the vicinity of x i .
Revised: [4] proposed to replace the delta functions δ x i ( x ) by some estimate of density in the vicinity of x i .

Original: Given a vicinaldistribution of x i as P x i ( x ) , the improved density estimate is P 
Revised: Given a vicinal distribution of x i as P x i ( x ) , the improved density estimate is P ν ( x, y )

Original: ν ( x, y ) = 1 N (cid:80) Ni =
Revised: = 1 N (cid:80) Ni =

Original: 1 P x i ( x ) δ y i ( y ) .
Revised: 1 P x i ( x ) δ y i ( y ) .

Original: And the corresponding vicinal risk is defined as
Revised: And the corresponding vicinal risk is deﬁned as

Original: [Equation]
Revised: [Equation]

Original: 3.2 Vicinal Loss
Revised: 3.2 Vicinal Loss

Original: In this paper, we assume the feature extractor is trained and fixed for novel tasks.
Revised: In this paper, we assume the feature extractor is trained and ﬁxed for novel tasks.

Original: Each task is
Revised: Each task is

Original: C -way K -shot, consisting of N = CK data support samples { ( x i , y i ) }
Revised: C -way K -shot, consisting of N = CK data support samples { ( x i , y i ) }

Original: Ni =1 .
Revised: Ni =1 .

Original: To perform VRM, weassume the feature z i of each sample x i in the support set admits a Gaussian vicinal distribution
Revised: To perform VRM, we assume the feature z i of each sample x i in the support set admits a Gaussian vicinal distribution

Original: N ( µ i , Σ i ) such that a sample ˜ z is drawn from the vicinal distribution with probability
Revised: N ( µ i , Σ i ) such that a sample ˜ z is drawn from the vicinal distribution with probability

Original: P z
Revised: P z

Original: i
Revised: i

Original: (˜ z )
Revised: (˜ z )

Original: =det(2 π Σ i ) − 12 exp( − 12 (˜ z − µ i ) 
Revised: = det(2 π Σ i ) − 12 exp( − 12 (˜ z − µ i ) (cid:62) Σ − 1 i

Original: ⊤ Σ − 1 i (˜ z − µ i )) , where µ i and Σ
Revised: (˜ z − µ i )) , where µ i and Σ

Original: i is the mean and variance of thevicinal distribution of z i .
Revised: i is the mean and variance of the vicinal distribution of z i .

Original: 3.2.1 Vicinal Cross-Entropy Loss
Revised: 3.2.1 Vicinal Cross-Entropy Loss

Original: We first focus on linear classifier with Cross-Entropy loss.
Revised: We ﬁrst focus on linear classiﬁer with Cross-Entropy loss.

Original: A linear classifier f θ ( z ) =
Revised: A linear classiﬁer f θ ( z ) =

Original: ( W ⊤ z + b )maps a feature z ∈ R d to its logits, where W ∈ R d × C and b ∈ R C is the parameter of the classifier.
Revised: ( W (cid:62) z + b ) maps a feature z ∈ R d to its logits, where W ∈ R d × C and b ∈ R C is the parameter of the classiﬁer.

Original: The cross-entropy loss is defined as ℓ ( h, y ) 
Revised: The cross-entropy loss is deﬁned as (cid:96) ( h, y ) =

Original: = − e ⊤ y log( softmax ( f θ ( z ))) , where y ∈ 1 , 2 , ..., C is thelabel; e y is a one-hot vector with the y -th element being 1 .
Revised: − e (cid:62) y log( softmax ( f θ ( z ))) , where y ∈ 1 , 2 , ..., C is the label; e y is a one-hot vector with the y -th element being 1 .

Original: The expected loss of cross entropy over
Revised: The expected loss of cross entropy over

Original: Gaussian vicinal distribution of x is defined as
Revised: Gaussian vicinal distribution of x is deﬁned as

Original: [Equation]
Revised: [Equation]

Original: Due to the softmax function, the expected loss (1) cannot be computed in closed form.
Revised: Due to the softmax function, the expected loss (1) cannot be computed in closed form.

Original: Instead, weutilize Taylor expansion to derive an approximation of Eqn.(1).
Revised: Instead, we utilize Taylor expansion to derive an approximation of Eqn.(1).

Original: By taking the second-order Taylorexpansion of ℓ ( f θ (˜ z ) , y ) around the mean of the vicinal distribution
Revised: By taking the second-order Taylor expansion of (cid:96) ( f θ (˜ z ) , y ) around the mean of the vicinal distribution

Original: z i , i.e. µ , we obtain
Revised: z i , i.e. µ , we obtain

Original: [Equation]
Revised: [Equation]

Original: where G is the gradient and H is the Hessian w.r.t.
Revised: where G is the gradient and H is the Hessian w.r.t.

Original: input µ .
Revised: input µ .

Original: Note that H
Revised: Note that H

Original: = W ( diag ( h ) − 
Revised: = W ( diag ( h ) − hh (cid:62) )

Original: hh ⊤ ) W ⊤where h = softmax ( f θ ( µ )) and diag ( h ) is a diagonal matrix whose diagonal elements are equal to h[26].
Revised: W (cid:62) where h = softmax ( f θ ( µ )) and diag ( h ) is a diagonal matrix whose diagonal elements are equal to h [32].

Original: By taking the expectation over ˜ z , we have
Revised: By taking the expectation over ˜ z , we have

Original: [Equation]
Revised: [Equation]

Revised: [Table]

Original:  where Σ is the variance of ˜ z , h k is the k -th elements of h , W k is the k -th column of W .
Revised: [Table] [Table] [Table] where Σ is the variance of ˜ z , h k is the k -th elements of h , W k is the k -th column of W .

Original: We define
Revised: We deﬁne

Original: R θ (Σ) as the RHS of Eqn.
Revised: R θ (Σ) as the RHS of Eqn.

Original: Plugging (2), (3) and (4) into (1), we obtain the approximated expectedvicinal cross-entropy loss for the linear classifier as
Revised: Plugging (2), (3) and (4) into (1), we obtain the approximated expected vicinal cross-entropy loss for the linear classiﬁer as

Original: [Equation]
Revised: [Equation]

Original: 3.2.2 Vicinal SVM
Revised: 3.2.2 Vicinal SVM

Original: SVM is used as classifier for meta learning algorithms in [16, 37].
Revised: SVM is used as classiﬁer for meta learning algorithms in [19, 43].

Original: We follow [16] to use the Crammerand Singer formulation of multi-class SVM [5].
Revised: We follow [19] to use the Crammer and Singer formulation of multi-class SVM [7].

Original: In [16, 5], the dual formulation of the objective for alinear multi-class SVM is obtained as:
Revised: In [19, 7], the dual formulation of the objective for a linear multi-class SVM is obtained as:

Original: [Equation]
Revised: [Equation]

Original: where α ki is the dual variable and λ is a regularization parameter.
Revised: where α ki is the dual variable and λ is a regularization parameter.

Original: The linear classifier for thek -th class is f kθ ( z ) =
Revised: The linear classiﬁer for the k -th class is f kθ ( z ) =

Original: w ⊤ k z , where w k = (cid:80) i{+Ni =
+}
Revised: w (cid:62) k z , where w k = (cid:80) {+Ni =
+}

Original: α ki z i .
Revised: α ki z i .

Original: Note that here z is an unseen data and weassume only labeled data admit vicinal distributions while unseen data does not admit vicinaldistributions.
Revised: Note that here z is an unseen data. We assume only labeled data admit vicinal distributions while unseen data does not admit vicinal distributions.

Original: If each z i admits a vicinal distribution N ( µ i , Σ i ) , the vicinal expectation for theterm z ⊤ i
Revised: If each z i admits a vicinal distribution N ( µ i , Σ i ) , the vicinal expectation for the term z (cid:62) i

Original: z j in (6) becomes
Revised: z j in (6) becomes

Original: E ˜ z i ∼N ( µ i , Σ i ) E ˜
Revised: [Equation]

Original: z j ∼N ( µ j , Σ j ) [˜ z ⊤ i ˜ z j ] = µ ⊤ i µ j , and the classifier becomes173 E ˜ z
Revised: and the classiﬁer becomes

Original: i ∼N ( µ i , Σ i ) [ f kθ (˜ z i )] = ( (cid:80) i α ki µ i ) ⊤ z.
Revised: [Equation]

Original: If µ i =
Revised: If µ i =

Original: z i , the objective of the vicinal SVM reduces to theoriginal SVM objective in Eqn.
Revised: z i , the objective of the vicinal SVM reduces to the original SVM objective in Eqn.

Original: Even when µ i ̸ =
Revised: Even when µ i (cid:54) =

Original: z i , the objective of the vicinal SVM stilldoes not take the variance of the vicinal distribution into consideration and may result in suboptimalperformance.
Revised: z i , the objective of the vicinal SVM still does not take the variance of the vicinal distribution into consideration and may result in suboptimal performance.

Original: To exploit the variance information to improve the vicinal SVM classifier, we considera nonlinear classifier, i.e. the kernel-based SVM.
Revised: To exploit the variance information to improve the vicinal SVM classiﬁer, we consider a nonlinear classiﬁer, i.e. the kernel-based SVM.

Original: Multi-class kernel-based SVM uses a kernel K ( · , · ) .
Revised: Multi-class kernel-based SVM uses a kernel K ( · , · ) .

Original: It admits a dual formulation similar to (6), butreplaces the inner product of z i and z j in (6) by K ( z i , z j ) .
Revised: It admits a dual formulation similar to (6), but replaces the inner product of z i and z j in (6) by K ( z i , z j ) .

Original: And the classifier becomes f kθ ( z ) =(cid:80) i α ki K
Revised: And the classiﬁer becomes f kθ ( z ) = (cid:80) i α ki K

Original: ( z i , z ) .
Revised: ( z i , z ) .

Original: In this paper, we consider the widely-used RBF kernel K ( z, z′ ) =
Revised: In this paper, we consider the widely-used RBF kernel K ( z, z

Original: exp( − || z − z ′
Revised: [Equation]

Original: || 2 2 σ) ,where σ is a hyper-parameter.
Revised: ) , where σ is a hyper-parameter.

Original: In vicinal multi-class kernel-based SVM, we define M ( z i , z j ) =
Revised: In vicinal multi-class kernel-based SVM, we deﬁne M ( z i , z j ) =

Original: E ˜ z
Revised: E ˜ z

Original: i ∼N ( µ i , Σ i ) E ˜
Revised: i ∼N ( µ i , Σ i ) E ˜

Original: z
Revised: z

Original: j ∼N ( µ j , Σ j ) K
Revised: j ∼N ( µ j , Σ j ) K

Original: ( z i , z j )and Q ( z i , z ) =
Revised: ( z i , z j ) and Q ( z i , z ) =

Original: E ˜ z
Revised: E ˜ z

Original: i ∼N ( µ i , Σ i ) K ( z i , z ) and obtain their computation as 
Revised: i ∼N ( µ i , Σ i ) K ( z i , z ) and obtain their computation as [Equation] z i , [Equation]

Revised: =

Original: [Equation]
Revised: [Equation]

Original: The detailed derivation is available in Appendix A. Here, A ( σ 

Revised: The detailed derivation is available in Appendix A. Here, A ( σ

Original: ) ∈ R d × d is a diagonal matrix withdiagonal elements being σ 2 ; M i =
Revised: ) ∈ R d × d is a diagonal matrix with diagonal elements being σ 2 ; M i =

Original: ( A ( σ 2 ) + Σ i ) − 1 ; M ij =
Revised: ( A ( σ 2 ) + Σ i ) − 1 ; M ij =

Original: ( A ( σ 2 ) + Σ i + Σ j ) − 1 ; the operator∥ z ∥ 2 M = z ⊤ Mz .
Revised: ( A ( σ 2 ) + Σ i + Σ j ) − 1 ; the operator (cid:107) z (cid:107) 2 M = z (cid:62) Mz .

Original: Note that for computation simplification, in vicinal SVM, we assume the varianceΣ i is diagonal.
Revised: Note that for computation simpliﬁcation, in vicinal SVM, we assume the variance Σ i is diagonal and set its non-diagonal elements to zeros.

Original: Now we achieve the vicinal objective for multi-class kernel-based SVM as:
Revised: Now we achieve the vicinal objective for multi-class kernel-based SVM as:

Original: [Equation]
Revised: [Equation]

Original: with the classifier being f kθ ( z ) = (cid:80) i α ki Q
Revised: with the classiﬁer being f kθ ( z ) = (cid:80) i α ki Q

Original: ( z i , z ) .
Revised: ( z i , z ) .

Original: The objective in (9) is a quadratic program (QP)over dual variables { α ki } and can be solved by using a differentiable GPU-based QP solver [1].
Revised: The objective in (9) is a quadratic program (QP) over dual variables { α ki } and can be solved by using a differentiable GPU-based QP solver [1].

Original: 3.3 Lazy Random Walk for Adaptive Vicinal Distribution
Revised: 3.3 Lazy Random Walk for Adaptive Vicinal Distribution

Original: As illustrated in Figure 1, the performance of the model learned by VRM relies heavily on howclose the vicinal distribution is to the true distribution.
Revised: As illustrated in Figure 1, the performance of the model learned by VRM relies heavily on how close the vicinal distribution is to the true distribution.

Original: However, it is almost impossible to estimateaccurate proximate distributions from a small support set.
Revised: However, it is almost impossible to estimate accurate proximate distributions from a small support set.

Original: Therefore, we propose to exploit extra data,i.e., the unlabeled data from the same task, to reduce the estimation error of the statistical parametersfor the vicinal distribution and, thus, improve the performance of the model.
Revised: Therefore, we propose to exploit extra data, i.e., the unlabeled data from the same task, to reduce the estimation error of the statistical parameters for the vicinal distribution and, thus, improve the performance of the model.

Original: We assume that a well-trained feature extractor on the meta-training tasks can preserve the similaritybetween the data in the feature space.
Revised: We assume that a well-trained feature extractor on the meta-training tasks can preserve the similarity between the data in the feature space.

Original: Data from the same class in the novel task would have similarfeatures with a high probability.
Revised: Data from the same class in the novel task would have similar features with a high probability.

Original: Then, we apply the lazy random walk algorithm [42] to estimate theprobability that the unlabeled data will be visited starting from a support sample, and compute thestatistical parameters as the weighted mean and variance of the samples visited by the lazy randomwalk.
Revised: Then, we apply the lazy random walk algorithm [48] to estimate the probability that the unlabeled data will be visited starting from a support sample, and compute the statistical parameters as the weighted mean and variance of the samples visited by the lazy random walk.

Original: To perform lazy random walk, we first compute the distance D SU between the support sample
Revised: To perform lazy random walk, we ﬁrst compute the distance D SU between the support sample

Original: z i andthe unlabeled data { u j } N u j =1 and compute the distance D UU between the unlabeled data as: 
Revised: z i and the unlabeled data { u j } N u j =1 and compute the distance D UU between the unlabeled data as: [Equation]

Revised: SUij = (cid:107)

Original: [Equation]
Revised: [Equation]

Original: and
Revised: i

Original: [Equation]
Revised: [Equation]

Revised: (cid:107) 2 , and (10)

Original: [Equation]
Revised: [Equation]

Original: Secondly, we transform these distances into transition probabilities by softmaxing the distance matrixover columns with a temperature τ :
Revised: Secondly, we transform these distances into transition probabilities by softmaxing the distance matrix over columns with a temperature τ :

Original: [Equation]
Revised: [Equation]

Original: Note that we set D UUjj = ∞ such that P UUjj = 0 .
Revised: Note that we set D UUjj = ∞ such that P UUjj = 0 .

Original: Thirdly, we construct the transition probabilities matrix for z i in lazy random walk by combining
Revised: Thirdly, we construct the transition probabilities matrix for z i in lazy random walk by combining

Original: P SUi : and P UU and setting the lazy stay probability β ∈ (0 , 1) as:
Revised: P SUi : and P UU and setting the lazy stay probability β ∈ (0 , 1) as:

Original: [Equation]
Revised: [Equation]

Original: where P SUi : is the i -th row of P SU , 0 n c × 1 is a all zero vector, and I N u is an identity matrix.
Revised: where P SUi : is the i -th row of P SU , 0 n c × 1 is a all zero vector, and I N u is an identity matrix.

Original: We then compute the visit probability v i at the T -th step for z i:
Revised: We then compute the visit probability v i at the T -th step for z i

Original: [Equation]
Revised: [Equation]

Original: where e 1 ∈ R ( N u +1)
Revised: where e 1 ∈ R ( N u +1)

Original: × 1 is a one-hot vector with the first element being 1 .
Revised: × 1 is a one-hot vector with the ﬁrst element being 1 .

Revised: For computational efﬁciency, we keep only the top m nearest neighbors of a data visitable and set D UUij ( D SUij ) to zero

Revised: [Table]

Revised: if u j is not in the top m nearest neighbors of u i ( z i ).

Original: Finally, we obtain µ iand Σ i as the weighted mean and variance of z i and { u j } N u j =1 , with the weight being v i .
Revised: Finally [Table] we obtain µ i and Σ i as the weighted mean and variance of z i and { u j } N u j =1 , with the weight being v i .

Original: That [-is:µ i = 12 z i + 12 (cid:80) N u j =1 v ij u j , and Σ i = 
-]
Revised: That [-is:µ i = 12 z i + 12 (cid:80) N u j =1 v ij u j , and Σ i = 
-]is:

Original: ( z i − µ i )( z i − µ i)
Revised: [Equation] ij [Equation] i

Original: ⊤ + 12 (cid:80) N u j =1 v ij ( u j − µ i )( u j − µ i ) ⊤ .
Revised: [Equation]

Original:  Experiment
Revised:  Experiment

Original: In this section, we verify the effectiveness of the proposed approach in improving the generalizationof the few-shot learning.
Revised: In this section, we verify the effectiveness of the proposed approach in improving the generalization of the few-shot learning.

Original: We first introduce the implementation details.
Revised: We ﬁrst introduce the implementation details.

Original: Then we present the resultson the benchmarks, followed by a detailed study on the proposed approach.
Revised: Then we present the results on the benchmarks, followed by a detailed study of the proposed approach.

Original: We use three benchmarks for performance evaluation: mini ImageNet [33], CUB [34] and CIFARFS [2].
Revised: We use three benchmarks for performance evaluation: mini ImageNet [39], CUB [40] and CIFARFS [2].

Original: For all datasets, we construct 5-way 1-shot and 5-way 5-shot tasks by randomly sampling 5classes.
Revised: For all datasets, we construct 5-way 1-shot and 5-way 5-shot tasks by randomly samplingclasses.

Original: For performance evaluation, we construct 10 , 000 meta-testing tasks and report the averageclassification accuracy with a 95% confidence interval.
Revised: For performance evaluation, we construct 10 , 000 meta-testing tasks and report the average classiﬁcation accuracy with a 95% conﬁdence interval.

Original: [Table] Implementation details : we focus on meta-testing phase and directly use the WideResNet-2810(WRN) [40] feature extractors pre-trained following the previous work [19].
Revised: Implementation details : we focus on meta-testing phase and directly use the WideResNet-28- (WRN) [46] feature extractors pre-trained following the previous work [22].

Original: We use ADV-CE todenote the application of ADV in the linear classifier with vicinal cross-entropy loss and ADV-SVMto denote the application of ADV in SVM.
Revised: We use ADV-CE to denote the application of ADV in the linear classiﬁer with vicinal cross-entropy loss and ADV-SVM to denote the application of ADV in SVM.

Original: For ADV-CE, we  optimize the vicinal loss in (5) bygradient descent for 100 steps.
Revised: For ADV-CE, we initialize the weights of the classiﬁer by class prototypes and optimize the vicinal loss in (5) by gradient descent for 100 steps.

Original: The learning rate is determined by performing a grid search from0 .
Revised: The learning rate is determined by performing a grid search from 0 .

Original: 001 to 1 on the tasks constructed by the meta-validation set.
Revised: 001 to 1 on the tasks constructed by the meta-validation set.

Original: For ADV-SVM, we solve the QP in (9)by using a differentiable GPU-based QP solver [1].
Revised: For ADV-SVM, we solve the QP in (9) by using a differentiable GPU-based QP solver [1].

Original: The regularization parameter λ is set to 0 .
Revised: The regularization parameter λ is set to 0 .

Original: 1 andthe parameter σ for RBF kernel is obtained via grid search from 0 .
Revised: 1 and the parameter σ for RBF kernel is obtained via grid search from 0 .

Original: 1 to 10 .
Revised: 1 to 10 .

Original: In the lazy random walkalgorithm, the number of step T , the lazy stay probability β and the temperature τ  are obtained viagrid search in { 1 , 2 , 3 , 4 , 5 } , { 0 . 1 , 0 . 2 , 0 . 5 } , { 0 . 01 , 0 . 1 , 1 , 10 } ,  respectively.
Revised: In the lazy random walk algorithm, the number of step T , the lazy stay probability β and the temperature τ and the number of visitable neighbors m are obtained via grid search in { 1 , 2 , 3 , 4 , 5 } , { 0 . 1 , 0 . 2 , 0 . 5 } , { 0 . 01 , 0 . 1 , 1 , 10 } , { 5 , 7 , 9 } , respectively.

Original: Notably, ADV accessesthe whole query set as unlabeled data to perform lazy random walk.
Revised: Notably, ADV accesses the whole query set as unlabeled data to perform lazy random walk.

Original: Thus, ADV is a transductivemethod.
Revised: Thus, ADV is a transductive method.

Original: The number of query samples per class is 15 . 
Revised: The number of query samples per class is 15 . [Table] [Table]

Original: 4.1 Comparison with Baselines
Revised: 4.1 Comparison with Baselines

Original: We present the performance of ADV-CE, ADV-SVM with a WRN feature extractor and comparethem with existing works in Table 1.
Revised: We present the performance of ADV-CE, ADV-SVM with a WRN feature extractor and compare them with existing works in Table 1.

Original: The results of baselines are from [16], [22] and [11].
Revised: The results of baselines are from [19], [26] and [13].

Original: Weobserve in Table 1 that, in all datasets, ADV achieves the best performance in both 1-shot and5-shot tasks.
Revised: We observe in Table 1 that, in all datasets, ADV achieves the best performance in both 1-shot and 5-shot tasks.

Original: Notable, ADV outperforms the best competitor by at least 2% in 1-shot tasks.
Revised: Notable, ADV outperforms the best competitor by at least 2% in 1-shot tasks.

Original: Theextraordinary performance of ADV in 1-shot tasks indicates its effectiveness to exploit the informationof the unlabeled data from the query set to provide a more accurate distribution estimate to the truedistribution of each class, addressing the issue that the empirical distribution estimated from thesupport set may deviate significantly from the true distribution.
Revised: The extraordinary performance of ADV in 1-shot tasks indicates its effectiveness to exploit the information of the unlabeled data from the query set to provide a more accurate distribution estimate to the true distribution of each class, addressing the issue that the empirical distribution estimated from the support set may deviate signiﬁcantly from the true distribution.

Revised: We also present the experimental results regarding comparison with more SOTA algorithms, including Transductive Information Maximation (TIM) [3], LaplacianShot [49] and Obliquer Manifold (OM) [25].

Original: 4.2 Visualization of Vicinal Distributions
Revised: We compare with these 3 methods because they are all independent of the pre-training of the base model and do not require training any extra module in the meta-training phase. Note that the performance of few-shot learning is affected by both the base model obtained in the meta-training phase and the ﬁne-tuning strategy in the meta-testing phase. For a fair comparison in the meta-testing phase, the base model should be the same for different methods. We feed all 4 methods the same ﬁxed WRN feature extractor and the same 10,000 meta-testing tasks.

Revised: As TIM and LaplacianShot use the same pre-trained weights of a feature extractor, we use these pre-trained weights for all of themethods.

Revised: No data augmentation is applied to images in meta-testing tasks.

Revised: From the results in Table 2, we observe that ADV-CE outperforms all baselines in 1-shot learning and obtains the second best performance in 5-shot learning, which is slightly lower than TIM.

Revised: The major contributions of TIM lie in regularization on query sets, while ours can be deemed as regularization on support sets, which means its contributions are orthogonal to our contributions.

Revised: We are more interested in integrating TIM to further boost the proposed method.

Revised: When integrating TIM with ADV-CE and applying it to the WRN feature extractor pre-trained by [22] on miniImageNet,

Revised: ADV-CE+TIM achieves competitive performance to the SOTA methods as presented in Table 3, where the performances of the SOTA methods are reported in the original paper.

Revised: 4.2 Visualization of Vicinal Distributions

Original: The vicinal risk minimization principle in the proposed method is built on the vicinal distributionsobtained by a lazy random walk.
Revised: The vicinal risk minimization principle in the proposed method is built on the vicinal distributions obtained by a lazy random walk.

Original: Therefore, the performance of ADV depends on the exactness of thevicinal distribution estimation.
Revised: Therefore, the performance of ADV depends on the exactness of the vicinal distribution estimation.

Original: Here, we visualize the vicinal distributions to evaluate their exactness.
Revised: Here, we visualize the vicinal distributions to evaluate their exactness.

Original: We run the proposed lazy random walk algorithm on two 5-way 1-shot tasks randomly constructedon the meta-testing set of miniImageNet, generate 500 samples from each vicinal distribution, andperform t-SNE algorithm on all data.
Revised: We run the proposed lazy random walk algorithm on two 5-way 1-shot tasks randomly constructed on the meta-testing set of miniImageNet, generate 500 samples from each vicinal distribution, and perform t-SNE algorithm on all data.

Original: The visualization results are presented in Figure 2.
Revised: The visualization results are presented in Figure 2.

Original: [Table]
Revised: [Table]

Original: We can observe from Figure 2 that the vicinal distribution generated by ADV covers most of thequery samples of the same class.
Revised: We can observe from Figure 2 that the vicinal distribution generated by ADV covers most of the query samples of the same class.

Original: This verifies that the lazy random walk module in ADV can providea good estimate of the vicinal distribution, which overlaps with the ground truth distribution.
Revised: This veriﬁes that the lazy random walk module in ADV can provide a good estimate of the vicinal distribution, which overlaps with the ground truth distribution.

Original: Byoptimizing the expected loss over the vicinal distributions, the model trained by ADV has a smallerprobability of overfitting the support samples and provides better generalization to the query set.
Revised: By optimizing the expected loss over the vicinal distributions, the model trained by ADV has a smaller probability of overﬁtting the support samples and provides better generalization to the query set.

Original: 4.3 Effect of the Network Architecture of Feature Extractor
Revised: 4.3 Effect of the Network Architecture of Feature Extractor

Original: The proposed algorithm ADV is compatible with any pre-trained feature extractor.
Revised: The proposed algorithm ADV is compatible with any pre-trained feature extractor.

Original: Here, we validatethe effectiveness of ADV on three feature extractors with different backbones, which are trainedby different meta-learning methods.
Revised: Here, we validate the effectiveness of ADV on three feature extractors with different backbones, which are trained by different meta-learning methods.

Original: We consider WideResNet (WRN) [40], ResNet-12 [9] and ashallow network of 4 convolutional blocks (CONV) [2].
Revised: We consider WideResNet (WRN) [46], ResNet-12 [11] and a shallow network of 4 convolutional blocks (CONV) [2].

Original: We meta-train the WRN, ResNet, CONVfollowing [19], [16] and [2], respectively.
Revised: We meta-train the WRN, ResNet, CONV following [22], [19] and [2], respectively.

Original: We report the performance of a kernel SVM classifier and a linear classifier trained by crossentropy (denoted by CE) with and without ADV.
Revised: We report the performance of a kernel SVM classiﬁer and a linear classiﬁer trained by crossentropy (denoted by CE) with and without ADV.

Original: The results in Table 2 show that the performanceimprovement of ADV is consistent for three feature extractors.
Revised: The results in Table 4 show that the performance improvement of ADV is consistent for three feature extractors.

Original: This verifies that the effectiveness of
Revised: This veriﬁes that the effectiveness of

Original: ADV is independent of the feature extractor.
Revised: ADV is independent of the feature extractor.

Original: It is worth noting that the larger performance gains areobserved on the better feature extractors.
Revised: It is worth noting that the larger performance gains are observed on the better feature extractors.

Original: For example, the overall performance of WRN outperforms
Revised: For example, the overall performance of WRN outperforms

Original: CONV, and the performance boost of ADV-CE over CE with WRN in 1-shot tasks is 10.5%, whichis larger than the boost with CONV (4.8%).
Revised: CONV, and the performance boost of ADV-CE over CE with WRN in 1-shot tasks is 10.5%, which is larger than the boost with CONV (4.8%).

Original: This observation suggests that 1) even with a superiorfeature extractor that well preserves the similarity between data in feature space, the classifier trainedfrom a limited number of high-quality support data still overfits the data; 2) for a better featureextractor, ADV can exploit more information from the similarity between data to estimate the vicinaldistribution of support data and alleviate the overfitting.
Revised: This observation suggests that 1) even with a superior feature extractor that well preserves the similarity between data in feature space, the classiﬁer trained from a limited number of high-quality support data still overﬁts the data; 2) for a better feature extractor, ADV can exploit more information from the similarity between data to estimate the vicinal distribution of support data and alleviate the overﬁtting.

Original: 4.4 Ablation Study
Revised: 4.4 Ablation Study

Original: We perform an ablation study on ADV regarding the gain of the VRM principle.
Revised: We perform an ablation study on ADV regarding the gain of the VRM principle.

Original: To do this, we traina vanilla linear classifier with cross-entropy loss (denoted by CE), a linear classifier with Mixupvicinal distribution [41] (denoted by CE-Mixup), a linear classifier with Delta vicinal distributionwhose mean locates on the same mean estimate by our lazy random walk module and whose varianceis zero (denoted by ADV-CE-DD), and compare them with ADV-CE.
Revised: To do this, we train a vanilla linear classiﬁer with cross-entropy loss (denoted by CE), a linear classiﬁer with Mixup vicinal distribution [47] (denoted by CE-Mixup), a linear classiﬁer with Delta vicinal distribution (denoted by ADV-CE-Mean) whose mean locates on the same mean estimate by our lazy random walk module and whose variance is zero, i.e., P z i (˜ z ) = δ µ i (˜ z ) , and compare them with ADV-CE.

Original: We also train a multi-class SVM classifier, a multi-class kernel SVM classifier, and a kernel SVM classifier withDelta vicinal
Revised: We also train a multi-class SVM classiﬁer, a multi-class kernel SVM classiﬁer, and a kernel SVM classiﬁer with

Original: [Table]
Revised: [Table]

Original:  distribution (denoted by ADV-SVM-DD).
Revised: Delta vicinal distribution (denoted by ADV-SVM-Mean).

Original: The training of multi-class SVM classifiers all follows [5].
Revised: The training of multi-class SVM classiﬁers all follows [7].

Original: We report the performance on three benchmarks in Table 3.
Revised: We report the performance on three benchmarks in Table 5.

Original: The results in Table 3 show that the performance of CE-Mixup only competes with CE in miniImageNet and CIFAR-FS, and is even inferior to CE on the CUB dataset.
Revised: The results in Table 5 show that the performance of CE-Mixup only competes with CE in miniImageNet and CIFAR-FS, and is even inferior to CE on the CUB dataset.

Original: This suggests that the Mixupvicinal distribution fails to improve the generalization given a limited number of data.
Revised: This suggests that the Mixup vicinal distribution fails to improve the generalization given a limited number of data.

Original: In contrast,the vicinal distribution obtained by the lazy random walk algorithm is a more accurate estimate ofthe true distribution, as evidenced by the fact that the performance of ADV-CE-DD and ADV-CEis better than CE and CE-Mixup.
Revised: In contrast, the vicinal distribution obtained by the lazy random walk algorithm is a more accurate estimate of the true distribution, as evidenced by the fact that the performance of ADV-CE-Mean and ADV-CE is better than CE and CE-Mixup.

Original: To be concrete, the performance boost of ADV-CE-DD over CEdemonstrates that the mean of the vicinal distribution, which lies not on the original sample but on aweighted average of the data passed through by the lazy random walk, is closer to the ground-truthdistribution.
Revised: To be concrete, the performance boost of ADV-CE-Mean over CE demonstrates that the mean of the vicinal distribution, which lies not on the original sample but on a weighted average of the data passed through by the lazy random walk, is closer to the ground-truth distribution.

Original: The performance improvement of ADV-CE over ADV-CE-DD demonstrates that thevariance of the vicinal distribution helps to alleviate the overfitting problem.
Revised: The performance improvement of ADV-CE over ADV-CE-Mean demonstrates that the variance of the vicinal distribution helps to alleviate the overﬁtting problem.

Original: As shown in Table 3, the performance of Kernel SVM is worse than SVM in most cases.
Revised: As shown in Table 5, the performance of Kernel SVM is worse than SVM in most cases.

Original: The reasonmay be that the features extracted by WRN are linearly separable and the non-linear kernel techniquedoes not help to further improve the separability.
Revised: The reason may be that the features extracted by WRN are linearly separable and the non-linear kernel technique does not help to further improve the separability.

Original: Although the performance of Kernel SVM isinferior to SVM, we observe a performance boost of ADV-SVM, which is built on Kernel SVM.
Revised: Although the performance of Kernel SVM is inferior to SVM, we observe a performance boost of ADV-SVM, which is built on Kernel SVM.

Original: This indicates that learning from the vicinal distributions that are close to the true distribution wouldsufficiently improve the performance.
Revised: This indicates that learning from the vicinal distributions that are close to the true distribution would sufﬁciently improve the performance.

Original: We also observe the performance gain of ADV-SVM over
Revised: We also observe the performance gain of ADV-SVM over

Original: ADV-SVM-DD, again evidencing the importance of the variance of vicinal distributions in alleviatingthe overfitting.
Revised: ADV-SVM-DD, again evidencing the importance of the variance of vicinal distributions in alleviating the overﬁtting.

Original:  Conclusion and Limitations
Revised:  Conclusion and Limitations

Original: In this paper, we discuss the overfitting issue in few-shot learning, which is caused by the deviationof the empirical distribution from the ground-truth distribution.
Revised: In this paper, we discuss the overﬁtting issue in few-shot learning, which is caused by the deviation of the empirical distribution from the ground-truth distribution.

Original: To alleviate the overfitting, werevisit the Vicinal Risk Minimization principle in deep few-shot learning and propose to harnessunlabeled data and a lazy random walk algorithm to generate adaptive statistical parameters forthe vicinal distribution of each training sample.
Revised: To alleviate the overﬁtting, we revisit the Vicinal Risk Minimization principle in deep few-shot learning and propose to harness unlabeled data and a lazy random walk algorithm to generate adaptive statistical parameters for the vicinal distribution of each training sample.

Original: Following the VRM principle, we learn the modelby optimizing the resulting expected vicinal loss.
Revised: Following the VRM principle, we learn the model by optimizing the resulting expected vicinal loss.

Original: We derive the vicinal loss for linear classifierstrained with cross-entropy loss and for SVM classifier and develop the resulting Adaptive Vicinalclassifiers as ADV-CE and ADV-SVM, respectively.
Revised: We derive the vicinal loss for linear classiﬁers trained with cross-entropy loss and for SVM classiﬁer and develop the resulting Adaptive Vicinal classiﬁers as ADV-CE and ADV-SVM, respectively.

Original: We evaluate the performance of ADV-CE and
Revised: We evaluate the performance of ADV-CE and

Original: ADV-SVM on three benchmarks and validate the effectiveness of ADV in mitigating overfitting andimproving the generalization of classifiers learned from a limited number of samples.
Revised: ADV-SVM on three benchmarks and validate the effectiveness of ADV in mitigating overﬁtting and improving the generalization of classiﬁers learned from a limited number of samples.

Original: Currently,our proposed ADC is compatible with CE and SVM only.
Revised: Currently, our proposed ADC is compatible with CE and SVM only.

Original: We will extend it to other loss functions.
Revised: We will extend it to other loss functions.

Original: For the vicinal Cross-Entropy loss, we optimize an approximation obtained by second-order Taylorexpansion.
Revised: For the vicinal Cross-Entropy loss, we optimize an approximation obtained by second-order Taylor expansion.

Original: However, the approximation error can be large.
Revised: However, the approximation error can be large.

Original: Thus, the performance of ADV-CEmay be suboptimal and we can further improve it by deriving a better approximation for the vicinal
Revised: Thus, the performance of ADV-CE may be suboptimal and we can further improve it by deriving a better approximation for the vicinal

Original: Cross-Entropy loss.
Revised: Cross-Entropy loss.

Original: Checklist
Revised: Checklist

Original: The checklist follows the references.
Revised: The checklist follows the references.

Original: Please read the checklist guidelines carefully for information onhow to answer these questions.
Revised: Please read the checklist guidelines carefully for information on how to answer these questions.

Original: For each question, change the default [TODO] to [Yes], [No] or[N/A] .
Revised: For each question, change the default [TODO] to [Yes] [No] or [N/A] .

Original: You are strongly encouraged to include a justification to your answer, either by referencingthe appropriate section of your paper or providing a brief inline description.
Revised: You are strongly encouraged to include a justiﬁcation to your answer either by referencing the appropriate section of your paper or providing a brief inline description.

Original: For example:
Revised: For example:

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [Yes] See Section ?? .
Revised: [Yes] See Section ?? .

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [No] The code and the data areproprietary.
Revised: [No] The code and the data are proprietary.

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [N/A]Please do not modify the questions and only use the provided macros for your answers.
Revised: [N/A] Please do not modify the questions and only use the provided macros for your answers.

Original: Note that theChecklist section does not count towards the page limit.
Revised: Note that the Checklist section does not count towards the page limit.

Original: In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.1.
Revised: In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] .
Revised: [Yes] .

Original: See Section 5(c) Did you discuss any potential negative social impacts of your work?
Revised: See Section(c) Did you discuss any potential negative social impacts of your work?

Original: [N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: [N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] (b) Did you include complete proofs of all theoretical results?
Revised: [Yes] (b) Did you include complete proofs of all theoretical results?

Original: [Yes]3.
Revised: [Yes] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
Revised: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes](c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?[Yes]
Revised: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original:  (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
Revised: [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [No] 
Revised: [No] 4.

Original: If you are using existing assets (e g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] (b) Did you mention the license of the assets?
Revised: [Yes] (b) Did you mention the license of the assets?

Original: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: [N/A] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?

Original: [N/A] 
Revised: [N/A] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount
Revised: Here, we provide the derivation of vicinal kernel functions omitted in Section 3.2.2. The one-side vicinal kernel Q ( z, z i ) is derived as [Equation]

Original: spent on participant compensation?
Revised: And the two-side vicinal kernel M ( z i [Equation] z j ) is derived as [Equation]

Revised: The derivation follows the convolution integrals of normal distribution functions and is simpliﬁed by the assumption that A ( σ 2 ) , Σ i , Σ j are diagonal.

Original: [N/A]
Revised: We evaluate the proposed method on mini ImageNet [39], CUB [40] and CIFAR-FS [2]. mini ImageNet is a standard benchmark for few-shot image classiﬁcation benchmark. It consists of 100 classes from ImageNet dataset [30].

Revised: Each class contains 600 images of size 84 × 84.

Revised: These classes are split into 64, 16, and 20 classes for meta-training, meta-validation, and meta-testing, respectively [28].

Revised: CUB contains 200 classes with total 11,788 images of size 84 × 84.

Revised: Following previous works [5], the base, validation, and novel split are 100, 50, and 50 classes respectively.

Revised: CIFAR-FS is a variant of the CIFAR-100 dataset used for few-shot classiﬁcation.

Revised: It containsclasses, each with 600 images of 32 × 32 pixels.

Revised: The classes are randomly split into 64, 16, andfor meta-training, meta-validation, and meta-testing.