Original: Improve Task-Specific Generalization in Few-Shot Learning via Adaptive Vicinal Risk Minimization
Revised: Improving Task-Specific Generalization in Few-Shot Learning via Adaptive Vicinal Risk Minimization

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AffiliationAddress email
Revised: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Recent years have witnessed the rapid development of meta-learning in improvingthe meta generalization over tasks in few-shot learning.
Revised: Recent years have witnessed the rapid development of meta-learning in improvingthe meta generalization over tasks in few-shot learning.

Original: However, the task-specificlevel generalization is overlooked in most algorithms.
Revised: However, the task-specificlevel generalization is overlooked in most algorithms.

Original: For a novel few-shot learningtask where the empirical distribution likely deviates from the true distribution, themodel obtained via minimizing the empirical loss can hardly generalize to unseendata.
Revised: For a novel few-shot learningtask where the empirical distribution likely deviates from the true distribution, themodel obtained via minimizing the empirical loss can hardly generalize to unseendata.

Original: A viable solution to improving the generalization comes as a more accurateapproximation of the true distribution; that is, admitting a Gaussian-like vicinaldistribution for each of the limited training samples.
Revised: A viable solution to improving the generalization comes as a more accurateapproximation of the true distribution; that is, admitting a Gaussian-like vicinaldistribution for each of the limited training samples.

Original: Thereupon we derive theresulting vicinal loss function over vicinities of all training samples and minimizeit instead of the conventional empirical loss over training samples only, favorablyfree from the exhaustive sampling of all vicinal samples.
Revised: Thereupon we derive theresulting vicinal loss function over vicinities of all training samples and minimizeit instead of the conventional empirical loss over training samples only, favorablyfree from the exhaustive sampling of all vicinal samples.

Original: It remains challengingto obtain the statistical parameters of the vicinal distribution for each sample.
Revised: It remains challengingto obtain the statistical parameters of the vicinal distribution for each sample.

Original: Totackle this challenge, we further propose to estimate the statistical parameters asthe weighted mean and variance of a set of unlabeled data it passed by a randomwalk starting from training samples.
Revised: Totackle this challenge, we further propose to estimate the statistical parameters asthe weighted mean and variance of a set of unlabeled data it passed by a randomwalk starting from training samples.

Original: To verify the performance of the proposedmethod, we conduct experiments on four standard few-shot learning benchmarksand consolidate the superiority of the proposed method over state-of-the-art fewshot learning baselines.
Revised: To verify the performance of the proposedmethod, we conduct experiments on four standard few-shot learning benchmarksand consolidate the superiority of the proposed method over state-of-the-art fewshot learning baselines.

Original:  Introduction
Revised:  Introduction

Original: In recent years, deep learning algorithms have made tremendous progress and have been widelyapplied in vision [14], language [31] and speech [8] areas.
Revised: In recent years, deep learning algorithms have made tremendous progress and have been widelyapplied in vision [14], language [31] and speech [8] areas.

Original: Despite the prevalence, deep learningalgorithms rely heavily on a sufficient amount of labeled data and often fail when only a limitednumber of labeled samples are available.
Revised: Despite the prevalence, deep learningalgorithms rely heavily on a sufficient amount of labeled data and often fail when only a limitednumber of labeled samples are available.

Original: Considering the high cost of extensive data collection andannotation, it has attracted increasing attention to develop algorithms learning from a limited numberof training samples, which has been widely known as few-shot learning [33, 4].
Revised: Considering the high cost of extensive data collection andannotation, it has attracted increasing attention to develop algorithms learning from a limited numberof training samples, which has been widely known as few-shot learning [33, 4].

Original: The recent focus of few-shot learning is built on the meta-learning framework [6, 33, 27].
Revised: The recent focus of few-shot learning is built on the meta-learning framework [6, 33, 27].

Original: The goalof meta-learning is to train a model that can be quickly adapted to a new task with a small number oflabeled samples by leveraging the knowledge from previous related tasks.
Revised: The goalof meta-learning is to train a model that can be quickly adapted to a new task with a small number oflabeled samples by leveraging the knowledge from previous related tasks.

Original: Typically, a meta-learningmodel consists of the meta-training stage where a well-generalized feature extractor is learned frommeta-training tasks and the meta-testing stage during which a classifier is built on top of the featureextractor for novel tasks.
Revised: Typically, a meta-learningmodel consists of the meta-training stage where a well-generalized feature extractor is learned frommeta-training tasks and the meta-testing stage during which a classifier is built on top of the featureextractor for novel tasks.

Original: While the majority of meta-learning works revolved around developingbetter meta-training strategies to reduce the meta generalization error over all tasks, how to fullyleverage the meta-trained feature extractor to build a robust classifier and improve a task-specificgeneralization in the meta-testing stage remains less explored.
Revised: While the majority of meta-learning works revolved around developingbetter meta-training strategies to reduce the meta generalization error over all tasks, how to fullyleverage the meta-trained feature extractor to build a robust classifier and improve a task-specificgeneralization in the meta-testing stage remains less explored.

Original: The challenge to building such a classifier lies in the significant deviation of the empirical distribution(i.e.
Revised: The challenge to building such a classifier lies in the significant deviation of the empirical distribution(i.e.

Original: the distribution formed by the training data) from the population distribution (i.e. the true
Revised: the distribution formed by the training data) from the population distribution (i.e. the true

Original: [Figure]
Revised: [Figure]

Original: distribution).
Revised: distribution).

Original: As illustrated in Figure 1, a model well fitting the training data (support set) maygeneralize poorly on the testing data (query set) from the true distribution due to the bias in theempirical distribution.
Revised: As illustrated in Figure 1, a model well fitting the training data (support set) maygeneralize poorly on the testing data (query set) from the true distribution due to the bias in theempirical distribution.

Original: One viable solution is to approximate the true distribution more accurately –by viewing the empirical distribution as the assembling of delta functions located on each trainingsample, Chapelle et al.
Revised: One viable solution is to approximate the true distribution more accurately –by viewing the empirical distribution as the assembling of delta functions located on each trainingsample, Chapelle et al.

Original: [3] proposed the Vicinal Risk Minimization loss function where the deltafunctions are replaced by some density estimate in the vicinity of each training data.
Revised: [3] proposed the Vicinal Risk Minimization loss function where the deltafunctions are replaced by some density estimate in the vicinity of each training data.

Original: In the eraof deep learning, the VRM learning principle is implemented as Mixup [41, 32], which is simpleyet effective given a sufficiently large number of training data.
Revised: In the eraof deep learning, the VRM learning principle is implemented as Mixup [41, 32], which is simpleyet effective given a sufficiently large number of training data.

Original: However, as we have verified in
Revised: However, as we have verified in

Original: Table 3, the performance improvement by applying Mixup on the support set is limited in few-shotlearning where the vicinal distributions assembled by Mixup is still significantly biased.
Revised: Table 3, the performance improvement by applying Mixup on the support set is limited in few-shotlearning where the vicinal distributions assembled by Mixup is still significantly biased.

Original: To improvethe generalization of a few-shot classifier during meta-testing, we are motivated to revisit the VRMprinciple to 1) generate adaptive vicinal distribution for each samples in support set and 2) minimizethe expected vincial loss over the vicinal distributions.
Revised: To improvethe generalization of a few-shot classifier during meta-testing, we are motivated to revisit the VRMprinciple to 1) generate adaptive vicinal distribution for each sample in support set and 2) minimizethe expected vicinal loss over the vicinal distributions.

Original: However, both are challenging.
Revised: However, both are challenging.

Original: Without loss of generality, we assume that the vicinity of each training point admits a Gaussiandistribution.
Revised: Without loss of generality, we assume that the vicinity of each training point admits a Gaussiandistribution.

Original: In [3], the means of the vicinal distributions are located on the original training points.
Revised: In [3], the means of the vicinal distributions are located on the original training points.

Original: As illustrated in Figure 1(b), however, setting the mean in this way is not sufficient to approximatethe true distribution, as an original training point likely deviates from the mean of the ground-truthdistribution of its corresponding class.
Revised: As illustrated in Figure 1(b), however, setting the mean in this way is not sufficient to approximatethe true distribution, as an original training point likely deviates from the mean of the ground-truthdistribution of its corresponding class.

Original: In fact, it is difficult to estimate the mean and variance basedonly on the labeled training samples.
Revised: In fact, it is difficult to estimate the mean and variance basedonly on the labeled training samples.

Original: In this paper, we assume that unlabeled samples are alsoavailable, and leverage these unlabeled samples to estimate the mean and variance of the vicinaldistribution for each labeled sample.
Revised: In this paper, we assume that unlabeled samples are alsoavailable, and leverage these unlabeled samples to estimate the mean and variance of the vicinaldistribution for each labeled sample.

Original: Concretely, inspired by [42], we first construct a transitionprobability matrix based on pairwise distances between features of all samples, and secondly performthe lazy random walk among unlabeled samples starting from each labeled sample.
Revised: Concretely, inspired by [42], we first construct a transitionprobability matrix based on pairwise distances between features of all samples, and secondly performthe lazy random walk among unlabeled samples starting from each labeled sample.

Original: After walkinga certain number of steps, we obtain the visit probabilities of all the samples and infer the vicinaldistribution whose mean and variance are the weighted mean and variance of all samples visited.
Revised: After walkinga certain number of steps, we obtain the visit probabilities of all the samples and infer the vicinaldistribution whose mean and variance are the weighted mean and variance of all samples visited.

Original: All that remains is the loss function over the vicinal distributions of all training samples.
Revised: All that remains is the loss function over the vicinal distributions of all training samples.

Original: An intuitiveway to do this is to exploit the Monte Carlo method, which repeats random sampling to explicitlygenerate data points from each vicinal distribution and minimizes the loss on all the sampled datapoints.
Revised: An intuitiveway to do this is to exploit the Monte Carlo method, which repeats random sampling to explicitlygenerate data points from each vicinal distribution and minimizes the loss on all the sampled datapoints.

Original: Unfortunately, it requires to draw a large amount of samples to minimize the Monte Carloapproximation error, which significantly increases the computation cost.
Revised: Unfortunately, it requires to draw a large amount of samples to minimize the Monte Carloapproximation error, which significantly increases the computation cost.

Original: Instead, we follow [3] toderive a closed form of the expected vicinal loss over the vicinal distributions.
Revised: Instead, we follow [3] toderive a closed form of the expected vicinal loss over the vicinal distributions.

Original: In this paper, we focuson building an optimization-based classifier in optimzation-based meta-learning algorithms [6, 16],which has been proved with much more adaptability and generalization capacity in novel meta-testingtasks than a nearest-neighbour classifier in metric-based meta-learning algorithms [33, 27, 28, 21].
Revised: In this paper, we focuson building an optimization-based classifier in optimization-based meta-learning algorithms [6, 16],which has been proved with much more adaptability and generalization capacity in novel meta-testingtasks than a nearest-neighbour classifier in metric-based meta-learning algorithms [33, 27, 28, 21].

Original: Particularly, we consider both the cross-entropy loss and the SVM loss functions which are used asclassifier in the state-of-the-art meta-learing algorithms [37, 16].
Revised: Particularly, we consider both the cross-entropy loss and the SVM loss functions which are used asclassifier in the state-of-the-art meta-learning algorithms [37, 16].

Original: For the vicinal version of crossentropy loss, it cannot be computed in closed form.
Revised: For the vicinal version of crossentropy loss, it cannot be computed in closed form.

Original: We instead derive its quadratic approximation.
Revised: We instead derive its quadratic approximation.

Original: For SVM, we derive its closed-form vicinal loss.
Revised: For SVM, we derive its closed-form vicinal loss.

Original: In summary, our contributions are threefold:
Revised: In summary, our contributions are threefold:

Original: • We revisit vicinal risk minimization (VRM) in deep few-shot learning and derive the resultingexpected vicinal loss functions for both the cross-entropy loss and SVM.•
Revised: • We revisit vicinal risk minimization (VRM) in deep few-shot learning and derive the resultingexpected vicinal loss functions for both the cross-entropy loss and SVM.•

Original: We propose to harness unlabeled data and a lazy random walk algorithm to generate adaptivestatistical parameters for the vicinal distribution of each training sample.
Revised: We propose to harness unlabeled data and a lazy random walk algorithm to generate adaptivestatistical parameters for the vicinal distribution of each training sample.

Original: • We have validated the superiority of the proposed algorithm with extensive experiments on threebenchmars, i.e., miniImageNet, CUB, CIFAR-FS.
Revised: • We have validated the superiority of the proposed algorithm with extensive experiments on threebenchmarks, i e., miniImageNet, CUB, CIFAR-FS.

Original:  Related Works2.1 Few-Shot Learning and Meta-Learning
Revised:  Related Works2.1 Few-Shot Learning and Meta-Learning

Original: Few-shot learning aims to recognize novel classes with insufficient labeled instances.
Revised: Few-shot learning aims to recognize novel classes with insufficient labeled instances.

Original: The recentfocus of few-shot learning algorithms has been on the meta-learning framework, which trains themodel with multiple meta-episodes to enable quick learning and applies the model to improvethe learning in novel tasks.
Revised: The recentfocus of few-shot learning algorithms has been on the meta-learning framework, which trains themodel with multiple meta-episodes to enable quick learning and applies the model to improvethe learning in novel tasks.

Original: The meta-learning methods fall into two categories: metric-basedmethods and optimization-based methods.
Revised: The meta-learning methods fall into two categories: metric-basedmethods and optimization-based methods.

Original: Metric-based methods [13, 33, 27, 28, 20, 39] learn ametric to compare two samples’ feature embeddings encoded by the feature extractor and classifya sample in  query set based on its distances to the samples in the support set.
Revised: Metric-based methods [13, 33, 27, 28, 20, 39] learn ametric to compare two samples’ feature embeddings encoded by the feature extractor and classify asample in the query set based on its distances to the samples in the support set.

Original: Optimization-basedapproaches [6, 23, 25, 29, 35] learn a good initialization such that it can be fast adapted to novel tasksbased on a few instances.
Revised: Optimization-basedapproaches [6, 23, 25, 29, 35] learn a good initialization such that it can be fast adapted to novel tasksbased on a few instances.

Original: To tackle the data deficiency issue in few-shot learning, researchers have explored the idea ofexploiting unlabeled data and developed transductive learning or semi-supervised learning techniquesfor few-shot learning, either by leveraging nearest neighbor graph for propagating labels [36, 12, 18]or by leveraging predicted soft or hard labels on unlabeled samples to update the class prototypesor metric [24, 10, 15, 22, 39].
Revised: To tackle the data deficiency issue in few-shot learning, researchers have explored the idea ofexploiting unlabeled data and developed transductive learning or semi-supervised learning techniquesfor few-shot learning, either by leveraging nearest neighbor graph for propagating labels [36, 12, 18]or by leveraging predicted soft or hard labels on unlabeled samples to update the class prototypesor metric [24, 10, 15, 22, 39].

Original: Our method is not built on metric-based method.
Revised: Our method is not built on metric-based method.

Original: We make useof the unlabeled data to derive the vicinal distribution for each support sample and optimize theconsequential vicinal loss to learn a classifier.
Revised: We make useof the unlabeled data to derive the vicinal distribution for each support sample and optimize theconsequential vicinal loss to learn a classifier.

Original: Our approach shares a similar spirit of [37].
Revised: Our approach shares a similar spirit of [37].

Original: It [37]proposes to calibrate the distribution of the support samples by transferring statistics from the baseclasses and train the classifier on a set of samples sampled from the calibrated distributions.
Revised: It [37]proposes to calibrate the distribution of the support samples by transferring statistics from the baseclasses and train the classifier on a set of samples sampled from the calibrated distributions.

Original: Unlike[37], our method obtains more accurate vicinal distributions by making use of the unlabeled datafrom the same task.
Revised: Unlike[37], our method obtains more accurate vicinal distributions by making use of the unlabeled datafrom the same task.

Original: Besides, instead of sampling extensive data from the distributions, our modeldirectly optimizes the expected vicinal loss over the distributions, resulting in more efficient trainingand higher performance.
Revised: Besides, instead of sampling extensive data from the distributions, our modeldirectly optimizes the expected vicinal loss over the distributions, resulting in more efficient trainingand higher performance.

Original: 2.2 Vicinal Risk Minimization
Revised: 2.2 Vicinal Risk Minimization

Original: The vicinal Risk Minimization principle is introduced in [3, 30].
Revised: The vicinal Risk Minimization principle is introduced in [3, 30].

Original: It proposes to assume each trainingsample admits a vicinal distribution and to optimize the expected loss over the vicinal distributions.
Revised: It proposes to assume each trainingsample admits a vicinal distribution and to optimize the expected loss over the vicinal distributions.

Original: In [3], only binary VRM classifier are presented.
Revised: In [3], only binary VRM classifier are presented.

Original: Our method considers multi-class classifier andcross-entropy loss, which is more challenging to derive the expected vicinal loss.
Revised: Our method considers multi-class classifier andcross-entropy loss, which is more challenging to derive the expected vicinal loss.

Original: Besides, (author?)[3] consider Gaussian vicinities whose means lies on the original training samples.
Revised: Besides, Chapelleet al. [3] consider Gaussian vicinities whose means lie on the original training samples.

Original: Our approachalso considers Gaussian vicinities but estimates the mean by the unlabeled data, which is generallynot on the training samples.
Revised: Our approachalso considers Gaussian vicinities but estimates the mean by the unlabeled data, which is generallynot on the training samples.

Original: In the era of deep learning, the most popular VRM technique in deep learning algorithms is Mixup [41,32].
Revised: In the era of deep learning, the most popular VRM technique in deep learning algorithms is Mixup [41,32].

Original: Mixup constructs the vicinal distributions as the convex combination of the training data, whichcan be understood as a data augmentation technique that encourages the model to behave linearlyinter training examples.
Revised: Mixup constructs the vicinal distributions as the convex combination of the training data, whichcan be understood as a data augmentation technique that encourages the model to behave linearlyinter training examples.

Original: It has been demonstrated that Mixup is simple yet efficient and improves thegeneralization of deep model given sufficiently many training samples.
Revised: It has been demonstrated that Mixup is simple yet efficient and improves thegeneralization of deep model given sufficiently many training samples.

Original: Mixup is also introduced onthe meta-learning algorithm in [38, 7], but is applied on the query set for meta model optimization.
Revised: Mixup is also introduced onthe meta-learning algorithm in [38, 7], but is applied on the query set for meta model optimization.

Original: As shown by the experiment (Table 6) in [38], applying Mixup on the support set could even hurtthe performance.
Revised: As shown by the experiment (Table 6) in [38], applying Mixup on the support set could even hurtthe performance.

Original: Our approach instead leverages the unlabeled data for a more accurate vicinaldistribution estimate for an extremely small support set.
Revised: Our approach instead leverages the unlabeled data for a more accurate vicinaldistribution estimate for an extremely small support set.

Original:  Adaptive Vicinal Few-Shot Learning
Revised:  Adaptive Vicinal Few-Shot Learning

Original: In this section, we will introduce the proposed algorithm, AD aptive V icinal Few-Shot Learning(ADV).
Revised: In this section, we will introduce the proposed algorithm, AD aptive V icinal Few-Shot Learning(ADV).

Original: We will first introduce the VRM; then derive the expected vicinal loss for  linear classifierwith cross-entropy loss and for SVM; and finally propose tge algorithm based on lazy random walksto obtain the adaptive vicinal distribution for each labeled support sample.
Revised: We will first introduce the VRM; then derive the expected vicinal loss for a linear classifierwith cross-entropy loss and for SVM; and finally propose the algorithm based on lazy random walksto obtain the adaptive vicinal distribution for each labeled support sample.

Original: 3.1 Vicinal Risk Minimization
Revised: 3.1 Vicinal Risk Minimization

Original: Given data x, y following the distribution P ( x, y ) , the loss function L , we would like the optimize amodel F ( x; θ ) by minimizing the expected risk
Revised: Given data x, y following the distribution P ( x, y ) , the loss function L , we would like the optimize amodel f θ ( x ) by minimizing the expected risk

Original: [Equation]
Revised: [Equation]

Original: Here, x is the input data, y is its label and θ is the model parameters.
Revised: Here, x is the input data, y is its label and θ is the model parameters.

Original: In practice, P ( x, y ) is unknownand a set of training data { x i , y i } Ni =1 is available, where N is the size of training data.
Revised: In practice, P ( x, y ) is unknownand a set of training data { x i , y i } Ni =1 is available, where N is the size of training data.

Original: To learn amodel, one can minimize the empirical risk, which is equivalent to minimizing the expect loss withrespect to an empirical density estimate d P δ ( x, y )
Revised: To learn amodel, one can minimize the empirical risk, which is equivalent to minimizing the expected loss withrespect to an empirical density estimate d P δ ( x, y )

Original: = 1 N (cid:80) Ni
Revised: = 1 N (cid:80) Ni

Original: δ x i ( x ) δ y
Revised: δ x i ( x ) δ y

Original: i
Revised: i

Original: ( y ) , where δ a ( · ) is a Diracdelta function centered at a .
Revised: ( y ) , where δ a ( · ) is a Diracdelta function centered at a .

Original: Given a limited number of training sample, the empirical density estimate d P δ ( x, y ) could bias fromthe true density d P ( x, y ) .
Revised: Given a limited number of training sample, the empirical density estimate d P δ ( x, y ) could bias fromthe true density d P ( x, y ) .

Original: For a more accurate density estimate, Chapelle et al.
Revised: For a more accurate density estimate, Chapelle et al.

Original: [3] proposed toreplace the delta functions δ x i ( x ) by some estimate of density in the vicinity of x i .
Revised: [3] proposed toreplace the delta functions δ x i ( x ) by some estimate of density in the vicinity of x i .

Original: Given a vicinaldistribution of x i as P x i ( x ) , the improved density estimate is P
Revised: Given a vicinaldistribution of x i as P x i ( x ) , the improved density estimate is P

Original: ν
Revised: ν

Original: ( x, y ) = 1 N (cid:80) Ni =
Revised: ( x, y ) = 1 N (cid:80) Ni =

Original: 1 P x i ( x ) δ y i ( y ) .
Revised: 1 P x i ( x ) δ y i ( y ) .

Original: And the corresponding vicinal risk is defined as
Revised: And the corresponding vicinal risk is defined as

Original: [Equation]
Revised: [Equation]

Original: 3.2 Vicinal Loss
Revised: 3.2 Vicinal Loss

Original: In this paper, we assume the feature extractor is trained and fixed for novel tasks.
Revised: In this paper, we assume the feature extractor is trained and fixed for novel tasks.

Original: Each tasks is
Revised: Each tasks is

Original: C -way K -shot, consisting of N = CK data support samples { ( x i , y i ) }
Revised: C -way K -shot, consisting of N = CK data support samples { ( x i , y i ) }

Original: Ni =1 .
Revised: Ni =1 .

Original: To perform VRM, weassume the feature
Revised: To perform VRM, weassume the feature

Original: z i of each sample x i in the support set admits a Gaussian vicinal distribution
Revised: z i of each sample x i in the support set admits a Gaussian vicinal distribution

Original: N ( µ i , Σ i ) such that a sample ˜ zi is drawn from the vicinal distribution with probability
Revised: N ( µ i , Σ i ) such that a sample ˜ z is drawn from the vicinal distribution with probability

Original: P z
Revised: P z

Original: i
Revised: i

Original: (˜ z )
Revised: (˜ z )

Original: =det(2 π Σ i ) − 12 exp( − 12 ( x − µ i )
Revised: =det(2 π Σ i ) − 12 exp( − 12 (˜ z − µ i )

Original: ⊤ Σ − 1 i ( x − µ i )) , where µ i and Σ
Revised: ⊤ Σ − 1 i (˜ z − µ i )) , where µ i and Σ

Original: i is the mean and variance of thevicinal distribution of x i .
Revised: i is the mean and variance of thevicinal distribution of z i .

Original: 3.2.1 Vicinal Cross-Entropy Loss
Revised: 3.2.1 Vicinal Cross-Entropy Loss

Original: We first focus on linear classifier with Cross-Entropy loss.
Revised: We first focus on linear classifier with Cross-Entropy loss.

Original: A linear classifier f θ ( z ) =
Revised: A linear classifier f θ ( z ) =

Original: ( W ⊤ z + b )maps a feature z ∈ R d to its logits, where W ∈ R d × K and b ∈ R K is the parameter of the classifier.
Revised: ( W ⊤ z + b )maps a feature z ∈ R d to its logits, where W ∈ R d × C and b ∈ R C is the parameter of the classifier.

Original: The cross entropy loss is defined as ℓ ( h, y )
Revised: The cross-entropy loss is defined as ℓ ( h, y )

Original: = − e ⊤ y log( softmax ( f θ ( z ))) , where y ∈ 1 , 2 , ..., C isthe label; e y is a one-hot vector with  y -th element being 1 .
Revised: = − e ⊤ y log( softmax ( f θ ( z ))) , where y ∈ 1 , 2 , ..., C is thelabel; e y is a one-hot vector with the y -th element being 1 .

Original: The expected loss of cross entropy over
Revised: The expected loss of cross entropy over

Original: Gaussian vicinal distribution of x is defined as
Revised: Gaussian vicinal distribution of x is defined as

Original: [Equation]
Revised: [Equation]

Original: Due to the softmax function, the expected loss (1) cannot be computed in closed form.
Revised: Due to the softmax function, the expected loss (1) cannot be computed in closed form.

Original: Instead, weutilize Taylor expansion to derive an approximation of (1).
Revised: Instead, weutilize Taylor expansion to derive an approximation of (1).

Original: By taking second-order Taylor expansionof ℓ ( f θ (˜ z ) , y ) around the mean of its vicinal distribution µ , we obtain
Revised: By taking second-order Taylor expansionof ℓ ( f θ (˜ z ) , y ) around the mean of its vicinal distribution µ , we obtain

Original: [Equation]
Revised: [Equation]

Original: (2) where G is the gradient and H is the Hessian w.r.t.
Revised: (2) where G is the gradient and H is the Hessian w.r.t.

Original: input µ .
Revised: input µ .

Original: Note that H
Revised: Note that H

Original: = W ( diag ( h ) −
Revised: = W ( diag ( h ) −

Original: hh ⊤ ) W ⊤where h = softmax ( f θ ( µ )) and diag ( h ) is a diagonal matrix whose diagonal elements are equal to h[26].
Revised: hh ⊤ ) W ⊤where h = softmax ( f θ ( µ )) and diag ( h ) is a diagonal matrix whose diagonal elements are equal to h[26].

Original: By take expectation over ˜ z , we have
Revised: By taking expectation over ˜ z , we have

Original: [Equation]
Revised: [Equation]

Original: where Σ is the variance of ˜ z , h k is the k -th elements of h , and W k is the k -th column of W . Plugging(2), (3) and (4) into (1), we obtain the approximated expected vicinal cross-entropy loss for the linearclassifier as
Revised: where Σ is the variance of ˜ z , h k is the k -th elements of h , and W k is the k -th column of W . Plugging(2), (3) and (4) into (1), we obtain the approximated expected vicinal cross-entropy loss for the linearclassifier as

Original: [Equation]
Revised: [Equation]

Original: 3.2.2 Vicinal SVM
Revised: 3.2.2 Vicinal SVM

Original: SVM is used as classifer for meta learning algorithms in [16, 37].
Revised: SVM is used as classifier for meta learning algorithms in [16, 37].

Original: We follow [16] to use the Crammerand Singer formulation of multi-class SVM [5].
Revised: We follow [16] to use the Crammerand Singer formulation of multi-class SVM [5].

Original: In [16, 5], the dual formulation of the objective for alinear multi-class SVM is obtained as:
Revised: In [16, 5], the dual formulation of the objective for alinear multi-class SVM is obtained as:

Original: [Equation]
Revised: [Equation]

Original: where α ki is the dual variables and λ is a regularization parameter.
Revised: where α ki is the dual variable and λ is a regularization parameter.

Original: The linear classifier for thek -th class is f kθ ( z ) =
Revised: The linear classifier for the k -thclass is f kθ ( z ) =

Original: w ⊤ k z , where w k = (cid:80) i
Revised: w ⊤ k z , where w k = (cid:80) i

Original: α ki
Revised: α ki

Original: z i .
Revised: z i .

Original: Note that here z is a unseen data and does notassume to admit a vicinal distribution.
Revised: Note that here z is an unseen data and does notassume to admit a vicinal distribution.

Original: If each z i admits a vicinal distribution N ( µ i , Σ i ) , the vicinalexpectation for the term z ⊤ i
Revised: If each z i admits a vicinal distribution N ( µ i , Σ i ) , the vicinalexpectation for the term z ⊤ i

Original: z j in (6) becomes
Revised: z j in (6) becomes

Original: E ˜ z

Original: i ∼N

Original: [Equation]

Original: µ i , Σ i ) E˜ [Equation] [Equation] [Equation]
Revised: E

Original: ] = µ ⊤

Original: [Equation]
Revised: [Equation]

Original: , and the classifier becomes E ˜ z
Revised: and the classifier becomes E ˜ z

Original: i ∼N ( µ i , Σ i ) [ f kθ ( z  )] =
Revised: i ∼N ( µ i , Σ i ) [ f kθ (˜ z i )] =

Original: ( (cid:80) i
Revised: ( (cid:80) i

Original: α ki µ i )
Revised: α ki µ i )

Original: ⊤ z.
Revised: ⊤ z.

Original: If µ i =
Revised: If µ i =

Original: z i , the objective of thevicinal SVM coincides with the original one.
Revised: z i , the objective of thevicinal SVM coincides with the original one.

Original: Even when µ i ̸
Revised: Even when µ i ̸

Original: =
Revised: =

Original: z i , it still does not take the varianceof the vicinal distribution into consideration and may result in suboptimal performance.
Revised: z i , it still does not take the varianceof the vicinal distribution into consideration and may result in suboptimal performance.

Original: To exploitthe variance information to improve the SVM classifier, we consider nonlinear classifier, i.e. thekernel-based SVM.
Revised: To exploitthe variance information to improve the SVM classifier, we consider nonlinear classifier, i.e. thekernel-based SVM.

Original: Multi-class kernel-based SVM uses a kernel K ( · , · ) and admits a similar dual formulation to (6).
Revised: Multi-class kernel-based SVM uses a kernel K ( · , · ) and admits a similar dual formulation to (6).

Original: The176 difference lays in that it replace the inner product of z i and z j in (6) by K ( z i , z j ) and replace the innerproduct of z i and z in the classifier by K ( z i , z ) .
Revised: The difference lays in that it replaces the inner product of z i and z j in (6) by K ( z i , z j ) and replacethe inner product of z i and z in the classifier by K ( z i , z ) .

Original: For the vicinal multi-class kernel-based  SVM, these178 two terms are further replaced by a two-vicinal kernel M ( z i , z j ) and a one-vicinal kernel Q ( z, z i ) .
Revised: For the vicinal multi-class kernel-based 178 SVM, these two terms are further replaced by a two-vicinal kernel M ( z i , z j ) and a one-vicinal kernel Q ( z, z i ) .

Original: In this paper, we consider the widely-used RBF kernel K ( z, z ′ ) =
Revised: In this paper, we consider the widely-used RBF kernel K ( z, z ′ ) =

Original: exp( − || z − z ′
Revised: exp( − || z − z ′

Original: || 2 2 σ) and construct thevicinal kernels for it as:
Revised: || 2 2 σ) andconstruct the vicinal kernels for it as:

Original: [Equation]
Revised: [Equation]

Original: Here, A ( σ 2 ) ∈
Revised: Here, A ( σ 2 ) ∈

Original: R d × d is a diagonal matrix with diagonal elements being σ 2 ; M i =
Revised: R d × d is a diagonal matrix with diagonal elements being σ 2 ; M i =

Original: ( A ( σ 2 ) + Σ i ) − 1 ;
Revised: ( A ( σ 2 ) + Σ i ) − 1 ;

Original: M ij =
Revised: M ij =

Original: ( A ( σ 2 ) + Σ i + Σ j ) − 1 ; the operator ∥ z ∥ 2 M = z ⊤ Mz .
Revised: ( A ( σ 2 )+Σ i +Σ j ) − 1 ; the operator ∥ z ∥ 2 M = z ⊤ Mz .

Original: Note that for computation simplication,in vicinal SVM, we assume the variance Σ i is diagonal.
Revised: Note that for computation simplification,in vicinal SVM, we assume the variance Σ i is diagonal.

Original: The derivation is available in Appendix.
Revised: The derivation is available in Appendix.

Original: Now we achieve the vicinal objective for multi-class kernel-based SVM as:
Revised: Now we achieve the vicinal objective for multi-class kernel-based SVM as:

Original: [Equation]
Revised: [Equation]

Original: with the classifier being f kθ ( z ) =
Revised: with the classifier being f kθ ( z ) =

Original: (cid:80) i α ki Q
Revised: (cid:80) i α ki Q

Original: ( z i , z ) .
Revised: ( z i , z ) .

Original: The objective in (9) is a quadratic program (QP)over dual variables { α ki } and can be solved by using a differentiable GPU-based QP solver [1].
Revised: The objective in (9) is a quadratic program (QP)over dual variables { α ki } and can be solved by using a differentiable GPU-based QP solver [1].

Original: 3.3 Lazy Random Walk for Adaptive Vicinal Distribution
Revised: 3.3 Lazy Random Walk for Adaptive Vicinal Distribution

Original: As illustrated in Figure 1, the performance of the model learned by VRM has a heavy reliance onthe quality of the vicinal distribution to approximate the true distribution.
Revised: As illustrated in Figure 1, the performance of the model learned by VRM has a heavy reliance onthe quality of the vicinal distribution to approximate the true distribution.

Original: However, it is almostimpossible to estimate accurate proximate distributions from a limited number of support samples,as they may deviate from the true distribution.
Revised: However, it is almostimpossible to estimate accurate proximate distributions from a limited number of support samples,as they may deviate from the true distribution.

Original: Therefore, we propose to exploit extra data, i.e. theunlabeled data from the same task, to reduce the estimation error of the statistical parameters for thevicinal distribution and, thus, improve the performance of the model.
Revised: Therefore, we propose to exploit extra data, i.e. theunlabeled data from the same task, to reduce the estimation error of the statistical parameters for thevicinal distribution and, thus, improve the performance of the model.

Original: We assume that a well-trained feature extractor on the meta-training tasks can preserve the similaritybetween the data in the feature space.
Revised: We assume that a well-trained feature extractor on the meta-training tasks can preserve the similaritybetween the data in the feature space.

Original: Data from the same class in the novel task would have similarfeatures with high probability.
Revised: Data from the same class in the novel task would have similarfeatures with high probability.

Original: Then, we apply lazy random walks algorithm
Revised: Then, we apply lazy random walks algorithm

Original: [42] to estimate theprobability that the unlabeled data would be visited starting from a support sample, and compute thestatistical parameters as the weighted mean and variance of the data visited by the lazy random walksalgorithm.
Revised: [42] to estimate theprobability that the unlabeled data would be visited starting from a support sample, and compute thestatistical parameters as the weighted mean and variance of the data visited by the lazy random walks.

Original: To perform lazy random walk, we first compute the distance D SU between the support sample
Revised: To perform lazy random walk, we first compute the distance D SU between the support sample

Original: z i andthe unlabeled data { u j } N u j =1 and compute the distance D UU between the unlabeled data as:
Revised: z i andthe unlabeled data { u j } N u j =1 and compute the distance D UU between the unlabeled data as:

Original: [Equation]
Revised: [Equation]

Original:  and (10)
Revised: 2 , and [Equation] [Equation]

Original: [Equation]
Revised: [Equation]

Original: Secondly, we transform these distances into transition probabilities by softmaxing the distance matrixover columns with a temperature τ :
Revised: Secondly, we transform these distances into transition probabilities by softmaxing the distance matrixover columns with a temperature τ :

Original: [Equation]
Revised: [Equation]

Revised: Note that we set D UUjj = ∞ such that P UUjj = 0 .

Original: Thirdly, we construct the transition probabilities matrix for z i in lazy random walk by combine P SUi :and P UU and setting the lazy stay probability β ∈ (0 , 1) as:
Revised: Thirdly, we construct the transition probabilities matrix for z i in lazy random walk by combining P SUi : and P UU and setting the lazy stay probability β ∈ (0 , 1) as:

Original: [Equation]
Revised: [Equation]

Original: where 0 n c × 1 is a all zero vector, and I N u is an identity matrix.
Revised: where 0 n c × 1 is a all zero vector, and I N u is an identity matrix.

Original: We then compute the T step visit probability v i for z i :
Revised: We then compute the T step visit probability v i for z i :

Original: [Equation]
Revised: [Equation]

Original: where e 1 is a one-hot vector with the first element being 1 .
Revised: where e 1 is a one-hot vector with the first element being 1 .

Original: and finally we obtain u
Revised: and finally we obtain µ i and Σ

Original: i and Σ i as theweighted mean and variance of z {+i and { u j } N u j =1 , with the weight being v i . That is: µ i =
 z i + 12 (cid:80) N u j =1 v j u j , and Σ i = 
 ( z i − µ i )( z i − µ i )+}
Revised: i asthe weighted mean and variance of z {+i and { u j } N u j =1 , with the weight being v i . That is: µ i =
 z i + 12 (cid:80) N u j =1 v j u j , and Σ i = 
 ( z i − µ i )( z i − µ i )+}

Original: i and { u j } N u j =1 , with the weight being V i  .
Revised: ⊤ + 12 (cid:80) N u j =1 v j ( u j − µ i )( u j − µ i ) ⊤ .

Original: We summarize the proposed ADaptive Vicinal Few-Shot Learning (ADV) in Algorithm 1.
Revised: We summarize the proposed ADaptive Vicinal Few-Shot Learning (ADV) in Algorithm 1.

Original:  Experiment
Revised:  Experiment

Original: In this section, we verify the effectiveness of the proposed approach in improve the generalizationfor few-shot learning.
Revised: In this section, we verify the effectiveness of the proposed approach in improving the generalizationfor few-shot learning.

Original: We first introduce the standard few-shot learning benchmark for performanceevaluation and the implementation details.
Revised: We first introduce the standard few-shot learning benchmark for performanceevaluation and the implementation details.

Original: Then we present the results on the benchmarks, followedby a detailed study on the proposed approach.
Revised: Then we present the results on the benchmarks, followedby a detailed study on the proposed approach.

Original: We use three benchmarks for performance evaluation: mini ImageNet [33], CUB [34] and CIFARFS [2].
Revised: We use three benchmarks for performance evaluation: mini ImageNet [33], CUB [34] and CIFARFS [2].

Original: For all datasets, we construct 5-way 1-shot and 5-way 5-shot tasks by randomly sampling 5classes.
Revised: For all datasets, we construct 5-way 1-shot and 5-way 5-shot tasks by randomly sampling 5classes.

Original: For performance evaluation, we construct 10 , 000 meta-testing tasks and report the averageclassification accuracy with a 95% confidence intervals.
Revised: For performance evaluation, we construct 10 , 000 meta-testing tasks and report the averageclassification accuracy with a 95% confidence interval.

Original: [Table]
Revised: [Table]

Original: 4.0.1 Implementation details
Revised: Implementation details

Original: We focus on meta-testing phase and directly use the WideResNet-28-10(WRN) [40] feature extractorspre-trained following the previous work [19].
Revised: We focus on meta-testing phase and directly use the WideResNet-28-10(WRN) [40] feature extractorspre-trained following the previous work [19] [Table]

Original: We denote the application of ADV to the linear classifier with vicinal cross-entropy loss by ADV-CEand denote the application of ADV to SVM by ADV-SVM.
Revised: We denote the application of ADV to the linear classifier with vicinal cross-entropy loss by ADV-CEand denote the application of ADV to SVM by ADV-SVM.

Original: For ADV-CE, we optimize the vicinalloss in (5) by gradient descent for 100 steps.
Revised: For ADV-CE, we optimize the vicinalloss in (5) by gradient descent for 100 steps.

Original: The learning rate is determined by performing gridsearch from 0 .
Revised: The learning rate is determined by performing gridsearch from 0 .

Original: 001 to 1 on the tasks constructed by meta-validation set.
Revised: 001 to 1 on the tasks constructed by meta-validation set.

Original: For ADV-SVM, we solve the
Revised: For ADV-SVM, we solve the

Original: QP in (9) by using a differentiable GPU-based QP solver [1].
Revised: QP in (9) by using a differentiable GPU-based QP solver [1].

Original: The regularization parameter λ is setto 0 .
Revised: The regularization parameter λ is setto 0 .

Original: 1 and the parameter σ for RBF kernel is obtained via grid search from 0 .
Revised: 1 and the parameter σ for RBF kernel is obtained via grid search from 0 .

Original: 1 to 10 .
Revised: 1 to 10 .

Original: The numberof step T , lazy stay probability β and temperature τ in the lazy random walks are obtained via gridsearch in { 1 , 2 , 3 , 4 , 5 } , { 0 . 1 , 0 . 2 , 0 . 5 } , { 0 . 01 , 0 . 1 , 1 , 10 } , respectively.
Revised: The numberof step T , lazy stay probability β and temperature τ in the lazy random walks are obtained via gridsearch in { 1 , 2 , 3 , 4 , 5 } , { 0 . 1 , 0 . 2 , 0 . 5 } , { 0 . 01 , 0 . 1 , 1 , 10 } , respectively.

Original: Notably, ADV accesses thequery set as unlabeled data to perform lazy random walks.
Revised: Notably, ADV accesses thequery set as unlabeled data to perform lazy random walks.

Original: Thus, ADV is a transductive methods.
Revised: Thus, ADV is a transductive method.

Revised: Thenumber of query samples per class is 15 .

Original: 4.1 Comparison with Baselines
Revised: 4.1 Comparison with Baselines

Original: We present the performance of ADV-CE, ADV-SVM with a WRN feature extractor and comparethe results with existing works in Table 1.
Revised: We present the performance of ADV-CE, ADV-SVM with a WRN feature extractor and comparethe results with existing works in Table 1.

Original: The results of baselines are from [16], [22] and [11].
Revised: The results of baselines are from [16], [22] and [11].

Original: From Table 1 ADV achieves the best performance in all the datasets for both 1-shot and 5-shot tasks.
Revised: From Table 1 ADV achieves the best performance in all the datasets for both 1-shot and 5-shot tasks.

Original: Notable, ADV outperforms the best competitor by at least 2% for 1-shot tasks.
Revised: Notable, ADV outperforms the best competitor by at least 2% for 1-shot tasks.

Original: In 1-shot tasks, theempirical distribution estimated by the support set can deviate from the true distribution by a largemargin.
Revised: In 1-shot tasks, theempirical distribution estimated by the support set can deviate from the true distribution by a largemargin.

Original: The extraordinary performance of ADV in 1-shot tasks indicates its effectiveness to exploitthe information of the unlabeled data from the query set for a more accurate distribution estimate tothe true distribution of each class.
Revised: The extraordinary performance of ADV in 1-shot tasks indicates its effectiveness to exploitthe information of the unlabeled data from the query set for a more accurate distribution estimate tothe true distribution of each class.

Original: 4.2 Visualization of Vicinal Distributions
Revised: 4.2 Visualization of Vicinal Distributions

Original: The vicinal risk minimization principle in the proposed method is built on the vicinal distributionsobtained by lazy random walks.
Revised: The vicinal risk minimization principle in the proposed method is built on the vicinal distributionsobtained by lazy random walks.

Original: Thus, the performance of ADV depends on the exactness of vicinaldistribution estimate.
Revised: Thus, the performance of ADV depends on the exactness of vicinaldistribution estimate.

Original: Here, we visualize the vicinal distributions to evaluate its exactness.
Revised: Here, we visualize the vicinal distributions to evaluate their exactness.

Original: We runthe proposed lazy random walks in Section 3.3 on tow 5-way 1-shot tasks randomly constructedon the meta-testing set of miniImageNet, generate 500 samples from each vicinal distribution for
Revised: Werun the proposed lazy random walks in Section 3.3 on two 5-way 1-shot tasks randomly constructedon the meta-testing set of miniImageNet, generate 500 samples from each vicinal distribution for

Original: [Table]
Revised: [Table]

Original: visualization, and perform t-SNE algorithm on all data.
Revised: visualization, and perform t-SNE algorithm on all data.

Original: The visualization results are presented in
Revised: The visualization results are presented in

Original: Figure 2.
Revised: Figure 2.

Original: From Figure
Revised: From Figure

Original: [Table]
Revised: [Table]

Original: we can observe that the vicinal distribution generated by ADV covers most of thequery samples from the same class.
Revised: we can observe that the vicinal distribution generated by ADV covers most of thequery samples from the same class.

Original: This validates that the lazy random walks module in ADV canprovide a good estimate of the vicinal distribution, which overlaps with the ground truth distribution.
Revised: This validates that the lazy random walks module in ADV canprovide a good estimate of the vicinal distribution, which overlaps with the ground truth distribution.

Original: By optimize the expected loss over the vicinal distributions, the model learned by ADV ishas asmaller probability of overfitting the support samples and has a better generalization to the query set.
Revised: By optimizing the expected loss over the vicinal distributions, the model learned by ADV has asmaller probability of overfitting the support samples and has a better generalization to the query set.

Original: 4.3 Effect of the Network Architecture of Feature Extractor
Revised: 4.3 Effect of the Network Architecture of Feature Extractor

Original: The proposed algorithm ADVis develop at feature level. Therefore, it can be applied on any pretrainedfeature extractor.
Revised: The proposed algorithm ADV can be applied on any pre-trained feature extractor.

Original: Here, we validate the effectiveness of ADV on three feature extractors with differentbackbones that  trained by different meta-learning methods.
Revised: Here, we validatethe effectiveness of ADV on three feature extractors with different backbones that are trained bydifferent meta-learning methods.

Original: We consider WideResNet (WRN) [40], ResNet-12 [9] and a shallow network of four convolutional blocks (CONV) [2].
Revised: We consider WideResNet (WRN) [40], ResNet-12 [9] and ashallow network of four convolutional blocks (CONV) [2].

Original: We meta-train the WRN, ResNet, CONV following [19], [16] and [2], respectively.
Revised: We meta-train the WRN, ResNet, CONVfollowing [19], [16] and [2], respectively.

Original: We report the performance of a kernel SVM classifier and a linear classifier trained by cross-entropy(denoted by CE) with and without ADV.
Revised: We report the performance of a kernel SVM classifier and a linear classifier trained by cross-entropy(denoted by CE) with and without ADV.

Original: The results in Table 2 show the performance improvementof ADV is consistent over all three different feature extractors.
Revised: The results in Table 2 show the performance improvementof ADV is consistent over all three different feature extractors.

Original: This verifies that the effectiveness of
Revised: This verifies that the effectiveness of

Original: ADV is feature extractor-independent.
Revised: ADV is feature extractor-independent.

Original: It is worth noting the larger performance gains are observedon the better feature extractor.
Revised: It is worth noting the larger performance gains are observedon the better feature extractor.

Original: For example, the overall performance of WRN is better than CONV,and the performance boost of ADV-CE over CE of 1-shot tasks of WRN is 10.5%, which is largerthan the boost on CONV (4.8%).
Revised: For example, the overall performance of WRN is better than CONV,and the performance boost of ADV-CE over CE of 1-shot tasks of WRN is 10.5%, which is largerthan the boost on CONV (4.8%).

Original: This observation indicates: 1) even with a superior feature extractorthat
Revised: This observation indicates: 1) even with a superior feature extractorthat

Original: well
Revised: well

Original: preserves the similarity between data in feature space, the classifier trained from a limitednumber of high-quality support data still overfitsto the data; 2) for a better feature extractor, ADVcan exploit more information from the similarity between data to estimate the vicinal distribution ofsupport data and alleviate the overfitting.
Revised: preserves the similarity between data in feature space, the classifier trained from a limitednumber of high-quality support data still overfits the data; 2) for a better feature extractor, ADVcan exploit more information from the similarity between data to estimate the vicinal distribution ofsupport data and alleviate the overfitting.

Original: 4.4 Ablation Study
Revised: 4.4 Ablation Study

Original: We perform  ablation study on ADV about the gain of  VRM principle.
Revised: We perform an ablation study on ADV about the gain of the VRM principle.

Original: To do this, we train avanilla linear classifier by cross-entropy loss (denoted by CE), a linear classifier with Mixup vicinaldistribution [41] (denoted by CE-Mixup), a linear classifier with Delta vicinal distribution whose
Revised: To do this, we train avanilla linear classifier by cross-entropy loss (denoted by CE), a linear classifier with Mixup vicinaldistribution [41] (denoted by CE-Mixup), a linear classifier with Delta vicinal distribution whose

Original: [Table]
Revised: [Table]

Original:  mean located on the same mean estimate by our lazy random walks module and  compare them with ADV-CE.
Revised: 87.41 ± 0. mean locates on the same mean estimate by our lazy random walks module and whose variance iszero (denoted by ADV-CE-DD), and compare them with ADV-CE.

Revised: Note that by using a Delta vicinaldistribution, only the mean will contributes to the loss.

Original: We also train a multi-class SVM classifier, a multi-class kernel SVM classifier, and akernel SVM classifier with Delta vicinal distribution (denoted by ADV-SVM-DD).
Revised: We also train a multi-class SVM classifier,a multi-class kernel SVM classifier, and a kernel SVM classifier with Delta vicinal distribution(denoted by ADV-SVM-DD).

Original: The training ofmulti-class SVM classifier all follows [5].
Revised: The training of multi-class SVM classifier all follows [5].

Original: We report the performance on three benchmarks in Table3.
Revised: We reportthe performance on three benchmarks in Table 3.

Original: Note that by using a Delta vicinal distribution, only the mean will contributes to the loss.

Original: In Table 3, we can observe that the performance of CE-Mixup is just competitive with CE inminiImageNet and CIFAR-FS, and even inferior to CE on CUB dataset.
Revised: In Table 3, we can observe that the performance of CE-Mixup is just competitive with CE inminiImageNet and CIFAR-FS, and even inferior to CE on CUB dataset.

Original: This suggests the Mixupvicinal distribution fails to improve the generalization given a limited number of data.
Revised: This suggests the Mixupvicinal distribution fails to improve the generalization given a limited number of data.

Original: In contrast,the vicinal distribution obtained by the lazy random walk algorithm is a more accurate estimate forthe true distribution, as evidenced by the fact that the performance of ADV-CE-DD and ADV-CEis better than CE and CE-Mixup.
Revised: In contrast,the vicinal distribution obtained by the lazy random walk algorithm is a more accurate estimate forthe true distribution, as evidenced by the fact that the performance of ADV-CE-DD and ADV-CEis better than CE and CE-Mixup.

Original: To be concrete, the performance boost of ADV-CE-DD over CEevidences the mean of vicinal distribution, which does not located on the original sample, but on theweighted mean of the data passing through by the lazy random walks, is closer to the ground-truthdistribution.
Revised: To be concrete, the performance boost of ADV-CE-DD over CEevidences the mean of vicinal distribution, which does not located on the original sample, but on theweighted mean of the data passing through by the lazy random walks, is closer to the ground-truthdistribution.

Original: The performance improvement of ADV-CE over ADV-CE-DD demonstrates that thevariance of the vicinal distribution helps to alleviate the overfitting problem.
Revised: The performance improvement of ADV-CE over ADV-CE-DD demonstrates that thevariance of the vicinal distribution helps to alleviate the overfitting problem.

Original: As shown in Table 3, the performance of Kernel SVM is worse than SVM in most cases, which maybe due to the linear separability of the features extracted by WRN, while kernel does not help toimprove the separability.
Revised: As shown in Table 3, the performance of Kernel SVM is worse than SVM in most cases, which maybe due to the linear separability of the features extracted by WRN, while kernel does not help toimprove the separability.

Original: Though the performance of Kernel SVM is inferior to SVM, we observeperformance boost of ADV-SVM, which is built on Kernel SVM.
Revised: Though the performance of Kernel SVM is inferior to SVM, we observeperformance boost of ADV-SVM, which is built on Kernel SVM.

Original: This indicates that learning from thevicinal distributions that are close to the true distribution would sufficiently improve the performance.
Revised: This indicates that learning from thevicinal distributions that are close to the true distribution would sufficiently improve the performance.

Original: We also observe the performance gain of ADV-SVM over ADV-SVM-DD, again evidencing theimportance of the variance of vicinal distributions in alleviating the overfitting.
Revised: We also observe the performance gain of ADV-SVM over ADV-SVM-DD, again evidencing theimportance of the variance of vicinal distributions in alleviating the overfitting.

Original:  Conclusion and Limitations
Revised:  Conclusion and Limitations

Original: In this paper, we discuss the overfitting issue in few-shot learning, which is caused by the deviationof empirical distribution from the ground-truth distribution.
Revised: In this paper, we discuss the overfitting issue in few-shot learning, which is caused by the deviationof empirical distribution from the ground-truth distribution.

Original: To alleviate the overfitting, we revisit the Vicinal Risk Minimization principle in deep few-shot learning and propose the harness unlabeleddata and a lazy random walk algorithm to generate apdative statistical parameters for the vicinaldistribution of each training sample.
Revised: To alleviate the overfitting, we revisitthe Vicinal Risk Minimization principle in deep few-shot learning and propose to harness unlabeleddata and a lazy random walk algorithm to generate adaptive statistical parameters for the vicinaldistribution of each training sample.

Original: Following  VRM principle, we learn the model by optimizingthe resulting expected vicinal loss.
Revised: Following the VRM principle, we learn the model by optimizingthe resulting expected vicinal loss.

Original: We derive the vicinal loss for lineare classifier trained with crossentropy loss and for SVM classifier and develop the resulting Adaptive Vicinal classifiers as ADV-CEand ADV-SVM.
Revised: We derive the vicinal loss for linear classifiers trained with crossentropy loss and for SVM classifier and develop the resulting Adaptive Vicinal classifiers as ADV-CEand ADV-SVM.

Original: We evaluate the performance of ADV-CE and ADV-SVM on three benchmarks andvalidate the effectiveness of ADV in mitigating overfitting and improving generalization of classifierslearned from a limited number of samples.
Revised: We evaluate the performance of ADV-CE and ADV-SVM on three benchmarks andvalidate the effectiveness of ADV in mitigating overfitting and improving generalization of classifierslearned from a limited number of samples.

Original: Currently, our proposed ADC  only compatible with CEand SVM.
Revised: Currently, our proposed ADC is only compatible with CE and SVM.

Original: We would extend it to other loss functions.
Revised: We would extend it to other loss functions.

Original: For the vicinal Cross-Entropy loss, we optimizean approximation obtained by second-order Taylor expansion.
Revised: For the vicinal Cross-Entropy loss, weoptimize an approximation obtained by second-order Taylor expansion.

Original: However, the approximation error canbe large.
Revised: However, the approximationerror can be large.

Original: Thus, the performance of ADV-CE may be suboptimal and we can further improve it byderiving a better approximation for the vicinal Cross-Entropy loss.
Revised: Thus, the performance of ADV-CE may be suboptimal and we can further improveit by deriving a better approximation for the vicinal Cross-Entropy loss.

Original: Checklist
Revised: Checklist

Original: The checklist follows the references.
Revised: The checklist follows the references.

Original: Please read the checklist guidelines carefully for information onhow to answer these questions.
Revised: Please read the checklist guidelines carefully for information onhow to answer these questions.

Original: For each question, change the default [TODO] to [Yes] , [No] or[N/A] .
Revised: For each question, change the default [TODO] to [Yes] , [No] or[N/A] .

Original: You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
Revised: You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.

Original: For example:
Revised: For example:

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [Yes] See Section ?? .
Revised: [Yes] See Section ?? .

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [No] The code and the data areproprietary.
Revised: [No] The code and the data areproprietary.

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [N/A]
Revised: [N/A]

Original: Please do not modify the questions and only use the provided macros for your answers.
Revised: Please do not modify the questions and only use the provided macros for your answers.

Original: Note that the
Revised: Note that the

Original: Checklist section does not count towards the page limit.
Revised: Checklist section does not count towards the page limit.

Original: In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
Revised: In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s

Original: contributions and scope?
Revised: contributions and scope?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes] .
Revised: [Yes] .

Original: See Section 5(c) Did you discuss any potential negative societal impacts of your work?
Revised: See Section (c) Did you discuss any potential negative societal impacts of your work?

Original: [N/A]
Revised: [N/A]

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?
Revised: them?

Original: [Yes]
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [Yes]3.
Revised: [Yes]3.

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?
Revised: mental results (either in the supplemental material or as a URL)?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?

Original: [Yes](c)
Revised: [Yes]

Original:  Did you report error bars (e
Revised: (c) Did you report error bars (e

Original: g., with respect to the random seed after running experiments multiple times)?
Revised: g., with respect to the random seed after running experiments multiple times)?

Original: [Yes]
Revised: [Yes]

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?

Original: [No]
Revised: [No]

Original: If you are using existing assets (e
Revised: If you are using existing assets (e

Original: g., code, data, models) or curating/releasing new assets...
Revised: g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A]
Revised: [N/A]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?
Revised: using/curating?

Original: [N/A]
Revised: [N/A]

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Original: information or offensive content?
Revised: information or offensive content?

Original: [N/A]
Revised: [N/A]

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: applicable?
Revised: applicable?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: spent on participant compensation?
Revised: spent on participant compensation?

Original: [N/A]
Revised: [N/A]