Original: Improving Task-Specific Generalization in Few-Shot Learning via Adaptive Vicinal Risk Minimization
Revised: Improving Task-Specific Generalization in Few-Shot Learning via Adaptive Vicinal Risk Minimization

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AffiliationAddress email
Revised: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Recent years have witnessed the rapid development of meta-learning in improvingthe meta generalization over tasks in few-shot learning.
Revised: Recent years have witnessed the rapid development of meta-learning in improvingthe meta generalization over tasks in few-shot learning.

Original: However, the task-specificlevel generalization is overlooked in most algorithms.
Revised: However, the task-specificlevel generalization is overlooked in most algorithms.

Original: For a novel few-shot learningtask where the empirical distribution likely deviates from the true distribution, themodel obtained via minimizing the empirical loss can hardly generalize to unseendata.
Revised: For a novel few-shot learningtask where the empirical distribution likely deviates from the true distribution, themodel obtained via minimizing the empirical loss can hardly generalize to unseendata.

Original: A viable solution to improving the generalization comes as a more accurateapproximation of the true distribution; that is, admitting a Gaussian-like vicinaldistribution for each of the limited training samples.
Revised: A viable solution to improving the generalization comes as a more accurateapproximation of the true distribution; that is, admitting a Gaussian-like vicinaldistribution for each of the limited training samples.

Original: Thereupon we derive theresulting vicinal loss function over vicinities of all training samples and minimizeit instead of the conventional empirical loss over training samples only, favorablyfree from the exhaustive sampling of all vicinal samples.
Revised: Thereupon we derive theresulting vicinal loss function over vicinities of all training samples and minimizeit instead of the conventional empirical loss over training samples only, favorablyfree from the exhaustive sampling of all vicinal samples.

Original: It remains challengingto obtain the statistical parameters of the vicinal distribution for each sample.
Revised: It remains challengingto obtain the statistical parameters of the vicinal distribution for each sample.

Original: Totackle this challenge, we further propose to estimate the statistical parameters asthe weighted mean and variance of a set of unlabeled data it passed by a randomwalk starting from training samples.
Revised: Totackle this challenge, we further propose to estimate the statistical parameters asthe weighted mean and variance of a set of unlabeled data it passed by a randomwalk starting from training samples.

Original: To verify the performance of the proposedmethod, we conduct experiments on four standard few-shot learning benchmarksand consolidate the superiority of the proposed method over state-of-the-art fewshot learning baselines.
Revised: To verify the performance of the proposedmethod, we conduct experiments on four standard few-shot learning benchmarksand consolidate the superiority of the proposed method over state-of-the-art fewshot learning baselines.

Original:  Introduction
Revised:  Introduction

Original: In recent years, deep learning algorithms have made tremendous progress and have been widelyapplied in vision [14], language [31] and speech [8] areas.
Revised: In recent years, deep learning algorithms have made tremendous progress and have been widelyapplied in vision [14], language [31] and speech [8] areas.

Original: Despite the prevalence, deep learningalgorithms rely heavily on a sufficient amount of labeled data and often fail when only a limitednumber of labeled samples are available.
Revised: Despite the prevalence, deep learningalgorithms rely heavily on a sufficient amount of labeled data and often fail when only a limitednumber of labeled samples are available.

Original: Considering the high cost of extensive data collection andannotation, it has attracted increasing attention to develop algorithms learning from a limited numberof training samples, which has been widely known as few-shot learning [33, 4].
Revised: Considering the high cost of extensive data collection andannotation, it has attracted increasing attention to develop algorithms learning from a limited numberof training samples, which has been widely known as few-shot learning [33, 4].

Original: The recent focus of few-shot learning is built on the meta-learning framework [6, 33, 27].
Revised: The recent focus of few-shot learning is built on the meta-learning framework [6, 33, 27].

Original: The goalof meta-learning is to train a model that can be quickly adapted to a new task with a small number oflabeled samples by leveraging the knowledge from previous related tasks.
Revised: The goalof meta-learning is to train a model that can be quickly adapted to a new task with a small number oflabeled samples by leveraging the knowledge from previous related tasks.

Original: Typically, a meta-learningmodel consists of the meta-training stage where a well-generalized feature extractor is learned frommeta-training tasks and the meta-testing stage during which a classifier is built on top of the featureextractor for novel tasks.
Revised: Typically, a meta-learningmodel consists of the meta-training stage where a well-generalized feature extractor is learned frommeta-training tasks and the meta-testing stage during which a classifier is built on top of the featureextractor for novel tasks.

Original: While the majority of meta-learning works revolved around developingbetter meta-training strategies to reduce the meta generalization error over all tasks, how to fullyleverage the meta-trained feature extractor to build a robust classifier and improve a task-specificgeneralization in the meta-testing stage remains less explored.
Revised: While the majority of meta-learning works revolved around developingbetter meta-training strategies to reduce the meta generalization error over all tasks, how to fullyleverage the meta-trained feature extractor to build a robust classifier and improve a task-specificgeneralization in the meta-testing stage remains less explored.

Original: The challenge to building such a classifier lies in the significant deviation of the empirical distribution(i.e.
Revised: The challenge to building such a classifier lies in the significant deviation of the empirical distribution(i.e.

Original: the distribution formed by the training data) from the population distribution (i.e. the true
Revised: the distribution formed by the training data) from the population distribution (i.e. the true

Original: [Figure]
Revised: [Figure]

Original: distribution).
Revised: distribution).

Original: As illustrated in Figure 1, a model well fitting the training data (support set) maygeneralize poorly on the testing data (query set) from the true distribution due to the bias in theempirical distribution.
Revised: As illustrated in Figure 1, a model well fitting the training data (support set) maygeneralize poorly on the testing data (query set) from the true distribution due to the bias in theempirical distribution.

Original: One viable solution is to approximate the true distribution more accurately –by viewing the empirical distribution as the assembling of delta functions located on each trainingsample, Chapelle et al.
Revised: One viable solution is to approximate the true distribution more accurately –by viewing the empirical distribution as the assembling of delta functions located on each trainingsample, Chapelle et al.

Original: [3] proposed the Vicinal Risk Minimization loss function where the deltafunctions are replaced by some density estimate in the vicinity of each training data.
Revised: [3] proposed the Vicinal Risk Minimization loss function where the deltafunctions are replaced by some density estimate in the vicinity of each training data.

Original: In the eraof deep learning, the VRM learning principle is implemented as Mixup [41, 32], which is simpleyet effective given a sufficiently large number of training data.
Revised: In the eraof deep learning, the VRM learning principle is implemented as Mixup [41, 32], which is simpleyet effective given a sufficiently large number of training data.

Original: However, as we have verified in
Revised: However, as we have verified in

Original: Table 3, the performance improvement by applying Mixup on the support set is limited in few-shotlearning where the vicinal distributions assembled by Mixup is still significantly biased.
Revised: Table 3, the performance improvement by applying Mixup on the support set is limited in few-shotlearning where the vicinal distributions assembled by Mixup is still significantly biased.

Original: To improvethe generalization of a few-shot classifier during meta-testing, we are motivated to revisit the VRMprinciple to 1) generate adaptive vicinal distribution for each sample in support set and 2) minimizethe expected vicinal loss over the vicinal distributions.
Revised: To improvethe generalization of a few-shot classifier during meta-testing, we are motivated to revisit the VRMprinciple to 1) generate adaptive vicinal distribution for each sample in support set and 2) minimizethe expected vicinal loss over the vicinal distributions.

Original: However, both are challenging.
Revised: However, both are challenging.

Original: Without loss of generality, we assume that the vicinity of each training point admits a Gaussiandistribution.
Revised: Without loss of generality, we assume that the vicinity of each training point admits a Gaussiandistribution.

Original: In [3], the means of the vicinal distributions are located on the original training points.
Revised: In [3], the means of the vicinal distributions are located on the original training points.

Original: As illustrated in Figure 1(b), however, setting the mean in this way is not sufficient to approximatethe true distribution, as an original training point likely deviates from the mean of the ground-truthdistribution of its corresponding class.
Revised: As illustrated in Figure 1(b), however, setting the mean in this way is not sufficient to approximatethe true distribution, as an original training point likely deviates from the mean of the ground-truthdistribution of its corresponding class.

Original: In fact, it is difficult to estimate the mean and variance basedonly on the labeled training samples.
Revised: In fact, it is difficult to estimate the mean and variance basedonly on the labeled training samples.

Original: In this paper, we assume that unlabeled samples are alsoavailable, and leverage these unlabeled samples to estimate the mean and variance of the vicinaldistribution for each labeled sample.
Revised: In this paper, we assume that unlabeled samples are alsoavailable, and leverage these unlabeled samples to estimate the mean and variance of the vicinaldistribution for each labeled sample.

Original: Concretely, inspired by [42], we first construct a transitionprobability matrix based on pairwise distances between features of all samples, and secondly performthe lazy random walk among unlabeled samples starting from each labeled sample.
Revised: Concretely, inspired by [42], we first construct a transitionprobability matrix based on pairwise distances between features of all samples, and secondly performthe lazy random walk among unlabeled samples starting from each labeled sample.

Original: After walkinga certain number of steps, we obtain the visit probabilities of all the samples and infer the vicinaldistribution whose mean and variance are the weighted mean and variance of all samples visited.
Revised: After walkinga certain number of steps, we obtain the visit probabilities of all the samples and infer the vicinaldistribution whose mean and variance are the weighted mean and variance of all samples visited.

Original: All that remains is the loss function over the vicinal distributions of all training samples.
Revised: All that remains is the loss function over the vicinal distributions of all training samples.

Original: An intuitiveway to do this is to exploit the Monte Carlo method, which repeats random sampling to explicitlygenerate data points from each vicinal distribution and minimizes the loss on all the sampled datapoints.
Revised: An intuitiveway to do this is to exploit the Monte Carlo method, which repeats random sampling to explicitlygenerate data points from each vicinal distribution and minimizes the loss on all the sampled datapoints.

Original: Unfortunately, it requires to draw a large amount of samples to minimize the Monte Carloapproximation error, which significantly increases the computation cost.
Revised: Unfortunately, it requires to draw a large amount of samples to minimize the Monte Carloapproximation error, which significantly increases the computation cost.

Original: Instead, we follow [3] toderive a closed form of the expected vicinal loss over the vicinal distributions.
Revised: Instead, we follow [3] toderive a closed form of the expected vicinal loss over the vicinal distributions.

Original: In this paper, we focuson building an optimization-based classifier in optimization-based meta-learning algorithms [6, 16],which has been proved with much more adaptability and generalization capacity in novel meta-testingtasks than a nearest-neighbour classifier in metric-based meta-learning algorithms [33, 27, 28, 21].
Revised: In this paper, we focuson building an optimization-based classifier in optimization-based meta-learning algorithms [6, 16],which has been proved with much more adaptability and generalization capacity in novel meta-testingtasks than a nearest-neighbour classifier in metric-based meta-learning algorithms [33, 27, 28, 21].

Original: Particularly, we consider both the cross-entropy loss and the SVM loss functions which are used asclassifier in the state-of-the-art meta-learning algorithms [37, 16].
Revised: Particularly, we consider both the cross-entropy loss and the SVM loss functions which are used asclassifier in the state-of-the-art meta-learning algorithms [37, 16].

Original: For the vicinal version of crossentropy loss, it cannot be computed in closed form.
Revised: For the vicinal version of crossentropy loss, it cannot be computed in closed form.

Original: We instead derive its quadratic approximation.
Revised: We instead derive its quadratic approximation.

Original: For SVM, we derive its closed-form vicinal loss.
Revised: For SVM, we derive its closed-form vicinal loss.

Original: In summary, our contributions are threefold:
Revised: In summary, our contributions are threefold:

Original: • We revisit vicinal risk minimization (VRM) in deep few-shot learning and derive the resultingexpected vicinal loss functions for both the cross-entropy loss and SVM.•
Revised: • We revisit vicinal risk minimization (VRM) in deep few-shot learning and derive the resultingexpected vicinal loss functions for both the cross-entropy loss and SVM.•

Original: We propose to harness unlabeled data and a lazy random walk algorithm to generate adaptivestatistical parameters for the vicinal distribution of each training sample.
Revised: We propose to harness unlabeled data and a lazy random walk algorithm to generate adaptivestatistical parameters for the vicinal distribution of each training sample.

Original: • We have validated the superiority of the proposed algorithm with extensive experiments on threebenchmarks,
Revised: • We have validated the superiority of the proposed algorithm with extensive experiments on threebenchmarks,

Original: i
Revised: i

Original: e., miniImageNet, CUB, CIFAR-FS.
Revised: e., miniImageNet, CUB, CIFAR-FS.

Original:  Related Works2.1 Few-Shot Learning and Meta-Learning
Revised:  Related Works2.1 Few-Shot Learning and Meta-Learning

Original: Few-shot learning aims to recognize novel classes with insufficient labeled instances.
Revised: Few-shot learning aims to recognize novel classes with insufficient labeled instances.

Original: The recentfocus of few-shot learning algorithms has been on the meta-learning framework, which trains themodel with multiple meta-episodes to enable quick learning and applies the model to improvethe learning in novel tasks.
Revised: The recentfocus of few-shot learning algorithms has been on the meta-learning framework, which trainsthe model with multiple episodes to enable quick learning and applies the model to improve thelearning in novel tasks.

Original: The meta-learning methods fall into two  categories: metric-basedmethods and optimization-based methods.
Revised: The meta-learning methods fall into two major categories: metric-basedmethods and optimization-based methods.

Original: Metric-based methods [13, 33, 27, 28, 20, 39] learn ametric to compare two samples’ feature embeddings encoded by the feature extractor and classify asample in the query set based on its distances to the samples in the support set.
Revised: Metric-based methods [13, 33, 27, 28, 20, 39] learn ametric to compare two samples’ feature embeddings encoded by the feature extractor and classify asample in the query set based on its distances to the samples in the support set.

Original: Optimization-basedapproaches [6, 23, 25, 29, 35] learn a good initialization such that it can be fast adapted to novel tasksbased on a few instances.
Revised: Optimization-basedapproaches [6, 23, 25, 29, 35] learn a good initialization such that it can be fast adapted to novel tasksbased on a few instances.

Original: To tackle the data deficiency issue in few-shot learning, researchers have explored the idea ofexploiting unlabeled data and developed transductive learning or semi-supervised learning techniquesfor few-shot learning, either by leveraging nearest neighbor graph for propagating labels [36, 12, 18]or by leveraging predicted soft or hard labels on unlabeled samples to update the class prototypesor metric [24, 10, 15, 22, 39].
Revised: To tackle the data deficiency issue in few-shot learning, researchers have explored the idea ofexploiting unlabeled data and developed transductive learning or semi-supervised learning techniquesfor few-shot learning, either by leveraging nearest neighbor graph for propagating labels [36, 12, 18]or by leveraging predicted soft or hard labels on unlabeled samples to update the class prototypesor metric [24, 10, 15, 22, 39].

Original: Our method is not built on  metric-based method.
Revised: Our method is not built on a metric-based method.

Original: We make useof the unlabeled data to derive the vicinal distribution for each support sample and optimize theconsequential vicinal loss to learn a classifier.
Revised: We make useof the unlabeled data to derive the vicinal distribution for each support sample and optimize theconsequential vicinal loss to learn a classifier.

Original: Our approach shares a similar spirit of [37].
Revised: Our approach shares a similar spirit of [37].

Original: It [37]proposes to calibrate the distribution of the support samples by transferring statistics from the baseclasses and train the classifier on a set of samples sampled from the calibrated distributions.
Revised: Yang et al.[37] proposed to calibrate the distribution of the support samples by transferring statistics from thebase classes and trained the classifier on a set of samples sampled from the calibrated distributions.

Original: Unlike[37], our method obtains more accurate vicinal distributions by making use of the unlabeled datafrom the same task.
Revised: Unlike [37], our method obtains more accurate vicinal distributions by making use of the unlabeleddata from the same task.

Original: Besides, instead of sampling extensive data from the distributions, our modeldirectly optimizes the expected vicinal loss over the distributions, resulting in more efficient trainingand higher performance.
Revised: Besides, instead of sampling extensive data from the distributions, ourmodel directly optimizes the expected vicinal loss over the distributions, resulting in more efficienttraining and higher performance.

Original: 2.2 Vicinal Risk Minimization
Revised: 2.2 Vicinal Risk Minimization

Original: The vicinal Risk Minimization  principle is introduced in [3, 30].
Revised: The Vicinal Risk Minimization (VRM) principle is introduced in [3, 30].

Original: It proposes to assume each trainingsample admits a vicinal distribution and to optimize the expected loss over the vicinal distributions.
Revised: It assumes each trainingsample admits a vicinal distribution and optimizes the expected loss over the vicinal distributions.

Original: In [3], only binary VRM classifier are presented.
Revised: In[3], only binary VRM classifiers are considered.

Original: Our method considers multi-class classifier andcross-entropy loss, which is more challenging to derive the expected vicinal loss.
Revised: Our method considers multi-class classifiers and thecross-entropy loss, which is more challenging to derive the expected vicinal loss.

Original: Besides, Chapelleet al.
Revised: Besides, Chapelleet al.

Original: [3] consider Gaussian vicinities whose means lie on the  original training samples. Our approachalso considers Gaussian vicinities but estimates the mean by the unlabeled data, which is generallynot on the training samples.
Revised: [3] consider Gaussian vicinities but only estimate the variances, keeping the mean the same asthe original training sample. Our approach further estimates the mean with the unlabeled data.

Original: In the era of deep learning, the most popularVRM technique in deep learning algorithms is Mixup [41,32].
Revised: In the era of deep learning, the most popular technique under the VRM principle is Mixup [41, 32].

Original: Mixup constructs the vicinal distributions as the convex combination of the training data, whichcan be understood as a data augmentation technique that encourages the model to behave linearlyinter training examples.
Revised: Mixup constructs the vicinal distributions as the convex combination of the training data, whichcan be understood as a data augmentation technique that encourages the model to behave linearlyin-between training examples.

Original: It has been demonstrated that Mixup is simple yet efficient and improves thegeneralization of deep model given sufficiently many training samples.
Revised: It has been demonstrated that Mixup is simple yet efficient andimproves the generalization of deep models given sufficiently many training samples.

Original: Mixup is also introduced onthe meta-learning algorithm in [38, 7], but is applied on the query set for meta model optimization.
Revised: Mixup is alsointroduced on the meta-learning algorithm in [38, 7], but is applied on the query set for meta-modeloptimization.

Original: As shown by the experiment (Table 6) in [38], applying Mixup on the support set could even hurtthe performance.
Revised: The experimental results (Table 6) in [38] show that applying Mixup on the support setcould even hurt the performance.

Original: Our approach instead leverages the unlabeled data for a more accurate vicinaldistribution estimate  for an extremely small support set.
Revised: Our method instead improves the performance by producing anaccurate estimate of the vicinal distribution for the small support set using unlabeled data.

Original:  Adaptive Vicinal Few-Shot Learning
Revised:  Adaptive Vicinal Few-Shot Learning

Original: In this section, we will introduce the proposed algorithm, AD aptive V icinal Few-Shot Learning(ADV).
Revised: In this section, we will introduce the proposed algorithm, AD aptive V icinal Few-Shot Learning(ADV).

Original: We will first introduce the VRM; then derive the expected vicinal loss for a linear classifierwith cross-entropy loss and for SVM; and finally propose the algorithm based on lazy random walksto obtain the adaptive vicinal distribution for each labeled support sample.
Revised: We will first introduce the VRM, then derive the expected vicinal loss for a linear classifiertrain with cross-entropy loss and for SVM, respectively, and finally propose the algorithm based onlazy random walks to obtain the adaptive vicinal distribution for each labeled support sample.

Revised: Wesummarize the proposed method in Algorithm 1.

Revised: Note that our method focuses on improving thetask-specific generalization in meta-testing set and is compatible with feature extractor trained by anymeta-learning algorithm.

Original: 3.1 Vicinal Risk Minimization
Revised: 3.1 Vicinal Risk Minimization

Original: Given data x, y following the distribution P ( x, y ) , the loss function L , we would like the optimize amodel f θ ( x ) by minimizing the expected risk
Revised: Given data x, y following the distribution P ( x, y ) , the loss function L , we would like to optimize amodel f θ ( x ) by minimizing the expected risk

Original: [Equation]
Revised: [Equation]

Original: Here, x is the input data, y is its label and θ is the model parameters.
Revised: Here, x is the input data, y is its label and θ is the model parameters.

Original: In practice, P ( x, y ) is unknownand a set of training data { x i , y i } Ni =1 is available, where N is the size of training data.
Revised: In practice, P ( x, y ) is unknownand a set of training data { x i , y i } Ni =1 is available, where N is the size of training data.

Original: To learn amodel, one can minimize the empirical risk, which is equivalent to minimizing the expected loss withrespect to an empirical density estimate d P δ ( x, y )
Revised: To learn amodel, one can minimize the empirical risk, which is equivalent to minimizing the expected loss withrespect to an empirical density estimate d P δ ( x, y )

Original: = 1 N (cid:80) Ni
Revised: = 1 N (cid:80) Ni

Original: δ x i ( x ) δ y
Revised: δ x i ( x ) δ y

Original: i
Revised: i

Original: ( y ) , where δ a ( · ) is a Diracdelta function centered at a .
Revised: ( y ) , where δ a ( · ) is a Diracdelta function centered at a .

Original: Given a limited number of training sample, the empirical density estimate d P δ ( x, y ) could bias fromthe true density d P ( x, y ) .
Revised: Given a limited number of training sample, the empirical density estimate d P δ ( x, y ) could bias fromthe true density d P ( x, y ) .

Original: For a more accurate density estimate, Chapelle et al.
Revised: For a more accurate density estimate, Chapelle et al.

Original: [3] proposed toreplace the delta functions δ x i ( x ) by some estimate of density in the vicinity of x i .
Revised: [3] proposed toreplace the delta functions δ x i ( x ) by some estimate of density in the vicinity of x i .

Original: Given a vicinaldistribution of x i as P x i ( x ) , the improved density estimate is P
Revised: Given a vicinaldistribution of x i as P x i ( x ) , the improved density estimate is P

Original: ν
Revised: ν

Original: ( x, y ) = 1 N (cid:80) Ni =
Revised: ( x, y ) = 1 N (cid:80) Ni =

Original: 1 P x i ( x ) δ y i ( y ) .
Revised: 1 P x i ( x ) δ y i ( y ) .

Original: And the corresponding vicinal risk is defined as
Revised: And the corresponding vicinal risk is defined as

Original: [Equation]
Revised: [Equation]

Original: 3.2 Vicinal Loss
Revised: 3.2 Vicinal Loss

Original: In this paper, we assume the feature extractor is trained and fixed for novel tasks.
Revised: In this paper, we assume the feature extractor is trained and fixed for novel tasks.

Original: Each tasks is
Revised: Each task is

Original: C -way K -shot, consisting of N = CK data support samples { ( x i , y i ) }
Revised: C -way K -shot, consisting of N = CK data support samples { ( x i , y i ) }

Original: Ni =1 .
Revised: Ni =1 .

Original: To perform VRM, weassume the feature
Revised: To perform VRM, weassume the feature

Original: z i of each sample x i in the support set admits a Gaussian vicinal distribution
Revised: z i of each sample x i in the support set admits a Gaussian vicinal distribution

Original: N ( µ i , Σ i ) such that a sample ˜ z is drawn from the vicinal distribution with probability
Revised: N ( µ i , Σ i ) such that a sample ˜ z is drawn from the vicinal distribution with probability

Original: P z
Revised: P z

Original: i
Revised: i

Original: (˜ z )
Revised: (˜ z )

Original: =det(2 π Σ i ) − 12 exp( − 12 (˜ z − µ i )
Revised: =det(2 π Σ i ) − 12 exp( − 12 (˜ z − µ i )

Original: ⊤ Σ − 1 i (˜ z − µ i )) , where µ i and Σ
Revised: ⊤ Σ − 1 i (˜ z − µ i )) , where µ i and Σ

Original: i is the mean and variance of thevicinal distribution of z i .
Revised: i is the mean and variance of thevicinal distribution of z i .

Original: 3.2.1 Vicinal Cross-Entropy Loss
Revised: 3.2.1 Vicinal Cross-Entropy Loss

Original: We first focus on linear classifier with Cross-Entropy loss.
Revised: We first focus on linear classifier with Cross-Entropy loss.

Original: A linear classifier f θ ( z ) =
Revised: A linear classifier f θ ( z ) =

Original: ( W ⊤ z + b )maps a feature z ∈ R d to its logits, where W ∈ R d × C and b ∈ R C is the parameter of the classifier.
Revised: ( W ⊤ z + b )maps a feature z ∈ R d to its logits, where W ∈ R d × C and b ∈ R C is the parameter of the classifier.

Original: The cross-entropy loss is defined as ℓ ( h, y )
Revised: The cross-entropy loss is defined as ℓ ( h, y )

Original: = − e ⊤ y log( softmax ( f θ ( z ))) , where y ∈ 1 , 2 , ..., C is thelabel; e y is a one-hot vector with the y -th element being 1 .
Revised: = − e ⊤ y log( softmax ( f θ ( z ))) , where y ∈ 1 , 2 , ..., C is thelabel; e y is a one-hot vector with the y -th element being 1 .

Original: The expected loss of cross entropy over
Revised: The expected loss of cross entropy over

Original: Gaussian vicinal distribution of x is defined as
Revised: Gaussian vicinal distribution of x is defined as

Original: [Equation]
Revised: [Equation]

Original: Due to the softmax function, the expected loss (1) cannot be computed in closed form.
Revised: Due to the softmax function, the expected loss (1) cannot be computed in closed form.

Original: Instead, weutilize Taylor expansion to derive an approximation of (1).
Revised: Instead, weutilize Taylor expansion to derive an approximation of Eqn.(1).

Original: By taking  second-order Taylor expansionof ℓ ( f θ (˜ z ) , y ) around the mean of its vicinal distribution  µ , we obtain
Revised: By taking the second-order Taylorexpansion of ℓ ( f θ (˜ z ) , y ) around the mean of the vicinal distribution z i , i.e. µ , we obtain

Original: [Equation]
Revised: [Equation]

Original: (2) where G is the gradient and H is the Hessian w.r.t.
Revised: where G is the gradient and H is the Hessian w.r.t.

Original: input µ .
Revised: input µ .

Original: Note that H
Revised: Note that H

Original: = W ( diag ( h ) −
Revised: = W ( diag ( h ) −

Original: hh ⊤ ) W ⊤where h = softmax ( f θ ( µ )) and diag ( h ) is a diagonal matrix whose diagonal elements are equal to h[26].
Revised: hh ⊤ ) W ⊤where h = softmax ( f θ ( µ )) and diag ( h ) is a diagonal matrix whose diagonal elements are equal to h[26].

Original: By taking  expectation over ˜ z , we have
Revised: By taking the expectation over ˜ z , we have

Original: [Equation]
Revised: [Equation]

Original: where Σ is the variance of ˜ z , h k is the k -th elements of h ,and W k is the k -th column of W . Plugging(2), (3) and (4) into (1), we obtain the approximated expected vicinal cross-entropy loss for the linearclassifier as
Revised: where Σ is the variance of ˜ z , h k is the k -th elements of h , W k is the k -th column of W . We define R θ (Σ) as the RHS of Eqn. Plugging (2), (3) and (4) into (1), we obtain the approximated expectedvicinal cross-entropy loss for the linear classifier as

Original: [Equation]
Revised: [Equation]

Original: 3.2.2 Vicinal SVM
Revised: 3.2.2 Vicinal SVM

Original: SVM is used as classifier for meta learning algorithms in [16, 37].
Revised: SVM is used as classifier for meta learning algorithms in [16, 37].

Original: We follow [16] to use the Crammerand Singer formulation of multi-class SVM [5].
Revised: We follow [16] to use the Crammerand Singer formulation of multi-class SVM [5].

Original: In [16, 5], the dual formulation of the objective for alinear multi-class SVM is obtained as:
Revised: In [16, 5], the dual formulation of the objective for alinear multi-class SVM is obtained as:

Original: [Equation]
Revised: [Equation]

Original: where α ki is the dual variable and λ is a regularization parameter.
Revised: where α ki is the dual variable and λ is a regularization parameter.

Original: The linear classifier for the k -thclass is f kθ ( z ) =
Revised: The linear classifier for thek -th class is f kθ ( z ) =

Original: w ⊤ k z , where w k = (cid:80) i
Revised: w ⊤ k z , where w k = (cid:80) i

Original: α ki
Revised: α ki

Original: z i .
Revised: z i .

Original: Note that here z is an unseen data and does notassume to admita vicinal distribution.
Revised: Note that here z is an unseen data and weassume only labeled data admit vicinal distributions while unseen data does not admit vicinaldistributions.

Original: If each z i admits a vicinal distribution N ( µ i , Σ i ) , the vicinalexpectation for the term z ⊤ i
Revised: If each z i admits a vicinal distribution N ( µ i , Σ i ) , the vicinal expectation for theterm z ⊤ i

Original: z j in (6) becomes
Revised: z j in (6) becomes

Original: E 
Revised: E ˜

Original: [Equation]
Revised: z i ∼N ( µ i , Σ i ) E ˜ z j ∼N ( µ j , Σ j ) [˜ z ⊤ i ˜

Original:  and the classifier becomes  E ˜ z
Revised: z j ] = µ ⊤ i µ j , and the classifier becomes 173 E ˜ z

Original: i ∼N ( µ i , Σ i ) [ f kθ (˜ z i )] =
Revised: i ∼N ( µ i , Σ i ) [ f kθ (˜ z i )] =

Original: ( (cid:80) i
Revised: ( (cid:80) i

Original: α ki µ i )
Revised: α ki µ i )

Original: ⊤ z.
Revised: ⊤ z.

Original: If µ i =
Revised: If µ i =

Original: z i , the objective ofthevicinal SVM coincides with the original one.
Revised: z i , the objective of the vicinal SVM reduces to theoriginal SVM objective in Eqn.

Original: Even when µ i ̸
Revised: Even when µ i ̸

Original: =
Revised: =

Original: z i , it still does not take the varianceof the vicinal distribution into consideration and may result in suboptimal performance.
Revised: z i , the objective of the vicinal SVM stilldoes not take the variance of the vicinal distribution into consideration and may result in suboptimalperformance.

Original: To exploitthe variance information to improve the  SVM classifier, we consider nonlinear classifier, i.e. thekernel-based SVM.
Revised: To exploit the variance information to improve the vicinal SVM classifier, we considera nonlinear classifier, i.e. the kernel-based SVM.

Original: Multi-class kernel-based SVM uses a kernel K ( · , · ) and admits a similar dual formulation to (6).
Revised: Multi-class kernel-based SVM uses a kernel K ( · , · ) .

Original: The difference lays in that it replaces the inner product of z i and z j in (6) by K ( z i , z j ) and replacethe inner product of z
Revised: It admits a dual formulation similar to (6), butreplaces the inner product of z i and z j in (6) by K ( z i , z j ) .

Original: i and z in the classifier by K ( zi , z ) . For the vicinal multi-class kernel-based 178 SVM, these two terms are further replaced by a two-vicinal kernel M ( z i , z j ) and a one-vicinal kernel
Revised: And the classifier becomes f kθ ( z ) =(cid:80) i α ki K

Original: Q (z, z i  ) .
Revised: ( z i , z ) .

Original: In this paper, we consider the widely-used RBF kernel K ( z, z ′ ) =
Revised: In this paper, we consider the widely-used RBF kernel K ( z, z ′ ) =

Original: exp( − || z − z ′
Revised: exp( − || z − z ′

Original: || 2 2 σ) andconstruct the vicinal kernels for it as:
Revised: || 2 2 σ) ,where σ is a hyper-parameter. In vicinal multi-class kernel-based SVM, we define M ( z i , z j ) = E ˜ z

Original: [Equation]
Revised: i ∼N ( µ i , Σ i ) E ˜ z j ∼N ( µ j , Σ j ) K ( z i , z j )and Q ( z i , z ) =

Original:  Here, A ( σ 2 ) ∈{+
+}
Revised: E ˜ z i ∼N ( µ i , Σ i ) K ( z i , z ) and obtain their computation as [Equation] The detailed derivation is available in Appendix A. Here, A ( σ {+
+}

Original:  R d × d is a diagonal matrix with diagonal elements being σ 2 ; M i =
Revised: ) ∈ R d × d is a diagonal matrix withdiagonal elements being σ 2 ; M i =

Original: ( A ( σ 2 ) + Σ i ) − 1 ; M ij =
Revised: ( A ( σ 2 ) + Σ i ) − 1 ; M ij =

Original: ( A ( σ 2 )+Σ i +Σ j ) − 1 ; the operator ∥ z ∥ 2 M = z ⊤ Mz .
Revised: ( A ( σ 2 ) + Σ i + Σ j ) − 1 ; the operator∥ z ∥ 2 M = z ⊤ Mz .

Original: Note that for computation simplification,in vicinal SVM, we assume the variance Σ i is diagonal.
Revised: Note that for computation simplification, in vicinal SVM, we assume the varianceΣ i is diagonal.

Original: The derivation is available in Appendix.

Original: Now we achieve the vicinal objective for multi-class kernel-based SVM as:
Revised: Now we achieve the vicinal objective for multi-class kernel-based SVM as:

Original: [Equation]
Revised: [Equation]

Original: with the classifier being f kθ ( z ) =
Revised: with the classifier being f kθ ( z ) =

Original: (cid:80) i α ki Q
Revised: (cid:80) i α ki Q

Original: ( z i , z ) .
Revised: ( z i , z ) .

Original: The objective in (9) is a quadratic program (QP)over dual variables { α ki } and can be solved by using a differentiable GPU-based QP solver [1].
Revised: The objective in (9) is a quadratic program (QP)over dual variables { α ki } and can be solved by using a differentiable GPU-based QP solver [1].

Original: 3.3 Lazy Random Walk for Adaptive Vicinal Distribution
Revised: 3.3 Lazy Random Walk for Adaptive Vicinal Distribution

Original: As illustrated in Figure 1, the performance of the model learned by VRM has a heavy reliance onthe quality of the vicinal distribution  toapproximate the true distribution.
Revised: As illustrated in Figure 1, the performance of the model learned by VRM relies heavily on howclose the vicinal distribution is to the true distribution.

Original: However, it is almostimpossible to estimate accurate proximate distributions from a limited number of support samples,as they may deviate from the true distribution.
Revised: However, it is almost impossible to estimateaccurate proximate distributions from a small support set.

Original: Therefore, we propose to exploit extra data, i.e. theunlabeled data from the same task, to reduce the estimation error of the statistical parameters for thevicinal distribution and, thus, improve the performance of the model.
Revised: Therefore, we propose to exploit extra data,i.e., the unlabeled data from the same task, to reduce the estimation error of the statistical parametersfor the vicinal distribution and, thus, improve the performance of the model.

Original: We assume that a well-trained feature extractor on the meta-training tasks can preserve the similaritybetween the data in the feature space.
Revised: We assume that a well-trained feature extractor on the meta-training tasks can preserve the similaritybetween the data in the feature space.

Original: Data from the same class in the novel task would have similarfeatures with  high probability.
Revised: Data from the same class in the novel task would have similarfeatures with a high probability.

Original: Then, we apply  lazy random walks algorithm [42] to estimate theprobability that the unlabeled data would be visited starting from a support sample, and compute thestatistical parameters as the weighted mean and variance of the data visited by the lazy random walks.
Revised: Then, we apply the lazy random walk algorithm [42] to estimate theprobability that the unlabeled data will be visited starting from a support sample, and compute thestatistical parameters as the weighted mean and variance of the samples visited by the lazy randomwalk.

Original: To perform lazy random walk, we first compute the distance D SU between the support sample
Revised: To perform lazy random walk, we first compute the distance D SU between the support sample

Original: z i andthe unlabeled data { u j } N u j =1 and compute the distance D UU between the unlabeled data as:
Revised: z i andthe unlabeled data { u j } N u j =1 and compute the distance D UU between the unlabeled data as:

Original: [Equation]
Revised: [Equation]

Original: 2 , and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Secondly, we transform these distances into transition probabilities by softmaxing the distance matrixover columns with a temperature τ :
Revised: Secondly, we transform these distances into transition probabilities by softmaxing the distance matrixover columns with a temperature τ :

Original: [Equation]
Revised: [Equation]

Original: Note that we set D UUjj = ∞ such that P UUjj = 0 .
Revised: Note that we set D UUjj = ∞ such that P UUjj = 0 .

Original: Thirdly, we construct the transition probabilities matrix for z i in lazy random walk by combining
Revised: Thirdly, we construct the transition probabilities matrix for z i in lazy random walk by combining

Original: P SUi : and P UU and setting the lazy stay probability β ∈ (0 , 1) as:
Revised: P SUi : and P UU and setting the lazy stay probability β ∈ (0 , 1) as:

Original: [Equation]
Revised: [Equation]

Original: where  0 n c × 1 is a all zero vector, and I N u is an identity matrix.
Revised: where P SUi : is the i -th row of P SU , 0 n c × 1 is a all zero vector, and I N u is an identity matrix.

Original: We then compute theT step visit probability v i  for z i :
Revised: We then compute the visit probability v i at the T -th step for z i :

Original: [Equation]
Revised: [Equation]

Original: where e 1  is a one-hot vector with the first element being 1 .
Revised: where e 1 ∈ R ( N u +1) × 1 is a one-hot vector with the first element being 1 .

Original: and finally we obtain µ i and Σ i asthe weighted mean and variance of z i and { u j } N u j =1 , with the weight being v i .
Revised: Finally, we obtain µ iand Σ i as the weighted mean and variance of z i and { u j } N u j =1 , with the weight being v i .

Original: That is: µ i [-=
-] z i + 12 (cid:80) N u j =1 v j u j , and Σ i = 

Revised: That is:µ i [-=
-]= 12 z i + 12 (cid:80) N u j =1 v ij u j , and Σ i =

Original: ( z i − µ i )( z i − µ i )
Revised: ( z i − µ i )( z i − µ i )

Original: ⊤ + 12 (cid:80) N u j =1 v j ( u j − µ i )( u j − µ i ) ⊤ .
Revised: ⊤ + 12 (cid:80) N u j =1 v ij ( u j − µ i )( u j − µ i ) ⊤ .

Original: We summarize the proposed ADaptive Vicinal Few-Shot Learning (ADV) in Algorithm 1.

Original:  Experiment
Revised:  Experiment

Original: In this section, we verify the effectiveness of the proposed approach in improving the generalizationfor few-shot learning.
Revised: In this section, we verify the effectiveness of the proposed approach in improving the generalizationof the few-shot learning.

Original: We first introduce thestandard few-shot learning benchmark for performanceevaluation and the implementation details.
Revised: We first introduce the implementation details.

Original: Then we present the results on the benchmarks, followedby a detailed study on the proposed approach.
Revised: Then we present the resultson the benchmarks, followed by a detailed study on the proposed approach.

Original: We use three benchmarks for performance evaluation: mini ImageNet [33], CUB [34] and CIFARFS [2].
Revised: We use three benchmarks for performance evaluation: mini ImageNet [33], CUB [34] and CIFARFS [2].

Original: For all datasets, we construct 5-way 1-shot and 5-way 5-shot tasks by randomly sampling 5classes.
Revised: For all datasets, we construct 5-way 1-shot and 5-way 5-shot tasks by randomly sampling 5classes.

Original: For performance evaluation, we construct 10 , 000 meta-testing tasks and report the averageclassification accuracy with a 95% confidence interval.
Revised: For performance evaluation, we construct 10 , 000 meta-testing tasks and report the averageclassification accuracy with a 95% confidence interval.

Original: [Table]
Revised: [Table]

Original: We focus on meta-testing phase and directly use the WideResNet-28-10(WRN) [40] feature extractorspre-trained following the previous work [19] [Table]
Revised: Implementation details : we focus on meta-testing phase and directly use the WideResNet-2810(WRN) [40] feature extractors pre-trained following the previous work [19].

Original: We denote the application of ADV to the linear classifier with vicinal cross-entropy loss by ADV-CEand denote the application of ADV to SVM by ADV-SVM.
Revised: We use ADV-CE todenote the application of ADV in the linear classifier with vicinal cross-entropy loss and ADV-SVMto denote the application of ADV in SVM.

Original: For ADV-CE, we optimize the vicinalloss in (5) by gradient descent for 100 steps.
Revised: For ADV-CE, we optimize the vicinal loss in (5) bygradient descent for 100 steps.

Original: The learning rate is determined by performing gridsearch from 0 .
Revised: The learning rate is determined by performing a grid search from0 .

Original: 001 to 1 on the tasks constructed by  meta-validation set.
Revised: 001 to 1 on the tasks constructed by the meta-validation set.

Original: For ADV-SVM, we solve the QP in (9) by using a differentiable GPU-based QP solver [1].
Revised: For ADV-SVM, we solve the QP in (9)by using a differentiable GPU-based QP solver [1].

Original: The regularization parameter λ is setto 0 .
Revised: The regularization parameter λ is set to 0 .

Original: 1 and the parameter σ for RBF kernel is obtained via grid search from 0 .
Revised: 1 andthe parameter σ for RBF kernel is obtained via grid search from 0 .

Original: 1 to 10 .
Revised: 1 to 10 .

Original: The numberof step T ,  lazy stay probability β and  temperature τin the lazy random walks are obtained via gridsearch in { 1 , 2 , 3 , 4 , 5 } , { 0 . 1 , 0 . 2 , 0 . 5 } , { 0 . 01 , 0 . 1 , 1 , 10 } , respectively.
Revised: In the lazy random walkalgorithm, the number of step T , the lazy stay probability β and the temperature τ are obtained viagrid search in { 1 , 2 , 3 , 4 , 5 } , { 0 . 1 , 0 . 2 , 0 . 5 } , { 0 . 01 , 0 . 1 , 1 , 10 } , respectively.

Original: Notably, ADV accesses thequery set as unlabeled data to perform lazy random walks.
Revised: Notably, ADV accessesthe whole query set as unlabeled data to perform lazy random walk.

Original: Thus, ADV is a transductive method.
Revised: Thus, ADV is a transductivemethod.

Original: Thenumber of query samples per class is 15 .
Revised: The number of query samples per class is 15 .

Original: 4.1 Comparison with Baselines
Revised: 4.1 Comparison with Baselines

Original: We present the performance of ADV-CE, ADV-SVM with a WRN feature extractor and comparethe results with existing works in Table 1.
Revised: We present the performance of ADV-CE, ADV-SVM with a WRN feature extractor and comparethem with existing works in Table 1.

Original: The results of baselines are from [16], [22] and [11].
Revised: The results of baselines are from [16], [22] and [11].

Original: From Table 1  ADV achieves the best performance inall the datasets for both 1-shot and 5-shot tasks.
Revised: Weobserve in Table 1 that, in all datasets, ADV achieves the best performance in both 1-shot and5-shot tasks.

Original: Notable, ADV outperforms the best competitor by at least 2% for 1-shot tasks.
Revised: Notable, ADV outperforms the best competitor by at least 2% in 1-shot tasks.

Original: In 1-shot tasks, theempirical distribution estimated by the support set can deviate from the true distribution by a largemargin. The extraordinary performance of ADV in 1-shot tasks indicates its effectiveness to exploitthe information of the unlabeled data from the query set for a more accurate distribution estimate tothe true distribution of each class.
Revised: Theextraordinary performance of ADV in 1-shot tasks indicates its effectiveness to exploit the informationof the unlabeled data from the query set to provide a more accurate distribution estimate to the truedistribution of each class, addressing the issue that the empirical distribution estimated from thesupport set may deviate significantly from the true distribution.

Original: 4.2 Visualization of Vicinal Distributions
Revised: 4.2 Visualization of Vicinal Distributions

Original: The vicinal risk minimization principle in the proposed method is built on the vicinal distributionsobtained by  lazy random walks.
Revised: The vicinal risk minimization principle in the proposed method is built on the vicinal distributionsobtained by a lazy random walk.

Original: Thus, the performance of ADV depends on the exactness of vicinaldistribution estimate.
Revised: Therefore, the performance of ADV depends on the exactness of thevicinal distribution estimation.

Original: Here, we visualize the vicinal distributions to evaluate their exactness.
Revised: Here, we visualize the vicinal distributions to evaluate their exactness.

Original: Werun the proposed lazy random walks in Section 3.3 on two 5-way 1-shot tasks randomly constructedon the meta-testing set of miniImageNet, generate 500 samples from each vicinal distribution for [Table] visualization, and perform t-SNE algorithm on all data.
Revised: We run the proposed lazy random walk algorithm on two 5-way 1-shot tasks randomly constructedon the meta-testing set of miniImageNet, generate 500 samples from each vicinal distribution, andperform t-SNE algorithm on all data.

Original: The visualization results are presented in Figure 2.
Revised: The visualization results are presented in Figure 2.

Original: From Figure

Original: [Table]
Revised: [Table]

Original: we can observe  that the vicinal distribution generated by ADV covers most of thequery samples from the same class.
Revised: We can observe from Figure 2 that the vicinal distribution generated by ADV covers most of thequery samples of the same class.

Original: This validates that the lazy random walks module in ADV canprovide a good estimate of the vicinal distribution, which overlaps with the ground truth distribution.
Revised: This verifies that the lazy random walk module in ADV can providea good estimate of the vicinal distribution, which overlaps with the ground truth distribution.

Original: By optimizing the expected loss over the vicinal distributions, the model learned by ADV has asmaller probability of overfitting the support samples and has a better generalization to the query set.
Revised: Byoptimizing the expected loss over the vicinal distributions, the model trained by ADV has a smallerprobability of overfitting the support samples and provides better generalization to the query set.

Original: 4.3 Effect of the Network Architecture of Feature Extractor
Revised: 4.3 Effect of the Network Architecture of Feature Extractor

Original: The proposed algorithm ADV can be applied on any pre-trained feature extractor.
Revised: The proposed algorithm ADV is compatible with any pre-trained feature extractor.

Original: Here, we validatethe effectiveness of ADV on three feature extractors with different backbones that are trained bydifferent meta-learning methods.
Revised: Here, we validatethe effectiveness of ADV on three feature extractors with different backbones, which are trainedby different meta-learning methods.

Original: We consider WideResNet (WRN) [40], ResNet-12 [9] and ashallow network of four convolutional blocks (CONV) [2].
Revised: We consider WideResNet (WRN) [40], ResNet-12 [9] and ashallow network of 4 convolutional blocks (CONV) [2].

Original: We meta-train the WRN, ResNet, CONVfollowing [19], [16] and [2], respectively.
Revised: We meta-train the WRN, ResNet, CONVfollowing [19], [16] and [2], respectively.

Original: We report the performance of a kernel SVM classifier and a linear classifier trained by cross-entropy(denoted by CE) with and without ADV.
Revised: We report the performance of a kernel SVM classifier and a linear classifier trained by crossentropy (denoted by CE) with and without ADV.

Original: The results in Table 2 show  the performance improvementof ADV is consistent over all threedifferent feature extractors.
Revised: The results in Table 2 show that the performanceimprovement of ADV is consistent for three feature extractors.

Original: This verifies that the effectiveness of
Revised: This verifies that the effectiveness of

Original: ADV is  feature extractor-independent.
Revised: ADV is independent of the feature extractor.

Original: It is worth noting  the larger performance gains are observedon the better feature extractor.
Revised: It is worth noting that the larger performance gains areobserved on the better feature extractors.

Original: For example, the overall performance of WRN is better than CONV,and the performance boost of ADV-CE over CE of 1-shot tasksof WRN is 10.5%, which is largerthan the boost on CONV (4.8%).
Revised: For example, the overall performance of WRN outperforms CONV, and the performance boost of ADV-CE over CE with WRN in 1-shot tasks is 10.5%, whichis larger than the boost with CONV (4.8%).

Original: This observation indicates: 1) even with a superior feature extractorthat well preserves the similarity between data in feature space, the classifier trained from a limitednumber of high-quality support data still overfits the data; 2) for a better feature extractor, ADVcan exploit more information from the similarity between data to estimate the vicinal distribution ofsupport data and alleviate the overfitting.
Revised: This observation suggests that 1) even with a superiorfeature extractor that well preserves the similarity between data in feature space, the classifier trainedfrom a limited number of high-quality support data still overfits the data; 2) for a better featureextractor, ADV can exploit more information from the similarity between data to estimate the vicinaldistribution of support data and alleviate the overfitting.

Original: 4.4 Ablation Study
Revised: 4.4 Ablation Study

Original: We perform an ablation study on ADV about the gain of the VRM principle.
Revised: We perform an ablation study on ADV regarding the gain of the VRM principle.

Original: To do this, we train avanilla linear classifier by cross-entropy loss (denoted by CE), a linear classifier with Mixup vicinaldistribution [41] (denoted by CE-Mixup), a linear classifier with Delta vicinal distribution whose 
Revised: To do this, we traina vanilla linear classifier with cross-entropy loss (denoted by CE), a linear classifier with Mixupvicinal distribution [41] (denoted by CE-Mixup), a linear classifier with Delta vicinal distributionwhose mean locates on the same mean estimate by our lazy random walk module and whose varianceis zero (denoted by ADV-CE-DD), and compare them with ADV-CE. We also train a multi-class SVM classifier, a multi-class kernel SVM classifier, and a kernel SVM classifier with Delta vicinal

Original: [Table]
Revised: [Table]

Original: 87.41 ± 0. mean locates on the same mean estimate by our lazy random walks module and whose variance iszero (denoted byADV-CE-DD), and compare them with ADV-CE. Note that by using a Delta vicinaldistribution, only the mean will contributes to the loss. We also train a multi-class SVM classifier,a multi-class kernel SVM classifier, and a kernel SVM classifier with Delta vicinal distribution(denoted by ADV-SVM-DD).
Revised: distribution (denoted by ADV-SVM-DD).

Original: The training of multi-class SVM classifier all follows [5].
Revised: The training of multi-class SVM classifiers all follows [5].

Original: We reportthe performance on three benchmarks in Table 3.
Revised: We report the performance on three benchmarks in Table 3.

Original: In Table 3, we can observe that the performance of CE-Mixup is just competitive with CE inminiImageNet and CIFAR-FS, and  even inferior to CE on  CUB dataset.
Revised: The results in Table 3 show that the performance of CE-Mixup only competes with CE in miniImageNet and CIFAR-FS, and is even inferior to CE on the CUB dataset.

Original: This suggests  the Mixupvicinal distribution fails to improve the generalization given a limited number of data.
Revised: This suggests that the Mixupvicinal distribution fails to improve the generalization given a limited number of data.

Original: In contrast,the vicinal distribution obtained by the lazy random walk algorithm is a more accurate estimate forthe true distribution, as evidenced by the fact that the performance of ADV-CE-DD and ADV-CEis better than CE and CE-Mixup.
Revised: In contrast,the vicinal distribution obtained by the lazy random walk algorithm is a more accurate estimate ofthe true distribution, as evidenced by the fact that the performance of ADV-CE-DD and ADV-CEis better than CE and CE-Mixup.

Original: To be concrete, the performance boost of ADV-CE-DD over CEevidences the mean of  vicinal distribution, which does notlocated on the original sample, but on theweighted mean of the data passing through by the lazy random walks, is closer to the ground-truthdistribution.
Revised: To be concrete, the performance boost of ADV-CE-DD over CEdemonstrates that the mean of the vicinal distribution, which lies not on the original sample but on aweighted average of the data passed through by the lazy random walk, is closer to the ground-truthdistribution.

Original: The performance improvement of ADV-CE over ADV-CE-DD demonstrates that thevariance of the vicinal distribution helps to alleviate the overfitting problem.
Revised: The performance improvement of ADV-CE over ADV-CE-DD demonstrates that thevariance of the vicinal distribution helps to alleviate the overfitting problem.

Original: As shown in Table 3, the performance of Kernel SVM is worse than SVM in most cases, which maybe due to the linear separability of the features extracted by WRN, while kernel does not help toimprove the separability.
Revised: As shown in Table 3, the performance of Kernel SVM is worse than SVM in most cases. The reasonmay be that the features extracted by WRN are linearly separable and the non-linear kernel techniquedoes not help to further improve the separability.

Original: Though the performance of Kernel SVM is inferior to SVM, we observeperformance boost of ADV-SVM, which is built on Kernel SVM.
Revised: Although the performance of Kernel SVM isinferior to SVM, we observe a performance boost of ADV-SVM, which is built on Kernel SVM.

Original: This indicates that learning from thevicinal distributions that are close to the true distribution would sufficiently improve the performance.
Revised: This indicates that learning from the vicinal distributions that are close to the true distribution wouldsufficiently improve the performance.

Original: We also observe the performance gain of ADV-SVM over ADV-SVM-DD, again evidencing theimportance of the variance of vicinal distributions in alleviating the overfitting.
Revised: We also observe the performance gain of ADV-SVM over ADV-SVM-DD, again evidencing the importance of the variance of vicinal distributions in alleviatingthe overfitting.

Original:  Conclusion and Limitations
Revised:  Conclusion and Limitations

Original: In this paper, we discuss the overfitting issue in few-shot learning, which is caused by the deviationof  empirical distribution from the ground-truth distribution.
Revised: In this paper, we discuss the overfitting issue in few-shot learning, which is caused by the deviationof the empirical distribution from the ground-truth distribution.

Original: To alleviate the overfitting, we revisitthe Vicinal Risk Minimization principle in deep few-shot learning and propose to harness unlabeleddata and a lazy random walk algorithm to generate adaptive statistical parameters for the vicinaldistribution of each training sample.
Revised: To alleviate the overfitting, werevisit the Vicinal Risk Minimization principle in deep few-shot learning and propose to harnessunlabeled data and a lazy random walk algorithm to generate adaptive statistical parameters forthe vicinal distribution of each training sample.

Original: Following the VRM principle, we learn the model by optimizingthe resulting expected vicinal loss.
Revised: Following the VRM principle, we learn the modelby optimizing the resulting expected vicinal loss.

Original: We derive the vicinal loss for linear classifiers trained with crossentropy loss and for SVM classifier and develop the resulting Adaptive Vicinal classifiers as ADV-CEand ADV-SVM.
Revised: We derive the vicinal loss for linear classifierstrained with cross-entropy loss and for SVM classifier and develop the resulting Adaptive Vicinalclassifiers as ADV-CE and ADV-SVM, respectively.

Original: We evaluate the performance of ADV-CE and ADV-SVM on three benchmarks andvalidate the effectiveness of ADV in mitigating overfitting and improving generalization of classifierslearned from a limited number of samples.
Revised: We evaluate the performance of ADV-CE and ADV-SVM on three benchmarks and validate the effectiveness of ADV in mitigating overfitting andimproving the generalization of classifiers learned from a limited number of samples.

Original: Currently, our proposed ADC isonly compatible with CE and SVM.
Revised: Currently,our proposed ADC is compatible with CE and SVM only.

Original: We would extend it to other loss functions.
Revised: We will extend it to other loss functions.

Original: For the vicinal Cross-Entropy loss, weoptimize an approximation obtained by second-order Taylor expansion.
Revised: For the vicinal Cross-Entropy loss, we optimize an approximation obtained by second-order Taylorexpansion.

Original: However, the approximationerror can be large.
Revised: However, the approximation error can be large.

Original: Thus, the performance of ADV-CE may be suboptimal and we can further improveit by deriving a better approximation for the vicinal Cross-Entropy loss.
Revised: Thus, the performance of ADV-CEmay be suboptimal and we can further improve it by deriving a better approximation for the vicinal Cross-Entropy loss.

Original: Checklist
Revised: Checklist

Original: The checklist follows the references.
Revised: The checklist follows the references.

Original: Please read the checklist guidelines carefully for information onhow to answer these questions.
Revised: Please read the checklist guidelines carefully for information onhow to answer these questions.

Original: For each question, change the default [TODO] to [Yes] , [No] or[N/A] .
Revised: For each question, change the default [TODO] to [Yes] , [No] or[N/A] .

Original: You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
Revised: You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.

Original: For example:
Revised: For example:

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [Yes] See Section ?? .
Revised: [Yes] See Section ?? .

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [No] The code and the data areproprietary.
Revised: [No] The code and the data areproprietary.

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [N/A] Please do not modify the questions and only use the provided macros for your answers.
Revised: [N/A]Please do not modify the questions and only use the provided macros for your answers.

Original: Note that the Checklist section does not count towards the page limit.
Revised: Note that theChecklist section does not count towards the page limit.

Original: In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
Revised: In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.1.

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s

Original: contributions and scope?
Revised: contributions and scope?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes] .
Revised: [Yes] .

Original: See Section (c) Did you discuss any potential negative societal impacts of your work?
Revised: See Section 5(c) Did you discuss any potential negative social impacts of your work?

Original: [N/A]
Revised: [N/A]

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?
Revised: them?

Original: [Yes]
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [Yes]3.
Revised: [Yes]3.

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?
Revised: mental results (either in the supplemental material or as a URL)?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?

Original: [Yes]
Revised: [Yes](c)

Original: (c) Did you report error bars (e g., with respect to the random seed after running experiments multiple times)?
Revised: Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes]
Revised: [Yes]

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?

Original: [No]
Revised: [No]

Original: If you are using existing assets (e
Revised: If you are using existing assets (e

Original: g., code, data, models) or curating/releasing new assets...
Revised: g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A]
Revised: [N/A]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?
Revised: using/curating?

Original: [N/A]
Revised: [N/A]

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Original: information or offensive content?
Revised: information or offensive content?

Original: [N/A]
Revised: [N/A]

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: applicable?
Revised: applicable?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: spent on participant compensation?
Revised: spent on participant compensation?

Original: [N/A]
Revised: [N/A]