REPRESENTATIVE PROTOTYPE WITH CONSTRASTIVE LEARNING FOR SEMI-SUPENVISED FEW-SHOT CLASSIFICATION
Authors: Hexiu Lin
Daming Shi
Yukun Liu
Venue: Submitted to ICLR 2023 (2023)
Keywords: SEMI-SUPENVISED FEW-SHOT CLASSIFICATION

Abstract: Few-shot learning aims to learn novel classes in the dataset with few samples per class, which is a very challenging task. To mitigate this issue, the prior work obtain representative prototypes with semantic embeddin based on prototypical networks. While the above methods do not meet the requirement of few-shot learning, which requires abundant labeled samples. Therefore, We propose a new model framework to get representative prototypes with semi-supervised learning. Specifically, we introduces the dataset containing unlabeled samples to assist training the model. More importantly, to fully utilize these unlabeled samples, we adopt conditional variational autoencoder to construct more representative prototypes. Simultaneously, we develop novel contrastive loss to improve the model generalization ability. We evaluate our method on miniImageNet and tieredImageNet benchmarks for both 1-shot and 5-shot settings and achieve better performance over the state-of-the-art semisupervised few-shot method.

Under review as a conference paper at ICLR 2023
REPRESENTATIVE PROTOTYPE WITH CONSTRASTIVE LEARNING FOR SEMI-SUPENVISED FEW-SHOT CLAS- SIFICATION
Anonymous authors Paper under double-blind review
ABSTRACT
Few-shot learning aims to learn novel classes in the dataset with few samples per class, which a very challenging task. To mitigate this issue, the prior work obtain representative prototypes with semantic embedding based on prototypi- cal networks. While the above methods do not meet the requirement of few- shot learning, which requires abundant labeled samples. Therefore, We propose a new model framework to get representative prototypes with semi-supervised learning. Specifically,
introduces the dataset containing unlabeled samples assist training the model. More importantly, to fully utilize these unlabeled sam- ples, we adopt conditional variational autoencoder to construct more representa- tive prototypes. Simultaneously, we develop novel contrastive loss to improve the model generalization ability. We evaluate our method on minilmageNet and tieredImageNet benchmarks for both 1-shot and 5-shot settings and achieve better performance over the state-of-ther -frtt---ttt--t---- semi-supervised few-shot method.
INTRODUCTION
In real life, humans are able to quickly establish awareness of new concepts from just one or a few examples. However, conventional machine learning usually learn with abundant labeled samples to ensure its generalization ability. Actually, obtaining a plentiful of labeled samples is exceedingly hard on account of security and the high cost time and money. Motivated by this, many researchers turn to few-shot learning (FSL). In the field of image classification, FSL means getting better image classification accuracy in small dataset. Generally, prior knowledge obtained from the base classes and then applied to the novel classes, which contains a few labeled samples (Fei-Fei et al., 2006) (Wang et al. 2020).
Existing studies on FSL roughly fall into four types. (1) Metric-based method (Koch et al., 2015) (Vinyals et al.) 2016) (Zhang et al.] 2019b). The type of methods is a space mapping method, which aims to learn a good feature space. In this space, all data is converted into feature vectors, and the feature vectors of similar samples are close, while the feature vectors of dissimilar samples far, so as to distinguish samples, and the distance usually use Euclidean distance (Snell et al. 2017) or cosine distance Chen et al. 2019a). (2) Optimization-basee method. the meta-learning framework, the method first learns group of good and potential parameters for the network model with a large number of similar tasks, and then uses this group of parameters as the initial value to train on specific tasks, tasks, such as: (Finn et al. 2017) (Lee et al., 2019) (Fallah et al., 2020). (3) Data augmentation- based method (Alfassy that samples is few, so it can be solved by increasing the diversity of samples. For example. (Zhang al. (019g) proposed to segment the image into foreground and background, and then combine the foreground and background of different pictures, so as to expand the dataset. (4) Semantics-based method (Chen et al., 2019b) (Xing et al. 2019) (Li et al. 2020) (Zhang et al. (202)) (Xu & Le 2022). This method is recent research hotspot, which is mainly inspired by zero-shot learning (ZSL). This series of methods use semantic information as auxiliary information to enhance classification performance. In some cases, visual information is richer, while in some cases. semantic information is richer (Xing et al.) 2019). which explains that fusing cross modal information plays an important role in constructing representative class prototypes.
achieve the convergence effect long as fine-tuning on the new
al. 2019) (Schwartz
2018). The fundamental problem of FSL is

---

Under review as a conference paper at ICLR 2023
Generally, most of these methods not used alone but integrated, and almost all are based on the meta-learning framework. However, class prototypes based on the meta- meta-learning framework are not representative enough due to the number of samples in support set is few. Therefore, a new model to construct representative class prototypes. For FSI, prototype-based is the typical method. Simply put, prototype-basee is construct a class prototype for each class using support set, and then keep test samples (from query set) close to the class prototype to which they belong and away from the other class prototypes. Prototypical networks (ProtoNet) firstly to tackle FSL (Snell et al. 2017), the basic idea of which is that samples in each class will be mapped to a feature space through neural network, and calculating the mean features of all samples of each class in this space as class prototype. And then there are lot of work around ProtoNet.
propose
The novel extensions of ProtoNet (Ren et al., 2018) exploits unlabeled samples when constructing prototypes, moreover, this paper makes the precise analysis of the distractor in unlabeled samples. Furthermore, a cosine similarity based prototypical network to select neighbor samples to augment support set (Liu et al. 2020) and training the regression model to restore the biased prototype with the Euclidean distance between the biased prototype and the real prototype (Xue & Wang 2020), etc. The setting of our work is similar to (Ren et al., 2018), the different is that we cluster unl unlabeled samples first, and then judge the labels of unlabeled samples according to cluster centers and class prototypes. Except that we adopt the generation model to make full use of unlabeled samples.
We discover that the key of the prototype-hh pasee method is how to use a few samples to construct representative class prototype, which is the challenging task account of few samples noise samples. The above approaches mentioned above rely solely on visual features for few-shot classifi- cation. Recently, inspired by ZSL, some work combined semantic embedding with prototype-base to enhance the performance. (Xing et al. (019)) utilize cross-modal information (visual features and semantic embedding) to generate visual and semantic prototypes and fuse the two prototypes accord- ing to different weights. (Zhang et al. (202T) takes attribute features as prior knowledge to complete the biased prototype. (Xu & Le 2022) first selects the representative samples in the base classes via assuming that the features of each class follow the Gaussian multivariate distribution. Then, conditional variational autoencoder (CVAE) is used to generate representative features with these representative samples, and constructs representative class prototypes with the generate features and the support features in the novel classes. This paper opens up ideas for constructing representative class prototypes, the one is data preprocessing, and the other is using generation models to augment data.
Many previous methods require large number of labeled samples training stage, which is not consistent with the common scenario in our life, so semi-supervised learning can be used in our work. In order to make full use of unlabeled samples, these samples and semantic information inputed CVAE to generate more features to construct representative class prototypes. Concurrently, the novel contrastive loss introduced, which improves model generalization ability. Please note that this contrastive loss is calculated in feature space via feature extractor. Based on the above contents, we propose novel model framework via meta-learning. Our main contributions of this paper can be summarized as follows:
We propose a new prototype recovery framework based on meta-learning, which can effec- tively use unlabeled samples to construct representative class prototypes.
We develop novel contrastive loss, using class prototypes as the anchor, which allows better inter-class discriminability to mitigate generalization problem.
We evaluate our approach on two benchmark datasets for few-shot learning, namely mini- ImageNet and tieredImageNet. The experimental results show that our method achieves higher performance, outperforming semi-supervised few-shot learning baselines.
We summarize related works in Section 2. Section 3 provides rundown of our approach. Section 4 reports the main results obtained with our method. In section 5, we analyzed our methods from different aspects.
2

---

Under review as a conference paper at ICLR 2023
2
RELATED WORK
2.1 SEMI-SUPERVISED LEARNING
Many methods for FSL are supervised learning. However, in many fields, such as medical treatment and aerospace, there will be abundant information that has not been labeled, some of which are use- ful, but if all of them manually labeled, it will be very time-consuming and laborious. Therefore, FSL based on sunessuper learning paradigm is more practical (Ren et al., 2018) (Liu et al.) 2019) (Liu al. 2020). supervise few-shot learning mainly improves performance with un- labeled samples. Semi-supervise paradigm is used firstly to tackle FSL in (Ren et al. (2018), which augmenting data by tagging unlabeled samples. Simultaneously, they proposed soft K-means, soft K-means + cluster and soft K-means + mask to get more representative prototypes based on Pro- toNet. (Liu et al. 2019) puts forward a new idea, which uses graph model and label propagation to predict the labels of query set. Although these methods have achieved good performance, they do not make further use of unlabeled samples, we propose to take these unlabeled samples as the input of CVAE, combined with the corresponding class attributes, to construct more representative prototypes and improve the classification performance.
2.2 CONSTRASTIVE LEARNING
The core idea of contrastive learning is to shorten the distance between the positive samples and the anchor sample in the vector representation space, and lengthen the distance between the negative samples and the anchor sample. This makes the boundary between positive and negative samples more obvious. The performance of contrastive learning has been demonstrated in the image do- main (Kipf et al.) 2019) (He et al.) (020)) (Chen et al.] 2020) (Han et al. 2021). Compared with the generative methods, which need to reconstruct pixel details to learn sample features, contrastive methods only need to learn discrimination in the feature space. Therefore, contrastive methods do not pay too much attention to pixel details, but can focus on abstract semantic information (more general knowledge), so to improve the generalization ability of models. For sample selection, in traditional contrastive learning, we regard the input samples as anchors, the samples after data aug- mentation as positive samples, and other samples in the same batch as negative samples. Differently, sample selection in our model is to use the original prototype as the anchor.
2.3 CONDITIONAL VARIATIONAL AUTOENCODER
VAE (Kingma & Welling 2013 belongs to the generation model family. Using VAE models for generating features conditioned on the corresponding semantic embedding is fairly common in ZSL methods (Mishra et al. 2018) (Schonfeld et al. 2019). (Mishra et al., 2018) is the first to propose to use a conditional VAE for ZSL. (Xu & Le| (022) is the first FSL method that uses a conditional VAE model to generate visual features, conditioned on the semantic emhedding of each class. In Xu & Lc 2022). focusing on generating more features for the novel classes and constructing rep- resentative class prototypes with the support features in the novel classes, while our method focuses on generating more features for the support set to construct the representative class prototypes at two stages: meta-learning training and meta-learning test.
SEMI-SUPENVISED FSL WITH REPRESENTATIVE PROTOTYPE
In this section, we introduce the overall framework of our model. As shown in Figure ! For N-way have two the image, y1 is the label of 2; has few labeled samples. The other is query set @ = {x:)) where Q is the number of images in Q contains unlabeled samples. The samples in both sets are from Dmovel- Samples in D Dnovel are few, so most work introduce an auxiliary dataset Doaee. During Meta-Training stage, add an additional dataset R [xx))] contains large of unlabeled samples. Noted that the classes in Drase and Dnovel are different, formalized as Chase O Cnove! = 0. Our goal is to classify query samples correctly using few labeled samples from the support set.
K-shot image classification,
is support set S ((x,, 1r N where x; is
3

---

Under review as a conference paper at ICLR 2023
Support
pollive sumples
eabedding
Recondmacion inages
Encoeer
Laten oads
Deoakr
Euankel Supprt
Roceairation Ino.
repaioesanpic mncher
Creo-crtiapy loa
Figure 1: Overall Framework. First, getting features representation of samples. Then labeling samples with GMM, the support set turn the extended support set with unlabeled samples. Finally, constructing class prototypes with CVAE. Training the model with loss function L = Let + Luee + Liest, where the loss Let according shrink the distance the anchor and the positive samples, while enlarging the distance between the anchor and the negative samples.
In this paper, we exploit convolutional neural network fo, to extract features from an input where f indicates a parameter of the network. We adopt ResNet12 feature extractor for both the support set S and the query set Q.
3.1 LABELING SAMPLES WITH GAUSSIAN MIXTURE MODEL
In this part, generating label for unlabeled samples by the Gaussian mixture model. Specifically,
First, defining a multivariate Gaussian mixture model with K classes to estimate the probability density of the samples. Its defined distribution Equation ( where μkg, Σg denotes the and covariance of the k-th multivariate Gaussian distribution, and ag denotes the probability of the k-th mixture component.
K
P(x) =
Σ On p(æ|uu, Ex) k=1
(1)
Next, calculating the parameters
{a {a,,02,.... a2, agy #l, U2. Mk: Σ₁ Ez...., Σg} with the EM algorithm, which is divided into two steps: E-Step: calculating the posterior probability that the sample Iy comes from the k-th multivariate Gaussian distribution, where denotes the number of R, formalized as Equation (
OKk p(a,lKk, E₁)
Yik
1,2,3 ...y
E p(zjllus Ex)
M-step: calculating new round of model parameter estimates (ar,, Me. Ex), formalized as Equation 5 Repeating the E-step and M-step until the model converges.
==
7ik
0₁-
(3)
> 75A 2z
14
H (4x) )4x
(4)
Σ R
E 7ik (xg
Σ
(5)
Σ Yikk
Finally, utilizing the labeled samples from S to label unlabeled samples. Specifically, after the model convergence is completed, R is clustered into K classes, expressed as C' {c1 f2,. .2, Ck The
model calculates the mean vector as the class prototypes pk the support set according to Equation (6), so have K class prototypes, expressed P {p1, {P1,P2, P2, ... . Then the probability of each
4

---

Under review as a conference paper at ICLR 2023
cluster c; be class k is estimated based the proximity between the c₁ C and pk E P over softmax (Brile 1990), formalized as Equation 77)
Σ
(6)
Sk
where St E S the subset of support belonging to class
expl d < Ci, Pk >)
P(c
k) Σx exp( ct Pk >)
where d <,> denotes the Euclidean distance of two vectors.
3.2 CONTRASTIVE Loss
Contrastive learning has been widely shown to capable of improving model generalization. Hence, develop
loss to further improve model generalization. Given the class prototype pa as the anchor, simultancously, we' 'v'e possessed the dataset T = {Ii, 42 ,2, having the sample label as pı (in section B which as the positive samples. Also, having negative dataset O {r1, f .2,...,, To J. which has different label from pę- Here we propose a prototype-based contrastive loss function, which adopts the class prototype the anchor, formalized as Equation Now, our goal is to shrink the distance between x4 E T and ps while enlarging the distance between I; E O and
pk-
En eapeepp( < Tz. Pk Eis exp( Iı Pk >)
L -log
(8)
where / denotes the number of positive samples, o denotes the number of negative samples, and <, denotes the Euclidean distance of vectors.
3.3 CONSTRUCTING CLASS PROTOTYPES WITH CVAE
Based on the above work, already have the labeled samples (defined dataset Š) according to the support set and the corresponding dataset C. To generate samples with semantic embedding, we combine 2; E S and the interrelated attributes embedding as the input of VAE. VAE consists of an encoder E(x, a), which encodes sample to a latent code and decoder D(x, a), which reconstructs æ from z. The objection loss can be defined as:
Leae = K LLgg(= a²) a") (pp((((a))))
- log p(z,(2,a"))
where a denotes the semantic embedding of class k: The first term is the regularization term, which aligns the variation term g( a) to the prior distribution p(2(a) through the Kullback-Leibler vergence. The second term is the reconstruction loss, which aims make the features from the encoder generated by the decoder approximate the original input features..
We acquire new dataset Š whose samples contain semantic information. Combining the previous datasets S, we can reconstruct representative class prototypes as follows:
1
Pk
Syll
(z,)
+
SA
Σ
(fer
(10)
(
(R.B))ES
where Sz E is the subset of support belonging to class k. S₁ E S is the subset of support belonging class k.
3.4 META-LEARNIE WITH REPRESENTATIVE PTOTOTYPE
We construct number of N-way K-shot tasks from Drase following the episodic training manner (Vinyals et al. 2016). Specially, in each task, we sample N classes from the base classes, K images in each class as the support set S, and Q images the query Q. Beyond that, we introduce R, having images. For each task, first, we access more labeled samples from with GMM, and then calculating contrastive loss with these labeled samples, finally constructing class prototypes via
5

---

Under review as a conference paper at ICLR 2023
VAE with semantic embedding. Next, calculating the Euclidean distance between the test samples, which from the query set, and the representative class prototypes to predict the labels:
exp( d < for (2,),
P( k|x,)= Ee exp(
(11)
J0, (zi),
Here, i denotes the final predicted label for 2;, and d <, > denotes the Euclidean similarity between of two vector. The largest probability value is taken as the predict label. Then the loss function is calculated as:
Crest = =eet=ce(juu ce(
(12)
Here, is denotes the final predicted label for Ti. denotes the real label for Ji, ce(, ) denotes the cross-entropy loss. Therefore, at the meta-learning training stage, the loss function as follow:
[ = Let + Lvee + Lteat
(13)
Finally, applying the meta-trained model to the novel class.
Table 1: Few-shot classification accuracies minilmageNet and teiredImageNet. All results are averaged over 600 test episodes. Top results are highlighted.
Method
Backbone
minilmageNet
teiredImage.Net
5-way 1-shot
5-way 5-shot
5-way 1-shot
5-way 5-shot
ProtoNet Snell et al.
2017
ConvNet-64
49.42 + 0.78
68.20 + 0.66
MAML
Finn al.
2017
ResNer-18
48.70 :: 1.84
63.11 +:2 0.92
Soft k- k-Means Ren ct
201
ConvNet-64
50.09 0.45
64.59 + 0.28
51.52 - 0.36
70.25 0.31
TPN-semi Liu et al.
2019
ConvNet-64
52.78
66.42
55.74
71.01
PSN, Semi-supervised (Simon et
2018
ConwNet-64
68.12 1: 0.67
71.15 0.67
Ours (Semi-supervee learning)
ResNet-12
53.10 0.03
68.14 0.02
57.14 + 0.14
74.13 0.02
4
EXPERIMENTS
4.1 BENCHMARKS
minilmageNet. 2016, Google DeepMind team extracted minilmageNet based on ImageNet, which contains 100 classes, each class has 600 images. DeepMind team first applied minilmagenet to few-shot learning research /Vinyals et al. (2016). since then minilmageNee has became the bench- mark for few-shot learning fields. In general, divided the dataset into 64 classes training sets, 16 as validation sets, and 20 as test sets.
feiredlmageNet. The data set is also subset of ImageNet, which proposed in (Ren et al. 2018). Ieaged is similar to Omniglot, its classification has the concept of hierarchy. The dataset is divided into 34 high-level classes (such as Instruments, tools, vehicles, etc.), each of which contains 10-30 more detailed sub-classes (such as Musical Instruments, including guitars, pianos, etc.). The 34 classes are divided into 20 training classes, validation classes and test classes.
4.2 IMPLEMENTATION DETAILS
Dataset Split. For each dataset, we first create an additional split to separate labeled and unlabeled samples. For minilmagenet, 40% of the data for labeled samples and 60% of data for unlabeled sam- ples. For teiredlmagenet, 10% of the data for labeled samples and the remaining 90% for unlabeled samples.
Training Details. We adopt ResNer12 architectures feature extractor, and the dimension of the feature representation is 512. The dimensions of semantic embedding are set to be 512, and are extracted from CLIP Radford al. (2021). And then, all parameters trained jointly with 100 episodes in epoch. All our models are trained with Adam KingaD (2013) and initial learning rate of 10-3

---

Under review as a conference paper at ICLR 2023
Input
Predicted label
update
ResNet12
classifier
GMM
Clustering
(b)
Figure 2: Clustering Visualization. (a) To update the parameters of ResNet12. (b) To compare features distribution the original and the clustered.
Evaluation. We conduct few-shot classification on 600 randomly sampled episodes from the test set and report the mean accuracy together with the 95% confidence interval. In each episode, we randomly sample 15 unlabeled images per class for augmenting the support set, and sample 15 query images per class for evaluation in 5-way 1-shot/5-shot tasks
4.3 RESULTS
Sablenpreeents the 5-way 1-shot and 5-way 5-shot classification results of our methods on minilm- ageNet and tieredImageNet in comparision with previous FSL methods. We compare our method with (Snell al. 2017) (Finn et al. 2017) (Ren et al. 2018) (Liu et al. 2019) (Simon et al. 2018). The first is classical meta-learning methods, ProtoNet and MAMI.. The second is Semi-supervised method, such as Soft k-Means and TPN-semi. Since we use the Semi-supervised method paradigm for training, choose these two methods for comparison. Our method outperforms existing semi- supervised methods, which demonstrates its effectiveness.
5 ANALYSES
In order to better illustrate the experimental results, we explained and visualized some details of the experiment, found the experimental results under different experimental settings, and found better experimental settings.
5.1 FEATURE DISTRIBUTION ANALYSIS
Clustering Visualization. As shown in Figure ((a) first extract features of the input samples to obtain 312-dimensional feature vectors. And then in feature space, clustering features of unlabeled samples. Finally, updating the parameters of feature extractor according the classifier. To sum up, we cluster unlabeled samples in feature space. To further understand the distribution clusters in the feature space. We use t-SNE (Hinton van der Maaten 2008) to map feature representation to 2-d space. Figure 2(b) shows the feature distribution. The left is the original feature distribution, and the right the clustered distribution. We find that clustering will change the feature distribution of the original data and cluster the features of the same samples. However, since the image is high- dimensional data, the label generated by clustering may not be correct, which also causes problems in downstream tasks. Therefore, we will study how to further improve the accuracy of clustering by GMM.
Features Visualization on Training, To verify whether feature extractor has learned discriminative features, we randomly select three classes to visualized the feature distribution different training stages. Figure 3(a) is the original features distribution. Figure3 (b) is the features distribution after epoch 1. Figure (c the features distribution after epoch 64. Figure3(d)iss is the features distribution after epoch 128. As shown in Figure find that with the increase of training epochs, the features extracted by feature extractor become more and more discriminative. This shows that the model obtained in this training stage is effective.

---

Under review as a conference paper at ICLR 2023
(a)
(b)
(c)
(d)
Figure 3: Feature Visualization on Training, (a) Epoch (b) Epoch 1. (c) Epoch 64. (d) Epoch 128.
5.2 PERFORMANCE ON DIFFERENT NUMBER OF UNLABELED SAMPLES
Figure shows the classification accuracy with different number of unlabeled samples. The model is carried out under the setting of way = 5. The number of unlabeled samples of each class increases from 0 to 25, and observe that the classification accuracy of the model also increases. This demonstrates that under the meta-learning framework, even under the semi-supervised paradigm, the model can learn to obtain better representation.
80
1-shot
70
5-shot
60
Test ACC.(5-wayy (5-way)
40
30
20
10
15
20
25
Number of Unlabeled Items Per Class
Figure 4: Model Performance minilmageNet with different numbers of unlabeled samples meta-lcarning test.
CONCLUSIONS
novel framework for few-shot learning construct representative class prototypes proposed. First, clustering unlabeled samples with GMM, and predict the labels of unlabeled samples accord- ing the support set with few labeled samples, thus the model conforms the supervised learning paradigm. Then, in order to learn more distinctive features, the prototype-base based contrastive loss developed. Finally, the corresponding semantic embedding the extended support set and the extended support set as the input of CVAE to reconstruct more features, so as to calculate more representative class prototypes. The whole model is based on the meta-learning framework, and the whole training loss composed of three parts. The results show that the performance of our method better than the existing semi- cemi-supervised learning methods. In the future, we will focus on how to get better clustering results in high-dimensional data.
8

---

Under review as a conference paper at ICLR 2023
REFERENCES
Amit Alfassy, Leonid Karlinsky, Amit Aides, Joseph Shtok, Sivan Harary, Rogerio Feris, Raja Giryes, and Alex M Bronstein. Laso: Label-set operations networks for multi-label few-shot learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recogni- tion, pp. 6548-6557, 2019.
John S Bridle. Probabilistic interpretation of feedforward classification network outputs, with rela- tionships to statistical pattern recognition. In Neurocomputiing, pp. 227-236. Springer. 1990.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pp. 1597-1607. PMLR, 2020.
Wei- Yu Chen, Yen-Cheng Liu, Zsolt Kira, Yu-Chiang Frank Wang, and Jia-Bin Huang. A closer look few-shot classification. arXiv preprint arXiv:1904.04232. 2019a.
Zitian Chen, Yanwei Fu, Yinda Zhang, Yu-Gang Jiang, Xiangyang Xue, and Leonid Sigal. Multi- level semantic feature augmentation for one-shot learning. IEEE Transactions on Image Process- ing, 28(9):4594 4605, 2019b.
Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar. On the convergence theory of gradient- based model-agnostic meta-learning algorithms. In International Conference on Artificial Intelli- gence and Statistics, pp. 1082-1092. PMLR, 2020.
Nanyi Fei, Zhiwu Lu, Tao Xiang, and Songfang Huang. Melr: Meta-learning via modeling episode-
level relationships for few-shot learning. In International Conference Learning Representa- fions, 2020.
Li Fei-Fei, Robert Fergus, and Pietro Perona. One-shot learning object categories. IEEE trans- actions pattern analysis and machine intelligence, 28(4):594-611, 2006.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning, pp. 1126-1135, PMLR, 2017.
Junlin Han, Mehrdad Shoeiby, Lars Petersson, and Mohammad Ali Armin. Dual contrastive learning for unsupervised image-to- tangetto-mmage translation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 746-755, 2021.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 9729-9738, 2020.
Hinton and L van der Maaten. Visualizing data using t-sne journal of machine learning research. 2008.
A KingaD. A metbodtrastcchatiippinzzzzzii Anon. International Conferenceon Learning Rep- resentations. SanDego: ICLR, 2015.
Diederik P Kingma and Max Welling. arXiv:1312.6114 2013.
Auto-encoding variational bayes. arXiv preprint
Thomas Kipf, Elise van der Pol, and Max Welling. Contrastive learning of structured world models. arXiv preprint 1911v:1911.1221 2019.
Gregory Koch, Richard Zemel, Ruslan Salakhutdinov, et al. Siamese neural networks for one-shot image recognition. In /CML deep learning workshop, volume 2, pp. 0. Lille, 2015.
Kwonjoon Lee, Subhransu Maji, Avinash Ravichandran, and Stefano Soatto. Meta-learning with differentiable convex optimization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10657-10665. 2019.
Aoxue Li, Weiran Huang, Xu Lan, Jiashi Feng, Zhenguo Li, and Liwei Wang. Boosting few-shot learning with adaptive margin loss. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 12576- 2020.

---

Under review as a conference paper at ICLR 2023
Jinlu Liu, Liang Song, and Yongqiang Qin. Prototype rectification for few-shot learning. In Euro- pean Conference on Computer Vision. pp. 741-756. Springer. 2020.
Yanbin Liu, Juho Lee, Minscop Park. Sachoon Kim, Eunho Yang. Sung Ju Hwang, and Yi Yang. Learning to propagate labels: Transductive propagation network for few-shot learning, In Inter- national Conference on Learning Representations, 2019.
Ashish Mishra, Shiva Krishna Reddy, Anurag Mittal, and Hema A Murthy. A generative model for zero shot learning using conditional variational autoencoders. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops. pp. 2188-2196, 2018.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, pp. 8748 8763. PMLR, 2021.
Mengye Ren, Eleni Triantafillou, Sachin Ravi, Jake Snell, Kevin Swersky, Joshua B Tenenbaum, Hugo Larochelle, and Richard S Zemel. Meta-learning for semi-supervised few-shot classifica- tion. In Proceedings of 6th International Conference on Learning Representations ICLR, 2018.
Edgar Schonfeld, Sayna Ebrahimi, Samarth Sinha, Trevor Darrell, and Zeynep Akata. Generalized zero-and few-shot learning via aligned variational autoencoders. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8247-8255, 2019.
Eli Schwartz, Leonid Karlinsky, Joseph Shtok, Sivan Harary, Mattias Marder, Abhishek Kumar, Rogerio Feris, Raja Giryes, and Alex Bronstein. Delta-encoder: effective sample synthesis method for few-shot object recognition. Advances in neural information processing systems, 31, 2018.
Christian Simon, Piotr Komiusz, and Mchrtash Harandi. Projective subspace networks for few-shot learning. 2018
Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. Ad- vances in neural information processing systems, 30, 2017.
Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. Advances in neural information processing systems, 29, 2016.
Yaqing Wang, Quanming Yao, James Kwok, and Lionel Ni. Generalizing from few examples: A survey on few-shot learning. ACM computing surveys (csur), 53(3):1-34, 2020.
Chen Xing, Negar Rostamzadeh, Boris Oreshkin, and Pedro O O Pinheiro. Adaptive cross-modal few-shot learning, Advances Neural Information Processing Systems, 32, 2019.
Jingyi Xu and Hieu Le. Generating representative samples for few-shot classification. In Proceed- ings of the IEEF/CVF Conference on Computer Vision and Pullern Recognition, pp. 9003-9013, 2022.
Wanqi Xue and Wei Wang. One-shot image classification by learning to restore prototypes. In Proceedings of the AAAI Conference Artificial Intelligence, volume 34, pp.
2020.
Baoquan Zhang, Xutao Li, Yunming Ye, Zhichao Huang, and Lisai Zhang. Prototype completion with primitive knowledge for few-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3754-3762, 2021.
Hongguang Zhang, Jing Zhang, and Piotr Koniusz. Few-shot learning via saliency-guided halluci- nation of samples. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 2770-2779, 2019a.
Jian Zhang, Chenglong Zhao, Bingbing Ni, Minghao Xu, and Xiaokang Yang. Variational few-shot learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp- 1685-1694 2019b.
Manli Zhang. Jianhong Zhang, Zhiwu Lu, Tao Xiang, Mingyu Ding, and Songfang Huang. lept: Instance-level and episode-level pretext tasks for few-shot learning. In International Conference Learning Representations, 2020.
10


Reviews:
[{"review_id": "etiisn_tdB", "reviewer": "Reviewer_xsww", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a new method for few-shot image classification. The method contains a new framework for obtaining representative prototypes, by leveraging an external unlabeled dataset to assist training. The method also includes a conditional variational autoencoder and a contrastive loss. Experimental results are shown for 1-shot and 5-shot settings on two datasets (miniImageNet and tieredImageNet).", "summary_of_the_review": "The paper addresses an important problem but falls short in terms of experimental and analytical insights.  There are several modules in the method, but no ablation study is provided to establish the utility of each module.", "strength_and_weaknesses": "*Strengths*\n- The topic of FSL is important finding effective prototypes is important for improving FSL.\n- \n\n*Weaknesses*\n- Many details about the experimental setting are missing. For example how were the dataset splits obtained? Were they sampled at random, or was another protocol used? \n- tieredImageNet is an interesting dataset to use, but the paper does not explore various few-shot settings that are possible in tieredImageNet. For example, splitting the dataset on high-level vs low-level. \n- No ablation studies on the different components on the method (Contrastive Loss, CVAE protypes, Meta Learning, etc.)\n- only one architecture is used. Experiments should be replicated on various architectures (eg. ViT, ResNet, etc.)", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "YZ1hxR4Sqh", "reviewer": "Reviewer_7ZBH", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper deals with the problem of Few Shot learning were too few samples are available to perform heavy training of the model. To alleviate this problem authors proposed to use semi-supervised learning. An additional set of unlabelled samples is thus introduced. This additional set of samples is exploited with GMM clustering performed on the set of features and further associated to support classes with a specific metric in the latent space of a Contextual Variational Auto Encoder. Those pseudo-classified additional samples are then used to update class protoypes. \nAll the network used (feature extractor, CVAE) are trained with episodic training and dedicated losses (constrastive loss for prototype, CVAE loss, cross entropy for accuracy efficiency).\nExperiments are carried on miniImageNet and tieredImageNet data sets and compared with classical approaches. Additional analyses are provided to illustrate obtained clusters and to show the influence of introducing the additional unlabelles samples.", "summary_of_the_review": "This paper propose to use semi-supervised learning to enhance Few Shot learning scenarios. However the proposed approach turns out to be very similar to the transductive setting used in Few Shot Learning. No reference to such transductive setting is made here which is mandatory here. Furthermore reported results are far away from those obtained by even very early works in transductive settings. \n\nFurther considering the low clarity and quality of the paper, this paper is not of sufficient quality to be accepted here. So I recommend to strongly reject this paper. ", "strength_and_weaknesses": "Strength: \n---\nthis paper illustrates that using semi-supervised learning, some gain may be obtained relatively to inductive settings\n\nWeakness:\n---\n1. This paper is clearly lacking of recent works dealing with transductive setting in the context of FSL. Indeed transductive settings directly exploit the unlabelled samples from the query set to help improve the design of the classifier without having to introduce the $\\mathcal{R}$ dataset which is questionable. So the proposed 'semi-supervised' setting should be clearly compared to this transductive setting.  \n2. The benefirs of the use of CVAE is not clear here in the paper\n3. Experiments are really weak and more datasets should be considered here. \n4. Performance reported are really poor with respect to typical performance reported in transductive settings (see [1-3] for early work example on transductive settings). Especially [3] is already exploiting soft clustering of unlabelled samples to improve FSL\n5. Feature distribution analysis in section 5.1 is very naive. Comparison with epoch 0 of the training is somehow useless, since the proposed features are rather random ones. A typical instructing comparison would be to compare with final converged solution when using various losses or use of CVAE or not.\n6. equation 10 is incorrect when defining prototypes. The propotypes from the support and extended unlabelled sets should be (weighted-)averaged rather than sumed.\n\nSome early references to be considered here about transductive settings: (although they are not the most recent ones)\n\n[1]. Ziko et al, \"Laplacian regularized few shot learning\", ICML2020\n\n[2] Lichtenstein et al, \"TAFSSL: Task adaptive feature sub-space learning for few shot classification', ECCV 2020\n\n[3] Kye et al, \"Meta learned confidence for few shot learning\", CVPR 2020\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "2E3qVtv6ID", "reviewer": "Reviewer_R3n9", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose a method for few-shot classification via constructing representative prototypes. Here additional unlabeled data is used (transductive setting). They first use GMM to label samples in an EM manner. A conservative loss is used to push class centers far away from each other. A conditional VAE is trained to generate new samples that are used together with few-shot samples to construct class prototypes. ", "summary_of_the_review": "The paper doesn't introduce novel ideas and is poorly presented.", "strength_and_weaknesses": "Strength:\n- The method is simple.\n\nWeaknesses:\n- The paper was written in a rush with many typos, unfinished sentences, and wrong mathematical formulas. \n- The motivation is weak. It's unclear what is the main idea. Using unlabeled data is common in transductive settings. The contrastive loss here is not new.\n- The novelty is weak. Using GMM to label unlabeled data is a common practice. Using CVAE to generate additional features is not new. Constructing representative prototypes is not new.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}]

Author rebuttal:

Decision: Reject