U NLEASHING THE P OTENTIAL OF CNN S FOR I NTERPRETABLE F EW -S HOT L EARNING
Anonymous authors
Paper under double-blind review
A BSTRACT
Convolutional neural networks (CNNs) have been generally acknowledged as one of the driving forces for the advancement of computer vision.
Despite their promising performances on many tasks, CNNs still face major obstacles on the road to achieving ideal machine intelligence.
One is the difﬁculty of interpreting them and understanding their inner workings, which is important for diagnosing their failures and correcting them.
Another is that standard CNNs require large amounts of annotated data, which is sometimes very hard to obtain.
Hence, it is desirable to modify them so that they can learn from few examples.
In this work, we address these two blind sides of CNNs by developing novel and interpretable models for few-shot learning.
Our models are based on the idea of encoding objects in terms of visual concepts, which are interpretable visual cues represented within CNNs.
We ﬁrst use qualitative visualizations and quantitative statistics, to uncover several key properties of feature encoding using visual concepts.
Motivated by these properties, we present two intuitive models for the problem of few-shot learning.
Experiments show that our models achieve competitive performances, while being much more ﬂexible and interpretable than previous state-ofthe-art few-shot learning methods.
We conclude that visual concepts expose the natural capability of CNNs for few-shot learning.
I NTRODUCTION
After their debut (LeCun et al., 1998) in 1998, Convolutional Neural Networks (CNNs) have played an ever increasing role in computer vision, particularly after their triumph (Krizhevsky et al., 2012) on the ImageNet challenge (Deng et al., 2009).
Some researchers have even claimed that CNNs have surpassed human-level performance (He et al., 2015), though other work suggests otherwise (Zhu et al., 2017).
Nevertheless, the successes of CNNs have inspired the computer vision community to develop more sophisticated models (He et al., 2016; Szegedy et al., 2017).
Despite the gains in performance due to these increasingly complex models, we have only limited understanding of
CNNs.
Recent studies show that they fail in unexpected ways and are vulnerable to adversarial attacks (Goodfellow et al., 2015).
We only have limited insights into why CNNs are effective and it is hard to determine the visual cues used in these networks.
This is unsatisfactory from a scientiﬁc perspective and arguably slows the progress of developing more powerful CNNs.
In particular, the ever-increasing depth and complicated structures of CNNs mean that the internal representations produced by CNNs are highly abstract and hard to understand.
This means that the problem of understanding deep neural networks is non-trivial.
Notably, for conventional deep neural network structures, there are three major challenges for understanding their mechanisms.
Firstly, the use of non-linear layers makes it almost impossible to obtain theoretical results for understanding neural networks.
Secondly, for some commonly used networks such as ResNet (He et al., 2016) and Inception (Szegedy et al., 2017), the deep network features processed by many convolutional layers are squeezed into a vector before the prediction layer, which does not correspond to the 2 D spatial structure of images.
Thirdly, the intermediate-level feature maps within deep neural networks, despite preserving spatial information, are hard to relate to the ﬁnal recognition decisions of the networks.
To address the difﬁculty of interpreting deep neural networks, we attempt to gain understanding by building on recent work on Visual Concepts (Wang et al., 2015).
These Visual Concepts (VCs) are
[Figure]
extracted representations of object parts from CNNs (see visualizations in Figure 1).
They relate to the ﬁndings that CNNs have internal representations corresponding to objects in scenes (Zhou et al., 2015).
In Section 3, we will review VC in detail.
Brieﬂy speaking, VCs are extracted by clustering intermediate-level raw features of CNNs, e.g., features produced by the Pool-4 layer of
VGG-16 (Simonyan & Zisserman, 2015).
Serving as the cluster centers in feature space, VCs divide intermediate-level deep network features into a discrete dictionary.
Visualizations (as in Figure 1) qualitatively suggest that these dictionary elements correspond to semantic parts of objects.
VCs have also been quantiﬁed as detectors for semantic parts (Wang et al., 2017).
More speciﬁcally, our starting point is recent work in preparation on VC-Encoding which represents objects in terms of binary codes of VCs.
Using VC-Encodings we obtain the following three preliminary ﬁndings.
Firstly, some VCs are closely related to speciﬁc object categories, i.e. , they mainly occur in images from a certain category.
Secondly, VC-Encodings result in spatial patterns in images represented by the heat maps in Figure 3b.
Combined with the ﬁrst property, this indicates that VC-encoding capture spatial information.
Thirdly, VCs are training cost effective in the sense that several images are sufﬁcient for extracting VCs for VC-Encoding.
These preliminary ﬁndings encouraged us to investigate whether VC-Encodings can be applied to
Few-Shot Learning .
Traditionally, machine learning algorithms require large annotated datasets which is problematic for some real world applications.
Moreover, the ability to learn from a few examples is a characteristic of human intelligence and would be strongly desirable for an ideal machine learning system.
Previous works on few-shot learning mainly lies in two directions – learning a metric and learning to learn .
These methods achieve high accuracies on few-shot benchmarks but they lack ﬂexibilities ( i.e. have to be trained separately for each task) and are not always easy to interpret.
By exploiting
VC-Encoding we develop a novel approach to few-shot learning that is simple, interpretable, and ﬂexible.
This is motivated by the three properties described above: (1) the third property suggests that VCs can be extracted from few images, and (2) the ﬁrst two properties suggest that we can then perform image recognition using those encodings.
Based on these considerations we propose two intuitive models, nearest neighbor and factorizable likelihood model based on the VC-Encoding.
We emphasize that these models, unlike other few-shot learning methods, are not deliberately trained to address speciﬁc few-shot learning scenarios, e.g., 5-way 5-shot.
Instead, they are ﬂexible and can be applied to any-way any-shot without additional training.
Despite their simplicity, these models achieve comparable results to the state-of-the-art few-shot learning methods (using only the simplest versions of our approach).
From a deeper perspective, these results show that CNNs naturally have the potential for few-shot learning on novel classes but to achieve this potential requires studying the internal structures of CNNs to re-express them in simpler and more interpretable terms.
Overall, our major contributions are two-fold:
(1) We shed additional light on CNNs by studying VC-Encodings and ﬁnding three novel properties of these encodings.
(2) Based on these three properties, we present two simple, interpretable, and ﬂexible models for few-shot learning.
These models unleash the potential of CNNs for ﬂexible and interpretable few-shot learning.
They yield competitive results compared to the state-of-the-art methods on speciﬁc few-shot learning tasks but can also be applied directly, without additional training, to other few-shot scenarios.
R ELATED W ORK
Our work lies at the intersection of attempts to understand neural networks and work on few-shot learning.
Therefore, we review here previous literature on these two topics.
2. N EURAL N ETWORK U NDERSTANDING
Recently, there have been numerous studies aimed at understanding the behavior of neural networks.
The dominant theme of these explorations has been to probe the internal representations of CNNs.
Some literature tries to visualize inner representations by sampling (Zeiler & Fergus, 2014), generating (Simonyan et al., 2013; Nguyen et al., 2016) or by backpropagating (Mahendran & Vedaldi, 2015) images maximizing the activations of hidden units.
In addition, Zhou et al.
(2015) shows that object detectors emerge in CNNs and tries to interpret scene classiﬁcations as compositions of object detections.
These attempts mainly focus on tracing back the activities from hidden units to inputs while ignoring the causal relations between hidden units and recognition results.
Conversely, other works investigate the discriminative power of the hidden features of CNNs by assessing them on speciﬁc problems (Sharif Razavian et al., 2014;
Bau et al., 2017; Agrawal et al., 2014;
Yosinski et al., 2014).
The overall ﬁndings suggest that deep networks have internal representations of object parts.
The most relevant work is the study of visual concepts which discovered mid-level visual cues in the internal features of CNNs and showed relationships between these visual cues and semantic parts (Wang et al., 2015; 2017).
[Equation]
EARNING
There have been growing attempts to perform few-shot learning motivated by attempts to mimic human abilities and to avoid some of the limitations of conventional data-demanding learning.
An early attempt was made building on probabilistic program induction (Lake et al., 2015) and another attempt exploited object parts (Wong et al., 2017).
More recent efforts at few-shot learning can be broadly categorized into two classes.
The ﬁrst is to design methods to embed the inputs into a feature space friendly to few-shot settings (Koch et al., 2015; Vinyals et al., 2016).
The second is meta-learning which efﬁciently trains an ordinary model with the budget of few examples (Ravi &
Larochelle, 2017; Finn et al., 2017).
In addition, Qiao et al.
(2017) performs few-shot learning by estimating parameters of the prediction layer using regression from previously learned objects.
We emphasize that our approach differs from these works, most of which are tailored for a few speciﬁc few-shot learning scenarios, while our methods are simple and ﬂexible, so that they work both in normal and almost all few-shot settings.
B ACKGROUND : V ISUAL C ONCEPTS
In Wang et al.
, VCs are discovered as internal representations within deep networks which roughly correspond to mid-level semantic visual cues.
VCs play a core role in our work on understanding properties of CNNs and developing our interpretable few-shot learning approach,
In this section, we formalize VCs following Wang et al.
We ﬁrst summarize the formalization of VCs, which are also illustrated in Figure 2.
CNNs contain a hierarchy of lattices L l , where l 2 0 ; 1 ; : : : stands for the layer of the lattice.
In particular, the input images are deﬁned over the lattice L 0 and those lattices on which we derive VCs are speciﬁed as L k .
We denote the spatial mappings from L 0 to L k by (cid:25) 0 7!
k and from L k to L 0 by (cid:25) k 7! 0 .
Thus we can deﬁne F k = f f p : p 2 L k g as the feature vector set at L k of all images with p referring to a pixel in
[Figure]
lattices L k .
These feature vectors are computed by f p =
f
( I p ) , where the function f is speciﬁed by the neural network
and I p is a subregion of the input image I , centered on a point (cid:25) k 7!
( p ) on L 0 .
Note that by collecting feature vectors into F k we remove all spatial information.
Now we describe how VCs are extracted.
The approach assumes that the visual concepts are represented by a population code of the CNN feature vectors.
They can be extracted using an unsupervised grouping algorithm.
In our implementation, we assume that the feature vectors are generated by a mixture of von Mises-Fisher distributions (Hasnat et al., 2017) and learn this mixture by the EM algorithm (Banerjee et al., 2005).
This yields a set of VCs, which are mean directions of the learnt von Mises-Fisher distributions.
By contrast, K-means clustering was used in Wang et al.
Finally, we denote the set of VCs by V =
f f v : v 2 Vg .
To observe the basic semantic nature of VCs, we compute the distances from the original feature vectors to the VCs as follows:
[Equation]
(1) where d p;v denotes the distance between f p and the visual concept v .
We select those feature vectors with the smallest distances to each VC v and trace them back to the original input image using (cid:25) k 7! 0 .
This yields visualizations patches of VCs, shown in Figure 1.
We observe that these patches roughly correspond to the semantic parts of objects, which justiﬁes our assertion that the VCs are semantic visual cues.
Visual concepts have been applied to detection and classiﬁcation tasks.
In the original paper (Wang et al., 2015) VCs were evaluated as semantic part detectors.
Later work (Wang et al., 2017) showed that VCs could be combined by voting to detect semantic parts and performed well even if the parts were partially occluded.
In more recent work (in preparation) VCs were used to encode semantic parts and objects using VCEncoding and used for detection tasks in the presence of occlusion.
This VC-Encoding is described in the next section.
We emphasize that none of this prior work on visual concepts addressed few-shot learning and restricted itself to situations where there were many training examples.
F EW -S HOT L EARNING FROM VC S
This section describes the technical ideas of our paper.
In Section 4.1, we introduce VC-Encoding and then discuss three important properties which motivate its use for few-shot learning.
We note that VC-Encoding was developed for another research project (in preparation) but the three properties are contributions of this paper.
Then in Section 4.2 and Section 4.3, we propose two simple, interpretable models for few-shot learning based on VC-encoding.
4. VC-E NCODING AND I TS P ROPERTIES
We assume that objects can be parsed into semantic parts.
From the perspective of VCs, this means that most f p should be assigned to a single VC .
This requires specifying an explicit relationships between the f p and the VCs.
A natural choice is to compute the distances d p;v between the f p and the VCs and threshold it to provide a binary value b p;v ( i.e. b p;v
= 1 if d p;v < T ).
We refer to f b p;v g as the VC-Encoding .
We use two criteria to specify a good encoding, coverage and firerate ,
[Figure] [Figure]
following:
[Equation]
The choice of the encoding threshold T is a trade-off between requiring sufﬁcient coverage and a ﬁring rate that it is close to one (the threshold will be found by a grid-search which outputs the smallest threshold ensuring that coverage > = 0 : 8 with step size 0 : 001 ).
This yields b p;v =( d
p;v < T ) .
Next we analyze VC-encoding and ﬁnd the following three properties.
Class Sensitivity The ﬁrst insight is that many VCs tend to occur for a speciﬁc object class.
In
Figure 3a, we calculate the frequency distributions of several VCs for 6 object classes out the 12 in
PASCAL3D+ (Xiang et al., 2014).
In each column, a single object class dominates the frequency of VC occurrences.
This suggests that VCs are useful for classiﬁcation decisions.
Moreover, the corresponding visualized patches on the top support our understanding that VCs have this class sensitivity because they capture the semantic parts of objects.
Spatial Clustering
The second discovered property of VC-Encoding is that VCs, despite being learnt by ignoring spatial position, naturally cluster together to form regular spatial patterns (as shown in Figure 3b).
This is consistent with the conjecture that the spatial patterns of semantic parts play a vital role in object recognition.
More speciﬁcally, the VC-encoding capture the spatial patterns of the semantic parts of objects.
Training Cost Effectiveness After showing that VC-encoding is potentially useful for recognition (the ﬁrst two properties) we also study how many examples are needed to learn good VCs.
We ﬁnd that we only need a few images in order to extract VCs which, when used for VC-encoding, obey the ﬁrst two properties and hence are suitable for object classiﬁcation.
To illustrate this third property, we calculate the frequencies of VCs occurring on the 12 object classes (after thresholding) using only 5 randomly picked images per class in Figure 3c.
These frequencies are very similar to those when many images are used to train the visual concepts.
We observe, for example, that only the car and bus respond to the wheel visual concept.
As we will show, these frequencies can be used to build models learnt from few-shots.
4. N EAREST N EIGHBOR ON VC-E NCODINGS
First, we propose a simple template matching method which is similar to traditional nearest neighbor algorithms.
The only novelty is that we use a similarity metric between VC-encodings which is “fuzzy” so that it can tolerate small spatial shifts of the semantic parts in images.
Formally, the similarity metric takes the following form:
[Equation]
[Figure]
where K ( b; b ′ ) is the similarity between the binary
VC-encodings b and b ′ .
n ( p ) deﬁnes the set of neighboring positions of p (as shown in Figure 4).
During testing, we classify an image to the category of the training example with the largest similarity.
One motivation for this method is that sequential convolutional operations carried in a neural network can be considered as embedding input images into a hierarchy of feature spaces.
Each convolutional layer can be treated as a different level of decomposition of the inputs since the convolution along with non-linear activation, which take the form
Y = (cid:27) ( W (cid:1) X + B ) , is composed of matching templates W and using non-linear function (cid:27) to ﬁlter out patterns based on the threshold B .
In light of this interpretation, and the three properties described above, it is reasonable that VC-Encoding will yield an explicit semantic decomposition.
4. F ACTORIZABLE L IKELIHOOD M ODEL
Apart from the intuitive nearest neighbor method, we present a second method which models the likelihood of the VC-Encoding.
We observe that we can specify a distribution over the VC-Encoding b p;v using a bernoulli distribution with probability (cid:18) p;v .
Following Na¨ıve Bayes, we assume all the elements of the VC-Encoding b are independent (making it possible to learn the distribution for a very small number of examples).
Hence we can express the likelihood of b as following:
[Equation]
For each object class y , we derive a probabilistic distribution (cid:18) y from the training examples.
Thus the prediction of object class given the VC-encoding b is given by:
[Equation]
Note that by doing this, we are in fact implementing a discriminative model obtained from a generative distribution.
We smooth each distribution (cid:18)
y using a Gaussian ﬁlter to guard against unlikely events.
E VALUATIONS UNDER F EW -S HOT S ETTINGS
Few-Shot learning, unlike conventional machine learning, resembles the learning process of human beings.
It requires the efﬁciency to learn generalizable knowledge from strictly limited examples, such as a few training images.
Hence few-shot learning is a challenging task.
Nevertheless, the third property of VC-Encoding in Section 4.1 suggests that a few images may be enough for learning object models represented by VC-Encodings.
Indeed our experiments show that both our two visualconcepts-based few-shot learning methods are competitive in performance with methods designed speciﬁcally for few-shot learning such as Ravi & Larochelle (2017).
In addition, while previous fewshot methods always work in speciﬁc few-shot scenarios, such as 5-way classiﬁcations, our methods can be applied to a large range of few-shot scenarios using the same CNN trained only once.
The experimental results convey the message that trained CNNs have the potential to recognize novel objects from few examples by exploiting VC-Encoding.
[Table]
last column we use the same models as in the middle and we omit those that are not directly applicable to this setting.
5. M INI -I MAGE N ET
To assess the capability of our few-shot methods, we ﬁrst evaluate them on a common few-shot learning benchmark, namely Mini-ImageNet.
The Mini-ImageNet dataset was ﬁrst proposed by
Vinyals et al.
as a benchmark for evaluating few-shot learning methods.
It selects 100 classes out of 1000 classes in ImageNet with 600 examples per class.
We use the split proposed by Ravi & Larochelle (2017) consisting of 64 training classes, 16 validation classes and 20 testing classes.
In accordance with the convention for Mini-ImageNet, we perform numerous trials of few-shot learning during testing.
In each trial, we randomly sample 5 unseen classes from a preserved testing set.
Each class is composed of 5 training images for the 5-shot setting and 1 training image for the 1-shot setting.
During evaluation, we randomly select 15 images for each class following Ravi &
Larochelle (2017).
As Table 1 illustrates, we compare our methods against two baselines in line with the ones in Ravi & Larochelle (2017).
In addition, we present the performances of state-of-the-art few-shot learning methods including matching network (Vinyals et al., 2016), Meta-Learner (Ravi & Larochelle, 2017) and MAML (Finn et al., 2017).
Regarding our methods, we train a VGG-13 on the training and validation set and then extract 200 visual concepts from the Pool-3 layer.
The reason for choosing
Pool-3 features is that a grid in Pool-3 lattices L 3 correspond to a 36 (cid:2) 36 patch in the original 84 (cid:2) 84 image, which is a plausible size for a semantic part.
For the gaussian ﬁlter used to smooth the factorizable likelihood model, we use a (cid:27) of 1 : 2 .
To directly examine the impact of the VCs, we also include the result of nearest neighbor matching using raw features from the pool-3 layer (referred as Pool3-Neareast-Neighbor in Table 1).
Moreover, we attempt to evaluate the few
[Table]
[Figure]
airplane bicycle
Distribution
[Figure]
sofa train tvmonitor
Decision: tvmonitor
Class-wise Likelihood tvmonitor train sofa boat bicycle airplane
[Figure]
airplane bicycle boat sofa train tvmonitor
VGG
[Figure] [Figure]
I
[Figure]
Transform input to
Pool
[Figure] [Figure]
Given a set of VC distribution, calculate Pixel
[Figure]
Likelihood.
Pixel
[Figure]
Aggregate Pixel
[Figure]
Likelihood
[Figure]
Likelihood.
Step V
[Figure]
Choose
[Figure]
class
[Figure]
highest likelihood.
[Figure]
After calculating the likelihood for each pixel using distributions derived from a few examples, we obtain the pixel-wise likelihood.
Then we use the likelihood to make the ﬁnal decision.
For better visualization, we rescale the variance of aggregated likelihood to 1 .
For all the visualizations, we use the same Universal Color Bar.
This ﬁgure is best viewed in color.
learning ability with the variance in the number of classes (as shown in the last column of Table 1) and further extend the few-shot learning evaluation with other randomly selected settings (as shown in Table 2), where we need to use exactly the same model as the middle columns in Table 1 trained once on the training and validation set.
The results indicate our VCs-based methods compare well with prior methods which are speciﬁcally designed for few-shot learning.
Considering meta-learning-based methods, we achieve higher accuracy than Meta-Learner both in 1-shot and 5-shot set-ups, while just slightly behind MAML.
Compared with metric-based methods, which are more similar to ours, we marginally outperform the matching network, which is the state-of-the-art method of this category.
These results conﬁrm our assumption that low level visual cues within trained CNNs can naturally perform few-shot learning.
We observe that on the 5-shot scenario, our likelihood model is signiﬁcantly better than our nearest neighbor model.
A possible explanation is that the likelihood model combines several training examples into a distribution while nearest neighbor can only use examples individually.
For instance, if a front wheel of cars appears in one training example and a rare wheel occurs in another example, the likelihood can combine these two wheels into a distribution while nearest neighbor can only match testing examples with either the front wheel or the rare wheel.
Finally, using the same model in the middle columns of Table 1, some previous methods like Meta-Learner LSTM are not applicable to various extended settings in Table 2 and the last column in Table 1.
In fact, these methods can directly deal only with changes in the number of shots but cannot deal with changes in the number of classes.
But unlike these methods, our few-shot learning methods based on VC-Encoding can be easily extended (with minimal re-training) to any number of shots and any number of classes.
5. PASCAL3D+
To delve deeper into our methods, we apply them to PASCAL3D+, a dataset with larger high quality images than Mini-ImageNet.
PASCAL3D+
(Xiang et al., 2014) is a dataset augmenting 12 rigid categories of the PASCAL VOC 2012 (Everingham et al.).
It is originally tailored for 3D object detection and pose estimation.
We choose PASCAL3D+
as our testbed since it provides high quality images in comparable image size to ImageNet.
We interpret our few-shot recognition mainly by visualizing every step of the inference.
With input images of sufﬁcient sizes, we can obtain large
VC-Encoding distribution maps whose visualizations are friendly to human observation.
[Table]
The simplicity of our methods makes the inference process of few-shot recognition very transparent.
On PASCAL3D+, we ﬁrst qualitatively analyze this procedure based on VCs.
In Figure 5, we thoroughly visualize every step of our method based on an example VC.
Among the closest patches, the corner of TV Monitor occurs the most times.
So we may assume this VC relate to the corner of TV Monitor.
The distributions of this VC suggest it mainly responds to the upper right of TV
Monitors since only the upper right corner of TV Monitor’s distribution map shows high frequency (see the distributions in Figure 5).
Using this VC, we convert the original deep network features into
VC-Encoding.
The VC-Encoding implies this VC ﬁres on the upper right corner of the input (see the
VC-Encoding map in Figure 5).
After calculating the pixel-wise likelihood using the distributions from a few images, it is obvious that except for the TV Monitor, each class has a low likelihood area on that corner (dark parts of pixel-wise likelihood maps in Figure 5).
Finally, we aggregate the likelihood and make the correct classiﬁcation decision.
Meanwhile, we quantitatively evaluate our methods on PASCAL3D+.
More speciﬁcally, we employ
PASCAL3D+ as our testing dataset.
For training, we use the ImageNet (Deng et al., 2009) classiﬁcation dataset without object classes related to 12 rigid categories ( 956 classes left).
We train an ordinary VGG-16 as our starting point which achieves 71 : 27% top-1 accuracy.
For testing, we crop the objects out using annotated bounding boxes provided by Xiang et al.
and resize them into 224 (cid:2) 224 .
Then we use Pool-4 features produced by the VGG-16 to implement our few-shot methods instead of our conﬁgurations in 5.1.
The main reason for this change is the increased input image size of 224 in this task.
As a comparison, we propose 2 baseline models.
One (referred to as Baseline-nearest-neighbor in Table 3) is a nearest neighbor method based on raw Pool-4 features using the cosine distance metric.
The other (referred to as Baseline-SVM in Table 3) is an
Exemplar-SVM trained using hinge loss.
Both of these baselines use the same pre-trained VGGas our methods.
During evaluation, we set 20 trials of both 5-shot and 1-shot learning over 12 classes on PASCAL3D+.
We also assess our methods using different numbers of VCs to see impacts of the number of VCs.
The results are shown in Table 3.
In light of our testing results, we conclude that VC-Encoding is a powerful semantic decomposition of images.
In general, our methods based on VCs outperform 2 baselines.
In particular, the difference between our nearest neighbor methods and baseline baseline nearest neighbor methods is a series of operations based on VCs.
In view of the accuracy gap between this pair, we claim that decomposing fuzzy features ( i.e. deep network features) into explicit semantic cues helps both in interpretability and performance.
We also notice that our methods are not sensitive to the number of
VCs because changes of the number of VCs only cause slight differences among accuracies.
C ONCLUSION
In this paper we address the challenge of understanding the internal visual cues of CNNs and making use of them for few-shot learning.
We start by using visual concepts which enable us to represent objects in terms of VC-Encodings.
Then we demonstrate three properties of VC-Encoding, both qualitatively and quantitatively, which suggest that VCs can be used for few-shot learning.
Following these ideas we propose two novel, but closely related, models to few-shot learning which are simple, interpretable and ﬂexible.
Our methods show comparable performances to the state-ofthe-art methods which are specialized for speciﬁc few-shot learning scenarios.
We demonstrate the ﬂexibility of our two models by showing that they can apply to a range of different few-shot scenarios with minimal re-training.
In summary, we show that VC-Encodings enable ordinary CNNs to perform few-shot learning.
We emphasize that in this paper we have concentrated on developing the core ideas of our two few-shot learning models and that we have not explored variants of our ideas which could lead to better performance by exploiting standard performance enhancing tricks.
Future improvements would include improving the quality of extracted VCs and extending this approach to few-shot detection.