Safety in Numbers: Learning Categories from Few Examples with Multi Model
Knowledge Transfer
Tatiana Tommasi1,2 , Francesco Orabona3 , Barbara Caputo1
1
Idiap Research Institute, Martigny CH
2Ecole Polytechnique Federale de Lausanne, EPFL, Lausanne CH
3
DSI, Universita’ degli Studi di Milano, Milan IT
ttommasi@idiap.ch, orabona@dsi.unimi.it, bcaputo@idiap.ch

Abstract
Learning object categories from small samples is a challenging problem, where machine learning tools can in general provide very few guarantees. Exploiting prior knowledge may be useful to reproduce the human capability ofrecognizing objects even from only one single view. This
paper presents an SVM-based model adaptation algorithm
able to select and weight appropriately prior knowledge
coming from different categories. The method relies on thesolution of a convex optimization problem which ensures tohave the minimal leave-one-out error on the training set.
Experiments on a subset of the Caltech-256 database showthat the proposed method produces better results than bothchoosing one single prior model, and transferring from all
previous experience in a flat uninformative way.

1. Introduction
The ability to learn from few samples is a hallmark of human intelligence. We rapidly and reliably learn many kindsof regularities and this enables us to make inductive inference even from only small amount of data [1].
Although current state of the art categorization methods
reach impressive results on difficult datasets [6], they don’t
handle well small training sets. Without additional information, learning from few examples always reduces to an
ill-posed optimization problem. A possible solution is exploiting prior knowledge, a strategy known in the literature
as learning to learn, knowledge transfer or transfer learning. The basic intuition is that, if a system has alreadylearned k categories, learning the k + 1 should be easier,even from one or few training samples [21]. Besides boosting the learning process, knowledge transfer can give threeother advantages ([19], see Figure 1): (1) higher start: theinitial performance is higher (one-shot learning); (2) higher
slope: performance grows faster, and (3) higher asymptote:

Figure 1. Three ways in which transfer might improve learning [19].

the final performance is better.
The contribution of this paper is a method for learning
object categories from few examples. We focus on three
key issues for knowledge transfer: how to transfer, what totransfer and when to transfer [16]. We propose a discriminative method based on Least Square Support Vector Machine (LS-SVM) [20] (how to transfer) that learns the newclass through adaptation. We define the prior knowledge as
the hyperplanes of the classifiers wj0 , j = 1, · · · , k of the
k classes already learned (what to transfer). Hence knowledge transfer is equivalent to constrain the hyperplanes wof the k + 1 new category to be close to those of a subset of the k classes. This strategy is in between the choiceof transferring acritically from all previously learned models [7] and transferring from one single model [22]. We
learn the sub-set of classes from where to transfer, and how
much to transfer from each of them, via the Leave-One-Out(LOO) error on the training set. Determining how much totransfer helps avoiding negative transfer. Therefore, in caseof non-informative prior knowledge, transfer might be disregarded completely (when to transfer).
Experiments on various subsets of the Caltech-256 [12]database show that our approach consistently reproduces
the curve depicted in Figure 1 with a higher start, and higher
slope compared to what is obtained by not exploiting priorknowledge, and to current state of the art knowledge tran-sfer approaches [22, 7]. Furthermore, when the number kof known classes grows, our algorithm presents a one-shotlearning behaviour.
The rest of the paper is organized as follows: we give an
overview of previous work in Section 2. Section 3 describes
LS-SVM and the knowledge transfer algorithm in [22], on
which we build. Section 4 describes our new algorithm anddiscusses its properties. Experimental results are reported
in Section 5. We conclude with an overall discussion andpointing out possible avenues for future research.

2. Related Work
Several authors addressed in the past the issues of what,
how and when to transfer. We review below the most prominent approaches.
What to Transfer. We can find three answers to thisquestion in the literature (see [16] for a survey). The first is
the instance-transfer approach: although the source domain
data cannot be reused directly, there are certain parts ofthem that can still be considered together with a few labelleddata in the target domain. A second solution is definedby transferring feature representations. It means learning
a common feature structure, e.g. a kernel in SVM-based
approaches, from different domains that can bridge related
tasks. The third strategy can be described as parametertransfer approach. It assumes that the source task and thetarget tasks share some parameters of their model or priors.
How to Transfer. Wu and Dietterich transferred source
training examples either as support vectors or as constraints(or both) and demonstrated improved image classification
by SVMs [24]. Fei-Fei et al. [7] proposed a Bayesian transfer learning approach for object recognition that learns acommon prior over visual classifier parameters. Zweig andWeinshall [25] investigated transfer learning with a method
based on combining object classifiers from different hierarchical levels into a single classifier. Using discriminative (maximum margin) object models, Fink [9] developed amethod that learns distance metrics from related problems.
Quattoni et al. [17] proposed to use knowledge transfer inan unsupervised setting learning a representation based onkernel distances to available unlabelled data.
When to Transfer. Works focusing on when to transfer evaluate the limit of transfer learning power. Rosenstain
et al. [18] showed empirically that if two tasks are dissimilar, then the transferring hurts the performance on the target task. Ideally, a transfer method should produce positivetransfer between appropriately related tasks while avoiding
negative transfer when the tasks are not a good match. However it might be easier to avoid negative transfer if, given
multiple source tasks, one transfers from several or all ofthem.
Research on knowledge transfer is still in its infancy,
especially applied to object recognition. Although there

are many publication in this area, given the slightly different tasks defined in each paper, none of them compare
against the others. Moreover there is not an official testbed
database, nor a standard experimental setup. In this work
we propose a reproducible experimental setting that can beused in the future to test new knowledge transfer algorithms
and we benchmark our algorithm against two other methods
in literature [22, 7]. We address three open problems of [8]:(1) the possibility that a sophisticated multimodal prior isbeneficial in learning; (2) if it is easier to learn new categories which are similar to some of the “prior” categories;(3) if exist another point of view besides the Bayesian one
that allows to incorporate prior knowledge. We present
a discriminative method which exploits a combination ofmultiple visual features and selects automatically the mostuseful prior knowledge models to use when learning a new
category.

3. Problem Statement
Consider the following scenario. We have k visual categories and a classifier trained to distinguish each of themfrom background. This corresponds to define k functions
fj (x) → {1, −1}, j = 1, ..., k, such that the image x is assigned to the j th category if and only if fj (x) = 1. Now
suppose that we want to learn a new k + 1 category fromjust one or few instances, plus some background examples.
To obtain fk+1 we can train using only the available data, or
we can take advantage of what already learned. In the following we briefly review the LS-SVM theory and how it canbe used in a model adaptation framework [15]. We review
how this approach can be formulated to derive a knowledgetransfer algorithm that exploits prior knowledge from only
one of the k classes [22] (Section 3.1). The contribution ofthis paper is how to extend this method to exploit all thesuitable prior knowledge. The used strategy is presented in
Section 4.

3.1. LS-SVM Adaptation Method: learning fromsmall samples
Suppose to have a binary problem and a set of l samples{xi , yi }li=1 , where xi ∈ X ⊂ Rd is an input vector describing the ith sample and yi ∈ Y = {−1, 1} is its label.
We want to learn a linear function f (x) = w · φ(x) + b
which assigns the correct label to an unseen test sample x.
φ(x) is used to map the input samples to a high dimensionalfeature space, induced by a kernel function K(x, x0 ) =φ(x) · φ(x0 ) [5]. In LS-SVM the model parameters (w, b)
are found by solving the following optimisation problem:lCX
1[yi − w · φ(xi ) − b]2 .
min kwk2 +
w,b 2
2 i=1

(1)

It can be shown [20] that the optimal w is expressed byPlα, b) are found solving
w = i=1 αi φ(xi ), and (α

K + C1 I
1T

1
0



α
b




=

y
0


,

(2)

where K is the kernel matrix. Let us call G the first term in
left-hand side of (2). The least-square optimisation problem
can be solved by simply inverting G. Another advantage of
the LS-SVM formulation is that it gives the possibility towrite the LOO error in closed form [4]. The LOO error isan unbiased estimator of the classifier generalization error
and can be used for model selection [4].
Slightly changing the classical LS-SVM regularization
term, it is possible to define a learning method based on
adaptation [15]. The idea is to constrain a new model to beclose to one of a set of pre-trained models:lowest one will identify the best prior knowledge model touse for adaptation.
To further increase robustness to unbalanced distributions of the data, the model parameters (w, b) can be found
via minimisation of a regularised weighted least-square loss
function [20]:lCX1ζi [yi − w · φ(xi ) − b]2 .
min kwk2 +
w,b 2
2 i=1

(8)

This introduces just a small variation in the LS-SVM soluα, b) are defined by ation: the optimal model parameters (αmodified system of linear equations [4]:

K + C1 W
1T

1
0



α
b




=

y
0


,

(9)

l
CX
1
0 2
[yi − w · φ(xi ) − b]2 , (3)min kw − βw k +
w,b 2
2 i=1

where W = diag{ζ1−1 , ζ2−1 , . . . , ζl−1 } and ζi are defined
as in (7). Hence the model adaptation method changes to its
weighted formulation [22]:where w0 is the parameter describing the old model and β
is a scaling factor in (0, 1) necessary to control the degree
to which the new model is close to the old one. The LOO
error in the modified formulation is:lCX1ζi [yi − w · φ(xi ) − b]2 . (10)min kw − βw0 k2 +
w,b 2
2 i=1

(−i)

ri

=

αi
αi0
−
β
,
G−1
G−1
ii
ii

(5)

In this way the weighting factors ζi take into account thatthe proportion of positive and negative examples in the
training data are known not to be representative of the operational class frequencies. In particular they help to balance
the contribution of the sets of positive and negative examples to the data misfit term [22].
Experiments show that this method is able to learn new
visual categories from few examples. However, the algorithm can choose only one prior known model. As wewill show in Section 5, this is not always the best solution.
Moreover this approach can suffer for instability in time, i.e.
when the number of training images increases.

(6)

4. Multi Model Knowledge Transfer(4)T
where αi0 = G−1(−i) [ŷ1 , . . . , ŷi−1 , ŷi+1 , . . . , ŷl , 0] , G(−i) is
the matrix obtained when the ith sample is omitted in G andŷi = (w0 · φ(xi )), i.e. ŷi is the prediction of the old model(−i)
on the ith sample. ri
is then used to obtain an estimateof the Weighted Error Rate (WER) [4]:W ER =lX(−i)

ζi Ψ{yi ri

− 1}

i=1

1
with Ψ{z} =1 + exp{−10 ∗ z} lif yi = +12l+and ζi =l
if yi = −1 .
−
2l

(7)

Here l+ and l− represent the number of positive and negative examples respectively. Introducing the weighting factors ζi is asymptotically equivalent to re-sampling the dataso that object and non-object examples are balanced [4].
Hence, without explicitly running cross validation experiments, the best learning parameters which maximise the
LS-SVM model generalisation performance can be found as(−i)
those minimising WER. Since ri
depends on β, for each
known model it is possible to find the best β producing thelowest WER. Then, comparing all the criterion errors, theConsider the following situation. Suppose to be given
the task to learn from few examples the class motorbike,having already learned the categories bicycle, car, dog andcat. We would expect to achieve better results by transferring from bicycle and car, rather than transferring from bicycle or car. Also, we would expect better results compared
to transferring equally from all known categories, as cat anddog might induce negative transfer.
This kind of scenario motivates us to design a knowledgetransfer algorithm able to find autonomously the best subset of known models from where to transfer. In the rest of
the Section we define the new model (Section 4.1) and we
discuss its properties (Section 4.2).

4.1. Multi Model Knowledge Transfer: Definition
We start from Equation (10) and we rewrite it substituting the single coefficient β with a vector β containing as
many elements as the number of prior models, k:kX1w−
βj wj0
min
w,b 2
j=1

2

l

CX
+
ζi (yi − w · φ(xi ) − b)2 .
2 i=1

(11)
β k2 ≤ 1.
Here β has to be chosen in the unitary ball, i.e. kβ
It is similar to the regularization term used in LS-SVM in
Equation (1), and it is a natural generalization of the original
constraint 0 ≤ β ≤ 1. This term is necessary to avoid
overfitting problems. They can happen when the number ofknown models is large compared to the number of training
samples. With this new formulation the optimal solution isw=

kX

βj w 0 j +

j=1

l
X

αi φ(xi ) .

(12)

i=1

Hence w is expressed as a weighted sum of the pre-trainedmodels scaled by the parameters βj , plus the new model
built on the incoming training data.
To find the optimal β we use again the possibility of LSSVM to write the LOO error in closed form:
(−i)

ri

= yi − ỹi =

k
0
X
αi(j)
αi
−
β
,
j
G−1
G−1
ii
ii
j=1

(13)

j
j
j
j
j
0
T
where αi(j)= G−1(−i) [ŷ1 , . . . , ŷi−1 , ŷi+1 , . . . , ŷl , 0] , ŷi =(wj0 ·φ(xi )) and ỹi are the LOO predictions. By multiplyingby yi we obtain:


k
0
X
α
αi
i(j)
yi ỹi = 1 − yi  −1 −βj −1  .
(14)
Gii
G
ii
j=1

The best values of βj are those minimizing the LOO error,
i.e. the values producing positive values for yi ỹi , for eachi. However minimizing directly the sign of those quantities
would result in a non-convex formulation with many local
minima. We propose instead the following loss function:loss(yi , ỹi ) = ζi max [1 − yi ỹi , 0]
 


k
0
X
α
α
i(j)
i
βj −1  , 0 .
= max yi ζi  −1 −
Gii
G
ii
j=1

(15)

This loss function is similar to the hinge loss used in Support Vector Machines. It is a convex upper bound to the
LOO misclassification loss and favours solution in which ỹi
has a value of 1, beside having the same sign of yi . Moreover it has a smoothing effect, similar to the function in (6).

Finally, the objective function is:J=lX

loss(yi , ỹi ) s.t.

β k2 ≤ 1 .
kβ

(16)

i=1

Notice that this formulation is equivalent to the more comβ k22 + CJ for a proper
mon optimization problem (1/2)kβ
choice of C [5]. By minimizing J we can find the best values of βj to weight the known prior models in the transfer
learning process. The scaling factors ζi are introduced in
the loss function to take care of the data unbalance between
positive and negative samples in the training set, as in [22].
We implement the optimization process using a simple
projected sub-gradient descent algorithm, where at each itβ k2 ≤ 1.
eration β is projected onto the l2 -sphere, kβ4.2. Multi Model Knowledge Transfer: Properties
The main advantage of our approach is the ability totransfer from multiple prior model, instead of choosing justone. At the same time, the knowledge is not transfered ina flat, uninformative way, but we evaluate the importanceof each model and their interaction. The loss used is convex
and the constraint in (16) is convex too, hence the minimizerof (16) is unique. This is opposed to the formulation proposed in [22], where (7) is non-convex. This means that thealgorithm in [22] can have many local minima.
An important property of this new formulation is also
its “stability”. Stability here means that the behaviour of
the algorithm does not change much if a point is removed
or added. This notion is closely related to the LOO error,
which is exactly calculated measuring the performance ofthe model every time a point is removed. Recent works
have shown that a stable algorithm has a better generalization ability [3]. The algorithm in [22] can choose only
one model at each time step, to be used to transfer knowledge. This means that everytime the algorithm “changes its
mind”, i.e. it chooses a different prior model on two consecutive time steps, the behaviour of the algorithm will change
completely. On the other hand, our method selects morethan one prior model at each time step, so we expect thatdifferences between steps in the vector β will be small. The
regularization is also important in this sense [3]. In Section
5.2 we show empirically that this is true.
From a computational point of view the current algorithm’s runtime is O(l3 + kl2 ), with l the number of training samples (of the order of 10 images) and k the numberof known prior models. The first term is related to invertingG, while the second term is the computational complexityof (13). We match the complexity of a plain SVM, whichin the worst case is known to be O(l3 ) [13], and is thestandard out-of-the-shelf classification method commonlyused on datasets with more than 103 images. The computational complexity of each step of the projected sub-gradientdescent is O(kl) and it is extremely fast. For instance,
our MATLAB implementation takes just half a second withl = 12 and k = 3.

5. Experiments
We present here three sets of experiments designed toillustrate how our algorithm performs (a) when the priorknowledge is related/unrelated to the new class (Section5.2); (b) when prior knowledge increases (Section 5.3); (c)compared to the current state of the art [7, 8] (Section 5.4).
We first describe the experimental setup (Section 5.1) andthen we report our findings in the three scenarios describedabove. The algorithms presented here were implemented in
MATLAB. The code for the Multi Model Knowledge Transfer method and all the scripts used for the experiments areavailable online1 .

5.1. Experimental Setting
Our working assumption is to have k category models
stored in memory, built using LS-SVM. We used the Gaussian kernel K(x, x0 ) = exp(−γkx − x0 k2 ) for all our experiments; the parameters C and γ were chosen by crossvalidation. When the new k + 1 category comes, the systemstarts learning.
All experiments are run on subsets of the Caltech-256
database [12]. We selected in total 41 classes + backgroundclass, obtaining a data set with a fair amount of clutter andscale variation. We didn’t perform any image selection
or preprocessing. The training data consists of m images
from the background dataset and an increasing number of
instances of the new category from 1 to m. The test set consists of 100 images, half from the background and half fromthe new category. Images are chosen randomly by splitting each class into two disjoint sets: m training images are
drawn randomly for the first, a set of 50 are taken from thesecond. As we focus on learning from small samples, we
varied m from 1 to 6, repeating the experiments 10 times
for each value and using different sets of training and test
images. To get a reliable estimate of the performance on
all the categories, we used a leave-one-class-out approach,considering in turn each class for adaptive learning and using all the rest as prior knowledge.
We used the pre-computed features of [10] which the authors made available on their website2 . Specifically, weused four different image descriptors: PHOG Shape Descriptors [2], Appearance Descriptors [14], Region Covariance [23] and Local Binary Patterns. All of the image descriptors were computed in a spatial pyramid, we considered just the first level (i.e. informations extracted from the1 http://www.idiap.ch/

˜ttommasi/source_code.html
˜pgehler/

2 http://www.vision.ee.ethz.ch/

projects/iccv09/

whole image) and combined the features using the averagekernel.
In the following we will compare the performance of ourMulti Model Knowledge Transfer aglorithm (Multi-KT) tothat obtained with a flat average mixture of prior knowledge (Average-KT) and to the method presented in [22]that we call here Single-KT. We also benchmark all the
results against No Adapt. This corresponds to learn fromscratch using weighted-LS-SVM, i.e. solving the optimization problem in Equation (10) with β = 0. The significanceof the comparisons are evaluated through the sign test [11].

5.2. Related/Unrelated Prior Knowledge
In the first set of experiments we considered different
groups of related and unrelated categories. The goal is to
study how Multi-KT chooses the reliable prior knowledge,
and its impact on performance.
Related Classes. We considered two sets of 6 classes
belonging respectively to the Caltech-256 general classes“transportation, ground, motorized” ( bulldozer, firetruck,
motorbikes, schoolbus, snowmobile, car-side) and “foodedibles” (cake, hamburger, hot-dog, ice-cream-cone,
spaghetti, sushi). Figure 2(a)-(d) show the respective classification results. In both cases we see that all KT algorithms
obtain an impressive advantage over starting from scratch.
As Figure 2(c)-(f) shows, Multi-KT performs clearly better
than Single-KT, with (p < 0.02) for less than four images
in both cases. This confirms the intuition that it pays off
to transfer from multiple sources, as opposed to one, when
they all bring useful information. There is no significant
difference in accuracy between Multi-KT and Average-KT(Figure 2(b)-(e)). This suggests that, when all prior knowledge is useful, learning the weights does not give a real advantage over a flat average.
Mixed Classes. To consider what happens in a more confused situation, we selected the following 10 classes: dog,
horse, zebra, helicopter, fighter-jet, motorbikes, car-side,
dolphin, goose and cactus. The classification results in Figure 2(g) show that here Multi-KT performs better both than
Average-KT and Single-KT (Figure 2(h)-(i), in both cases
p < 0.02 for less than four images). This experiment illustrates very clearly the power of our approach: when theprior knowledge is partially related to the new class, transferring from only one model does not exploit fully previous
experience. At the same time, using acritically all the priorknowledge induces partial negative transfer behaviours, that
affect the overall performance. Notice that the situation ofknowledge transfer from a mixture of related and unrelatedclasses is the most common.
We can also compare Multi-KT to Single-KT in terms ofstability. Let us consider the unique β used by Single-KT as
an element of the β vector where all the remaining elements
are zero. There are 6 steps in time corresponding to a new

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

(i)

Figure 2. (a-d-g) Classification performance as a function of the number of object training images, when learning respectively one out of six
related categories “transportation, ground, motorized”, “food, edibles”, and one out of ten mixed categories. The results shown correspondto average recognition rate over the categories, considering each class-out experiment repeated 10 times; (b-e-h) average difference inclassification performance ± stand. dev., obtained by Multi-KT with respect to Average-KT; (c-f-i) average difference in classification
performance ± stand. dev., obtained by Multi-KT with respect to Single-KT.

positive sample entering the training set. For each couple ofsubsequent steps we calculated the difference between theobtained β vectors of Single-KT. We did the same with theβ vectors produced by the Multi-KT algorithm. Figure 3
shows the average norm of these differences. It is evident
that choosing a combination of the prior known models fortransfer learning is more stable (lower average variations inthe β vectors) than relying on just a single known category.

5.3. Increasing Prior Knowledge
Here we studied how performance varies when the number of known category grows. We are especially interested
in how Multi-KT behaves when learning from a single pos-Figure 3. Norm of the differences between two β vectors correspondent to two subsequent steps in time. The norms are averaged
both on the classes and on the splits. These results are obtained
considering 10 randomly chosen classes.

(a)

(b)

Figure 4. (a) One-shot learning performance of Multi-KT, Average-KT and Single-KT respect to No Adapt when varying the number ofprior known categories; (b) classification performance as a function of the number of training images when learning on 30 object categories.
The results correspond to average recognition rate over the 30 categories (each class out repeated 10 times), we run this experiment 3 times,
the error bars denote ± standard deviation.

itive image (one-shot learning). We selected 30 classes3 ,extracting 3 visually related classes from 10 general categories of Caltech-256. We run six set of experiments,
considering 3/5/7/10/15/20 categories plus a final one withall the 30 categories. We first extracted three categories
through random selection and then we went on adding newones till covering the whole 30 class dataset. We repeatedthe experiments three times: Figure 4(a) shows the average
recognition rate and the corresponding standard deviations
when training only on one object image. We expect thatthe overall performance will increase along with the number of stored models, since there is a larger probability tohave stored useful prior knowledge. This intuition is confirmed by the increasing accuracy of the one-shot learningfor Multi-KT. Average-KT shows a decreasing behaviour,
indicating that as the prior knowledge grows, the number of
unrelated classes in memory usually outnumbers the related
one. The performance of Single-KT is more or less constant
except for an evident jump in performance passing from 3
to 5 categories. Finally, Figure 4(b) shows the average classification results in case of 30 categories. It is evident here
that, when learning from few samples (≤ 4), Multi-KT outperforms both Average-KT and Single-KT. These results,
jointly with those reported in the previous section, makeus conclude that Multi-KT is the most effective knowledgetransfer algorithm, compared to Average-KT and Single-KT.

5.4. Comparison with previous work
The most famous one-shot learning algorithm in thecomputer vision literature is [7, 8], where the authors ex3 “transportation, ground, motorized”:car-side, fire-truck, motorbike;“animal,land”: dog, horse, zebra; “animal,water”: goldfish, dolphin,killer-whale; “transportation, water”: canoe, kayak, speed-boat; “music,
stringed”: electric-guitar, harp, mandolin; “food, containers”: beer-mug,coffee-mug, teapot; “transportation, air”: airplanes, helicopter, fighter-jet;“animals, air”’: duck, goose, swan; “plants”: bonsai, cactus, fern; “structures, buildings”: light-house, windmill, smokestack.

Figure 5. Classification performance as a function of the numberof object training images, when learning one out of four unrelated
categories. The results showed correspond to average recognition
rate over the four categories, considering each class-out experiment repeated 10 times.
Algorithm

Error Rate(%) on 5
pos. images

Best Rec. Rate(%) on 1
pos. image

Multi-KT

8-29

airplanes: 90.8

Single-KT [22]10-29

airplanes: 88.1

Remarks+6 backgr.
images
+6 backgr.
images[7]
8-22
faces: 82.0
Table 1. Comparison between our Multi-KT algorithm, Single-KT[22], and the Bayesian One-Shot learning method presented in[7]. Since both Multi-KT and Single-KT are a discriminative approaches, besides the positive samples we need few backgroundimages in the training set.

tract a “general knowledge” from previously learned categories. Their approach makes no assumptions on the reliability of prior knowledge, which is always considered as
an average of all the known classes. To compare against
this method, we repeated the four classes experiment presented in [7]. Unfortunately it was not possible to reproduce exactly their experimental setting, as the features used
are no more available4 , and the algorithm was not publicly
4 L. Fei Fei, personal communication.

released. We opted therefore for benchmarking our results
against those reported in Table 1 in [7].
We considered the classes faces, motorbikes, leopards(originally spotted-cats) and airplanes. The average recognition rate over the categories as a function of the number ofobject training images is shown in Figure 5. Table 1 compares the results of Multi-KT and Single-KT to that reported
in [7] considering also the best one-shot result per class.
This analysis confirms that our method performs better thanSingle-KT, and it obtains results comparable to [7].

6. Conclusions
We presented an SVM-based method for learning object categories from few examples. The algorithm transfers prior knowledge selecting a subset of the known classes
and weighting them appropriately. It decides automaticallyfrom where and how much to transfer, adapting the oldmodels to the incoming data and solving a convex optimization problem which minimizes an estimate of the generalization error. Experiments show that it outperforms both theresults obtained in [22] and those produced using an averageof all the previous experience. This last choice can induce
negative transfer, in particular when the number of knowncategory increases. On the contrary, when prior knowledge
grows our method shows a one-shot learning behaviour. By
using the features provided in [10] and making available ourcode we are offering to the community a reproducible experimental setting that can be used in the future to test new
knowledge transfer algorithms. By using several features
we also showed that the behaviour of the method is not affected by the feature’s choice. Future work will investigate
ways to reduce the computational complexity of the algorithm for large number of known categories and analyze its
asymptotical behaviour when the number of training samples increases.

7. Acknowledgments
This work was supported by the EU project DIRAC(FP6-0027787), by the EMMA project thanks to the Hasler
foundation (www.haslerstiftung.ch) (T.T. & B.C), and by
the PASCAL Network of Excellence (F.O.). We are thankful to Nicolò Cesa-Bianchi for his helpful comments andsuggestions.

References[1] I. Biederman. Recognition-by-components: A theory of human image understanding. Psychological Review, 1987.
[2] A. Bosch, A. Zisserman, and X. Munoz. Representing shapewith a spatial pyramid kernel. In CIVR, 2007.
[3] O. Bousquet and A. Elisseeff. Stability and generalization.
JMLR, 2:499–526, Mar 2002.

[4] G. Cawley. Leave-one-out cross-validation based model selection criteria for weighted LS-SVMs. In IJCNN, 2006.
[5] N. Cristianini and J. Shawe-Taylor. An Introduction to Support Vector Machines. Cambridge University Press, 2000.
[6] M. Everingham, L. V. Gool, C. K. I. Williams, J. Winn,
and A. Zisserman.
The PASCAL Visual Object
Classes Challenge 2008 Results.
http://www.pascalnetwork.org/challenges/VOC/.
[7] L. Fei-Fei, R. Fergus, and P. Perona. A bayesian approachto unsupervised one-shot learning of object categories. In
ICCV, 2003.
[8] L. Fei-Fei, R. Fergus, and P. Perona. One-shot learning ofobject categories. PAMI, 28:594–611, 2006.
[9] M. Fink. Object classification from a single example utilizing class relevance metrics. In NIPS, 2004.
[10] P. Gehler and S. Nowozin. On feature combination for multiclass object classification. In ICCV, 2009.
[11] J. Gibbons. Nonparametric Statistical Inference. New York:
Marcel Dekker, 1985.
[12] G. Griffin, A. Holub, and P. Perona. Caltech 256 object category dataset. Technical Report UCB/CSD-04-1366, California Institue of Technology, 2007.
[13] D. Hush, P. Kelly, C. Scovel, and I. Steinwart. QP algorithms with guaranteed accuracy and run time for support
vector machines. JMLR, 2006.
[14] D. Lowe. Distinctive image features from scale-invariant
keypoints. IJCV, 60(2):91–110, 2004.
[15] F. Orabona, C. Castellini, B. Caputo, E. Fiorilla, and G. Sandini. Model adaptation with least-squares SVM for handprosthetics. In ICRA, 2009.
[16] S. J. Pan and Q. Yang. A survey on transfer learning. Technical report, Hong Kong University of Science and Technology, Hong Kong, China, November 2008.
[17] A. Quattoni, M. Collins, and T. Darrell. Transfer learning forimage classification with sparse prototype representations.
CVPR, 2008.
[18] M. Rosenstein, Z. Marx, and L. P. Kaelbling. To transfer
or not to transfer. In NIPS Workshop on Transfer Learning,
2005.
[19] E. Soria, J. Martin, R. Magdalena, M. Martinez, and A. Serrano. Handbook of Research on Machine Learning Applications, chapter L. Torrey and J. Shavlik, Transfer Learning.
IGI Global, 2009.
[20] J. Suykens, T. V. Gestel, J. D. Brabanter, B. D. Moor, andJ. Vanderwalle. Least Squares Support Vector Machines.
World Scientific, 2002.
[21] S. Thrun. Is learning the n-th thing any easier than learning
the first? In NIPS, 1996.
[22] T. Tommasi and B. Caputo. The more you know, the lessyou learn: from knowledge transfer to one-shot learning ofobject categories. In BMVC, 2009.
[23] O. Tuzel, F. Porikli, and P. Meer. Human detection via classification on riemannian manifolds. In CVPR, 2007.
[24] P. Wu and T. G. Dietterich. Improving SVM accuracy bytraining on auxiliary data sources. In ICML, 2004.
[25] A. Zweig and D. Weinshall. Exploiting object hierarchy:
Combining models from different category levels. In ICCV,2007.