EASY TRANSFER LEARNING BY EXPLOITING INTRA-DOMAIN STRUCTURES
Jindong Wang1 , Yiqiang Chen1,? , Han Yu2 , Meiyu Huang3 , Qiang Yang41

Beijing Key Lab. of Mobile Computing and Pervasive Device, Inst. of Computing Tech., CAS
School of Computer Science and Engineering, Nanyang Technological University, Singapore
3
Qian Xuesen Lab. of Space Tech., China Academy of Space Tech., Beijing, China
4
Hong Kong University of Science and Technology, Hong Kong, China{wangjindong, yqchen}@ict.ac.cn, han.yu@ntu.edu.sg, huangmeiyu@qxslab.cn, qyang@cse.ust.hk
2

ABSTRACTTransfer learning aims at transferring knowledge from a welllabeled domain to a similar but different domain with limited
or no labels. Unfortunately, existing learning-based methods
often involve intensive model selection and hyperparametertuning to obtain good results. Moreover, cross-validation is
not possible for tuning hyperparameters since there are often
no labels in the target domain. This would restrict wide applicability of transfer learning especially in computationallyconstraint devices such as wearables. In this paper, we propose a practically Easy Transfer Learning (EasyTL) approach
which requires no model selection and hyperparameter tuning, while achieving competitive performance. By exploiting intra-domain structures, EasyTL is able to learn both nonparametric transfer features and classifiers. Extensive experiments demonstrate that, compared to state-of-the-art traditional and deep methods, EasyTL satisfies the Occam’s Razor principle: it is extremely easy to implement and use whileachieving comparable or better performance in classification
accuracy and much better computational efficiency. Additionally, it is shown that EasyTL can increase the performance ofexisting transfer feature learning methods.
Index Terms— Transfer Learning, Domain Adaptation,
Cross-domain Learning, Non-parametric Learning
1. INTRODUCTION
The success of multimedia applications depends on the availability of sufficient labeled data to train machine learningmodels. However, it is often expensive and time-consuming
to acquire massive amounts of labeled data. Transfer learning(TL), or domain adaptation [12] is a promising strategy to enhance the learning performance on a target domain with few
or none labels by leveraging knowledge from a well-labeled
source domain. Since the source and target domains have difCorresponding author: Y. Chen. J. Wang and Y. Chen are also affiliatedwith University of Chinese Academy of Sciences, Beijing 100190, China.
978-1-5386-1737-3/18/$31.00 2019 IEEE.

Transfer learning tasks

From existing transfer learning to EasyTL
Begin
For

do

For

do

Transfer
End ForEnd ForEnd

Existing Work

Begin
EasyTLEnd

Easy Transfer Learning

Fig. 1. A brief illustration of EasyTL
Table 1. Comparison between EasyTL and recent TL methods on Office-Home [19] dataset
MethodAccuracy
Hyperparameter
Network

MEDA [23]60.3λ, η, pDANN [5] CDAN [8]
57.6
62.8
λ
λ
ResNet50

EasyTL
63.3
None

ferent distributions, numerous methods have been proposed
to reduce the distribution divergence [24, 23, 21, 26, 6].
Unfortunately, despite the great success achieved by existing TL methods, it is notoriously challenging to apply them toa real situation since we cannot determine the best TL modeland their optimal hyperparameters. The reasons are threefold. Firstly, most traditional and deep TL methods are parametric methods [23, 24, 26, 5] that have to go through an intensively expensive and time-consuming process to tune a lotof hyperparameters (e.g. Fig. 1; MEDA, DANN, and CDAN
in Table 1). Secondly, cross-validation, which is the mostcommon strategy to select models and tune hyperparameters,
is not available in TL since there are often no labeled data inthe target domain [12]. Thirdly, although the recent AutoML
methods can automatically tune the hyperparameters via tree
pruning, boosting, or neural architecture search [15], they areunable to handle the different distributions between domains
in TL and typically take a long time to converge.
These challenges seriously restrict the real application ofTL, especially on small devices that require instant local computing with limited resources such as wearables. Is it possible
to develop an easy but powerful TL algorithm to circumventmodel selection and parameter tuning, but have competitive
performance, i.e. satisfying Occam’s Razor principle [16]?
In this paper, we make the first attempt towards addressing this challenge by proposing a practically Easy Transfer Learning (EasyTL) approach. EasyTL is able to perform knowledge transfer across domains without the need formodel selection and hyperparameter tuning (Table 1). By
exploiting intra-domain stuctures, EasyTL learns both nonparametric transfer features by intra-domain alignment andtransfer classifier by intra-domain programming. Thus, it isable to avoid negative transfer [12]. Furthermore, EasyTL
can also increase the performance of existing TL methods byserving as their final classifier via intra-domain programming.
We conduct extensive experiments on public TL datasets.
Both visual domain adaptation and cross-domain sentiment
analysis demonstrated significant superiority of EasyTL inclassification accuracy and computational efficiency overstate-of-the-art traditional and deep TL methods. In short,
EasyTL has the following characteristics:• Easy: EasyTL is extremely easy to implement and use.
Therefore, it eliminates the need for model selection or
hyperparameter tuning in transfer learning.
• Accurate: EasyTL produces competitive results in several popular TL tasks compared to state-of-the-art traditional and deep methods.
• Efficient: EasyTL is significantly more efficient thanother methods. This makes EasyTL more suitable forresource-constrained devices such as wearables.
• Extensible: EasyTL can increase the performance ofexisting TL methods by replacing their classifier withintra-domain programming.

2. RELATED WORK
EasyTL significantly differs from existing work in the following three aspects:
Transfer learning. Existing TL methods can be summarized into two main categories: (a) instance reweighting, which reuses samples from the source domain according to some weighting technique; and (b) feature transformation, which performs subspace learning or distribution adaptation [24, 23, 13, 17, 20]. Unfortunately, these methods
are all parametric approaches. They depend on extensive
hyperparameter tuning through cross-validation for the feature transformation [13, 24], or the prediction model [6, 17],or both [20, 23]. Most methods require multiple iterationsof training [20, 24]. The L2T framework [25] is similar to
EasyTL in spirit, but L2T is still based on model iterationand parameter tuning. Deep TL methods [8, 9, 19, 5] require
heavy hyperparameter tuning. In TL, cross-validation is often
not available since there are almost no labeled data in the target domain [12]. In contrast, EasyTL is a non-parametric TL
approach that directly learns from intra-domain structures,

which requires no model selection and hyperparameter tuning and much more efficient than existing methods.
Non-parametric learning. Nearest-neighbor (NN) classifier is the most common non-parametric method. However,NN computes the distance between each sample in two domains, which is more likely to be influenced by domain shift.
Naive Bayes NN (NBNN) classifier is used for domain adaptation [18], which still requires hyperparameter tuning anditerations. Nearest Centroid (NC) classifier is based on the
distance between each target sample to the class center of thesource domain. EasyTL uses a linear programming to get thesoftmax probabilities (float weights), while NC is basically
0/1 weights. This means that EasyTL could not only consider
the relationship between sample and center, but also the relations of other samples. Open set DA [14] has similar ideawith EasyTL, while it solves a binary programming and requires other classifiers.
Automated machine learning. Recent years have witnessed the advance of Automated Machine Learning (AutoML) [15]. AutoML produces results without model selection and parameter tuning. However, no existing AutoML
frameworks can handle TL tasks since they assume the training and test data are in the same distribution. AutoDIAL [10]is similar to EasyTL, which includes automatic domain alignment layers for deep networks. However, AutoDIAL still requires many parameters in the network to be tuned.
3. EASY TRANSFER LEARNING
We follow the most common TL settings in existing work [23,24, 26]. We are given a labeled source domain Ωs =st{(xsi , yis )}ni=1
and an unlabeled target domain Ωt = {xtj }nj=1.
The feature space Xs = Xt , label space Ys = Yt , but
marginal distribution Ps (xs ) 6= Pt (xt ) with conditional distribution Qs (ys |xs ) 6= Qt (yt |xt ). The goal is to predict thelabels yt ∈ Yt for the target domain.
3.1. Motivation
It is challenging to design a TL method that requires no model
selection and hyperparameter tuning. On one hand, a simple
NN (Nearest Neighbor) classifier may suffice, while NN suffers in handling the distribution divergence between domains.
On the other hand, existing TL methods such as GFK [6]and BDA [20] could reduce the distribution divergence, while
they require tuning a lot of hyperparameters. Combining bothof them without considering domain structures may easily result in negative transfer [12], which dramatically hurts theperformance of TL. Currently, there is no such effort.
In this work, we propose a practically Easy Transfer
Learning (EasyTL) approach to learn non-parametric transfer features and classifier with competitive performance toexisting heavy methods. In short, to be consistent with Occam’s Razor principle [16]. In light of the recent advance濦瀂瀈瀅濶濸

瀖瀖濧濴瀅濺濸瀇

Intra-domain
Programming

Intra-domain
Alignment

Fig. 2. The procedure of EasyTL. The colored boxes and circles denote samples from the source and target domains. The
red line denotes the classifier.
in representation learning [7], incorporating knowledge of locality can greatly improve the representations quality. Therefore, Instead of learning sample-wise distance, EasyTL focuses on exploiting the intra-domain structure. The mainpart of EasyTL is a novel non-parametric Intra-domain programming classifier, while remains open for adopting existing
methods for Intra-domain alignment. Intra-domain programming is able to learn discriminative transfer information fromdomains. Fig. 2 shows the procedure of EasyTL. In the following sections, we first introduce the proposed intra-domain
programming. Then, we show how to adapt a non-parametricfeature learning method for intra-domain alignment.

j

瀖瀖

濃濁濅

澤澢澨

濃濁濄

瀖瀖

濃濁濆

濃濁濅

濃濁濅

瀖瀖

澤澢澨

濃濁濅

濃濁濅

瀖瀖

The class center hc on Ωs can be calculated as:
hc =

1

ns
X

(c)
|Ωs |

i

xsi · I(yis = c),(3)where I(·) is an indicator function which evaluates to 1 if thecondition is true, and 0 otherwise.
Consider the constraints to minimize the cost function in
Eq. (1). Firstly, note that the value of Mcj is a probability
measuring the confidence of xtj belonging to class c, suchthat the sum of the probability of one particular sample xtj
belonging to all existing classes is 1. This is ensured by thefollowing constraint:C
X

Mcj = 1, ∀j ∈ {1, · · · , nt }.

(4)Secondly, since Ωs and Ωt have the same label space (i.e.
Ys = Yt ), there must be at least one sample for any givenclass c. This is ensured by the following constraint:
Mcj = arg max Mrj , r ∈ {1, · · · , C}, ∀c ∈ {1, · · · , C}, ∃j.
r(5)
In fact, the ideal state of Mcj should be a binary value (0or 1), i.e. Mcj = 1 iff xj belongs to class c, otherwise Mcj =0. Therefore, we use the following formulation to replace
Eq. (5) in the computation without affecting the results:nt
X

Mcj ≥ 1, ∀c ∈ {1, · · · , C}.

Learning objective. Combining the cost function in
Eq. (1) and the constraints in Eq. (4) and Eq. (6), the final
learning objective becomes:

(1)

nt X
C
X
j

c

where the distance value Dcj is an entry in a distance matrix
D. Dcj denotes the distance between xtj and the c-th class(6)j

min J =
Dcj Mcj ,澤澢澩

cIntra-domain programming directly learns a transfer classifier for TL problem and provides reliable likelihood information for intra-domain alignment. We introduce the concept of the Probability Annotation Matrix, based on which we
can build the non-parametric transfer classifier. Formally, letc ∈ {1, · · · , C} denote the class label, a matrix M ∈ RC×nt
with its element 0 ≤ Mcj ≤ 1 is a probability annotation
matrix. Here, the entry value Mcj of M indicates the annotation probability of xtj belonging to class c. Fig. 3 illustrates
the main idea of M. There are 4 classes and nt target samples
with example activation values. Similar to the widely-adopted
softmax classifier in neural networks, the highest probability
value for xt1 is 0.4, which indicates that it belongs to C4 . The
same goes for xt2 and xt3 , etc. The real probability annotation
values are to be learned by our proposed approach.
Instead of learning yt directly, the algorithm focuses onlearning the probability annotation matrix M. In this way, thecost function can be formalized as:
J =

濃濁濅

Fig. 3. An example of the probability annotation matrix.

3.2. Intra-domain Programming

nt X
C
X

濃濁濄

s.t.

Dcj Mcj .

c
0P≤ Mcj ≤ 1
Cc Mcj = 1, ∀j ∈ {1, · · · , nt }P ntj Mcj ≥ 1, ∀c ∈ {1, · · · , C}

(7)

(c)

center of the source domain Ωs .
(c)
We denote hc as the c-th class center of Ωs . Then, Dcj
can be computed by the Euclidean distance:
Dcj = ||xtj − hc ||2 .

(2)

Solving Eq. (7) requires us to solve a linear programming (LP) problem. There are existing solution packages
such as PuLP 1 for solving the above linear programming
1 https://pypi.python.org/pypi/PuLP/1.1

problem efficiently. Then, M can be obtained. Eventually,
the label of xtj is given by the softmax function:
Mrj
yjt = arg max PCr
c Mcj

for r ∈ {1, · · · , C}.

(8)

It is noticeable that this classifier does not involve any parameters to tune explicitly. This is significantly different fromwell-established classifiers such as SVM that needs to tune
numerous hyperparameters. In fact, intra-domain programming can be used alone for TL problems.
3.3. Intra-domain Alignment
Intra-domain alignment serves as the transfer feature learning methods for EasyTL. Although EasyTL can be used directly in TL with good performance, it can also be combinedwith transfer feature learning to eliminate feature distortionsin the original space. On the other hand, existing transfer feature learning methods can also be extended using the previous
intra-domain programming to enhance their performance.
Recall our inspiration that learning the structure of locality will help learn useful representations [7]. Therefore, if
we perform personalized transfer feature learning within eachsubspace, we could further reduce the domain divergence. For
simplicity and efficiency, we only perform feature learningfrom the source domain to the subspace of the target domain.
Inspired by the non-parametric feature learning method
CORAL [17], the intra-programming process of EasyTL canbe formulated as:(
11
xr · (cov(xs ) + Es )− 2 · (cov(xt ) + Et ) 2 if r = srz =xr
if r = t(9)where cov(·) is the covariance matrix. Es and Et are identity
matrices with equal sizes to Ωs and Ωt , respectively. We cantreat this step as a re-coloring process of each subspace [17].
Eq. (9) aligns the two distributions by re-coloring whitenedsource features with the covariance of target distributions.
Remark: Other than CORAL, EasyTL can also choose
other popular methods for feature learning such as BDA [20]and GFK [6]. We choose CORAL for its computational efficiency and that it contains no other parameters to tune.
3.4. Computational Complexity
We use the big-O notation to analyze the complexity of
EasyTL. Intra-domain alignment takes at most O(Cn3s ). The
complexity of intra-domain programming is O(n3t C 3 ) giventhat it is a linear programming problem (Eq. (7)) [11].
EasyTL can be made more efficient with the low-rank representations and other fast computing algorithms.
As an intuitive comparison of classifiers, we compare
intra-domain programming with the well-established SVM. A
single round of SVM training and prediction takes O((ns +Algorithm 1 EasyTL: Easy Transfer Learning
Input: Feature matrix xs , xt for Ωs and Ωt , respectively; andlabel vector ys for Ωs
Output: Predicted label vector yt for target domain.
1: (Optional) Perform intra-domain alignment via Eq. (9)2: Solve Eq. (7) to obtain the probability annotation matrix
M and compute yt using Eq. (8)3: return Label vector yt for Ωtnt )3 ) computations [1]. This implies that the time complexityof intra-domain programming is comparable to a single roundof SVM. However, SVM still needs a couple of rounds for
hyperparameter tuning (e.g. kerneltype, constraints) before
getting optimal performance. Therefore, EasyTL is theoretically more efficient than SVM. We will experimentally show
the efficiency of EasyTL in the next sections.
4. EXPERIMENTAL EVALUATION
4.1. Experimental Setup
We use four popular TL datasets: 1) Amazon Review [2] isa cross-domain sentiment analysis dataset that contains positive and negative reviews of four kinds of products: Kitchen
appliance (K), DVDs (D), Electronics (E), and Books (B).
2) Office-Caltech [6] contains 10 common classes of images
in Amazon (A), DSLR (D), Webcam (W), and Caltech (C).
3) Image-CLEF DA [9] contains 12 categories of images belonging to 3 domains: Caltech (C), ImageNet (I), and Pascal (P). 4) Office-Home [19] contains 15,500 images of 65
categories from 4 domains: Art (Ar), Clipart (Cl), Product (Pr), and Real-world (Rw). Within each dataset, any two
domains can be source and target to construct TL tasks.
State-of-the-art traditional and deep comparison methods: NN (Nearest Neighbor), SVM with linear kernel [17],TCA (Transfer Component Analysis) [13], GFK (Geodesic
Flow Kernel) [6], SA (Subspace Alignment) [4], CORAL(CORrelation ALignment) [17], BDA (Balanced Distribution Adaptation) [20], JGSA (Joint Geometrical and Statistical Alignment) [26], D-GFK [24], ResNet50, DANN(Domain-adversarial Neural Networks) [5], JAN (Joint Adaptation Networks) [9], and CDAN (Conditional Adversarial
Adaptation Networks) [8]. The source code of EasyTL isavailable at http://transferlearning.xyz/code/
traditional/EasyTL.
Specifically, we use EasyTL(c) to denote intra-domain
programming since it can serve as a TL method alone, while
EasyTL denotes the full method. By following the standard protocol in [17], we adopt linear SVM for traditional
TL methods. For the amazon dataset, we use the 400dimensional features [22]. For image datasets, we use the2048-dimensional ResNet50 finetuned features [22]. For
Office-Caltech datasets, we adopt the SURF features [24].

Table 2. Accuracy (%) on Image-CLEF DA and Office-Home datasets using ResNet features
ID
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
-

Task
C→I
C→P
I→C
I→P
P→C
P→I
AVG
Ar → Cl
Ar → Pr
Ar → Rw
Cl → Ar
Cl → Pr
Cl → Rw
Pr → Ar
Pr → ClPr → Rw
Rw → Ar
Rw → Cl
Rw → Pr
AVG
Avg rank [3]ResNet
78.0
65.5
91.5
74.8
91.2
83.9
80.7
34.9
50.0
58.0
37.4
41.9
46.2
38.5
31.2
60.4
53.9
41.2
59.9
46.1
12

1NN
83.5
71.3
89.0
74.8
76.2
74.0
78.1
45.3
60.1
65.8
45.7
57.0
58.7
48.1
42.9
68.9
60.8
48.3
74.7
56.4
7

SVM
86.0
73.2
91.2
76.8
85.8
80.2
82.2
45.3
65.4
73.1
43.6
57.3
60.2
46.8
39.1
69.2
61.1
45.6
75.9
56.9
6

TCA
89.3
74.5
93.2
77.5
83.7
80.8
83.2
38.3
58.7
61.7
39.3
52.4
56.0
42.6
37.5
64.1
52.6
41.7
70.5
51.3
9

GFK
86.3
73.3
93.0
75.5
82.3
78.0
81.4
38.9
57.1
60.1
38.7
53.1
55.5
42.2
37.6
64.6
53.8
42.3
70.6
51.2
10

BDA
90.8
73.7
94.0
75.3
83.5
77.8
82.5
38.9
54.8
58.2
36.2
53.1
50.2
42.1
38.2
63.1
50.2
44.0
68.2
49.8
11

Table 3. Accuracy (%) on Amazon Review dataset
Method
B→D
B→E
B→K
D→B
D→E
D→K
E→B
E→D
E→K
K→B
K→D
K→E
AVG

1NN
49.6
49.8
50.3
53.3
51.0
53.1
50.8
50.9
51.2
52.2
51.2
52.3
51.3

TCA
63.6
60.9
64.2
63.3
64.2
69.1
59.5
62.1
74.8
64.1
65.4
74.5
65.5

GFK
66.4
65.5
69.2
66.3
63.7
67.7
62.4
63.4
73.8
65.5
65.0
73.0
66.8

SA
67.0
70.8
72.2
67.5
67.1
69.4
61.4
64.9
70.4
64.4
64.6
68.2
67.3

BDA
64.2
62.1
65.4
62.4
66.3
68.9
59.2
61.6
74.7
62.7
64.3
74.0
65.5

CORAL
71.6
65.1
67.3
70.1
65.6
67.1
67.1
66.2
77.6
68.2
68.9
75.4
69.1

JGSA
66.6
75.0
72.1
55.5
67.3
65.6
51.6
50.8
55.0
58.3
56.4
51.7
60.5

EasyTL(c)
78.4
77.5
79.2
79.5
77.4
80.4
73.0
73.1
84.6
75.2
73.8
82.0
77.8

CORAL
83.0
71.5
88.7
73.7
72.0
71.3
76.7
42.2
59.1
64.9
46.4
56.3
58.3
45.4
41.2
68.5
60.1
48.2
73.1
55.3
8

DANN
87.0
74.3
96.2
75.0
91.5
86.0
85.0
45.6
59.3
70.1
47.0
58.5
60.9
46.1
43.7
68.5
63.2
51.8
76.8
57.6
5

JAN
89.5
74.2
94.7
76.8
91.7
88.0
85.8
45.9
61.2
68.9
50.4
59.7
61.0
45.8
43.4
70.3
63.9
52.4
76.8
58.3
4

CDAN
91.2
77.2
96.7
78.3
93.7
91.2
88.1
46.6
65.9
73.4
55.7
62.7
64.2
51.8
49.1
74.5
68.2
56.9
80.7
62.8
2

EasyTL(c)
85.5
72.0
93.3
78.5
91.0
89.5
85.0
51.6
68.1
74.2
53.1
62.9
65.3
52.8
45.8
73.5
62.2
50.2
76.0
61.3
3

EasyTL
91.5
77.7
96.0
78.7
95.0
90.3
88.2
52.8
72.1
75.9
55.0
65.9
67.6
54.4
46.9
74.7
63.8
52.3
78.0
63.3
1

Table 4. Average rank, parameter, and running time
EasyTL
79.8
79.7
80.9
79.9
80.8
82.0
75.0
75.3
84.9
76.5
76.3
82.5
79.5

Deep TL methods are only used in image datasets whose results are obtained from existing work. Classification accuracy
is acting as the evaluation metric [13, 6, 17].
4.2. Results: Accuracy and Efficiency
The results on Amazon Review is shown in Table 3. Table 2
shows the results on ImageCLEF DA (IDs 1∼6) and OfficeHome (IDs 7∼18). Results on Office-Caltech are in the code
page due to space limit. We also report the average rank [3],parameter and running time (train+test) of several methods
in Table 4. The results demonstrate that EasyTL outperforms all comparison methods in both sentiment and image
data. EasyTL(c) can also achieve competitive performance
than most methods. Note that except 1NN and EasyTL, othermethods all require hyperparameter tuning. It clearly indicates the superiority of EasyTL in accuracy and efficiency.
There are more insightful findings. 1) On the larger andmore challenging Office-Home dataset, the performance of
EasyTL(c) is only slightly worse than CDAN, while it outperforms all other comparison methods. Compared to deep TL
methods (DANN, JAN, and CDAN), although EasyTL takes
finetuned features as inputs, it only requires one distinct fine-

MethodAvg rank
Parameter
Time (s)TCA
9d, λ, C, γ
119

GFK
10
d, C, γ
127.4

CORAL
8
C, γ
126.5

DANN
4
λ
>1000

CDAN
2
λ
>1000

EasyTL(c)
3
None
23.8

EasyTL1
None
59.6

tuning process while deep methods have to run the network
multiple times to get optimal parameters. 2) Moreover, in
real applications, it is rather difficult and almost impossible tosearch parameters since there are often little or none labeleddata in the target domain. It seems that adversarial learning
provides little contribution on TL compared to the power ofResNet in representation learning, and it is rather difficult totrain an adversarial network. 3) The results imply that the “2stage” procedure (finetune+EasyTL) is better than “1-stage”
methods (deep TL) even if they are end-to-end. All methods
have their advantages and disadvantages. 4) EasyTL tends toperform consistently well on rather balanced datasets (Amazon Review and Image-CLEF DA). While on the unbalanced
Office-Caltech dataset, the performance of EasyTL is limited.
The research of improving EasyTL for unbalanced datasets
remains as future research.
4.3. Evaluation of Extensibility
We evaluate the extensibility of EasyTL by extending existing TL methods with the classifier of EasyTL. We use PCA,TCA [13], GFK [6], and CORAL [17] for intra-domain alignment and compare the performance of +SVM and +EasyTL
in Fig. 4. The results show that: 1) By using feature learning
methods other than CORAL, EasyTL can still achieve comparable performance to existing approaches. 2) More importantly, EasyTL does not rely on any particular feature learning methods to achieve good performance. 3) EasyTL can
increase the performances of existing TL methods. It clearly
implies the extensibility of EasyTL in transfer learning.

90

65+EasyTL+SVM

Accuracy (%)

Accuracy (%)

+SVM

85

80

75

PCA

TCA

GFK

CORAL

[2] M. Chen, Z. Xu, K. Weinberger, and F. Sha. Marginalized
denoising autoencoders for domain adaptation. In ICML, 2012.

+EasyTL

60[3] J. Demšar. Statistical comparisons of classifiers over multiple
data sets. JMLR, 7(Jan):1–30, 2006.

55

50[4] B. Fernando et al. Unsupervised visual domain adaptation using subspace alignment. In ICCV, pages 2960–2967, 2013.
PCA

TCA

GFK

MethodMethod(a) Image-CLEF DA(b) Office-Home

CORAL[5] Y. Ganin and V. Lempitsky. Unsupervised domain adaptation
by backpropagation. In ICML, pages 1180–1189, 2015.

Fig. 4. Extending existing methods with EasyTL[6] B. Gong, Y. Shi, et al. Geodesic flow kernel for unsupervised
domain adaptation. In CVPR, pages 2066–2073, 2012.

5. CONCLUSIONS AND FUTURE WORK

[7] R. D. Hjelm et al. Learning deep representations by mutual
information estimation and maximization. In ICLR, 2019.

In this paper, we propose EasyTL as the first non-parametrictransfer learning approach by exploiting intra-domain structures features to learn non-parametric transfer features andclassifiers. Thus, it requires no model selection and hyperparameter tuning. EasyTL is easy, accurate, efficient, and extensible, and can be applied directly in the resource-constrained
devices such as wearables. Experiments demonstrate the superiority of EasyTL in accuracy and efficiency over several
popular traditional and deep TL methods. EasyTL can also
increase the performance of other TL methods.
EasyTL opens up some new research opportunities in TL:
Practical Transfer Learning. Instead of pursuing good
performance in accuracy, we do hope that EasyTL could open
up a new door in research on practical transfer learning that
focuses more on the actual usage of the methods: parametertuning and model validation.
Automated Domain Selection. Currently, EasyTL focuseson how to transfer. In real applications, another important
problem is what to transfer. Some existing work focused on
this issue, but they also require heavy parameter tuning. We
will further explore this part by extending EasyTL.
Multi-label Transfer Learning. EasyTL can be used in
multi-label transfer learning by changing the value of probability annotation matrix. This makes EasyTL suitable formulti-label tasks. Existing multi-label transfer methods haveto modify their classifiers. EasyTL can be more advantageous
for these applications.

[8] M. Long, Z. Cao, J. Wang, et al. Conditional adversarial domain adaptation. In NeurIPS, pages 1647–1657, 2018.
[9] M. Long, J. Wang, et al. Deep transfer learning with joint adaptation networks. In ICML, pages 2208–2217, 2017.
[10] F. Maria Carlucci, L. Porzi, et al. Autodial: Automatic domainalignment layers. In ICCV, pages 5067–5075, 2017.
[11] N. Megiddo. On the complexity of linear programming. 1986.
[12] S. J. Pan and Q. Yang. A survey on transfer learning. IEEE
TKDE, 22(10):1345–1359, 2010.
[13] S. J. Pan, Q. Yang, et al. Domain adaptation via transfer component analysis. IEEE TNN, 22(2):199–210, 2011.
[14] P. Panareda Busto and J. Gall. Open set domain adaptation. In
ICCV, pages 754–763, 2017.
[15] Y. Quanming et al. Taking human out of learning applications: A survey on automated machine learning. arXiv
preprint:1810.13306, 2018.
[16] C. E. Rasmussen and Z. Ghahramani. Occam’s razor. In
NeurIPS, pages 294–300, 2001.
[17] B. Sun, J. Feng, and K. Saenko. Return of frustratingly easy
domain adaptation. In AAAI, volume 6, page 8, 2016.
[18] T. Tommasi and B. Caputo. Frustratingly easy nbnn domain
adaptation. In ICCV, pages 897–904, 2013.
[19] H. Venkateswara et al. Deep hashing network for unsupervised
domain adaptation. In CVPR, pages 5018–5027, 2017.
[20] J. Wang, Y. Chen, S. Hao, et al. Balanced distribution adaptation for transfer learning. In ICDM, pages 1129–1134, 2017.
[21] J. Wang, Y. Chen, L. Hu, et al. Stratified transfer learning forcross-domain activity recognition. In PerCom, 2018.

6. ACKNOWLEDGMENTS[22] J. Wang et al. Everything about transfer learning and domain
adaptation. http://transferlearning.xyz, 2018.

This work is supported by National Key R & D Programof China (2016YFB1001200), NSFC (61572471, 61702520),
Hong Kong CERG projects (16209715, 16244616), andNanyang Technological University, Nanyang Assistant Professorship (NAP).

[23] J. Wang, W. Feng, Y. Chen, et al. Visual domain adaptation
with manifold embedded distribution alignment. In ACM MM,
pages 402–410, 2018.

7. REFERENCES[25] Y. Wei, Y. Zhang, and Q. Yang. Transfer learning via learning
to transfer. In ICML, 2018.

[1] A. Abdiansah and R. Wardoyo. Time complexity analysis
of support vector machines in libsvm. J. Comput. Appl.,
128(3):28–34, 2015.

[24] J. Wei, J. Liang, R. He, and J. Yang. Learning discriminative
geodesic flow kernel for unsupervised domain adaptation. In
ICME, pages 1–6, 2018.

[26] J. Zhang, W. Li, and P. Ogunbona. Joint geometrical and statistical alignment for visual domain adaptation. In CVPR, 2017.