2017 IEEE International Conference on Data Mining

Balanced Distribution Adaptation for Transfer Learning
Jindong Wang∗†‡ , Yiqiang Chen∗†‡ , Shuji Hao§ , Wenjie Feng∗‡†† , Zhiqi Shen∗ Beijing Key Laboratory of Mobile Computing and Pervasive Device†† CAS Key Laboratory of Network Data Science & Technology† Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China‡ University of Chinese Academy of Sciences,§ Institute of High Performance Computing, A*STAR School of Computer Science and Engineering, Nanyang Technological University, Singapore
Email:{wangjindong,yqchen}@ict.ac.cn, haosj@ihpc.a-star.edu.sg, fengwenjie@software.ict.ac.cn, zqshen@ntu.edu.sg

achieve better performance. The work of [2], [4], [7] also
proposed several approaches to adapt the joint distribution.
However, those two distributions are often treated equallyin existing methods, while the importance of each other is
not leveraged. When the datasets are much more dissimilar,
it means the marginal distributions are more dominant;when the datasets are similar, it means the conditional
distributions needs more attention. Hence, it will deteriorate
the performance of algorithms by only adding them together
with equal weight. Therefore, how to adaptively leverage theimportance of each distribution is a critical problem.
Moreover, class imbalance often exists in many transfer
learning scenarios. When the class proportion of domains
is highly imbalanced, it needs to be considered carefullyfor distribution adaptation. Existing methods [2], [4] often
ignore this issue by treating the classes as balanced acrossdomains, or they only handle the bias on one domain [3],and this may hinder the effectiveness of transfer learning.
Therefore, how to handle the class imbalance situation intransfer learning is another important challenge.
In this paper, we propose two novel methods to tackle theabove two issues. For distribution adaptation, we propose
Balanced Distribution Adaptation (BDA). BDA can not
only adapt both the marginal and conditional distributions
between domains, but also leverage the importance of those
two distributions, thus it can be effectively adjusted to
speciﬁc transfer learning tasks. Several existing methods
can be regarded as special cases of BDA. Based on BDA,
we also propose a novel Weighted Balanced Distribution
Adaptation (W-BDA) algorithm to tackle the class imbalance issue in transfer learning. The proposed W-BDA canadaptively change the weight of each class when performing
distribution adaptation. To evaluate BDA and W-BDA, we
conduct extensive experiments on ﬁve image datasets.
To sum up, our contributions are mainly three-fold:
1) We propose a novel transfer learning method, whichis named as BDA to balance the marginal and conditional
distribution adaptation. BDA can adaptively adjust the importance of those two distances and can achieve a better
performance. Several transfer learning algorithms can be
regarded as special cases of BDA.

Abstract—Transfer learning has achieved promising resultsby leveraging knowledge from the source domain to annotate
the target domain which has few or none labels. Existing
methods often seek to minimize the distribution divergence
between domains, such as the marginal distribution, the conditional distribution or both. However, these two distances areoften treated equally in existing algorithms, which will resultin poor performance in real applications. Moreover, existing
methods usually assume that the dataset is balanced, which also
limits their performances on imbalanced tasks that are quite
common in real problems. To tackle the distribution adaptation
problem, in this paper, we propose a novel transfer learning
approach, named as Balanced Distribution Adaptation (BDA),
which can adaptively leverage the importance of the marginal
and conditional distribution discrepancies, and several existing
methods can be treated as special cases of BDA. Based on
BDA, we also propose a novel Weighted Balanced Distribution
Adaptation (W-BDA) algorithm to tackle the class imbalanceissue in transfer learning. W-BDA not only considers the
distribution adaptation between domains but also adaptively
changes the weight of each class. To evaluate the proposed
methods, we conduct extensive experiments on several transfer
learning tasks, which demonstrate the effectiveness of our
proposed algorithms over several state-of-the-art methods.
Keywords-Transfer learning, domain adaptation, distribution
adaptation, class imbalance

I. I NTRODUCTION
Preparing labeled data is crucial for training machinelearning algorithms. However, it is often expensive and timeconsuming to obtain sufﬁcient labeled data in real applications. In this case, transfer learning [1] has been a promising
approach by transferring knowledge from a labeled source
domain to the target domain. Transfer learning often assumes
the training and testing data are from similar but different
distributions [1]. For instance, the images of an object
taken in different angles, backgrounds and illuminations
could lead to different marginal or conditional distributions.
By observing this, existing transfer learning methods aremainly focusing on distribution adaptation to minimize the
distribution divergence between domains [2], [3], [4].
Most of the existing distribution adaptation methods adapt
either marginal distribution [5], conditional distribution [6]or both [2], [4]. It is shown in [2] that adapting both could
2374-8486/17 $31.00 © 2017 IEEE
DOI 10.1109/ICDM.2017.150

1129

A. Problem Deﬁnition

2) We also propose another novel method W-BDA byextending BDA to handle the class imbalance problem whichis common in transfer learning. The proposed W-BDA not
only considers the distribution adaptation of domains but
also adaptively changes the weight of each class, thus it canhandle the class imbalance problem for transfer learning.
3) We conduct extensive experiments on ﬁve image
datasets to evaluate the BDA and W-BDA methods, indicating their superiority against other state-of-the-art methods.

Given a labeled source domain {xsi , ysi }ni=1 , an unlabeled target domain {xtj }m
j=1 , and assume feature space
Xs = Xt , label space Ys = Yt but marginal distributions
Ps (xs ) = Pt (xt ) with conditional distributions Ps (ys |xs ) =Ps (yt |xt ). Transfer learning aims to learn the labels yt of
Dt using the source domain Ds .
Balanced distribution adaptation solves the transfer learning problem by adaptively minimizing the marginal andconditional distribution discrepancy between domains, andhandle the class imbalance problem, i.e. to minimize thediscrepancies between: 1) P (xs ) and P (xt ), 2) P (ys |xs )and P (yt |xt ).

II. R ELATED W ORK
Transfer learning has been widely applied to activity
recognition [8], incremental learning [9], and online learning [10], [11]. Our proposed BDA and W-BDA are mainly
related to the feature-based transfer learning methods. Thus,in this section, we present a detailed discussion on thiscategory, speciﬁcally on two aspects.
Joint distribution adaptation. [12] proposed to jointly
select feature and preserve structural properties. Long etal. [2] proposed joint distribution adaptation method (JDA) to match both marginal and conditional distribution
between domains. Others extended JDA by adding structural consistency [4], domain invariant clustering [7], andtarget selection [13]. Those methods tend to ignore theimportance between two distinct distributions by just adding
them together. However, when there is a large discrepancy
between both distributions, those methods cannot evaluate
the importance of each distribution, and may not generalize
well in most cases. Our work is capable of investigating
the importance of each distribution. Thus it can be moregeneralized to transfer learning scenarios with complex data
distributions.
Class imbalance problem. Previous sample re-weighting
methods [14] only learned weights of speciﬁc samples,but ignore the class weights balance for different classes.
[15] developed a Closest Common Space Learning (CCSL)
method to adapt the cross-domain weights. CCSL is an
instance selection method, while ours is a feature based
approach. Multiset feature learning was proposed in [16] tolearn discriminant features. [3] proposed weighted maximum
mean discrepancy to construct a source reference collection
on the target domain but it only adapted the prior of source
domain, while our method could adapt the priors from both
source and target domains. [17] tackled the imbalance issue
when target domain has some labels, while in our method,target domain has no labels. [18] adjusted the weights ofdifferent samples according to their predictions, while ourwork focuses on adjusting the weight of each class.

B. Balanced Distribution Adaptation
Transfer learning methods often seek to adapt both themarginal and conditional distributions between domains [2],[7]. Speciﬁcally, this refers to minimizing the distance
D(Ds , Dt ) ≈ D(P (xs ), P (xt ))+ D(P (ys |xs ), P (yt |xt ))(1)

However, simply matching both distributions is not enough. Existing methods usually assume they are equallyimportant, and that implicit assumption does not hold. In thissection, we propose Balanced Distribution Adaptation (BDA) to adaptively adjust the importance of both the marginal
and conditional distributions based on each speciﬁc tasks.
Concretely speaking, BDA exploits a balance factor μ toleverage the different importance of distributions:
D(Ds , Dt ) ≈ (1 − μ)D(P (xs ), P (xt ))+ μD(P (ys |xs ), P (yt |xt ))(2)

where μ ∈ [0, 1]. When μ → 0, it means the datasetsare more dissimilar, so the marginal distribution is more
dominant; when μ → 1, it reveals the datasets are similar,
so the conditional distribution is more important to adapt.
Therefore, the balance factor μ can adaptively leverage theimportance of each distribution and lead to good results.
It is worth noting that, since the target domain Dt has
no labels, it is not feasible to evaluate the conditional
distribution P (yt |xt ). Instead, we use the class conditional
distribution P (xt |yt ) to approximate P (yt |xt ). Because
P (xt |yt ) and P (yt |xt ) can be quite involved according to
the sufﬁcient statistics when sample sizes are large [2]. In
order to compute P (xt |yt ), we apply prediction on Dt using
some base classiﬁer trained on Ds to get the soft labels for
Dt . The soft labels may be less reliable, so we iteratively
reﬁne the them.
In order to compute the marginal and conditional distribution divergences in Eq. (2), we adopt maximum mean
discrepancy (MMD) [5] to empirically estimate both distribution discrepancies. As a nonparametric measurement,
MMD has been widely applied to many existing transfer

III. BALANCED D ISTRIBUTION A DAPTATIONThis section elaborates our proposed algorithms. First,
we introduce the problem deﬁnition. Then, we present theBalanced Distribution Adaptation (BDA) approach. Finally,
the Weighted BDA (W-BDA) method is introduced.

1130

Learning algorithm: Denote Φ = (φ1 , φ2 , · · · , φd ) as
Lagrange multipliers, then Lagrange function for Eq. (4) islearning approaches [5], [2]. Formally speaking, Eq. (2) canbe represented as

2
 n

m

1 

1

D(Ds , Dt ) ≈(1 − μ) 
x
−
x
si
t
n
m j=1 j 
 i=1

H

2


C




1
1

+μ
x si −
x tj 

 nc

m
c
(c)
(c)
c=1 

xsi ∈Ds
xtj ∈Dt

L = tr A X (1 − μ)M0 + μ+ λA2F + tr

(3)
where H denotes the reproducing kernel Hilbert space(RKHS), c ∈ {1, 2, · · · , C} is the distinct class label, n, m
denote the number of samples in the source / target domain,(c)(c)
and Ds and Dt denote the samples belonging to class c(c)in source and target domain, respectively. nc = |Ds |, mc =(c)(c)
|Dt |, denoting the number of samples belonging to Ds(c)and Dt , respectively. The ﬁrst term denotes the marginal
distribution distance between domains, while the second
term is the conditional distribution distance.
By further taking advantage of matrix tricks and regularization, Eq. (2) can be formalized as:

min tr A X (1 − μ)M0 + μ

C


Mc





X A

+ λA2F

X A

(7)



(I − A XHX A)Φ

C





Mc



X + λI A = XHX AΦ
(8)

C. Weighted Balanced Distribution Adaptation0≤μ≤1

BDA is able to adaptively leverage the importance ofmarginal and conditional distributions between domains.
BDA indicates when the marginal distributions are relatively close, the performance of transfer learning is highly
dependent on the conditional distribution distance. Whencomputing the conditional distributions, BDA utilizes classconditional distributions instead, i.e. P (x|y) is used toapproximate P (y|x). This implicitly assumes that the probability of this class in each domain is similar, which is usually
not the case in real world. In this section, we propose a morerobust approximation of the conditional distribution for classimbalance problem:

There are two terms in Eq. (4): the adaptation of marginal
and conditional distribution with balance factor (term 1),
and the regularization term (term 2). λ is the regularization
2
parameter with ·F the Frobenius norm. Two constraints
are involved in Eq. (4): the ﬁrst constraint ensures thatthe transformed data (A X) should preserve the innerproperties of the original data. The second constraint denotes
the range of the balance factor μ.
More speciﬁcally, in Eq. (4), X denotes the input data
matrix composed of xs and xt , A denotes the transformation
matrix, I ∈ R(n+m)×(n+m) is the identity matrix, and H
is the centering matrix i.e. H = I − (1/n)1. Similar as
in work [2], M0 and Mc are MMD matrices and can beconstructed in the following ways:⎧
1⎪
x i , x j ∈ Ds
⎨ n2 ,
1(M0 )ij = m2 ,(5)x i , x j ∈ Dt⎪
⎩ 1
− mn , otherwise(Mc )ij =



Mc



Finally, the optimal transformation matrix A can be obtainedby solving Eq. (8) and ﬁnding its d smallest eigenvectors.
Estimation of μ: Note that μ is technically not a freeparameter like λ and it has to be estimated according to data
distributions. However, there is no effective solution for its
estimation. For now, we evaluate the performance of μ bysearching its values in experiments. For real application, we
recommend getting the optimal μ through cross-validation.

(4)

⎧1
⎪
n2c ,
⎪
⎪
⎪
1
⎪
⎪
⎨ m2 ,

xi , xj ∈ Ds⎪
− mc1nc ,
⎪
⎪
⎪
⎪
⎪
⎩
0,

xi ∈ Ds , xj ∈ Dt(c)(c)
xi ∈ Dt , xj ∈ Ds
otherwise

c

c=1





c=1

c=1

s.t. A XHX A = I,
X (1 − μ)M0 + μ





C


Set derivative ∂L/∂A = 0, the optimization can be derived
as a generalized eigendecomposition problem

H







2

P (ys |xs ) − P (yt |xt )H
2
 P (ys )P (yt )

P
(x
P
(x
=
|y
)
−
|y
)
s
s
t
t
 P (xs )P (xt )
2= αs P (xs |ys ) − αt P (xt |yt )H
Technically, we approximate αs and αt by the classprior of both domains. To this end, weighted balanced
distribution adaptation (W-BDA) is proposed to balance theclass proportion of each domain. Evaluating the conditional
distribution divergence in Eq. (9) requires to estimate themarginal distributions P (xs ) and P (xt ). However, it is nontrivial. Since BDA is fully capable of adapting P (xs ) andP (xt ), we do not estimate them in this step and assume they
are unchanged. Then, we construct a weight matrix Wc for

(c)
(c)

xi , xj ∈ Dt

(c)

(c)

(9)

H

(6)

1131

each class:(Wc )ij =⎧P (ys(c) )
⎪
,
⎪
2
⎪
nc 
⎪
⎪
(c)
⎪
P
y
⎪
t
⎪
⎨ m2 ,

IV. E XPERIMENTS
In this section, we evaluate the performance of the proposed methods through extensive experiments.

(c)xi , xj ∈ Ds

(c)

c 
 

(c)
(c)
P yt
P ys

xi , xj ∈ Dt

A. Datasets
We adopt ﬁve widely-used datasets: USPS + MNIST,
COIL20 and Ofﬁce + Caltech. Table I shows the detailsof the datasets. USPS (U) and MNIST (M) are standard
digit recognition datasets containing handwritten digits from
0-9. USPS consists of 7,291 training images and 2,007
test images. MNIST contains 60,000 training images and10,000 test images. COIL20 (CO) includes 1,440 images
belonging to 20 objects. Ofﬁce is composed of three realworld object domains: Amazon, Webcam and DSLR. It
has 4,652 images with 31 object categories. Caltech-256(C) contains 30,607 images and 256 categories. Detailed
descriptions about those datasets can be found in [2]. For all
the datasets, we follow [2] to construct 16 different tasks.

(c)
(c)
⎪
xi ∈ Ds , xj ∈ Dt
⎪
⎪
−
,
⎪
(c)
(c)
m
n
c c
⎪
⎪
xi ∈ Dt , xj ∈ Ds⎪⎪⎩0,otherwise
(10)


(c)
(c)
where P ys
and P ytdenote the class prior on classc in the source and target domain, respectively.
Embedding Eq. (10) into BDA, we get the trace optimization problem of W-BDA:

min tr A X (1 − μ)M0 + μ

C





Wc



X A

+ λA2F

c=1





s.t. A XHX A = I,

0≤μ≤1
(11)

Table II NTRODUCTION OF THE FIVE DIGIT / OBJECT DATASETS .
DatasetType#Sample #Feature#Class
Domain
USPS
Digit
1,800
256
10
U
MNIST
Digit
2,000
256
10
M
COIL20Object
1,440
1,024
20
CO1, CO2
Ofﬁce
Object1,410
800
10A, W, D
CaltechObject
1,123
800
10
C

Remark: Eq. (5) of BDA and Eq. (10) of W-BDA are
much similar in spirit. Their differences are: 1) Eq. (5) of
BDA only considers the number of samples in each class,while Eq. (10) also considers the class prior. 2) Eq. (10)provides more accurate approximation to the conditional
distributions than Eq. (5) when handling the class imbalance.
Kernelization: When applied to nonlinear problems, we
can use a kernel map ψ: x → ψ(x), and a kernel matrix
K = ψ(X) ψ(X). The kernel matrix K ∈ R(n+m)×(n+m)
can be constructed using linear or RBF kernel.
In summary, Algorithm 1 presents the detail of BDA and
W-BDA methods.

B. Comparison Methods
We choose six state-of-the-art comparison methods:• 1 Nearest Neighbor classiﬁer (1NN)• Principal Component Analysis (PCA) + 1NN• Geodesic Flow Kernel (GFK) [19] + 1NN• Transfer Component Analysis (TCA) [5] + 1NN• Joint Distribution Adaptation (JDA) [2] + 1NN• Transfer Subspace Learning (TSL) [20] + 1NN
Among those methods, 1NN and PCA are traditional
learning methods, while GFK, TCA, JDA, and TSL are stateof-the-art transfer learning approaches.

Algorithm 1 BDA: Balanced Distribution Adaptation
Input:
Source and target feature matrix Xs and Xt ,source label vector ys , #dimension d, balance factor μ,
regularization parameter λOutput:Transformation matrix A and classiﬁer f
1: Train a base classiﬁer on Xs and apply prediction on
Xt to get its soft labels ŷt . Construct X = [Xs , Xt ],initialize M0 and Mc by Eq. (5) and (6) (or Wc using
Eq. (10) for W-BDA)
2: repeat3:
Solve the eigendecomposition problem in Eq. (8) (orEq. (11) for W-BDA) and use d smallest eigenvectors
to build A
4:
Train a classiﬁer f on {A Xs , ys }5:Update the soft labels of Dt : ŷt = f (A Xt )6:
Update matrix Mc using Eq. (6) (or update Wc using
Eq. (10) for W-BDA)
7: until Convergence
8: return Classiﬁer f

C. Implementation Details
PCA, TCA, JDA, TSL, and BDA are acting as dimensionality reduction process, then 1NN is applied. For GFK,
1NN is applied after we get the geodesic ﬂow kernel. For
BDA and W-BDA, μ is searched in {0, 0.1, · · · , 0.9, 1.0}.
Since BDA can achieve a stable performance under a wide
range of parameter values, for the comparison study, we set
d = 100; λ = 0.1 for MNIST + USPS / Ofﬁce + Caltech
datasets and λ = 0.01 for COIL20 dataset. For the kernelbased methods, we use linear kernel. The iteration number
for JDA and TCA is set to be T = 10. The codes of BDA
and W-BDA are available online1 . Classiﬁcation accuracy
on target domain is adopted as the evaluation metric which
is widely used in literatures [2], [3].
1 Code available at http://tinyurl.com/yd3ol4om

1132

100

Table II
ACCURACY (%) OF BDA AND OTHER METHODS ON 16 TASKS .
PCA
44.95
66.22
84.72
84.03
36.95
32.54
38.22
34.73
35.59
27.39
26.36
31.00
77.07
29.65
32.05
75.93
47.34

GFK
46.45
67.22
72.50
74.17
41.02
40.68
38.85
40.25
38.98
36.31
30.72
29.75
80.89
30.28
32.05
75.59
48.48

TCA
52.20
54.28
88.61
96.25
44.89
36.61
45.86
40.78
37.63
31.85
27.16
30.69
90.45
32.50
31.52
87.12
51.78

220

64

200

62

180
160
140

TCA
JDA
BDA

120
100

5

10

15

JDA
57.45
62.89
97.22
86.39
42.90
38.64
47.13
38.82
37.29
40.13
25.29
31.84
90.45
30.99
32.25
91.19
53.18

TSL
53.75
66.06
88.06
87.92
44.47
34.24
43.31
37.58
33.90
26.11
29.83
30.27
87.26
28.50
27.56
85.42
50.27

60
58
56
54

TCA
JDA
BDA

52
20

BDA
59.35
69.78
97.22
96.81
44.89
38.64
47.77
40.78
39.32
43.31
28.94
32.99
91.72
32.50
33.09
91.86
55.56

50

# Iterations

5

10

15

20

# Iterations(a) MMD distance(b) Accuracy
Figure 1. MMD distance and classiﬁcation accuracy comparison of TCA,JDA and BDA on U → M. It can be noted that BDA achieves better
accuracy with relatively small MMD distance.

D. Performance Evaluation of BDA
1) Classiﬁcation accuracy: We test the performance of
BDA and the other comparison methods on 16 cross-domain
learning tasks. The results are shown in Table II, based on
which, we can draw the following observations.
First, BDA outperforms most of the existing methods (15out of 16 tasks). Speciﬁcally, the average classiﬁcation
accuracy of BDA is 55.56%, which shows an average
improvement of 2.38% compared to the best comparison
method JDA. JDA is only capable of adapting the marginal
and conditional distribution with the equal weight (μ = 0.5).
Thus JDA can be considered as a special case of BDA.
However, BDA can dramatically improve the accuracy byadjusting the balance parameter μ to adapt various scenarios.
Second, TCA is also a special case of BDA (μ = 0)
since it only adapts the marginal distribution. Therefore, theperformance of TCA is worse than JDA and BDA.
Third, TSL only adapts the marginal distributions whichhighly relies on the distribution density. The performanceof GFK is better on object recognition tasks. The reason isthat GFK learns a global geodesic ﬂow kernel on the lowdimension representation, which may be enough to transit
smoothly for the object datasets. But as for the digit tasks,
it may not enough to construct smooth transit when themarginal distribution distance is large.
Last, all transfer learning methods perform better than

A
D
A
W
CO2
CO1
M
U

90
80

Accuracy(%)

1NN
44.70
65.94
83.61
82.78
23.70
25.76
25.48
26.00
29.83
25.48
19.86
22.96
59.24
26.27
28.50
63.39
40.84

Accuracy (%)

MMD distance

Task
U→M
M→U
CO1 → CO2
CO2 → CO1
C→A
C→W
C→D
A→C
A→W
A→D
W→C
W→A
W→D
D→C
D→A
D→W
Average

70
60
50
40
30

0

0.2

0.4

0.6

0.8

1

Figure 2. Classiﬁcation accuracy w.r.t. μ on different tasks. Dashed lines
are the best comparison methods.

traditional learning approaches due to the large distribution
gap between different domains. This indicates the effectiveness of transfer learning methods, among which BDA could
achieve the best performance.
2) Effectiveness of distribution adaptation: We further
verify the effectiveness of BDA by comparing its distribution
adaptation with other two distribution adaptation methods:
TCA and JDA. Speciﬁcally, we investigate their performancewith MMD distances calculated using Eq. (4).
Fig. 1(a) and Fig. 1(b) show the MMD distance andaccuracy of TCA, JDA, and BDA with increasing iteration,
respectively. Based on the results, we can observe: a) MMD
distances of all methods can be reduced. This indicates theeffectiveness of TCA, JDA and BDA; b) MMD distance of
TCA is not reduced largely as it only adapts the marginal
distribution distance and requires no iteration; c) MMD
distance of JDA is obviously larger than BDA, since BDA
could balance the importance of marginal and conditional
distribution via μ; d) BDA achieves the best performance.
E. Effectiveness of Balance Factor
In this section, we evaluate the effectiveness of the balancefactor μ. We run BDA with μ ∈ {0, 0.1, · · · , 1.0} on sometasks and compare the performances with the best baseline
method. Fig. 2 shows the results. It is obvious that the
optimal μ varies on different tasks, indicating the importance to balance the marginal and conditional distributions
between domains. In comparison, the best baseline JDA(dash lines) is only the special case of BDA (μ = 0.5),
which means to treat those distributions equally. However,this assumption does not hold. In task A → W with optimal
μ = 0.8, it means the marginal distributions are almost thesame so the performance of transfer learning mostly depends
on conditional distributions. In task M → U with optimal
μ = 0.1, it means the marginal distributions contribute mostto the discrepancy, so μ is relatively small. In other 13 tasks,
the observations are similar. It indicates in cross-domain
learning problems, μ is extremely important to balance boththe marginal and conditional distributions. Therefore, BDA
is more capable of achieving good performance.
To be noticed, there may be more than one optimal μ forsome tasks (A → D), and the tendency of μ is not always1133

stable (CO2 → CO1). The problems behind those facts stillneed to be addressed in future research.
F. Effectiveness of Weighted BDA
We extensively verify the effectiveness of proposed WBDA. We choose some tasks with highly imbalanced class
distributions and compare the performance of W-BDA with
BDA and JDA. TABLE. III demonstrates the classiﬁcation
accuracy of 6 tasks. Note that for comparison, classes ontasks 1 ∼ 4 are rather imbalanced, while classes are rather
balanced on tasks 5 ∼ 6.
Table III
ACCURACY OF JDA, BDA AND W-BDA ON SOME TASKS .
Index
Task
JDA
BDA
WBDA
1
C→D
47.13
47.77
48.41
2
W→C
25.29
28.94
31.08
3
U→M
57.45
59.15
59.35
4
A→W
37.29
39.32
40.68
5
C→A
42.90
44.89
45.20
6
CO1 → CO2 97.22
97.22
96.69

From the results, we can observe: 1) JDA achieves theworst results since it does not consider the gap between
marginal and conditional distributions. BDA is able tohandle the distribution discrepancy and outperforms JDA in
all situations. 2) For the ﬁrst four tasks which are underimbalanced class distributions, W-BDA could improve theperformance by adaptively weighting each class. In the othertwo tasks where class distributions are rather balanced, WBDA still achieves comparable results. On the other 10 tasks,
the results follow the same tendency. To sum up, the results
demonstrate that in transfer learning, W-BDA remains aneffective method to balance the different class distribution
between domains.
In addition, BDA and W-BDA have other two parameters:feature dimension d and regularization parameter λ. Theirsensitivity evaluations are omitted due to page limit. In ouractual experiments, BDA and W-BDA are relatively robust
to those two parameters.
V. C ONCLUSION
Balancing the probability distributions and class distributions between domains are both two important problems
in transfer learning. In this paper, we propose Balanced
Distribution Adaptation (BDA) to adaptively weight theimportance of both marginal and conditional distribution
adaptations. Thus, it could signiﬁcantly improve the transfer
learning performance. Moreover, we consider handling theclass imbalance problem for transfer learning by proposing
Weighted BDA (W-BDA). Extensive experiments on ﬁve
image datasets demonstrate the superiority of our methods
over several state-of-the-art methods. In the future, we willcontinue the exploration in these two aspects: by developing
more strategies to leverage the distributions and handle theclass imbalance problem.

ACKNOWLEDGMENT
This work is supported in part by National Key R& D Plan of China (No.2016YFB1001200), NSFC
(No.61572471), and Beijing Municipal Science &Technology Commission (No.Z161100000216140 &Z171100000117013).
R EFERENCES[1] S. J. Pan and Q. Yang, “A survey on transfer learning,” TKDE,
vol. 22, no. 10, pp. 1345–1359, 2010.
[2] M. Long, J. Wang, G. Ding, J. Sun, and P. S. Yu, “Transferfeature learning with joint distribution adaptation,” in ICCV,2013, pp. 2200–2207.
[3] H. Yan, Y. Ding, P. Li, Q. Wang, Y. Xu, and W. Zuo, “Mindthe class weight bias: Weighted maximum mean discrepancy
for unsupervised domain adaptation,” in CVPR, 2017.
[4] C.-A. Hou, Y.-H. H. Tsai, Y.-R. Yeh, and Y.-C. F. Wang,“Unsupervised domain adaptation with label and structuralconsistency,” TIP, vol. 25, no. 12, pp. 5552–5562, 2016.
[5] S. J. Pan, I. W. Tsang, J. T. Kwok, and Q. Yang, “Domain
adaptation via transfer component analysis,” TNN, vol. 22,
no. 2, pp. 199–210, 2011.
[6] S. Satpal and S. Sarawagi, “Domain adaptation of conditional probability models via feature subsetting,” in PKDD.
Springer, 2007, pp. 224–235.
[7] J. Tahmoresnezhad and S. Hashemi, “Visual domain adaptation via transfer feature learning,” Knowl. Inf. Syst., 2016.
[8] J. Wang, Y. Chen, S. Hao, X. Peng, and L. Hu, “Deeplearning for sensor-based activity recognition: A survey,”
arXiv preprint arXiv:1707.03502, 2017.
[9] L. Hu, Y. Chen, S. Wang, J. Wang, J. Shen, X. Jiang, andZ. Shen, “Less annotation on personalized activity recognition
using context data,” in UIC, July 2016, pp. 327–332.
[10] S. Hao, P. Zhao, Y. Liu, S. C. H. Hoi, and C. Miao, “Online
multi-task relative similarity learning,” in IJCAI, 2017.
[11] Y. Chen, Y. Gu, X. Jiang, and J. Wang, “Ocean: A new opportunistic computing model for wearable activity recognition,”in UbiComp. ACM, 2016, pp. 33–36.
[12] J. Li, J. Zhao, and K. Lu, “Joint feature selection and structure
preservation for domain adaptation,” in IJCAI, 2016.
[13] C.-A. Hou, Y.-R. Yeh, and Y.-C. F. Wang, “An unsupervised
domain adaptation approach for cross-domain visual classiﬁcation,” in AVSS. IEEE, 2015, pp. 1–6.
[14] S. Ando and C. Y. Huang, “Deep over-sampling framework for classifying imbalanced data,” arXiv preprint arXiv:1704.07515, 2017.
[15] T. Ming Harry Hsu, W. Yu Chen, C.-A. Hou, and H. T.
et al., “Unsupervised domain adaptation with imbalanced
cross-domain data,” in ICCV, 2015, pp. 4121–4129.
[16] F. Wu, X.-Y. Jing, S. Shan, W. Zuo, and J.-Y. Yang, “Multiset
feature learning for highly imbalanced data classiﬁcation,” in
AAAI, 2017.
[17] P.-H. Hsiao, F.-J. Chang, and Y.-Y. Lin, “Learning discriminatively reconstructed source data for object recognition withfew examples,” TIP, vol. 25, no. 8, pp. 3518–3532, 2016.
[18] S. Li, S. Song, and G. Huang, “Prediction reweighting fordomain adaptation,” TNNLS, no. 99, pp. 1–14, 2016.
[19] B. Gong, Y. Shi, F. Sha, and K. Grauman, “Geodesic ﬂow
kernel for unsupervised domain adaptation,” in CVPR, 2012.
[20] S. Si, D. Tao, and B. Geng, “Bregman divergence-based
regularization for transfer subspace learning,” TKDE, vol. 22,
no. 7, pp. 929–942, 2010.

1134