# Classifier Adaptation Based on Modified Label Propagation for Unsupervised Domain Adaptation

Authors: Yongjie Du; Deyun Zhou; Yu Xie; Weiren Kong; Xiaoyang Li; Jiao Shi; Yu Lei
Journal: Wireless Communications and Mobile Computing
 (2022)

---

## Abstract

Unsupervised domain adaptation endeavors to learn a desirable classifier for a target domain by transferring knowledge learned from a related (source) domain. Existing approaches focus on deriving domain-invariant feature representations by aligning the domain distributions. However, those works often require an extra classifier. In contrast, this paper proposes a classifier adaptation method based on modified label propagation (CAMLP) for unsupervised domain adaptation. Inspired by pseudolabeling, CAMLP proposes a simple but effective measurement for relationships among cross-domain samples. Thus, samples from distinct domains are constructed in a same graph. The true labels can then propagate from the source domain to the target one along the graph. We also propose a consistency-aware pseudolabel annotation to alleviate the problem of negative transfer caused by unreliable pseudo labels. Extensive experiments on several benchmark datasets confirm that the proposed method performs favorably against the state-of-the-art approaches.


---

## Body

## 1. Introduction

Domain adaptation [1–3] transfers knowledge from a label-well source domain to the label-scarce target domain, and there is usually a certain degree of data distribution discrepancy between the source and target domains. Domain adaptation has a wide range of applications in medical diagnosis [4, 5], intelligent traffic [6–8], and other IOT (Internet of Things) fields [9, 10]. For example, domain adaptation is widely used in image semantic segmentation tasks to alleviate the huge cost of dense annotation [11–14]. And it is also applied to strengthen the generalization of person reidentification models [6, 15–17]. In fact, machine learning has made a amazing progress in these fields recently, but this relies on vast amounts of label-accurate training data and requires same distribution of training data (source domain) and test data (target domain). In real world scenarios, however, annotating samples for every scenario is uneconomical or even impossible. Domain adaptation tackles this problem by applying the source-domain-learned ability to the target domain. Experience on riding a bike, for example, is helpful for riding a motorcycle. Due to the distribution shift (discrepancy), the model performance will deteriorate when applied directly from one domain (source domain) to another (target domain). Domain adaptation mitigates this effect through bridging the gaps between domains. In general, domain adaptation is mainly classified into semi-supervised domain adaptation (SSDA) [18–20] with a few labels of target samples being available, unsupervised domain adaptation (UDA) [21] without any target label. The SSDA uses the limited labeled target samples to facilitate the model training with the label-rich source data. Therefore, the SSDA-learned classifier usually has a more easily distinguishable classification boundary in the target domain than the UDA-learned one [22].In this study, we focus mainly on the UDA, which is more challenging and practical. The key of UDA is how to exploit common knowledge across domains. Existing approaches generally aligned distinct domains by projection or transformation, such as searching common latent feature representations in Hilbert space [23–25]. Maximum mean discrepancy (MMD) is a classic and efficient one of these strategies. The structural (geometric) information of samples has recently attracted the interest of researchers, which can efficiently characterize the relationship between samples, according to certain relevant research [26–28]. By investigating the underlying structure across all (source and target) samples, the graph-based technique offers a high-efficiency way to develop a classifier [26]. All approaches above align the domain distribution but require an additional classifier. In contrast, this paper learns a domain-invariant classifier directly.To this end, a novel UDA method, classifier adaptation based on modified label propagation (CAMLP), is proposed by exploiting cross-domain and within-domain structural information. Samples from different domains are constructed into ak-nearest neighbor graph. The nodes of the graph represent the observations in the source or target domain. The edges of the graph indicate the relationships between these observations, which are used to be measured in traditional ways, such as cosine similarity. However, the traditional similarity metrics may fail to accurately reflect the affinity relationships between cross-domain samples due to domain discrepancy. Our modified label propagation (MLP) proposes to measure these relationship by pseudo labeling, simply yet effectively. Then, the true labels transfer from the source domain to the target one along the graph. In addition, we propose a consistency-aware pseudolabel annotation (CPLA) to alleviate the negative transfers caused by incorrect labels. In conclusion, the contributions of this study are as follows:
(i)
A unsupervised domain adaptation method, classifier adaptation based on modified label propagation (CAMLP), is proposed through exploiting structural information and pseudo labeling(ii)
A simple yet effective approach is proposed to measure the relationship between cross-domain samples. Thus, samples of different domains can be constructed into the same graph(iii)
A consistency-aware pseudolabel annotation method is presented, which significantly alleviates the impact of incorrect labels(iv)
Extensive experiments on four benchmark datasets demonstrate that our method outperforms the mainstream domain adaptation methodsThe rest of this paper is organized as follows. Section2 introduces relevant topics of unsupervised domain adaptation and some basics of this work. Section 3 delves into the specifics of our proposed method. Sections 4 and 5 provide extensive experiments and discussion. Finally, our work is concluded in Section 6.


## 2. Related Work

### 2.1. Unsupervised Domain Adaptation

Mainstream unsupervised domain adaptation methods learn a domain-invariant feature representation by narrowing the domain discrepancy. These methods can align the domain distribution well but may damage the category discrimination within the target domain. Accordingly, there are two important challenges in UDA: aligning the domain distribution at the class level and maintaining the category differentiability within the domain. To this end, two important tools are proposed: the pseudolabeling and structural exploitation for UDA.The pseudolabeling was commonly utilized to reduce domain discrepancy at the category level, which helped to efficiently apply classifiers learned from the source domain to the target one [29]. Pseudolabels were often generated by a source-domain-trained classifier [25, 30]. Due to domain shift, however, these pseudolabels may contain incorrect ones, which would cause the negative transfer problem. Pei et al. [31] alleviated the problem by the soft labeling method. Refs. [28, 32, 33] proposed the confidence-aware label filtering method, which only allowed the high-confidence pseudolabels to narrow domain discrepancy. PACET [34] effectively reduced the uncertainty in pseudo labels by filtering the target samples progressively and describing relations among samples with class confidence scores.UDA differs from unsupervised learning mainly in the availability of unlabeled test (target) data. The data can be regarded as coming from a distribution supported by a low-dimensional manifold embedded in a high-dimensional space in terms of geometric structure [35]. In a sense, closed samples intend to belong to the same class. The underlying structure within the target domain helps distinguish the different categories, so that it is often used as an auxiliary knowledge to enhance domain adaptive methods. For example, GTL [27] managed UDA by optimizing the empirical likelihood while maintaining the geometric structure at the same time. JDA [23], LSC [36] and GAKT [26] exploited the structure in the target domain to increase the confidence of pseudolabels. For better aligning the conditional distribution between two domains, CCSL [29] used both the label and structural information within and across domains.In contrast, this paper proposes a classifier adaptive method-based modified label propagation (CAMLP) for unsupervised domain adaptation. GTL [27] and SPL [28] are the most similar to our method. In several ways, CAMLP differs significantly from them. To begin with, GTL optimizes both empirical likelihood and geometric structure concurrently to tackle domain adaptation tasks, but CAMLP keeps the geometric structure while aligning the conditional distribution. Both CAMLP and SPL employ pseudolabeling to reduce the conditional distribution discrepancy. CAMLP immediately learns a domain-invariant classifier, whereas SPL tries to learn a domain-invariant feature representation.


### 2.2. Label Propagation

Graphs are widely used to model structured and relational data. The label propagation is a classifying function that is sufficiently smooth to inherent the structure disclosed by available labeled and unlabeled graph nodes. Given a graphG=X,W, where X represents the graph nodes and W indicates graph edges. X=XL;XU and the labels of XL are available (YL) whereas the labels of XU are not. W represents the affinity matrix of X, whose element wij represents the relationship between nodes xi and xj. Label propagation is able to propagate the labels from XL to XU by learning the indicated matrix F. Specifically, F is computed by minimizing the following cost function:
(1)QF=12∑i,j=1l+uwij1diFi−1djFj2+12μ∑i=1lFi−Yi2,where l=XL and u=XU are the corresponding node numbers. The first element on the right causes neighboring samples to have similar indicated vectors F∗, whereas the second guarantees the prediction close to their ground truth labels. The trade-off between these two constraints is captured by the hyper-parameter μ. Y∗ indicates the one-hot vector of the corresponding ground truth label. Proved by [37], the optimal F is computed by the following equations:
(2)F=1−αI−αS−1Y,S=D−1/2WD−1/2,D=diagd1,d2,⋯,dn,di=∑j=1nwij,where α=11+μ, S is the propagating matrix, and D is the degree matrix of W. Y is the one-hot matrix with Yij=1 if xi is labeled as yi=j and Yij=0 otherwise. Then, xi can be predicted by the following:
(3)y^xi=argmax1≤j≤cFij′.In summary, the affinity matrixW and matrix Y may identify the labels of nodes uniquely.


### 2.3. Spectral Clustering

Spectral clustering (SC) [38] is a classical unsupervised learning method that can split unlabeled data into several groups. SC firstly projects the training data into a low-dimensional space, then groups them by K-means or K-medoids. The dimension of the low-dimensional space is based on the eigenvectors of the Laplacian matrix about W. And W is the affinity matrix describing the relation between samples. The specific implementation process of spectral clustering is shown in Algorithm 1. W is constructed by calculating the similarity between samples. Note that Laplacian matrix L is normalized by the Shi-Malik method [39] in Line 4.Algorithm 1: spectralCluster.
Input: X, cluster number C.Output: Y∧clu.1 Construct the affinity matrixW from X;2 Calculate the degree matrix:D=diagd1,d2,⋯,dn, di=∑j=1nwij;3 Calculate Laplacian matrix:L=D−W;4 Normalize Laplacian matrix:Lnorm=D−1L;5V=eigenvectorLnorm,V∈ℝn×C;6 ClusterV into C categories by k-means clustering;7 Obtain clustering labelsY∧clu∈ℝn×1;


## 2.1. Unsupervised Domain Adaptation

Mainstream unsupervised domain adaptation methods learn a domain-invariant feature representation by narrowing the domain discrepancy. These methods can align the domain distribution well but may damage the category discrimination within the target domain. Accordingly, there are two important challenges in UDA: aligning the domain distribution at the class level and maintaining the category differentiability within the domain. To this end, two important tools are proposed: the pseudolabeling and structural exploitation for UDA.The pseudolabeling was commonly utilized to reduce domain discrepancy at the category level, which helped to efficiently apply classifiers learned from the source domain to the target one [29]. Pseudolabels were often generated by a source-domain-trained classifier [25, 30]. Due to domain shift, however, these pseudolabels may contain incorrect ones, which would cause the negative transfer problem. Pei et al. [31] alleviated the problem by the soft labeling method. Refs. [28, 32, 33] proposed the confidence-aware label filtering method, which only allowed the high-confidence pseudolabels to narrow domain discrepancy. PACET [34] effectively reduced the uncertainty in pseudo labels by filtering the target samples progressively and describing relations among samples with class confidence scores.UDA differs from unsupervised learning mainly in the availability of unlabeled test (target) data. The data can be regarded as coming from a distribution supported by a low-dimensional manifold embedded in a high-dimensional space in terms of geometric structure [35]. In a sense, closed samples intend to belong to the same class. The underlying structure within the target domain helps distinguish the different categories, so that it is often used as an auxiliary knowledge to enhance domain adaptive methods. For example, GTL [27] managed UDA by optimizing the empirical likelihood while maintaining the geometric structure at the same time. JDA [23], LSC [36] and GAKT [26] exploited the structure in the target domain to increase the confidence of pseudolabels. For better aligning the conditional distribution between two domains, CCSL [29] used both the label and structural information within and across domains.In contrast, this paper proposes a classifier adaptive method-based modified label propagation (CAMLP) for unsupervised domain adaptation. GTL [27] and SPL [28] are the most similar to our method. In several ways, CAMLP differs significantly from them. To begin with, GTL optimizes both empirical likelihood and geometric structure concurrently to tackle domain adaptation tasks, but CAMLP keeps the geometric structure while aligning the conditional distribution. Both CAMLP and SPL employ pseudolabeling to reduce the conditional distribution discrepancy. CAMLP immediately learns a domain-invariant classifier, whereas SPL tries to learn a domain-invariant feature representation.


## 2.2. Label Propagation

Graphs are widely used to model structured and relational data. The label propagation is a classifying function that is sufficiently smooth to inherent the structure disclosed by available labeled and unlabeled graph nodes. Given a graphG=X,W, where X represents the graph nodes and W indicates graph edges. X=XL;XU and the labels of XL are available (YL) whereas the labels of XU are not. W represents the affinity matrix of X, whose element wij represents the relationship between nodes xi and xj. Label propagation is able to propagate the labels from XL to XU by learning the indicated matrix F. Specifically, F is computed by minimizing the following cost function:
(1)QF=12∑i,j=1l+uwij1diFi−1djFj2+12μ∑i=1lFi−Yi2,where l=XL and u=XU are the corresponding node numbers. The first element on the right causes neighboring samples to have similar indicated vectors F∗, whereas the second guarantees the prediction close to their ground truth labels. The trade-off between these two constraints is captured by the hyper-parameter μ. Y∗ indicates the one-hot vector of the corresponding ground truth label. Proved by [37], the optimal F is computed by the following equations:
(2)F=1−αI−αS−1Y,S=D−1/2WD−1/2,D=diagd1,d2,⋯,dn,di=∑j=1nwij,where α=11+μ, S is the propagating matrix, and D is the degree matrix of W. Y is the one-hot matrix with Yij=1 if xi is labeled as yi=j and Yij=0 otherwise. Then, xi can be predicted by the following:
(3)y^xi=argmax1≤j≤cFij′.In summary, the affinity matrixW and matrix Y may identify the labels of nodes uniquely.


## 2.3. Spectral Clustering

Spectral clustering (SC) [38] is a classical unsupervised learning method that can split unlabeled data into several groups. SC firstly projects the training data into a low-dimensional space, then groups them by K-means or K-medoids. The dimension of the low-dimensional space is based on the eigenvectors of the Laplacian matrix about W. And W is the affinity matrix describing the relation between samples. The specific implementation process of spectral clustering is shown in Algorithm 1. W is constructed by calculating the similarity between samples. Note that Laplacian matrix L is normalized by the Shi-Malik method [39] in Line 4.Algorithm 1: spectralCluster.
Input: X, cluster number C.Output: Y∧clu.1 Construct the affinity matrixW from X;2 Calculate the degree matrix:D=diagd1,d2,⋯,dn, di=∑j=1nwij;3 Calculate Laplacian matrix:L=D−W;4 Normalize Laplacian matrix:Lnorm=D−1L;5V=eigenvectorLnorm,V∈ℝn×C;6 ClusterV into C categories by k-means clustering;7 Obtain clustering labelsY∧clu∈ℝn×1;


## 3. Methodology

In this section, we offer the problem formulation first and then introduce the suggested method: classifier adaptation based on modified label propagation (CAMLP) for unsupervised domain adaptation in detail.


### 3.1. Problem Formulation

This work focuses on UDA with label-well source domain (Ds) and label-scarce target domain (Dt). Specifically, the labeled data Xs,Ys∈Ds, unlabeled data Xt∈Dt. Xs and Ys (Xt and Yt) represent the feature space and label space of the source (target) domain, respectively. Our work focuses the scenario: label space Ys=Yt, whereas the marginal probability PsXs≠PtXt and conditional probability QsXsYs≠QtXtYt. The purpose of UDA is to learn a classifier f to accurately predict the target samples, i.e., fXs,Ys,Xt⟶Y∧t. Table 1 shows the primary notations and descriptions used in this study.Table 1
Notations and descriptions.
NotationDescriptionSource/target domainXs/XtSource/target sample matrixYs/YtSource/target label matrixns/ntNumber of source/target samplesp1/p2Nearest neighbor number in LP/SCd/d2Feature dimension before/after PCAWAffinity matrix


### 3.2. The Overall Framework

In this paper, we proposed a classifier adaptation-based modified label propagation (CAMLP) for unsupervised domain adaptation, as illustrated in Figure1. To begin with, a k-nearest neighbor (knn) graph is constructed for two distinct domains. The labels are then transferred from the source domain to the target one via the modified label propagation. When constructing graphs, it is difficult to accurately measure the affinity relationship between cross-domain data only by conventional methods, due to domain discrepancy. Inspired by the pseudolabeling technique, our modified label propagation presents a novel way for solving this challenge. In this method, the accuracy of pseudolabels is crucial, because misclassified samples frequently degrade performance. To this end, this work presents a consistency-aware pseudolabel annotation method. The idea is that samples in the same cluster are more likely to belong to the same class, according to the clustering assumption. The structure within the target domain helps to obtain more accurate predicted labels. Our approach is outlined in depth in the following subsections.Figure 1
Illustration of the proposed approach (best viewed in color). The distinct domain samples are constructed into the same graph. The ground truth labels then transfer from source domain to target one along the graph.


### 3.3. Graph Construction

LetX=Xs;XtT∈ℝn×d, Y=Ys;0T∈ℝn×c where n=ns+nt, ns=Xs, nt=Xt. In high-dimensional data, there is information redundancy increasing the computational cost. We use principal component analysis (PCA) to reduce the dimension of data, similar to many other UDA approaches [23, 28], i.e., PCAX⟶X~. The feature vector in X~ is then subjected to L2 normalization, which compels samples from the two domains to be dispersed on the surface of same hypersphere. During the domain adaptation technique, we retain the inherent data distributions of the two domains by preserving the geometric structure. To properly depict the structural relationship in feature space, we construct a knn graph G=X~,W with n vertexes, where W is the affinity matrix of X~ and its entries on the diagonal are 0.


### 3.4. Transfer Labels via Modified Label Propagation

Calculating the affinity matrixW is critical for labeling propagation, according to Section 2.2. W is represented as a partitioned matrix in the following way for convenience:
(4)W=WssWstWstTWtt,where Wss and Wtt denote the relationships within the source and target domains, respectively; Wst denotes the relationships between cross-domain samples. Surely, Wst is difficult to measure precisely by the traditional way because of domain discrepancy. To tackle this issue, we propose a label-aware representation of relationship between cross-domain samples. Specifically, we firstly annotate target samples. Then, the relationship between two samples is 1 if their labels are the same, and 0 otherwise. In a nutshell, Wst and Wss are calculated by the following equation:
(5)wij=1,yxi=y^xj∩i≠j0,otherwises.t.1≤i≤ns,1+ns≤j≤n∪1≤i,j≤ns,where y^∗ and y∗ denote the pseudolabel and the ground truth label, respectively. Wtt is calculated through the Euclidean distance, as demonstrated in the following:
(6)Distij=dist<xi,xj>,xi∈Npxj,xj∈Npxi0,otherwisewij=e−Distij2s.t.ns+1≤i,j≤nwhere Npxj, Npxi are the p nearest neighbor set of xj and xi, respectively. According to Section 2.2, the target labels Y∧t are obtained via the original label propagation, as shown as Algorithm 2. It is worth noting that the incorrect ones of pseudo labels may damage the model (result in negative transfer problem). To solve this problem, we propose a consistency-aware strategy to correct these wrong labels.Algorithm 2: Prediction via MLP.
Input: X~s, Ys, X~t, Y∧t′, class num C, hyperparameter αOutput: Y∧t.1 Construct the affinity matrix:W via equations (5) and (6);2 Construct one-hot-label matrix:Y=Ys;0n×C;3 CalculateD=diagd1,d2,⋯,dn,di=∑j=1nwij;4S=D−1/2WD−1/2;5F=1−αI−αS−1Y;6y^i=argmax1≤j≤CFij;7 Return the predicted labels:Y∧t=y^i1≤i≤n.


### 3.5. Consistency-Aware Pseudolabel Annotating

The existing pseudolabel initialization schemes are entirely dependent on the source domain. In contrast, our method exploits the knowledge of the source domain and the target-domain structural information together to predict target samples. We construct a new knn graph for the target sample and then split the graph intoC subgraphs through spectral clustering. In other words, the target samples are grouped into C clusters. A ballot-aware alignment method is proposed to match these cluster labels with the known categories. Specifically, a source-domain-learned classifier predicts the initial target-domain pseudolabels Y∧t′′. The drop points of Y∧t′′ on each target cluster are calculated. We define that i is the category label for a certain cluster, if there are the most points belonging to i in the cluster according to the calculated drop-points. Therefore, the new pseudolabels Y∧sc are obtained. At this point, Y∧t′′ contains the source-domain knowledge, and Y∧sc contains target-domain underlying structure. Based on this, we propose a consistency-aware strategy: only reserving the same elements in Y∧t′′ and Y∧sc as pseudo labels.Algorithm 3: Consistency-aware pseudolabel annotation.
Input: X~s, Ys, X~t.Output: pseudolabels Y∧t′.1 Learn a classifierf:fX~s,Ys,X~t⟶Y∧t;2Y∧t′′⟵fX~t;3Y∧clu⟵spectralClusterX~t;4Y∧sc=0nt×1;5fori =1: Cdo6 Calculatenij1≤j≤c;7y^=argmax1≤j≤cnij;8Y∧scY∧clu==i=y^;9 end.10Y∧t′=y∧t′i1≤i≤nt, y^it′=y^it′′,y^it′′=y^isc0,otherwiseThe specific implementation process of the method is shown in Algorithm3. In Lines 1-2, a source-domain-learned classifier predicts the target samples. In Line 3, the spectral clustering groups target samples into C clusters. In Lines 4-9, the target clusters are matched with the known categories, where nij interprets the number of pseudolabels equaling i in the jth cluster. Line 10 shows the consistency-aware label filtering strategy. It is worth noting that an iterative framework is utilized. Specifically, during each training session, the prediction of target samples are used as the pseudo labels for the next iteration. An incremental training strategy is used where the pseudo labels with high confidence are fixed in each iteration, and only the remaining pseudo labels are updated in subsequent iterations.


## 3.1. Problem Formulation

This work focuses on UDA with label-well source domain (Ds) and label-scarce target domain (Dt). Specifically, the labeled data Xs,Ys∈Ds, unlabeled data Xt∈Dt. Xs and Ys (Xt and Yt) represent the feature space and label space of the source (target) domain, respectively. Our work focuses the scenario: label space Ys=Yt, whereas the marginal probability PsXs≠PtXt and conditional probability QsXsYs≠QtXtYt. The purpose of UDA is to learn a classifier f to accurately predict the target samples, i.e., fXs,Ys,Xt⟶Y∧t. Table 1 shows the primary notations and descriptions used in this study.Table 1
Notations and descriptions.
NotationDescriptionSource/target domainXs/XtSource/target sample matrixYs/YtSource/target label matrixns/ntNumber of source/target samplesp1/p2Nearest neighbor number in LP/SCd/d2Feature dimension before/after PCAWAffinity matrix


## 3.2. The Overall Framework

In this paper, we proposed a classifier adaptation-based modified label propagation (CAMLP) for unsupervised domain adaptation, as illustrated in Figure1. To begin with, a k-nearest neighbor (knn) graph is constructed for two distinct domains. The labels are then transferred from the source domain to the target one via the modified label propagation. When constructing graphs, it is difficult to accurately measure the affinity relationship between cross-domain data only by conventional methods, due to domain discrepancy. Inspired by the pseudolabeling technique, our modified label propagation presents a novel way for solving this challenge. In this method, the accuracy of pseudolabels is crucial, because misclassified samples frequently degrade performance. To this end, this work presents a consistency-aware pseudolabel annotation method. The idea is that samples in the same cluster are more likely to belong to the same class, according to the clustering assumption. The structure within the target domain helps to obtain more accurate predicted labels. Our approach is outlined in depth in the following subsections.Figure 1
Illustration of the proposed approach (best viewed in color). The distinct domain samples are constructed into the same graph. The ground truth labels then transfer from source domain to target one along the graph.


## 3.3. Graph Construction

LetX=Xs;XtT∈ℝn×d, Y=Ys;0T∈ℝn×c where n=ns+nt, ns=Xs, nt=Xt. In high-dimensional data, there is information redundancy increasing the computational cost. We use principal component analysis (PCA) to reduce the dimension of data, similar to many other UDA approaches [23, 28], i.e., PCAX⟶X~. The feature vector in X~ is then subjected to L2 normalization, which compels samples from the two domains to be dispersed on the surface of same hypersphere. During the domain adaptation technique, we retain the inherent data distributions of the two domains by preserving the geometric structure. To properly depict the structural relationship in feature space, we construct a knn graph G=X~,W with n vertexes, where W is the affinity matrix of X~ and its entries on the diagonal are 0.


## 3.4. Transfer Labels via Modified Label Propagation

Calculating the affinity matrixW is critical for labeling propagation, according to Section 2.2. W is represented as a partitioned matrix in the following way for convenience:
(4)W=WssWstWstTWtt,where Wss and Wtt denote the relationships within the source and target domains, respectively; Wst denotes the relationships between cross-domain samples. Surely, Wst is difficult to measure precisely by the traditional way because of domain discrepancy. To tackle this issue, we propose a label-aware representation of relationship between cross-domain samples. Specifically, we firstly annotate target samples. Then, the relationship between two samples is 1 if their labels are the same, and 0 otherwise. In a nutshell, Wst and Wss are calculated by the following equation:
(5)wij=1,yxi=y^xj∩i≠j0,otherwises.t.1≤i≤ns,1+ns≤j≤n∪1≤i,j≤ns,where y^∗ and y∗ denote the pseudolabel and the ground truth label, respectively. Wtt is calculated through the Euclidean distance, as demonstrated in the following:
(6)Distij=dist<xi,xj>,xi∈Npxj,xj∈Npxi0,otherwisewij=e−Distij2s.t.ns+1≤i,j≤nwhere Npxj, Npxi are the p nearest neighbor set of xj and xi, respectively. According to Section 2.2, the target labels Y∧t are obtained via the original label propagation, as shown as Algorithm 2. It is worth noting that the incorrect ones of pseudo labels may damage the model (result in negative transfer problem). To solve this problem, we propose a consistency-aware strategy to correct these wrong labels.Algorithm 2: Prediction via MLP.
Input: X~s, Ys, X~t, Y∧t′, class num C, hyperparameter αOutput: Y∧t.1 Construct the affinity matrix:W via equations (5) and (6);2 Construct one-hot-label matrix:Y=Ys;0n×C;3 CalculateD=diagd1,d2,⋯,dn,di=∑j=1nwij;4S=D−1/2WD−1/2;5F=1−αI−αS−1Y;6y^i=argmax1≤j≤CFij;7 Return the predicted labels:Y∧t=y^i1≤i≤n.


## 3.5. Consistency-Aware Pseudolabel Annotating

The existing pseudolabel initialization schemes are entirely dependent on the source domain. In contrast, our method exploits the knowledge of the source domain and the target-domain structural information together to predict target samples. We construct a new knn graph for the target sample and then split the graph intoC subgraphs through spectral clustering. In other words, the target samples are grouped into C clusters. A ballot-aware alignment method is proposed to match these cluster labels with the known categories. Specifically, a source-domain-learned classifier predicts the initial target-domain pseudolabels Y∧t′′. The drop points of Y∧t′′ on each target cluster are calculated. We define that i is the category label for a certain cluster, if there are the most points belonging to i in the cluster according to the calculated drop-points. Therefore, the new pseudolabels Y∧sc are obtained. At this point, Y∧t′′ contains the source-domain knowledge, and Y∧sc contains target-domain underlying structure. Based on this, we propose a consistency-aware strategy: only reserving the same elements in Y∧t′′ and Y∧sc as pseudo labels.Algorithm 3: Consistency-aware pseudolabel annotation.
Input: X~s, Ys, X~t.Output: pseudolabels Y∧t′.1 Learn a classifierf:fX~s,Ys,X~t⟶Y∧t;2Y∧t′′⟵fX~t;3Y∧clu⟵spectralClusterX~t;4Y∧sc=0nt×1;5fori =1: Cdo6 Calculatenij1≤j≤c;7y^=argmax1≤j≤cnij;8Y∧scY∧clu==i=y^;9 end.10Y∧t′=y∧t′i1≤i≤nt, y^it′=y^it′′,y^it′′=y^isc0,otherwiseThe specific implementation process of the method is shown in Algorithm3. In Lines 1-2, a source-domain-learned classifier predicts the target samples. In Line 3, the spectral clustering groups target samples into C clusters. In Lines 4-9, the target clusters are matched with the known categories, where nij interprets the number of pseudolabels equaling i in the jth cluster. Line 10 shows the consistency-aware label filtering strategy. It is worth noting that an iterative framework is utilized. Specifically, during each training session, the prediction of target samples are used as the pseudo labels for the next iteration. An incremental training strategy is used where the pseudo labels with high confidence are fixed in each iteration, and only the remaining pseudo labels are updated in subsequent iterations.


## 4. Experiments

Firstly, we elaborate the benchmark datasets and the experimental setup in this part. The results of comparisons with baseline domain adaptation techniques are then provided and analyzed. Finally, we test the sensitivity of various parameters and give the visualization of the confusion matrix.


### 4.1. Datasets and Evaluation Criteria

Office-Caltech10 [40] is a popular dataset in the object recognition area, which contains 2533 samples across four domains: Amazon (A), Caltech (C), Webcam (W), and DSLR (D). Each domain contains 10 object classes, such as backpacks, bicycles, and calculators.ImageCLEF-DA is a benchmark dataset for ImageCLEF 2014 Domain Adaptation Challenge https://www.imageclef.org/2014/adaptation, which are composed of 1800 samples across three domains: Caltech (C), Imagenet (I), and PASCAL (P). Each domain provides 12 classes of objects, including airplane, bike, and people.Office31 [41] is a medium-size dataset for the object recognition, which has 4110 images in 31 categories. These images are divided into three domains: Amazon (A), DSLR (B), and Webcam (W) according to their source. Specifically, A contains 2817 images with clean background and unified scale; B has 498 low-noise high-resolution pictures (4288×2848); W are composed of 795 low-resolution images with flaws, such as significant noise, color, and white balance artifacts.Office-Home [42] is a larger dataset than Office31 in object recognition area, which includes 15588 images in 65 categories. There are four domains: Art (A), Clipart (C), Product (P), and Real-word (R). Specifically, A has 2427 images in the form of sketches, paintings, etc.; C contains 4365 clipart-images; P consists of the set with 4439 clean-background pictures of products; R includes 15500 images in real world.These four benchmark datasets are widely utilized in UDA and some samples of them are shown in Figure2. Decaf6 features [43] of Office-Caltech10 and Resnet50 features [44] of others datasets are utilized in our experiments. The information of four datasets is shown in Table 2. In terms of every dataset, subtasks are performed by training in one domain and testing in another. For example, C⟶I indicates training on the domain Caltech while testing on the domain Imagenet. Similar to works [27, 28, 45], the predicted accuracy of target samples is utilized as the assessment criteria. The accuracy is computed by the following equation:
(7)Accurary=x:x∈Dt∧fx=yxx:x∈Dt,where f∗ and y∗ represent predicted label and the ground truth label, respectively.Figure 2
Objects in different domain for four datasets.Table 2
Statistics of datasets.
Dataset#Feature#Domain#Object#SampleOffice-caltech1040964102533ImageCLEF-DA20483122391Office3120483314110Office-Home204846515500


### 4.2. Environment

All our experiments ran on a same desktop-computer with a single CPU (Intel(R) Core(TM) i7-4790 CPU @ 3.60 GHz), 16.0 GB RAM, and 256 G SSD. The programming language is MATLAB.


### 4.3. Experimental Setting

The label propagation and spectral clustering both employ thek-nearest neighbor graph, however, the neighbor-node number differs between them, p1=15 for the former, p2=20 for the latter. The dimension of PCA space is d1=256. And the label propagation parameter α is empirically set as 0.9 [37]. In the succeeding subsections, the impact of these hyperparameters on model will be examined in detail.


### 4.4. Baseline Methods

Our strategy is compared to a nonadaptation approach as well as several standard methodologies to assess performance. The follows are a brief introduction of these comparative approaches: the1NN [46] is a baseline method without adaptation. Specifically, a k-nearest neighbor classifier (k=1) learned from the source domain directly predicts the target samples. Using MMD-penalized kernel PCA, the transfer component analysis (TCA) [24] found common latent features to reduce the marginal distribution discrepancy. The latent characteristics were then used to train a classifier. The joint distribution adaptation (JDA) [23] attempted to construct a robust feature representation by simultaneously narrowing the shift in both the marginal distribution and conditional distribution between domains. A classifier was then trained using these features. By integrating an infinite number of subspaces, the geodesic flow kernel (GFK) [40] learned a robust feature representations. Then, a classifier was learned on these representation. The modified A-distance and sparse filtering (MASF) [47] learned common sparse representations by minimizing the modified A-distance between two domains. Then, a classifier was learned from these representations. The joint probability distribution adaptation (JPDA) [48] learned common latent features using modified-MMD (DJP-MMD). Then, a classifier was learned on the latent features. The collaborative representation with curriculum classifier boosting (CRCCB) [49] is aimed at learning directly an adaptive classifier by multistage inference and instance rearranging.


### 4.5. Results and Analysis

Tables3–6 show our classification accuracy results as well as the comparable ones for all subtasks (“source”⟶ “target”) in the four datasets. The average accuracy of several subtasks is also provided. For statistical verification, we repeat our experiment 20 times on each subtask and give the statistical results. The best and second best results in each subtask are represented by bold and underlined fonts, respectively.Table 3
Accuracy (%) on ImageCLEF-DA.
Method1NNTCAJDAGFKMAFSJPDACRCCBCAMLPC⟶I85.1686.6790.8384.6789.8384.0089.8393.70±0.07C⟶P69.1672.0873.4370.3972.8373.6071.9176.35±1.06I⟶C91.1693.3393.6793.0093.1786.8394.1694.67±0.00I⟶P73.1675.6376.4876.1476.8376.3178.8477.70±0.08P⟶C81.3385.3383.8386.0085.3380.6794.5094.33±0.00P⟶I74.5079.1778.5080.0080.8379.1790.8392.30±0.07Aver79.0882.0482.7981.7083.1480.1086.6888.17±0.20Table 4
Accuracy (%) on Office-Caltech10.
Method1NNTCAJDAGFKMAFSJPDACAMLPA⟶C70.3484.3384.6879.5287.3673.8289.49±0.00A⟶D64.9683.4481.5380.8986.6284.7184.2±1.59A⟶W57.2874.9276.2768.4781.0272.8885.08±1.44C⟶A85.6989.3590.4087.7990.8184.7692.17±0.00C⟶D74.5284.7187.2687.2689.8186.6292.99±0.78C⟶W66.1082.0386.4480.3487.4681.0288.34±1.11D⟶A62.7388.6291.0286.4390.4086.5391.96±0.00D⟶C52.0980.5084.5977.5685.7581.7587.44±0.00D⟶W89.1599.3299.3298.6498.98100.0091.80±0.15W⟶A62.5280.5888.5275.8991.3485.3992.38±0.00W⟶C60.3776.6783.7077.1185.0480.2387.80±0.00W⟶D98.72100.00100.00100.0099.36100.0093.50±2.79Aver70.3785.3787.8183.3389.5084.8189.76±0.40Table 5
Accuracy (%) on Office31.
Method1NNTCAJDAGFKMAFSJPDACRCCBCAMLPA⟶D76.9177.5178.7177.5181.1277.7188.5588.71±1.13A⟶W76.4875.7278.1173.8476.9876.9884.5286.89±0.96D⟶A65.3267.1668.6265.2562.6668.1970.8275.04±1.18D⟶W97.9996.8697.1196.8692.0897.2396.8591.62±0.36W⟶A62.7664.6866.8863.2663.6167.7768.2272.96±1.08W⟶D99.4099.2099.0098.5995.7899.0099.3994.54±1.28Aver79.8180.1981.4079.2278.7181.1584.7384.96±0.56Table 6
Accuracy (%) on Office-Home.
Method1NNTCAJDAGFKMAFSJPDACRCCBCAMLPA⟶C35.1235.7636.0628.5029.5542.1547.1945.30±0.00A⟶P56.7958.0858.6650.4847.0856.8865.7171.57±0.71A⟶R64.0360.9861.5355.5756.1464.4771.6376.95±0.65C⟶A48.4542.6043.2238.7736.1847.3063.7361.80±0.50C⟶P57.8554.1656.1650.8946.6560.8969.5275.76±0.51C⟶R59.4955.3457.1051.6247.4958.3070.5776.60±0.34P⟶A51.5547.1047.5142.4438.9049.2464.3161.03±0.78P⟶C40.7341.0141.7635.1435.0344.3148.0148.15±0.50P⟶R68.6365.8067.2061.0760.7868.3078.0178.04±0.34R⟶A58.9255.5856.1249.5349.4055.5870.0462.66±1.06R⟶C42.7340.5543.1435.3338.5647.3851.4544.61±0.42R⟶P73.0870.0671.4866.8867.5271.6880.1381.11±1.19Aver54.7852.2553.3347.1946.1155.5465.0365.30±0.11From Tables3–6, it is concluded that our approach achieves the highest average accuracy consistently on all four datasets. Specifically, on the ImageCLEF-DA dataset (Table 3), our method performs best on almost all subtasks, only slightly worse than CRCCB on two subtasks (1.14%: I⟶P, 0.17%: P⟶C). Our method accomplishes an average accuracy of 88.17%, which is increased by 9.09% (1NN), 6.14% (TCA), 5.38% (JDA), 6.47% (GFK), 5.03% (MASF), 8.07% (JPDA), and 1.49% (CRCCB). On the Office-Caltech10 dataset (Table 4), the average accuracy of ours achieves the best performance. With almost subtasks (9/12), the proposed method wins the first place. Compared with the rest comparison methods, our average accuracy is improved by 19.39% (1NN), 4.39% (TCA), 1.95% (JDA), 6.44% (GFK), 0.26% (MASF), and 4.95% (JPDA).In terms of Office31 dataset (Table5), our method performs the best in four out of six subtasks and ranks first with the average accuracy of 84.96%. Compared with remainder approaches, our average accuracy increases by 5.15% (1NN), 4.77% (TCA), 3.56% (JDA), 5.74% (GFK), 6.26% (MASF), 3.81% (JPDA), and 0.23% (CRCCB). On the Office-Home dataset (Table 6), the average accuracy of ours is 65.30%, which improves by 10.52% (1NN), 13.05% (TCA), 11.97% (JDA), 18.11% (GFK), 19.19% (MASF), 9.76% (JPDA), and 0.27% (CRCCB). In general, the CRCCB and our method performs roughly the same on the two large datasets (Office31, Office-Home), both far better than the other methods. And our method performs best on two relatively small datasets (ImageCLEF-DA, Office-Caltech10).


### 4.6. Ablation Study

An ablation research is designed to determine how various aspects of our work contributes to the ultimate result. To this goal, we look into various combinations of components such as modified label propagation (MLP) and consistency-aware pseudolabel annotation (CPLA). Specifically, three groups of experiments are conducted. To predict target samples, the first group exclusively uses modified label propagation (MLP) and the initial pseudolabels annotated through a source-domain-trained classifier; the second group directly predicts target samples by CPLA; and the third group combines the MLP and CPLA. To be fair, the three groups use the same 1NN classifier learned only from the source domain. Table7 shows the average accuracy for the four datasets. It is obvious that MLP and CPLA are ineffective on their own. However, combining them significantly improves their performance.Table 7
Results of the ablation study. “✓” indicates using the corresponding method, “×” otherwise.
CPLAMLPaverage accuracy (%)ImageCLEF-DAOffice-CaltechOffice31Office-Home×✓80.2883.6579.8154.80✓×75.7977.5970.9542.29✓✓88.17±0.2089.76±0.4084.96±0.5665.3±0.11


### 4.7. Parameter Sensitivity Analysis

Our approach has four hyperparameters:d1—dimension of PCA space, p1—number of the nearest neighbors in label propagation, p2—number of the nearest neighbors in spectral clustering, and T—the maximum iteration. Setting each hyperparameter to a series of different values while fixing others allows us to see how it affects performance. Figure 3 shows the average accuracy across all subtasks, with the parameters used in this paper marked by a red dotted circle. A larger d1 is obviously necessary for datasets with more categories (Office31, Office-Home), whereas d1 has little influence on model performance for tasks with few categories (ImageCLEF-DA, Office-Caltech), as shown in Figure 3(a). This is owing to the fact that as the number of categories grows, more features are required. When p1 is miniscule, the performance drops dramatically on Office31, but less on others datasets. Different p2 has the greatest influence on the performance of Office-Home (Figure 3(d)). For the four datasets, different T has little effect on the performance, implying that the spectral clustering pseudo labels are nearly the same in each iteration (Figure 3(c)). Our approach swiftly converges after employing the consistent strategy (only one or two iterations).Figure 3
Sensitivity analysis of hyperparameters: (a) PCA dimensiond1, (b) the number of nearest neighbors p1 in label propagation, (c) the maximum iteration T, and (d) the number of nearest neighbors p2 in spectral clustering.
(a)(b)(c)(d)


### 4.8. Confusion Matrix Visualization

In this part, the performance of CAMLP is compared with that of 1NN (without domain adaptation). Figure4 illustrates the visualizations of confusion matrix for the two methods on tasks P⟶I and C⟶P from ImageCLEF-DA. Figures 4(a) and 4(b) directly reflect the negative impact brought by domain discrepancy. Clearly, the confusion for most classes is significantly reduced in CAMLP by comparing (a) and (b) and (c) and (d) in Figure 4. For example, the confused boat, bus, and horse are correctly identified by CAMLP in task P⟶I, and the confusion of bike, boat, bottle, and people also is reduced by CAMLP in task C⟶P. Unfortunately, neither CAMLP nor 1NN can handle the problem of misclassifying boat as car on task C⟶P. These observations further reveal that CAMLP has excellent discrimination in the target domain.Figure 4
The confusion matrix visualization of 1NN and CAMLP (best viewed in color).
(a)
1NN (P → I)(b)
CAMLP (P → I)(c)
1NN (C → P)(d)
CAMLP (C → P)


## 4.1. Datasets and Evaluation Criteria

Office-Caltech10 [40] is a popular dataset in the object recognition area, which contains 2533 samples across four domains: Amazon (A), Caltech (C), Webcam (W), and DSLR (D). Each domain contains 10 object classes, such as backpacks, bicycles, and calculators.ImageCLEF-DA is a benchmark dataset for ImageCLEF 2014 Domain Adaptation Challenge https://www.imageclef.org/2014/adaptation, which are composed of 1800 samples across three domains: Caltech (C), Imagenet (I), and PASCAL (P). Each domain provides 12 classes of objects, including airplane, bike, and people.Office31 [41] is a medium-size dataset for the object recognition, which has 4110 images in 31 categories. These images are divided into three domains: Amazon (A), DSLR (B), and Webcam (W) according to their source. Specifically, A contains 2817 images with clean background and unified scale; B has 498 low-noise high-resolution pictures (4288×2848); W are composed of 795 low-resolution images with flaws, such as significant noise, color, and white balance artifacts.Office-Home [42] is a larger dataset than Office31 in object recognition area, which includes 15588 images in 65 categories. There are four domains: Art (A), Clipart (C), Product (P), and Real-word (R). Specifically, A has 2427 images in the form of sketches, paintings, etc.; C contains 4365 clipart-images; P consists of the set with 4439 clean-background pictures of products; R includes 15500 images in real world.These four benchmark datasets are widely utilized in UDA and some samples of them are shown in Figure2. Decaf6 features [43] of Office-Caltech10 and Resnet50 features [44] of others datasets are utilized in our experiments. The information of four datasets is shown in Table 2. In terms of every dataset, subtasks are performed by training in one domain and testing in another. For example, C⟶I indicates training on the domain Caltech while testing on the domain Imagenet. Similar to works [27, 28, 45], the predicted accuracy of target samples is utilized as the assessment criteria. The accuracy is computed by the following equation:
(7)Accurary=x:x∈Dt∧fx=yxx:x∈Dt,where f∗ and y∗ represent predicted label and the ground truth label, respectively.Figure 2
Objects in different domain for four datasets.Table 2
Statistics of datasets.
Dataset#Feature#Domain#Object#SampleOffice-caltech1040964102533ImageCLEF-DA20483122391Office3120483314110Office-Home204846515500


## 4.2. Environment

All our experiments ran on a same desktop-computer with a single CPU (Intel(R) Core(TM) i7-4790 CPU @ 3.60 GHz), 16.0 GB RAM, and 256 G SSD. The programming language is MATLAB.


## 4.3. Experimental Setting

The label propagation and spectral clustering both employ thek-nearest neighbor graph, however, the neighbor-node number differs between them, p1=15 for the former, p2=20 for the latter. The dimension of PCA space is d1=256. And the label propagation parameter α is empirically set as 0.9 [37]. In the succeeding subsections, the impact of these hyperparameters on model will be examined in detail.


## 4.4. Baseline Methods

Our strategy is compared to a nonadaptation approach as well as several standard methodologies to assess performance. The follows are a brief introduction of these comparative approaches: the1NN [46] is a baseline method without adaptation. Specifically, a k-nearest neighbor classifier (k=1) learned from the source domain directly predicts the target samples. Using MMD-penalized kernel PCA, the transfer component analysis (TCA) [24] found common latent features to reduce the marginal distribution discrepancy. The latent characteristics were then used to train a classifier. The joint distribution adaptation (JDA) [23] attempted to construct a robust feature representation by simultaneously narrowing the shift in both the marginal distribution and conditional distribution between domains. A classifier was then trained using these features. By integrating an infinite number of subspaces, the geodesic flow kernel (GFK) [40] learned a robust feature representations. Then, a classifier was learned on these representation. The modified A-distance and sparse filtering (MASF) [47] learned common sparse representations by minimizing the modified A-distance between two domains. Then, a classifier was learned from these representations. The joint probability distribution adaptation (JPDA) [48] learned common latent features using modified-MMD (DJP-MMD). Then, a classifier was learned on the latent features. The collaborative representation with curriculum classifier boosting (CRCCB) [49] is aimed at learning directly an adaptive classifier by multistage inference and instance rearranging.


## 4.5. Results and Analysis

Tables3–6 show our classification accuracy results as well as the comparable ones for all subtasks (“source”⟶ “target”) in the four datasets. The average accuracy of several subtasks is also provided. For statistical verification, we repeat our experiment 20 times on each subtask and give the statistical results. The best and second best results in each subtask are represented by bold and underlined fonts, respectively.Table 3
Accuracy (%) on ImageCLEF-DA.
Method1NNTCAJDAGFKMAFSJPDACRCCBCAMLPC⟶I85.1686.6790.8384.6789.8384.0089.8393.70±0.07C⟶P69.1672.0873.4370.3972.8373.6071.9176.35±1.06I⟶C91.1693.3393.6793.0093.1786.8394.1694.67±0.00I⟶P73.1675.6376.4876.1476.8376.3178.8477.70±0.08P⟶C81.3385.3383.8386.0085.3380.6794.5094.33±0.00P⟶I74.5079.1778.5080.0080.8379.1790.8392.30±0.07Aver79.0882.0482.7981.7083.1480.1086.6888.17±0.20Table 4
Accuracy (%) on Office-Caltech10.
Method1NNTCAJDAGFKMAFSJPDACAMLPA⟶C70.3484.3384.6879.5287.3673.8289.49±0.00A⟶D64.9683.4481.5380.8986.6284.7184.2±1.59A⟶W57.2874.9276.2768.4781.0272.8885.08±1.44C⟶A85.6989.3590.4087.7990.8184.7692.17±0.00C⟶D74.5284.7187.2687.2689.8186.6292.99±0.78C⟶W66.1082.0386.4480.3487.4681.0288.34±1.11D⟶A62.7388.6291.0286.4390.4086.5391.96±0.00D⟶C52.0980.5084.5977.5685.7581.7587.44±0.00D⟶W89.1599.3299.3298.6498.98100.0091.80±0.15W⟶A62.5280.5888.5275.8991.3485.3992.38±0.00W⟶C60.3776.6783.7077.1185.0480.2387.80±0.00W⟶D98.72100.00100.00100.0099.36100.0093.50±2.79Aver70.3785.3787.8183.3389.5084.8189.76±0.40Table 5
Accuracy (%) on Office31.
Method1NNTCAJDAGFKMAFSJPDACRCCBCAMLPA⟶D76.9177.5178.7177.5181.1277.7188.5588.71±1.13A⟶W76.4875.7278.1173.8476.9876.9884.5286.89±0.96D⟶A65.3267.1668.6265.2562.6668.1970.8275.04±1.18D⟶W97.9996.8697.1196.8692.0897.2396.8591.62±0.36W⟶A62.7664.6866.8863.2663.6167.7768.2272.96±1.08W⟶D99.4099.2099.0098.5995.7899.0099.3994.54±1.28Aver79.8180.1981.4079.2278.7181.1584.7384.96±0.56Table 6
Accuracy (%) on Office-Home.
Method1NNTCAJDAGFKMAFSJPDACRCCBCAMLPA⟶C35.1235.7636.0628.5029.5542.1547.1945.30±0.00A⟶P56.7958.0858.6650.4847.0856.8865.7171.57±0.71A⟶R64.0360.9861.5355.5756.1464.4771.6376.95±0.65C⟶A48.4542.6043.2238.7736.1847.3063.7361.80±0.50C⟶P57.8554.1656.1650.8946.6560.8969.5275.76±0.51C⟶R59.4955.3457.1051.6247.4958.3070.5776.60±0.34P⟶A51.5547.1047.5142.4438.9049.2464.3161.03±0.78P⟶C40.7341.0141.7635.1435.0344.3148.0148.15±0.50P⟶R68.6365.8067.2061.0760.7868.3078.0178.04±0.34R⟶A58.9255.5856.1249.5349.4055.5870.0462.66±1.06R⟶C42.7340.5543.1435.3338.5647.3851.4544.61±0.42R⟶P73.0870.0671.4866.8867.5271.6880.1381.11±1.19Aver54.7852.2553.3347.1946.1155.5465.0365.30±0.11From Tables3–6, it is concluded that our approach achieves the highest average accuracy consistently on all four datasets. Specifically, on the ImageCLEF-DA dataset (Table 3), our method performs best on almost all subtasks, only slightly worse than CRCCB on two subtasks (1.14%: I⟶P, 0.17%: P⟶C). Our method accomplishes an average accuracy of 88.17%, which is increased by 9.09% (1NN), 6.14% (TCA), 5.38% (JDA), 6.47% (GFK), 5.03% (MASF), 8.07% (JPDA), and 1.49% (CRCCB). On the Office-Caltech10 dataset (Table 4), the average accuracy of ours achieves the best performance. With almost subtasks (9/12), the proposed method wins the first place. Compared with the rest comparison methods, our average accuracy is improved by 19.39% (1NN), 4.39% (TCA), 1.95% (JDA), 6.44% (GFK), 0.26% (MASF), and 4.95% (JPDA).In terms of Office31 dataset (Table5), our method performs the best in four out of six subtasks and ranks first with the average accuracy of 84.96%. Compared with remainder approaches, our average accuracy increases by 5.15% (1NN), 4.77% (TCA), 3.56% (JDA), 5.74% (GFK), 6.26% (MASF), 3.81% (JPDA), and 0.23% (CRCCB). On the Office-Home dataset (Table 6), the average accuracy of ours is 65.30%, which improves by 10.52% (1NN), 13.05% (TCA), 11.97% (JDA), 18.11% (GFK), 19.19% (MASF), 9.76% (JPDA), and 0.27% (CRCCB). In general, the CRCCB and our method performs roughly the same on the two large datasets (Office31, Office-Home), both far better than the other methods. And our method performs best on two relatively small datasets (ImageCLEF-DA, Office-Caltech10).


## 4.6. Ablation Study

An ablation research is designed to determine how various aspects of our work contributes to the ultimate result. To this goal, we look into various combinations of components such as modified label propagation (MLP) and consistency-aware pseudolabel annotation (CPLA). Specifically, three groups of experiments are conducted. To predict target samples, the first group exclusively uses modified label propagation (MLP) and the initial pseudolabels annotated through a source-domain-trained classifier; the second group directly predicts target samples by CPLA; and the third group combines the MLP and CPLA. To be fair, the three groups use the same 1NN classifier learned only from the source domain. Table7 shows the average accuracy for the four datasets. It is obvious that MLP and CPLA are ineffective on their own. However, combining them significantly improves their performance.Table 7
Results of the ablation study. “✓” indicates using the corresponding method, “×” otherwise.
CPLAMLPaverage accuracy (%)ImageCLEF-DAOffice-CaltechOffice31Office-Home×✓80.2883.6579.8154.80✓×75.7977.5970.9542.29✓✓88.17±0.2089.76±0.4084.96±0.5665.3±0.11


## 4.7. Parameter Sensitivity Analysis

Our approach has four hyperparameters:d1—dimension of PCA space, p1—number of the nearest neighbors in label propagation, p2—number of the nearest neighbors in spectral clustering, and T—the maximum iteration. Setting each hyperparameter to a series of different values while fixing others allows us to see how it affects performance. Figure 3 shows the average accuracy across all subtasks, with the parameters used in this paper marked by a red dotted circle. A larger d1 is obviously necessary for datasets with more categories (Office31, Office-Home), whereas d1 has little influence on model performance for tasks with few categories (ImageCLEF-DA, Office-Caltech), as shown in Figure 3(a). This is owing to the fact that as the number of categories grows, more features are required. When p1 is miniscule, the performance drops dramatically on Office31, but less on others datasets. Different p2 has the greatest influence on the performance of Office-Home (Figure 3(d)). For the four datasets, different T has little effect on the performance, implying that the spectral clustering pseudo labels are nearly the same in each iteration (Figure 3(c)). Our approach swiftly converges after employing the consistent strategy (only one or two iterations).Figure 3
Sensitivity analysis of hyperparameters: (a) PCA dimensiond1, (b) the number of nearest neighbors p1 in label propagation, (c) the maximum iteration T, and (d) the number of nearest neighbors p2 in spectral clustering.
(a)(b)(c)(d)


## 4.8. Confusion Matrix Visualization

In this part, the performance of CAMLP is compared with that of 1NN (without domain adaptation). Figure4 illustrates the visualizations of confusion matrix for the two methods on tasks P⟶I and C⟶P from ImageCLEF-DA. Figures 4(a) and 4(b) directly reflect the negative impact brought by domain discrepancy. Clearly, the confusion for most classes is significantly reduced in CAMLP by comparing (a) and (b) and (c) and (d) in Figure 4. For example, the confused boat, bus, and horse are correctly identified by CAMLP in task P⟶I, and the confusion of bike, boat, bottle, and people also is reduced by CAMLP in task C⟶P. Unfortunately, neither CAMLP nor 1NN can handle the problem of misclassifying boat as car on task C⟶P. These observations further reveal that CAMLP has excellent discrimination in the target domain.Figure 4
The confusion matrix visualization of 1NN and CAMLP (best viewed in color).
(a)
1NN (P → I)(b)
CAMLP (P → I)(c)
1NN (C → P)(d)
CAMLP (C → P)


## 5. Discussion

We propose a classifier adaptation method based on modified label propagation. And this method builds different domains in the same graph. We innovatively propose determining relationships between cross-domain samples through pseudolabeling. To alleviate the damage of wrong labels to the model, a consistency-aware pseudolabel annotation method is proposed to improve the accuracy of pseudo labels. Therefore, our method is more suitable for structurally obvious tasks.According to the experimental results in Section4, the proposed method achieves better performance in most tasks but has the problem of negative transfer in a few tasks, e.g., D⟶W(Office31), W⟶D(Office31), and W⟶D(Office-Caltech10). Clearly, the three tasks have one common character: very small domain discrepancy. We attribute this to the fact that our method give more weight to the structural information in the target domain than the knowledge from the source domain. In addition, our method is insensitive to the number of iterations. This maybe be due to the knowledge learned in each iteration is not well inherited. For example, the relationships within the target domain are not updated after each iteration. Addressing this problem will give full play to the advantages of iterative optimization, which is one of the key points of our next work.


## 6. Conclusion

We propose a classifier adaptation method-based modified label propagation (CAMLP) in this paper. The suggested method consists of two parts: the modified label propagation (MLP) and the consistency-aware pseudolabel annotation (CPLA). MLP builds different-domain samples into a same graph and then transfers the labels from the source domain to the target domain. The CPLA coordinates the source-domain knowledge and the target-domain underlying structure to correct pseudo labels. On the four real-world datasets, sufficient experiments show that the suggested method is superior to state-of-the-art methods. The importance of graph structure information in UDA is demonstrated in this research. For future work, we will investigate the usage of graph neural networks (GNN) in unsupervised domain adaptation, considering that GNN can better extract and utilize structural information among data.



---

*Source: 2963195-2022-08-09.xml*