Multi‐metric domain adaptation for unsupervised transfer learning

IET Image ProcessingVolume 14, Issue 12 p. 2780-2790 Research ArticleFree Access Multi-metric domain adaptation for unsupervised transfer learning Hongwei Yang, Hongwei Yang School of Computer Science and Technology, Harbin Institute of Technology, Harbin, People's Republic of ChinaSearch for more papers by this authorHui He, Corresponding Author Hui He hehui@hit.edu.cn School of Computer Science and Technology, Harbin Institute of Technology, Harbin, People's Republic of ChinaSearch for more papers by this authorTao Li, Tao Li School of Computer Science and Technology, Harbin Institute of Technology, Harbin, People's Republic of ChinaSearch for more papers by this authorYawen Bai, Yawen Bai School of Computer Science and Technology, Harbin Institute of Technology, Harbin, People's Republic of ChinaSearch for more papers by this authorWeizhe Zhang, Weizhe Zhang School of Computer Science and Technology, Harbin Institute of Technology, Harbin, People's Republic of China Peng Cheng Laboratory, Shenzhen, People's Republic of ChinaSearch for more papers by this author Hongwei Yang, Hongwei Yang School of Computer Science and Technology, Harbin Institute of Technology, Harbin, People's Republic of ChinaSearch for more papers by this authorHui He, Corresponding Author Hui He hehui@hit.edu.cn School of Computer Science and Technology, Harbin Institute of Technology, Harbin, People's Republic of ChinaSearch for more papers by this authorTao Li, Tao Li School of Computer Science and Technology, Harbin Institute of Technology, Harbin, People's Republic of ChinaSearch for more papers by this authorYawen Bai, Yawen Bai School of Computer Science and Technology, Harbin Institute of Technology, Harbin, People's Republic of ChinaSearch for more papers by this authorWeizhe Zhang, Weizhe Zhang School of Computer Science and Technology, Harbin Institute of Technology, Harbin, People's Republic of China Peng Cheng Laboratory, Shenzhen, People's Republic of ChinaSearch for more papers by this author First published: 11 September 2020 https://doi.org/10.1049/iet-ipr.2019.1434Citations: 1AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract Unsupervised domain adaptation aims to learn a classifier for the unlabelled target domain by leveraging knowledge from a labelled source domain. This study presents a novel domain adaptation framework from global and local transfer perspectives, referred to as multi-metric domain adaptation (MMDA) for unsupervised transfer learning. At the global level, MMDA minimises the marginal and within-class distances and maximises the between-class distance between domains while maintaining the features of the source domain to improve the cross-domain adaptability. At the local level, MMDA exploits both in- and cross-domain manifold structures embedded in data samples to increase the discriminative ability. The authors learn a coupled transformation that projects the source and target domain data onto respective subspace where the statistical and geometrical divergences are reduced simultaneously. They formulate global and local adaptation methods in an optimisation problem and derive an analytic solution to the objective function. Extensive experiments demonstrate that MMDA shows improvements in classification accuracy compared with several existing state-of-the-art methods. Nomenclature source domain data target domain data kernel matrix , source/target domain projection matrix , Laplacian matrices of the in-domain manifolds , Laplacian matrices of the cross-domain manifolds n, m number of samples in source/target domain p number of nearest neighbours weight of manifold adaptation , trade-off parameters for the maximum mean discrepancy adaptation trade-off parameter for source domain between-class scatter v trade-off parameter for the in- and cross-domain manifolds 1 Introduction Transfer learning has been successfully applied to numerous real-world applications. Domain adaptation [1] is a major subtopic of transfer learning that aims to solve the problem in the target domain by using the training data in the related source domain even when these domains may have different distributions. Domain adaptation has been remarkably successful in various applications, such as sentiment classification [2], natural language processing [3], and visual recognition [4]. Domain adaptation [5] can be divided into supervised, semi-supervised, and unsupervised. Supervised and semi-supervised methods [6, 7] need a certain amount of labelled data in the target domain, while unsupervised approaches [8, 9] do not need labelled data. In this study, we focus on the unsupervised domain adaptation, which is considered to be more challenging and practical than the other methods. Metric learning [10] has attracted increasing attention due to its critical role in image processing and classification. Many metric learning-based domain adaptation studies are currently emerging, such as Riemann distance-based transfer learning [11, 12] and other metric-based methods [13, 14]. The most common approaches of unsupervised domain adaptation at present are instance- and feature-based adaptation methods [15, 16]. In general, we can solve the problem of domain adaptation by reducing the cross-domain discrepancy. For this reason, existing methods can be classified into two categories: (i) sample reweighing [17–19], which reuses the instances in the source domain to better match the distribution of the target domain; (ii) feature transformation, which utilises subspace learning theory to study the geometrical structure of the subspaces [20–23] or align the distributions to minimise the marginal or conditional distribution divergence between domains [8]. This study follows the idea of feature transformation. Two basic kinds of feature transformation are available [5]: data- and subspace-centric methods. Data-centric methods aim to find a transformation that projects data from the source and target domains into a domain-invariant space to minimise the divergence of marginal or conditional distributions or both while preserving the data properties in the original spaces [5, 8, 24]. However, they fail to exploit the specific feature in two domains because of the lack of a common subspace where the distributions of the two domains are sufficiently similar and the features are preserved at the same time. The subspace-centric methods conduct subspace transformation to obtain improved feature representation. However, the subspace-centric methods only manipulate the subspaces of the two domains without explicitly considering the within and between-class distances between the projected data of the two domains. The major problem of domain adaptation is the way to improve the ability of cross-domain adaptation. A mainstream approach to this problem is to learn domain invariant features by minimising distance metrics of cross-domain discrepancy [24]. However, when the cross-domain discrepancy is large, some source instances are independent of the target domain and even use the domain-invariant features, which may cause a large deviation to the transfer classifier. Another important strategy is to boost the ability of the classifier for distinguishing samples to accurately identify similar samples in domains [8, 21]. However, when the cross-domain discrepancy is large, a large number of source data will be degraded, which results in a small effective set of data for the training transfer classifier. This condition will result in a transfer classifier with poor generalisation ability but high variance, which is not robust for the large cross-domain discrepancy. In the aforementioned challenging scenario, either improving the adaptation ability or increasing the discriminative ability can reduce but not remove the domain discrepancy. The adaptation and discriminative abilities should be enhanced jointly for robust domain adaptation. As far as we know, no previous study considers the adaptation ability in the global transfer perspective and the discriminative ability in the local manifold view. In this study, we propose a unified multi-metric domain adaptation (MMDA) framework motivated by two key insights of domain adaptation, as illustrated in Fig. 1. The first insight is to improve global adaptability by preserving features in the source domain on the one hand and by minimising the marginal maximum mean discrepancy (MMD), decreasing the within-class MMD, and increasing the between-class MMD on the other hand. Another insight is to increase local discriminative ability by minimising the weighted manifold adaptation between domains. Data from source and target domains are embedded in in- and cross-domain manifolds. Thus, the distance between samples from the same class but different domains cannot be equal to the distance between samples from the same class and domain. MMDA pays close attention to the local relationship between domains. After the coupled transformation, MMDA minimises the weighted manifolds jointly to reduce the domain mismatch. Fig. 1Open in figure viewerPowerPoint The main idea of our approach (a) Original data, (b) Globally aligned data, (c) Locally adapted data The main contributions of this study are summarised as follows: (1) We extend the distribution matching to discriminative information adaptation based on the non-parametric MMD to globally increase the discriminative ability. (2) We propose a novel graph structure under the framework of graph embedding to locally adapt the in- and cross-domain manifolds. (3) A large number of experiments on eight real-world image datasets validate that MMDA outperforms not only traditional methods but also some end-to-end deep methods. The rest of the paper is organised as follows: in Section 2, we briefly review related works on domain adaptation. We describe the proposed MMDA framework in detail in Section 3. The results of experiments conducted on public large-scale datasets are presented in Section 4. We make a conclusion in Section 5. 2 Related works In this section, we introduce the related works from the perspective of transfer learning and the current mainstream approaches to domain adaptation. With the continuous development and progress of artificial intelligence technology, many different transfer learning methods have emerged. Sparse representation [25, 26] of high-dimensional data is a major research topic in the field of machine learning and computer vision in recent years, and its representation results have a good performance in pattern recognition. Many of the transfer learning systems often choose sparse representation as their key module [27–30]. At present, the non-deep domain adaptation methods can be divided into distribution, geometrical, and manifold adaptation. Distribution adaptation. Some existing domain adaptation studies align marginal distribution, such as transfer component analysis [24]; others align conditional distribution; the rest align marginal and conditional distributions, such as joint distribution alignment (JDA) [8], adaptation regularisation-based transfer learning (ARTL) [4], transfer joint matching (TJM) [31], joint geometrical and statistical alignment (JGSA) [5], and distribution matching machine (DMM) [32]. Among these methods, ARTL and DMM aim to build a domain-invariant classifier based on structural risk minimisation [33]. Most existing methods adapt marginal and conditional distributions with equal weights and do not evaluate the relative importance of the two distributions [4, 5, 30, 34, 35]. Balanced distribution adaptation [36] and manifold embedded distribution alignment [37] attempt to adapt two distributions with balanced weights and achieves better results. In this study, we minimise the difference between domains by minimising the marginal MMD, decreasing the within-class MMD, and increasing the between-class MMD rather than minimising the marginal or conditional distribution shift. The distinction between MMDA and these methods is that MMDA not only improves global adaptability by adapting discriminative class information and preserving the source domain features but also increases local discriminative ability by optimising the underlying data characteristics from the manifold perspective. Geometrical and manifold adaptation. Some domain adaptation approaches adapt to geometrical characteristics. Subspace alignment [9] performs subspace alignment. Subspace distribution alignment [38] aligns the distributions of the subspace. Geodesic flow kernel (GFK) [21] extends the idea of a sample in a Grassmann manifold and proposes to study the GFK between the source and the target domain. Correlation alignment [39] aligns subspaces in second-order statistics without aligning the distributions. Scatter component analysis (SCA) [40] transforms samples into a set of subspaces and then reduces the distance between the subspaces based on Fisher's linear discriminant analysis (FLDA). Optimal transport [41] is an optimal transportation model to conduct the alignment of the representations in the source and target domains. Some other domain adaptation studies perform manifold adaptation, which belongs to the category of spectral clustering [42–44]. Laplacian manifold regularisation is an important spectral clustering method, which has been applied to the study of domain adaptation and obtained satisfactory results, such as [4, 37, 45]. The main difference between MMDA and these methods is that MMDA not only minimises the domain discrepancy by locally exploiting manifold properties to reduce domain divergence but also globally reduces the domain shift by optimising discriminative information adaptation and maintaining the source domain scatters. With the progress of deep learning, a large number of deep transfer learning algorithms have emerged [46–48]. These deep transfer learning methods can be roughly divided into three categories [45]: (i) distance-based approaches [47, 49, 50], which minimise the distribution distance between source and target domains; (ii) adversarial-based approaches [51–53], which perform domain adaptation based on deep adversarial network; (iii) reconstruction-based approaches [49, 54], which exploit the cross-domain relations. Compared with non-deep domain adaptation methods, deep transfer learning approaches integrate feature learning and knowledge transfer into a unified framework. In general, deep transfer learning methods perform better than non-deep ones. 3 Multi-metric domain adaptation In this section, we present the MMDA approach in detail. Notations, which are frequently used in this paper, are summarised in Nomenclature section. 3.1 Problem definition Definition 1.A labelled source domain is denoted as and an unlabelled target domain is denoted as . We assume the feature space , the label space , but marginal probability distribution with conditional probability distribution . The purpose of unsupervised domain adaptation [37] is to learn a classifier to predict the labels for the target domain using the related labelled source domain . The source and target domain data are denoted as and , respectively. According to the classical FLDA [55], a ratio of scatters exists (1)where and are between- and within-class scatter matrices, respectively. Maximising FLDA increases the separation of the data points with respect to the class cluster. However, the classification performance of FLDA will be decreased when the domain divergence between and is considerably large. Thus, we must add additional constraints to minimise the distribution divergence while learning a domain invariant classifier f. 3.2 Main idea MMDA is formulated by finding two coupled transformations ( and for source and target domains, respectively) to obtain new representations of respective domains. Then, MMDA reduces the domain mismatches from the global and local levels. At the global level, MMDA preserves features of the source domain and optimises the discriminative information adaptation between domains by minimising the marginal MMD and within-class MMD and maximising the between-class MMD to improve the cross-domain adaptability. At the local level, MMDA exploits manifold adaptation to increase the discriminative ability between domains. Thus, our objective can be formulated as (2)where indicates the discriminative information adaptation, which is used to reduce the marginal MMD, decrease within-class MMD, and increase between-class MMD simultaneously. represents the weighted manifold regularisation, which is used to preserve the local manifold structures. and indicate the between- and within-class scatters in the source domain, respectively, which are used to preserve features in the source domain. 3.2.1 Source domain feature preservation Given that labels in the source domain are available, we can use the label information to constrain the new representation of the data of the source domain to be discriminative [5]. Then, the between- and within-class scatters can be denoted as (3) (4)where , , C is the number of classes, is the set of source samples that belong to class c, , , and is the centring matrix of data within class c, is the identity matrix, is a column vector with all ones, and is the number of source samples in class c. 3.2.2 Discriminative information adaptation We perform discriminative information adaptation from minimising the marginal distribution shift, minimising the within-class distance, and maximising the between-class distance. Formally, the discriminative information adaptation can be formulated as (5)where and are trade-off parameters, and parameter is either equal to or equal to 0. We compute the marginal distribution shift, within-class, and between-class distances by exploiting the projected MMD methods [4, 36, 56]. The marginal MMD can be denoted as (6)the within-class MMD can be denoted as (7)correspondingly, the between-class MMD can be represented as (8)where , , and . and are the sets of examples that belong to class c in source and target domains, respectively. and are the true and predicted labels of and . and are the sets of samples that belong to class c in source and target domains, respectively. Given that no labels are present in the target domain data, ARTL [4] uses the pseudo-target labels predicted by certain supervised classifiers trained on the source domain. Thereafter, the pseudo labels are refined iteratively to reduce the discrepancy between two domains. We exploit their idea in our framework. Thus, the final discriminative information adaptation can be transformed into its matrix form as follows: (9)where (10) (11) (12) (13)where (14) (15) (16) 3.2.3 Weighted manifold adaptation The idea of manifold adaptation is that a sample tends to have the same label as its nearest neighbours. In domain adaptation, data from the same class should stay close during the distribution matching. As we have stated before, data from the source and target domains are embedded in in- and cross-domain manifolds and the distance between samples from the same class but different domains cannot be equal to the distance between samples from the same class and same domain. Thus, we should minimise the weighted in- and cross-domain manifolds together to maximise the discriminative ability of the learned model. We let denote a graph with n nodes, and each node corresponds to a sample in feature space In-domain manifold: An edge is placed between nodes i and j if , and is the p nearest neighbours of . Cross-domain manifold: An edge is placed between nodes i and j if , and is the p nearest neighbours of . We jointly minimise in-domain manifolds (e.g. and ) and cross-domain manifolds (e.g. and ) to maximise the local discriminative ability. The weighted manifold adaptation can be transformed to the matrix form as (17)where (18) (19)where , is a diagonal matrix with elements on the main diagonal (20)where is the set of p-nearest neighbours of point . 3.2.4 Overall reformulation We follow [5, 40] to impose the constraint that is sufficiently small to control the scale of solution . We encourage subspaces and to be close by minimising . Substituting with (3), (4), (9), and (17). Then, the objective function (2) can be reformulated as (21)where (22)is the scatter matrix of the target domain, is the column vector with all elements equal to 1, and , , , , is the identity matrix. 3.3 Problem optimisation To facilitate the optimisation solution, we denote (23)and we obtain the optimisation function that can be denoted as follows: (24)We can reformulate (24) as (25)We rewrite it in the form of Lagrange function as follows: (26)Then, setting the first derivative to zero, we get (27)where are the k leading eigenvalues and contains the corresponding eigenvectors. Algorithm 1 (see Fig. 2) provides a complete summary of MMDA. Fig. 2Open in figure viewerPowerPoint Algorithm 1: MMDA 3.4 Computational complexity Here, we analyse the computational complexity of the MMDA Algorithm 1 (Fig. 2) by using the big O notation. We denote T as the number of iterations, the subspace bases k are not , and T is not . Thus, and . The computational cost is detailed as follows. Computing matrices , , and take , , and (Line 1), respectively; constructing matrices , , , and take , , , and (Line 1), respectively; for solving the generalised eigendecomposition problem (Line 4); for all other computational processes. In summary, the total computational complexity of Algorithm 1 (Fig. 2) is . 4 Experiments and evaluations In this section, we summarise the complete domain adaptation results over a range of cross-domain object recognition tasks. The image samples are shown in Fig. 3 and the description is displayed in Table 1. Fig. 3Open in figure viewerPowerPoint Image samples from USPS, MNIST, Office, Caltech-256, COIL20, ImageNet, and VOC2007 datasets. Details are described in Section 4.1. Table 1. Eight benchmark datasets we used in this study Dataset Name Data Features Classes Domain Office-10 1410 800(4096) 10 A,W,D Caltech-10 1123 800(4096) 10 C Office-31 4652 4,096 31 A,W,D USPS 1800 256 10 U MNIST 2000 256 10 M ImageNet 7341 4096 5 I VOC2007 3376 4096 5 V COIL20 1440 1,024 20 COIL1,COIL2 4.1 Data setup The public image datasets in our experiments are Office+Caltech10, Office-31, USPS+MNIST, ImageNet+VOC2007, and COIL20, which are popular benchmark image datasets that are widely used to evaluate transfer learning algorithms, such as [4, 5, 21]. Office+Caltech10 [37, 57] has 2533 images in ten different subcategories. The dataset contains four real-world object domains Amazon (A), DSLR (D), Webcam (W), and Caltech (C). Given that the features in Office and Caltech follow different distributions, domain adaptation will help perform cross-domain recognition. Formally, ten classes are used in their respective datasets. In our experiment, we choose Office+Caltech10 from [21], which consists of 12 tasks AC, AD, AW, …, and DW. We use AB to denote the knowledge transfer from the source domain A to the target domain B. USPS+MNIST (U, M) [37] consists of raw images sampled from datasets of handwritten digits from 0 to 9. Given that the same digits in two datasets follow different distributions, domain adaptation can be conducted. MNIST has 60,000 training images and 10,000 test images with a size of 2828. USPS contains 7291 training images and 2007 test images of 1616. Two tasks, namely, MU and UM, are constructed. ImageNet+VOC2007 (I, V) [37] are standard large-scale image recognition datasets. Each dataset can be viewed as one domain and the images from the same classes of two domains follow different distributions. In this study, we use the datasets in [58] to perform domain adaptation tasks. Five classes common to both datasets, namely, , , , , and , are selected. Two other tasks, namely IV and VI, are also considered. COIL20 [59] includes 20 objects with 1440 greyscale images. For each object, 72 images with a size of 3232 are taken at pose intervals of 5 rotating through 360. Following [8, 59], we split the dataset into two subsets: COIL1 and COIL2. Specifically, COIL1 consists of all the images at the directions of and COIL2 contains images of . COIL1 and COIL2 include the same objects with diverse shooting degrees. Thus, they follow different but related distributions. Two additional tasks, COIL1COIL2 and COIL2COIL1, are considered. 4.2 Baseline methods We compare the MMDA approach with some state-of-the-art methods, namely, GFK [21], JDA [8], TJM [31], SCA [40], JGSA [5], DMM [32], unsupervised transfer metric learning (UTML) [60], locality preserving joint transfer (LPJT) [45], and domain invariant and class discriminative feature learning (DICD) [59]. Three deep approaches, namely, AlexNet [61], deep domain confusion (DDC) [62], and deep adaptation network (DAN) [47], are also tested for comparison. We exploit classification accuracy on as the evaluation criterion, which is commonly used in many approaches [21, 24, 36], i.e. (28)where and are the truth and predicted labels in the target domain, respectively. All baseline methods are evaluated by searching the optimal parameter settings from parameter spaces and the best results are reported. The number of subspace k is searched from for GFK, while parameter is searched from for JGSA. The best results are summarised in Tables 2–4. Figs. 4 and 5 show the sensitivity of different parameters. Fig. 4Open in figure viewerPowerPoint Parameter sensitivity study of MMDA on different types of datasets (a) Classification accuracy w.r.t. parameter p, (b) Classification accuracy w.r.t. parameter , (c) Convergence analysis of iteration Fig. 5Open in figure viewerPowerPoint Parameter sensitivity. The results suggest that should be set a small value (), v is optimal in the range of , and the best k depend on different tasks (a) Parameter , (b) Parameter k, (c) Parameter v Table 2. Accuracy(%) on MNIST+USPS, ImageNet+VOC2007, and COIL20datasets Tasks GFK JDA TJM SCA JGSA DICD UTML LPJT MMDA(P) MU 61.2 67.3 63.3 65.1 80.4 77.8 76.1 73.3 UM 46.5 59.7 52.3 48.0 68.2 65.2 59.8 63.6 IV 59.5 63.4 63.7 60.1 52.3 55.9 60.4 63.5 VI 73.8 70.2 73.0 66.7 70.6 69.3 71.0 74.2 COIL1COIL2 72.5 89.3 91.5 88.6 91.3 91.9 92.9 94.1 COIL2COIL1 74.2 88.5 91.8 89.2 91.3 93.3 90.8 94.2 Average 64.6 73.1 72.6 69.6 75.7 76.2 75.0 77.0 Table 3. Accuracy(%) on Office+Caltech10 datasets using speeded up robust features speeded up Robust features (SURF) features Tasks GFK JDA TJM SCA JGSA UTML DICD LPJT MMDA(P) MMDA(L) MMDA(R) CA 46.0 45.6 46.8 45.6 51.5 51.7 47.3 57.4 58.2 57.3 CW 37.0 41.7 39.0 40.0 45.4 51.2 46.4 51.6 52.6 53.3 CD 40.8 45.2 44.6 47.1 45.9 54.8 49.7 46.5 51.0 49.7 AC 40.7 39.4 39.5 39.7 41.5 44.4 42.4 44.5 43.8 43.4 AW 37.0 38.0 42.0 34.9 45.8 47.8 45.1 44.4 49.2 49.2 AD 40.1 39.5 45.2 39.5 47.1 42.0 38.9 48.5 51.0 48.5 WC 24.8 31.2 30.2 31.1 33.2 33.6 34.2 35.3 34.2 34.7 WA 27.6 32.8 30.0 30.0 39.9 41.0 34.1 40.4 40.0 41.2 WD 85.4 89.2 89.2 87.3 90.5 91.1 89.8 88.5 91.1 91.8 DC 29.3 31.5 31.4 30.7 29.9 34.6 34.3 31.8 33.4 33.5 DA 28.7 33.1 32.8 31.6 38.0 36.7 34.5 42.6 41.4 40.4 DW 80.3 89.5 85.4 84.4 90.5 91.2 86.8 91.6 91.6 91.0 Average 43.1 46.4 46.3 45.2 50.0 52.0 49.0 52.5 52.8 53.3 Table 4. Accuracy(%) on Office+Caltech10 datasets using DeCaf6 features Tasks Non-deep methods Deep learning methods Our methods GFK JDA TJM SCA DMM JGSA DICD LPJT AlexNet DDC DAN MMDA(L) MMDA(P) MMDA(R) CA 88.2 90.2 88.8 89.5 92.4 91.4 91.0 92.7 91.9 91.9 92.0 92.6 92.5 CW 77.6 85.4 81.4 85.4 87.5 86.8 87.8 83.7 85.4 90.6 90.2 87.8 87.5 CD 86.6 86.0 84.7 87.9 90.4 93.6 93.6 89.8 87.1 88.8 89.3 94.3 94.3 AC 79.2 81.9 84.3 78.8 84.8 84.9 86.0 86.6 83.0 85.0 84.1 87.8 88.2 AW 70.9 80.7 71.9 75.9 84.7 81.0 81.4 87.5 79.5 86.1 91.8 88.1 87.8 AD 82.2 81.5 76.4 85.4 92.4 88.5 83.4 88.5 87.4 89.0 91.7 91.08 89.2 WC 69.8 81.2 83.0 74.9 81.7 85.0 84.0 82.6 73.0 78.0 81.2 87.3 86.2 WA 76.8 90.7 87.6 85.0 86.5 90.7 89.7 90.7 83.8 84.9 92.1 91.8 92.1 WD 98.7 99.4 99.4 99.4 99.4 DC 71.4 80.3 83.8 78.1 83.3 86.2 86.1 80.0 79.0 81.1 80.3 85.8 84.9 DA 76.3 92.0 90.3 90.0 90.7 92.0 92.2 90.9 87.1 89.5 90.0 92.4 DW 99.3 99.3 99.3 98.6 99.3 99.7 99.0 97.7 98.2 98.5 95.3 99.0 95.3 Average 81.5 87.4 86.0 85.9 89.4 90.0 89.9 89.7 86.1 88.2 90.1 91.7 90.9 Our method involves eight parameters: subspace dimension k, number of nearest neighbours p, regularisation parameters , , v, , and . The number of iterations of all tasks is set to . Furthermore, we denote MMDA (P), MMDA (L), and MMDA (R) as MMDA with primal kernel, linear kernel, and Gauss kernel, respectively. 4.3 Results and analysis In this section, we comprehensively eva