W asserstein Selective Transfer Learning for Cross-domain Text Mining (2021)

Abstract: Transfer learning (TL) seeks to improve the learning of a data-scarce target domain by using information from source domains. However, the source and target domains usually have different data distributions, which may lead to negative transfer. To alleviate this issue, we propose a Wasserstein Selective Transfer Learning (WSTL) method. Specifically, the proposed method considers a reinforced selector to select helpful data for transfer learning. We further use a Wasserstein-based discriminator to maximize the empirical distance between the selected source data and target data. The TL module is then trained to minimize the estimated Wasserstein distance in an adversarial manner and provides domain invariant features for the reinforced selector. We adopt an evaluation metric based on the performance of the TL module as delayed reward and a Wasserstein-based metric as immediate rewards to guide the reinforced selector learning. Compared with the competing TL approaches, the proposed method selects data samples that are closer to the target domain. It also provides better state features and reward signals that lead to better performance with faster convergence. Extensive experiments on three real-world text mining tasks demonstrate the effectiveness of the proposed method.

## introduction transfer learning (tl) is a type of classical machine learning methods to leverage information from data-rich source domains to help a data-scarce target domain @xcite . recently, transfer learning based on deep neural networks, referred to as deep transfer learning @xcite , has been widely used on various tasks in natural language processing @xcite @xcite and computer vision @xcite @xcite . despite its success in various applications, vanilla deep transfer learning approaches may suffer from the negative transfer problem @xcite @xcite @xcite , as the source and target domains usually have different data distributions. one solution to solve this problem is instance-based transfer learning, which properly selects instances from the source domain to alleviate or avoid the negative transfer. the recent instance-based transfer learning methods incorporate reinforcement learning (rl) for data selection @xcite @xcite . these methods jointly train an rl based data selector and the transfer learning module, which is demonstrated to be better than previous methods. however, the aforementioned methods suffer from the following two challenges. first, the transfer learning module used in the previous reinforced instance-based tl methods is a simple fully-shared model, which may not be able to learn clean domain-invariant feature representations that are discriminative in prediction @xcite . this leads to sub-optimal transfer results and the transfer learning module cannot further provide good state representations for the rl module. second, the environment built by the transfer learning module provides sparse "delayed" rewards and with high variance during the training stage, which makes the rl policy difficult to optimize and cannot provide reasonable signals to guide the data selection process. the rl policy is performed at the instance level, but the delayed reward is at the batch level. thus, this is a challenging sparse reward problem, i.e., to update batched sequential decisions with a single delayed reward. in light of these challenges, we propose a wasserstein distance-based selective transfer learning (wstl) method to select helpful data from the source domain to help the target. our method is built on top of the reinforced transfer learning framework, but differently, we introduce a wasserstein-distance based discriminator. the advantages of the discriminator are two-fold. first, the discriminator is trained to distinguish features between the source and target, and further guides the transfer learning module to minimize the estimated wasserstein distance in an adversarial manner. hence, features learned in the transfer learning module are domain-invariant (i.e., more transferrable) and also discriminative in prediction (i.e., yields better transfer learning results). second, the discriminator also provides wasserstein distancebased metric to serve as an immediate reward signal to help guide rl policy, which can solve the sparse reward problem. in this way, the proposed method can select high-quality source data to help the target in an efficient and effective manner. to demonstrate the benefits of the proposed wstl method, we evaluate it on three real-world text mining tasks including paraphrase identification, natural language inference, and review text analysis. experimental results on all these datasets show that the proposed method outperforms the state-of-the-art methods by a large margin. empirical studies also confirm that the proposed method can select source domain data that are close to the target domain, thus indeed helps to reduce domain discrepancy and alleviate negative transfer. we summarize the contributions as follows. ## proposed method we formulate the problem in a standard transfer learning setting. given a source domain d @xmath0, y e i } ne @xmath1, y t i } nt @xmath2, our model aims to improve the performance in d t using the rich knowledge in d e which is usually much larger than d t . ## conclusions to alleviate the negative transfer issues, we propose a wasserstein selective transfer learning (wstl) method that builds a wasserstein discriminator to maximize the empirical distance between the selected source and target domain data. the tl module is trained to minimize the estimated wasserstein distance in an adversarial manner, and the discriminator provides immediate rewards further coupled with the delayed rewards from the tl module to guide the reinforced data selector. extensive experiments on three real-world datasets show the proposed method significantly outperforms the competing methods.