Simultaneous Selection and Adaptation of Source Data via Four-Level Optimization (2024)

Abstract: In many NLP applications, to mitigate data deficiency in a target task, source data is collected to help with target model training. Existing transfer learning methods either select a subset of source examples that are close to the target domain or try to adapt all source examples into the target domain, then use selected or adapted source examples to train the target model. These methods either incur significant information loss or bear the risk that after adaptation, source examples which are originally already in the target domain may be outside the target domain. To address the limitations of these methods, we propose a four-level optimization based framework which simultaneously selects and adapts source data. Our method can automatically identify in-domain and out-of-domain source examples and apply example-specific processing methods: selection for in-domain examples and adaptation for out-of-domain examples. Experiments on various datasets demonstrate the effectiveness of our proposed method.

## introduction transfer learning (tl) @xcite , which aims at improving a model in a target domain by utilizing data from a source domain, has been broadly studied in natural language processing. one paradigm of tl methods @xcite @xcite @xcite focuses on selecting a subset of source examples that are close to the target domain and using selected examples as additional training data for the target model. another paradigm of tl methods @xcite @xcite focuses on adapting the entire set of source examples into the target domain and using adapted source examples as additional training data for the target model. the problem of selec-tion based methods is that unselected examples are discarded. though having a domain discrepancy with target data, unselected examples still contain useful information that can be leveraged to improve the target model. discarding them would lead to information loss. the problem of adaptation based methods is that some source examples may already be in the target domain; performing domain adaptation on these source examples is a waste of effort; and, even worse, after adaptation, these source examples may be outside the target domain. to address the limitations of both paradigms of methods, we propose a new approach which simultaneously performs selection and adaptation of source examples: our method automatically identifies which source examples are in the same domain as target data (referred to as in-domain source data) and which are not (referred to as out-of-domain source data); for in-domain source data, they are directly used to train the target model; for out-of-domain source data, they are first adapted, then utilized to train the target model. compared with previous methods, our approach has the following advantage: instead of using a single way to deal with all source examples (either performing selection or adaptation), our method applies example-specific ways to deal with different examples based on whether they are in-domain or out-of-domain. our method is based on four-level optimization, which performs the following four stages end-to-end. at the first stage, a domain distance network is trained based on self-supervised learning. at the second stage, the domain distance network is used to identify out-of-domain source examples and adapt them into the target domain. at the third stage, in-domain source examples selected by the domain distance network and adapted source examples are used to train a target model. at the fourth stage, the trained target model is evaluated on a validation set and data weights of the domain distance network are updated by minimizing the validation losses. experiments on a variety of datasets demonstrate the effectiveness of our proposed method. to summarize, the major contributions of this work is that we propose a four-level optimization based framework for simultaneous selection and adaptation of source examples in transfer learning and demonstrate the effectiveness of our proposed method on two nlp applications. data selection in transfer learning. many methods @xcite @xcite @xcite @xcite @xcite @xcite @xcite have been developed for selecting source data that is suitable for training target models, based on reinforcement learning @xcite @xcite , adversarial learning @xcite , curriculum learning @xcite , entropy @xcite , bayesian optimization @xcite , multi-task learning @xcite , and bi-level optimization (blo) @xcite @xcite @xcite . blo based approaches select data by minimizing a validation loss, where the lower-level optimization problem trains network weights on a training dataset and the upperlevel optimization problem learns data selection variables on a validation set. these methods select part of source data and discard the rest, which incurs information loss. transfer learning (tl). tl @xcite @xcite @xcite aims at training a better target model by utilizing source data. many tl methods have been developed, including those based on 1) distribution alignment @xcite @xcite , 2) regularization @xcite @xcite , 3) adversarial domain-invariant representation learning @xcite @xcite , and 4) latent space projection @xcite @xcite . these methods either select part of source data or adapt all source data, which leads to information loss, waste of efforts, and the risk of adapting in-domain source data outside the target domain. bi-level optimization (blo). blo finds broad applications in hyperparameter tuning @xcite , data selection @xcite @xcite , training data generation @xcite , neural architecture search @xcite , learning rate adaptation @xcite , meta learning @xcite , etc. in blo-based methods, model weights are learned by solving an inner optimization problem and meta parameters are learned by solving an outer optimization problem. the two optimization problems are nested. different from existing blo-based methods, our method is based on four-level optimization. ## methods in this section, we present the method for simultaneous selection and adaptation of source examples based on four-level optimization. we aim to train a target model for a specific target domain using dataset d t . in practical scenarios, the target domain often suffers from a lack of labeled training data. this lack can lead to overfitting on the training data and poor generalization on test data. to address this issue, we utilize a dataset from a source domain, d s , which has an abundance of labeled examples. however, there is a notable discrepancy between the source and target data. we categorize d s examples into two types: those that belong to the same domain as d t (in-domain source data) and those that do not (out-of-domain source data). we aim to select in-domain source data and adapt out-of-domain source data into the target domain, and use selected and adapted source data to train the target model. the overall framework is shown in figure 1 . the notations are shown in ## experiments in this section, we present experimental results on text classification and visual question answering on pathology images. following the common data assumption in transfer learning, the amount of labeled source data in our experiments is significantly larger than that of target data. every experiment runs 4 times with different random initializations. for all experiments, we performed significance tests using double-sided t-tests. the p-values of our methods against baselines are all less than 0.001, which shows that our methods are significantly better than baselines. ## conclusions and discussion we propose a framework for simultaneous selection and adaptation of source examples in transfer learning. our method automatically identifies which source examples are in or out of the target domain, and performs example-specific operations (either selection or adaptation). this is different from previous methods, which 1) discard out-of-domain source examples, leading to information loss; or 2) try to adapt all source examples into the target domain, incurring a risk of moving in-domain source examples outside the target domain. our framework is based on four-level optimization. experiments on text classification and visual question answering demonstrate the effectiveness of our proposed method.