D omain I nv: Domain Invariant Fine Tuning and Adversarial Label Correction For Unsupervised QA Domain Adaptation (2024)

Abstract: Existing Question Answering (QA) systems are limited in their ability to answer questions from unseen domains or any out-of-domain distributions, making them less reliable for deployment in real scenarios. Importantly, all existing QA domain adaptation methods are either based on generating synthetic data or pseudo-labeling the target domain data. Domain adaptation methods relying on synthetic data and pseudo-labeling suffer from either the need for extensive computational resources or an additional overhead of carefully selecting the confidence threshold to distinguish noisy examples from the training dataset. In this paper, we propose unsupervised domain adaptation for an unlabeled target domain by transferring the target representation close to the source domain without using supervision from the target domain. To achieve this, we introduce the idea of domain-invariant fine-tuning along with adversarial label correction (DomainInv) to identify target instances that are distant from the source domain. This involves learning the domain invariant feature encoder to minimize the distance between such target instances and source instances class-wisely. This eliminates the possibility of learning features of the target domain that are still close to the source support but are ambiguous. The evaluation of our QA domain adaptation method, namely DomainInv, on multiple target QA datasets reveals a performance improvement over the strongest baseline.

## introduction over the past few years, machine learning models have been widely deployed in production. however, making them work satisfactorily in production requires a substantial amount of high-quality annotated data, which is expensive and time-consuming. therefore, it is of utmost importance to build generalizable models that can perform well on unseen datasets. however, due to the mechanism of domain shift or bias in the training dataset @xcite , it is challenging to directly transfer knowledge from the model trained on the source domain to the unlabeled target domain. in this paper, we studied this phenomenon specifically for the case of extractive question answering (qa) systems. extractive qa systems perform the task of identifying the most relevant answer for a given question within a context or paragraph. the answer is represented as a sub-span of the context, with start and end positions predicted by the qa model. the training data for qa essentially consists of triplets specifying the question, answer, and context. the input to the model is a question and context presented as running text separated by a separator. the model is trained to predict the most relevant start and end positions in the context @xcite @xcite . these qa systems also face performance degradation at test time, as questions and contexts can vary widely in complexity. the same question may be phrased in the simplest or most complex ways, and the answer may involve reasoning or follow a complex extraction pattern that is challenging to generalize with a limited annotated training dataset. recent works @xcite @xcite have explored this issue, proposing solutions such as using labeled target domain data or incorporating feedback during training @xcite @xcite . others @xcite have employed synthetic or pseudo-labeled data to train these systems and enhance their generalization to out-of-domain distributions. however, it is important to note that pseudo-labeled data is prone to noise, and obtaining accurately labeled data requires considerable human labeling effort. in this paper, we focus on unsupervised domain adaptation (uda), which does not require labeled target domain data. there are numerous works towards achieving domain-invariant representations, categorized into 1) optimizing the discrepancy between domain representations @xcite , and 2) adversarial learning @xcite . however, in general there exists different distance metrics for minimizing domain discrepancy, for example @xcite leverage maximum mean discrepancy (mmd) as the distance measure between source and target domain distributions. similar to mmd, cmd (central moment discrepancy) @xcite , wasserstein distance (wd) @xcite , sliced wasserstein distance (swd) @xcite , multi-kernel mmd @xcite , joint mmd @xcite are other alternative measures. inspired by generative adversarial networks (gan) @xcite , adaptation methods based on adversarial learning have also shown promising results @xcite @xcite @xcite . adversarial learning methods propose the idea of using the domain discriminator to distinguish whether the incoming sample is from the source or target domain, while the feature generator tries to fool the discriminator by generating domain-invariant features. during the process of creating domain-invariant representations, the generator positions the target representation near the source domain decision boundaries. however, these representations are misaligned with respect to the source classes, leading to a degradation in performance @xcite . some works rely on high-confidence pseudolabels @xcite for the target domain. however, this method of generating synthetic data for the target domain imposes an additional computational overhead. moreover, target pseudo-labeling can have adverse effects on adaptation if it generates too many incorrect labels above the confidence threshold. some works propose minimizing the distance between tokens from the target instances and those of the source support contrastively @xcite . however, in practical scenarios, the target domain can be completely asymmetrical, necessitating the alignment of the pre-trained source model with the target domain before optimizing for domain-invariant representations. in this paper, we propose an adaptation framework called domaininv (illustrated in figure 1 ), which can perform domain adaptation without training an answer classifier with noisy pseudo-labeled data. this eliminates the need to filter out that noise before training on the target domain, as opposed to the existing sota method in @xcite . our approach involves learning domaininvariant features through domain-invariant finetuning along with adversarial label correction. this is done to identify target instances that are far apart from the source domain and optimize them to lie near the source support, class wisely. main contributions of this paper are as follows: ## related work in the past few years, there has been an increasing interest in learning generalized representations through various learning paradigms, namely, unsupervised, multi-tasking, and transfer learning @xcite @xcite @xcite . specifically, recent studies have explored the generalization capability of reading comprehension systems @xcite @xcite @xcite . our interest in this paper lies solely in unsupervised approaches for domain adaptation, where target domain data is unlabelled. the approaches used for unsupervised pretrained qa encoder pretrained qa encoder answer classifier c 1 answer classifier c 2 target question and context source question and context qa encoder source question and context answer classifier c 1 a) pre-training qa model b) domain invariant fine tuning c) adversarial label correction fine tuned qa encoder target question and context answer classifier c 1 answer classifier c 2 domain tranformation layer weights shared loss swd source start & end positions loss ce inference training back propagation source start & end positions loss ce frozen training figure 1: domaininv: a robust framework for qa domain adaptation. it proposed to utilize domain invariant fine-tuning followed by adversarial label correction to overcome the limitations associated with domain invariant fine-tuning, demonstrating the noise free domain adaptation. contrastive learning: contrastive learning methods @xcite @xcite @xcite aim to learn a feature encoder that generates similar features for the same input (obtained from different augmentations) and different features for any other input and its augmentations. specifically for qa, @xcite have generated synthetic qa samples through question generation (qg). leveraging these samples improves performance in out-of-domain distribution @xcite @xcite @xcite @xcite . additionally, contrastive learning has been applied to minimize the discrepancy between the source and target domains using maximum mean discrepancy (mmd) @xcite . they learned to minimize the distance for averaged token features @xcite among answer and non-answer tokens in source and target domains and maximize the distance between them. self-supervision: there are many works in computer vision that have explored the use of selfsupervision for unsupervised domain adaptation, all aligned with the common objective of minimizing the discrepancy (distance) between domains @xcite @xcite . although the objective is similar to that of contrastive learning, models learned through contrastive learning have been shown to perform better @xcite . apart from the mmd @xcite criterion used in @xcite , other metrics like central moment discrepancy (cmd) used in @xcite directly match order-wise differences of central moments. wasserstein distance, employed to measure the distance between two probability distributions, has been explored in @xcite . the method in @xcite learns sentence representations for text matching between asymmetrical domains. in our approach, we consider the use of sliced wasserstein distance @xcite . instead of minimizing the distance between representations for domains, this distance is applied to minimize the distribution learned for the start and end tokens in qa domain adaptation. ## setup problem setup for unsupervised domain adaptation(uda) consider the labeled source domain d s and unlabelled target domain d t . the goal is to maximize the performance on target domain by only training with labeled source domain data and unlabelled target domain data as in @xcite @xcite . data: specifically, for the case of qa domain adaptation we describe the labeled source domain d s data as samples consisting of triplets, ## conclusion in this paper, we proposed a novel qa domain adaptation framework called domaininv. it is an unsupervised algorithm that does not require the use of labeled target domain data, nor does it depend on synthetic data or pseudo-labeled target domain. domaininv comprises two key components: 1) domain invariant fine tuning, which fine-tunes the qa model using the target style on the source domain, and 2) adversarial label correction, which identifies target distributions that are far apart from the source domain and optimizes the feature generator to bring them closer to the source support class wisely. evaluation of domaininv showed that it outperforms all baselines, achieving superior performance and establishing a new benchmark.