Unified Feature and Instance Based Domain Adaptation for Aspect-Based Sentiment Analysis (2020)

Abstract: The supervised models for aspect-based sentiment analysis (ABSA) rely heavily on labeled data. However, fine-grained labeled data are scarce for the ABSA task. To alleviate the dependence on labeled data, prior works mainly focused on feature-based adaptation, which used the domain-shared knowledge to construct auxiliary tasks or domain adversarial learning to bridge the gap between domains, while ignored the attribute of instance-based adaptation. To resolve this limitation, we propose an end-to-end framework to jointly perform feature and instance based adaptation for the ABSA task in this paper. Based on BERT, we learn domain-invariant feature representations by using part-of-speech features and syntactic dependency relations to construct auxiliary tasks, and jointly perform word-level instance weighting in the framework of sequence labeling. Experiment results on four benchmarks show that the proposed method can achieve significant improvements in comparison with the state-of-the-arts in both tasks of cross-domain End2End ABSA and cross-domain aspect extraction.

## introduction aspect extraction and aspect sentiment classification are two important sub-tasks in aspect based sentiment analysis (absa) @xcite , which aim to extract aspect terms and predict the sentiment polarities of the given aspect terms, respectively. since these two sub-tasks have been well studied in the literature, a number of recent studies focus on the end2end absa task by employing a unified tagging scheme to tackle the two sub-tasks in an end-to-end manner @xcite @xcite . the unified tagging scheme fuses aspect boundary tags {b, i, o} and sentiment polarities {pos, neg, neu} together, and formulates end2end absa as a sequence labeling problem. for example, given a sentence "the price is reasonable, although the service is poor.", the end2end absa task aims to jointly extract aspect terms and detect sentiment polarities over them. the extracted pairs in this example are {"price": positive; "service": negative}. however, these existing studies heavily rely on supervised learning over a large amount of labeled data, which is usually hard to obtain for absa due to the intensive nature of human annotation. therefore, it will be very attractive to explore the end2end absa task in a cross-domain setting, which allows us to train a robust absa model for a resource-poor target domain based on enough annotated data in a resource-rich source domain. traditional domain adaptation methods primarily focus on coarse-grained sentiment classification @xcite @xcite @xcite @xcite . most of these methods can be grouped into two categories: feature-based domain adaptation and instance-based domain adaptation. featurebased methods focus on finding a new feature representation which could reduce domain discrepancy. instance-based methods aim to re-weight training samples in source domain which essentially attempts to assign higher weights to instances similar to the target domain, and lower weights to instances different from the target domain. in contrast, due to the difficulty in fine-grained domain adaptation, only a few approaches have been proposed for cross-domain absa. most of them explored cross-domain absa from the feature-based adaptation perspective, aiming to induce domain-invariant representations for each word. specifically, @xcite and @xcite proposed to use domain-shared syntactic knowledge to construct auxiliary tasks to reduce domain disparity. more recently, @xcite used the memory network to model the syntactic relations between words and designed a selective adversarial learning strategy to achieve word-level adaptation. however, all these methods are still based on traditional neural network architectures. as we all know, with the recent trend of pre-training in nlp @xcite @xcite , many pre-trained text encoders such as bert have demonstrated their strong capability for domain-invariant representation learning, which poses new challenges for domain adaptation. based on our preliminary experiments, we find that simply using bert without domain adaptation has already obtained indistinguishable performance compared with previous domain adaptation methods. therefore, it will be more attractive to extend these feature-based adaptation approaches to pre-trained models and further improve the domain adaptation performance. apart from the feature-based domain adaptation methods, @xcite pointed out the importance of performing instance-based adaptation for different nlp tasks. as revealed by the theoretical analysis in @xcite , the domain discrepancy mainly comes from feature mismatches and instance mismatches, and needs to be jointly modeled from two attributes. however, previous studies only demonstrated the importance of instance-based domain adaptation in coarse-grained sentiment classification @xcite , and it is still unclear how to perform instance adaptation for the absa task. to address the two challenges mentioned above, we first utilize bert to learn domain-invariant features for the absa task, followed by proposing an instance weighting method for cross-domain absa. finally, we integrate them into an end-toend framework to jointly perform feature and instance adaptation. specifically, for feature-based adaptation, we use the domain-shared part-ofspeech information and dependency relations as self-supervised signals to enhance bert to learn domain-invariant representation for cross-domain absa. for instance-based adaptation, since absa is typically modeled as a word-level prediction task, we propose to leverage a domain classier to dynamically learn an importance weight for each word and re-weight different words from the source domain during supervised training. finally, we propose a unified framework to jointly perform feature and instance-based adaptation via sequential learning and joint learning, respectively. experimental results on four benchmark datasets show that our method can significantly improve the performance of cross-domain end2end absa and cross-domain aspect extraction, and we further carry out ablation studies to quantitatively measure the effectiveness of each component in our unified framework. the main contributions of this paper can be summarized as follows: ## related work aspect extraction and aspect-level sentiment classification are two important subtasks in aspect-based sentiment analysis (absa), which aim to extract aspect terms and identify the sentiment orientations towards them, respectively @xcite . as two fundamental tasks, aspect extraction (qiu @xmath0, bearing, hated, beauty, ##mi, amazement, ##ant, canned, mistake, madden, accused, nicely, employee, proud, difficulty, impressive, likely, catalogue, ##working @xmath1, lesson, strongly, reality, comfortably, artwork, food, loving, dissatisfaction, spice, ##kind, fork, appears, weary, desk, projects, monster, covering, recipients, purchases @xmath2, desk, robust, lightly, capable, waking, satisfactory, birthday, releasing, kitchen, noises, appearing, experiences, sophisticated, extreme, providing, nuts, interaction, recommendations since these two tasks are strongly related with each other, a number of previous studies propose to tackle them together in an end-to-end manner @xcite . some recent studies have further demonstrated that a unified tagging scheme can effectively eliminate the error propagation issue of traditional pipeline methods, and thus achieve the state-of-the-art performance. however, since annotating each word with fine-grained label is time-consuming, it is next to impossible to obtain enough annotated data for the absa task in every new domain. therefore, in this work, we resort to transfer learning, and focus on proposing an effective domain adaptation approach for the absa task. existing domain adaptation studies in sentiment analysis primarily focus on coarse-grained domain adaptation problem. most of them can be grouped into two categories: feature-based methods @xcite @xcite @xcite @xcite and instance-based methods @xcite @xcite . the former one attempts to learn a domain-invariant representation with auxiliary tasks or domain adversarial learning, while the latter one tries to re-weight source instances in order to assign higher weights to instances similar to the target domain and lower weights to instances different from the target domain. due to the challenges in fine-grained domain adaptation, only a few studies have explored the absa task in cross-domain settings. @xcite and @xcite used domain general syntactic relations to construct auxiliary task to bridge the domains. @xcite proposed a selective adversarial learning method to learn domain-invariant representations for aspect words. however, these methods are still based on traditional networks such as lstm, but fail to resort to recent pre-trained text encoders such as bert. moreover, all these methods only perform featurebased adaptation, but ignore instance-based adaptation. in contrast, our work aims to propose a unified feature and instance-based method based on bert for cross-domain absa. besides, it is worth noting that @xcite explored bert for cross-domain aspect sentiment classification, where the aspect terms or categories are provided for both source and target domains. different from their work, we primarily focus on the cross-domain end2end absa task in this work, which aims to first extract aspect terms followed by identifying the sentiment towards each detected aspect term. ## conclusion in this paper, we explored the potential of bert to domain adaptation, and proposed a unified feature and instance-based adaptation approach for both tasks of cross-domain end2end absa and cross-domain aspect extraction. in feature-based domain adaptation, we use domain-shared syntactic relations and pos tags to construct auxiliary tasks, which can help learn domain-invariant representations for domain adaptation. in instance-based domain adaptation, we employ a domain classifier to learn to assign appropriate weights for each word. extensive experiments on four benchmark datasets demonstrate the superiority of our unified domain adaptation (uda) approach over existing methods in both cross-domain end2end absa and cross-domain aspect extraction.