Generative Cross-Domain Data Augmentation for Aspect and Opinion Co-Extraction (2022)

Abstract: As a fundamental task in opinion mining, aspect and opinion co-extraction aims to identify the aspect terms and opinion terms in reviews. However, due to the lack of fine-grained annotated resources, it is hard to train a robust model for many domains. To alleviate this issue, unsupervised domain adaptation is proposed to transfer knowledge from a labeled source domain to an unlabeled target domain. In this paper, we propose a new Generative Cross-Domain Data Augmentation framework for unsupervised domain adaptation. The proposed framework is aimed to generate target-domain data with fine-grained annotation by exploiting the labeled data in the source domain. Specifically, we remove the domain-specific segments in a source-domain labeled sentence, and then use this as input to a pre-trained sequence-to-sequence model BART to simultaneously generate a target-domain sentence and predict the corresponding label for each word. Experimental results on three datasets demonstrate that our approach is more effective than previous domain adaptation methods.

## introduction aspect and opinion co-extraction is a fundamental task in opinion mining. it aims to extract aspect terms and opinion terms from review sentences. given a review "the keyboard is great", the aspect term is keyboard and the opinion term is great. most existing works formulate aspect and opinion co-extraction as a sequence labeling task. with the support of deep learning, many supervised methods have achieved remarkable results @xcite @xcite . however, owing to the high cost of fine-grained annotations, the scarcity of labeled data in many new domains makes them fail to obtain a robust performance. to alleviate the deficiency of labeled data, unsupervised domain adaptation is proposed to transfer knowledge from a labeled source domain to an unlabeled target domain. the main difficulty of unsupervised domain adaptation lies in the distribution discrepancy between data from different domains. specifically, reviews from different domains may have distinct aspect terms, opinion terms, and expression patterns. to tackle the distribution discrepancy problem, many domain adaptation methods were proposed for coarse-grained sentiment classification @xcite @xcite . however, due to the complexity of fine-grained domain adaptation for aspect and opinion extraction, only a handful of studies attempt to address this issue. most of them are based on the following three paradigms: however, the paradigms mentioned above suffer from the following problems. for rule-based adaptation methods, it is difficult to design high-quality manual rules and opinion set, which may bring low precision results. for feature-based adaptation methods, although they can bridge the domain gap by using the unlabeled data from both domains, the main task classifier is only trained by the source labeled data, which fails to exploit the important supervision signals in the target domain. although our recent work @xcite has shown the superiority of data augmentation-based adaptation methods, its main limitation lies in that it only replaces source-specific aspects and opinions in the source domain review with target-specific aspects and opinions, but ignoring other domain-specific attributes such as collocations and expression styles, which limits the quality and diversity of generated target-domain data. to this end, we propose a generative cross-domain data augmentation framework for unsupervised domain adaptation, which generates target labeled data from source labeled data based on a pre-trained sequence-to-sequence model, i.e., bart @xcite . specifically, given the labeled samples in the source domain, we first train a classifier to assign pseudo labels for each unlabeled sample in the target domain. with the labeled and pseudo labeled samples from both domains, we then remove their domain-specific features to obtain domain-independent samples. next, we employ the bart model by feeding the domain-independent samples and their token labels to the encoder and reconstructing their original texts and corresponding labels in the decoder. in the inference stage, given a source-domain sentence in which the domain-specific features are removed, the model can generate a target-domain sentence by integrating target-specific features into the source context while predicting its token-level labels. compared with the previous method @xcite , our new approach generates highquality target-domain data with more flexible syntactic patterns by making full use of the knowledge from the domain-invariant source contexts. with the label prediction, our approach can easily obtain the adaptive token labels for each generated target-domain sample. our main contributions in this work can be summarized as follows: ## problem formulation in this paper, we focus on the aspect and opinion co-extraction task and formulate it as a sequence labeling problem. we denote a review as a sequence of tokens @xmath0, x 2 , ..., x n ], and the task aims to predict the label sequence of the review @xmath1, y 2 , ..., y n ], where y @xmath2, i-asp, b-op, i-op, o}. for unsupervised domain adaptation, we are given a set of sourcedomain labeled reviews d @xmath3, y s i )} n s @xmath4 the goal is to predict token-level labels on the test set from the target domain d @xmath5. ## methodology in this paper, we propose a generative cross-domain data augmentation framework, which generates target-domain reviews with fine-grained annotation from source-domain labeled reviews. figure 1 illustrates the overall architecture of our generative framework. specifically, we leverage the labeled samples in the source domain to train a classifier, and then assign pseudo labels for each unlabeled sample in the target domain. given the labeled and pseudo labeled sentences from both domains, we employ a segment mask module to remove their domain-specific features while preserving their context and original labels. next, a pre-trained sequence-to-sequence model bart concatenates the domain-invariant context and original labels as the input of the encoder, followed by recovering the original sentences and predicting their token-level labels via the decoder. during the inference stage, given a labeled source-domain sentence, the model removes its domain-specific features, and then employs the fine-tuned bart model to generate a target-domain sentence with its token-level labels. ## conclusion in this paper, we proposed a generative cross-domain data augmentation framework for unsupervised domain adaptation, which leverages the labeled source-domain data to directly generate labeled target-domain data based on a fine-tuned sequence-to-sequence model bart. experiments on three benchmark datasets show that our generative approach generates high-quality target-domain data with fine-grained annotation and outperforms previous domain adaptation methods for aspect and opinion co-extraction.