Conditional Augmentation for Aspect Term Extraction via Masked Sequence-to-Sequence Generation (2020)

Abstract: Aspect term extraction aims to extract aspect terms from review texts as opinion targets for sentiment analysis. One of the big challenges with this task is the lack of sufficient annotated data. While data augmentation is potentially an effective technique to address the above issue, it is uncontrollable as it may change aspect words and aspect labels unexpectedly. In this paper, we formulate the data augmentation as a conditional generation task: generating a new sentence while preserving the original opinion targets and labels. We propose a masked sequence-to-sequence method for conditional augmentation of aspect term extraction. Unlike existing augmentation approaches, ours is controllable and allows to generate more diversified sentences. Experimental results confirm that our method alleviates the data scarcity problem significantly. It also effectively boosts the performances of several current models for aspect term extraction.

## introduction aspect term extraction (ate), which aims to identify and extract the aspects on which users express their sentiments @xcite , is a fundamental task in aspect-level sentiment analysis. for example, in the sentence of "the screen is very large and crystal clear with amazing colors and resolution", "screen", "colors" and "resolution" are the aspect terms to extract in this task. ate is typically formulated as a sequence labeling problem @xcite @xcite , where each word is appended with a label indicating if it identifies an aspect. sentence and label sequence are both used to train a ate model. one of the remaining challenges with this task is the shortage of annotated data. while data augmentation appears to be a solution to this problem, it * corresponding author. faces two main obstacles here. first, the new sentences must adhere to their original label sequences strictly. as shown in figure 1 , the generation a is an effective augmentation as the original label sequence is preserved, whereas b is not even though it can be a valid review. second, a noun phrase is regarded as aspect term only if it is an opinion target. in the generation d of figure 1 , although the term "screen" remains where it is in the original sentence, the new context makes it just an ordinary mention rather than an opinion target. to sum up, the real difficulty of data augmentation in ate is generating a new sentence while aligning with the original label sequence and making the original aspect term remain an opinion target. existing augmentation models such as gan @xcite and vae @xcite tend to change the opinion target unpredictably and thus are not applicable for this task. another genre of augmentation strategy is based on word replacement. it generates a new sentence by replacing one or multiple words with their synonyms @xcite or with words predicted by a language model @xcite . this approach seems to be able to address the above issue in ate augmentation, yet it only brings very lim-ited changes to the original sentences and cannot produce diversified sentences. intuitively, augmentation strategies are effective when they increase the diversity of training data seen by a model. we argue in this paper that the augmentation for aspect term extraction calls for a conditional approach, which is to be formulated as a masked sequence-to-sequence generation task. specifically, we first mask several consecutive tokens for an input sentence. then, our encoder takes the partially masked sentence and its label sequence as input, and our decoder tries to reconstruct the masked fragment based on the encoded context and label information. the process of reconstruction keeps the opinion target unchanged and is therefore controllable. moreover, compared with replacementbased approaches @xcite which replace words separately, ours replaces a segment each time and has the potential to generate more diversified new sentences in content. to implement the above conditional augmentation strategy, we adopt transformer @xcite as our basic architecture and train it like mass @xcite , a pre-trained model for masked sequence-to-sequence generation. the contributions of this work are as follows. ## conditional augmentation for ate as mentioned before, we formulate the data augmentation of aspect term extraction (ate) as a conditional generation task. in this section, we first introduce the problem formulation, and then describe our augmentation method in detail. ## experiment in this section, we first introduce the experimental datasets and several popular ate models. then, we report the experimental results, which are obtained by averaging five runs with different initializations. ## discussion in this section, we present more qualitative analysis and discussions about our augmentation approach. ## conclusion in this paper, we have presented a conditional data augmentation approach for aspect term extraction. we formulated it as a conditional generation problem and proposed a masked sequence-to-sequence generation model to implement it. unlike existing augmentation approaches, ours is controllable to generate qualified sentences, and allows more diversified new sentences. experimental results on two review datasets confirm its effectiveness in this conditional augmentation scenario. we also conducted qualitative studies to analyze how this augmentation approach works, and tested other language models to explain why our masked sequence-tosequence generation framework is favored. moreover, the proposed augmentation method tends not to be unique to the current task and could be applied to other low-resource sequence labeling tasks such as chunking and named entity recognition.