Progressive Self-Training with Discriminator for Aspect Term Extraction (2021)

Abstract: Aspect term extraction aims to extract aspect terms from a review sentence that users have expressed opinions on. One of the remaining challenges for aspect term extraction resides in the lack of sufficient annotated data. While self-training is potentially an effective method to address this issue, the pseudo-labels it yields on unlabeled data could induce noise. In this paper, we use two means to alleviate the noise in the pseudo-labels. One is that inspired by the curriculum learning, we refine the conventional self-training to progressive self-training. Specifically, the base model infers pseudo-labels on a progressive subset at each iteration, where samples in the subset become harder and more numerous as the iteration proceeds. The other is that we use a discriminator to filter the noisy pseudo-labels. Experimental results on four SemEval datasets show that our model significantly outperforms the previous baselines and achieves state-of-the-art performance.

## introduction aspect term extraction (ate) is a crucial task in aspect-level sentiment analysis, aiming to extract all aspect terms present in the sentence @xcite . for example, given a restaurant review "i looove their eggplant pizza, as well as their pastas!", ate system aims to extract "eggplant pizza" and "pastas". many researchers typically formulated ate as a sequence labeling problem or a token-level classification problem. the current state-of-the-art neural models can be classified into two categories. one designs the sophisticated model with a variety of techniques, such as history attention @xcite , sequence to sequence @xcite , and constituency lattice @xcite . although these models achieve satisfactory performance, its sufficient condition is the availability of sufficient * corresponding author. training data. however, labeling a large amount of aspect data may not be practical due to its cost. the other aims at addressing the data insufficiency issue from a different perspective. for example, @xcite generated the reviews while preserving the original aspects via formulating the data augmentation as a conditional generation task. however, varying only a small number of consecutive non-aspect words in the reviews would limit the semantic diversity of the sample. chen and qian (2020) tackled long-tail distributions problem of aspect terms and context words in the training sets with soft prototypes. nevertheless, the way that the soft template implicitly uses external data discounts the usefulness of external data. different from previous approaches, in this paper, we use self-training @xcite to alleviate the labeled data insufficiency. in self-training, a base model trained on the labeled data is used to infer pseudo-labels on the unlabeled data, and then a new base model is trained to optimize the loss on human labels and pseudo-labels jointly. we can iterate this algorithm a few times by using new base model to relabel the unlabeled data and retraining a new base. hence, we can supplement the labeled data with some pseudo-labeled data. however, the approaches relying on self-training typically suffer from noise induced by pseudolabels. in this paper, we use two means to mitigate the negative effects of pseudo-labels. one is to refine the conventional self-training to progressive self-training. here, we use a progressive subset at each iteration instead of the entire unlabeled data set. during the iterative process, the unlabeled samples in the subset become harder and more numerous. our motivation stems from curriculum learning @xcite , where we expect to infer pseudo-labels for unlabeled data in the order of easy to hard and few to many. in this process, easy unlabeled data will bring in little noise, and model that have been previously learned will be better and thus generate less noise at later stages. the other is to use a discriminator to filter out as much noise as possible from the pseudo-labels. inspired @xcite , we construct a question sequence based on sentence and a pseudo-label aspect term in sentence that is fed to the discriminator to make true-false prediction. to train the discriminator, we fabricate several positive and negative samples from the training set. here, the negative samples are constructed based on the left and right boundary errors of aspect terms and the errors of non-aspect terms with the same pos tag as aspect terms. a pseudo-labeled sentence is used to train the new base model only if its all aspect terms are true; otherwise, it is filtered out. moreover, we can also apply self-training method to train the discriminator to enhance its discriminatory power. our method follows multiple steps. overall, we make the following contributions: (a) to the best of our knowledge, we are the first to use self-training to address the problem of insufficient labeled data in ate; (b) to mitigate the noise introduced by self-training, we refine the general self-training to progressive self-training and bring in a discriminator to filter the noisy pseudo-labels; (c) experimental results on four ate datasets show that our method outperforms the baselines and achieves the state-of-the-art performance. furthermore, we conduct extensive experiments to verify its effectiveness and generalization. ## conclusion in this paper, we focus on the problem of insufficient labeled data in ate, and try to solve it via selftraining. to mitigate the noise in pseudo-labels, we make two efforts. (i) motivated by curriculum learning, we refine the conventional self-training to progressive self-training, expecting to reduce the generation of noisy pseudo-labels; (ii) we introduce a discriminator to filter the noisy pseudo-labels. experimental results show that our method beats the baselines and achieves sota performance. moreover, we verify its effectiveness and generalization through extensive experiments.