Unsupervised Data Augmentation for Aspect Based Sentiment Analysis (2022)

Abstract: Recent approaches to Aspect-based Sentiment Analysis (ABSA) take a co-extraction approach to this span-level classification task, performing the subtasks of aspect term extraction (ATE) and aspect sentiment classification (ASC) simultaneously. In this work, we build on recent progress in applying pre-training to this co-extraction task with the introduction of an adaptation of Unsupervised Data Augmentation in semi-supervised learning. As originally implemented, UDA cannot accommodate span-level classification since it relies on advanced data augmentation techniques, such as back-translation, that alter the sequence lengths of the original data and cause index mismatches. We introduce an adaptation of UDA using Masked Language Model (MLM) unmasking that accommodates this index-match constraint and test the approach on standard ABSA benchmark datasets. We show that simple augmentations applied to modest-sized datasets along with consistency training lead to competitive performance with the current ABSA state-of-the-art in the restaurant and laptop domains using only 75% of the training data.

## introduction aspect-based sentiment analysis (absa) is a subset of sentiment analysis (sa), operating at the phrase-rather then sentence-or documentlevel. as with other forms of sa, the goal is to determine the sentiment associated with a given text segment, though, in the case of absa, these phrasal segments are typically "aspects" or features associated with products, services, or experiences, such as "waitstaff" or "ambience." as with other span-level classification tasks, such as named entity recognition (ner), a major challenge of absa is class imbalance, as the majority of token labels typically refer to non-aspect terms @xcite and the terms themselves are of inconsistent phraselevel categories. this introduces considerable variance in aspect term labels and makes it difficult for models to effectively generalize to example terms outside those explicitly shown in the training data. ## discussion in our experiments, we found that, not surprisingly, downsampling the training data results in a degradation of model performance, but we also found that uda achieves sotacompetitive performance with 75% of the data. the original uda paper @xcite showed good performance with only 20 training examples for sentiment classification. in our dataset, we achieved performance competitive with bert-based-uncased @xcite with only 25% of the training data. in addition to benchmarks and ablation experiments, we explored the role of different types of confidence thresholding (ct) on the final performance of our models. the confidence threshold, φ, filters out unsupervised examples that fall below φ during uda, so that the model does not reinforce its own errors, and φ is typically increased during training on a schedule @xcite , namely, linear, log, and exp. we found that ct during uda is important to the stability and convergence of our models. figure 2 shows the training curves for test data micro-f1 with and without ct. our models converge faster with ct, and interestingly, the curve for our models without ct shows a kink after ∼1 epoch, where micro-f1 appears to saturate only to rebound as training continues. we hypothesize that this kink is the result of the model initially learning the wrong features, which are subsequently relearned during further training. however, the final performance of our models without ct never reaches those with ct see ## summary we introduced a span-level modification of the uda procedure, which, along with posttraining on bert, achieves performance competitive with state-of-the-art on the restaurant and laptop domains for absa with 75% of the data. while post-training contributed the most to overall performance, uda may be more efficient on a per data/compute basis. we observed that confidence thresholding is essential to stabilize model training and achieve greater performance, and that linear confidence threshold scheduling achieved the best performance along with single augmentations compared to multiple augmentations. this work reveals the benefit of using uda for span-level tasks and with post-trained language models.