A Self-enhancement Multitask Framework for Unsupervised Aspect Category Detection (2023)

Abstract: Our work addresses the problem of unsupervised Aspect Category Detection using a small set of seed words. Recent works have focused on learning embedding spaces for seed words and sentences to establish similarities between sentences and aspects. However, aspect representations are limited by the quality of initial seed words, and model performances are compromised by noise. To mitigate this limitation, we propose a simple framework that automatically enhances the quality of initial seed words and selects high-quality sentences for training instead of using the entire dataset. Our main concepts are to add a number of seed words to the initial set and to treat the task of noise resolution as a task of augmenting data for a low-resource task. In addition, we jointly train Aspect Category Detection with Aspect Term Extraction and Aspect Term Polarity to further enhance performance. This approach facilitates shared representation learning, allowing Aspect Category Detection to benefit from the additional guidance offered by other tasks. Extensive experiments demonstrate that our framework surpasses strong baselines on standard datasets.

## introduction aspect category detection (acd), aspect term extraction (ate), and aspect term polarity (atp) are three challenging sub-tasks of aspect based sentiment analysis, which aim to identify the aspect categories discussed (e.i., food), all aspect terms presented (e.i., 'fish', 'rolls'), and determine the polarity of each aspect term (e.i., 'positive', 'negative') in each sentence, respectively. to better understand these problems consider the example in unsupervised acd has mainly been tackled by clustering sentences and manually mapping these clusters to corresponding golden aspects using top representative words @xcite @xcite . however, this approach faces a major problem with the mapping process, requiring manual labeling and a strategy to resolve aspect label inconsistency within the same cluster. recent works have introduced using seed words to tackle this problem @xcite @xcite . these works direct their attention to learning the embedding space for sentences and seed words to establish similarities between sentences and aspects. as such, aspect representations are limited by a fixed small number of the initial seed words. @xcite is one of the few works that aims to expand the seed word set from the vocabulary of a pretrained model. however, there is overlap among the additional seed words across different aspects, resulting in reduced discriminability between the aspects. another challenge for the unsupervised acd task is the presence of noise, which comes from out-of-domain sentences and incorrect pseudo labels. this occurs because data is often collected from various sources, and the pseudo labels are usually generated using a greedy strategy. current methods handle noisy sentences by either treating them as having a general aspect @xcite or forcing them to have one of the desired aspects @xcite @xcite . to address incorrect pseudo labels, @xcite attempt to assign lower weights to uncertain pseudo-labels. however, these approaches might still hinder the performance of the model as models are learned from a large amount of noisy data. in this paper, we propose a self-enhancement multitask (asem) framework to address these limitations. firstly, to enhance the understanding of aspects and reduce reliance on the quality of initial seed words, we propose a seedword enhancement component (sec) to construct a high-quality set {while the fish @xcite is unquestionably fresh, rolls @xcite tend to be inexplicably bland. another contribution to our work concerns the recommendation of multi-tasking learning for unsupervised acd, ate and atp in a neural network. intuitively, employing multi-task learning enables acd to leverage the benefits of ate and atp. referring to the example in figure 1 , ate extracts opinion target expressions (otes): 'fish' and 'rolls', which requires understanding the emotion 'positive' (expressed as 'unquestionably fresh') for 'fish' and 'negative' (expressed as 'inexplica-bly bland') for 'rolls'. meanwhile, acd wants to detect the aspect category of the sentence, requiring awareness of the presence of "fish" and "rolls" (terms related to the aspect of "food") within the sentence. despite the usefulness of these relationships for acd, the majority of existing works do not utilize this information. @xcite is one of the few attempts to combine learning the representations of aspect and polarity at the sentence level before passing them through separate classifiers. our contributions are summarized as follows: ## related works topic models were once the dominant approach @xcite @xcite for unsupervised aspect category detection. however, they can produce incoherent aspects. recently, neural network-based methods have been developed to address this challenge. cluster mapping-based resolvers: these methods utilize neural networks to cluster effectively and manually map (many-to-many mapping) the clusters to their corresponding aspects. they employ attention-based autoencoders @xcite or contrast learning approach @xcite for clustering. @xcite further enhance performance by using knowledge distillation to learn labels generated after clustering. seed words-based resolvers: these approaches automate the aspect category mapping process by utilizing seed words that indicate aspect appearance. @xcite use the weighted sum of seed word representations as aspect representations, allowing mapping one-to-one in the auto-encoder model. recent works focus on learning embedding spaces for sentences and seed words, generating pseudo labels for weakly supervised learning. they use skip-gram @xcite for embedding space learning and convolutional neural networks or linear layers for classification @xcite . @xcite jointly learn acd with sentence-level atp, while @xcite consider the uncertainty of the initial embedding space. without any human supervision, @xcite rely solely on label names, similar to seed words. tulkens and van cranenburgh (2020) detect aspects using cosine similarity between pre-trained aspect and label name representations, while @xcite train the clustering model with instancelevel and concept-level constraints. our framework called asem (short for a selfenhancement mutitask framework), consists of three key components: (i) pseudo-label generation, (ii) retrieval-based data augmentation, and (iii) classification. figure 1 presents an overview of the framework. initially, we extract a small subset of the training data to serve as the task-specific in-domain data. based on the quality of the initial seed words in this dataset, we utilize sec to expand the set of seed words in order to enhance its quality. by feeding the task-specific in-domain data and enhanced seed words to the pseudo-label generation, we obtain high-quality pseudo labels for the task-specific in-domain data. then, we leverage the retrieval-based augmentation to enhance the number of training samples from the data bank (the remaining part of the training data), based on our prior knowledge of the target task (seed words, taskspecific in-domain data with high-quality pseudo labels). to this end, the high-quality pseudo labels and augmented data are passed through a multitask classifier to predict the task outputs. ## evaluation first, we report the results of our framework on the acd task in ## analysis in this section, we thoroughly analyze the performance of our framework concerning the quality of seed words and training data. ## conclusion in this work, we propose a novel framework for acd that achieves three main goals: (1) enhancing aspect understanding and reducing reliance on initial seed words, (2) effectively handling noise in the training data, and (3) self-boosting supervised signals through multitask-learning three unsupervised tasks (acd, ate, atp) to improve performance. the experimental results demonstrate that our model outperforms the baselines and achieves state-of-the-art performance on three benchmark datasets. in the future, we plan to extend our framework to address other unsupervised problems.