Transition-based Opinion Generation for Aspect-based Sentiment Analysis (2024)

Abstract: Recently, the use of pre-trained generation models for extracting sentiment elements has resulted in significant advancements in aspect-based sentiment analysis benchmarks. However, these approaches often overlook the importance of explicitly modeling structure among sentiment elements. To address this limitation, we present a study that aims to integrate general pre-trained sequence-to-sequence language models with a structure-aware transition-based approach. Therefore, we propose a transition system for opinion tree generation, designed to better exploit pre-trained language models for structured fine-tuning. Our proposed transition system ensures the structural integrity of the generated opinion tree. By leveraging pre-trained generation models and simplifying the transition set, we are able to maximize the accuracy of opinion tree generation. Extensive experiments show that our model significantly advances the state-of-the-art performance on several benchmark datasets. In addition, the empirical studies also indicate that the proposed opinion tree generation with transition system is more effective in capturing the sentiment structure than other generation models.

## introduction aspect-based sentiment analysis (absa) has been garnering increasing interest within the community. this area encompasses four key subtasks: aspect term extraction, opinion term extraction, aspect term category classification, and aspect-level sentiment classification. the initial two subtasks focus on extracting aspect terms and opinion terms from within a given sentence. the subsequent two subtasks aim to identify the category of the extracted aspect term and determine its sentiment polarity. through these subtasks, absa offers a comprehensive approach to analyzing sentiment at a more granular, aspect-based level. previously, most absa tasks were formulated as either sequence-level @xcite @xcite or token-level classification problems @xcite . however, these approaches often encountered significant challenges due to error propagation, as the overall prediction performance was heavily dependent on the accuracy of each individual step @xcite . as a result, recent studies have shifted towards tackling the absa problem with a unified generative approach @xcite @xcite @xcite . this new approach offers a promising direction for absa research, as it aims to mitigate the issues caused by error propagation in traditional methods. despite their promise, these unified generative approaches for absa suffer from certain limitations. a notable limitation is lack of a structural guarantee for sentiment elements. this absence means that the model may generate string outputs that do not conform to a valid opinion tree structure, necessitating additional post-processing steps to establish the necessary relationships between sentiment elements. in this study, we propose the transition-based opinion generation model to address the above challenges. our model leverages the generative capabilities of pre-trained language models while also capturing the explicit structure of sentiment elements. as depicted in figure 1 , we devise a transition system consisting of a concise set of fundamental actions. this system forms the foundation of our model, enabling the generation of structurally sound outputs. integrating this approach not only enhances output integrity but also reduces the need for extensive post-processing, thereby streamlining the overall sentiment analysis process. utilizing the developed transition system, we introduce a neural transition-based opinion tree generation model. unlike previous methods @xcite , our model takes a review sentence as input and aims to produce an action sequence adhering to the predefined transition system. this generated action sequence facilitates seamless reconstruction of the opinion tree and sentiment elements, ensuring structural stability and alignment with the original sentiment structure. the detailed evaluation shows that our model significantly advances the state-of-the-art performance on several benchmark datasets. in addition, the empirical studies also indicate that the proposed transition-based opinion tree generation is more effective in capturing the sentiment structure than other generative methods. ## related work aspect-based sentiment analysis (absa) has drawn wide attention during the last decade. early studies focus on the prediction of a single element, such as extracting the aspect term @xcite , detecting the mentioned aspect category @xcite , and predicting the sentiment polarity for a given aspect @xcite @xcite . some works further consider the joint detection of two sentiment elements, including the pairwise extraction of aspect and opinion term @xcite ; the prediction of aspect term and its corresponding sentiment polarity @xcite ; and the co-extraction of aspect category and sentiment polarity @xcite . recently, aspect sentiment triplet and quadruple prediction tasks are proposed in absa, they employ end-to-end models to predict the sentiment elements in triplet or quadruple format @xcite @xcite @xcite @xcite . more recently, there are some attempts on tackling absa problem in a sequence-to-sequence manner @xcite , either treating the class index @xcite or the desired sentiment element sequence @xcite as the target of the generation model. for example, @xcite treated the absa as a text generation problem, and employ a sequenceto-sequence pre-trained model to generate the sequence of aspect terms and opinion words directly. @xcite proposed a paraphrase model that utilized the knowledge of the pre-trained model via casting the original task to a paraphrase generation process. they employed the paraphrase to represent aspect-based quads. @xcite employed a generation model to generate all the sentiment elements as a tree structure. @xcite simultaneously detected aspect categories and co-extract aspect-opinionsentiment triplets, can absorb deeper interactions between sentiment elements without error propagation. our study differs from previous research in that we integrate pre-trained sequence-to-sequence language models with a transition-based approach for opinion tree parsing. this integration allows us to explore the complementarity between these two powerful techniques and assess their combined potential for enhancing sentiment analysis. ## preliminaries as shown in figure 2 , our proposed approach involves several key steps. firstly, we introduce a transition system that serves to normalize sentiment elements into an opinion tree structure. next, we employ a neural transition-based opinion tree generation model to generate an action sequence from a given review text. following the generation of the action sequence, we construct the opinion tree based on this sequence and the transition system. finally, since all sentiment elements are normalized into the opinion tree, it becomes straightforward to recover them from the tree. in this section, we give the definition of the aspect-based sentiment analysis task and the con- the wine list has a good price. review sentence encoder decoder action sequence constrained decoding wine list (aspect) price (category) good (opinion) positive (polarity) opinion tree aspect term wine list aspect category price opinion term good sentiment polarity positive sentiment quadruple arc p 0 op 0 arc r quad 1 … root transition-based opinion tree generation @xmath0. quad sentiment quadruple. ap virtual node of aspect. non op virtual node of opinion. -terminal at aspect term. ca category of aspect. ot opinion term. sp sentiment polarity. terminal aspect e.g., wine list, snacks category e.g., food quality opinion e.g., good, delicious polarity e.g., positive, negative ## transition system in this section, we present a transition system specifically designed for opinion tree generation. figure 3 shows a parsed example of a transition action sequence and the graph structure from opinion tree. diverging from previous transition-based approaches, our transition system operates without employing traditional data structures like stacks or buffers. in particular, receiving a source sentence @xmath1, x 2 , ..., x n }, our transition system undertakes a left-to-right scan of the sentence using a cursor c t , where @xmath2, 2, ..., n}. transition actions involve either shifting the cursor by one token forward or generating multiple nodes and edges while the cursor points to the same token. the transition process concludes when the final word in the sentence is shifted, signifying the completion of parsing processing. available actions for our transition system are as follows: within a transition system, nodes are exclusively generated @xmath3. this offers an opportunity to leverage the pre-trained vocabulary on the target side of the generation model, thereby maximizing the utilization of linguistic knowledge acquired during pre-training. furthermore, the use of a cursor variable in the transition system disentangles node referencing from source tokens, enabling the generation of multiple nodes and edges under the same token, even constructing the entire opinion tree structure if necessary. this imparts more expressiveness and flexibility to opinion tree generation, particularly when aspect term or opinion term is implicit. in summary, our proposed transition system ensures the preservation of structural coherence within the generated opinion tree. through the strategic utilization of pre-trained generation model and streamlining the transition set, we are able to achieve optimal accuracy in opinion tree generation. ## transition-based opinion tree generation in this section, we employ a sequence-to-sequence model to generate the action sequence of the transition system via a transformer-based encoderdecoder architecture. as discussed in the above sections, the opinion tree can be generated based on the action sequence and the transition system. ## experiments in this section, we introduce the datasets used for evaluation and the baseline methods employed for comparison. we then report the experimental results conducted from different perspectives. ## conclusion in this study, we introduce a novel approach named transition-based opinion tree generation, which seeks to bridge the gap between general pre-trained sequence-to-sequence language models and a structure-aware transition-based methodology. our approach diverges from traditional methods by incorporating a transition system specifically tailored for opinion tree generation, designed to leverage the power of pre-trained language models through structured fine-tuning. comprehensive experiments demonstrate that our model achieves substantial improvements over the state-of-the-art performance on multiple benchmark datasets. furthermore, empirical studies reveal that our proposed transition-based opinion tree generation not only outperforms generative models but also excels in capturing the intricate sentiment structure within text.