On the Strength of Sequence Labeling and Generative Models for Aspect Sentiment Triplet Extraction (2023)

Abstract: Generative models have achieved great success in aspect sentiment triplet extraction tasks. However, existing methods ignore the mutual informative clues between aspect and opinion terms and may generate false paired triplets. Furthermore, the inherent limitations of generative models, i.e., the token-by-token decoding and the simple structured prompt, prevent models from handling complex structures especially multi-word terms and multi-triplet sentences. To address these issues, we propose a sequence labeling enhanced generative model. Firstly, we encode the dependency between aspect and opinion into two bidirectional templates to avoid false paired triplets. Secondly, we introduce a marker-oriented sequence labeling module to improve generative models’ ability of tackling complex structures. Specifically, this module enables the generative model to capture the boundary information of aspect/opinion spans and provides hints to decode multiple triplets with the shared marker. Experimental results on four datasets prove that our model yields a new state-of-art performance. Our code and data are available at https://github.com/NLPWM-WHU/SLGM .

## introduction aspect sentiment triplet extraction (aste) aims at extracting all triplets in a sentence, consisting of the aspect/opinion terms and the sentiment polarity on them. given the example "their twist on pizza is healthy, but full of flavor." in fig. 1 (a), the goal is to extract two triplets (twist on pizza, healthy, positive) and (flavor, full, positive). conventional approaches to aste include pipeline @xcite , table filling @xcite , sequence tagging @xcite , and hybrid ones @xcite . more recently, there is an emerging trend in adopting generative models for aste @xcite ; c ) illustrate the difference between our proposed generative method and existing ones for this task, where x is the input sentence, and t denotes the target triplet. x a /x o contains the prompt prefix to define the decoding order (aspect or opinion first) while y a /y o indicates the generated sequences following the order in x a /x o . mosl is our marker-oriented sequence labeling module to improve the generative model's ability of handling complex structures. @xcite to alleviate error propagation and exploit full label semantics. current generative aste models employ a classical encoder-decoder architecture and follow a paradigm that first generates a target sequence y and then recovers the triplets t from the sequence y . the model needs to pre-define an output template ψ(•) to convert aste into text generation and then calculates the loss between the triplet and the generated sequence for model training, as shown in fig. 1 (b ). the template ψ(•) constructed by existing methods is in the form of ψ @xmath0, reflecting the unidirectional dependency from aspect to opinion, or vice versa. however, the aspect and opinion terms that appear together in one sentence might hold informative clues to each other @xcite and there is no intrinsic order between them @xcite . hence, modeling unidirectional dependency may mislead the model to generate false paired triplets like (twist on pizza, full, positive). existing generative aste models also suffer from another challenging problem, i.e., lacking the ability to handle complex structures especially multi-word terms and multi-triplet sentences. on one hand, the token-by-token decoding manner makes the model focus only on the next token at each time step of decoding without grasping the whole information of the aspect/opinion term with multiple words. on the other hand, generative models often deploy the simple-structured prompt template to ensure the generation quality. when handling the sentence with multiple triplets, a generative model needs to invoke a template several times, which may lead to an information confusion for the same marker in the template. to address the aforementioned issues, we propose a sequence labeling enhanced generative model for aste. firstly, we design two bidirectional templates with different decoding orders to simultaneously capture the mutual dependency between the aspect and opinion terms. in particular, we add two types of prompt prefix before the input sentence to indicate the decoding order, and we also present two output templates ψ @xmath1, both consisting of the markers {aspect, opinion, sentiment} and the corresponding labels {a, o, s}. in this way, the decoder can generate two sentences reflecting dependency from aspect to opinion and that from opinion to aspect. secondly, we propose a marker-oriented sequence labeling (mosl) module, which can enhance the generative model's ability to handle complex structures. specifically, the decoding is conducted after the mosl module at the training stage. hence the bio tags obtained in mosl help the generative model capture the boundary information of multi-word aspect/opinion terms in advance. moreover, while the generative model needs to invoke the output templates several times for the multitriplet sentence, we adopt different marker vectors in mosl for the same marker in the generative model. by doing this, we can share the markers without causing confusion. since the markers encode information across multiple triplets in one sentence, previous markers can contribute to the decoding of subsequent triplets. the illustration of our proposed method is shown in fig. 1 (c ). we conduct extensive experiments on four datasets with both full supervised and low-resource settings. the results demonstrate that our model significantly outperforms the state-of-art baselines for the aste task. ## related work aspect-based sentiment analysis traditionally involves three basic tasks, including aspect extraction @xcite @xcite , aspect-level sentiment classification @xcite @xcite , and opinion extraction @xcite . to meet the practical need, some recent studies propose to extract two or more elements simultaneously, including aspect opinion pair extraction @xcite @xcite , end-to-end aspect-based sentiment analysis @xcite @xcite , and aspect sentiment triplet extraction. among them, atse is regarded as a near complete task and is of the most challenge. earlier work in atse can be sorted into four streams, i.e., pipeline @xcite , table filling @xcite , sequence tagging @xcite , and hybrid ones @xcite @xcite . these methods do not fully utilize the rich label semantics and some of them may encounter the error propagation problem. another line of research in aste performs this task in a generative manner @xcite . for example, @xcite model the extraction and classification tasks as the generation of pointer indexes and class indexes. @xcite introduce the structured extraction language and structural schema instructor to unify all information extraction tasks. while getting better performance, current generative models are prone to generate false paired triplets and are not suitable for tackling complex structures. our generative model addresses these issues with the proposed bidirectional templates and the marker-oriented sequence labeling module. ## our method given a review sentence x with l words, the goal of aste is to extract all triplets @xmath2, o, s)} n @xmath3, where n is the number of triplets, and a, o, and s denotes aspect term, opinion term, and sentiment polarity, respectively. we first introduce the overall architecture of our proposed sequence labeling enhanced generative (1) to capture the mutual information between the aspect and opinion terms, we construct two bidirectional templates at both the input and output ends, shown as x a /x o and ψ @xmath4. 2 . ## conclusion in this paper, we exploit the power of text generation and sequence labeling for aste. we propose two bidirectional templates to reflect the mutual aspect-opinion dependency for filtering false paired triplets. we also present a marker-oriented sequence labeling module to help the text generation module tackle complex structures in the subsequent decoding process. experiment results show that our framework consistently outperforms all generative and non-generative baselines under both the full supervised and low-resource settings.