i ACOS : Advancing Implicit Sentiment Extraction with Informative and Adaptive Negative Examples (2024)

Abstract: Aspect-based sentiment analysis (ABSA) have been extensively studied, but little light has been shed on the quadruple extraction consisting of four fundamental elements: aspects, categories, opinions and sentiments, especially with implicit aspects and opinions. In this paper, we propose a new method iACOS for extracting Implicit Aspects with Categories and Opinions with Sentiments. First, iACOS appends two implicit tokens at the end of a text to capture the context-aware representation of all tokens including implicit aspects and opinions. Second, iACOS develops a sequence labeling model over the context-aware token representation to co-extract explicit and implicit aspects and opinions. Third, iACOS devises a multi-label classifier with a specialized multi-head attention for discovering aspect-opinion pairs and predicting their categories and sentiments simultaneously. Fourth, iACOS leverages informative and adaptive negative examples to jointly train the multi-label classifier and the other two classifiers on categories and sentiments by multi-task learning. Finally, the experimental results show that iACOS significantly outperforms other quadruple extraction baselines according to the F1 score on two public benchmark datasets.

## introduction aspect-based sentiment analysis (absa) has gained continuous attention during the last decade due to its broad application @xcite @xcite . absa aims to extract tuples consisting of closely related elements including the aspect term, opinion term, aspect category and sentiment polarity. the aspect term refers to a specific word or phrase in a text that is being evaluated, while the opinion term is a subjective statement in the text that expresses a personal sentiment on the aspect term. both the aspect term and opinion term are typically classified into a predefined category and sentiment polarity, respectively. most of the existing works only extract explicit aspects and opinions but completely ignore the implicit ones that are absent from texts. some works consider the extraction of implicit aspects @xcite @xcite , implicit opinions @xcite , or both @xcite @xcite @xcite . in particular, the study @xcite firstly attempts to extract implicit aspects and opinions simultaneously, because real textual reviews often contain a significant amount of implicit aspects and opinions. for example, in the product review "looks nice and the surface is smooth, but certain apps take seconds to respond" @xcite , "surface" is an aspect term and classified into the "design" category, "smooth" is the opinion term toward this aspect with the "positive" sentiment. the four elements constitute an quadruple "surface-design-smooth-positive". obviously, there are two more quadruples: "null-design-nice-positive" and "apps-software-null-negative", where "null" stands for an implicit aspect or opinion term that does not appear in the given text. recently, the two studies @xcite have improved implicit quadruple extraction based on contrastive learning, a method that constructs positive and negative examples for each anchor (i.e., a training example). this method attempts to minimize the distance between the anchor and positive examples and maximize the distance between the anchor and negative examples in the latent representation space. unfortunately, most existing studies suffer from two limitations: (1) uninformative negative examples. in contrastive learning, it is crucial to sample informative negative examples that are difficult to distinguish from the positive examples @xcite . however, exist-ing studies often fail to generate such informative negative examples due to the intrinsic nature of random perturbation methods. (2) non-adaptive negative examples. the negative examples lack adaptiveness, as their sampling is not influenced by the current model parameters @xcite . as a result, there is significant scope to enhance performance in extracting aspect-categoryopinion-sentiment quadruples from texts. therefore, this paper proposes a new method based on informative and adaptive negative examples, namely iacos, for extracting implicit aspects with categories and opinions with sentiments. first, iacos employs the pre-trained encoder bert @xcite to get the context-aware token representation of a text, by which a large amount of knowledge contained in bert can be transferred into iacos. meanwhile, iacos appends two implicit tokens at the end of texts to capture the semantic representation of implicit aspects and opinions, respectively. second, iacos builds a sequence labeling model over the context-aware token representation by extending the bioes foot_0 tagging scheme to co-extract explicit and implicit aspects and opinions; the aspectsopinion co-extraction is preferentially executed, since it is a relatively simple task @xcite @xcite and our extended sequence labeling model can accurately generate aspect and opinion candidates for other subsequent tasks. third, iacos develops a multi-label classifier with a specialized multi-head attention to predict the category-sentiment combination label of each aspect-opinion candidate pair; this classifier is an end-to-end method for discovering aspect-opinion pairs and predicting their categories and sentiments at the same time to alleviate error propagation in the pipeline solution @xcite . fourth, iacos constructs negative examples based on the aspectopinion co-extraction results to train the classifier. these negative examples are informative and adaptive to current model parameters due to two reasons. the main contributions are listed below: the rest of this paper is organized as follows. we highlight related work in section 2. our iacos is presented in section 3, followed by experimental evaluation in section 4. finally, section 5 concludes this paper. ## related work this section reviews the recent advances of sentiment quadruple extraction and contrastive learning. quadruple extraction. recently, @xcite introduce a new task, named aspectcategory-opinion-sentiment quadruple extraction, construct two new datasets for this new task, and benchmark the task with four baseline systems. later, most works @xcite @xcite @xcite apply the sequence-to-sequence model t5 to generate a list of quadruples for a given sentence and can be differentiated from one another in terms of the formats of quadruples. for instance, the literature @xcite represents each quadruple as a paraphrase sentence, the reference @xcite formats all quadruples as an opinion tree with linearized order, and the research @xcite denotes each quadruple as an independent path of a tree without linearized order. other studies @xcite @xcite develop a unified generative framework based on the t5 model with instructional prompts for a variety of absa tasks including quadruple extraction. ## the proposed iacos we define the research problem in section 3.1, introduce the inference process in sections 3.2-3.3, and present the training process in sections 3.4-3.5. ## experiments we present the evaluation setup in section 4.1 and experimental results in section 4.2. ## conclusion in this paper, we propose iacos, a novel approach for extracting implicit sentiment quadruples with multi-label classifier and multi-head attention over the context-aware representation of implicit aspects and opinions. furthermore, we devise an informative and adaptive sample construction method for generating negative examples to train multiple classifiers by multi-task learning. experiment results have verified our method's effectiveness and superiority in comparison to existing strong baselines.