Seeking Common but Distinguishing Difference, A Joint Aspect-based Sentiment Analysis Model (2021)

Abstract: Aspect-based sentiment analysis (ABSA) task consists of three typical subtasks: aspect term extraction, opinion term extraction, and sentiment polarity classification. These three subtasks are usually performed jointly to save resources and reduce the error propagation in the pipeline. However, most of the existing joint models only focus on the benefits of encoder sharing between subtasks but ignore the difference. Therefore, we propose a joint ABSA model, which not only enjoys the benefits of encoder sharing but also focuses on the difference to improve the effectiveness of the model. In detail, we introduce a dual-encoder design, in which a pair encoder especially focuses on candidate aspect-opinion pair classification, and the original encoder keeps attention on sequence labeling. Empirical results show that our proposed model shows robustness and significantly outperforms the previous state-of-the-art on four benchmark datasets.

## introduction sentiment analysis is a task that aims to retrieve the sentiment polarity based on three levels of granularities: document level, sentence level, and entity and aspect level @xcite , which is under the urgent demands of several society scenarios @xcite @xcite . recently, the aspect-based sentiment analysis (absa) task @xcite , focusing on excavating the specific aspect from an annotated review, has aroused much attention from researchers, in which this paper mainly concerns the aspect/opinion term extraction and sentiment classification task. the latest benchmark proposed by @xcite formulates the relevant information into a triplet: target aspect object, opinion clue, and sentiment polarity the view is spectacular, and the food is great. to tackle the error cascading phenomenon on the pipeline model, a growing trend of jointly modeling these subtasks in one shot appears. @xcite proposed a joint model using a sequence tagging method, based on the bidirectional long short-term memory (lstm) and conditional random fields (crf). however, they found that if a tagged mention belongs to more than one triplet, this method will be ineffective. @xcite proposed a multi-task learning approach with the aid of dependency parsing on tail word pair of corresponding aspect-opinion pair. however, this non-strict dependency parsing may miss capturing structural information of term span. meanwhile, the many-target to one-opinion issue is not effectively handled. the promising results achieved by machine reading comprehension (mrc) frameworks on solving many other nlp tasks @xcite ) also inspires the aste task. @xcite and @xcite attempted to design question-answer pair in terms of mrc to formulate the triplet extraction. nevertheless, both need to make the converted question correspond one-to-one to the designed question manually, hence increasing computation complexity. among these joint models, wu et al. (2020) transformed the sequence representation into the two-dimension space and argued that the word-pair under at least one assumption could represent the aspect-opinion pair as input of different encoders. although this model indicated significant improvement, it treated the word-pair without taking span boundary of aspect term and opinion term into consideration and incorporated nonexistent pre-defined aspect-opinion pairs. considering the problems mentioned above, we propose a dual-encoder model based on a pretrained language model by jointly fine-tuning multiple encoders on the absa task. similar to prior work, our framework uses a shared sequence encoder to represent the aspect terms and opinion terms in the same embedding space. moreover, we introduce a pair encoder to represent the aspectopinion pair on the span level. thus, our dualencoder model could learn from the absa subtasks individually and benefit from each other in an end-to-end manner. experiments on benchmark datasets show that our model significantly outperforms previous approaches at the aspect level. we also conduct a series of experiments to analyze the gain of additional representation from the proposed dualencoder structure. the contributions of our work are as follows: ## case study to investigate why our model far exceeds the baseline models, we conduct a case study of three typical cases from 14lap test dataset of aste-data-v2, as shown in from example-1, we observe that our model is able to handle the one-to-one case. however, our dual-encoder structure is more biased towards coordinative relation between colors and speedy. more cases we investigated further demonstrating that our model performs slightly worse on one-toone than one-to-many and many-to-many relation types. from example-2, we see that our model can tackle the one-opinion to many-target problem. however, most previous works are even unable to tackle one-opinion to two-target. from example-3, we observe that our model is capable of well handling the one-target to many-opinion problem, which is neglected by most of the existing work but important for triplet extraction. because many sentences compose conflicting sentiments on target, the model will fail to recognize the opposite polarity of the same at when the incorrect at extraction happens. finally, we also observe that our model accurately inferences the boundary of osx lion span, which demonstrates the usefulness of our transformation that utilizes span to replace the word. from example-4, we notice that our model could efficiently handle the complex situation of many-opinion to many-target with long-range dependency, which was particularly paid attention to but not solved well by @xcite . it is due to incorporating the self-attention mechanism and gru in two dimensions, and our model is sensitive to the difference between the proposed dual-encoder architecture. collectively, these aforementioned cases demonstrate the robustness of our dual-encoder model. ## related work recently, nlp has been developed rapidly @xcite @xcite @xcite , and the process is further by deep neural networks @xcite and pre-trained language models @xcite . aspect-based sentiment analysis was proposed by @xcite and also received lots of attention in recent years. ## conclusion in this paper, we observe the significant differences between the at/ot extraction subtask and the sc subtask of absa for the joint model. specifically, the results on 8 benchmark datasets with significant improvement over state-of-the-art baselines verify the effectiveness of our proposed model. furthermore, to distinguish such differences and keep the shared part between different modules simultaneously, we construct a dual-encoder framework with representation learning and self-attention mechanism. in addition to the encoder-sharing approach, our dual-encoder framework can capture the difference between the subtasks by interconnecting encoders at each layer to share the critical information.