Target-specified Sequence Labeling with Multi-head Self-attention for Target-oriented Opinion Words Extraction (2021)

Abstract: Opinion target extraction and opinion term extraction are two fundamental tasks in Aspect Based Sentiment Analysis (ABSA). Many recent works on ABSA focus on Target-oriented Opinion Words (or Terms) Extraction (TOWE), which aims at extracting the corresponding opinion words for a given opinion target. TOWE can be further applied to Aspect-Opinion Pair Extraction (AOPE) which aims at extracting aspects (i.e., opinion targets) and opinion terms in pairs. In this paper, we propose Target-Specified sequence labeling with Multi-head Self-Attention (TSMSA) for TOWE, in which any pre-trained language model with multi-head self-attention can be integrated conveniently. As a case study, we also develop a Multi-Task structure named MT-TSMSA for AOPE by combining our TSMSA with an aspect and opinion term extraction module. Experimental results indicate that TSMSA outperforms the benchmark methods on TOWE significantly; meanwhile, the performance of MT-TSMSA is similar or even better than state-of-the-art AOPE baseline models.

## introduction aspect-based sentiment analysis (absa) @xcite has attracted much attention of researchers in recent years. in absa, aspect (or called opinion target) extraction and opinion term extraction are two fundamental tasks. aspect is the word or phrase in the reviews referring to the object towards which users show attitudes, while opinion terms are those words or phrases representing users' attitudes @xcite . for example, in the sentence "the dim sum is delicious.", the phrase "dim sum" is an aspect and the word "delicious" is an opinion term. see the upper part of and opinion term extraction @xcite @xcite ; moreover, some studies combine these two tasks into a multi-task structure to extract aspects and opinion terms simultaneously @xcite @xcite . however, one critical deficiency in the researches mentioned above is that they ignore the relation of aspects and opinion terms, which leads to the birth of target-oriented opinion words (or terms) extraction (towe) @xcite for extracting the corresponding opinion terms of a given opinion target. subsequently, aspect-opinion pair extraction (aope) @xcite and pair-wise aspect and opinion terms extraction (paote) @xcite have emerged, which both aim at extracting aspects and opinion terms in pairs. aope and paote are exactly the same task, only named differently. in the following, we use aope to denote this task for simplicity. it can be considered that aope contains aspect and opinion word extraction and towe. since aspect extraction has been fully studied and satisfactory results have been obtained, towe, which aims at mining the relation between aspects and opinion terms, is the key to the aope task. as shown in the lower part of the challenge of towe is the learning of representations of the given opinion target accurately and a few works focus on this task. for instance, @xcite propose an inward-outward lstm to pass target information to the left context and the right context of the target respectively, and then they combine the left, right, and global context to encode the sentence. recently, sdrn @xcite and spanmlt @xcite both adopt a pre-trained language model to learn contextual representations for aope. in sdrn, a double-channel recurrent network and a synchronization unit are applied to extract aspects, opinion terms and their relevancy. in spanmlt, the terms are extracted under annotated span boundaries with contextual representations, and then the relations between every two span combinations are identified. however, apart from hyper-parameters in the pre-trained language model, these two methods introduce many other hyper-parameters (e.g., the hidden size, thresholds and recurrent steps in sdrn, and the span length, top k spans and the balanced factor of different tasks in spanmlt). some of these hyper-parameters have a significant impact on the model performance. motivated by the previous work and to address the challenges mentioned above, we propose a target-specified sequence labeling method based on multi-head self-attention @xcite (tsmsa). the sentence is first processed in the format "[sep] aspect [sep]" (e.g., "the [sep] food [sep] is delicious."), which is inspired by @xcite who utilized a special symbol "[sep]" to label all entities and output their corresponding representations. then we develop a sequence labeling model based on multi-head self-attention to identify the corresponding opinion terms. by using the special symbol and self-attention mechanism, tsmsa is capable of capturing the information of the specific aspect. to improve the performance of our model, we apply pre-trained language models like bert @xcite which contain a multi-head self-attention module as the encoder. as a case study, we integrate aspect and opinion term extraction, and towe into a multi-task architecture named mt-tsmsa to validate the effectiveness of our method on the aope task. in addition, apart from hyper-parameters in the pre-trained language model, we only need to adjust the balanced factor of different tasks in mt-tsmsa. in summary, our main contributions are as follows: extensive experiments validate that our tsmsa can achieve the best performance on towe, and mt-tsmsa performs quite competitive on aope. the rest of this paper is organized as follows. section 2 introduces the existing studies on towe and aope, respectively. section 3 details the proposed tsmsa and mt-tsmsa. section 4 presents our experimental results and discussions. finally, we draw conclusions in section 5. ## conclusions in this paper, we propose a target-specified sequence labeling method based on multi-head selfattention (tsmsa) and a multi-task version (mt-tsmsa) to deal with towe and aope, respectively. in our methods, the encoder is capable of capturing the information of the specific aspect which is labeled by a special symbol "[sep]". experimental results demonstrate that tsmsa and mt-tsmsa achieve quite competitive performance in most cases. when combining aspect and opinion words extraction with towe, our mt-tsmsa can slightly improve the performance as compared with tsmsa. in the future, we plan to extend our approaches to sentiment classification of pairs and explore an efficient model with a one-stage inference process to reduce the time complexity on aope.