A Sequence-to-Structure Approach to Document-level Targeted Sentiment Analysis (2023)

Abstract: Most previous studies on aspect-based sentiment analysis (ABSA) were carried out at the sentence level, while the research of document-level ABSA has not received enough attention. In this work, we focus on the document-level targeted sentiment analysis task, which aims to extract the opinion targets consisting of multi-level entities from a review document and predict their sentiments. We propose a Sequence-to-Structure (Seq2Struct) approach to address the task, which is able to explicitly model the hierarchical structure among multiple opinion targets in a document, and capture the long-distance dependencies among affiliated entities across sentences. In addition to the existing Seq2Seq approach, we further construct four strong baselines with different pretrained models. Experimental results on six domains show that our Seq2Struct approach outperforms all the baselines significantly. Aside from the performance advantage in outputting the multi-level target-sentiment pairs, our approach has another significant advantage - it can explicitly display the hierarchical structure of the opinion targets within a document. Our source code is publicly released at https://github.com/NUSTM/Doc-TSA-Seq2Struct.

## introduction aspect-based sentiment analysis (absa) has received wide attention in nlp for nearly two decades @xcite . most of the previous studies have focused on sentence-level absa. however, a review text often consists of multiple sentences, and the opinion targets expressed in these sentences are often interrelated. conducting sentence-level absa on each individual sentence cannot capture the interrelated opinion targets in the entire document. in comparison, documentlevel absa is more suitable for practical appli-input : such a great restaurant . service is quick and friendly , even when it 's crowded . staff was really welcoming and recommended the lobster roll , but price was very expensive . cations, yet it has not received enough attention. only a limited amount of work attempted to identify the sentiments towards all aspect categories in a document (e.g., the food quality category in a restaurant domain), without involving explicit opinion target terms (e.g., entities or aspects) @xcite @xcite @xcite . recently, @xcite introduced a new task called document-level targeted sentiment analysis (document-level tsa), aiming to discover the opinion target consisting of multi-level entities (or aspects) in a review document, and predict the sentiment polarity label (positive, negative or mixed) towards the target. in sentence-level absa, an opinion target is usually a single entity or aspect. while in document-level tsa, it often involves multiple entities and aspects throughout the review document, and their relation is affiliated rather than flat. as shown in figure 1 , "restaurant -lobster_roll -price" is an opinion target consisting of three-level entities foot_0 , indicating the price of the lobster roll sold in the restaurant. @xcite accordingly proposed a sequence-tosequence (seq2seq) framework to solve the task. by using bart as the backbone, they took the review document as the input, and output a sequence indicating a set of target and sentiment tuples. for example, the tuple "<@xmath0. as shown in figure 1 (b), there is actually a hierarchical structure among multiple opinion targets in the document: the first layer is the "restaurant" entity, the second layer contains three entities (or aspects) affiliated to "restaurant" ("service", "lobster_roll" and "staff "), and the third layer is the "price" aspect of "lobster_roll". although the seq2seq method appears simple and straightforward, it is imperfect to model such complex hierarchical structure. on one hand, it outputs the structural information by a sequence of tuples, where the previous tuples affects the generation of subsequent ones. on the other hand, the inherent encoder-decoder architecture is less flexible and effective to model the long-distance dependencies among affiliated entities/aspects across sentences. to address the aforementioned issues, we propose a sequence-to-structure (seq2struct) approach in this work for the document-level tsa task. our approach still takes a document as the input, but the output is no longer a sequence, but a structure as shown in figure 1 @xcite . it is a hierarchical structure with multiple layers of related entities, where each entity is assigned a predicted sentiment polarity. seq2struct contains four mains steps. we firstly identify flat opinion entities and their sentiments from the document. secondly, we propose a multi-grain graphical model based on graph convolutional network (gcn), to better learn the semantic relations between document, sentences and entities. thirdly, we employ a table-filling method to identify the affiliation relations in flat entities and consequently get the hierarchical opinion target structure. we finally incorporate the sentiments of the flat entities into the hierarchical structure and parse out the target-sentiment pairs as defined in @xcite . we evaluate our approach on the document-level tsa dataset containing six domains. in addition to the existing approach, we further construct four strong baselines with different pretrained models. the experimental results show that our seq2struct approach outperforms all the baselines significantly on average f1 of the target-sentiment pairs. aside from the advantage of performance, our approach can further explicitly display the hierarchical structure of the opinion targets in a document. we also make in-depth discussions from the perspectives of document length, the number of levels in opinion target, etc., verifying the effectiveness of our approach in capturing the long-distance dependency among across-sentence entities in document-level reviews. ## task description in document-level tsa task, an opinion target often consists of multi-level entities with affiliated relations (e.g., "restaurant -lobster_roll -price"). we call it "multi-level opinion target" in contrast to the opinion target at the sentence level. a document normally contain a set of multi-level opinion targets, which constitute a hierarchical structure as shown in figure 1(b) . similar as @xcite , we formulate document-level tsa as a task to detect a set of target-sentiment pairs from a document @xmath1, x 2 , ..., x n ] with n tokens: the document-level tsa task is challenging, as the opinion target consisting of multi-level entities, and a predicted opinion target is considered to be correct if and only if entities at all levels match the ground-truth exactly. for instance, "restaurantlobster-price" is the correct target only if the first, second, and third levels are predicted as "restaurant","lobster", and "price", respectively. ## approach as shown in figure 2 , we propose a sequenceto-structure (seq2struct) approach to address the document-level tsa task, which consists of four main modules. ## conclusion in this work, we focus on a document-level absa task, called document-level tsa, which aims to extract the opinion targets consisting of multilevel entities from a review document and predict the corresponding sentiments. different from the existing seq2seq mythology, we propose a sequence-to-structure (seq2struct) approach to address this task, to model the hierarchical structure among multiple opinion targets and capture the long-distance dependencies among affiliated entities. experiments have verified the advantages of our seq2struct approach in more accurately extracting multi-level opinion targets and predicting their sentiments, and more explicitly displaying the hierarchical structure of the opinion targets in a document.