Target-Guided Structured Attention Network for Target-Dependent Sentiment Analysis (2020)

Abstract: Target-dependent sentiment analysis (TDSA) aims to classify the sentiment of a text towards a given target. The major challenge of this task lies in modeling the semantic relatedness between a target and its context sentence. This paper proposes a novel Target-Guided Structured Attention Network (TG-SAN), which captures target-related contexts for TDSA in a fine-to-coarse manner. Given a target and its context sentence, the proposed TG-SAN first identifies multiple semantic segments from the sentence using a target-guided structured attention mechanism. It then fuses the extracted segments based on their relatedness with the target for sentiment classification. We present comprehensive comparative experiments on three benchmarks with three major findings. First, TG-SAN outperforms the state-of-the-art by up to 1.61% and 3.58% in terms of accuracy and Marco-F1, respectively. Second, it shows a strong advantage in determining the sentiment of a target when the context sentence contains multiple semantic segments. Lastly, visualization results show that the attention scores produced by TG-SAN are highly interpretable

## introduction target-dependent sentiment analysis (tdsa) is an actively studied research topic with the aim to determine the sentiment polarity of a text towards a specific target. for example, given a sentence ''the food is so good and so popular that waiting can really be a nightmare'', the target-dependent sentiments of food and waiting are positive and negative, respectively. the major challenge of tdsa lies in modeling the semantic relatedness between the target and its context sentence @xcite . most recent progress in this area benefits from the attention mechanism, which captures the relevance between the target and every other word in the sentence. based on such word-level correlations, several models have already been proposed for constructing target-related sentence representations for sentiment prediction @xcite @xcite @xcite . one important underlying assumption in existing attention-based models is that words can be used as independent semantic units for modeling the context sentence when performing tdsa. this assumption neglects the fact that a sentence is oftentimes composed of multiple semantic segments, where each segment may contain multiple words expressing a certain meaning or sentiment collectively. furthermore, different semantic segments may even contribute differently to the sentiment of a certain target. figure 1 shows an example of a restaurant review, which contains two salient semantic segments (highlighted in blue). intuitively, a tdsa model should be able to identify both segments and determine that the second one is more relevant to the writer's sentiment towards the target @xcite . existing methods, however, would only attend important words (highlighted in red) such as ''good'', ''popular'', ''really'', and ''nightmare'' individually through the aforementioned assumption. we hypothesize that the ability to uncover multiple semantic segments and their relatedness with the target from a context sentence will be beneficial for tdsa. in this light, we propose a fine-to-coarse tdsa framework, namely, target-guided structured attention network (tg-san) figure 1: a motivating example, where darker shades denote higher contributions to the sentiment of the target @xcite . (a) a tdsa model should be able to identify two salient segments from the sentence, and that the second one is more important for determining the target's sentiment. (b) existing attention-based models would attend important words individually and fail to determine their relatedness with the target. in this paper. the core components of tg-san include a structured context extraction unit (scu) and a context fusion unit (cfu). as opposed to using word-level attention, the scu utilizes a target-guided structured attention mechanism to encode multiple semantic segments of a sentence as a structured embedding matrix, where each vector in the matrix can be viewed as one target-related context. the cfu then fuses the extracted contexts based on their relatedness with the target to construct the ultimate context representation of the target for sentiment classification. our contributions are summarized as follows: given a target and its context sentence, the major challenge of tdsa lies in identifying targetrelated contexts in the sentence for determining the target's sentiment. early work adopted rulebased methods or statistical methods to solve this problem @xcite @xcite . these methods relied either on handcrafted features, rules, or sentiment lexicons, all of which required massive manual efforts. in recent years, neural networks have achieved great success in various fields for their strong representation capability. they have also been proven effective in modeling the relatedness between the target and its contexts. recursive neural networks were first used by @xcite and nguyen and shirai (2015) for tdsa. specifically, the target was first converted into the root node of a parsing tree, and then it contexts were composed based on syntactic relations in the tree. as such approaches rely strongly on dependency parsing, they fall short when analyzing nonstandard texts such as comments and tweets, which are commonly used for sentiment analysis. another line of work applied recurrent neural network (rnn) and its extensions to tdsa for their natural way of encoding sentences in a sequential fashion. for instance, @xcite utilized two rnns to individually capture the left and the right contexts of the target, and then combined the two contexts for sentiment prediction. @xcite elaborated on this idea by using a gate to leverage the contributions of the two contexts for sentiment prediction. however, such rnn-based methods place more emphasis on the words near the target while ignoring the distant ones, regardless of whether they are target-related. recently, attention mechanisms have become widely used for modeling the relatedness between every context word and the target for tdsa @xcite @xcite . for example, @xcite assigned attention scores to each context word according to their relevance to the target, and combined all context words with their attention scores to constitute the context representation of the target for sentiment classification. the aforementioned attention-based methods used a single attention layer to capture targetrelated contexts. one drawback of this has been recently examined by @xcite and li the memory builder (section 3.2) takes a sequence of dense word vectors @xmath0, . . . , x i , . . . , x l } as input, and obtains the contextualized word representations @xmath1, . . . , h i , . . . , h l } via a bi-lstm. h is then split into a context memory m c and a target memory m t based on the positions of the target. the scu (section 3.3) applies a self-attentive operation on the target memory to obtain a structured target representation r t , which is used to guide the extraction of r target-related segments r c from the context memory through a structured attention mechanism. the cfu (section 3.4) generates the target vector r t through a self-attentive operation on r t , and then learns the contribution of each context to obtain the ultimate context vector r c . finally, the output layer (section 3.5) composes the context vector and the target vector for predicting the target's sentiment. et al. @xcite . they argued that using one layer of attention to attend all context words may introduce noises and degrade classification accuracy. to alleviate this problem, @xcite proposed refining the attended words in an iterative manner, whereas @xcite used a convolutional neural network to extract n-gram features whose contributions were decided by their relative positions to the target in the context sentence. to the best of our knowledge, no existing study has explicitly considered uncovering a sentence's semantic segments and learning their contributions to a target's sentiment. we address this problem with a novel target-guided structured attention network in this work. ## approach we first mathematically formulate the tdsa problem addressed in this paper, and then describe the proposed tg-san. figure 2 depicts the architecture of tg-san. ## conclusions and future work in this paper, we develop a novel target-guided structured attention network (tg-san) for target-dependent sentiment analysis (tdsa). as opposed to the simple word-level attention mechanism used by existing models, tg-san uses a fine-to-coarse attention framework to uncover multiple target-related contexts and then fuse them based on their relatedness with the target for sentiment classification. the effectiveness of tg-san is validated through comprehensive experiments on three public benchmark datasets. it also demonstrates superior ability in handling multi-segment sentences, which contain multiple targets or multiple mentions of the same target. in addition, the attention results it produces are highly interpretable as visualization results shown. as future work, we may extend this study in two directions. first, the scu is currently utilized once to extract target-related contexts from a sentence, but extending such fine-to-coarse framework through iterative use of multiple scus is also feasible from the model perspective. second, we would like to explore the effectiveness of our model in other tasks where semantic relatedness plays an important role as in tdsa, such as the answer sentence selection task for questionanswering. by the enterprise support scheme (ess) of the hong kong innovation and technology fund (no. b/e022/18). any opinions, findings, conclusions or recommendations expressed in this paper do not reflect the views of the government of the hong kong special administrative region, the innovation and technology commission, or the ess assessment panel.