# Incorporating Word Significance into Aspect-Level Sentiment Analysis
###### Abstract
Taking the sentence \"Great food, REASONABLE prices, makes for an evening that can't be beat\" as an example, human attention is drawn to the word \"REASONABLE\" with respect to the aspect word \"prices\" because of the capitalization. Current methods bypass this step resulting in most methods encompassing of embedding, contextual, attention and output layers [2; 8; 12], resulting in attention not being directed to the most significant words. The second problem in current models is overlooking important words that may be at the end of a sentence. Such models ignore the notion that humans make incremental interpretations of a sentence by maximizing the degree each new encountered word affects sentence interpretation [18; 19]. This implies that the ultimate sentiment polarity keeps being reviewed as more words are accessed. For example in this sentence \"My day off after a wedding consist of wii zelda and chinese food. I feel like im in jr high. This rocks! !\", the polarity towards the aspect word \"wii\" is neutral if only \"My day off after a wedding consist of wii zelda and chinese food\" is considered, the polarity is still neutral if the part sentence \"My day off after a wedding consist of wii zelda and chinese food. I feel like im in jr high\" is considered. The ultimate sentiment polarity is established when the ending words \"This rocks! !\" are considered, then the polarity towards the aspect word \"wii\" is positive. We therefore argue that incorporating incremental interpretation into aspect-level sentiment analysis may tackle sentences where there the relevant aspect and context words are far apart. The last problem current models have is determining sentiment polarity when an aspect word and the relevant context word are far apart from each other. Current models overlook the concept of decaying human reaction to repeated stimuli [20; 21] and therefore word importance decreases as sentence length increases until there is a complex stimuli triggered by an encountered context word [22; 23]. Current methods do not take this quality into consideration but instead use global attention based models which do not take explicit positions of a word [5; 6; 24; 25], while position based attention assumes that a context word has higher importance if it is closer to an aspect word [11; 12]. We argue that computational methods could benefit from taking language processing psychological evidence into consideration in aspect-level sentiment analysis. To mitigate the first problem we propose a unified module dubbed as word significance as an attention filter to guide our model to pay more attention to the most significant words. The word significance module also acts as a propagation mechanism that allows a high significance context word that may be far from an aspect word to still have influence on the sentiment polarity. The word significance factor is made up of two aspects being an increasing incremental interpretation factor as a solution to the second problem, which is counter balanced by a novelty decay factor as a solution to the third problem. We solve incremental interpretation using a growth function that increases as more words are accessed in a sentence, as previously used in determining research paper significance growth [21; 26]. We propose to represent novelty decay using the stretched exponential law which has proven successful in representing phenomena relating to human nature dynamics with a natural time span decay. To the best of our knowledge the use of the stretched exponential has not been used in NLP, particularly in aspect-level sentiment analysis. Experimental results show that our model performs competitively with current models. The aim of this paper is to demonstrate the importance of word significance in directing attention to the most attention worthy words in aspect-level sentiment analysis. The contributions of this paper are as follows:  
* We use the word significance factor to model attention worthiness in aspect-level sentiment analysis using the Significant Attention Network (SAN) model.