# Incorporating Word Significance into Aspect-Level Sentiment Analysis
###### Abstract  
Aspect-level sentiment analysis has drawn growing attention in recent years, with higher performance achieved through the attention mechanism. Despite this, previous research does not consider some human psychological evidence relating to language interpretation. This results in attention being paid to less significant words especially when the aspect word is far from the relevant context word or when an important context word is found at the end of a long sentence. We design a novel model using word significance to direct attention towards the most significant words, with novelty decay and incremental interpretation factors working together as an alternative for position based models. The interpretation factor represents the maximization of the degree each new encountered word contributes to the sentiment polarity and a counter balancing stretched exponential novelty decay factor represents decaying human reaction as a sentence gets longer. Our findings support the hypothesis that the attention mechanism needs to be applied to the most significant words for sentiment interpretation and that novelty decay is applicable in aspect-level sentiment analysis with a decay factor \(\beta=0.7\). aspect-level sentiment analysis; attention mechanism; novelty decay; incremental interpretation; stretched exponential law +
Footnote †: journal:  
Recent advances in aspect-level sentiment analysis have gravitational towards neural network based models [3; 4]. Such models learn text representations from high dimensional data without careful feature engineering and capture semantic relations between context and aspect words in a more scalable manner [5; 6]. Researchers have further used the attention mechanism in conjunction with Recurrent Neural Networks (RNNs) as effective sequence modeling techniques to achieve higher model accuracy [7; 8]. Continuing from the sentence example provided earlier, one can see that the context word \"graceless\" is more relevant to the aspect word \"manager,\" the word \"rude\" is more applicable to the word \"bartender\", while the words \"20 min\" are more pertinent to the word \"waiting.\" In this example, the attention mechanism is used to highlight the most influential context word with respect to an aspect word. In existing approaches, the magnitude of attention on a context word correlates to its relevance to the aspect word in predicting the output at each position of a sentence [9; 10]. These existing approaches model the attention weight based on explicit positional encodings [11; 12] or generate target specific sentence representations [5; 7] but there are still some outstanding limitations in these models as they overlook the notion that humans use attention filters to direct attention towards items deemed as important. The significance of attention filters is highlighted by psychologists, who define attention as a basic component of human cognitive biology which is limited in terms of capacity and duration [13; 14]. An everyday example showing that there is a limit to items humans can pay attention to is, one can not effectively listen to a telephone conversation if someone else in the room is simultaneously giving them complex instructions. Consequently selective towards stimuli [15] and this is supported by theories behind language reading comprehension. These theories illustrate that text elements are first processed at a minimal level and graded for significance, then extra attention is paid to elements in proportion to their significance [16; 17]. In this backdrop, the first problem with current aspect-level sentiment analysis models is paying attention to less significant words as they allocate attention weights to words without grading for significance. For example, it is common for a writer to use italics or bold to draw attention to important words that may otherwise be missed in a sentence.