Octa: Omissions and Conflicts in Target-Aspect Sentiment Analysis (2020)

Abstract: Sentiments in opinionated text are often determined by both aspects and target words (or targets). We observe that targets and aspects interrelate in subtle ways, often yielding conflicting sentiments. Thus, a naive aggregation of sentiments from aspects and targets treated separately, as in existing sentiment analysis models, impairs performance. We propose Octa, an approach that jointly considers aspects and targets when inferring sentiments. To capture and quantify relationships between targets and context words, Octa uses a selective self-attention mechanism that handles implicit or missing targets. Specifically, Octa involves two layers of attention mechanisms for, respectively, selective attention between targets and context words and attention over words based on aspects. On benchmark datasets, Octa outperforms leading models by a large margin, yielding (absolute) gains in accuracy of 1.6% to 4.3%.

## introduction people share their opinions about almost anything: tourist attractions, restaurants, car dealerships, and products. such opinionated texts do not merely help people make decisions in their daily life, but also help businesses measure consumer satisfaction to improve their offerings. sentiment analysis involves many aspects of natural language processing, e.g., negation handling @xcite , entity recognition @xcite , topic modeling @xcite . importantly, opinionated texts often convey conflicting sentiments. distinct sentiments may refer to distinct aspects of the domain in questione.g., food quality of a restaurant or battery life of a smartphone. these predefined domain aspects may or may not appear in the texts. aspect-based sentiment analysis (absa) approaches @xcite @xcite predict sentiments from text about a given aspect. and, target-based sentiment analysis (tbsa) approaches @xcite @xcite @xcite predict sentiments of targets that appear in an opinionated text. targets are usually entities in a review: e.g., a dish for a restaurant and a salesperson for a car dealership. we posit that aspects and targets provide subtle, sometimes contradictory, information about sentiment and should therefore be modeled, not in isolation, but jointly. considering them separately, as absa and tbsa approaches do, impairs performance. take this review sentence from semeval-15 as an example: conflicting sentiments on aspect we both had the filet, very good, didn't much like the frites that came with. if we ask about aspect food#quality, by disregarding targets during training, absa models fail to address the contradiction in sentiment about filet and frites, as do tbsa models, which focus on targets and disregard aspects. in the following review sentence from semeval-16, the target fish is associated with opposite sentiments: positive for food#quality and negative for food#style options. ## problem definition the input of our sentiment analysis task is a sequence of words, with an aspect, or a target, or both. our goal is to identify the sentiment polarity associated with the aspect and the target. formally, octa has three inputs, the remaining words that are not part of the target are context words @xmath0, . . . , c n -m }. ## octa model overview figure 1 shows the octa architecture. to infer the sentiment for an aspect and a target composed of words from the sequence, first, octa uses bert to generate word embeddings. second, octa uses a selective attention mechanism to compute context word and target attention weights and applies them to word embeddings to generate targeted contextual embeddings. third, octa constructs aspect embeddings and uses the embeddings to compute aspect attention over target and context words. octa uses a multihead architecture to learn attention in diverse embedding subspaces. it fuses and normalizes embeddings from each head and uses a linear classification layer with a softmax activation for sentiment classification. to introduce nonlinearity, octa uses feed-forward networks (shown in grey), each comprising two fully connected layers followed by a nonlinear activation. ## related work sentiment analysis has received substantial attention over the last few years. we highlight here only the works most relevant to octa. ## conclusion the main innovation of octa is to jointly consider aspects and targets. it uses selective attention to model the relationships between target and context words, and aspects to attend to targeted contexts to predict sentiments. users can "query" octa about sentiment of a particular aspect or target, or both. our evaluation shows that octa outperforms state-of-the-art models on semeval, sentihood, and conflicting sentiment datasets. our ablation study shows that jointly modeling aspects and targets with selective attention is superior to selective attention only, aspect attention only, and aspect with self-attention. in natural language processing (emnlp), pages 4560-4570, hong kong. zhe zhang and munindar singh. 2018. limbic: author-based sentiment aspect modeling regularized with word embeddings and discourse relations. in proceedings of the 23 rd conference on empirical methods in natural language processing (emnlp), pages 3412-3422, brussels. zhe zhang and munindar singh. 2019. leveraging structural and semantic correspondence for attributeoriented aspect sentiment discovery. in proceedings of the 24 th conference on empirical methods in natural language processing (emnlp), hong kong. xiaodan zhu, hongyu guo, saif mohammad, and svetlana kiritchenko. 2014. an empirical study on the effect of negation words on sentiment. in proceedings of the 52 nd annual meeting of the association for computational linguistics (acl), pages 304-313, baltimore.