Sentiment Interpretable Logic Tensor Network for Aspect-Term Sentiment Analysis (2022)

Abstract: Aspect-term sentiment analysis (ATSA) is an important task that aims to infer the sentiment towards the given aspect-terms. It is often required in the industry that ATSA should be performed with interpretability, computational efficiency and high accuracy. However, such an ATSA method has not yet been developed. This study aims to develop an ATSA method that fulfills all these requirements. To achieve the goal, we propose a novel Sentiment Interpretable Logic Tensor Network (SILTN). SILTN is interpretable because it is a neurosymbolic formalism and a computational model that supports learning and reasoning about data with a differentiable first-order logic language (FOL). To realize SILTN with high inferring accuracy, we propose a novel learning strategy called the two-stage syntax knowledge distillation (TSynKD). Using widely used datasets, we experimentally demonstrate that the proposed TSynKD is effective for improving the accuracy of SILTN, and the SILTN has both high interpretability and computational efficiency.

## introduction 1.motivation aspect-term sentiment analysis (atsa) is a finegrained task in sentiment analysis, which aims is to recognize the sentiment polarity of the given aspect-term in a sentence @xcite . early research for atsa was developed based on manually extracted features. for example, @xcite designed hand-crafted dependency rules to obtain aspect-related words, which are then fed into the machine learning methods to infer the sentiment polarity. with the development of deep learning, deep neural networks (dnns) have dominated the study. generally, the performance of dnns is superior to traditional machine learning methods when the labeled training data is sufficient. more recently, there are two classes of methods that have received attention: (i) syntax-aware neural network (sann), which integrates syntax knowledge into attention-based neural network that increases the predictor's performance and interpretability @xcite @xcite ; (ii) large pre-trained language model (plm) for atsa (e.g., bert @xcite @xcite ), which learns knowledge from largescale corpus and stably exceeds the other baseline by a significant margin. despite the effectiveness of prior work, atsa in real-world remains a challenge for several reasons. first, dnns usually perform as a "black box", because they cannot explicitly explain the process of the analysis; therefore, they cannot be applied in cases where explanations are required. second, the performance of sanns relies on the intricate knowledge integration mechanism, which introduces more trainable parameters and brings extra computational costs. furthermore, based on our empirical observation, the sann achieves merely limited performance improvements on most attention-based models (see methods 9-12 in ## conclusion in this paper, we propose a novel sentiment interpretable logic tensor network (siltn). siltn is interpretable because it is constructed followed by first-order logic language (fol). the sentiment inferring process can be decomposed into different grounding, which is constructed by the simple neural network; therefore, siltn is computationally efficient. to achieve high inferring accuracy, we propose a two-stage syntax knowledge distillation (tsynkd) strategy. tsynkd consists of two distillation stages with three networks: bert, as-dgcn and siltn. the first distill stage refers to the output distillation, which makes the bert output logits as the training objective of asdgcn. in asdgcn, we develop a novel attention structure to learn the aspect-specific dependency knowledge through output distillation. the second stage is feature distillation, which allows a siltn to learn from asdgcn's intermediate feature representations. extensive experiments have been conducted on 5 real-world datasets. the experimental results show that the proposed siltn with the tsynkd strategy significantly outperforms the conventional attention-based methods, and achieve compatible results compared with the state-of-theart bert-based methods for aspect-term sentiment analysis.