Interpreting Sentiment Composition with Latent Semantic Tree (2023)

Abstract: As the key to sentiment analysis, sentiment composition considers the classification of a constituent via classifications of its contained sub-constituents and rules operated on them. Such compositionality has been widely studied previously in the form of hierarchical trees including untagged and sentiment ones, which are intrinsically suboptimal in our view. To address this, we propose semantic tree, a new tree form capable of interpreting the sentiment composition in a principled way. Semantic tree is a derivation of a context-free grammar (CFG) describing the specific composition rules on difference semantic roles, which is designed carefully following previous linguistic conclusions. However, semantic tree is a latent variable since there is no its annotation in regular datasets. Thus, in our method, it is marginalized out via inside algorithm and learned to optimize the classification performance. Quantitative and qualitative results demonstrate that our method not only achieves better or competitive results compared to baselines in the setting of regular and domain adaptation classification, and also generates plausible tree explanations.

## introduction sentiment classification is a task to determine the sentiment polarity of a sentence @xcite . current researches on this task are gradually shifting from improving model performance to interpretability. as the most known stream, feature-based explanation tries to figure out which input feature, say word, has the most influence on the prediction, in the form of the salience score or rationale, and in both self and post-hoc settings @xcite @xcite @xcite . however, this task requires sentiment composition 1 data and code implementation is available at https:// github.com/changmenseng/semantic_tree . figure 1 : different tree structures for explanining sentiment composition, where semantic tree can explain the sentiment composition in the inverted-v structure, as shown in the box of (c). @xcite , which is beyond the ability of these feature-based explanations. to be concrete, sentiment composition considers the classification of a constituent via 1) classifications of its contained sub-constituents and 2) rules operated on them @xcite , as shown in figure 1(c ). thus, the classification of a sentence is decomposed into hierarchical sentiment compositions of its sub-constituents. such compositionality has been widely studied previously in the form of hierarchical trees including untagged tree and sentiment tree, as shown in figure 1 . untagged tree is usually modeled as a latent variable and learned via the task objective @xcite @xcite @xcite . @xcite is adopted to encode the sentence following the hierarchy for the final prediction. however, untagged tree is limited because it can only explain the hierarchy but not give labels on all nodes. sentiment tree takes a further step that every node within has a polarity score or label. as the most representative example, @xcite creates stanford sentiment treebank (sst) that has senti-ment tree annotation. sentiment tree also appears as a post-hoc explanation giving hierarchical attribution scores @xcite . however, in fact, not every constituent is sentimental, some of which are somewhat more functional. for example, while a negator "not" is sentimentally neural, it can functionally flip the sentiment of a constituent. sentiment labels are therefore not sufficient to explain such phenomenon. to overcome those defects, we propose semantic tree, a new tree form capable of explicitly and principally interpreting the sentiment composition. in the semantic tree, each node is assigned a label in semantic labels including sentimental and functional ones, and each local inverted-v structure reveals the rule composing adjacent constituents, as shown in figure 1(c ). inspired by @xcite , formally, the semantic tree is a derivation of a context-free grammar (cfg) @xcite defined by non-terminal symbols (semantic labels), terminal symbols (word vocabulary), rules, and root symbols (positive and negative). the challenge of designing such grammar lies in designing semantic labels and rules, which requires linguistic knowledge of sentiment composition. to address this, we follow previous work about sentiment composition @xcite @xcite to carefully design 11 semantic labels and 62 rules. we believe the grammar could cover most cases in sentiment analysis, as shown in we aim to learn a model capable of extracting the semantic tree using data consisting of only sentence-label pairs, which is challenging because the semantic tree is latent without full annotation. to address this, we first build a semantic tree parser, and then marginalize out the semantic tree to induce a sentiment classifier to conduct supervised training on such data. fortunately, this marginalization over the exponential tree space is computationally tractable resorting to the inside algorithm @xcite . this process could be abstracted as a module, namely sentiment composition module (scm), which computes the compatibility of a prediction in the view of sentiment composition but not only pattern recognition. accompanying an arbitrary neural text encoder with the proposed scm, we can build a self-explanatory model that can not only predict the sentiment label but also generate a semantic tree as the explanation. to learn more plausible semantic trees, we further propose two extra objectives to guide the preterminals in the semantic tree, and to make the tree structure more syntactically meaningful. we conduct experiments on three datasets including mr @xcite , sst2 @xcite and amazon @xcite in the setting of regular and cross-domain classification. quantitative and qualitative results demonstrate that our method not only achieves better or competitive results compared to baselines, and also generates plausible tree explanations. ## experiments in this section, we conduct experiments to illustrate that the proposed scm module is able to improve the accuracy performance. ## related works sentiment composition is one of the key to sentiment analysis, which considers the semantic of a constituent from both recognition and composition views @xcite . that is, it decomposes the classification of a sentence into a hierarchical tree structure explicitly showing how the polarity of the sentence come from the composition of its subconstituents. early works are mainly based on manual rules and semantic lexicon that is constructed either manually @xcite or automatically @xcite . nowadays, represented via different forms of tree, sentiment composition is often learned explicitly or implicitly in the endto-end learning manner of neural network models. common tree forms include untagged tree and sentiment tree, while the learning paradigm is also varied in literature. to be concrete, untagged tree can either be directly obtained from the external syntactic parser @xcite @xcite , or serve as a latent variable learned implicitly @xcite @xcite @xcite . compared to the untagged one, sentiment tree offers more information about sentiment polarity of each constituent in the tree. as the most representative resource in this form, sst @xcite formalizes sentiment composition as a parsing task, motivating lots of works to learn the tree supervisedly @xcite @xcite . sentiment tree is also a popular explanation form for post-hoc interprebility since it can provide hierahical attribution scores @xcite . while both existing forms are useful, they are suboptimal due to their in-ability to explicitly interpret sentiment composition, which our proposed semantic tree fills this gap. ## conclusions in this paper, we present semantic tree to explicitly interpret sentiment compositions in sentiment classification. we carefully design a grammar under each compositions from the linguistic inspiration, and learn to extract semantic tree explanations without full annotations. quantitative and qualitative results demonstrate that our method is effective and can generate plausible tree explanations.