Can Emotion Carriers Explain Automatic Sentiment Prediction? A Study on Personal Narratives (2022)

Abstract: Deep Neural Networks (DNN) models have achieved acceptable performance in sentiment prediction of written text. However, the output of these machine learning (ML) models cannot be natively interpreted. In this paper, we study how the sentiment polarity predictions by DNNs can be explained and compare them to humans’ explanations. We crowdsource a corpus of Personal Narratives and ask human judges to annotate them with polarity and select the corresponding token chunks - the Emotion Carriers (EC) - that convey narrators’ emotions in the text. The interpretations of ML neural models are carried out through Integrated Gradients method and we compare them with human annotators’ interpretations. The results of our comparative analysis indicate that while the ML model mostly focuses on the explicit appearance of emotions-laden words (e.g. happy, frustrated), the human annotator predominantly focuses the attention on the manifestation of emotions through ECs that denote events, persons, and objects which activate narrator’s emotional state.

## introduction neural data-driven models have managed to perform comparably well in various tasks related to natural language processing @xcite . nevertheless, the definition and the training processes of such models have made their decision non-natively interpretable. several studies and experiments have been conducted to address this issue and explain the decision outputs of such models in various tasks such as emotion prediction @xcite , question answering @xcite , the classification of linguistic styles @xcite , and lexicon-based sentiment prediction @xcite . sentiment analysis is a well-established field of research that aims to extract sentiment and its aspects in a written text. its performances have reached acceptable levels in different domains such as product reviews @xcite , movie reviews @xcite , social media @xcite , financial news @xcite , and personal narratives (pn) which are recollections of real-life events that are experienced by the narrator @xcite . recently, a deeper understanding of the expressed sentiment and emotion has gained growing research interest @xcite @xcite . these works focus on a more fine-grained analysis on the expressed sentiment/emotion by identifying the emotion carriers (entities or actions that explain, cause or carry the emotion). the concept of emotion carriers (ec) was first introduced by @xcite for german pns. in this genre of text, the identification of ecs may help in better understanding the emotional state of the narrator and what has caused distress @xcite . in this work, we address the problem of analyzing and comparing the text chunks used by machines and humans when predicting the sentiment polarity of text documents. for this study we have selected the personal narrative genre since it is rich with entities and relations which are sparsely distributed. we identify the tokens that contribute to the model's prediction according to their attributions given by integrated gradients @xcite , an explainable-ai technique, and compare them with the tokens tagged as ecs by the human annotator. our comparative analysis shows the human annotator identifies the tokens that explain an event or its participants as the carrier of emotions and sentiments, which clearly convey the activation of the emotional state in the narrator, even though they are not explicitly manifesting a sentiment. meanwhile, the dnn model bases its decision mostly on a limited set of tokens which belong to the category of emotion-laden words (see figure 1 for an example). we summarize our contribution as follows: ai explainability there have been several interesting works to address the unexplainability of neural architectures. danilevsky et al. (2020) conducted a survey study on explainable ai (xai) in natural language processing, summarizing the various xai methods used by researchers. bodria et al. (2020) proposed an attention model to investigate the words that contribute to the sentiment prediction, by adding an additional attention layer on top of the bert architecture to fuse the token embeddings in one vector used to compute the prediction. bacco et al. (2021) used the attention weights technique to extract summaries of reviews to explain the sentiment prediction of a transformer-based model, by using a simplified model with 2 layers and one attention head per layer. torres et al. (2021) designed a deep neural network with an interpretable decision process to recognize emotions from the electroencephalography (eeg) signals. while the approaches based on attention weights require a change in the architecture of the model, lime (local model-agnostic explanations) @xcite and the integrated gradients technique @xcite can be applied to any model without changing the architecture. using lime, @xcite extracted a sentiment lexicon used as a weak classifier to categorize unseen examples to augment the initial training set. @xcite used lime and hard-attention to extract spans of text that convey personal attacks. furthermore, @xcite used the integrated gradients to compare most relevant tokens for the human and the machine in predicting the linguistic style of a text. ## data collection & annotation we used an extended version of the dataset of pns from users receiving cognitive behavioural therapy to handle their distress more effectively, introduced previously by @xcite . each pn encompasses a real-life personal event that has activated the narrator's emotional state, the participants of the event as well as the details about the user's thought and emotions. during two periods of 3 months, we collected 481 personal narratives written by 45 italian speaker users, with the average length of 51 tokens per narrative and overall dictionary size of 5875 tokens. ## prediction decision explainability we investigate the explainability of the automatic sentiment prediction by comparing the tokens influencing the prediction with those selected by the human judge as ecs. in order to detect the tokens crucial to the model's prediction, we use the attribution assigned to each token by the integrated gradients @xcite technique. integrated gradients (intgrad) is an attribution method for explainable ai which builds on top of the classic backward gradient analysis. given our sentiment prediction model f (f u ), where fu is the functional unit f @xmath0, w 2 , .., w n } and w @xmath1, the backward gradient is given by: ## conclusion in this work we studied whether the sentiment prediction decision of dnn models can be explained by emotion carriers, spans of text that convey and carry emotions. we have focused our study on personal narratives which encompass real-life events and experiences that activate the emotional state of the narrator. we have collected a dataset of personal narratives and conducted an annotation task for sentiment polarity and emotion carrier selection at the functional unit for each narrative. we have then developed a sentiment prediction model based on alberto architecture @xcite . we have investigated whether the decision of the model is based on the emotion carriers that the human annotator selected to explain the sentiment of the text. furthermore, we have studied the impact of the emotion carriers on the confidence score of the polarity prediction model. our analysis has shown that the human annotators tend to focus on manifestation of emotions through words describing actions and events that have activated the emotional state of the narrator. however, the model bases its decision on explicit representations of sentiment such as emotion-laden words.