Sentiment Forecasting in Dialog (2020)

Abstract: Sentiment forecasting in dialog aims to predict the polarity of next utterance to come, and can help speakers revise their utterances in sentimental utterances generation. However, the polarity of next utterance is normally hard to predict, due to the lack of content of next utterance (yet to come). In this study, we propose a Neural Sentiment Forecasting (NSF) model to address inherent challenges. In particular, we employ a neural simulation model to simulate the next utterance based on the context (previous utterances encountered). Moreover, we employ a sequence influence model to learn both pair-wise and seq-wise influence. Empirical studies illustrate the importance of proposed sentiment forecasting task, and justify the effectiveness of our NSF model over several strong baselines.

## introduction developing intelligent chatbots is of great appealing to both the industry and the academics. however it is challenging to build up such an intelligent chatbot which involves a series of high-level natural language processing techniques, such as sentiment analysis of utterances in dialog. previous studies on sentiment classification focus on determining polarity (positive or negative) in a single document @xcite . in comparison, only few studies focus on determining polarity of utterances in dialog @xcite . however, all of these studies focus on determining the polarity of existing utterances. it may be more important to predict the polarity of next utterance yet to come. given the example in figure 1 , although b expresses a positive sentiment in second utterance, a still shows a negative sentiment in his response. in this case, if b know that a would be very upset after his first utterance, he may revise his utterance to let a feel more comfortable. hence, predicting the polarity of the next utterance can help a speaker to improve their utterances, which is important in automatic dialogue, such as customer service. to the above purpose, we propose a new task, calls sentiment forecasting in dialog, which aims to predict the polarity of next utterance yet to come. in this paper, we focus on tackling two inherent challenges, one is how to simulate next utterance, and the other is how to learn the influence from context towards next utterance. the motivations behind are that, since next utterance is yet to come, it would be helpful if we can simulate next utterance from the context. moreover, the polarity of next utterance can be much influenced by the context, it is necessary to consider the influence of the context for next utterance. in this paper, we propose a neural sentiment forecasting (nsf) model to address above challenges. in particular, a neural simulation model is employed to simulate next utterance based on existed utterances. in addition, a hierarchical influence model is employed to learn the influence from existed utterances by considering both pair-wise and sequence-wise influence between existed utterance sequence and next utterance. empirical studies illustrate the importance of proposed sentiment forecasting task in dialog, and also show the effectiveness of our proposed nsf model over several strong baselines. a: john, i've asked you not to smoke in here! i don't want to see you smoking in my office again. b: i'm sorry, ms. fairbanks. i won't let it happen again. a: negative (that's what you said the last time! if you want to smoke, you'll have to use your break time and go outside! ) ## related work our task is related to document-level sentiment classification @xcite for various neural network models have been used, including convolutional neural networks @xcite , recursive neural network @xcite and recurrent neural network @xcite . more recently, researches focus on aspect level sentiment analysis @xcite @xcite and user-based or product-based sentiment classification @xcite @xcite . different from document-level sentiment classification, sentiment classification in dialog aims to detect polarity of utterances in dialog by considering the influence of the whole dialog. as a prime study, @xcite stated that sentiment plays a pivotal role in conversations. @xcite studied the impact of sentimental text on the customer's perception of the service agent. on the basic, @xcite used svm to classify sentiment in customer support dialogs by integrating both text based turn and dialog features. recently, @xcite proposed a multi-task hierarchical recurrent network to classify sentiment and dialog act jointly. @xcite proposed a recurrent neural networks to track of the individual states throughout the dialog and employed this information for sentiment classification. different from previous studies which focus on detecting the polarity of existing utterances, we propose a novel and important task to forecast polarity of next utterance yet to come. ## neural sentiment forecasting modeling as illustrated in figure 1 , given a existed utterance sequence {u 1 , u 2 , ..., u n-1 } in a dialog d, we aim to predict the polarity (positive, negative, or neutral) of next utterance u n . note that, next utterance u n does not exist in dialog, and we do not know the polarity of utterances in existed dialog sequence. figure 2 shows the overview of proposed neural sentiment forecasting (nsf) model. we first learn the representation of each utterance u i . secondly, we employ a neural simulation model to simulate the representation of next utterance ûn based on the existed utterance representations. thirdly, we employ a hierarchical sequence influence model to learn the influence from existed utterance sequence to the next utterance ûn . finally, we predict polarity ŷ of next utterance ûn based on the simulation model and the influence from existed utterances. in the following, we discuss these issues one by one. ## analysis and discussion in this section, we give some statistic and analysis to discuss our motivations and illustrate the importance of propose sentiment forecasting task. ## conclusion in this paper, we propose a novel but important task, called sentiment forecasting in dialog, which aims to forecast the polarity of next utterance to come. there are two challenges in this task, one is that how to simulate the next utterance for predicting its' polarity, and another is how to learn the influence of existing utterance sequence for forecasting next utterance's polarity. in this paper, we propose a neural sentiment forecasting (nsf) model to address above challenges. in particular, a neural simulation model is used to simulate the next utterance based on existed utterances sequence. in addition, a hierarchical influence model is used to learn the influence of existing utterance by considering both pair-wise and sequencewise influence. empirical studies illustrate the importance of our proposed sentiment forecasting task, and show the effectiveness of our proposed nsf model over several strong baselines.