La sentiment analysis come strumento di studio del parlato emozionale?
Authors: Paolo Mairano, Enrico Zovato, Vito Quinci (University of Lille, Università di Torino, 2018)
Topic: Sentiment, Emotion, Irony, Hate

Abstract
Abundant literature has shown that English. Abundant literature has shown that emotional speech is characterized by various acoustic cues. However, most studies focused on sentences produced by actors, disregarding ecologically elicited speech due to difficulties in finding suitable emotional data. In this contribution we explore the possibility of using sentiment analysis for the selection of emotional chunks from speech corpora. We used the LibriSpeech corpus and extracted sentiment analysis scores at word and sentence levels, as well as several acoustic and spectral parameters of emotional voice. The analysis of the relation between textual and acoustic indices revealed significant but small effects. This suggests that these two levels tend to be fairly independent, making it improper to use sentiment analysis for the selection of acoustically emotional speech.

Introduction
The expression of emotions can occur through different components at various linguistic levels (Reilly & Seibert, 2003): lexical (modal verbs, intensifiers, attenuators, or evaluative elements), syntactic (e.g., relative clauses can comment on actions and behaviors), acoustic (prosody, voice quality), and paralinguistic (facial expressions, gestures). Traditional frameworks for the analysis of emotions are based on categories (Ekman, 2000) or on dimensions (Russell, 1980). The former distinguishes various emotional states (anger, joy, fear, sadness, etc.), while the latter tends to define emotions as coordinates in a multidimensional space, where each dimension represents a property of an emotional state. Among the many existing frameworks, Russell (1980) hypothesized two dimensions: valence (positive vs. negative) and arousal (high vs. low). The classification of emotional states through linguistic cues has proven to be a difficult task for both categorical and dimensional frameworks, and the interaction of the various linguistic levels further complicates the situation: it is still unclear whether the lexical/syntactic component should be considered dependent or complementary to the acoustic component. Despite these challenges, many studies have analyzed emotional speech with the goal of identifying specific acoustic correlates of various emotional states. Some studies have shown that systematic variations in the fundamental frequency (both in terms of pitch range and average pitch) accompany realizations of speech with positive valence (Burkhardt & Sendlmeier, 2000). However, other prosodic parameters also seem to play an important role in the communication of emotions: effects of intensity and speaking rate have been observed (Johnstone & Scherer, 2000); in fact, various acoustic measures (standard deviation of the fundamental frequency, average energy, duration of segments, spectral dropoff, etc.) have been used to predict the judgments of native speakers (Banse & Scherer, 1996), and various other parameters have been used in other studies (see Schröder et al., 2001, and Audibert, Aubergé & Rilliard, 2005). However, one of the limitations of these studies concerns the reliability of the data: given the difficulty in eliciting controlled emotional speech, many studies use recordings of acted speech, which often appears stereotyped or exaggerated (Scherer, 2003). In this contribution, we investigate whether sentiment analysis (henceforth: SA) can be helpful in this regard. SA, the study of people's opinions, feelings, and reviews in textual form (Liu, 2003), is a rapidly growing NLP field, also thanks to a wide range of applications, such as email classification (Mohammad & Yang, 2011), novels (Mohammad, 2011), movie reviews (Sadikov, 2009), and reviews of purchased items or services (McGlohon, Glance & Reiter, 2010). SA systems range from relatively simple rule-based methods to advanced deep learning techniques—see Liu (2012) for an overview. In this study, we examine the relationship between SA values and the acoustic characteristics of naturally elicited read speech, extrapolated from audiobooks. The ultimate goal is to extend the analysis to spontaneous speech data; however, given the numerous issues that this type of speech entails, we preferred to start with read speech data in which emotions were not explicitly elicited. To measure the degree of emotion expressed by the audiobook text, SentiWordNet (Baccianella, Esuli & Sebastiani, 2010) and Vader (Gilbert & Hutto, 2014) were used, which operate mainly at the lexical level. At the acoustic level, we extracted various (mostly prosodic) indices described in the literature. A similar analysis to this one, studying the correlation between SA and acoustic parameters, was conducted by Charfuelan & Schröder (2012) on data from a single speaker and a single audiobook. Here, we extend the analysis to 251 audiobooks read by different speakers, hoping that the results will have both theoretical relevance (study of the interaction between lexical and acoustic levels in emotional speech) and practical application (using SA for selecting non-acted emotional speech).

Conclusion
The correlation between lexical and acoustic indices of emotional read speech seems to be significant, but limited in scope, both at the word and sentence levels. F0 indices seem to be influenced by the valence of the sentence and the word, but the explained variance remains low. These results confirm and extend what was reported by Charfuelan & Schröder (2012) on data from a single audiobook, where moderate correlations for F0 and energy indices were observed. The data show a great deal of inter-speaker variability: it is clear that speakers use different acoustic indices to express emotional states. Moreover, a limitation of our analysis lies in the use (inevitable, given the large amount of data analyzed) of automatic transcriptions and annotations, whose errors undoubtedly introduce some noise into the data, reducing the observable relationships among the various variables studied. Finally, the absence of punctuation in the LibriSpeech corpus makes it impossible (or very complex) to differentiate between direct and indirect speech, where one might expect more pronounced emotional speech. In the future, such hypotheses can be tested on more recent corpora constructed with more specific and suitable purposes, such as SynPaFlex (Sini et al., 2008). In conclusion, we return to the theme of the interaction between various linguistic levels for the expression of emotions in speech. The results of our study suggest that the different linguistic levels analyzed (lexical and acoustic) are relatively independent of each other in terms of emotional expression. This means that, for a given sentence, speakers tend to rely on only one of the two levels analyzed for expressing the emotional state. This may be particularly true for read speech, in which the speaker is not directly involved, especially in the case of an audiobook narrator. Thus, using SA to study emotional speech does not appear entirely appropriate for selecting emotionally marked material, as it relies on the assumption that lexical and acoustic indices of emotionality go hand in hand and tend to co-occur. However, the correlation between lexical and acoustic variables for other types of speech remains to be explored, particularly for spontaneous speech—in which speakers are more directly involved with the semantic content.