## 4 Results
### Impression Estimation from Video and Prosody Correlation
Figure 7: Histograms of \(\delta_{v}\) (**top left**) and \(\delta_{a}\) (**top right**) and \(||\overrightarrow{\delta_{E}}||\) (**bottom**) for every utterance generated in the “Talk to Kotaro” experiment. metric [55], which measures the distance between a point and a distribution, to determine which emotion changes were outliers, with a Mahalanobis distance threshold of 3. From the emotional estimates obtained from each frame of every interaction with [PERSON], we have seen that the maximum valence shown was of 0.831, while the lowest valence was of \(-0.823\), while the average was \(-0.172\), with a standard deviation of 0.291. For arousal, the highest estimate was of 0.815 and the lowest estimate was of \(-0.061\). The average arousal was of 0.176 with a standard deviation of 0.134. Another metric that can be explored, for the video of a volunteer listening to a given GS utterance, is averaging the emotion estimates for each frame. This way, we can obtain an overall feeling of the emotion elicited by the interaction. Calculating such a metric for every video sample and averaging the average emotion, we obtained an average of all average emotion estimates \(E_{avg,avg}=(-0.248,0.161)\), with standard deviations of 0.293 for valence and 0.137 for arousal. The lowest and highest valence averages for each interaction were \(-0.693\) and 0.831, respectively. The lowest and highest arousal averages were \(-0.051\) and 0.767, respectively. Out of all 734 video samples, 130 video samples had a non-negative average valence and 695 had non-negative average arousal; and 37 video samples had both non-negative valence and arousal averages. Such results show that the majority of non-Yulean gibberish speech did generate moderately negative feelings on listeners, but still, few interactions had a positive average valence. Considering participants that had more than a single exchange with [PERSON], it is possible to analyze how their emotional state changed along the overall interaction. Out of the 37 participants, 33 had multiple exchanges and 7 had interactions across multiple days. We used linear regression to detect the tendency of the evolution of the average valence of each interaction within the the same day (a participation session) and across multiple days, for the volunteers who participate multiple days (multiple sessions). Out of the 65 participation sessions, volunteers had their average valence decrease in 30 sessions, while  
Figure 8: Every emotion change \(\overrightarrow{\delta_{E}}\) in the data set represented in the impression space. in the remaining 35, the average valence of the interactions increased. For arousal, out of the 65 sessions, only in 28 could we see the valence increasing. For volunteers who participated in multiple sessions across distinct days, the average valence decreased across different sessions for four volunteers, while it increased only for three of them. Arousal, on the other hand, increased only for two volunteers across multiple sessions, decreasing for the remaining five. The results of such analysis can be seen in Figure 9, where the average valences of the different sessions are represented in different colors and the line resulting from the linear regression for each session has the same color as the points. The longest line represent the changes across multiple sessions. Some volunteers had their emotions improve, while other had their emotional state deteriorate while listening to the GS utterances. For the top left image, we can see a result were the line fits the data very well, but for most volunteers, that is not the case, showing that continued interactions are not a good predictor of how well listeners will react to the different. Especially when looking at the bottom right graph of Figure 9, where we can see that in the first session there is a tendency of improving valence, but in the next session, there is a strong decrease in valence as the volunteer listened to the GS utterances.