# Generating Robotic Speech Prosody for Human Robot Interaction: A Preliminary Study
## 3 Method
### Materials  
A small humanoid robot NAO was used in the experiment (see Figure 1). The robot has a main software for controlling, called NAOqi. It is a programming framework and capable of commuicating between multi modules. The _ALTextToSpeech_ module was used to command the NAO to speak by using the text-to-speech engine, and the value for pitch in NAOqi was coded as _vct=value_. NAO uses the Altalk (This software is powered by AI Japan: [[https://www.ai-j.jp/english](https://www.ai-j.jp/english)]([https://www.ai-j.jp/english](https://www.ai-j.jp/english)).) speech synthesizer for Japanese language and has values ranging from 50 to 200. To transform the values into pitch Hertz values, the Equation (1) was used. Here, \(P_{value}\) is the value of NAO pitch in NAOqi, and \(P_{pitch}\) is the pitch in Hertz. To define Equation (1), [PERSON] speaking was recorded and converted using WaveSurfer (WaveSurfer was developed at KTH in Stockholm, Sweden). \[P_{pitch}=2.864P_{value}+4.6727. \tag{1}\]  
The pitch values were taken referring to the values of _FO_ and _mel scale_ from the study of [PERSON] and [PERSON] [29] as Equation (2) and were applied to three different voices: neutral, joy, and sadness, as in Table 1.  
\[mel=(1000/log2)log(f/1000+1). \tag{2}\]  
The robotic voices in both experiments were validated. We have extracted the features by using openSMILE [30]. A part of IS09-emotion.conf [31] was used for the feature \begin{table}
\begin{tabular}{c c c c} \hline \hline  & **Neutral** & **Positive (Joy)** & **Negative (Sadness)** \\ \hline Pitch (Hz) & 292 & 314 & 251 \\ Timing (\%) & 100 & 100 & 100 \\ Loudness (\%) & 100 & 100 & 100 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The pitch, timing, and loudness values in each session. Figure 1: The experiments of Section 3.3.1 (experiment A) and Section 3.3.2 (experiment B). The robot mainly talks, and human subjects try understanding what it says in experiment A, while robot and human have a conversation in the other experiment. The tools for measuring the emotional states are different in the two experiments: questionnaire in experiment A and autonomous recognition in experiment B.  
set. The original IS09-emotion.conf has 384 features that contained 32 speech and music-related features (Signal energy, Loudness, Mel-/Bark-/Octave-spectra, MFCC, PLP-CC, Pitch, Voice quality, Formants, LPC, Line Spectral Pairs, Spectral Shape descriptors), and 12 statistical functionals (Means, Extremes, Moments, Segments, Samples, Peaks, Linear and quadratic regression, Percentiles, Durations, Onsets, DCT coefficients, Zero-crossings, Modulation spectrum), but, here, we used the most important 11 features: \(\triangle\)RMSenergy-max, \(\triangle\) MFCC[9]-linregc2, RMSenergy-linregc2, \(\triangle\)voiceProb-maxPos, MFCC[1]-linregc2, \(\triangle\) MFCC[12]-linregc2, \(\triangle\) MFCC[10]-max, \(\triangle\) MFCC[4]-linregc1, \(\triangle\) MFCC[10]-min, ZCR-amen, and \(\triangle\) MFCC[8]-linregc1. These features were extracted through the FSS (Forward Stepwise Selection) method used by [PERSON] el al. [32]. After extracting the features, we trained the robotic voice dataset by using the Weka software, which is a machine learning workbench [33]. For classification, we used sequential minimal optimization (SMO) [34]. k-fold cross validation was used for the validation method in Weka. The value k in the dataset used in the experiment A was 13, and that for the dataset used in the experiment B was 10. In both datasets (datasets from the experiment A and B), the distribution is highly skewed, which means that the neutral emotion resulted in higher recognition. As shown in Figure 2, when the robot spoke about the weather forecast with some natural small talk, this provided a better prediction result (Figure 2B) than when the robot only talked about the weather forecast(Figure 2A). In case of Negative prediction, both speeches moderately same results.