# Generating Robotic Speech Prosody for Human Robot Interaction: A Preliminary Study
## 3 Method
### Protocols
#### 3.3.2 Experiment B  
In Section 3.3.1, the contents of [PERSON] speech were only about the weather forecast and focused on delivering information to the users. Considering the applications of this study and the target users being elderly people, we applied realistic scenarios to the contents of the conversation by adding greetings, self-introduction, small talk, and non-verbal cues [35; 36] apart from the weather forecast, as shown in Figure 3.  
The interaction time is approximately 5 min, and, by adding longer pause between the sentences, [PERSON] speaks more slowly than in the experiment in Section 3.3.1. An emotion detection software called Mood mirror (Mood mirror was developed by Dr. [PERSON] of MIT Media Lab) was used to recognize if the participants felt emotions (anger, sadness, fear, joy). This software employed the validated toolkit, affectiva SDK, for recognizing the facial expressions of humans [37]. This toolkit is one of the most used toolkits that is a cross-platform to recognize multi-face expression in real-time by using the facial action coding system (FACS). It has been showing a high accuracy and reliable system for several application [38; 39; 40]. The length of time for each recognized emotions was calculated and analyzed to determine if the emotions presented in the robot's voice reflected on the emotional states of users. Note that the loudness was 80% instead of 100% because a number of participants commented the voice of robot was too loud considering the physical distance between them in the experiment of Section 3.3.1. However, this revision did not influence on the studies because the same number was used to all sessions (neutral voice session, positive and negative ones).  
Three participants (all males, aged 22) took part in interaction sessions in which [PERSON] spoke in three different voices (i.e., neutral, positive, and negative voices). In these sessions, [PERSON] started the conversation with small talk for ice-breaking and natural conversation. Each participant engaged in three sessions and was recorded by a small web-camera in front of the [PERSON]. The video data were used for analyzing the emotional states of the participants. The participants interacted with the [PERSON] robot for approximately 5 min, mainly by verbal communication, as follows:  
_Hello. My name is [PERSON]. Can you tell me your name? I am a humanoid robot. Have you ever seen me somewhere? Recently, there are some friends who started working at the hotel. There are some friends playing soccer. Don't you think that this figure looks like a Pepper robot? Actually, I was built by the same company with Pepper robot. [PERSON] is like a brother. [PERSON] is also the same robot as I am talking to everyone in various places, such as a company and nursing home. I also like chatting with everyone. That's it. I want to know more about you. Now, I will ask you a couple of questions, so please answer to my questions. Thank you. Well then, what's your favorite color? (pause a little bit) I like that color! In addition, I like blue. I think that is cool. Alright, I will ask another question. What's your favorite food? (pause a little bit) Alright. I have not eaten yet, but I'd like to eat hot pot. By the way, I saw the weather forecast for tomorrow on today's news. Let tell you. The weather tomorrow is cloudy, the probability of precipitation is 30 percent. If you go out, I think it would be better to have an umbrella. In addition, it seems to be tomorrow, so it is better to keep your cold measures steady. It is about time to have the questionnaire answered. Today, thank you for chatting with me. Then, please answer the questionnaire from now.  
Figure 3: The contents of interactions with NAO in experiment A (Section 3.3.1) and experiment B (Section 3.3.2).  
After interacting with [PERSON] in each session, participants answered a questionnaire with questions, such as \"Was it easy to understand?\" and \"Did you feel the emotions of robot?\"