# Technology Enhanced Learning Using Humanoid Robots
## 5 Experimental Results
### User Experience  
In this section we will show the results of the user experience related to the interaction with the robot. We demoed the functionalities of the robot for about 30 minutes and then asked 10 participants to use the robot for performing four tasks: Sentiment analysis, conversation with the robot, ask the robot to perform some action, and ask the robot to identify some object. We took advantage of this session to gather further feedback about new potential use cases. Each of them had to fill a two-parts survey about their experience. The first part included six open questions (the first four also included five levels of reply, very bad, bad, normal, good, and very good), whereas the second part was a standard System Usability Scale (SUS) ([[https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html](https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html)]([https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html](https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html)) (accessed on 25 January 2021)) questionnaire to assess the usability of the application. Here we summarize their answers to the open questions. Q1. How do you find the interaction with the robot? Five users considered it good to use and three of them found it very good. The last two suggested to improve the speech-to-text capabilities because the robot could not understand well their statements. We investigated why and it turned out that the problems derived from speech impediment (they were Italians that talked in English to the robot). Q2. How effectively did the robot support you in classifying positive and negative sentences? Five users stated that the robot had an extremely positive accuracy (very good), whereas three found some errors on the ability of recognizing the correct sentiment (normal). Q3. How effectively did the robot perform the action commands you gave? Nine users agreed on the same score (good) commenting that the robot always performed the action (from a list of possible actions) that was told to. One user (normal) commented that the robot had some problem identifying his commands. The problem was that the user employed some words not covered within the dictionary of the robot action ontology module and it was easy for us to add the missing terms. Q4. How effectively did the robot support you in identifying objects? Eight users gave a score of good commenting that the robot identified the correct object they showed to her cameras. Two users (normal) commented that the robot did not correctly identify the objects they showed. The problem was that they were too far from the robot when performing this action. We fixed this by letting the robot say when the object detection module was triggered, to maintain a certain distance from her. Q5. What are the main weaknesses of the resulting interaction? Users did not flag any particular weakness. One of them pointed out that weaknesses might appear when used extensively. Q6. Can you think of any additional features to be included in the robot interaction? The suggested features were: (1) To have the robot say better instructions when she was turned on and each time a certain module was triggered and (2) the ability of saving all the data pertaining to the interaction with the user. The SUS questionnaire confirmed a good opinion of the users, scoring 81.2/100, equivalent to an A grade and places our architecture in the 91% percentile rank (percentiles of SUS [[https://measuringu.com/interpret-sus-score/](https://measuringu.com/interpret-sus-score/)]([https://measuringu.com/interpret-sus-score/](https://measuringu.com/interpret-sus-score/)) (accessed on 25 January 2021)). All users felt very confident in playing with the proposed architecture (with an average score of 4.2 \(\pm\) 0.5) and thought that it was easy to use (4.2 \(\pm\) 0.6).