## 5 Experimental Setup
### Results
#### 5.3.1 Qualitative Evaluation: SASSI Questionnaire
Regarding habitability, results obtained more average scores than the rest of the evaluation items. In this case, although they were given instructions about the task and interactions, participants were not sure about what to say to the system not because it was not clear, but because they were afraid the system would fail if they were _too natural_ in their interactions. Finally, scores for **speed** were different between use cases. In the guide use case, results for this aspect were more variate, and for the bin-picking use case were more constant. Since the complexity of the guide use case was higher than in the bin-picking use case, in some cases KIDE4I needed more time to process user commands (_vid._ Section 5.3.3). However, users considered the system to be fast in general terms. On the other hand, the **most appreciated** aspects among users were **response accuracy**, **likeability**, **cognitive demand** and ** verbosity**, with more consensus between participants. The most relevant results are the ones obtained for **cognitive demand**, as this aspect refers to one of the main objectives of KIDE4I and validates the easiness of use of the system, which makes it highly suitable in industrial scenarios. All in all, the system has been considered to be **accurate**, **useful**, **efficient**, **not demanding**, **flexible**, **fast** and that, in general, **the amount of information provided is correct**. For additional questions for the bin-picking use case--in Figure 8--they have also been evaluated very positively, and it is specially relevant that the feeling of security by using this system is very high, reaching nearly a perfect score among users. Furthermore, these results show that most users consider that the system would be a plus in their productivity. Table 5 shows the scores for the different evaluated aspects in the two user studies, averaged over the three age groups stated in Tables 3 and 4 (24-34, 35-44, 45-54). The table shows that the scores given by participants of different ages is roughly the same, with relatively small standard deviation values. However, and despite the small standard deviation values, these figures show some tendencies according to age. In general, the 24-34 age group assessed more positively the evaluated aspects, except for verbosity and annoyance, in which the results show the opposite, being the 45-54 group the most satisfied. This can be associated to the fact that older people tend to appreciate to be provided a good amount of information, whereas younger people prefer a fast execution rather than information. completed**. In the case of completed tasks, dialogues were classified between _fully completed_ or _partially completed_, depending on whether the user had to reformulate the query at some point of the dialogue (see Example (8)). 1. **Initial--not correctly interpreted:** \"Tengo sed\"
2. **R Reformulation--correctly interpreted:** \"Quiero beber\"
3. **R Reformulation--correctly interpreted:** \"Quiero beber\"
4. **I want to drink\"  
Table 6 shows the percentage of the dialogue completion rates for each use case. As it can be observed, the results are very positive, where the successful completion rates reach an 84.34% and 82.67%, respectively. In the guide use case, an 8.43% of these dialogues were classified as partially completed. Regarding **dialogue steps**, Table 7 shows the average number of steps required to complete the dialogue. It is important to keep in mind that each dialogue includes 1 or 2 steps that are included by default in each adaptation as part of their design: in the KIDE4 Guide case, the system initiates the dialogue by presenting itself and, in both KIDE4 Guide and KIDE4 BinPicking, when an action is obtained, the system asks the user for confirmation. Come as it may, the number of dialogues required to achieve the user's goal is positive enough to determine that KIDE4I allows an agile interaction between the user and the system.