# Measuring Quality of Service in a Robotized Comprehensive Geriatric Assessment Scenario
## 5 Experimental Evaluation
### Preliminary Experiments
Barthel #4, #5 and #6 (fifth-seventh columns, Table 2) incorporate new situations, such as questions not answered (QNA), patients suddenly undetected (NPD), calls to the doctor (CDR) and patients that pause the test (PR). We also emulate the scenario where patients sometimes answer in the first attempt but, in other cases, they answer in the second attempt. For example, in Barthel #4, the average and last values of the Performance metric are similar to the ones obtained in the execution of the Barthel #3. However, the standard deviation of this metric in the execution of the Barthel #4 is greater. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Question** & **Barthel \#1** & **Barthel \#2** & **Barthel \#3** & **Barthel \#4** & **Barthel \#5** & **Barthel \#6** \\ \hline
3 & 1 st, VR & 1 st, TS & 2 nd, TS & QNA & PR, 1 st, TS & NPD, 2 nd, TS \\ \hline
4 & 1 st, VR & 1 st, TS & 2 nd, TS & 1 st, TS & 2 nd, TS & 2 nd, TS \\ \hline
5 & 1 st, VR & 1 st, TS & 2 nd, TS & 2 nd, TS & PR, 2 nd, RC & NPD, 1 st, TS \\ \hline
6 & 1 st, VR & 1 st, RC & 2 nd, TS & QNA & CDR, 1 st, TS & QNA \\ \hline
7 & 1 st, VR & 1 st, RC & 2 nd, TS & 1 st, VR & 1 st, TS & CDR, 1 st, TS \\ \hline
8 & 1 st, VR & 1 st, RC & 2 nd, TS & 2 nd, VR & 1 st, TS & 2 nd, TS \\ \hline
9 & 1 st, VR & 1 st, RC & 2 nd, TS & 1 st, TS & 1 st, TS & PR, 1 st, TS \\ \hline
10 & 1 st, VR & 1 st, RC & 2 nd, TS & 2 nd, TS & 1 st, TS & 2 nd, TS \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Bathel tests used to validate our model. Legend: 1 st (FirstAttemptAnswered), 2 nd (SecondAttemptAnswered), VR (UserInteraction:VOICE_RECOGNITION), RM (UserInteraction: REMOTE_CONTROL), TS (UserInteraction::TOUCH_SCREEN), CDR (CallingDoctor and then TestState:RESTARTED), PR (TestState:PAUSED and then TestState:RESTARTED), QNA (Question not answered. Defined as SecondAttemptAnswered=false) and NPD (PersonDetected=false and then PersonDetected=true). The reason is that the patient in Barthel #4 sometimes answers in the first attempt and sometimes in the second attempt (or does not answer). All these situations have an impact on the evolution of the Performance property. Significantly, the fact that the simulated patient in Barthel #4 answered four of the questions in the first attempt provokes that the Interaction metric provides an average value of 0.61. This value is greater than the one obtained for the patient that run the Barthel #3. But it is a low value. Sometimes the patient answered fluently, and sometimes she needed a second attempt or did not provide an answer. The metrics associated to the execution of the Barthel #5 and #6 are affected by the multitude of uncommon situations included in both executions. In the execution of the Barthel #5, the simulation includes one call to the doctor and two pauses on the session. The result is a very poor Performance (these situations implied that the robot was not able to perform the complete test by itself). However, having a look to the evolution of Barthel #5 (sixth column, Table 2) we can infer that, after receiving help by the doctor (question 6), the human-robot interaction was fluent. The final Interaction value is therefore high (0.94), correctly capturing this fact. Finally, Barthel #6 shows the typical response when the robot deals with a patient that is really uncomfortable in the interaction with the robot. As the QoS metrics are being continuously monitored, it is important to analyse their evolution over time. Table 4 (Right) depicts how the Performance, Interaction and VoiceInteractionQuality metrics evolve during the execution of the test Barthel #4. Table 4 (Left) summarizes the observations detected by the Complex Event Processor, linked to the instant of time on which they were generated, and to the context stored in the DSR. With respect to Table 2, we can note the presence of two new context values in the Table 4 (Left): PD indicates that the person is detected and the session can start; and FINISHED states that the 10 questions were captured and the session can end.