## 5 Evaluation of the Proposed Monitoring Approach
### Results on Action Recognition  
First, we trained and tested the model with a fixed crop position, cropping a center square image portion for each frame of the sequences. Subsequently, we used a people detector to center the crop on the subject. We explored this matter to understand if the use of a person detector can help the models to reach better classification performance. Results in Table 7 shows that the use of a person detector allows models to achieve a better action classification. Most relevant performance increment is in Alerting group, follow by Daily-life, as expected the increment for Status is very small, this could be due to already high performance without the use of a people detector. As reported, a bigger increment of performance is obtained in Alerting and Daily-life, where the number of the classes is higher and the difference between some actions is concentrated in small details or minor movements of subjects, faced the results the use of a people detector is desirable.  
In parallel, we performed experiments comparing the two different architectures, presented before in Section 4.2. The results highlight that I3D with its architecture based on 3D convolutions made it possible to learn properly spatio-temporal features, reaching good performance in action classification. This means that the expansion of 2D image classification models to 3D convolutional networks allows learning efficiently spatio-temporal relations. I3D performs better in all conditions, reaching the best performance on ALMOND when combined with the subject detector. As speculative observation 3D convolution architecture could perform better than DeepHAR because the latter cannot reach the same expressive power and it is potentially limited by the pose estimation. It is worth noticing that also for the I3D architecture it results difficult to discriminate actions with similar movements and small differences, like those appearing in _Drinking_ and _Eating_. This behavior can be mainly noticed into Daily-life group, and reported in confusion matrix in Figure 6. In conclusion, I3D architecture obtains the best results for action classification; however it is not free from misclassification problems also with the use of a people detector. As shown in Figure 6e there is some confusion between the actions _Drinking_, _Eating_ and _Using phone_, which differ only for the object that the subject held on hand. The same situation is visible for _touching_ actions in Alerting group.  
\begin{table}
\begin{tabular}{l c c} \hline \hline
**Model** & **[EMAIL_ADDRESS]** & **AP** \\ \hline \multicolumn{3}{c}{2 cUnrotated} \\ \hline Faster R-CNN orig. & 0.819 & 0.516 \\ Faster R-CNN fineT. & 0.801 & 0.497 \\ \hline \multicolumn{3}{c}{2 cRotated +90} \\ \hline Faster R-CNN orig. & 0.246 & 0.106 \\ Faster R-CNN fineT. & 0.651 & 0.345 \\ \hline \multicolumn{3}{c}{2 cRotated -90} \\ \hline Faster R-CNN orig. & 0.244 & 0.105 \\ Faster R-CNN fineT. & 0.648 & 0.341 \\ \hline \hline \end{tabular}
\end{table}
Table 6: People detectors performance, comparison between pre-trained detector and fine-tuned.  
\begin{table}
\begin{tabular}{l c c c} \hline \hline
**Subject Localization** & **Action Group** & **I3D [33]** & **DeepHAR [42]** \\ \hline \multirow{3}{*}{Fixed central crop} & Status & 0.947 & 0.664 \\  & Alerting & 0.770 & 0.518 \\  & Daily-life & 0.630 & 0.446 \\ \hline \multirow{3}{*}{People detector} & Status & 0.974 & 0.723 \\  & Alerting & 0.829 & 0.613 \\ \cline{1-1}  & Daily-life & 0.715 & 0.498 \\ \hline \hline \end{tabular}
\end{table}
Table 7: Results of average per-class accuracy for compared architectures. We conduct experiments with fixed central crop and with the use of people detector for both selected models.  
Figure 6: Action classification confusion matrix for each defined actions group. Left column refers to I3D, right to DeepHAR. Results refer to experiments with the people detector.