# Complex Human-Object Interactions Analyzer Using a DCNN and SVM Hybrid Approach
## 4 Experimental Results  
The experiments were performed on the CAD-120 human daily activities dataset which consisted of ten high-level activities performed by four different subjects: making cereal, taking medicine, stacking objects, unstacking objects, microwaving food, picking objects, cleaning objects, taking food, arranging objects, and having a meal. The data consisted of RGB images, depth images and skeleton joint coordinates, object locations, and object types for each high-level activity. This data can be downloaded from the following URL: _[[http://prcs.cornell.edu/humanactivities/data.plp_](http://prcs.cornell.edu/humanactivities/data.plp_)]([http://prcs.cornell.edu/humanactivities/data.plp_](http://prcs.cornell.edu/humanactivities/data.plp_)). The sample RGB images in the CAD-120 dataset are shown in Figure 8. The performance of the proposed system was evaluated by the leave-one-subject-out-cross-validation method, which uses three subjects' data as training data, and data for the other subject as test data. Therefore, we alternately trained the DCNN and multi-class SVM using data from three different subjects, and used data from the remaining subject as test data. We performed these experiments four times by alternately training and testing using data from four subjects. For accurately recognizing the interactions involved in \"stacking objects\" and \"unstacking objects\", we added some rules based on spatial features of the objects, because the obvious difference between those two interactions is the spatial feature of (total width) at the start and end times of the interactions. In the case of \"stacking objects\", the width of all objects at the beginning is larger than that at the end. However, the opposite is the case for the \"unstacking objects\", as shown in Figure 9. The graphical form of the result of recognizing \"making cereal\" interaction is shown in  
Figure 7: Illustration of training and testing phases of multi-class SVM using object usage probability (OUP) data. Figure 10. A detailed confusion matrix for recognizing the actions in the CAD-120 dataset is shown in Table 6.  
Figure 8. Sample RGB images from the CAD-120 dataset. Figure 10. A detailed confusion matrix for recognizing the actions in the CAD-120 dataset is shown in Table 6.  
Figure 9. Illustration of spatial features between stacking and unstacking objects (First row: RGB image; Second row: DCNN input image; yellow line indicates total object width). \begin{table}
\begin{tabular}{|c|c|c|c|c|c|c|c|c|c|c|} \hline  & \multicolumn{10}{c|}{**Recognized Actions**} \\ \cline{2-11}  & & **\#P** & **\#P** & **\#P** & **\#P** & **\#P** & **\#P** & **\#P** & **\#P** & **\#P** & **\#P** \\ \hline \multirow{8}{*}{**Actual Actions**} & making cereal & 100 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ \cline{2-11}  & taking medicine & 0 & 100 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ \cline{2-11}  & stacking objects & 0 & 0 & 91.67 & 0 & 0 & 0 & 0 & 0 & 8.33 & 0 \\ \cline{2-11}  & unstacking objects & 0 & 0 & 0 & 91.67 & 0 & 0 & 0 & 0 & 8.33 & 0 \\ \cline{2-11}  & microwawing food & 0 & 0 & 0 & 0 & 91.67 & 0 & 0 & 8.33 & 0 & 0 \\ \cline{2-11}  & picking objects & 0 & 0 & 0 & 0 & 0 & 83.33 & 0 & 0 & 8.33 & 8.33 \\ \cline{2-11}  & cleaning objects & 0 & 0 & 0 & 0 & 0 & 0 & 100 & 0 & 0 & 0 \\ \cline{2-11}  & taking food & 0 & 0 & 0 & 0 & 8.33 & 8.33 & 0 & 83.33 & 0 & 0 \\ \cline{2-11}  & arranging objects & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 91.67 & 0 \\ \cline{2-11}  & having a meal & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 100 \\ \hline \end{tabular} The percentage of correctly recognized actions in the CAD-120 dataset are highlighted in yellow color.  
\end{table} 
Table 6. Confusion matrix for recognizing interactions in the CAD-120 dataset. We calculated the overall accuracy (Recognition Rate) using the following Equation (10).