# Complex Human-Object Interactions Analyzer Using a DCNN and SVM Hybrid Approach
## 3 Proposed System
### Interaction Recognition using a Deep Convolutional Neural Network (DCNN)  
Deep convolutional neural networks (DCNN), also known as deep learning, are a kind of neural network that includes grid-type operations of convolution and pooling over grid-type input data such as images. The main function of convolution operations is to extract key information from raw input data through multiplication in a predefined kernel matrix. Pooling operations are used for reducing the dimensions of each DCNN output layer. Pooling can be performed by the mathematical operations of taking averages or maximum values for the data which exist within the defined matrix. In every DCNN layer, a decision must be made on whether the results of convolution or pooling should be produced as output and sent to the next layer. This decision-making process is performed by the  
Figure 4: Sample DCNN input images.  
Figure 5: Three kinds of region connection states between microwave (cyan), bowl (pink), milk bottle (blue), cup (brown), and medicine box (green).  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Objects** & **R** & **G** & **B** & **Color** & **Objects** & **R** & **G** & **B** & **Color** \\ \hline medicine box & 0 & 255 & 0 & cloth & 255 & 0 & 255 & \\ \hline microwave & 0 & 255 & 255 & bowl & 255 & 0 & 128 & \\ \hline remote control & 0 & 128 & 255 & book & 255 & 0 & 0 & \\ \hline milk & 0 & 0 & 255 & cup & 128 & 0 & 0 & \\ \hline plate & 128 & 0 & 255 & box & 0 & 0 & 128 & \\ \hline \hline \end{tabular}
\end{table}
Table 3: Color representation of each object category in the CAD-120 dataset.  
activation function. Various activation functions have been discussed in the literature on DCNN, including sigmoid, hyperbolic tangent, and rectified linear units (ReLU). Each function has its own properties. For example, the output of the sigmoid activation function is within the range of 0 and 1, and the output of the hyperbolic tangent function is from \(-1\) to 1. In the proposed system, the rectified linear unit (ReLU) activation function was applied in all DCNN layers. ReLU produces original input values as outputs if those values are greater than 0 and produces 0 if input values are less than 0.  
The most recent DCNN research indicates that DCNN achieves superior performance in visual object and pattern recognition. Therefore, we applied DCNN in the proposed system for recognizing patterns of human motion and object interaction. As shown in Figure 6, the DCNN architecture comprises three convolution (_Conv_) layers and three pooling (_Pool_) layers, followed by one fully connected (_F_) layer and an output (_O_) layer. The total number of output neurons for each DCNN layer was 32, 64, 64, and 10. In addition, each layer was followed by a drop-out (_D_) layer for minimizing the data overfitting problem, using drop-out ratios of 1%, 2%, 3%, and 4%. Convolution operations were performed in three hidden layers using kernel sizes of \(7\times 7\), \(5\times 5\), and \(3\times 3\). For pooling operations, \(2\times 2\) kernels were used. Then, one fully connected layer was applied before the output layer in order to combine DCNN features in a one-dimensional vector with a length of 64. The next layer was an output layer with ten neurons for recognizing ten human-object interactions. For transforming the results for DCNN output layers into corresponding probability values (_P_DCNN_), a Soft-Max operation was applied. Weight initialization for all layers was done using the MSRA method, which was developed by Microsoft Research Asia (MSRA) [17]. The stochastic gradient descent algorithm [18] was used for weight updating, and the whole network was constructed using the Matcaffe deep-learning framework [19; 20].