# Driver Behavior Analysis via Two-Stream Deep Convolutional Neural Network
## 5 Conclusions  
Driver distraction is one of the major causes of traffic accidents, especially in Taiwan. It would be worth developing a system to detect driver distraction for automatic vehicles. Inspired by the famous two-stream CNN model, we proposed a driver behavior analysis system using CNNs to analyze the input consecutive frames, and the feature maps from the last convolution layer were extracted as the spatial and temporal features for further classification. Unlike previous studies using manually defined weights or weights obtained via the optimization process, a fusion network was designed to integrate the modality features for classification. In addition, a self-compiled dataset of 10 actions in the vehicle was established. According to the experimental results, the proposed system can increase the accuracy rate by nearly 30% compared to the original two-stream CNN model. In the future, a comprehensive network configuration could be designed for more fine-grained actions and various scene challenges, such as behavior recognition in the night.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **Spatial Stream ConvNet** & **Temporal Stream ConvNet** & **Fusion Result** \\ \hline
**Two-Stream Method 27** & 9.52 & 39.68 & 38.10 \\ \hline
**Two-Stream Method 27** & 12.98 & 41.27 & 39.68 \\
**with Pre-trained model** & 33.34 & 9.52 & 34.92 \\ \hline
**Proposed Method with** & **49.21** & **65.08** & **68.25** \\
**Pre-trained models** & & & \\ \hline \hline \end{tabular}
\end{table}
Table 2: Accuracy rate in the self-compiled dataset.  
\begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline  & Safe & Text & Phone & Text & Phone & Adjusting & \multirow{2}{*}{Drinking} & Reaching & Hair or & Talking to \\  & Driving & Right & Right & Left & Left & Radio & & & \\ \hline Safe Driving & **0.80** & 0.0 & 0.0 & 0.20 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\ \hline Text Right & 0.0 & **0.75** & 0.0 & 0.0 & 0.0 & 0.0 & 0.25 & 0.0 & 0.0 & 0.0 \\ \hline Phone Right & 0.0 & 0.0 & 0.33 & 0.0 & 0.0 & 0.0 & 0.17 & 0.0 & 0.50 & 0.0 \\ \hline Text Left & 0.0 & 0.0 & 0.0 & **1.00** & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\ \hline \hline Adjusting & \multirow{2}{*}{0.11} & \multirow{2}{*}{0.22} & \multirow{2}{*}{0.0} & \multirow{2}{*}{0.0} & \multirow{2}{*}{0.0} & \multirow{2}{*}{**0.56**} & \multirow{2}{*}{0.0} & \multirow{2}{*}{0.11} & \multirow{2}{*}{0.0} & \multirow{2}{*}{0.0} \\ Radio & & & & & & & & & \\ \hline Drinking & 0.0 & 0.13 & 0.25 & 0.0 & 0.0 & 0.0 & **0.5** & 0.0 & 0.12 & 0.0 \\ \hline Reaching & \multirow{2}{*}{0.0} & \multirow{2}{*}{0.0} & \multirow{2}{*}{0.0} & \multirow{2}{*}{0.0} & \multirow{2}{*}{0.0} & \multirow{2}{*}{0.0} & \multirow{2}{*}{**0.75**} & \multirow{2}{*}{0.0} & \multirow{2}{*}{0.0} \\ Behind & & & & & & & & & \\ \hline Hair or & \multirow{2}{*}{0.0} & \multirow{2}{*}{0.0} & \multirow{2}{*}{0.0} & \multirow{2}{*}{0.14} & \multirow{2}{*}{0.0} & \multirow{2}{*}{0.0} & \multirow{2}{*}{0.0} & \multirow{2}{*}{0.0} & \multirow{2}{*}{**0.86**} & \multirow{2}{*}{0.0} \\ Makeup & & & & & & & & & \\ \hline Talking to & \multirow{2}{*}{0.0} & \multirow{2}{*}{0.0} & \multirow{2}{*}{0.0} & \multirow{2}{*}{0.11} & \multirow{2}{*}{0.0} & \multirow{2}{*}{0.0} & \multirow{2}{*}{0.0} & \multirow{2}{*}{0.0} & \multirow{2}{*}{**0.67**} \\ Passenger & & & & & & & & & \\ \hline \hline \end{tabular}
\end{table}
Table 3: Confusion matrix in the self-compiled dataset.  
**Author Contributions:** Methodology, J.-C.C. and C.-Y.L.; Software, C.-Y.L. and P.-Y.H.; Visualization, P.-Y.H.; Writing--original draft preparation, C.-Y.L.; Writing--review & editing, J.-C.C. and C.-R.L. All authors have read and agreed to the published version of the manuscript.  
**Funding:** This work was supported by the Ministry of Science and Technology of Taiwan, R.O.C., under grant No. 108-2221-E-992-032-.  
**Conflicts of Interest:** The authors declare no conflict of interest.