# An Intelligent Real-Time Driver Activity Recognition System Using Spatio-Temporal Features
## Appendix A  
Table 11 illustrates the complexity of the proposed model including the trainable parameters and feature memory usage. According to the table, it can be observed that the trainable parameters and the feature memory of the proposed model were around 16 million and 90 M, respectively. To investigate the optimal number of frame inputs to the designed LRCN model, a different number of frames are compared, as displayed in Table 12. Based on the result obtained, three frames of input have shown the poorest performance; this is mainly because the proposed model has an LSTM network that expects an adequate sequence input or a long-term dependence to study the dynamic temporal information of the driver's image to determine the final driver activity. As such, three frame inputs could be too few a number of frames to correctly predict the final driver state. On the other hand, with four, five, and six number frame inputs, the model has shown a similar performance, with five frame inputs yielding the highest accuracy. As shown in Table 11, although six frame inputs achieved a similar accuracy to that of the five frame inputs, the model exhibited a long training time when increasing the number of frame inputs. This is because the LSTM unit requires more time when the sequence input increases to analyze the dynamical temporal information of each sequence input. This can cause a delay in  
real-time prediction and can be computationally expensive. Due to these facts, five frame inputs is selected as the final input of the model.  
The statistical result of the real-time prediction for the remaining three drivers during the daytime and nighttime scenarios are displayed in Figures 11 and 12 for the multiclass prediction and binary prediction, respectively. From Figure 11, it can be observed that the proposed model achieved accuracies of 75.8%, 92.0%, and 74.8% during the daytime, while 80.8%, 88.33%, and 76.4% accuracies are achieved during the nighttime for driver 2, driver 3, and driver 4, respectively. (Driver 1's results can be found in the main manuscript). From Figure 12, it can be observed that the model achieved accuracies of 99.6%, 96.2%, and 99.6% during the daytime. Also, accuracies of 98.8%, 95.8%, and 96.1% were achieved during the nighttime for the three subsequent drivers in real-time conditions. Figure 13 shows the failure prediction results when using RGB sensors for nighttime during the real-time prediction. As shown in the results, it is obvious that the performance of RGB images for nighttime recognition is poor. In addition, the real-time assessment output when using IR sensors in the daytime is displayed in Figure 14. Compared to the RGB images at night, IR images have shown a better performance during the daytime.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Number of Frames** & **Test Accuracy (\%)** \\ \hline
3 frames & 69.6 \\
4 frames & 86.7 \\
5 frames & 88.7 \\
6 frames & 88.1 \\ \hline \hline \end{tabular}
\end{table}
Table 11: Number of trainable parameters and feature memory of the proposed model.  
Figure A1: Real-time assessment results for multiclass classification: (**a**) RGB images in the daytime for driver 2 (accuracy of 75.8%), (**b**) IR images in the nighttime for driver 2 (accuracy of 80.8%), (**c**) RGB images in the daytime for driver 3 (accuracy of 92.9%), (**d**) IR images in the nighttime for driver 3 (accuracy of 88.3%) (**e**) RGB images in the daytime for driver 4 (accuracy of 74.8%), and (**f**) IR images in the nighttime for driver 4 (accuracy of 75.4%).