## 4 Experimental Implementation and Results
### Results  
This section presents a detailed analysis of the experimental results, highlighting the performance of the proposed two-stream CNN-LSTM model for excavator activity recognition. During the training phase, the model demonstrated exceptional learning capabilities, with an overall training accuracy of 99.62%. As depicted in Figure 7a, activities such as Digging, Dumping, Idle, and Leveling were classified with 100% accuracy, indicating the model's robustness in recognizing their distinct features. The Approach activity also exhibited outstanding performance, with 98.72% of its instances correctly identified, and only a single misclassification (1.28%) into the Dumping category. This near-perfect training performance across all classes underscores the model's strong learning capacity with the optimized hyperparameters.  
The true measure of the model's efficacy lies in its generalization performance on unseen data. When evaluated on the 30% test dataset, the trained model achieved an overall test accuracy of 94.67%, an F1-score of 94.78%, and a test loss of 5.10%. The testing confusion matrix in Figure 7b provides a detailed breakdown of these results. The model achieved perfect accuracy for \"Digging and Idle* activities, classifying 100% of their instances correctly. For the \"Dumping\" activity, 95.65% of instances were correctly identified, with one instance (4.35%) being misclassified as Digging. The Leveling activity with 91.30% accuracy, while 8.70% of its instances were misclassified as Approach. Lastly, the Approach activity had an 86.96% correct classification rate, with misclassifications occurring, such  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Epochs**} & \multicolumn{4}{c}{**Learning Rate**} \\ \cline{2-5}  & **0.01** & **0.001** & **0.0001** & **0.00001** \\ \hline
20 & 27.83 & 20 & 90.43 & 84.35 \\ \hline
25 & 38.26 & 86.96 & 93.91 & 86.09 \\ \hline
30 & 43.48 & 22.61 & 94.67 & 86.09 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Testing the accuracies of the trained model at different hyperparameters.  
Figure 6: Learning curve graphs for training and testing.  
as Dumping (8.70%) and Leveling (4.35%). These specific errors highlight subtle overlaps in visual cues or movement patterns between certain activity pairs during real-world operation, such as the initial phases of Approach and Dumping, or the characteristics shared by Leveling and Approach activities performed by the excavator movements.  
The error in predicted labels for activity recognition is visualized in Figure 8, comparing the predicted labels with the ground truth (GT). The experimental results explicitly demonstrate the robust performance of the multi-modal two-stream CNN-LSTM model for excavator activity recognition. The systematic hyperparameter tuning, particularly the identification of an optimal learning rate and epoch count, significantly contributed to the model's ability to achieve high accuracy and strong generalization capabilities. While minor ambiguities exist between specific activity pairs, resulting in a small number of misclassifications, the overall high precision and low error rates validate the proposed methodology's potential for accurately and automatically monitoring excavator earthwork operations.  
Figure 8: Example of activity recognition errors in the ground truth and prediction.  
Figure 7: Confusion matrix of the prediction results **(a)** for the training dataset and **(b)** for the test dataset. The bold values represent the overall performance metrics for all activity classes combined, reflecting the final results of the proposed method.