## 5 Evaluation and Discussion  
To evaluate the effectiveness of the proposed two-stream CNN-LSTM model, its performance was compared to two baseline single-stream models, one using only RGB video frames and the other using 3D point cloud data. Both models have the identical architecture, namely CNN-LSTM, and were trained and tested separately on the same dataset to ensure \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Parameter** & **Accuracy (\%)** & **F1-Score (\%)** & **Loss (\%)** & **Observation** \\ \hline \multicolumn{5}{c}{Learning Rate} \\
0.01 & 43.48 & 40.25 & 58.33 & Unstable training \\
0.001 & 22.61 & 21.9 & 75.14 & Divergence or slow learning \\
**0.0001** & **94.67** & **94.78** & **5.33** & **Optimal setting** \\
0.00001 & 86.09 & 85.4 & 16.05 & Underfitting \\ Epochs & 20 & 90.43 & 91.12 & 9.57 & Early stopping risk \\
25 & 93.91 & 94.1 & 6.09 & Better generalization \\
**30** & **94.67** & **94.78** & **5.33** & **Best performance** \\ Sequence Length & 10 & 89.32 & 89.5 & 10.68 & Temporal context insufficient \\
**15** & **94.67** & **94.78** & **5.33** & **Balanced setting** \\
20 & 94.12 & 94.3 & 5.88 & Slight improvement, higher cost \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Sensitivity analysis of hyperparameters on model performance. fair comparison. Similar to the proposed multi-stream approach, the single-stream input CNN-LSTM network extracts spatial features from camera RGB or point cloud data using a CNN and then utilizes these features as input for the LSTM to capture the temporal aspects of the input data. It integrates spatial and temporal features to recognize excavator activities and uses CNN layers to analyze input frames with optical flow, estimated via the Farneback algorithm. These spatiotemporal and optical flow features are then processed through an LSTM for activity recognition and classification. The dataset was split into training and testing data with a 70% and 30% ratio, respectively, to create distinct subsets for training and testing purposes. Figure 9 displays the training and testing results of two models based on a single-stream architecture using CNN-LSTM. Both models were trained on the same dataset to recognize five key excavator activities: Approach, Digging, Dumping, Idle, and Leveling. The figure includes the confusion matrix, along with class-specific precision, recall, and loss metrics, as well as overall accuracy and F1-score for training and testing. The RGB-based model achieved a training accuracy of 98.53%, with high classification performance across all activity categories. The training confusion matrix indicates that the model learned most class boundaries effectively, with perfect classification in \"Idle\" and \"Leveling\". The overall F1-score was 98.54, and the average loss was low (1.47). However, during the test phase, performance declined, with a test accuracy of 90.67%, an F1-score of 90.78, and a notable increase in class-wise losses (average of 9.33%). Misclassifications were more frequent between classes, such as \"Approach\" and \"Leveling\" or \"Dumping\", likely due to similar motion patterns and RGB's sensitivity to viewpoint and lighting changes. The cloud-based point model exhibited perfect classification performance during training, achieving 100% in accuracy, precision, recall, and F1-score, with zero loss across  
Figure 9: Confusion matrix for the training and testing single-stream CNN-LSTM. all activity classes. This indicates potential overfitting. Nonetheless, the model performed better than the RGB model on the test set, achieving a higher test accuracy of 92.00%, an F1-score of 92.00, and a reduced average loss of 8.00. The confusion matrix shows fewer misclassifications, particularly in separating geometrically distinct actions such as \"Leveling\" and \"Idle\". The results demonstrate the better robustness of point cloud data to visual occlusions and lighting variations, likely due to its 3D spatial structure and depth perception capabilities.