## 2 Research Framework/Methodology  
Excavators are used for performing four basic activities: excavation, hauling, dumping, and returning, usually. These activities are defined as filling the bucket with dirt/rock, moving the filled bucket to the dump point, emptying the bucket into the truck/dump point, and moving back to the excavation point with an empty bucket, respectively. The excavator usually shows two types of sequential patterns. The first type of sequential pattern is one in which the excavator shows its sequential visual features pattern during a single activity. For example, when the excavator is lifting-up during the digging and hauling while the bucket is moving downward. Similarly, for the hauling and swinging, the main body of the excavator rotates, keeping the bucket and arm at a specific speed and position. The other sequential pattern is the operation cycle of the excavator, in which the excavator starts a specific activity and then works in a loop. In most cases, the first activity \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Author** & **Year** & **Accuracy** & **Classifier** & **Goal** \\ \hline [PERSON] et al. [17] & 2011 & 79 & 3D-Harris feature + Bayesian learning classifier & Classifying actions of construction workers and equipment \\ \hline [PERSON] et al. [10] & 2013 & 86.33 & 3D HOG feature + SVM classifier & Activities of excavator and dump truck status \\ \hline [PERSON] et al. [31] & 2016 & 57 & HOG, HOE, MBH features + SVM classifier & Action recognition of construction worker \\ \hline [PERSON] et al. [32] & 2018 & 80.5 & Faster R-CNN detector + Relevance network & Recognizing diverse construction activities \\ \hline [PERSON] et al. [13] & 2018 & & Faster R-CNN & Reduce the test time and memory requirements, enhanced VGG-16 net precision \\ \hline [PERSON] et al. [12] & 2018 & P = 95.7, R = 94.9 & Faster R-CNN & Detect non-hardhat-use \\ \hline [PERSON] [20] & 2017 & 91.21 & \multirow{2}{*}{CNN-BDLSTM} & Novel CNN-BDLSTM method for activity recognition (human activities) \\ \cline{3-5}  & & 87.64 & & \\ \hline [PERSON] et al. [33] & 2019 & up to 99.88 & SVM, ANN, Decision tree & Detecting excavator anomalies \\ \hline [PERSON] et al. [6] & 2019 & 79.8 & CNN & Excavator detector \\ \cline{3-5}  & & 90.9 & CNN-LSTM & Excavator tracking \\ \cline{3-5}  & & 93.8 & CNN-DLSTM & Excavator activity recognition \\ \hline [PERSON] et al. [34] & 2019 & up to 98 & \
\begin{tabular}{c} Different pre-trained CNN \\ model and Transfer learning \\ \end{tabular} &
\begin{tabular}{c} Classification of full/empty-load trucks in \\ earthmoving operations \\ \end{tabular} \\ \hline [PERSON] et al. [11] & 2020 & 87.6 & Faster R-CNN + Deep SORT & Excavator detection \\ \cline{3-5}  & & tracker + 3.D ResNet classifier & Tracking, activity recognition of excavators \\ \hline [PERSON] et al. [35] & 2021 & 78 & YOLOV3 & Activity detection and classification \\ \hline [PERSON] et al. [36] & 2022 & 99.7 & YOWO & Vision-based autonomous excavator productivity \\ \hline [PERSON] et al. [37] & 2022 & 86 & Zero-shot learning method & Productivity analysis in earthmoving \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Summary of techniques used in the excavator detection and activity recognitions. of an earthwork operation is \"digging\" which is followed by the \"hauling\", \"dumping\", and \"swing\". Therefore, most probably, the last activity of the excavator would be the \"dumping\". The general views of the four activities are shown in Figure 1.  
This study classifies the basic activities of excavators based on the sequential pattern of video frames. The pre-trained CNN used in this research is \"Googlenet\", the winner of the ILSVRC (ImageNet Large Scale Visual recognition Competition) 2014 [38]. Video clips of each activity are fed into a pre-trained CNN to extract the features from video frames. The sequenced feature vectors are the input for the bidirectional long short-term memory (BiLSTM) to classify the activities.