Therefore, the authors have investigated the integrated method based on a pre-trained CNN, Googlenet and a bidirectional long short-term memory (BiLSTM) for the activity classification of the excavator. This study implemented a pre-trained convolutional neural network \"Googlenet\" with the recurrent neural network \"LSTM\" for classification based on the sequential pattern of video frames. The goal of this study is to practically demonstrate the applicability of Googlenet for feature extraction in the field of the construction industry. Additionally, the application of BiLSTM for the different activities of the excavator and its response is observed. This research's main objective is to evaluate the effectiveness of a pre-trained CNN-based bidirectional long short-term memory (CNN-BiLSTM) for the activity recognition of an excavator in a construction site. Excavators are the most common equipment for earthwork and construction sites with a cycle of activities: excavation, hauling, dumping, and swinging. The surveillance videos of the excavator are used as input to a pre-trained convolutional neural network to learn the sequential pattern of video frames. Based on the findings of the pre-trained model, BiLSTM classifies the different activities of the excavator. The rest of this paper is organized as follows. The first section of this paper will identify the key information relevant to the vision-based activity classification and provide an overview of the existing studies on vision-based activity recognition in the next section. The next section will then elaborate on the architecture of the proposed research following by the training of the model with the video input of the earthwork site in successive sections. The model is trained with the video input of the construction site to validate the effectiveness of the framework. The experimental results will be analyzed in the second-to-last section, and finally, the last section will provide the conclusion and the research contribution with the future direction. Related Works  
Many efforts have been made to recognize the activities of construction equipment with the help of vision-based techniques. Recently, different vision-based techniques have been utilized to investigate the productivity, safety, cost and automation of the construction process. Surveillance cameras have been installed to record the activities at construction sites [21]. Cameras should be placed at a high and appropriate locations so that the occlusion may be minimal [22]. Visual data collected from these surveillance cameras is used for the activity recognition and safety of the workers and construction equipment such as excavators, dump trucks and dozens. There are numerous vision-based methods available for the detection of objects, humans or construction equipment. SURF is a local feature detector and descriptor of the interesting points used to construct the feature vector. Additionally, SVM classifies different classes by constructing hyperplanes in a multidimensional space. A technique based on the Speeded-up Robust Features (SURF) and Support Vector Machine (SVM) was presented for facial recognition on Yalefaces and the UMIST dataset with an accuracy of 97.78% and 97.87%, respectively [23]. A part-based object recognition model was presented to detect the excavator at different poses using a discriminately trained HOG classifier [9]. A convolution neural network IFaster R-CNN method, consisting of a Region Proposal Network (RPN) and an R-CNN, was presented to detect workers and excavators in real-time [24]. The accuracy of the IFaster R-CNN model to detect the workers and the excavator was 91% and 95%, respectively. A deep learning method, Faster R-CNN, was used to detect non-hardhat-use at construction sites by surveillance videos [12]. The precision and recall for the test dataset were 95.7% and 94.9%, respectively. To date, various methods have been developed and introduced to detect construction equipment and recognize the activities of excavators in earthwork sites.