## 5 Results
### Preliminary  
The comparison study was based on a large-scale landscape transformation project, which adopts the counting FEQS logic, and thus requires the full/empty-load judgment for earthmoving trucks. The project site was located in an open field that is quarantined from the public, and thus no interference from external vehicles existed. All earthmoving trucks were uniform in models and loading capacity, and truck buckets were uncovered. The surveillance camera system was deployed on the route of truck transportation. Hence, the surveillance video was used as the data source. Before the model comparison, enough truck images were collected to form the data sets for training, validation, and testing. The principles of image collecting in this study were:  
* **Number:** Over 500 images for each full-load or empty-load truck state should be collected. * **Labeling:** Truck images are taken from the surveillance video by a screenshot, and are manually labeled to guarantee their correctness. * **Size:** Truck images should be uniform in size; in this case, the size should be around \(350\times 250\) pixels, and the truck should be in the middle of the image and occupying about one third to one half of the frame. * **Visibility:** The truck bucket should be visible so that the full-load and empty-load condition can be distinguished, and the truck in the image should be distinguishable from the background, i.e., the image should be collected and become usable data only if its truck loading condition can be distinguished by human eyes. In practice, 2454 images were taken and included as the data for the deep learning study following the above principles, thus they were uniform in size, manually labeled, and have good distinguishability by visual sight. The image collecting spanned three months (July to September) under different lighting or weather conditions. Among 2454 images, 1588 were full-load truck images and 866 were empty-load truck images. On the basis of fully utilizing the image data, all collected images were randomly divided into three sets according to the ratio of training set-validation set:testing set = 6:3:1. For the three sets: (i) The training set included labeled images of full/empty-load trucks and was used to train deep learning models; (ii) the validation set included labeled images of full/empty-load trucks, which did not participate in the model training but was used to test the training performance of models at the end of each training epoch; (iii) the testing set included unlabeled images of trucks, which was used to test the generalization ability of trained models, i.e., the actual working performance. Details of the three data sets are shown in Table 1 (in Appendix A). Based on the three data sets, the 12 deep learning models were trained, validated, and tested. The training set was first uploaded to train deep learning models. During the training, images in the training or validation sets were used to test the model training performance. Finally, after finishing all training epochs, the trained model was tested by the testing set in full/empty-load trucks classification and working performance of models can be revealed. Notably, classification errors in the test results were manually obtained. Training/testing results of the 12 models can be seen in Table  
Figure 5: Deep learning model comparison study. Since the data sets of training and validation here were disproportional in image numbers of the full-load and empty-load state, a contrast that adopts proportional data sets was attached to show the possible effect of disproportions. Hence, the number of full-load truck images in the original training and validation sets were reduced to the equal number of empty-load truck images, and the 12 deep learning models were trained and tested again under the reduced but proportional data sets. Details of these proportional data sets and training/testing results of this new round are shown in Tables A3 and A4 (in Appendix A).