# Raw Material Flow Rate Measurement on Belt Conveyor System Using Visual Data
## 4 Experimental Evaluations
### Results & Discussion
The sum aggregation performed the worst for small window sizes without any feature selection technique, see Figure 18. However, it performs adequately for large window sizes without the feature selection technique as can be seen from Figure 19. Furthermore, when trained  
Figure 19: Leave-one-out results with recursive feature engineering. with the feature selection, it performed well on all window sizes. The mean aggregation has shown promising results for linear models without the feature selection technique; however, the rest of the models have not performed very well. However, mean aggregation with feature selection tends to give better accuracy for larger window sizes. Finally, the combination of sum and mean aggregation does not show any significant difference in their performance. However, adding both sum and mean feature increase the overall training time for the models and may add multicollinearity, which undermines the statistical significance of an independent variable. The leave-one-out validation might work well for small data. However, a similar evaluation is also carried out with k-fold cross-validation and the results using RMSE metric are shown in Figures 20 and 21. The leave-one-out validation can have high variance as its value changes more for different data samples than the value for k-fold cross-validation. So, to maintain the bias-variance ratio, it is also evaluated with k-fold cross-validation. The mean value of frames in a window has proved to be the most reliable aggregation method in k-fold cross-validation with and without the feature selection technique. However, the sum has given the best results for smaller window sizes. The different machine learning algorithms have shown similar results as the leave-one-out cross-validation. The linear model has performed better without the feature selection, and with feature selection they have produced poor results. The Decision Trees, Random Forest, and Bagging regressor models have produced the best results with the feature selection technique. Without the feature selection technique, the linear models tend to produce better results for larger window sizes. With feature selection, the window size does not have much impact on the final results. The best result for k-fold cross-validation is given by Gamma regressor for 2000 window size without using the feature selection technique. After analysis, we selected the optimal settings for all the given parameters and trained the machine learning algorithms. The final model was evaluated with a split of training  
Figure 21: K-fold results with recursive feature engineering. Figure 20: K-fold results without recursive feature engineering. data and test data. For training the final model, the complete dataset was used. For the test set, a new set of input data was utilized, and the model's performance was reported based on those predictions. The results are shown in Table 3. We have trained our final model without feature selection since the results are better without the feature selection for our problem. Nevertheless, the situation can be changed if the performance is considered more important than accuracy. Apart from it, a window size of 1000 is chosen for performing the mean aggregation function. The evaluation statistics presented in Table 3 show that the best mean absolute error and mean absolute percentage error are achieved by the Bayesian ridge and the Thiel-Sen regressor, whereas the best mean squared error and root mean squared error is achieved by the Bagging regressor model. To further investigate the performance of all these models, the mean absolute error (MAE) is computed for each of the four test video sequences and the results are presented in Figure 22. These results reaffirm the superior performance of the Thiel-Sen regressor which shows appreciable MAE. The Bayesian ridge also produced good estimates in three tests, however, the difference between the actual (1647 kg/min) and estimated flow rate (1613 kg/min) in the third test video is slightly high.