# Object Detection for Industrial Applications: Training Strategies for AI-Based Depalletizer
## 3 Results and Discussion
### Comparison II Results  
In this subsection, the comparisons among the cases, or approaches, are reported and discussed. Table 1 reports all the pairwise comparisons that were statistically different when comparing the F1-score among all the cases given a fixed size of the training set. Such an analysis was conducted on both experimental conditions, _Dataset 1_ and _Dataset 2_.  
First of all, considering the trends reported in Figure 5a and Table 1, it can obliviously be stated that the data augmentation was always preferable if, of course, the computation resources can manage a bigger training set within a reasonable time. This can be noticed by comparing Case 1a vs. Case 1b and Case 3a vs. Case 3b, given a fixed training size. Most of such differences were not statistically significant due to the Bonferroni correction for multiple comparison, but can easily be observed within Figure 5a. This finding is true for both experimental conditions _Dataset 1_ and _Dataset 2_.  
Another important aspect concerns the results of Case 2a, which studied the possibility of using a pretrained network on other dataset of different boxes without performing fine-tuning on the new box images. Comparing Case 2a with the two Case 3s (Case 3a and Case 3b), it emerged that the fine-tuning always allowed reaching better performances, and this was true for both experimental conditions _Dataset 1_ and _Dataset 2_ and for most of the training sizes, as shown in Figure 7. Figure 7A,B highlight the differences between Approaches 2a and 3b considering _Dataset 1_: the model of Case 2a was not able to deal with new boxes, while the model of Case 3b, fine-tuned on a new boxes, correctly detected all the visible boxes. Figure 7C shows the improved predictions related to the fine-tuning of a pretrained model with _Dataset 2_; Figure 7D highlights the missed prediction of the model  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Training Set Size**} & \multicolumn{2}{c}{_Dataset 1_} & \multicolumn{2}{c}{_Dataset 2_} \\ \cline{2-5}  & **Pairwise** & _p-Value_ & **Pairwise** & _p-Value_ \\ \hline \multirow{2}{*}{\(15\)} & 1a–3b & 0.0002 & 1b–2a & 0.0137 \\  & & & 2a–3b & 0.0228 \\ \hline \multirow{2}{*}{\(29\)} & 1a–3b & 0.0069 & 1b–2a & 0.0198 \\  & & & 2a–3b & 0.0127 \\ \hline \multirow{2}{*}{\(58\)} & 1b–2a & 0.0148 & 1b–2a & 0.0050 \\  & 2a–3b & 0.0109 & 2a–3b & 0.0031 \\ \hline \multirow{2}{*}{\(116\)} & 1a–3b & 0.0274 & 2a–3b & 0.0036 \\  & 2a–3b & 0.0016 & 1b–2a & 0.0080 \\ \hline \multirow{2}{*}{\(232\)} & 1a–3b & 0.0261 & 2a–3b & 0.0036 \\  & 2a–3b & 0.0016 & & \\ \hline \multirow{2}{*}{\(464\)} & 2a–3b & 0.0006 & 1b–2a & 0.0036 \\  & & & 2a–3b & 0.0008 \\ \hline \multirow{2}{*}{\(696\)} & 1a–3b & 0.0271 & 1b–2a & 0.0080 \\  & 2a–3b & 0.0008 & 2a–3b & 0.0006 \\  & 2a–3a & 0.0197 & & \\ \hline \multirow{2}{*}{\(928\)} & 2a–3a & 0.0109 & 2a–3b & 0.0016 \\  & 2a–3b & 0.0009 & & \\ \hline \hline \end{tabular}
\end{table}
Table 1: Statistically significant differences of Comparison II.  
with _Dataset_ 2, due to the difficulty for the model to predict boxes with multiple textures and prints. However, comparing Case 2a with the two Case 1s (Case 1a and Case 1b), something different occurred. For the condition _Dataset_ 1, i.e., the new dataset contained simple boxes to be detected, it emerged that the results of Case 2a (the pretrained network was trained also with more complex boxes) were comparable to or better than the results of the Case 1s for a training set size smaller that 30 images, whereas, for the condition _Dataset_ 2, i.e., the new dataset contained complex boxes to be detected, it turned out that the results of Case 2a (the pretrained network was trained only with simple boxes) were always worse than the results of the Case 1s. This result is another important hint that can be considered when approaching the design and commissioning of a new depalletizer system.