# Application of Machine Learning Methods for Pallet Loading Problem
## 5 Results and Discussion
This classifier also performed best in terms of balanced accuracy by class, with a balanced accuracy for class P1 equal to 0.90, a balanced accuracy for class P2 equal to 0.88, and a balanced accuracy for class P3 equal to 0.95, showing a high predictive power. The Random Forest Classifier displayed the lowest confusion entropy by class, namely 0.23 for class P1, 0.22 for class P2, and 0.12 for class P3. The Confusion Matrix graphics of all five classifiers are depicted in Figures 7-11. In Figure 7, the Confusion Matrix for the ANN classifier is presented. It can be observed that the ANN classifier correctly classified 83% of the P1 classes, 79% of the P2 classes, and 81% of the P3 classes. Judging from Figure 8, which depicts the Confusion Matrix for the Decision Tree, the classifier managed to correctly predict 100% of classes P1, P2, and P3. Figure 9 shows the Confusion Matrix for the KNN classifier, which managed to correctly capture 97% of the P1 classes, 92% of the P2 classes, and 94% of the P3 classes. In Figure 10, the Confusion Matrix for the Random Forest classifier is shown. It can be observed that this classifier managed to predict 100% of classes P1, P2, and P3. Judging from Figure 11, which illustrates the Confusion Matrix for the SVM, the classifier managed to correctly capture 73% of the P1 classes, 75% of the P2 classes, and 77% of the P3 classes. According to Figure 12, average PUV values in P1, P2, and P3 were 90.31, 78.67, and 82.97, respectively. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multirow{2}{*}{**Class (Pallet)**} & \multirow{2}{*}{ \begin{tabular}{c} **Sensitivity** \\ **by Class** \\ \end{tabular}  } & \multirow{2}{*}{ \begin{tabular}{c} **Specificity** \\ **by Class** \\ \end{tabular}  } & \multirow{2}{*}{ \begin{tabular}{c} **Precision by** \\ **Class** \\ \end{tabular}  } & \multirow{2}{*}{ \begin{tabular}{c} **F1-Macro by** \\ **Class** \\ \end{tabular}  } & \multirow{2}{*}{\ \begin{tabular}{c} **Balanced** \\ **Accuracy by** \\ **Class** \\ \end{tabular}  } & \multirow{2}{*}{ \begin{tabular}{c} **Confusion** \\ **Entropy by** \\ **Class** \\ \end{tabular}  } \\ \hline \multirow{3}{*}{ANN} & P1 & 0.76 & 0.89 & 0.79 & 0.75 & 0.82 & 0.36 \\  & P2 & 0.72 & 0.82 & 0.65 & 0.65 & 0.77 & 0.42 \\  & P3 & 0.79 & 0.94 & 0.91 & 0.84 & 0.86 & 0.28 \\ \hline \multirow{3}{*}{Decision Tree} & P1 & 0.90 & 0.90 & 0.84 & 0.85 & 0.90 & 0.24 \\  & P2 & 0.80 & 0.94 & 0.85 & 0.81 & 0.87 & 0.25 \\  & P3 & 0.92 & 0.96 & 0.95 & 0.93 & 0.94 & 0.15 \\ \hline \multirow{3}{*}{KNN} & P1 & 0.79 & 0.88 & 0.78 & 0.77 & 0.84 & 0.36 \\  & P2 & 0.81 & 0.88 & 0.76 & 0.76 & 0.85 & 0.32 \\  & P3 & 0.84 & 0.96 & 0.94 & 0.87 & 0.90 & 0.20 \\ \hline \multirow{3}{*}{Random Forest} & P1 & 0.90 & 0.91 & 0.84 & 0.86 & 0.90 & 0.23 \\  & P2 & 0.81 & 0.96 & 0.88 & 0.82 & 0.88 & 0.22 \\  & P3 & 0.94 & 0.97 & 0.96 & 0.95 & 0.95 & 0.12 \\ \hline Support & P1 & 0.73 & 0.86 & 0.72 & 0.70 & 0.79 & 0.44 \\ Vector & P2 & 0 Figure 8: Normalized confusion matrix for Decision Tree. Figure 7: Normalized confusion matrix for ANN. Figure 10: Normalized confusion matrix for Random Forest. Figure 9: Normalized confusion matrix for KNN. The PUV values of 777 boxes were calculated for each pallet size, and the results are presented in Figure 13. It can be noted that the maximum PUV values were obtained in the case of P1 because of the smallest volume of the pallet in P1. Pallet size P3 provided the second best PUV value results because of its smaller volume compared with P1. Finally, the PUV values of the largest pallet volume (P3) were the highest because they had the largest volume. Figure 11: Confusion matrix for SVM classifier. Figure 12: Distribution of average PUV values for the 1 K instances for which the algorithm can be applied. Overall, the variety of results given in Section 5 provide an overall idea for shipment designers to choose better pallet or box sizes for the pallet loading.