# Application of Machine Learning Methods for Pallet Loading Problem
## 5 Results and Discussion  
In our previous study [10], 15 real-life datasets were obtained from the DHL supply chain, and the proposed algorithm was applied to this dataset. After a detailed analysis, an algorithm was described and used for the results of other algorithms for comparison. Thus, the results comprising the total number of boxes per pallet were compared. Consequently, an improvement of 6.7% was reported compared with the previous methods by uniformly considering the same 15 real-life datasets. It was concluded that the proposed algorithm manifested an improvement over the previous state-of-the-art research. In this research study, a current, more extensive dataset (given in Table 2) was used for training and testing with five different classification methods: ANN, DT, KNN, Random Forest, and SVM. The distribution of data according to three classes is depicted in Figure 6. It can be observed in this figure that all three classes are clustered around the box widths (\(Y\)-axis) and lengths (\(X\)-axis) of 5-20 units and 5-25 units, respectively. Table 3 shows that the Random Forest Classifier achieved the highest accuracy on the test dataset, namely 89%, and the highest overall balanced accuracy, namely 88%. The Random Forest Classifier was closely followed by the Decision Tree Classifier, with an overall accuracy of 88% and an overall balanced accuracy of 87%. The Support Vector Machine Classifier achieved the lowest overall accuracy (74%) and the lowest overall balanced accuracy (74%). Moreover, the Random Forest Classifier also ranked first in terms of Cohen's Kappa, with K = 0.84, meaning that the model's prediction depends entirely on the actual classification. The Decision Tree Classifier ranked second in terms of the Cohen's Kappa metric, with K = 0.82. The Support Vector Machine Classifier also ranked last in terms of Cohen's Kappa, with K = 0.62. In terms of Matthews Correlation Coefficient, the Random Forest Classifier was the best performing model, showing that this classifier has high predictive power. The Decision Tree classifier ranked second (0.83), followed by the KNN (0.74), the ANN (0.66), and the SVM (0.63). Regarding cross entropy, all the classifiers achieved similar results, with the Support Vector Machine classifier showing a slightly higher cross-entropy of 1.67. In terms of sensitivity by class, the Random Forest Classifier achieved the best outcomes, reaching a sensitivity of 0.90 for class P1, 0.81 for class P2, and 0.94 for class P3. The Decision Tree classifier achieved comparable results, with a sensitivity of 0.90 for class P1, 0.8 for class P2, and 0.92 for class P3. The results are presented in Table 4. \begin{table}
\begin{tabular}{c c c c c c} \hline
**Method** & **Accuracy** & **Overall Balanced Accuracy** & **Cohen’s Kappa** & **Matthew’s Correlation Coefficient** & **Cross Entropy** \\ \hline ANN & 0.76 & 0.76 & 0.64 & 0.66 & 1.66 \\ Decision Tree & 0.88 & 0.87 & 0.82 & 0.83 & 1.61 \\ KNN & 0.82 & 0.81 & 0.73 & 0.74 & 1.65 \\ Random Forest & 0.89 & 0.88 & 0.84 & 0.84 & 1.61 \\ Support Vector Machine & 0.74 & 0.74 & 0.62 & 0.63 & 1.67 \\ \hline \end{tabular}
\end{table} 
Table 3: Evaluation of the ANN, Decision Tree, KNN, Random Forest, and Support Vector Machine on the test dataset in terms of global metrics. Figure 6: Distribution of the 1K data points according to class. Additionally, the Random Forest Classifier achieved the highest specificity for all three classes considered, namely 0.91 for class P1, 0.96 for class P2, and 0.97 for class P3. It is worth mentioning that all classifiers achieved the highest specificity by class for class P3. In terms of precision by class, the Random Forest Classifier ranked first, with class P1 having a precision of 0.84, class P2 having a precision of 0.88, and class P3 having a precision of 0.96. The best precision by class was achieved for class P3. The Random Forest performed best in terms of the F1-macro score as well, achieving an F1-macro score of 0.86 for class P1, 0.82 for class P2, and 0.95 for class P3.