# Application of Machine Learning Methods for Pallet Loading Problem
## 3 Methodology
### Evaluation Metrics  
The evaluation metrics are typically used to quantify, analyze, and test the success of classification and estimation methods [56]. Global metrics such as overall accuracy, overall balanced accuracy, Cohen's Kappa, Matthew's Correlation Coefficient extended to multi-class, and confusion entropy ensure the quantification of multi-class classification and enable the comparison of various classification methods and techniques [57]. In the classification process, the ability of a model to perform a correct classification is quantified with the accuracy value [58], which is directly computed from the confusion matrix [57]. \[\text{Accuracy}=\frac{TP+TN}{TP+TN+FP+FN} \tag{4}\]  
Accuracy is a very intuitive metric that enables an overall understanding of the performance of a model on the test data [57]. However, when dealing with imbalanced datasets, accuracy has a tendency to conceal strong classification errors for classes with a limited number of units [57]. In this case, accuracy does not enable the identification of classes for which the model has suboptimal performance [57]. In the case of imbalanced datasets, balanced accuracy ensures that the prediction accuracy is not inflated because one or several classes have results that dominate the other classes [59]. This measure represents the arithmetic mean of the performance of a model for each class [59]. It is a balanced metric in the sense that it assigns the same weight and thus the same importance to each class studied [59]. \[\text{Balanced Accuracy}=\frac{1}{C}\times\sum\frac{p_{i}}{n_{i}}, \tag{5}\]  
where \(p_{i}\) is the correctly predicted number of data items in class \(i\) and \(n_{i}\) is the true number of data items in class \(i\). As can be inferred from the formula below, the Matthews Correlation Coefficient uses all the entries of the confusion matrix both in the numerator and in the denominator. Because of this fact, it can generally be considered a balanced indicator of performance [59]. This measure takes values in the range [1, 2, 1], with values close to 1 indicating a strong prediction and correlation between the predicted and true labels. Values close to \(-1\) indicate an inverse relationship between the predicted and true labels of the dataset and reflect systematic errors in the modeling process. In the case of values close to 0, the classifier makes random decisions when assigning the labels to the classes [59]. Matthews Correlation Coefficient for multi-class classification:Given multi-class Confusion Matrix C:  
\[K=\frac{c\times s-\sum p_{k}\times t_{k}}{\sqrt{\left(s^{2}-\sum p_{k}^{2}\right) \times\left(s^{2}-\sum t_{k}^{2}\right)}} \tag{6}\]  
\(c=\sum C_{kk}\) the total number of elements correctly predicted; \(s=\sum\sum C_{ij}\) the total number of elements; \(p_{k}=\sum C_{ki}\) the total number of times class \(k\) was predicted; \(t_{k}=\sum C_{ik}\) the total number of times that class \(k\) actually occurs. Cohen's Kappa for multi-class cases:  
The Cohen's Kappa Score possesses similarities to the Matthews Correlation Coefficient in the case of multi-class classification and leverages the expected accuracy, a measure that reflects the dependence obtained by chance between the predicted and the true labels [59]. The coefficient quantifies the agreement between the predicted and actual classes as follows: when \(K=0\), the model's prediction is independent of the actual classification, when \(K=1\), the model's prediction depends entirely on the actual classification, and when \(K=-1\), there is no agreement between the model's prediction and the actual classification [59]. Given multi-class Confusion Matrix C:  
\[K=\frac{c\times s-\sum p_{k}\times t_{k}}{s^{2}-\sum p_{k}\times t_{k}} \tag{7}\]  
\(c=\sum C_{kk}\) the total number of elements that were correctly predicted; \(s=\sum\sum C_{ij}\) the total number of elements; \(p_{k}=\sum C_{ki}\) the total number of times class \(k\) was predicted; \(t_{k}=\sum C_{ik}\) the total number of times that class \(k\) actually occurs.