## 4 Results
### Comparison among Different Classifiers
#### 4.2.1 K-Nearest Neighbours  
Three hyperparameters are considered here to assess their effects on the accuracy of the results: number \(k\) of neighbours; metric to measure the distance between neighbours; weights for the computed distances [39]. In particular, we recall that, given a feature \(q_{f}\), (\(f=1,\ldots,F\)) to classify and a dataset of \(x_{if}\) training samples, \(i=1,\ldots,N\), \(N\) being the dataset dimension, the distance can be defined as:  
\[d(q,x_{i})=\sum_{f=1}^{F}w_{f}\,\delta(q_{f},x_{if}) \tag{4}\]  
where \(w_{f}\) represents the weight adopted for the \(f\)-th feature, and \(\delta\) is the distance metric; relevant details are going to be provided in what follows. To improve the performance of the algorithm, a Bayesian, grid search, or random search optimization is adopted. Principal component analysis (PCA) [41] is also exploited in order to reduce the problem dimensionality. Results are gathered in Table 3 for seven different models characterized by a varying \(k\), which are representative of the entire set of analyses carried out, to get details about the effect of each hyperparameter varied in the investigation. The distance weights are selected to be either the inverse or the squared inverse of the distance between the data points, see [39]. The distance metric is assumed in most of the cases to be the city block one, given by \(\delta=\sum_{i=1}^{N}|q_{f}-x_{if}|\), with \(|\square|\) representing the modulus of the quantity \(\square\). For the highest values of \(k\) in the Table 3, results are also  
Figure 12: Patterns in the accuracies for 1000 randomized training and testing datasets, as obtained via the DT algorithm backed by a Bayesian optimizer. reported for the Euclidean distance metric and its generalization (Mahalanobis distance), see again [39] for details. The training accuracy varies between 80% and 91.7%, while the testing accuracy between 94.4% and 97.2%. It must be kept in mind that the training dataset is composed of 108 forms, so each one represents around the 0.9% of the solutions; the test dataset is instead composed of 36 forms, and each one indeed represents around the 2.8% of the corresponding solutions. Hence, for instance, showing that the accuracy is 97% during testing means that only one form is misclassified; the accuracies have accordingly to be judged on the basis of this kind of digital variation, with finite jumps in the classification outcome due to the limited number of available instances. In order to also assess the efficiency of the algorithm, results are reported in terms of the computing time required for the classification, which varies between 16.3 s to 64.6 s.  
The comparison among the seven models shows that, by changing the optimization scheme from Bayesian (model K2) to random search (K3), the training and testing accuracies are the same, but the computing time decreases from 38.1 s to 15.9 s. This is caused by the fact that, if the number of iterations is a priori set, the Bayesian optimizer is always recomputed (this requires a bit of time) according to the continuous update granted by the Bayes rule, while in a random search the next iteration in the convergence towards the optimal solution is simply randomly extracted. By increasing the number of iterations from 30 to 50, see model K5, the testing accuracy shows a positive trend, increasing from 94.4% to 97.2%; the computing time also increases significantly, from 38.1 s to 64.5 s, as expected. The grid search algorithm with 10 divisions is adopted in model K1: the accuracy is similar to the K2 and K3 ones, but the computing time turns out to be far larger, testifying that this optimization algorithm is not effective and efficient for the present case study.