## 6 Classification Models
Hence, a gradient descent algorithm is used to learn the network's weights and minimize a loss function on the given training set. In this work we considered three MLP models: MLP\({}_{1}\) is a shallow neural network with only one hidden layer, MLP\({}_{2}\) uses two hidden layers, while MLP\({}_{3}\) is the deeper neural network here considered which has three hidden layers. After some preliminary experiments, we chosen the following setting: 25 artificial neurons for each hidden layer, the _relu_ as activation function, the _cross-entropy_ as loss function, and the popular _Adam_ variant of the stochastic gradient descent algorithm. \begin{table}
\begin{tabular}{l l} \hline \hline
**Model** & **Description** \\ \hline RF & Random Forest classifier \\ \hline GBDT & Gradient Boosted Decision Tree classifier \\ \hline SVM & Support Vector Machine with radial basis function as kernel \\ \hline MLP\({}_{1}\) & Multi-Layer Perceptron with one hidden layer \\ \hline MLP\({}_{2}\) & Multi-Layer Perceptron with two hidden layers \\ \hline MLP\({}_{3}\) & Multi-Layer Perceptron with three hidden layers \\ \hline QDA & Quadratic Discriminant Analysis classifier \\ \hline NB & Naive Bayes classifier which fits Gaussian distributions \\ \hline kNN\({}_{\text{U}}\) & K-Nearest Neighbors with uniform weights \\ \hline kNN\({}_{\text{D}}\) & K-Nearest Neighbors with weights set on the basis of the distance in the features space \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Classification models considered in this work. The quadratic discriminant analysis (QDA) [66] and the naive Bayes (NB) [67] are two classifiers based on Bayesian probability theory [62]. QDA learns, for each target class, a multivariate Gaussian probabilistic model of the class conditional distribution on the given training set. Then, predictions are obtained using the Bayes rule and selecting the class that maximizes the posterior probability. Note that, QDA was preferred to its linear counterpart after performing some preliminary experiments. Regarding the NB classifier, it learns a simplified Bayesian network model of the given training set and uses the learned model to perform probabilistic predictions. Since in this work we are considering a real-valued dataset, the Gaussian-based NB model was adopted. Hence, it is interesting to note that, under this setting, NB can be seen as a simplified version of QDA where the learned covariance matrices are diagonal. The last method considered is K-nearest neighbors (kNN) [68], which does not learn a proper model of the class distributions but, conversely, memorizes the training set and performs predictions by outputting the most common target class among the \(k\) nearest training vectors to the (unlabeled) query vector. Therefore, kNN training only consists in organizing the training vectors in a suitable data structure for speeding up the computation of the \(k\) nearest vectors in the prediction phase [69]. This method was used in this work because our dataset is not huge. Moreover, two different variants of kNN were considered: kNN\({}_{\text{U}}\) and kNN\({}_{\text{D}}\). In kNN\({}_{\text{U}}\) all the neighbors count equally, while in kNN\({}_{\text{D}}\) their votes are weighted by the inverse distance from the query vector. We used the classical Euclidean distance function and, after some preliminary experiments, we set the number of neighbors \(k\) to 7.