## 2 Materials and Methods
### Methodology
#### 2.2.3 Classifiers
In the case of _dual_ = \"_False_\", the solver used is based on the primal formulation, and although we had fewer features than samples, the classifier performed better with this configuration than setting the _dual_ = \"_True_\". Furthermore, we utilized the coefficient \(C\) to tune the regularization strength. If \(C\) is too small, the model might overfit the training data, and if it is set too high, it might overly penalize the weights, leading to underfitting. In our case, the selection of the parameter \(C\) was \(C=2\). Logistic Regression Classifier  
Logistic Regression is a popular algorithm for binary classification. It models the probability that a given case belongs to a particular category. The logistic function (known also as sigmoid) is utilized to map predicted values into probabilities [56]. We used L2 penalization as in the SVM classifier; again, the dual parameter was selected as \"False\". Our choice of the solver was \"liblinear\" since we had a relatively small dataset (\"saga\" and \"sag\" are faster for large datasets). The tolerance parameter for stopping criteria (_tol_) was selected as the default option and the \(C\) parameter, which regulates the regularization strength, was selected as a float number \(C=2.0\). The Naive Bayes Classifier is a probabilistic machine learning algorithm, based on Bayes' Theorem, with a naive assumption that features are conditionally independent, given the class label. Although, at first glance, such an oversimplification looks \"naive\", indeed, the classifier performs well in most cases [57]. In our study, we used the _GaussianNB classifier_[54], where the likelihood of the features is assumed to be Gaussian. _The scikit-learn library_ provides a simple interface for creating neural network models through the \"MLPClassifier\" class, i.e., the Multi-layer Perceptron classifier [54]. This model optimizes the log-loss function using \"LBFGS\" or stochastic gradient descent. In our model, we used 1 hidden layer with 6 neurons (as the number of features), 11 neurons, and 100 neurons, and the activation function was the rectified linear unit function, i.e., \"relu\". The solver for weight optimization was \"adam\", which is a stochastic gradient-based optimizer. This selection works best for large datasets, so we had faster convergence and better performance using \"lbfgs\" as the solver, which is an optimizer in the family of quasi-Newton methods. Furthermore, for the strength of L2 regularization, the _alpha_ parameter was set to _alpha_ = 0.0001. The maximum number of iterations _max_iter_ parameter was set to 10,000 in order to achieve convergence since the algorithm performs more iterations. The solver iterates until convergence. After conducting a sufficiently large number of independent computer experiments, the best results were acquired when we utilized 11 neurons in the hidden layer. Evaluation of Classifiers  
For all seven classifiers, the overfitting factor was evaluated according to Equation (1), which quantifies the relative difference between training and testing accuracies [58]. In general, the overfitting factor is a metric used to quantify the degree of overfitting in a machine learning model. Overfitting occurs when a model learns the training data too well, capturing noise and details that are specific to the training set, but may not be able to generalize well to new unseen data. Furthermore, a higher overfit factor is an indication that the classifier performs well on the training set but there might be overfitting. A negative value may suggest that the model generalizes rather well on the test set. \[\text{Overfit Factor}=\frac{\textit{Train Acc}-\textit{Test Acc}}{\textit{Train Acc}} \tag{1}\]  
Once again it is obvious that the best results from the overfitting point of view are acquired when the dataset split is about 70% training set and 30% test set, according to the common consensus. It is worth mentioning that a Genetic Algorithm may optimize these results, and a better outcome may be predicted for other portioning as well.