# Low-Cost Handheld Spectrometry for Detecting Flavescence Doree in Vineyards
## 2 Materials and Methods
### Ensemble-Based Feature Selection
#### 2.3.2 Classification Models  
Among the variety of existing machine learning classifiers, we have chosen five different linear and non-linear binary classification approaches for their potential in this application domain and to determine which one is most effective at recognizing FD disease, namely, simple logistic regression (LR), support vector machines (SVMs), gradient boosting (XGBoost), random forest (RF), and Cubist regression (Cubist) classifiers. * Logistic regression (LR): The LR classifier is a simple yet effective technique that takes advantage of logistic function to model dependent variables that are linearly related to the log odds for binary classification problems. After ranking the features in terms of relevance, the model can be employed to forecast the probability of the classes based on its input features [35]. For the purpose of describing the phenomenon of classification, it is useful to tie the likelihood of a class to a group of explanatory features. In order to avoid overfit, the \"L2\" penalty is exploited in addition to the \"C\" value hyperparameter governing the degree of regularization. LR is widely employed for binary classification tasks in the remote sensing community due to its simplicity, parallelizability, and high interpretability producing extremely competitive results [36]. * Support vector machines (SVMs): SVMs are known to be one of the most promising classification tools as they are widely used within the remote sensing community different classification and regression tasks [37; 38; 39]. Its main objective is to find an  
\begin{table}
\begin{tabular}{l} \hline \hline
**Genetic Algorithm for Feature Selection** \\ \hline \hline \multicolumn{2}{l}{//Initialize variables} \\ \(k=0\), _NbrTotalFeatures = 100_, _stop_criterion = False_, \(\alpha\) = 0.1, \\ _MaxNbrGeneration = 4000_, _N = TotalNumberOfGenes_, _M = NbrOfPopulation_ \\ \(\backslash\) / Initialize a number of \(P_{k}\) binary chromosomes with random binary values of genes \\ \(\backslash\) 1.Generate initial population \(P_{k=0}\) \\ _While (stop_criterion \(\
eq\) True)_ \\ \(\backslash\) \\ \(\backslash\) /compute the score of each chromosome in the population using fitness function \\ 2.Evaluation \\ \(\backslash\) /Evaluate \(\mathrm{P_{k}}\) the binary population chromosomes chosen genes \\ _For \(P_{k}^{j}\in P_{k}\)_ given that \(j\in\)_I_,.., _M1_ // For each binary chromosome in Population \\ \(\backslash\) / transoform the binary chosen chromoroses to the classification feature space \\ \(\backslash\) \  
optimal hyperplane that separates a given dataset. Moreover, to cope with nonlinear patterns, an SVM kernel function aims at mapping the original data into a higher new dimensional space in which finding an optimal separation hyperplane between data becomes linearly feasible in the new transformed space. In our case, the radial basis function (RBF) has been exploited as an SVMs kernel. RBF function is widely used within the machine learning community, due to its flexibility, and similarity to the Gaussian distribution, it can overcome the feature space complexity. We put forth the SVMs classifier in this work motivated by their highly efficient performance in hyperdimensional classification tasks. * Random forest classifier (RF): A decision tree is a supervised learning method based on a set of rules arranged in the shape of a tree for solving both classification and regression problems [31]. RF, as its name suggests, is an extension of the single decision tree yet better suited for a complex dataset. In order to cope with decision tree overfitting limitations, it takes advantage of exploiting averaging on a bunch of decision trees during the training phase. In doing so, it is possible to solve the problem with improved generalization. * Gradient boosting classifier (XGBoost): Due to its high effectiveness and accuracy, extreme gradient boosting is one of the most used machine learning techniques.