## 4 Methodology
### Features Ranking  
The concept of this examination is to find the best set of features on the basis of the joint results of six different algorithms. In other words, the outcome of many feature rankings is combined to construct one final ranking.  
A batch of systematic experiments was performed to create feature rankings for each classifier. The aim of these experiments was to find optimal hyperparameters of each classifier, maximizing the chosen performance measure. For this purpose, the grid search approach was used. After each experiment, its' evaluation metrics were saved. Then, all approaches for a chosen classifier were compared in order to pick the one with the highest classification results. This phase of the research is displayed on the left side of Figure 1. The very first set of boxes on this chart represents all random forest tests, among which the most efficient one was chosen to focus on creating its ranking of features--\"Ranking of the best RF classifier\". A similar procedure was used to find rankings of features of the best variant of all other classifiers.  
As a final result of tests with six classifiers, described in the previous section, one has six optimized (i.e., with a set of parameters that offers the highest classification quality) classifiers. The validation of each classifier is performed with one of three evaluation measures by computing the test dataset. The three measures, accuracy, recall, and precision, are explained in Equation (1):  
\[accuracy=\frac{TP+TN}{TP+TN+FP+FN},\qquad recall=\frac{TP}{TP+FN},\qquad precision=\frac{TP}{TP+FP} \tag{1}\]  
where \(TP\) stands for the number of all samples from a specific class correctly classified, \(TN\) is the number of all samples from other classes correctly classified as not belonging to the specific class, \(FP\) is the number of all samples from other classes wrongly classified as the specific class, and \(FN\) stands for the number of samples of a specific class that are not classified as this class but as any other class.  
Apart from the final measures describing the quality, these six classifiers are also feature selectors--each of them produces an individual ranking of features. There are the six best rankings: one formed by random forest, one by Adaboost, etc. A weighted average position for each feature was created to build the final ranking. This position was based on the feature's exact positions in all six rankings. The weight of each component ranking position in these calculations is the accuracy of the classifier producing the given component ranking (see Figure 1):  
\[r_{i}=\sum_{j=1}^{6}(r_{max}-r_{i,j})\cdot q_{j}, \tag{2}\]  
where \(r_{i}\) stands for the final inverted ranking of the \(i\)-th feature, \(r_{i,j}\) is the component ranking of the \(i\)-th feature produced by the \(j\)-th classifier, \(r_{max}\) is the maximum rank value, and \(q_{j}\) is the quality measure (accuracy, precision, or recall) \(j\)-th among 6 classifiers (\(j=1,...6\)).  
Due to the inversion of ranks in Equation (2), the final weighted ranking is also inverted. So, at the end, features are sorted in descending order. The sorted list is an ultimate ranking of features, where the lower an index of features, the higher its importance.  
The novelty of this study, compared to the reviewed literature, is the use of a set of optimal classifiers to rank features, which, in this case, consists of six models. The selection of optimal classifiers involves checking which tuned parameters achieve the highest evaluation metrics.