# Stability of Feature Selection in Multi-Omics Data Analysis
## 2 Materials and Methods
### Evaluation of the Classification Models  
Classification accuracy is a metric that measures how often a model correctly classifies instances in a dataset. It is calculated as the ratio of correctly predicted instances to the total number of instances. High accuracy indicates that the model's predictions closely match the actual labels, which is generally desirable.  
Classification accuracy is a widely used evaluation metric. However, for unbalanced datasets, where one class has significantly more observations than the other, classification accuracy can be misleading. High accuracy values may not reflect good model quality, as the model might only perform well for the majority class. In our study, several datasets are unbalanced. To address this, we use balanced classification accuracy, which accounts for the number of observations in each class in the evaluated sample. Throughout this paper, any reference to classification accuracy refers specifically to balanced classification accuracy.  
In the computational procedure, we used 5-fold cross-validation, where 4/5 of the observations were used to fit the classifier model, and the remaining 1/5 were used for evaluation. The training and evaluation process was repeated multiple times, each with a different split between training and testing, following cross-validation guidelines. From our perspective, when evaluating a classifier, it is more important to consider the classification accuracy on the test subset as an unbiased estimate, reflecting the expected performance on new observations when the models are applied in practice.  
The second parameter we present from our experiments is the stability of feature selection, expressed using the Nogueira measure (Equation (9)). This stability was evaluated based on the compositions of feature subsets selected by classifier models trained on 15 dataset splits (using 3 repetitions of 5-fold cross-validation) as shown in Figure 1. A higher stability value indicates greater reproducibility of the feature subset compositionsand stronger independence of the classifier model from variations in the training set, which is beneficial in practical applications.  
Recognizing that both feature-selection stability and classification accuracy are crucial for assessing the quality of a predictive model, we propose combining these two parameters into a single metric using a weighted harmonic mean \(F_{\beta}\).  
\[F_{\beta}=\frac{1+\beta}{\frac{1}{acc}+\frac{\beta}{fss}} \tag{10}\]  
where \(acc\) denotes the classification accuracy value, and \(fss\) denotes the feature-selection stability value obtained for the tested classifier model. The parameter \(\beta\) is a weighting factor that determines the contribution of feature-selection stability to the \(F_{\beta}\) metric. The relative impact of the two evaluation metrics, feature-selection stability, and classification accuracy, on the calculation of \(F_{\beta}\) is controlled by the value of \(\beta\). A \(\beta\) value of 1 indicates an equal contribution from both metrics. Values of \(\beta>1\) increase the weight of feature-selection stability in assessing model quality, while values of \(0<\beta<1\) place greater emphasis on classification accuracy. The appropriate choice of \(\beta\) depends on the specific problem and the relative importance of each evaluation metric.