## 3 Materials and Methods
### Informative Variable Identifier  
In our proposal we use a recently proposed FS method called IVI [13], which is capable of identifying the informative, redundant, and noisy variables. It transforms the input-variable space distribution into a coefficient-feature space by using existing linear classifiers or a more efficient weight generator. Informative features and their relationships are determined by analyzing the joint distribution of these coefficients with resampling techniques. IVI selects the informative variables and then it passes them on to some linear or nonlinear classifier. Experiments have shown that IVI can outperform state-of-the-art algorithms in terms of feature identification capabilities, and even in classification performance when subsequent classifiers are used. The IVI algorithm is built on the initial hypothesis that weights, \(\mathbf{w}\), learned by a linear classifier method, \(\mathbf{y}_{l}=\mathbf{w}^{T}\mathbf{x}_{l}+b\), are capable of summarizing the relationship between each feature. The IVI algorithm introduced in the original work [13] was implemented with the covariance multiplication estimator (CME) as a weight generation method designed to be competitive with the standard linear algorithms. CME is a low-computational-cost weight generator which is built on the relationships among input features, and on the relationships between input and output variables. Given an input dataset and a class variable, \(\{\mathbf{X},\mathbf{y}\}\), where \(\mathbf{X}\in R^{N\times L}\) is the input data matrix, containing the input set of vectors in rows, with \(N\) samples or observations of \(L\) features and \(\mathbf{y}\in\{-1,+1\}\). \(\mathbf{C_{XX}}\) and \(\mathbf{C_{XY}}\), denote the sample covariance matrices estimated as  
\[\mathbf{C_{XX}}=\frac{1}{N}\mathbf{X}^{T}\mathbf{X}\in R^{L\times L} \tag{10}\] \[\mathbf{C_{XY}}=\frac{1}{N}\mathbf{X}^{T}\mathbf{y}\in R^{L\times 1} \tag{11}\]  
The \(l\)-dimensional coefficient vector of CME is defined as follows:  
\[\mathbf{w}=\Big{(}sign(\mathbf{C_{XX}})^{(s-1)}\odot\mathbf{C_{XX}}^{(s)} \Big{)}\mathbf{C_{XY}} \tag{12}\]  
where \(\odot\) denotes the Hadamard product or element-wise product, and \(g\) is the integer exponent of the element-wise power. Through the use of the IVI algorithm, we selected different groups of relevant features. The IVI algorithm consists of three stages that can be summarized as follows. The first step of the IVI algorithm consists of estimating the statistical distribution of the weights learned by a given criterion for each input feature. For this purpose, we transformed the input-feature space into a weight space by resampling the dataset and computing one set of weights for each input feature, and this transformation is summarized in Algorithm 1. Resampling techniques were used: every group of samples was trained with a linear model (CME), which provided us with an estimation of the empirical distribution of any statistical magnitude that can be built computationally. We sampled \(N_{g}\) rows in \(\mathbf{X}\) and \(\mathbf{y}\) to get the \(b^{th}\) resampling \(\mathbf{X}_{(b)}^{*}\), \(\mathbf{y}_{(b)}^{*}\). For every resample, we estimated the statistical magnitude of interest which, in our case, was the weight vector \(\mathbf{w}_{(b)}^{*}\). By repeating this procedure \(B\) times, we got an estimate of the marginal empirical distribution:  
\[\widehat{f}_{\mathbf{w}}^{*}(\mathbf{w})=\frac{1}{B}\sum_{b=1}^{B}\delta\Big{(} \mathbf{w}-\mathbf{w}_{(b)}^{*}\Big{)} \tag{13}\]  
where \(\delta(\mathbf{w})\) denotes the usual Dirac's delta function. The results of the first step of the IVI algorithm are the feature weights which were then used to identify feature relevance and redundancy in Algorithm 1. ```
0: Training set \(\mathbf{X}\) and \(\mathbf{y}\), and number of resamples, \(B\). 0: Resampled weight matrix, \(\mathbf{W}^{*}\in R^{B\times L}\). 1:for\(b\gets 1\) to \(B\)do
2: Generate a random subset of the training set \(\mathbf{X}_{(b)}\) and \(\mathbf{y}_{(b)}\), with size \(L_{b}\).