## 2 Background
### Random Forest for Feature Selection  
Random Forest (RF) is a machine learning algorithm designed by [PERSON] in 2001 [24] for classification and regression. RF uses multiple decision trees to classify a sample; each decision tree is built by using bootstrap sampling with a random feature selection implementation. Their predictions are used in a voting  
\begin{table}
\begin{tabular}{l l} \hline Mean of spectrum & \(F1=\frac{\sum_{k=1}^{K}X(k)}{K}\) \\ \hline Variance of spectrum & \(F2=\frac{\sum_{k=1}^{K}(X(k)-F1)^{2}}{K-1}\) \\ \hline Skewness of spectrum (Skewnessf) & \(F3=\frac{\sum_{k=1}^{K}}{(X(k)-F1)^{3}}K(\sqrt{F2})^{3}\) \\ \hline Kurtosis of spectrum & \(F4=\frac{\sum_{k=1}^{K}(X(k)-F1)^{4}}{K(F2)^{4}}\) \\ \hline Central Frequency & \(F5=\frac{\sum_{k=1}^{K}f_{k}X(k)}{\sum_{k=1}^{K}X(k)}\) \\ \hline STD of spectrum & \(F6=\sqrt{\frac{\sum_{k=1}^{K}}{(f_{k}-F5)^{2}X(k)}\sum_{k=1}^{K}X(k)}\) \\ \hline RMS of spectrum & \(F7=\sqrt{\frac{\sum_{k=1}^{K}f_{k}^{2}X(k)}{X(k)}}\) \\ \hline CP1 & \(F8=\frac{\sum_{k=1}^{K}(f_{k}-F5)^{3}X(k)}{K(F6)^{3}}\) \\ \hline CP2 & \(F9=\frac{F6}{F5}\) \\ \hline CP3 & \(F10=\frac{\sum_{k=1}^{K}(f_{k}-F5)^{\frac{1}{2}}X(k)}{K\sqrt{F6}}\) \\ \hline CP4 & \(F11=\frac{\sum_{k=1}^{K}(f_{k}-F5)^{3}X(k)}{F6^{K}K}\) \\ \hline CP5 & \(F12=\sqrt{\frac{\sum_{k=1}^{K}f_{k}^{4}X(k)}{\sum_{k=1}^{K}f_{k}^{2}X(k)}}\) \\ \hline Centroid of Spectrum & \(F13=\frac{\sum_{k=1}^{K}kX(k)}{\sum_{k=1}^{K}X(k)}\) \\ \hline Spectrum Spread & \(F14=\sqrt{\frac{\sum_{k=1}^{K}(-F13)^{2}X(k)}{\sum_{k=1}^{K}X(k)}}\) \\ \hline Entropy of spectrum & \(F15=-\sum_{k=1}^{K-1}P_{n}(k)log_{2}[P_{n}(k)]\) \\  & where \(P_{n}\) is the normalized total spectral energy \\  & \(P_{n}(k)=\frac{X(k)}{\sum_{k=1}^{K}X(k)}\) \\ \hline Total power & \(F16=\sum_{k=1}^{K}P(k)\) \\ \hline Median Frequency (MDF) & \(F17=\frac{1}{2}\sum_{k=1}^{K}P(k)\) \\ \hline Peak frequency (PKF) & \(F18=max(P(k)),\quad k=1,...K.\) \\ \hline First Spectral Moment & \(F19=\sum_{k=1}^{K}P(k)f_{k}\) \\ \hline Second Spectral Moment & \(F20=\sum_{k=1}^{K}P(k)f_{k}^{2}\) \\ \hline Third Spectral Moment & \(F21=\sum_{k=1}^{K}P(k)f_{k}^{3}\) \\ \hline Fourth Spectral Moment (SM4) & \(F22=\sum_{k=1}^{K}P(k)f_{k}^{4}\) \\ \hline Spectral moment ratio (VCF) & \(F23=\frac{F20}{F16}-(\frac{F19}{F16})^{2}\) \\ \hline Frequency ratio (FR) & \(F24=\sum_{LLC=f_{\min}}^{ULC=f_{\max}/2}P(k)/\sum_{LHC=\frac{f_{\max}}{2}+1}^{ UHC=f_{\max}}P(k)\) \\ \hline \end{tabular}
\end{table}
Table 1: Features.  
system where, from all trees, a majority class is calculated. The out-of-bag (OOB) error is mostly used to compute the expected model generalization performance [25].  
Feature Selection by MDA  
Given a data set \(\mathbf{D_{n}}=\{(\mathbf{X_{1}},Y_{1}),\ldots,(\mathbf{X_{n}},Y_{n})\}\) of \(n\) samples and \(p\) independent variables with \(\mathbf{X_{i}}=(X_{i}^{(1)},\ldots,X_{i}^{(p)})(i\in 1,\ldots,n)\) being a training sample, the importance of the j-th feature \(\mathbf{X}^{(j)}=X_{1}^{(j)},\ldots,X_{n}^{(j)}(j\in 1,\ldots,p)\), is calculated averaging the OBB errors of all permutations of trees [25].  
Denote \(D_{l,n}\) the out-of-bag data set of the \(l-th\) tree and \(D_{l,n}^{j}\) the same data set, where the values of \(X^{(j)}\) have been randomly permuted. Keeping in mind that the \(m_{n}(\cdot;\Theta_{l})\) represents the \(l\)-th tree estimate and where \(\Theta_{1},...,\Theta_{M}\) are independent random variable used to resample the training set before the growth of individual trees, MDA takes the form  
\[MDA(X^{(j)})=\frac{1}{M}\sum_{l=1}^{M}\Big{\{}R_{n}[m_{n}(\cdot;\Theta_{l}),D_ {l,n}^{j}]-R_{n}[m_{n}(\cdot;\Theta_{l}),D_{l,n}]\Big{\}} \tag{2}\]  
where \(R_{n}\) is defined for \(D=D_{l,n}\) or \(D=D_{l,n}^{j}\) by  
\[R_{n}\left[m_{n}(\cdot;\Theta_{l}),D\right]=\frac{1}{|D|}\sum_{i:(\mathbf{X_{i }},Y_{i})\in D}\left[Y_{i}-m_{n}(\mathbf{X_{i}};\Theta_{l})\right]^{2} \tag{3}\]