## 4 Methodology
### The Proposed Feature Importance Fusion Framework
#### 4.1.2 Ensemble Strategies  
Within our MME Framework, the importance calculated is stored in a matrix, \(\overline{\overline{V}}\), and the ensemble strategy, which can be obtained in several ways, is used to determine the final feature importance values from \(\overline{\overline{V}}\). The most common ensemble strategy, as discussed in Section 2, is to use the average values. However, this is not the most suitable ensemble strategy in cases where one or more of the feature importance approaches produce outliers compared with the majority of responses. Therefore, in addition to the mean, we also investigate data fusion using median, mode, box-whiskers, majority vote, Modified Thompson Tau test, and our novel fusion approach RATE. The mode of a continuous probability distribution is any value at which its probability density function has a locally maximum value. By employing box-whisker plots, the anomalous feature importance is identified and excluded from the ensemble if its value is outside of 1.5 times the interquartile range (IQR). IQR is the difference between the third and first quartiles of the feature importance values. For majority vote, each vector in the feature importance matrix has their features ranked based on their importance. Subsequently, the final feature importance is the average of the most common rank order for each feature. For example, feature \(X_{i}\) has a final rank vector of \([1,1,1,2]\), where each rank \(r_{k}\) is established by a different feature importance method \(k\). The final feature importance value for feature \(X_{i}\) is the average value from the three feature importance methods that ranked it as one. Modified Thompson Tau test is a statistical anomaly detection method using t-test to eliminate values that are above two standard deviations.  
RATE is our novel fusion approach that combines the statistical test feature rank, with majority vote. Rank correlation is the calculation of the statistical dependence between two rank variables. In this case, the rank variables are the ranking of feature importance by each combination of ML model and feature importance method pair. RATE combines the advantage of using a statistical approach to rank feature importance and anomaly removal with majority vote. The steps used in RATE are illustrated in Figure 3. The input to RATE is the feature importance matrix, \(\overline{\mathcal{V}}\). \(\overline{\mathcal{V}}\) is the matrix that has the individual feature importance from different models, and it has the shape of \(N*M\), where \(N\) is the feature importance vectors from different importance calculation methods and \(M\) is the number of features. We calculate the pairwise rank correlation between each feature importance vectors in the matrix \(\overline{\mathcal{V}}\) to obtain the rank and the general correlation coefficient values [43]. Using the rank and correlation value, we determine whether the correlation between the vectors is statistically significant (\(p\)-value less than 0.05). The \(p\)-values are stored in a separate matrix that is converted to a truth table. If the pairwise \(p\)-value is less than 0.05, it is given a value of 'TRUE'; otherwise, it is 'FALSE'. Using the truth table along with majority vote, we determine which feature importance vector overall does not correlate with the majority of vectors and should be discarded. The remaining feature importance vectors are averaged as the final feature importance.