# Feature-Weighted Sampling for Proper Evaluation of Classification Models
## 4 Discussion  
RBS is an efficient sampling method compared to the previous methods. The proposed FWS is a more improved method than RBS. Figure 10 shows how much more advanced FWS is than RBS. As shown in Figure 10a, the average _MAI_ value of FWS was 0.460, whereas that of RBS was 0.920. As we can see, the smaller the _MAI_, the better. Therefore, FWS improved MAI by 56% compared to RBS. Figure 10b compares the standard deviations of _the MAI_. The standard deviation of FWS was 0.403, whereas that of RBS was 0.779. In other words, this means the distribution of the _MAI_ value of the FWS was smaller than that of the RBS. FWS yielded more stable sampling results than RBS. Figure 10c shows the range of _MAI_. The range was calculated as (maximum of _MAI_) \(-\) (minimum of _MAI_). It also shows the distribution of _MAI_ values. The ranges of FWS and RBS were 2.359 and 4.619, respectively. The fluctuation range of the FWS was smaller than that of the RBS. All statistics in Figure 10 show that FWS is a more stable and accurate method than RBS. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Dataset** & **Classifier** & **MA** & **SD** & \begin{tabular}{c} **RBS** \\ **Accuracy** \\ \end{tabular}  & \begin{tabular}{c} **FWS** \\ **Accuracy** \\ \end{tabular}  & \ \begin{tabular}{c} **RBS** \\ **MAI** \\ \end{tabular}  &
\begin{tabular}{c} **FWS** \\ **MAI** \\ \end{tab  
Furthermore, it is proven that the similarity of distribution between the train/test sets and whole dataset is an important factor for their ideal splitting. In the development of the prediction model, the quality of the features determines the performance of the model. In general, the influence of features is greater than that of classification algorithms [17]. Therefore, considering the feature weight for distance calculation in the classification is reasonable. Figure 11 shows the influence of feature weighting in the FWS method. We compared FWS with and without feature weights. In the average of _MAI_, the \"without case\" was 0.634 whereas \"with case\" was 0.490 (Figure 10(a)). This means that feature weighting improved the performance of the FWS. In terms of the standard deviation, both cases were similar (Figure 10(b)). The ranges of \"with case\" and \"without case\" were 2.314 and 1.801, respectively (Figure 10(c)). This is because the maximum value of \"with case\" was large, which is less important than the standard deviation. We analyzed the variance in the _MAI_ according to the number of classes. In the result of RBS, the _MAI_ value of binary-class datasets was higher than that of multi-class datasets, whereas the difference was not large in FWS (Figure 12). This means that FWS is not influenced by the variance in the class number. FWS is a more stable method than RBS. Figure 11: FWS with and without feature-weighted distance. Figure 10: Comparison of RBS and FWS. In this study, we confirmed that the similarity of distribution between the original dataset and train/test sets is an important factor for accurate sampling. Furthermore, feature-weighted distance calculation can improve the sampling performance. If we use the proposed FWS for splitting train/test sets, we can more accurately evaluate the classification models. In our experiment, FWS performed better than RBS in 61 of 80 cases of train/test sets. This shows that the FWS has room for further improvement and is a topic for further research. Conceptualization, S.O. ; methodology, H.S. and S.O. ; software, H.S. ; validation, H.S. and S.O. ; formal analysis, H.S. and S.O. ; investigation, H.S. and S.O. ; resources, H.S. ; data curation, H.S. ; writing--original draft preparation, H.S. and S.O. ; writing--review and editing, H.S. and S.O. ; visualization, H.S. and S.O. ; supervision, S.O. ; project administration, S.O. ; funding acquisition, S.O. All authors have read and agreed to the published version of the manuscript. The present research was supported by the research fund of Dankook University in 2019. Not applicable. The authors declare no conflict of interest.