# Impact of Dataset Size on Classification Performance:
## 4 Discussion
### Large Datasets  
Interestingly, the classifiers exhibited varying reactions to the different size reduction scenarios. We used a Mann-Whitney U test at \(p\) < 0.05 in order to assess the statistical significance of the differences in performances in different scenarios. In each test, we compare two groups of values that represent the performance of one model on a dataset of two sizes. Each group contains the performance of the model in ten folds. Tables 15-18 in the Appendix A show the resulting \(p\)-value for all classification models in each reduction scenario, which show whether the scenario caused a significant decrease in the model performance with respect to size measures.  
Statistical tests revealed that DT is the most sensitive model to the size of the dataset since its performance decreases significantly in the majority of the scenarios (\(\sim\)70% of scenarios in Table 19). RF and NN showed a relatively similar response to the decrease of dataset size as they show significant performance degradation in 44% and 42% of the scenarios in Tables 15 and 16, respectively. Tree-based models are trained by splitting the data based on predictor variables to find pure subsets (i.e., instances that belong to the same class) that will be used to compute the conditional probabilities. Therefore, the model's predictions are based on considerably smaller data than the original dataset. For NN, the model learns by adjusting a large number of weights using backpropagation. Thus, more data allows further adjustment, and hence better performance. The next model is SVM, where its performance decreases significantly in 36% of the scenarios in Table A6. As is well known, the position of the SVM hyperplane is based only on the support vectors. Consequently, the size of the dataset is irrelevant as long as the data include the support vectors. AB and NB exhibited robust performance as they decrease significantly only in 13% and 19% of the scenarios in Tables A10 and A11, respectively. Since NB is a simple algorithm that assumes conditional independence between variables, it needs less data to train. This makes it a high-bias model, but immune to the most common issue of small training set: overfitting.  
Together, these results provide important insights into dataset size and classifiers performance. First, in support to our previous observation, the overall performance of classifiers depends on the extent to which the dataset represents the original distribution rather than its size. Second, it is clear from our experiments and statistical tests that the most robust model for small medical datasets appears to be AB and NB, followed by SVM, and then NN and RF, while the least robust model is DT. Third, on comparing the classifiers performance on small datasets (Tables 5 and A1, Table A2, Table A3, Table A4, Table A5) and their performance in the three reduction scenarios of datasets size (Tables A6-A11 and Figures 2 and 3), an interesting observation can be made: a robust machine learning model to dataset size reduction does not necessary imply that it provides the best performance compared to other models. This is evident by the observation that AB and NB were the most robust models to dataset size reduction, but they had the least average accuracy on the small datasets in Table 5, compared to other models. In addition, as explained in Section 3.2, AB was the least performing classifier with respect to most performance metrics in both large datasets.