# Data Quality Measures and Efficient Evaluation Algorithms for Large-Scale High-Dimensional Data
## 3 Methods
### Methods for Efficient Computation
#### 3.3.2 Bootstrapping
In addition, \(M_{var}\) and \(M_{var_{i}}\) measures in-class variability, which is another important factor to avoid unwanted inductive bias in trained models. Conceptualization, H.C. and S.L. ; methodology, H.C. and S.L. ; validation, H.C.; writing-original draft preparation, H.C.; writing-review and editing, S.L. ; supervision, S.L. All authors have read and agreed to the published version of the manuscript. Figure 5: Data quality plot using the two proposed quality measures. **Funding:** This research was supported by Basic Science Research Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Education(2018R1D1A1B07051383), and also by the MSIT(Ministry of Science and ICT), Korea, under the ITRC(Information Technology Research Center) support program(IITP-2020-0-01749) supervised by the IITP(Institute of Information & Communications Technology Planning & Evaluation). **Data Availability Statement:** The data presented in this study are openly available: MNIST ([[http://yann.lecun.com/exdb/mnist/](http://yann.lecun.com/exdb/mnist/)]([http://yann.lecun.com/exdb/mnist/](http://yann.lecun.com/exdb/mnist/))), notMNIST ([[https://www.kaggle.com/lubaroli/notmnist](https://www.kaggle.com/lubaroli/notmnist)]([https://www.kaggle.com/lubaroli/notmnist](https://www.kaggle.com/lubaroli/notmnist))), Linnaeus ([[http://chaladze.com/15/](http://chaladze.com/15/)]([http://chaladze.com/15/](http://chaladze.com/15/))), CIFAR10 ([[https://www.cs.toronto.edu/~kriz/cifar.html](https://www.cs.toronto.edu/~kriz/cifar.html)]([https://www.cs.toronto.edu/~kriz/cifar.html](https://www.cs.toronto.edu/~kriz/cifar.html))), STL10 ([[https://ai.stanford.edu/~acoates/stl10/](https://ai.stanford.edu/~acoates/stl10/)]([https://ai.stanford.edu/~acoates/stl10/](https://ai.stanford.edu/~acoates/stl10/))), SVHN ([[http://ufldl.stanford.edu/housenumbers/](http://ufldl.stanford.edu/housenumbers/)]([http://ufldl.stanford.edu/housenumbers/](http://ufldl.stanford.edu/housenumbers/))), and ImageNet ([[https://www.kaggle.com/c/tiny-imagenet](https://www.kaggle.com/c/tiny-imagenet)]([https://www.kaggle.com/c/tiny-imagenet](https://www.kaggle.com/c/tiny-imagenet))). **Conflicts of Interest:** The authors declare no conflict of interest. ## References

* [PERSON] and [PERSON] (2002) [PERSON]; [PERSON] Complexity measures of supervised classification problems. _IEEE Trans. Pattern Anal. Mach. Intell._**2002**, _24_, 289-300. * [PERSON] and [PERSON] (2006) [PERSON]; [PERSON] [PERSON] Data complexity assessment in undersampled classification of high-dimensional biomedical data. _Pattern Recognit. Lett._**2006**, _27_, 1383-1389. [CrossRef]
* [PERSON] et al. (2019) [PERSON] F.; [PERSON]; [PERSON] Spectral metric for dataset complexity assessment. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 15-21 June 2019. * [PERSON] et al. (2020) [PERSON]; [PERSON]; [PERSON] Fisher Discriminant Analysis (Linear Discriminant Analysis). _Dict. Bioinform. Comput. Biol._ Available online: [[https://doi.org/10.1002/9780471650126.dob0238.pub2](https://doi.org/10.1002/9780471650126.dob0238.pub2)]([https://doi.org/10.1002/9780471650126.dob0238.pub2](https://doi.org/10.1002/9780471650126.dob0238.pub2)) (accessed on 25 October 2020). * [PERSON] and [PERSON] (2001) [PERSON]; [PERSON] Random projection in dimensionality reduction: Applications to image and text data. In Proceedings of the ACM Special Interest Group on Knowledge Discovery in Data, San Francisco, CA, USA, 26-29 August 2001. * [PERSON] (1979) [PERSON] Bootstrap Methods: Another look at the jackknife. _Ann. Stat._**1979**, \(7\), 1-26. [CrossRef]
* [PERSON] and [PERSON] (2008) [PERSON]; [PERSON] _An Introduction to Kolmogorov Complexity and Its Applications_; Springer: Berlin, Germany, 2008. * [PERSON] et al. (2015) [PERSON]; [PERSON]; [PERSON] A set of complexity measures designed for applying meta-learning to instance selection. _IEEE Trans. Knowl. Data Eng._**2015**, _27_, 354-367.