# Data Quality Measures and Efficient Evaluation Algorithms for Large-Scale High-Dimensional Data
## 3 Methods
### Methods for Efficient Computation
#### 3.3.2 Bootstrapping
Additionally, we fixed the hyperparameters of Algorithm 1 as \(B=100\), \(R=0.25\), and \(nv=10\) since there was no big difference in performance when larger hyperparameter values were used. For comparison with other quality measures, we chose \(F1\), \(N1\), and \(N3\) from Ho and Basu [1] and \(CSG\) from [PERSON] et al. [3]. Here, \(N1\), \(N3\), and \(CSG\) are known to be highly correlated with test accuracy of classifiers [PERSON] et al. [3]. \(F1\) is similar to our \(M_{sep}\) in its basic idea. Other quality measures suggested in [PERSON] and Basu [1]showed very similar characteristics to \(F1\), \(N1\), \(N3\), and \(CSG\) and are therefore not included in the results. ### Datasets

To evaluate the representative performance of \(M_{sep}\) for class separability, we used various image datasets that are high-dimensional and popular in mobile applications. We chose ten benchmark image datasets for our experiments: MNIST, notMNIST, CIFAR10, Linnaeus, STL10, SVHN, ImageNet-1, ImageNet-2, ImageNet-3, and ImageNet-4. MNIST [19] consists of ten handwritten digits from 0 to 9. The dataset contains 60,000 training and 10,000 test data points. We sampled 10,000 data from the training data for a model training and measuring the quality, and we sampled 2500 data from the test data for assessing the model accuracy. The notMNIST [20] dataset is quite similar to MNIST, containing English letters from A to J in various fonts. It has 13,106 training and 5618 test samples. We sampled the data in the same way as MNIST. Linnaeus [21] consists of five classes: berry, bird, dog, flower, and others. Although the dataset is available in various image sizes, we chose \(32\times 32\) to reduce the computation time of \(N1\), \(N3\), and \(CSG\). CIFAR10 [22] is for object recognition with ten general object classes. It consists of 50,000 training data and 10,000 test data points. We sampled the CIFAR10 dataset in the same way as MNIST. STL10 [23] is also for object recognition with ten classes, and it has \(92\times 92\) images: we resized the images into \(32\times 32\) to reduce the computation time for the \(N1\), \(N3\), and \(CSG\). The dataset consists of 5000 training and 8000 test data points. We combined these two sets into a single dataset, and then sampled 10,000 data points from the combined set for model training and measuring quality. We also sampled 2500 data points from the combined set for assessing prediction model accuracy if necessary. SVHN [24] consists of street view house number images. The dataset contains 73,200 data points. We sampled 10,000 training data for a model training and measuring the quality, and we sampled 2500 data for assessing the model accuracy. ImageNet-1, ImageNet-2, ImageNet-3 and ImageNet-4 are subsets of Tiny ImageNet dataset [25]. The Tiny ImageNet dataset contains 200 classes, and each class has 500 images. They are consist of randomly selected ten classes of the Tiny ImageNet dataset (total 5000 data points). We used 4500 data points for model training and measuring the quality and 500 data points for assessing the model accuracy, respectively. We summarized the details of datasets in Table 1. The accuracy values in the Table 1 are calculated from the MLP-2 model since it showed good overall performance compared to the perceptron and the MLP-1 models. ### Representation Performance of the Class Separability Measure \(M_{sep}\)

Here, we show in experiments that how well our first quality measure \(M_{sep}\) represents class separability, compared to simple but popular classifiers and the existing data quality measures.