# Data Quality Measures and Efficient Evaluation Algorithms for Large-Scale High-Dimensional Data
## 3 Methods
### Proposed Data Quality Measures
#### 3.2.2 In-Class Variability  
Our second data quality measure gauges the in-class variability. Figure 1 shows one of the motivating examples to consider in-class variability for data quality. In the figure, we have two photos of the Bongeunsa temple in Seoul, Korea, taken by the same photographer. The photographer had been asked to take photos of Korean objects from several different angles, and it turned out that quite a few of the photos were taken in only marginal angle differences. Since the data creation was a government-funded project providing data for developing object recognition systems in academia and industry, low data variability was definitely an issue. Here, we define two types of in-class variability measure, the overall in-class variability of a given dataset \(M_{par}\) and the in-class variability of the \(i\)-th class, \(M_{var_{i}}\). First, the overall in  
Figure 1: An example of low in-class variability that similar images in the same class. The images are Bongeunsa temple in Seoul, Korea. (Source: Korean Type Object Image AI Training Dataset at [[http://www.aihub.or.kr/aidata/132](http://www.aihub.or.kr/aidata/132)]([http://www.aihub.or.kr/aidata/132](http://www.aihub.or.kr/aidata/132)), National Information Society Agency.) class variability \(M_{var}\) tries to capture the minimum variance of data points being projected onto any direction, based on the matrix \(\hat{S}_{w}\) defined in (2):  
\[M_{var}:=\min_{v\in\mathbb{R}^{n}:\|v\|=1}\frac{1}{c\cdot n}v^{T}\hat{S}_{w}v,\]  
where \(c\) is the number of class and \(n\) is the dimension of data. Unlike class separability, we added additional normalization factors \(c\) and \(n\), since the value \(\hat{S}_{w}\) is affected by the number of class and the data dimension. Second, the class-wise in-class variability \(M_{var_{i}}\) is based on the sample covariance matrix of each class:  
\[\hat{S}_{w_{i}}:=\frac{1}{m_{i}}\sum_{j=1}^{m_{i}}(x_{i,j}-\overline{x}_{i})( x_{i,j}-\overline{x}_{i})^{T},\]  
where \(m_{i}\) is the number of data points in the \(i\)-th class. The class-wise in-class variability measure \(M_{var_{i}}\) is defined as follows:  
\[M_{var_{i}}:=\min_{v\in\mathbb{R}^{n}:\|v\|=1}\frac{1}{c\cdot n}v^{T}\hat{S}_{ w_{i}}v\enspace.\]  
The normalization factors \(c\) and \(n\) are required for the same reason as in \(M_{var}\). The measure \(M_{var_{i}}\) represents the smallest variance of the data points in the same class after being projected onto any direction. As a matter of fact, \(M_{var}\) and \(M_{var_{i}}\) are the same as the smallest eigenvalue of \(\frac{1}{c\cdot n}\hat{S}_{w}\) and \(\frac{1}{c\cdot n}\hat{S}_{w_{i}}\) which can be computed for instance using the Lanczos algorithm Lanczos (1966) on the inverse of them with \(\mathcal{O}(n^{3})\) computation, which will be preventive for large data dimensions \(n\). We discuss a more efficient way to estimate the value in the next section, which can be computed alongside with our first data quality measure without significant extra cost. Using the \(M_{var}\) and \(M_{var_{i}}\), we can analyze the variety or redundancy in a given dataset. For instance, a very small \(M_{var}\) and \(M_{var_{i}}\) would indicate that we may have a small diversity issue where data points in the invested class are mostly alike. On the other hand, the overly large \(M_{var}\) and \(M_{var_{i}}\) may indicate a noise issue in the class, possibly including incorrect labeling. The difference between \(M_{var}\) and \(M_{var_{i}}\) is that the \(M_{var}\) aggregates the information of diversity for each class, and the \(M_{var_{i}}\) represents the information of diversity for a specific class. Since the \(M_{var}\) aggregates the information of variability of data points in each class, we can use this for comparing the in-class variability between datasets. On the other hand, we can use \(M_{var_{i}}\) for the datasets analysis, i.e., data points of a specific class with less \(M_{var_{i}}\) than other classes may cause low generalization performance.