# A Study on Data Profiling: Focusing on Attribute Value Quality Index
## Materials and Methods
### Data Quality Diagnosis
#### 2.3.1 Calculation of Data Quality Errors  
The value which diagnoses the quality level of the database is diagnosed from the viewpoint of data value, structure, and standardization, and the result is formulated and quantified as the data quality error rate [8]. Equation (1) is a calculation formula that applies a weight to the data value, structure, and standardization, which are quality factors of the data quality error rate [8].  
\[Quality\;error\;rate=\;\sum_{i=1}^{n}(E_{i}\times W_{i}), \tag{1}\]  
where \(E\) is the error rate per quality factor, and \(W\) is the weight per quality factor. Value (accuracy) error rate (\(E_{1}\)) refers to error level for the value of data. Structure (completeness) error rate (\(E_{2}\)) refers to the degree that the structure of a database is not faithful. Normalization (consistency) error rate (\(E_{3}\)) refers to the degree to which the conformity to a database standard is insufficient. The weight of value (\(W_{1}\)), weight of structure (\(W_{2}\)), and weight of normalization (\(W_{3}\)) are 0.7, 0.1, and 0.2, respectively [8].  
\[Value\;(accuracy)\;error\;rate\;(\%)=\;E_{1}\;=\;\frac{\sum_{i=1}^{n}e_{i}}{ \sum_{i=1}^{n}s_{i}}\;\times 100, \tag{2}\]  
where \(i\) is a value diagnostic item, \(s\) is the number of total data, and \(e\) is the number of error data.  
\[Structure\;(completeness)\;error\;rate\;(\%)=\;E_{2}=\;\frac{1}{n}\;\times\;\sum_{i =1}^{n}e_{i}, \tag{3}\]  
where \(n\) is the number of structural diagnosis items, \(i\) is the structural diagnosis item, and \(e\) is the error rate of each structural diagnosis item.  
\[Normalization\;(consistency)\;error\;rate\;(\%)=\;E_{3}=\;\frac{1}{n}\;\times\; \sum_{i=1}^{n}e_{i}, \tag{4}\]  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Quality Diagnosis Method** & **Method Explanation** \\ \hline \multirow{4}{*}{Value diagnostic profiling} & \(\circ\) The method to analyze the data value error itself, such as the \\  & validity and accuracy of the data value \\  & - Diagnosis centered on the accuracy of data values through column \\  & analysis, date analysis, pattern analysis, and code analysis \\  & \(\circ\) The method to diagnose the error of unstructured data, such as \\  & documents, images, or videos, through a human’s manual \\ \multirow{2}{*}{Unstructured survey} & confirmation (actual measurement) \\  & - Views information directly or manually checks the document \\  & without separate tools \\ \hline \hline \end{tabular}
\end{table}
Table 2: Quality diagnosis methods.  
where \(n\) is the number of standard diagnostic items, \(i\) is the standard diagnostic item, and \(e\) is the error rate per standard diagnostic item. Equations (2)-(4) show the calculation formulas of the value, structure, and normalization error rates (%). The calculation standard of the database quality diagnosis error data for structured data is defined; however, the target attribute in principle performs data quality diagnosis for all attributes. Also, for unstructured data, the performer should carry out data value quality diagnosis through actual measurement. The value (accuracy) error rate calculation method is an inefficient method for massive data profiling.