## 4 The Concept for the Assessment of Data Set Completeness
### Metric for Data Set Completeness
The higher the distance to the equally distributed vector, the higher the imbalance of vector \(\overset{\rightarrow}{v}_{j}\). The distance \(d\) of the two vectors is calculated based on the total variation distance (9). Thereby, the equally distributed vector \(\overset{\rightarrow}{e}_{j}\) consists of the same number of sections, contains the same number of observations (\(\sum\eta\)), but shows an equal distribution of these observations. \[\mathrm{d}\left(\overset{\rightarrow}{v}_{j},\overset{\rightarrow}{e}_{j} \right)=0.5\sum\limits_{z=1}^{K_{j}}\left|\overset{\rightarrow}{v}_{jz}- \overset{\rightarrow}{e}_{jz}\right|\qquad\forall i=[1\ldots n] \tag{9}\]  
In order to make the distance \(\mathrm{d}\left(\overset{\rightarrow}{v}_{j},\overset{\rightarrow}{e}_{j}\right)\) interpretable and comparable, it is normalized to a range between 0 and 1. In order to do so, the distance measure is divided by the distance between the equally distributed vector \(\overset{\rightarrow}{e}_{j}\) and the worst distributed class vector \(\overset{\rightarrow}{\omega}_{j}\). The quotient then expresses the degree of imbalance in percentage. This worst distributed class vector \(\overrightarrow{\omega_{j}}\) complies with the following requirements: \(\overrightarrow{\omega_{j}}\) contains the same number of observations as \(\overrightarrow{v}_{j}\); minority sections of \(\overrightarrow{v}_{j}\) (\(\overrightarrow{v}_{jz}\) \(\subset\)\(\overrightarrow{e}_{jz}\)) are set to 0; majority sections of \(\overrightarrow{v}_{j}\) (\(\overrightarrow{v}_{jz}\) \(\supset\)\(\overrightarrow{e}_{jz}\)) are set to \(\overrightarrow{e}_{jz}\) except for one section containing all remaining observations. Furthermore, the resulting quotient is multiplied by a correction factor taking into account the size of the vector and the number of minority sections within the vector. Otherwise the vector \(\overrightarrow{v}_{A}=\overrightarrow{\omega}_{A}=[20,\,0]\) receives the same imbalance score as vector \(\overrightarrow{v}_{B}=\overrightarrow{\omega}_{B}=[20,\,10,\,10,\,0]\), since both are maximal imbalanced for a number of minority sections of 1. However, a vector with less minority sections in relation to the total number of sections, e.g., \(\overrightarrow{v}_{B}\), is less imbalanced than a vector with more minority sections, e.g., \(\overrightarrow{v}_{A}\). Therefore, the imbalance metric is multiplied by the quotient of the number of minority sections \(\mu_{j}\) and the maximum of potential minority sections \(K_{j}-1\). This quotient achieves the maximum of 1, when all sections except for one are minority sections. The minimum of 0 is achieved when zero minority sections are observed which is only the case if the vector \(\overrightarrow{v}_{j}\) equals \(\overrightarrow{e}_{j}\). In this way, vector \(\overrightarrow{v}_{A}\) reveals an imbalance degree of 1 while vector \(\overrightarrow{v}_{B}\) reveals an imbalance degree of 0.33. The final section balance \(SB_{j}\) of class \(j\) equals 1 minus the derived imbalance degree (10). Since the quotient of the two distances \(d\) is between 0 and 1, and the correction factor is also between 0 and 1, the imbalance degree as well as the section balance \(SB_{j}\) achieves maximal 1 for perfectly good data quality and minimal 0 for perfectly poor data quality. \[SB_{j}=1-\left(\frac{\mathrm{d}\left(\overrightarrow{v}_{j},\overrightarrow{e }_{j}\right)}{\mathrm{d}\left(\overrightarrow{\omega}_{j},\overrightarrow{e }_{j}\right)}\cdot\frac{\mu_{j}}{K_{j}-1}\right)\quad \begin{array}{l}\forall j =[1\dots k_{i}]\\ \forall i=[1\dots n]\end{array}  \tag{10}\]  
The metric for the class volume \(CV_{j}\) compares the number of observations of class \(j\) with the number of observations of the other classes of variable \(i\) (\(\overrightarrow{s}\)). In the best case, all classes contain the same number of observations leading to a balanced distribution across the variable \(i\).