# A Study on the Calibrated Confidence of Text Classification Using a Variational Bayes
## 4 Experimental Results
### Comparison of Representations in Latent Space Depending on Labels in Documents  
This subsection describes the results of the statistical tests for Hypothesis 2. Table 3 shows the distribution of target variables. In the raw data, the patents related to litigation (_Litigation_ = Y) are very few at 125 cases (1.092%). The percentages of high-grade patents (_Valuation_ = Y) and transferred patents (_Transfer_ = Y) are 10.154% and 23.086%, respectively. Through this, it is evident that the patent labels are imbalanced; therefore, this study aims to compare how the proposed methodology works depending on the label ratio. To evaluate the generative model \(G(X,\ y)\) and classifier \(f(X,\ y)\), the raw data were divided into training data and test data in a 7:3 ratio.  
Figure A1 in Appendix B summarizes the architecture of \(G(X,\ y)\) used in this paper. In the experiment, we defined the dimension \(J\) of the latent space of \(G(X,\ y)\) as 2 for the statistical test of Hypothesis 2. The design of the statistical test for Hypothesis 2 is as follows. First, the latent space of \(G(X,\ y)\) is divided into two-dimensional \(z_{i}^{Y}\) and \(z_{i}^{N}\), respectively. However, the general Kolmogorov-Smirnov test deals with the homogeneity of the distribution of one-dimensional data. Therefore, the general Kolmogorov-Smirnov  
\begin{table}
\begin{tabular}{c c c c c c c c c c c} \hline \hline
**Case** & **Statistics** & **Claim** & **Inventor** & **ipc** & **cpc** & **cite\({}_{\text{P}}\)** & **cite\({}_{\text{C}}\)** & **cite\({}_{\text{C}}\)** & **cite\({}_{\text{R}}\)** & **fam\({}_{\text{P}}\)** & **fam\({}_{\text{C}}\)** & **fam\({}_{\text{E}}\)** \\ \hline \multirow{10}{*}{**Litigation**} & Avg. Y & 23.536 & 1.544 & 3.656 & 4.656 & 223.584 & 3.600 & 66.664 & 74.832 & 5.560 & 20.400 \\  & Std. Y & 12.981 & 1.874 & 4.785 & 9.004 & 430.433 & 2.626 & 140.082 & 178.146 & 3.965 & 23.832 \\  & Avg. N & 18.168 & 1.680 & 3.128 & 3.880 & 69.560 & 2.408 & 8.288 & 3.960 & 32.928 & 8.216 \\  & Std. N & 10.662 & 1.831 & 2.558 & 4.705 & 390.268 & 2.113 & 34.262 & 3.275 & 150.234 & 9.228 \\  & Levene \({}^{1}\) & 0204 & 0.747 & 0.114 & 0.396 & 0.007 & 0.001 & \(<\)0.001 & 0.090 & 0.002 & \(<\)0.001 \\  & \(t\)-test2 & \(<\)0.001 & 0.564 & 0.280 & 0.396 & 0.003 & \(<\)0.001 & \(<\)0.001 & 0.046 & 0.001 & \(<\)0.0  
test and the multidimensional version of the Kolmogorov-Smirnov test [73,74] are applied depending on the dimensions of the latent vector. Then, we can determine whether the latent space for each condition is disentangled through statistical hypothesis testing. Since the latent variables are not entangled in \(G(X,\,y)\), only data belonging to a specific label can be generated. Furthermore, we conducted experiments depending on the types of predictors and generative models to compare their results. In Table 4, Quant and Text are the results of using only quantitative indicators and text of documents, respectively. In addition, VAE refers to a generative model that does not assume conditions for a specific label in C-VAE.  
In Table 4, \(\left(z^{Y}_{i,\,1},\,z^{N}_{i,\,1}\right)\) and \(\left(z^{Y}_{i,\,2},\,z^{N}_{i,\,2}\right)\) are vectors depending on the labels of documents obtained in latent space. In the table, \(z^{Y}_{i,\,\cdot}\) denotes a two-dimensional vector obtained by merging \(z^{Y}_{i,\,1}\) and \(z^{Y}_{i,\,2}\). As a result of the experiment, when the generative model was C-VAE and the quantitative indicators and document texts were used as predictors, Hypothesis 2 was not rejected for all target labels. Therefore, the proposed method evidently disentangles the document representation for each label in the latent space.