# A Study on the Calibrated Confidence of Text Classification Using a Variational Bayes
## 4 Experimental Results
### Comparison of Improvements in Calibrated Confidence in Document Classification
As such, there are appropriate generative models and predictors depending on the label of the target. However, as a result of statistical testing for Hypothesis 2, only the proposed methodology could disentangle the data characteristics depending on the labels in latent space. Therefore, we need to test Hypothesis 3 on the results obtained using the proposed methodology. \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline \multirow{2}{*}{**Case**} & \multicolumn{3}{c}{**Performance Measure**} & \multicolumn{3}{c}{**Statistical Test**} \\ \cline{2-7}  & **Avg\_Baseline** & **Avg\_\_Gen** & **F1-Score** & **KS-Test 1** & **Paired \(t\)-Test 2** & **Wilcoxon 3** \\ \hline Litigation & 0.010 & 0.040 & 0.065 & \textless{}0.001 & \textless{}0.001 & \textless{}0.001 \\ Valuation & 0.157 & 0.353 & 0.395 & \textless{}0.001 & \textless{}0.001 & \textless{}0.001 \\ Transfer & 0.286 & 0.445 & 0.465 & \textless{}0.001 & \textless{}0.001 & \textless{}0.001 \\ \hline \hline \end{tabular}  
* 1 KS test for the two-sided test with log-likelihood. 2 Paired \(t\)-test for one-sided test with log-likelihood. 3 Wilcoxon Signed Rank one-sided test. The null hypotheses for the one and two-sided tests are “the two distributions are equal” and “the log-likelihood is lower than before”, respectively.  
\end{table} 
Table 5: Results of statistical tests for Hypothesis 3. Figure 3: Results of comparing the confidence of the baseline and proposed methodology (when the generative model was C-VAE and the predictors were quantitative indicators and document texts). (**a**) As a result, the proposed method calibrated the confidence better than the baseline; (**b**) the proposed method calibrated the confidence better than the baseline; (**c**) the proposed method calibrated the confidence of transferred patents better than the baseline; (**d**) the difference between the mean of the confidence and the accuracy was smaller for the proposed method than the baseline; (**e**) confidence was calibrated better than confidence in _Litigation_ cases; (**f**) confidence was calibrated better in _Transfer_ cases than in other cases. The confidence obtained using the proposed method tended to be higher than the baseline for all tasks. Thus, we compared the homogeneity of the confidence obtained through the baseline and the proposed method to test Hypothesis 3. Table 5 shows the results of rejecting the null hypothesis in the Kolmogorov-Smirnov test, paired _t_-test, and the Wilcoxon Signed Rank test. Therefore, it is possible to calibrate confidence in document classification through the proposed method. The results of the experiments conducted in this paper are as follows. First, the quantitative indicators of the patents differ depending on the purpose of document classification. Second, in the latent space of the generative models, documents have disentangled representations depending on their labels. Finally, the proposed method can increase the confidence of predictions by reducing uncertainty in document classification.