# A Study on the Calibrated Confidence of Text Classification Using a Variational Bayes
## 6 Conclusions  
This paper proposed a methodology to calibrate confidence by using a generative model to reduce uncertainty about TM when analyzing patents with imbalanced labels. Research hypotheses were presented to ensure the proposed method's validity. The first hypothesis is that the quantitative indicators of patents differ depending on the purpose of document classification. Patents are data that sufficiently reflect the TMs; therefore, predicting TM using these data requires that the quantitative indicator of a patent must first be able to explain the target variable. Second, the latent variable obtained through the generative model is disentangled in accordance with the label of the patent. The proposed method generates data under the condition of a specific label for a patent. If the patent is entangled in a latent space, more noise is added, and uncertainty may increase. Finally, we assume that the confidence of the TM is calibrated through the proposed method. Thus, the proposed method is effective at reducing uncertainty. The experiment was conducted to examine the practical applicability of the proposed method and to verify the research hypotheses. For the experiment, 10 quantitative indicators were extracted from 11,444 US patents. The text of each patent was transformed into a 384-dimensional vector through a transformer-based model for document embedding. Using these variables, we applied the proposed methods for _Litigation_, _Valuation_, and _Transfer_, which are representative TMs. The results of testing Hypothesis 1 showed that most quantitative indicators had statistically significant differences depending in the target variables. In other words, the quantitative indicators of patents are suitable for predicting TM. Next, by testing Hypothesis 2, we confirmed that the latent vector of the patent obtained through the generative model was disentangled in accordance with the label. Therefore, the proposed method can be used to calibrate confidence by selectively generating only a specific label. In the experiment, when the target variable is Valuation,we confirmed that the proposed method reduced the confidence from a maximum of 0.238 to 0.042. Similarly, when the target variables were _Litigation_ and _Transfer_, the confidence decreased from its maximum of 0.179 to its minimum of 0.020. It was found that the proposed method calibrated the confidence for the three TMs because Hypothesis 3 was statistically significant. In the future, it will be necessary to develop an architecture that combines the generative and predictive models. The proposed method increases the precursors of the training data as a generative model to reduce uncertainty about the prediction. A disadvantage of this approach is that the results may fluctuate depending on the predictive model. Therefore, we expect that sustainable TM will be possible through the development of a methodology that can merge generative and predictive models. [PERSON] and [PERSON] conceived and designed the experiments; [PERSON] analyzed the data to illustrate the validity of this study; [PERSON] and [PERSON] wrote the paper and performed all of the research steps. All authors have read and agreed to the published version of the manuscript. This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (No. NRF\(-\)2022R1 IIA1A01069422). Not applicable. The data presented in this study are available on request from the author [contact: [EMAIL_ADDRESS]]. The authors declare no conflict of interest. Appendix A Appendix A describes the architecture of the generative model used in the experiment. We used Python 3.7.3 and TensorFlow 2.5.0, which is a deep learning framework, to implement C-VAE. Figure 11 shows the developed encoder and decoder of C-VAE. The encoder input data include 394 variables, and the labels are extracted from patent data. The input is the result of concatenating 10 quantitative indicators and a 384-dimensional text vector.