# A Study on the Calibrated Confidence of Text Classification Using a Variational Bayes
## 1 Introduction  
Methods for finding patterns in data and predicting the future from data have grown into an important field of research, which has brought about advances in information technology [1, 2, 3]. [PERSON] et al. expected that predictive models that can analyze images, natural language, and signals will advance further in the near future [4]. In fact, much modern decision making is being carried out after insights were gained from predictive models, from self-driving cars to medical diagnoses [5, 6, 7, 8]. However, the reliability and efficiency of predictive models are affected by various factors such as noise in data [9, 10, 11, 12]. Thus, we have to consider many factors--such as _aleatoric_ uncertainty and _epistemic_ uncertainty--to secure confidence in a predictive model [13, 14, 15]. [PERSON] and [PERSON] argued that _aleatoric_ uncertainty and _epistemic_ uncertainty are, respectively, irreducible noise contained in data and reducible errors where the model cannot explain the data [16]. Therefore, modern machine learning and deep learning models should ensure confidence. In other words, confidence, which is the likelihood that a predicted label is correct, needs to be calibrated to reflect the ground truth accuracy. Calibration refers to the statistical consistency between the accuracy and probability of prediction [17, 18]. In this respect, the model should not only provide accurate predictions but also calibrated confidence [19]. Calibrated confidence is required for modeling fraud detection and healthcare, in addition to self-driving cars, as mentioned above. This is because the uncertainty in the model can pose a direct risk to objects such as persons. As services that use natural language processing are becoming increasingly common, research on named entity recognition, parts of speech, and question answering is beingactively conducted [20; 21; 22]. However, research on models with calibrated confidence for text classification is insufficient because the risk to an object is low relative to other applications. Although Technology Management (TM) using documents such as patents involves many factors that can threaten companies and research institutes, there has been insufficient discussion of the uncertainty. The reasons to consider calibrated confidence in TM based on data analysis are as follows:  
* TM, implemented with strategies such as patent litigation, technology transfer and valuation, is a time-consuming and expensive task [23; 24];
* Patent litigation causes the proliferation of patent wars between companies because it can inflict huge losses on the accused by prohibiting manufacturing, sales, and imports [25; 26; 27];
* Technology valuation can be used for the early commercialization of excellent technology, which can offer opportunities to expand business models [28; 29];
* Technology transfer can save on the time invested in technology development and can further serve as a cornerstone for mergers and acquisitions (M&A) between companies [30; 31];
* Most technologies do not go through a TM, and data labels can become imbalanced [32]. Based on the above evidence, TM is potentially a high-risk application. In particular, the uncertainty of the predictive model for these activities may be high. Therefore, we need to investigate ways of lowering the uncertainty in the model. Many previous studies have pointed out that the confidence of a predictive model is uncalibrated compared to the dramatically improved accuracy. [PERSON] proposed a method for converting the output of a predictive model into a probability using a logistic function [33]. He laid down the groundwork for measuring the confidence of many models and comparing with their accuracy. In particular, [PERSON] et al., in an extension of his research, contributed to reducing overconfidence by developing a function that could return a softened probability [34]. [PERSON] and [PERSON] judged whether their model's confidence was calibrated by visualizing the expected accuracy and observed accuracy [35].