# Uncertainty-Aware Active Meta-Learning for Few-Shot Text Classification
## 4 Methodology
### Uncertainty Quantification via Bayesian Neural Networks  
Due to its excellent performance, deep learning has received much attention as a major machine learning methodology that can be applied in various fields. However, for typical deep learning models, these tools for regression and classification cannot measure model uncertainty. In other words, deep learning works well, but it is difficult to explain why. There are various approaches to quantifying the uncertainty of these deep learning models.  
A Bayesian neural network is a type of deep learning algorithm that enables probabilistic analysis by assuming the probability distribution for the model's weights. The Bayesian neural network converges to a Gaussian process as the number of weight parameters increases, which means that the limitations of the Gaussian process can be overcome and the uncertainty of model prediction can be expressed simultaneously. With these advantages, Bayesian neural networks are recently attracting attention in various fields, including computational science [34; 35; 36].  
In general, the uncertainty of deep learning models is divided into aleatoric and epistemic uncertainty [36]. Aleatoric uncertainty refers to the uncertainty naturally occurring in the observed data, and an example is the problem of mapping the same observation to different data due to power supply noise in the sensor. In the case of aleatoric uncertainty, uncertainty cannot be reduced. Since the measurement error is constant for all samples, it is also called dynamic variance uncertainty. On the other hand, epistemic uncertainty occurs because the training data does not represent the entire dataset of the population and can explain what the model does not know when performing prediction and classification. Epistemic uncertainty is reduced when sufficient training data are given.  
\[p(w|X,\ Y)=\frac{p(Y|X,\ w)p(w)}{P(Y|X)} \tag{3}\]  
\[p(y^{*}|x^{*},\ X,\ Y)=\int p(y^{*}\Big{|}x^{*},\ w)p(w\Big{|}X,\ Y)dw \tag{4}\]  
A representative way to quantify the uncertainty of such a deep learning model is to use a Bayesian neural network [37]. The Bayesian neural network enables probabilistic interpretation of neural network models by assuming a prior distribution for the model weights as in Equation (3) and calculating the posterior distribution for the weights through the Bayesian inference process. When the set \(X\) of the input data and the set \(Y\) of the target data are given, the output \(y^{*}\) of the Bayesian neural network is expressed as a posterior predictive distribution as shown in Equation (4), and this process is called inference.  
\[KL(q_{\theta}(w)||p(w|Y|X))=\int q_{\theta}(w)log\frac{q_{\theta}(w)}{p(w|X,Y)}dw \tag{5}\]  
\[ELBO=\int q_{\theta}(w)logp(Y|X,w)dw-KL(q_{\theta}(w)||p(w)) \tag{6}\]  
The goal of this research is to find a way to quantify and measure the uncertainty of deep learning models and to find the estimated distribution most similar to the posterior. The Kullback-Leibler Divergence (KLD), which calculates the similarity between two different probability distributions, is used to determine the posterior distribution and the estimated distribution as in Equation (5). Calculating the similarity between the problem of minimizing the KLD between the posterior distribution and the estimated distribution is the same as the problem of maximizing the Evidence Lower Bound (ELBO), as shown in Equation (6). The process of training a neural network to solve such an optimization problem is called Bayes by backprop.