# Cost-Sensitive Variational Autoencoding Classifier for Imbalanced Data Classification
## 3 Method and Algorithm
### Cost-Sensitive Variational Auto-Encoder Classifier
#### 3.2.1 Variational Autoencoder  
Variational autoencoder (VAE) is an important generation model proposed by [PERSON] and [PERSON] in 2013 [35]. The graph model of VAE is shown in Figure 2, where \(x\) denotes the data that we can observe, \(z\) denotes the hidden variable, and VAE assumes \(x\) is produced by the hidden variable \(z\). \(z\to x\) is the generation model, which is the working process of the decoder from the view point of the autoencoder, and \(x\to z\) is the recognition model, which is the working process of the encoder from the view point of the autoencoder. \(\theta\) and \(\omega\) represent the parameters of the process and \(N\) stands for \(N\) samples. Figure 1: The diagram of the proposed Cost-Sensitive Variational AutoEncoding Classifier algorithm. \(X\) denotes the original data, \(Z\) denotes the hidden variable, \(Z^{\prime}\) denotes the sample of \(Z\), \(Y\) denotes the data label, and \(X^{\prime}\) denotes the reconstruction of the original data. The encoder module is used to obtain the distribution of hidden variable \(Z\) from the original data \(X\), the decoder module is used to reconstruct samples and classify them, and the cost matrix module is used to adapt to imbalanced data. The basic idea of the VAE is to assume that the data sample \(X\) is generated by the hidden variable \(Z\) in a certain way, i.e., \(X=g(Z)\). If we can learn the distribution of the hidden variable \(Z\) from the original data, i.e., determine the \(f\) in \(Z=f(X)\), then we can obtain a new \(Z\) by sampling on \(Z\) and generate new data by \(\hat{X}=g(\hat{Z})\). The VAE model is shown in Figure 3 as being mainly composed of encoder and decoder. The encoder accepts the input data \(X\) and outputs the distribution of the hidden variable. In the VAE, it is assumed that the hidden variable \(z\) satisfies the Gaussian distribution, so the encoder outputs the expectation and variance of the Gaussian distribution. The decoder accepts the mean of the hidden variable \(Z\), and then outputs the reconstructed sample of the input data (\(\hat{X}\)), where \(X\) and \(\hat{X}\) should be as close as possible. In the process of data generation, a new \(\hat{Z}\) is first sampled from the distribution obtained by the encoder and then input into the decoder to generate new sample data. We assumed that there were some hidden variables \(z\), but we did not know what the distribution was, and all we could observe was sample \(x\). Therefore, we had to derive the distribution of \(z\) given \(x\); specifically, we had to calculate \(p(z|x)\). \(p(z|x)\) can be given by Equation (3). \[p(z|x)=p(x|z)*p(z)/p(x). \tag{3}\]  
However, this integral of the Equation (3) cannot be calculated directly because we cannot directly calculate \(p(x|z)\) (\(x\) may have a high dimension). We could use the MCMC  
Figure 3: Structure of the proposed variational autoencoder. The encoder accepts the input data \(X\) and outputs the distribution of the hidden variable. The decoder accepts the mean of the hidden variable \(Z\) and then outputs the reconstructed sample of the input data (\(\hat{X}\)). \(X\) and \(\hat{X}\) should be as close as possible. Figure 2: Graph model of VAE. method to sample and then estimate the integral directly; however, in practice, this method is not feasible because it would be too slow. To solve this problem, we introduced the variational method by introducing an additional encoder \(q(z|x)\) to model the relationship between \(x\) and \(z\); that is, we used \(q(z|x)\) to approximate \(p(x|z)\). Therefore, our goal is as shown in Equation (4). \[minKL(q(z|x)||p(z|x)). \tag{4}\]  
_KL_ is the Kullback-Leibler divergence, which is used to measure the difference between two probability distributions in the same event space. The _KL_ divergence is an asymmetric measure of the difference between two probability distributions. For \(q(z|x)\) to be as close as possible to \(p(x|z)\), the _KL_ value must be minimized. According to the calculation formula of _KL_ divergence, Equation (5) can be obtained. \[