# Cost-Sensitive Variational Autoencoding Classifier for Imbalanced Data Classification
## 3 Method and Algorithm
### Cost-Sensitive Variational Auto-Encoder Classifier
#### 3.2.3 Classifier Design Based on Cost-Sensitive Factor  
Considering the data imbalance, cost-sensitive factors can be introduced on the basis of the above model to adapt to the imbalanced data and improve the classification performance of the model on the imbalanced data. Commonly used cost-sensitive factors include cost sampling, cost matrix, learning rate adjustment, and threshold shift. A cost-sensitive method based on a cost matrix was adopted in this study. The cost matrix presents the cost of category misclassification in the form of a matrix. The form of the cost matrix is given in Equation (1). The number \(c\) represents the number of data categories, e.g., in a binary classification task, and the cost-sensitive matrix is a \(2*2\) matrix. The element \(cost[i,j]\) in the matrix represents the cost of category \(i\) samples being misclassified into category \(j\). To improve the classification performance of the minority class, the overall design idea of the cost-sensitive matrix is to add a high cost to the minority class and a low cost to the majority class. In the cost-sensitive variational autoencoder proposed in this paper, domain knowledge was introduced to design the cost-sensitive matrix. The specific operation method is described in detail in the experimental section (Section 4.2). After the cost sensitivity is added, the cost function of the model is as shown in Equation (9).  
\[L=-E_{Z\sim q(z|x)}[logP(x|z)]+KL(q(z|x)||p(z))+\psi*L(\hat{y},y)*cost[y\_true,y\_predict] \tag{9}\]  
where \(-E_{Z~{}q(z|x)}[logP(x|z)]\) is the reconstruction error of the input data, \(KL(q(z|x)||p(z))\) is the \(KL\) error, \(L(\hat{y},y)\) is the classification error, \(\psi\) is the weight of the classification error, and \(cost[y\_true,y\_predict]\) is the corresponding misclassification cost.  
In summary, we obtained the overall algorithm for the cost-sensitive VAE. First, the cost-sensitive matrix was designed based on the domain knowledge. Next, the original data \(x\) were input, and the distribution of the hidden variable \(z\) was obtained by the encoder. Then, the new hidden variable \(z^{\prime}\) was obtained by sampling on the distribution of the hidden variable. Finally, the reconstruction \(\hat{x}\) and the label \(\hat{y}\) of the original data \(x\) were obtained through the decoder. The specific procedure utilized by the cost-sensitive VAE is shown in Algorithm 1.  
Figure 4: Structure of the variational autoencoding classifier. Compared with the variational autoencoder, the variational autoencoding classifier includes a new output \(Y.X\) represents the input data, \(Y\) represents the label of the input data, and \(\hat{X}\) represents the reconstruction of the input data.  
```
0: training set \(X=\{x_{t}\}_{t=1}^{N}\), corresponding labels \(Y=\{y_{t}\}_{t=1}^{N}\), loss weight \(\psi\), cost matrix \(cost\)
0: Model \(\theta_{final}\), \(\omega_{final}\)
1: Initialize model parameters \(\theta\) and \(\omega\)
2:for\(t\) in iterationdo
3: Sample \(x_{t}\) in the minibatch: \(out_{i}=fp(x_{i},\theta_{i})\).
4: Encoder: \(u_{z_{t}},\ \sigma_{z_{t}}\leftarrow\ell_{\omega}(x_{t})\).
5: Sampling: \(z_{t}\gets u_{z_{t}}+c\odot\sigma_{z_{t}},\quad\epsilon\ N(0,I)\).
6: Decoder: \(\hat{x}_{t},\ \hat{y}_{t}\gets f_{\theta}(z_{t})\).
7: Compute reconstruction loss:
8:\(\hat{\mathcal{L}}_{rec}=-\log p_{\theta}(x_{t}|z_{t})\).
9: Compute KL loss:
10:\(\hat{\mathcal{L}}_{KL}=KL(q(z_{t}|x_{t})||p(z_{t}))\).
11: Compute classification loss:
12:\(\hat{\mathcal{L}}_{cla}=L(\hat{y}_{t},y_{t})\).
13: Fuse the three losses:
14:\(\hat{\mathcal{L}}(\theta,\omega)=-\log p_{\theta}(x_{t}|z_{t})+KL(q(z_{t}|x _{t})||p(z_{t}))+\psi*L(\hat{y}_{t},y_{t})*cost[y_{t}\text{,}true,y_{t}\text{,} predict]\).
15: Back propagation: \(grad_{t}=bp(\theta_{t},\omega_{t})\).
16: Update: \(\theta_{t+1}=\theta_{t}-lr*grad_{t}\).
17:endfor
18:return Model with parameters \(\theta_{final}\) and \(\omega_{final}\)
```  
**Algorithm 1** Cost-sensitive Variational Autoencoding Classifier