# Generative Modeling for Imbalanced Credit Card Fraud Transaction Detection
## 3 Generative Models
### Variational Autoencoder (VAE)  
In this paper, we utilized a Variational Autoencoder architecture to balance our credit card dataset. VAEs, introduced by [PERSON] et al. (2015), extend traditional Autoencoders by incorporating variational inference, a statistical technique for approximating complex distributions. VAEs are generative models that use Variational Bayes Inference to model data generation through a probabilistic distribution. Unlike traditional Autoencoders, VAEs include an additional sampling layer along with the encoder and decoder layers [PERSON] and [PERSON] (2014). During training, the input data are encoded as a distribution over the latent space, and the latent vector is sampled from this distribution [PERSON] and [PERSON] (2014). This latent vector is then decoded, and the reconstruction error is computed and backpropagated through the network as described in Figure 3.  
Figure 3: Variational Autoencoder architecture.  
Probabilistically, a VAE consists of a latent representation \(z\), drawn from a prior distribution \(p(z)\), and the data point \(x\), drawn from a conditional likelihood distribution \(p(x\mid z)\), which is referred to as the probabilistic decoder. This can be expressed as follows:  
\[p(x,z)=p(x\mid z)p(z) \tag{4}\]  
The model's inference is examined by computing the posterior of the latent vector using [PERSON] theorem, as shown in the equation below:  
\[p(z\mid x)=\frac{p(x\mid z)p(z)}{p(x)} \tag{5}\]  
Using any distribution variant, such as Gaussian, variational inference can approximate the posterior. The reliability of this approximation can be assessed through the Kullback-Leibler divergence, which measures the information loss during approximation. The architecture and algorithm used for the VAE implementation in our study are detailed in Algorithm 2. This algorithm outlines the training process for a VAE on fraudulent transaction samples, with carefully chosen hyperparameters to balance model complexity, prevent overfitting, and enhance the model's ability to capture meaningful representations. The encoder architecture, with hidden layers of 64 and 32 units, reduces the data dimensionality, capturing complex patterns without overfitting. ReLU activation is used to mitigate the vanishing gradient problem and accelerate convergence, while batch normalization stabilizes training and improves generalization. Dropout rates of 0.2 and 0.3 are applied to prevent overfitting by randomly deactivating neurons during training. The encoder outputs the mean (\(\mu\)) and log-variance (\(\log(\sigma^{2})\)) for stochastic sampling, allowing the model to effectively learn from the data. The decoder mirrors the encoder structure for symmetric data reconstruction, with a sigmoid output layer suitable for binary classification. The loss function combines binary cross-entropy for reconstruction and KL divergence for regularization, promoting both accurate reconstruction and a structured latent space. The model is trained using the Adam optimizer with a learning rate of 0.001 for efficient training, for 100 epochs with a batch size of 32 to balance convergence speed and computational cost.